Commits

Branch selector

 main

User selector

All users

Datepicker

All time

Commit History

Commits on Mar 2, 2021

• allow for setting embedding dimension to be different than model dimension

  lucidrains

  committed
  c40815f

  Copy full SHA for c40815f

Commits on Feb 8, 2021

• add an assert for relative positional keyword arguments

  lucidrains

  committed
  6b93c21

  Copy full SHA for 6b93c21

Commits on Jan 17, 2021

• fix residual gating

  lucidrains

  committed
  64ee68b

  Copy full SHA for 64ee68b

Commits on Jan 16, 2021

• link to txl for RL paper

  lucidrains

  committed
  83fbd35

  Copy full SHA for 83fbd35

• add diagram for gating feature

  lucidrains

  committed
  172985a

  Copy full SHA for 172985a

• add gating at residuals, from deepminds paper for stabilizing txl for RL paper

  lucidrains

  committed
  25dee85

  Copy full SHA for 25dee85

Commits on Jan 15, 2021

• fix bug with prenorm, introduced when adding residual attention

  lucidrains

  committed
  08cc84e

  Copy full SHA for 08cc84e

Commits on Jan 12, 2021

• allow floats for ff_mult

  lucidrains

  committed
  b82afd7

  Copy full SHA for b82afd7

Commits on Jan 5, 2021

• fix bug with default empty memories for txl

  lucidrains

  committed
  208d70d

  Copy full SHA for 208d70d

Commits on Jan 4, 2021

• fix some more issues with T5 rel pos bias, thanks to @adrian-spataru

  lucidrains

  committed
  fb7a5d4

  Copy full SHA for fb7a5d4

• untie embedding, after learning T5 switched to untied classifier weights in latest version, aligning with my experience that it is better

  lucidrains

  committed
  058bc89

  Copy full SHA for 058bc89

Commits on Jan 3, 2021

• bump bug fix release

  lucidrains

  authored
  399efcb

  Copy full SHA for 399efcb

• Merge pull request lucidrains#10 from wangcongcong123/patch-2

  Show description for a06f479
  lucidrains

  authored
  a06f479

  Copy full SHA for a06f479

• bug fix for _relative_position_bucket

  Show description for 6d0c641
  wangcongcong123

  authored
  6d0c641

  Copy full SHA for 6d0c641

• avoid potential issue with PAR and transformer-xl recurrence

  lucidrains

  committed
  6dda487

  Copy full SHA for 6dda487

• add PAR, credit goes to @lunixbochs

  lucidrains

  committed
  af23656

  Copy full SHA for af23656

• fix bugs introduced from attention intermediates and also with feedforward glu, thanks to @wangcongcong123

  lucidrains

  committed
  9cf7001

  Copy full SHA for 9cf7001

Commits on Jan 1, 2021

• cite shortformer

  lucidrains

  committed
  67877d6

  Copy full SHA for 67877d6

• cleanup

  lucidrains

  committed
  b5c06bc

  Copy full SHA for b5c06bc

• remove funnel transformer

  lucidrains

  committed
  be45942

  Copy full SHA for be45942

• make shortformer possible

  lucidrains

  committed
  15b327a

  Copy full SHA for 15b327a

• allow users to access post softmax attention maps across layers with return_attn=True

  lucidrains

  committed
  20a0dc1

  Copy full SHA for 20a0dc1

• fix bug

  lucidrains

  committed
  6cdc8e8

  Copy full SHA for 6cdc8e8

• recurrence in 0.6

  lucidrains

  committed
  ab4b439

  Copy full SHA for ab4b439

• make transformer-xl recurrence work

  lucidrains

  committed
  3334ca4

  Copy full SHA for 3334ca4

• fix bug with absolute pos emb

  lucidrains

  committed
  8e8200d

  Copy full SHA for 8e8200d

• cite

  lucidrains

  committed
  d5975c2

  Copy full SHA for d5975c2

Commits on Dec 31, 2020

• save the user

  lucidrains

  committed
  9cef438

  Copy full SHA for 9cef438

• small cleanup

  lucidrains

  committed
  e2cf489

  Copy full SHA for e2cf489

• add position infused attention, from DETR and Shortformer papers

  lucidrains

  committed
  7d46572

  Copy full SHA for 7d46572

• add intent to implement

  lucidrains

  committed
  c886a5d

  Copy full SHA for c886a5d

Commits on Dec 28, 2020

• add quote

  lucidrains

  committed
  a5b8dde

  Copy full SHA for a5b8dde

• add ability to residualize cross attention as well, after noting an improvement in a toy task

  lucidrains

  committed
  4255d93

  Copy full SHA for 4255d93

• allow user to use a projection to logits (vs weight tying) with tie_embedding keyword argument

  lucidrains

  committed
  ae9fd01

  Copy full SHA for ae9fd01

• fix bug with residual attention

  lucidrains

  committed
  acb4c65

  Copy full SHA for acb4c65

Pagination

Previous

Next