last update

yousufmo · Feb 15, 2023 · 6ad2005 · 6ad2005
1 parent 0aaabf0
commit 6ad2005
Showing 1 changed file with 2 additions and 0 deletions.
diff --git a/README.md b/README.md
@@ -14,6 +14,8 @@ Update 2: <a href="https://api.wandb.ai/links/lucidrains/d4v6c8sl">experiments</
 
 Update 3: Dividing the learning rate by 3, seeing better early results than Adam. Maybe Adam has been dethroned, after nearly a decade.
 
+Update 4: using the 10x smaller learning rate rule of thumb from the paper resulted in the worst run. so I guess it still takes a bit of tuning
+
 ## Install
 
 ```bash