From 6ad200578a31c0cd70e14de96f89b21fb7fbe7df Mon Sep 17 00:00:00 2001
From: Phil Wang <lucidrains@gmail.com>
Date: Wed, 15 Feb 2023 12:12:38 -0800
Subject: [PATCH] last update

---
 README.md | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/README.md b/README.md
index dcebd0b..2fb4de8 100644
--- a/README.md
+++ b/README.md
@@ -14,6 +14,8 @@ Update 2: <a href="https://api.wandb.ai/links/lucidrains/d4v6c8sl">experiments</
 
 Update 3: Dividing the learning rate by 3, seeing better early results than Adam. Maybe Adam has been dethroned, after nearly a decade.
 
+Update 4: using the 10x smaller learning rate rule of thumb from the paper resulted in the worst run. so I guess it still takes a bit of tuning
+
 ## Install
 
 ```bash