Update README.md

016mm · May 18, 2023 · a8a6809 · a8a6809
1 parent cb32ded
commit a8a6809
Showing 1 changed file with 2 additions and 0 deletions.
diff --git a/README.md b/README.md
@@ -200,6 +200,8 @@ x = x + pos_emb_x + pos_emb_y
 
 ### Misc
 
+Maybe we can improve memorization by simply repeating the context (I guess 2 times is enough). Example:  Reference -> Reference(again) -> Question -> Answer
+
 #### Idea: Bytes-aware Embedding
 
 The idea is to make sure each token in vocab understand its length and raw UTF-8 bytes.