update readme

bht2016 · Dec 21, 2019 · 4b7dbf2 · 4b7dbf2
1 parent ee92100
commit 4b7dbf2
Show file tree

Hide file tree

Showing 7 changed files with 13 additions and 12 deletions.
diff --git a/.idea/workspace.xml b/.idea/workspace.xml
diff --git a/README.md b/README.md
@@ -333,16 +333,19 @@ class Decoder(nn.Module):
 ## 2. Experiments
 
 I use Multi30K Dataset to train and evaluate model <br>
-You can check detail of dataset [here](https://arxiv.org/abs/1605.00459)
+You can check detail of dataset [here](https://arxiv.org/abs/1605.00459) <br>
+I follow original paper's parameter settings. (below) <br>
+
+![conf](image/transformer-model-size.jpg)
+### 2.1 Transformer - Baseline
 
-### 2.1 Transformer - Small
 * total parameters = 69,919,919
 * model size = 273.2MB
 * lr scheduling : ReduceLROnPlateau
 
 #### 2.1.1 configuration
 
-* batch_size = 128
+* batch_size = 64
 * max_len = 50
 * d_model = 512
 * n_layers = 6
@@ -358,10 +361,8 @@ You can check detail of dataset [here](https://arxiv.org/abs/1605.00459)
 * epoch = 1000
 * clip = 1
 
-
 #### 2.1.2 Training Result
-![table](saved/transformer-small/train_result.jpg)
-<br><br>
+
 * Minimum Training Loss = 2.852672759656864
 * Minimum Validation Loss = 3.2048025131225586 
 <br><br>

diff --git a/image/transformer-model-size.jpg b/image/transformer-model-size.jpg
diff --git a/models/layers/__pycache__/multi_head_attention.cpython-36.pyc b/models/layers/__pycache__/multi_head_attention.cpython-36.pyc
diff --git a/models/layers/__pycache__/scale_dot_product_attention.cpython-36.pyc b/models/layers/__pycache__/scale_dot_product_attention.cpython-36.pyc
diff --git a/result/test.txt b/result/test.txt
@@ -1 +1 @@
-[5.760715872049332, 5.430302619934082, 5.130149096250534, 5.124002903699875, 5.081889361143112, 5.049355149269104, 4.9583838284015656, 4.919993549585342, 4.879352629184723]
+[5.869744032621384, 5.464778572320938, 5.381483763456345, 5.198367655277252, 5.1747742891311646, 5.201104640960693]
diff --git a/result/train.txt b/result/train.txt
@@ -1 +1 @@
-[6.487048629096951, 5.643908386188457, 5.413760770259975, 5.2877601085780475, 5.231303647751325, 5.191577813698856, 5.153288614382303, 5.098250471022686, 5.04833714657418]
+[7.047564462418073, 5.7127689853113655, 5.4532069918342625, 5.285951928420213, 5.216039784679329, 5.174198560252589]
Original file line number	Diff line number	Diff line change
		@@ -1 +1 @@
		[5.760715872049332, 5.430302619934082, 5.130149096250534, 5.124002903699875, 5.081889361143112, 5.049355149269104, 4.9583838284015656, 4.919993549585342, 4.879352629184723]
		[5.869744032621384, 5.464778572320938, 5.381483763456345, 5.198367655277252, 5.1747742891311646, 5.201104640960693]
Original file line number	Diff line number	Diff line change
		@@ -1 +1 @@
		[6.487048629096951, 5.643908386188457, 5.413760770259975, 5.2877601085780475, 5.231303647751325, 5.191577813698856, 5.153288614382303, 5.098250471022686, 5.04833714657418]
		[7.047564462418073, 5.7127689853113655, 5.4532069918342625, 5.285951928420213, 5.216039784679329, 5.174198560252589]