Correcting the optimization formula for Adam

umhan35 · May 26, 2018 · e2ac0ab · e2ac0ab
1 parent 9eb299f
commit e2ac0ab
Showing 1 changed file with 2 additions and 2 deletions.
diff --git a/2- Improving Deep Neural Networks/Readme.md b/2- Improving Deep Neural Networks/Readme.md
@@ -550,8 +550,8 @@ L2-regularization relies on the assumption that a model with small weights is si
   	# can be mini-batch or batch gradient descent
   	compute dw, db on current mini-batch                
 
-  	vdW = (beta1 * dW) + (1 - beta1) * dW     # momentum
-  	vdb = (beta1 * db) + (1 - beta1) * db     # momentum
+  	vdW = (beta1 * vdW) + (1 - beta1) * dW     # momentum
+  	vdb = (beta1 * vdb) + (1 - beta1) * db     # momentum
 
   	sdW = (beta2 * dW) + (1 - beta2) * dW^2   # RMSprop
   	sdb = (beta2 * db) + (1 - beta2) * db^2   # RMSprop