Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix typo in Backpropagation Post #35

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
fix typo
  • Loading branch information
nohtanoj committed Aug 2, 2017
commit 8a33167029820cd66c1864e0ce11e03c608d6d92
2 changes: 1 addition & 1 deletion posts/2015-08-Backprop/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -136,7 +136,7 @@ <h2 id="factoring-paths">Factoring Paths</h2>
<p>Instead of just naively summing over the paths, it would be much better to factor them:</p>
<p><span class="math">\[\frac{\partial Z}{\partial X} = (\alpha + \beta + \gamma)(\delta + \epsilon + \zeta)\]</span></p>
<p>This is where “forward-mode differentiation” and “reverse-mode differentiation” come in. They’re algorithms for efficiently computing the sum by factoring the paths. Instead of summing over all of the paths explicitly, they compute the same sum more efficiently by merging paths back together at every node. In fact, both algorithms touch each edge exactly once!</p>
<p>Forward-mode differentiation starts at an input to the graph and moves towards the end. At every node, it sums all the paths feeding in. Each of those paths represents one way in which the input affects that node. By adding them up, we get the total way in which the node is affected by the input, it’s derivative.</p>
<p>Forward-mode differentiation starts at an input to the graph and moves towards the end. At every node, it sums all the paths feeding in. Each of those paths represents one way in which the input affects that node. By adding them up, we get the total ways in which the node is affected by the input, it’s derivative.</p>
<div style="width:70%; margin-left:auto; margin-right:auto; margin-bottom:17px; margin-top:17px;">
<img src="img/chain-forward-greek.png" alt>
</div>
Expand Down