Skip to content

Commit

Permalink
Fix description
Browse files Browse the repository at this point in the history
  • Loading branch information
astonzhang committed Jul 31, 2023
1 parent 397dcbe commit cd42a8c
Showing 1 changed file with 1 addition and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ $\mathbf{h}_{t}$ as both the key and the value. Note that $\mathbf{c}_{t'}$ is t
using the additive attention scoring function
defined by :eqref:`eq_additive-attn`.
This RNN encoder-decoder architecture
using attention is depicted in :numref:`fig_s2s_attention_details`. Note that later this model was modified such as to include the already generated tokens in the decoder as further context (i.e., the attention sum does stop at $T$ but rather it proceeds up to $t'-1$). For instance, see :citet:`chan2015listen` for a description of this strategy, as applied to speech recognition.
using attention is depicted in :numref:`fig_s2s_attention_details`. Note that later this model was modified such as to include the already generated tokens in the decoder as further context (i.e., the attention sum does not stop at $T$ but rather it proceeds up to $t'-1$). For instance, see :citet:`chan2015listen` for a description of this strategy, as applied to speech recognition.

![Layers in an RNN encoder-decoder model with the Bahdanau attention mechanism.](../img/seq2seq-details-attention.svg)
:label:`fig_s2s_attention_details`
Expand Down

0 comments on commit cd42a8c

Please sign in to comment.