Skip to content

Commit

Permalink
Minor improvement: more details on API calls (d2l-ai#2312)
Browse files Browse the repository at this point in the history
* Minor improvement: more details on API calls

Considering pytorch, I believe this is the first time `nn.Parameter` is used.
I checked the previous chapters, and `tf.Variable` was already used but not really introduced (?)...and it's not entirely clear to me if/how differs from what pytorch does

For sure is the first time that `Sequential()` is used across the 3 frameworks.

All these are (at least to me) not complicated things to understand but having an explanation doesn't hurt.

* Update chapter_multilayer-perceptrons/mlp-implementation.md

Integrated suggestion: missing URL link

Co-authored-by: Anirudh Dagar <[email protected]>

* Update chapter_multilayer-perceptrons/mlp-implementation.md

integrated suggestion: rewording

Co-authored-by: Anirudh Dagar <[email protected]>

* Update chapter_multilayer-perceptrons/mlp-implementation.md

integrated suggestion

Co-authored-by: Anirudh Dagar <[email protected]>

Co-authored-by: Anirudh Dagar <[email protected]>
  • Loading branch information
finale80 and AnirudhDagar authored Oct 6, 2022
1 parent b899e96 commit f6f8895
Showing 1 changed file with 36 additions and 0 deletions.
36 changes: 36 additions & 0 deletions chapter_multilayer-perceptrons/mlp-implementation.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,22 @@ one weight matrix and one bias vector.
As always, we allocate memory
for the gradients of the loss with respect to these parameters.

:begin_tab:`mxnet`
In the code below, we first define and initialize the parameters
and then enable gradient tracking.
:end_tab:

:begin_tab:`pytorch`
In the code below we use [`nn.Parameter()`](https://pytorch.org/docs/stable/generated/torch.nn.parameter.Parameter.html)
from the `torch` standard library to automatically register
a class attribute as a parameter to be tracked by [`autograd`](https://pytorch.org/docs/stable/autograd.html).
:end_tab:

:begin_tab:`tensorflow`
In the code below we use [`tf.Variable()`](https://www.tensorflow.org/api_docs/python/tf/Variable)
to `tensorflow` standard library to define the model parameter.
:end_tab:

```{.python .input n=5}
%%tab mxnet
class MLPScratch(d2l.Classifier):
Expand Down Expand Up @@ -198,6 +214,26 @@ class MLP(d2l.Classifier):
tf.keras.layers.Dense(num_outputs)])
```

Notice the use of the `Sequential()` API and
the lack of definition of `forward()`.

Until now, we defined a `forward` method to transform the inputs
using the network parameters.
However, those operations are essentially a pipeline:
you take an input and
apply a transformation (e.g.,
matrix multiplication with weights, plus bias addition);
take the output of such transformation and use it as
input to next transformation, etc.

The default `d2l.Module.forward()`
simply invokes `self.net(X)` which is now defined
as a sequence of transformations.

In other words, the `Sequential()` API abstracts the forward process
enabling us to focus on the transformations.


### Training

[**The training loop**] is exactly the same
Expand Down

0 comments on commit f6f8895

Please sign in to comment.