briannaflynn / DL-PyTorch-Notes Public

Notes from diving deep into PyTorch for Deep Learning: General Concepts with Code

Notifications

Name		Name	Last commit message	Last commit date
Latest commit History 21 Commits
.ipynb_checkpoints		.ipynb_checkpoints
DL_consolidated_notes-Part1.ipynb		DL_consolidated_notes-Part1.ipynb
How-to-train-your-model.ipynb		How-to-train-your-model.ipynb
Matrix-Mul-Methods_Forward-Backward-Pass.ipynb		Matrix-Mul-Methods_Forward-Backward-Pass.ipynb
README.md		README.md
callbacks-math-deepdive.ipynb		callbacks-math-deepdive.ipynb

Repository files navigation

DL-PyTorch-Notes

Notes from diving deep into implementing PyTorch for Deep Learning: General Concepts with Code Examples

Goes over:

Why good initialization matters - and why we want activations to have a mean close to 0 and a standard deviation close to 1
Variance, Standard Deviation, and Mean Absolute Deviation - when to use each and why
Activation explosion
Basic training loop
Convolutions
Callbacks and why to use them
Math essentials
Hooks (Pytorch's version of callbacks)
When and why to use Softmax vs Binomial when interpreting activations
Batch Normalization
Exponentially weighted moving averages
Small Batch / Batch of 1 Normalization Techiques