forked from udlbook/udlbook
-
Notifications
You must be signed in to change notification settings - Fork 0
/
index.html
118 lines (104 loc) · 14.8 KB
/
index.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
<h1>Understanding Deep Learning</h1>
by Simon J.D. Prince
<br>
To be published by MIT Press Dec 5th 2023.<br>
<img src="https://raw.githubusercontent.com/udlbook/udlbook/main/UDLCoverSmall.jpg" alt="front cover">
<h2> Download draft PDF </h2>
<a href="https://github.com/udlbook/udlbook/releases/download/v1.1.1/UnderstandingDeepLearning_26_07_23_C.pdf">Draft PDF Chapters 1-21</a><br> 2023-07-26. CC-BY-NC-ND license
<br>
<img src="https://img.shields.io/github/downloads/udlbook/udlbook/total" alt="download stats shield">
<br>
<ul>
<li> Appendices and notebooks coming soon
<li> Report errata via <a href="https://github.com/udlbook/udlbook/issues">github</a> or contact me directly at [email protected]
<li> Follow me on <a href="https://twitter.com/SimonPrinceAI">Twitter</a> or <a href="https://www.linkedin.com/in/simon-prince-615bb9165/">LinkedIn</a> for updates.
</ul>
<h2>Table of contents</h2>
<ul>
<li> Chapter 1 - Introduction
<li> Chapter 2 - Supervised learning
<li> Chapter 3 - Shallow neural networks
<li> Chapter 4 - Deep neural networks
<li> Chapter 5 - Loss functions
<li> Chapter 6 - Training models
<li> Chapter 7 - Gradients and initialization
<li> Chapter 8 - Measuring performance
<li> Chapter 9 - Regularization
<li> Chapter 10 - Convolutional networks
<li> Chapter 11 - Residual networks
<li> Chapter 12 - Transformers
<li> Chapter 13 - Graph neural networks
<li> Chapter 14 - Unsupervised learning
<li> Chapter 15 - Generative adversarial networks
<li> Chapter 16 - Normalizing flows
<li> Chapter 17 - Variational autoencoders
<li> Chapter 18 - Diffusion models
<li> Chapter 19 - Deep reinforcement learning
<li> Chapter 20 - Why does deep learning work?
<li> Chapter 21 - Deep learning and ethics
</ul>
<h2>Resources for instructors </h2>
<p></p>Instructor answer booklet available with proof of credentials via <a href="https://mitpress.mit.edu/9780262048644/understanding-deep-learning"/> MIT Press</a></p>
<p></p>Figures in PDF (vector) / SVG (vector) / Powerpoint (images):
<ul>
<li> Chapter 1 - Introduction: <a href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap1PDF.zip">PDF Figures</a> / <a href="https://drive.google.com/uc?export=download&id=1udnl5pUOAc8DcAQ7HQwyzP9pwL95ynnv"> SVG Figures</a> / <a href="https://docs.google.com/presentation/d/1IjTqIUvWCJc71b5vEJYte-Dwujcp7rvG/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint Figures</a>
<li> Chapter 2 - Supervised learning: <a href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap2PDF.zip">PDF Figures</a> / <a href="https://drive.google.com/uc?export=download&id=1VSxcU5y1qNFlmd3Lb3uOWyzILuOj1Dla"> SVG Figures</a> / <a href="https://docs.google.com/presentation/d/1Br7R01ROtRWPlNhC_KOommeHAWMBpWtz/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint Figures</a>
<li> Chapter 3 - Shallow neural networks: <a href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap3PDF.zip">PDF Figures</a> / <a href="https://drive.google.com/uc?export=download&id=19kZFWlXhzN82Zx02ByMmSZOO4T41fmqI"> SVG Figures</a> / <a href="https://docs.google.com/presentation/d/1e9M3jB5I9qZ4dCBY90Q3Hwft_i068QVQ/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint Figures</a>
<li> Chapter 4 - Deep neural networks: <a href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap4PDF.zip">PDF Figures</a> / <a href="https://drive.google.com/uc?export=download&id=1ojr0ebsOhzvS04ItAflX2cVmYqHQHZUa"> SVG Figures</a> / <a href="https://docs.google.com/presentation/d/1LTSsmY4mMrJbqXVvoTOCkQwHrRKoYnJj/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint Figures</a>
<li> Chapter 5 - Loss functions: <a href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap5PDF.zip">PDF Figures</a> / <a href="https://drive.google.com/uc?export=download&id=17MJO7fiMpFZVqKeqXTbQ36AMpmR4GizZ"> SVG Figures</a> / <a href="https://docs.google.com/presentation/d/1gcpC_3z9oRp87eMkoco-kdLD-MM54Puk/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint Figures</a>
<li> Chapter 6 - Training models: <a href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap6PDF.zip">PDF Figures</a> / <a href="https://drive.google.com/uc?export=download&id=1VPdhFRnCr9_idTrX0UdHKGAw2shUuwhK"> SVG Figures</a> / <a href="https://docs.google.com/presentation/d/1AKoeggAFBl9yLC7X5tushAGzCCxmB7EY/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint Figures</a>
<li> Chapter 7 - Gradients and initialization: <a href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap7PDF.zip">PDF Figures</a> / <a href="https://drive.google.com/uc?export=download&id=1TTl4gvrTvNbegnml4CoGoKOOd6O8-PGs"> SVG Figures</a> / <a href="https://docs.google.com/presentation/d/11zhB6PI-Dp6Ogmr4IcI6fbvbqNqLyYcz/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint Figures</a>
<li> Chapter 8 - Measuring performance: <a href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap8PDF.zip">PDF Figures</a> / <a href="https://drive.google.com/uc?export=download&id=19eQOnygd_l0DzgtJxXuYnWa4z7QKJrJx"> SVG Figures</a> / <a href="https://docs.google.com/presentation/d/1SHRmJscDLUuQrG7tmysnScb3ZUAqVMZo/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint Figures</a>
<li> Chapter 9 - Regularization: <a href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap9PDF.zip">PDF Figures</a> / <a href="https://drive.google.com/uc?export=download&id=1LprgnUGL7xAM9-jlGZC9LhMPeefjY0r0"> SVG Figures</a> / <a href="https://docs.google.com/presentation/d/1VwIfvjpdfTny6sEfu4ZETwCnw6m8Eg-5/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint Figures</a>
<li> Chapter 10 - Convolutional networks: <a href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap10PDF.zip">PDF Figures</a> / <a href="https://drive.google.com/uc?export=download&id=1-Wb3VzaSvVeRzoUzJbI2JjZE0uwqupM9"> SVG Figures</a> / <a href="https://docs.google.com/presentation/d/1MtfKBC4Y9hWwGqeP6DVwUNbi1j5ncQCg/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint Figures</a>
<li> Chapter 11 - Residual networks: <a href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap11PDF.zip">PDF Figures</a> / <a href="https://drive.google.com/uc?export=download&id=1Mr58jzEVseUAfNYbGWCQyDtEDwvfHRi1"> SVG Figures</a> / <a href="https://docs.google.com/presentation/d/1saY8Faz0KTKAAifUrbkQdLA2qkyEjOPI/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint Figures</a>
<li> Chapter 12 - Transformers: <a href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap12PDF.zip">PDF Figures</a> / <a href="https://drive.google.com/uc?export=download&id=1txzOVNf8-jH4UfJ6SLnrtOfPd1Q3ebzd"> SVG Figures</a> / <a href="https://docs.google.com/presentation/d/1GVNvYWa0WJA6oKg89qZre-UZEhABfm0l/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint Figures</a>
<li> Chapter 13 - Graph neural networks: <a href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap13PDF.zip">PDF Figures</a> / <a href="https://drive.google.com/uc?export=download&id=1lQIV6nRp6LVfaMgpGFhuwEXG-lTEaAwe"> SVG Figures</a> / <a href="https://docs.google.com/presentation/d/1YwF3U82c1mQ74c1WqHVTzLZ0j7GgKaWP/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint Figures</a>
<li> Chapter 14 - Unsupervised learning: <a href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap14PDF.zip">PDF Figures</a> / <a href="https://drive.google.com/uc?export=download&id=1aMbI6iCuUvOywqk5pBOmppJu1L1anqsM"> SVG Figures</a> / <a href="https://docs.google.com/presentation/d/1A-lBGv3NHl4L32NvfFgy1EKeSwY-0UeB/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true"> Powerpoint Figures</a>
<li> Chapter 15 - Generative adversarial networks: <a href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap15PDF.zip">PDF Figures</a> / <a href="https://drive.google.com/uc?export=download&id=1EErnlZCOlXc3HK7m83T2Jh_0NzIUHvtL"> SVG Figures</a> / <a href="https://docs.google.com/presentation/d/10Ernk41ShOTf4IYkMD-l4dJfKATkXH4w/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint Figures</a>
<li> Chapter 16 - Normalizing flows: <a href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap16PDF.zip">PDF Figures</a> / <a href="https://drive.google.com/uc?export=download&id=1B9bxtmdugwtg-b7Y4AdQKAIEVWxjx8l3"> SVG Figures</a> / <a href="https://docs.google.com/presentation/d/1nLLzqb9pdfF_h6i1HUDSyp7kSMIkSUUA/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint Figures</a>
<li> Chapter 17 - Variational autoencoders: <a href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap17PDF.zip">PDF Figures</a> / <a href="https://drive.google.com/uc?export=download&id=1SNtNIY7khlHQYMtaOH-FosSH3kWwL4b7"> SVG Figures</a> / <a href="https://docs.google.com/presentation/d/1lQE4Bu7-LgvV2VlJOt_4dQT-kusYl7Vo/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint Figures</a>
<li> Chapter 18 - Diffusion models: <a href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap18PDF.zip">PDF Figures</a> / <a href="https://docs.google.com/presentation/d/1x_ufIBtVPzWUvRieKMkpw5SdRjXWwdfR/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true"> PowerPoint Figures</a>
<li> Chapter 19 - Deep reinforcement learning: <a href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap19PDF.zip">PDF Figures</a> / <a href="https://drive.google.com/uc?export=download&id=1a5WUoF7jeSgwC_PVdckJi1Gny46fCqh0"> SVG Figures</a> / <a href="https://docs.google.com/presentation/d/1TnYmVbFNhmMFetbjyfXGmkxp1EHauMqr/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true"> PowerPoint Figures </a>
<li> Chapter 20 - Why does deep learning work?: <a href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap20PDF.zip">PDF Figures</a> / <a href="https://drive.google.com/uc?export=download&id=1M2d0DHEgddAQoIedKSDTTt7m1ZdmBLQ3"> SVG Figures</a> / <a href="https://docs.google.com/presentation/d/1coxF4IsrCzDTLrNjRagHvqB_FBy10miA/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true"> PowerPoint Figures</a>
<li> Chapter 21 - Deep learning and ethics: <a href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLChap21PDF.zip">PDF Figures</a> / <a href="https://drive.google.com/uc?export=download&id=1jixmFfwmZkW_UVYzcxmDcMsdFFtnZ0bU"> SVG Figures</a>/ <a href="https://docs.google.com/presentation/d/1EtfzanZYILvi9_-Idm28zD94I_6OrN9R/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">PowerPoint Figures</a>
<li> Appendices - <a href="https://github.com/udlbook/udlbook/raw/main/PDFFigures/UDLAppendixPDF.zip">PDF Figures</a> / <a href="https://drive.google.com/uc?export=download&id=1k2j7hMN40ISPSg9skFYWFL3oZT7r8v-l"> SVG Figures</a> / <a href="https://docs.google.com/presentation/d/1_2cJHRnsoQQHst0rwZssv-XH4o5SEHks/edit?usp=drive_link&ouid=110441678248547154185&rtpof=true&sd=true">Powerpoint Figures</a>
</ul>
Instructions for editing figures / equations can be found <a href="https://drive.google.com/uc?export=download&id=1T_MXXVR4AfyMnlEFI-UVDh--FXI5deAp/">here</a>.</p>
<h2>Resources for students</h2>
<p>Answers to selected questions: <a href="https://github.com/udlbook/udlbook/raw/main/UDL_Answer_Booklet_Students.pdf">PDF</a></p>
<p>Python notebooks:</p>
<ul>
<li> Chapter 1 - Introduction: <a href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap01/1_1_BackgroundMathematics.ipynb">1.1</a>
<li> Chapter 2 - Supervised learning: <a href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap02/2_1_Supervised_Learning.ipynb">2.1</a>
<li> Chapter 3 - Shallow neural networks: <a href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap03/3_1_Shallow_Networks_I.ipynb">3.1</a>, <a href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap03/3_2_Shallow_Networks_II.ipynb">3.2</a>, <a href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap03/3_3_Shallow_Network_Regions.ipynb">3.3</a>, <a href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap03/3_4_Activation_Functions.ipynb">3.4</a>
<li> Chapter 4 - Deep neural networks: <a href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap04/4_1_Composing_Networks.ipynb">4.1</a>, <a href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap04/4_2_Clipping_functions.ipynb">4.2</a>, <a href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap04/4_3_Deep_Networks.ipynb">4.3</a>
<li> Chapter 5 - Loss functions: <a href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap04/4_3_Deep_Networks.ipynb">5.1</a>, <a href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap05/5_2_Binary_Cross_Entropy_Loss.ipynb">5.2</a>, <a href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap05/5_3_Multiclass_Cross_entropy_Loss.ipynb">5.3</a>
<li> Chapter 6 - Training models <a href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap06/6_1_Line_Search.ipynb">6.1</a>, <a href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap06/6_2_Gradient_Descent.ipynb">6.2</a>, <a href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap06/6_3_Stochastic_Gradient_Descent.ipynb">6.3</a>, <a href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap06/6_4_Momentum.ipynb">6.4</a>, <a href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap06/6_5_Adam.ipynb">6.5</a>
<li> Chapter 7 - Gradients and initialization <a href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap07/7_1_Backpropagation_in_Toy_Model.ipynb">7.1</a>, <a href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap07/7_2_Backpropagation.ipynb">7.2</a>, <a href="https://github.com/udlbook/udlbook/blob/main/Notebooks/Chap07/7_3_Initialization.ipynb">7.3</a>
<li> Chapter 8 - Measuring performance (coming soon)
<li> Chapter 9 - Regularization (coming soon)
<li> Chapter 10 - Convolutional networks (coming soon)
<li> Chapter 11 - Residual networks (coming soon)
<li> Chapter 12 - Transformers (coming soon)
<li> Chapter 13 - Graph neural networks (coming soon)
<li> Chapter 14 - Unsupervised learning (coming soon)
<li> Chapter 15 - Generative adversarial networks (coming soon)
<li> Chapter 16 - Normalizing flows (coming soon)
<li> Chapter 17 - Variational autoencoders (coming soon)
<li> Chapter 18 - Diffusion models (coming soon)
<li> Chapter 19 - Deep reinforcement learning (coming soon)
<li> Chapter 20 - Why does deep learning work? (coming soon)
<li> Chapter 21 - Deep learning and ethics (coming soon)
</ul>
<br>
<h2>Citation:</h2>
<pre><code>
@book{prince2023understanding,
author = "Simon J.D. Prince",
title = "Understanding Deep Learning",
publisher = "MIT Press",
year = 2023,
url = "http://udlbook.com"
}
</code></pre>