forked from udlbook/udlbook
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
1 changed file
with
298 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,298 @@ | ||
| { | ||
| "nbformat": 4, | ||
| "nbformat_minor": 0, | ||
| "metadata": { | ||
| "colab": { | ||
| "provenance": [], | ||
| "authorship_tag": "ABX9TyMTtWwAtwIZJoIsfGehxMMC", | ||
| "include_colab_link": true | ||
| }, | ||
| "kernelspec": { | ||
| "name": "python3", | ||
| "display_name": "Python 3" | ||
| }, | ||
| "language_info": { | ||
| "name": "python" | ||
| } | ||
| }, | ||
| "cells": [ | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": { | ||
| "id": "view-in-github", | ||
| "colab_type": "text" | ||
| }, | ||
| "source": [ | ||
| "<a href=\"https://colab.research.google.com/github/udlbook/udlbook/blob/main/CM20315_Intro.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "source": [ | ||
| "\n", | ||
| "# **CM20315 Background Mathematics**\n", | ||
| "\n", | ||
| "The purpose of this Python notebook is to make sure you can use CoLab and to familiarize yourself with some of the background mathematical concepts that you are going to need to understand deep learning. <br><br> It's not meant to be difficult and it may be that you know some or all of this information already.<br><br> Maths is *NOT* a spectator sport. You won't learn it by just listening to lectures or reading books. It really helps to interact with it and explore yourself. <br><br> Work through the cells below, running each cell in turn. In various places you will see the words **\"TO DO\"**. Follow the instructions at these places. If you can't figure out what is going on or how to complete the section, feel free to ask your fellow students or the TAs in the practical session or via Moodle. This does not count toward your final marks, but complete=ing it will help you do well in your coursework and exam and reduce your stress levels later on.\n" | ||
| ], | ||
| "metadata": { | ||
| "id": "s5zzKSOusPOB" | ||
| } | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": { | ||
| "id": "aUAjBbqzivMY" | ||
| }, | ||
| "outputs": [], | ||
| "source": [ | ||
| "# Imports math library\n", | ||
| "import numpy as np\n", | ||
| "# Imports plotting library\n", | ||
| "import matplotlib.pyplot as plt" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "source": [ | ||
| "**Linear functions**<br> In this course, we will be using the term *linear equation* to mean a weighted sum of inputs plus an offset. If there is just one input $x$, then this is a straight line:\n", | ||
| "\n", | ||
| "\\begin{equation}y=\\beta+\\omega x,\\end{equation} <br>\n", | ||
| "\n", | ||
| "where $\\beta$ is the y-intercept of the linear and $\\omega$ is the slope of the line. When there are two inputs $x_{1}$ and $x_{2}$, then this becomes:\n", | ||
| "\n", | ||
| "\\begin{equation}y=\\beta+\\omega_1 x_1 + \\omega_2 x_2.\\end{equation} <br><br>\n", | ||
| "\n", | ||
| "Any other functions are by definition **non-linear**.\n", | ||
| "\n", | ||
| "\n" | ||
| ], | ||
| "metadata": { | ||
| "id": "WV2Dl6owme2d" | ||
| } | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "source": [ | ||
| "# Define a linear function with just one input, x\n", | ||
| "def linear_function_1D(x,beta,omega):\n", | ||
| " # TODO -- replace the code lin below with formula for 1D linear equation\n", | ||
| " y = x\n", | ||
| " return y" | ||
| ], | ||
| "metadata": { | ||
| "id": "WeFK4AvTotd8" | ||
| }, | ||
| "execution_count": null, | ||
| "outputs": [] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "source": [ | ||
| "# Plot the 1D linear function \n", | ||
| "\n", | ||
| "# Define an array of x values from 0 to 10 with increments of 0.1\n", | ||
| "# https://numpy.org/doc/stable/reference/generated/numpy.arange.html\n", | ||
| "x = np.arange(0.0,10.0, 0.01) \n", | ||
| "# Compute y using the function you filled in above\n", | ||
| "beta = 0.0; omega = 1.0\n", | ||
| "y = linear_function_1D(x,beta,omega)\n", | ||
| "\n", | ||
| "# Plot this function\n", | ||
| "fig, ax = plt.subplots()\n", | ||
| "ax.plot(x,y,'r-')\n", | ||
| "ax.set_ylim([0,10]);ax.set_xlim([0,10])\n", | ||
| "ax.set_xlabel('x'); ax.set_ylabel('y')\n", | ||
| "plt.show\n", | ||
| "\n", | ||
| "# TODO -- experiment with changing the values of beta and omega\n", | ||
| "# to understand what they do. Try to make a line\n", | ||
| "# that crosses the y-axis at y=10 and the x-axis at x=5" | ||
| ], | ||
| "metadata": { | ||
| "id": "eimhJ8_jpmEp" | ||
| }, | ||
| "execution_count": null, | ||
| "outputs": [] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "source": [ | ||
| "Now let's investigate a 2D linear function" | ||
| ], | ||
| "metadata": { | ||
| "id": "AedfvD9dxShZ" | ||
| } | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "source": [ | ||
| "# Code to draw 2D function -- read it so you know what is going on, but you don't have to change it\n", | ||
| "def draw_2D_function(x1_mesh, x2_mesh, y):\n", | ||
| " fig, ax = plt.subplots()\n", | ||
| " fig.set_size_inches(7,7)\n", | ||
| " ax.contourf(x1_mesh, x2_mesh, y, levels=256 ,cmap = 'hot', vmin=-10,vmax=10.0)\n", | ||
| " ax.set_xlabel('x1');ax.set_ylabel('x2')\n", | ||
| " levels = np.arange(-10,10,1.0)\n", | ||
| " ax.contour(x1_mesh, x2_mesh, y, levels, cmap='winter')\n", | ||
| " plt.show()" | ||
| ], | ||
| "metadata": { | ||
| "id": "57Gvkk-Ir_7b" | ||
| }, | ||
| "execution_count": null, | ||
| "outputs": [] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "source": [ | ||
| "# Define a linear function with two inputs, x1 and x2\n", | ||
| "def linear_function_2D(x1,x2,beta,omega1,omega2):\n", | ||
| " # TODO -- replace the code line below with formula for 2D linear equation\n", | ||
| " y = x1\n", | ||
| " return y" | ||
| ], | ||
| "metadata": { | ||
| "id": "YxeNhrXMzkZR" | ||
| }, | ||
| "execution_count": null, | ||
| "outputs": [] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "source": [ | ||
| "# Plot the 2D function\n", | ||
| "\n", | ||
| "# Make 2D array of x and y points\n", | ||
| "x1 = np.arange(0.0, 10.0, 0.1)\n", | ||
| "x2 = np.arange(0.0, 10.0, 0.1)\n", | ||
| "x1,x2 = np.meshgrid(x1,x2) # https://www.geeksforgeeks.org/numpy-meshgrid-function/\n", | ||
| "\n", | ||
| "# Compute the 2D function for given values of omega1, omega2\n", | ||
| "beta = 0.0; omega1 = 1.0; omega2 = -0.5\n", | ||
| "y = linear_function_2D(x1,x2,beta, omega1, omega2)\n", | ||
| "\n", | ||
| "# Draw the function. \n", | ||
| "# Color represents y value (brighter = higher value)\n", | ||
| "# Black = -10 or less, White = +10 or more\n", | ||
| "# 0 = mid orange\n", | ||
| "# Lines are conoturs where value is equal\n", | ||
| "draw_2D_function(x1,x2,y)\n", | ||
| "\n", | ||
| "# TODO\n", | ||
| "# Predict what this plot will look like if you set omega_1 to zero\n", | ||
| "# Change the code and see if you are right.\n", | ||
| "\n", | ||
| "# TODO\n", | ||
| "# Predict what this plot will look like if you set omega_2 to zero\n", | ||
| "# Change the code and see if you are right.\n", | ||
| "\n", | ||
| "#TODO \n", | ||
| "# Predict what this plot will look like if you set beta to -5\n", | ||
| "# Change the code and see if you are correct\n" | ||
| ], | ||
| "metadata": { | ||
| "id": "rn_UBRDBysmR" | ||
| }, | ||
| "execution_count": null, | ||
| "outputs": [] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "source": [ | ||
| "Often we will want to compute many linear functions at the same time. For example, we might have three inputs, $x_1$, $x_2$, and $x_3$ and want to compute two linear functions giving $y_1$ and $y_2$. Of course, we could do this by just running each equation separately,<br><br>\n", | ||
| "\n", | ||
| "\\begin{eqnarray}y_1 &=& \\beta_1 + \\omega_{11} x_1 + \\omega_{12} x_2 + \\omega_{13} x_3\\\\\n", | ||
| "y_2 &=& \\beta_2 + \\omega_{21} x_1 + \\omega_{22} x_2 + \\omega_{23} x_3.\n", | ||
| "\\end{eqnarray}<br>\n", | ||
| "\n", | ||
| "However, we can write it more compactly with vectors and matrices:\n", | ||
| "\n", | ||
| "\\begin{equation}\n", | ||
| "\\begin{bmatrix} y_1\\\\ y_2 \\end{bmatrix} = \\begin{bmatrix}\\beta_{1}\\\\\\beta_{2}\\end{bmatrix}+ \\begin{bmatrix}\\omega_{11}&\\omega_{12}&\\omega_{13}\\\\\\omega_{21}&\\omega_{22}&\\omega_{23}\\end{bmatrix}\\begin{bmatrix}x_{1}\\\\x_{2}\\\\x_{3}\\end{bmatrix},\n", | ||
| "\\end{equation}<br>\n", | ||
| "or \n", | ||
| "\n", | ||
| "\\begin{equation}\n", | ||
| "\\mathbf{y} = \\boldsymbol\\beta +\\boldsymbol\\Omega\\mathbf{x}.\n", | ||
| "\\end{equation}\n", | ||
| "\n", | ||
| "for short. Here, lowercase bold symbols are used for vectors. Upper case bold symbols are used for matrices.\n", | ||
| "\n" | ||
| ], | ||
| "metadata": { | ||
| "id": "i8tLwpls476R" | ||
| } | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "source": [ | ||
| "# Define a linear function with three inputs, x1, x2, and x_3\n", | ||
| "def linear_function_3D(x1,x2,x3,beta,omega1,omega2,omega3):\n", | ||
| " # TODO -- replace the code below with formula for a single 3D linear equation\n", | ||
| " y = x1\n", | ||
| " return y" | ||
| ], | ||
| "metadata": { | ||
| "id": "MjHXMavh9IUz" | ||
| }, | ||
| "execution_count": null, | ||
| "outputs": [] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "source": [ | ||
| "Let's compute two linear equations, using both the individual equations and the vector / matrix form and check they give the same answer" | ||
| ], | ||
| "metadata": { | ||
| "id": "fGzVJQ6N-mHJ" | ||
| } | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "source": [ | ||
| "# Define the parameters\n", | ||
| "beta1 = 0.5; beta2 = 0.2\n", | ||
| "omega11 = -1.0 ; omega12 = 0.4; omega13 = -0.3\n", | ||
| "omega21 = 0.1 ; omega22 = 0.1; omega23 = 1.2\n", | ||
| "\n", | ||
| "# Define the inputs\n", | ||
| "x1 = 4 ; x2 =-1; x3 = 2\n", | ||
| "\n", | ||
| "# Compute using the individual equations\n", | ||
| "y1 = linear_function_3D(x1,x2,x3,beta1,omega11,omega12,omega13)\n", | ||
| "y2 = linear_function_3D(x1,x2,x3,beta2,omega21,omega22,omega23)\n", | ||
| "print(\"Individual equations\")\n", | ||
| "print('y1 = %3.3f\\ny2 = %3.3f'%((y1,y2)))\n", | ||
| "\n", | ||
| "# Define vectors and matrices\n", | ||
| "beta_vec = np.array([[beta1],[beta2]])\n", | ||
| "omega_mat = np.array([[omega11,omega12,omega13],[omega21,omega22,omega23]])\n", | ||
| "x_vec = np.array([[x1], [x2], [x3]])\n", | ||
| "\n", | ||
| "# Compute with vector/matrix form\n", | ||
| "y_vec = beta_vec+np.matmul(omega_mat, x_vec)\n", | ||
| "print(\"Matrix/vector form\")\n", | ||
| "print('y1= %3.3f\\ny2 = %3.3f'%((y_vec[0],y_vec[1])))\n" | ||
| ], | ||
| "metadata": { | ||
| "id": "Swd_bFIE9p2n" | ||
| }, | ||
| "execution_count": null, | ||
| "outputs": [] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "source": [ | ||
| "(Optional, but highly recommended) Think that you have this down? Here's some stuff to think about:<br>\n", | ||
| "\n", | ||
| "1. A single linear equation with three inputs (i.e. **linear_function_3D()**) associates a value y with each point in a 3D space ($x_1$,$x_2$,$x_3$). Is it possible to visualize this? What value is at position (0,0,0)?\n", | ||
| "\n", | ||
| "2. Write code to compute three linear equations with two inputs ($x_1$, $x_2$) using both the individual equations and the matrix form (you can make up any values for the inputs $\\beta_{i}$ and the slopes $\\omega_{ij}$." | ||
| ], | ||
| "metadata": { | ||
| "id": "3LGRoTMLU8ZU" | ||
| } | ||
| } | ||
| ] | ||
| } |