Merge pull request briandalessandro#33 from Z-Momin/master

Bias Variance lecture edits

ipython/python35/Lecture_BiasVariance_3.ipynb

@@ -40,9 +40,7 @@
   {
    "cell_type": "code",
    "execution_count": 1,
-   "metadata": {
-    "collapsed": false
-   },
+   "metadata": {},
    "outputs": [
     {
      "data": {
@@ -99,9 +97,7 @@
   {
    "cell_type": "code",
    "execution_count": 2,
-   "metadata": {
-    "collapsed": false
-   },
+   "metadata": {},
    "outputs": [
     {
      "data": {
@@ -158,9 +154,7 @@
   {
    "cell_type": "code",
    "execution_count": 3,
-   "metadata": {
-    "collapsed": false
-   },
+   "metadata": {},
    "outputs": [
     {
      "name": "stderr",
@@ -208,9 +202,7 @@
   {
    "cell_type": "code",
    "execution_count": 5,
-   "metadata": {
-    "collapsed": false
-   },
+   "metadata": {},
    "outputs": [
     {
      "data": {
@@ -257,13 +249,21 @@
     "\n",
     "<ul>\n",
     "    <li><u>Family of functions $\\mathbb{F}$</u>: This is a finite set of functions that share some structural properties. For instance, all decision trees on a set of features with depth=10, or all linear functions of the form $f(X)=\\beta^T \\: X$. Generally any classification algorithm can be placed into some particular family $\\mathbb{F}$.</li><br>\n",
+    "</ul>\n",
     "\n",
+    "<ul>\n",
     "    <li><u>Expected vs. Empirical risk:</u>The expected risk is a theoretical property we could know if we knew the true distribution $P(X,Y)$. For MSE the expected risk of a given function $f$ is $E[MSE(f)]= \\int ((y-f)^2 \\: dP(X,Y)$. Since we generally don't know $P(X,Y)$, we have to approximate the expected risk with the empirical risk, which is defined as: $E_j[MSE(f)] =n^{-1}\\sum ((y_i - f_i)^2$, where the sum is taken over some finite data set $j$.</li><br>\n",
+    "</ul>\n",
     "\n",
+    "<ul>\n",
     "    <li><u>The true risk minimizer $f^*$</u>: This is the function $f^*$ that best approximates the true $E[Y|X=x]$ and is the one that minimizes the expected risk across all families of functions. It can be defined as: $f^*= \\underset{f} {\\mathrm{argmin}} \\int((Y-f)^2 \\: dP(X,Y)$. In other words, this is the absolute truth that we wish we knew, but generally have to approximate using a finite data sample and some estimation procedure.</li><br>\n",
+    "</ul>\n",
     "\n",
+    "<ul>\n",
     "    <li><u>The expected risk minimizer from family $\\mathbb{F}$,   $\\:\\:f^{\\mathbb{F}}$</u>:This is similar to the true risk minimizer, only that we restrict the exploration of candidate functions to only those included in $\\mathbb{F}$. The exact definition is: $f^{\\mathbb{F}}= \\underset{f \\in \\mathbb{F}} {\\mathrm{argmin}} \\int((Y-f)^2 \\: dP(X,Y)$. One way to think about this is, using the above example, imagine we had infinite data but could only fit some non-linear data scatter with a linear model - the result wold be the expected risk minimizer from the family of linear functions on $X$.</li><br> \n",
+    "</ul>\n",
     "\n",
+    "<ul>\n",
     "    <li><u>The emprical risk minimizer from family $\\mathbb{F}$,   $\\:\\:f_j^{\\mathbb{F}}$</u> This is similar to the above definition, but using the empirical risk definition as opposed to the expected risk definition. When we fit a model on finite data, this is generally the function we end up with.</li>\n",
     "</ul><br><br>\n",
     "\n",
@@ -319,9 +319,7 @@
   {
    "cell_type": "code",
    "execution_count": 11,
-   "metadata": {
-    "collapsed": false
-   },
+   "metadata": {},
    "outputs": [
     {
      "data": {
@@ -395,23 +393,23 @@
  "metadata": {
   "anaconda-cloud": {},
   "kernelspec": {
-   "display_name": "Python [Root]",
+   "display_name": "Python 3",
    "language": "python",
-   "name": "Python [Root]"
+   "name": "python3"
   },
   "language_info": {
    "codemirror_mode": {
     "name": "ipython",
-    "version": 2
+    "version": 3
    },
    "file_extension": ".py",
    "mimetype": "text/x-python",
    "name": "python",
    "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython2",
-   "version": "2.7.12"
+   "pygments_lexer": "ipython3",
+   "version": "3.8.5"
   }
  },
  "nbformat": 4,
- "nbformat_minor": 0
+ "nbformat_minor": 1
 }