correlation_regression_ANOVA.ipynb

{"cells":[{"metadata":{},"cell_type":"markdown","source":"<h2> Correlation and Causation</h2>"},{"metadata":{},"cell_type":"markdown","source":"<b> By Michael Kumakech</b>"},{"metadata":{},"cell_type":"markdown","source":"In this section, we will explore several methods to see if certain characteristics or features can be used to predict car price."},{"metadata":{},"cell_type":"markdown","source":"<b>Correlation:</b> a measure of the extent of interdependence between variables.\n\n<b>Causation: </b>the relationship between cause and effect between two variables"},{"metadata":{},"cell_type":"markdown","source":"Pearson Correlation is the default method of the function \"corr\". Like before we can calculate the Pearson Correlation of the of the 'int64' or 'float64' variable"},{"metadata":{"trusted":true},"cell_type":"code","source":"import pandas as pd\nimport numpy as np","execution_count":2,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"path='https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/DA0101EN/automobileEDA.csv'\ndf = pd.read_csv(path)\ndf.head()","execution_count":3,"outputs":[{"output_type":"execute_result","execution_count":3,"data":{"text/plain":"   symboling  normalized-losses         make aspiration num-of-doors  \\\n0          3                122  alfa-romero        std          two   \n1          3                122  alfa-romero        std          two   \n2          1                122  alfa-romero        std          two   \n3          2                164         audi        std         four   \n4          2                164         audi        std         four   \n\n    body-style drive-wheels engine-location  wheel-base    length  ...  \\\n0  convertible          rwd           front        88.6  0.811148  ...   \n1  convertible          rwd           front        88.6  0.811148  ...   \n2    hatchback          rwd           front        94.5  0.822681  ...   \n3        sedan          fwd           front        99.8  0.848630  ...   \n4        sedan          4wd           front        99.4  0.848630  ...   \n\n   compression-ratio  horsepower  peak-rpm city-mpg highway-mpg    price  \\\n0                9.0       111.0    5000.0       21          27  13495.0   \n1                9.0       111.0    5000.0       21          27  16500.0   \n2                9.0       154.0    5000.0       19          26  16500.0   \n3               10.0       102.0    5500.0       24          30  13950.0   \n4                8.0       115.0    5500.0       18          22  17450.0   \n\n  city-L/100km  horsepower-binned  diesel  gas  \n0    11.190476             Medium       0    1  \n1    11.190476             Medium       0    1  \n2    12.368421             Medium       0    1  \n3     9.791667             Medium       0    1  \n4    13.055556             Medium       0    1  \n\n[5 rows x 29 columns]","text/html":"<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>symboling</th>\n      <th>normalized-losses</th>\n      <th>make</th>\n      <th>aspiration</th>\n      <th>num-of-doors</th>\n      <th>body-style</th>\n      <th>drive-wheels</th>\n      <th>engine-location</th>\n      <th>wheel-base</th>\n      <th>length</th>\n      <th>...</th>\n      <th>compression-ratio</th>\n      <th>horsepower</th>\n      <th>peak-rpm</th>\n      <th>city-mpg</th>\n      <th>highway-mpg</th>\n      <th>price</th>\n      <th>city-L/100km</th>\n      <th>horsepower-binned</th>\n      <th>diesel</th>\n      <th>gas</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>3</td>\n      <td>122</td>\n      <td>alfa-romero</td>\n      <td>std</td>\n      <td>two</td>\n      <td>convertible</td>\n      <td>rwd</td>\n      <td>front</td>\n      <td>88.6</td>\n      <td>0.811148</td>\n      <td>...</td>\n      <td>9.0</td>\n      <td>111.0</td>\n      <td>5000.0</td>\n      <td>21</td>\n      <td>27</td>\n      <td>13495.0</td>\n      <td>11.190476</td>\n      <td>Medium</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>3</td>\n      <td>122</td>\n      <td>alfa-romero</td>\n      <td>std</td>\n      <td>two</td>\n      <td>convertible</td>\n      <td>rwd</td>\n      <td>front</td>\n      <td>88.6</td>\n      <td>0.811148</td>\n      <td>...</td>\n      <td>9.0</td>\n      <td>111.0</td>\n      <td>5000.0</td>\n      <td>21</td>\n      <td>27</td>\n      <td>16500.0</td>\n      <td>11.190476</td>\n      <td>Medium</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>1</td>\n      <td>122</td>\n      <td>alfa-romero</td>\n      <td>std</td>\n      <td>two</td>\n      <td>hatchback</td>\n      <td>rwd</td>\n      <td>front</td>\n      <td>94.5</td>\n      <td>0.822681</td>\n      <td>...</td>\n      <td>9.0</td>\n      <td>154.0</td>\n      <td>5000.0</td>\n      <td>19</td>\n      <td>26</td>\n      <td>16500.0</td>\n      <td>12.368421</td>\n      <td>Medium</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>2</td>\n      <td>164</td>\n      <td>audi</td>\n      <td>std</td>\n      <td>four</td>\n      <td>sedan</td>\n      <td>fwd</td>\n      <td>front</td>\n      <td>99.8</td>\n      <td>0.848630</td>\n      <td>...</td>\n      <td>10.0</td>\n      <td>102.0</td>\n      <td>5500.0</td>\n      <td>24</td>\n      <td>30</td>\n      <td>13950.0</td>\n      <td>9.791667</td>\n      <td>Medium</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>2</td>\n      <td>164</td>\n      <td>audi</td>\n      <td>std</td>\n      <td>four</td>\n      <td>sedan</td>\n      <td>4wd</td>\n      <td>front</td>\n      <td>99.4</td>\n      <td>0.848630</td>\n      <td>...</td>\n      <td>8.0</td>\n      <td>115.0</td>\n      <td>5500.0</td>\n      <td>18</td>\n      <td>22</td>\n      <td>17450.0</td>\n      <td>13.055556</td>\n      <td>Medium</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n  </tbody>\n</table>\n<p>5 rows × 29 columns</p>\n</div>"},"metadata":{}}]},{"metadata":{},"cell_type":"markdown","source":"<h2> Descriptive Statistical Analysis</h2>"},{"metadata":{},"cell_type":"markdown","source":"The describe function automatically computes basic statistics for all continuous variables. Any NaN values are automatically skipped in these statistics.\n\nThis will show:\n\nthe count of that variable\nthe mean\nthe standard deviation (std)\nthe minimum value\nthe IQR (Interquartile Range: 25%, 50% and 75%)\nthe maximum value"},{"metadata":{"trusted":true},"cell_type":"code","source":"df.describe()","execution_count":5,"outputs":[{"output_type":"execute_result","execution_count":5,"data":{"text/plain":"        symboling  normalized-losses  wheel-base      length       width  \\\ncount  201.000000          201.00000  201.000000  201.000000  201.000000   \nmean     0.840796          122.00000   98.797015    0.837102    0.915126   \nstd      1.254802           31.99625    6.066366    0.059213    0.029187   \nmin     -2.000000           65.00000   86.600000    0.678039    0.837500   \n25%      0.000000          101.00000   94.500000    0.801538    0.890278   \n50%      1.000000          122.00000   97.000000    0.832292    0.909722   \n75%      2.000000          137.00000  102.400000    0.881788    0.925000   \nmax      3.000000          256.00000  120.900000    1.000000    1.000000   \n\n           height  curb-weight  engine-size        bore      stroke  \\\ncount  201.000000   201.000000   201.000000  201.000000  197.000000   \nmean    53.766667  2555.666667   126.875622    3.330692    3.256904   \nstd      2.447822   517.296727    41.546834    0.268072    0.319256   \nmin     47.800000  1488.000000    61.000000    2.540000    2.070000   \n25%     52.000000  2169.000000    98.000000    3.150000    3.110000   \n50%     54.100000  2414.000000   120.000000    3.310000    3.290000   \n75%     55.500000  2926.000000   141.000000    3.580000    3.410000   \nmax     59.800000  4066.000000   326.000000    3.940000    4.170000   \n\n       compression-ratio  horsepower     peak-rpm    city-mpg  highway-mpg  \\\ncount         201.000000  201.000000   201.000000  201.000000   201.000000   \nmean           10.164279  103.405534  5117.665368   25.179104    30.686567   \nstd             4.004965   37.365700   478.113805    6.423220     6.815150   \nmin             7.000000   48.000000  4150.000000   13.000000    16.000000   \n25%             8.600000   70.000000  4800.000000   19.000000    25.000000   \n50%             9.000000   95.000000  5125.369458   24.000000    30.000000   \n75%             9.400000  116.000000  5500.000000   30.000000    34.000000   \nmax            23.000000  262.000000  6600.000000   49.000000    54.000000   \n\n              price  city-L/100km      diesel         gas  \ncount    201.000000    201.000000  201.000000  201.000000  \nmean   13207.129353      9.944145    0.099502    0.900498  \nstd     7947.066342      2.534599    0.300083    0.300083  \nmin     5118.000000      4.795918    0.000000    0.000000  \n25%     7775.000000      7.833333    0.000000    1.000000  \n50%    10295.000000      9.791667    0.000000    1.000000  \n75%    16500.000000     12.368421    0.000000    1.000000  \nmax    45400.000000     18.076923    1.000000    1.000000  ","text/html":"<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>symboling</th>\n      <th>normalized-losses</th>\n      <th>wheel-base</th>\n      <th>length</th>\n      <th>width</th>\n      <th>height</th>\n      <th>curb-weight</th>\n      <th>engine-size</th>\n      <th>bore</th>\n      <th>stroke</th>\n      <th>compression-ratio</th>\n      <th>horsepower</th>\n      <th>peak-rpm</th>\n      <th>city-mpg</th>\n      <th>highway-mpg</th>\n      <th>price</th>\n      <th>city-L/100km</th>\n      <th>diesel</th>\n      <th>gas</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>count</th>\n      <td>201.000000</td>\n      <td>201.00000</td>\n      <td>201.000000</td>\n      <td>201.000000</td>\n      <td>201.000000</td>\n      <td>201.000000</td>\n      <td>201.000000</td>\n      <td>201.000000</td>\n      <td>201.000000</td>\n      <td>197.000000</td>\n      <td>201.000000</td>\n      <td>201.000000</td>\n      <td>201.000000</td>\n      <td>201.000000</td>\n      <td>201.000000</td>\n      <td>201.000000</td>\n      <td>201.000000</td>\n      <td>201.000000</td>\n      <td>201.000000</td>\n    </tr>\n    <tr>\n      <th>mean</th>\n      <td>0.840796</td>\n      <td>122.00000</td>\n      <td>98.797015</td>\n      <td>0.837102</td>\n      <td>0.915126</td>\n      <td>53.766667</td>\n      <td>2555.666667</td>\n      <td>126.875622</td>\n      <td>3.330692</td>\n      <td>3.256904</td>\n      <td>10.164279</td>\n      <td>103.405534</td>\n      <td>5117.665368</td>\n      <td>25.179104</td>\n      <td>30.686567</td>\n      <td>13207.129353</td>\n      <td>9.944145</td>\n      <td>0.099502</td>\n      <td>0.900498</td>\n    </tr>\n    <tr>\n      <th>std</th>\n      <td>1.254802</td>\n      <td>31.99625</td>\n      <td>6.066366</td>\n      <td>0.059213</td>\n      <td>0.029187</td>\n      <td>2.447822</td>\n      <td>517.296727</td>\n      <td>41.546834</td>\n      <td>0.268072</td>\n      <td>0.319256</td>\n      <td>4.004965</td>\n      <td>37.365700</td>\n      <td>478.113805</td>\n      <td>6.423220</td>\n      <td>6.815150</td>\n      <td>7947.066342</td>\n      <td>2.534599</td>\n      <td>0.300083</td>\n      <td>0.300083</td>\n    </tr>\n    <tr>\n      <th>min</th>\n      <td>-2.000000</td>\n      <td>65.00000</td>\n      <td>86.600000</td>\n      <td>0.678039</td>\n      <td>0.837500</td>\n      <td>47.800000</td>\n      <td>1488.000000</td>\n      <td>61.000000</td>\n      <td>2.540000</td>\n      <td>2.070000</td>\n      <td>7.000000</td>\n      <td>48.000000</td>\n      <td>4150.000000</td>\n      <td>13.000000</td>\n      <td>16.000000</td>\n      <td>5118.000000</td>\n      <td>4.795918</td>\n      <td>0.000000</td>\n      <td>0.000000</td>\n    </tr>\n    <tr>\n      <th>25%</th>\n      <td>0.000000</td>\n      <td>101.00000</td>\n      <td>94.500000</td>\n      <td>0.801538</td>\n      <td>0.890278</td>\n      <td>52.000000</td>\n      <td>2169.000000</td>\n      <td>98.000000</td>\n      <td>3.150000</td>\n      <td>3.110000</td>\n      <td>8.600000</td>\n      <td>70.000000</td>\n      <td>4800.000000</td>\n      <td>19.000000</td>\n      <td>25.000000</td>\n      <td>7775.000000</td>\n      <td>7.833333</td>\n      <td>0.000000</td>\n      <td>1.000000</td>\n    </tr>\n    <tr>\n      <th>50%</th>\n      <td>1.000000</td>\n      <td>122.00000</td>\n      <td>97.000000</td>\n      <td>0.832292</td>\n      <td>0.909722</td>\n      <td>54.100000</td>\n      <td>2414.000000</td>\n      <td>120.000000</td>\n      <td>3.310000</td>\n      <td>3.290000</td>\n      <td>9.000000</td>\n      <td>95.000000</td>\n      <td>5125.369458</td>\n      <td>24.000000</td>\n      <td>30.000000</td>\n      <td>10295.000000</td>\n      <td>9.791667</td>\n      <td>0.000000</td>\n      <td>1.000000</td>\n    </tr>\n    <tr>\n      <th>75%</th>\n      <td>2.000000</td>\n      <td>137.00000</td>\n      <td>102.400000</td>\n      <td>0.881788</td>\n      <td>0.925000</td>\n      <td>55.500000</td>\n      <td>2926.000000</td>\n      <td>141.000000</td>\n      <td>3.580000</td>\n      <td>3.410000</td>\n      <td>9.400000</td>\n      <td>116.000000</td>\n      <td>5500.000000</td>\n      <td>30.000000</td>\n      <td>34.000000</td>\n      <td>16500.000000</td>\n      <td>12.368421</td>\n      <td>0.000000</td>\n      <td>1.000000</td>\n    </tr>\n    <tr>\n      <th>max</th>\n      <td>3.000000</td>\n      <td>256.00000</td>\n      <td>120.900000</td>\n      <td>1.000000</td>\n      <td>1.000000</td>\n      <td>59.800000</td>\n      <td>4066.000000</td>\n      <td>326.000000</td>\n      <td>3.940000</td>\n      <td>4.170000</td>\n      <td>23.000000</td>\n      <td>262.000000</td>\n      <td>6600.000000</td>\n      <td>49.000000</td>\n      <td>54.000000</td>\n      <td>45400.000000</td>\n      <td>18.076923</td>\n      <td>1.000000</td>\n      <td>1.000000</td>\n    </tr>\n  </tbody>\n</table>\n</div>"},"metadata":{}}]},{"metadata":{},"cell_type":"markdown","source":"The Pearson Correlation measures the linear dependence between two variables X and Y.\n\nThe resulting coefficient is a value between -1 and 1 inclusive, where:\n\n1: Total positive linear correlation.\n0: No linear correlation, the two variables most likely do not affect each other.\n-1: Total negative linear correlation."},{"metadata":{"trusted":true},"cell_type":"code","source":"\ndf.corr()","execution_count":4,"outputs":[{"output_type":"execute_result","execution_count":4,"data":{"text/plain":"                   symboling  normalized-losses  wheel-base    length  \\\nsymboling           1.000000           0.466264   -0.535987 -0.365404   \nnormalized-losses   0.466264           1.000000   -0.056661  0.019424   \nwheel-base         -0.535987          -0.056661    1.000000  0.876024   \nlength             -0.365404           0.019424    0.876024  1.000000   \nwidth              -0.242423           0.086802    0.814507  0.857170   \nheight             -0.550160          -0.373737    0.590742  0.492063   \ncurb-weight        -0.233118           0.099404    0.782097  0.880665   \nengine-size        -0.110581           0.112360    0.572027  0.685025   \nbore               -0.140019          -0.029862    0.493244  0.608971   \nstroke             -0.008245           0.055563    0.158502  0.124139   \ncompression-ratio  -0.182196          -0.114713    0.250313  0.159733   \nhorsepower          0.075819           0.217299    0.371147  0.579821   \npeak-rpm            0.279740           0.239543   -0.360305 -0.285970   \ncity-mpg           -0.035527          -0.225016   -0.470606 -0.665192   \nhighway-mpg         0.036233          -0.181877   -0.543304 -0.698142   \nprice              -0.082391           0.133999    0.584642  0.690628   \ncity-L/100km        0.066171           0.238567    0.476153  0.657373   \ndiesel             -0.196735          -0.101546    0.307237  0.211187   \ngas                 0.196735           0.101546   -0.307237 -0.211187   \n\n                      width    height  curb-weight  engine-size      bore  \\\nsymboling         -0.242423 -0.550160    -0.233118    -0.110581 -0.140019   \nnormalized-losses  0.086802 -0.373737     0.099404     0.112360 -0.029862   \nwheel-base         0.814507  0.590742     0.782097     0.572027  0.493244   \nlength             0.857170  0.492063     0.880665     0.685025  0.608971   \nwidth              1.000000  0.306002     0.866201     0.729436  0.544885   \nheight             0.306002  1.000000     0.307581     0.074694  0.180449   \ncurb-weight        0.866201  0.307581     1.000000     0.849072  0.644060   \nengine-size        0.729436  0.074694     0.849072     1.000000  0.572609   \nbore               0.544885  0.180449     0.644060     0.572609  1.000000   \nstroke             0.188829 -0.062704     0.167562     0.209523 -0.055390   \ncompression-ratio  0.189867  0.259737     0.156433     0.028889  0.001263   \nhorsepower         0.615077 -0.087027     0.757976     0.822676  0.566936   \npeak-rpm          -0.245800 -0.309974    -0.279361    -0.256733 -0.267392   \ncity-mpg          -0.633531 -0.049800    -0.749543    -0.650546 -0.582027   \nhighway-mpg       -0.680635 -0.104812    -0.794889    -0.679571 -0.591309   \nprice              0.751265  0.135486     0.834415     0.872335  0.543155   \ncity-L/100km       0.673363  0.003811     0.785353     0.745059  0.554610   \ndiesel             0.244356  0.281578     0.221046     0.070779  0.054458   \ngas               -0.244356 -0.281578    -0.221046    -0.070779 -0.054458   \n\n                     stroke  compression-ratio  horsepower  peak-rpm  \\\nsymboling         -0.008245          -0.182196    0.075819  0.279740   \nnormalized-losses  0.055563          -0.114713    0.217299  0.239543   \nwheel-base         0.158502           0.250313    0.371147 -0.360305   \nlength             0.124139           0.159733    0.579821 -0.285970   \nwidth              0.188829           0.189867    0.615077 -0.245800   \nheight            -0.062704           0.259737   -0.087027 -0.309974   \ncurb-weight        0.167562           0.156433    0.757976 -0.279361   \nengine-size        0.209523           0.028889    0.822676 -0.256733   \nbore              -0.055390           0.001263    0.566936 -0.267392   \nstroke             1.000000           0.187923    0.098462 -0.065713   \ncompression-ratio  0.187923           1.000000   -0.214514 -0.435780   \nhorsepower         0.098462          -0.214514    1.000000  0.107885   \npeak-rpm          -0.065713          -0.435780    0.107885  1.000000   \ncity-mpg          -0.034696           0.331425   -0.822214 -0.115413   \nhighway-mpg       -0.035201           0.268465   -0.804575 -0.058598   \nprice              0.082310           0.071107    0.809575 -0.101616   \ncity-L/100km       0.037300          -0.299372    0.889488  0.115830   \ndiesel             0.241303           0.985231   -0.169053 -0.475812   \ngas               -0.241303          -0.985231    0.169053  0.475812   \n\n                   city-mpg  highway-mpg     price  city-L/100km    diesel  \\\nsymboling         -0.035527     0.036233 -0.082391      0.066171 -0.196735   \nnormalized-losses -0.225016    -0.181877  0.133999      0.238567 -0.101546   \nwheel-base        -0.470606    -0.543304  0.584642      0.476153  0.307237   \nlength            -0.665192    -0.698142  0.690628      0.657373  0.211187   \nwidth             -0.633531    -0.680635  0.751265      0.673363  0.244356   \nheight            -0.049800    -0.104812  0.135486      0.003811  0.281578   \ncurb-weight       -0.749543    -0.794889  0.834415      0.785353  0.221046   \nengine-size       -0.650546    -0.679571  0.872335      0.745059  0.070779   \nbore              -0.582027    -0.591309  0.543155      0.554610  0.054458   \nstroke            -0.034696    -0.035201  0.082310      0.037300  0.241303   \ncompression-ratio  0.331425     0.268465  0.071107     -0.299372  0.985231   \nhorsepower        -0.822214    -0.804575  0.809575      0.889488 -0.169053   \npeak-rpm          -0.115413    -0.058598 -0.101616      0.115830 -0.475812   \ncity-mpg           1.000000     0.972044 -0.686571     -0.949713  0.265676   \nhighway-mpg        0.972044     1.000000 -0.704692     -0.930028  0.198690   \nprice             -0.686571    -0.704692  1.000000      0.789898  0.110326   \ncity-L/100km      -0.949713    -0.930028  0.789898      1.000000 -0.241282   \ndiesel             0.265676     0.198690  0.110326     -0.241282  1.000000   \ngas               -0.265676    -0.198690 -0.110326      0.241282 -1.000000   \n\n                        gas  \nsymboling          0.196735  \nnormalized-losses  0.101546  \nwheel-base        -0.307237  \nlength            -0.211187  \nwidth             -0.244356  \nheight            -0.281578  \ncurb-weight       -0.221046  \nengine-size       -0.070779  \nbore              -0.054458  \nstroke            -0.241303  \ncompression-ratio -0.985231  \nhorsepower         0.169053  \npeak-rpm           0.475812  \ncity-mpg          -0.265676  \nhighway-mpg       -0.198690  \nprice             -0.110326  \ncity-L/100km       0.241282  \ndiesel            -1.000000  \ngas                1.000000  ","text/html":"<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>symboling</th>\n      <th>normalized-losses</th>\n      <th>wheel-base</th>\n      <th>length</th>\n      <th>width</th>\n      <th>height</th>\n      <th>curb-weight</th>\n      <th>engine-size</th>\n      <th>bore</th>\n      <th>stroke</th>\n      <th>compression-ratio</th>\n      <th>horsepower</th>\n      <th>peak-rpm</th>\n      <th>city-mpg</th>\n      <th>highway-mpg</th>\n      <th>price</th>\n      <th>city-L/100km</th>\n      <th>diesel</th>\n      <th>gas</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>symboling</th>\n      <td>1.000000</td>\n      <td>0.466264</td>\n      <td>-0.535987</td>\n      <td>-0.365404</td>\n      <td>-0.242423</td>\n      <td>-0.550160</td>\n      <td>-0.233118</td>\n      <td>-0.110581</td>\n      <td>-0.140019</td>\n      <td>-0.008245</td>\n      <td>-0.182196</td>\n      <td>0.075819</td>\n      <td>0.279740</td>\n      <td>-0.035527</td>\n      <td>0.036233</td>\n      <td>-0.082391</td>\n      <td>0.066171</td>\n      <td>-0.196735</td>\n      <td>0.196735</td>\n    </tr>\n    <tr>\n      <th>normalized-losses</th>\n      <td>0.466264</td>\n      <td>1.000000</td>\n      <td>-0.056661</td>\n      <td>0.019424</td>\n      <td>0.086802</td>\n      <td>-0.373737</td>\n      <td>0.099404</td>\n      <td>0.112360</td>\n      <td>-0.029862</td>\n      <td>0.055563</td>\n      <td>-0.114713</td>\n      <td>0.217299</td>\n      <td>0.239543</td>\n      <td>-0.225016</td>\n      <td>-0.181877</td>\n      <td>0.133999</td>\n      <td>0.238567</td>\n      <td>-0.101546</td>\n      <td>0.101546</td>\n    </tr>\n    <tr>\n      <th>wheel-base</th>\n      <td>-0.535987</td>\n      <td>-0.056661</td>\n      <td>1.000000</td>\n      <td>0.876024</td>\n      <td>0.814507</td>\n      <td>0.590742</td>\n      <td>0.782097</td>\n      <td>0.572027</td>\n      <td>0.493244</td>\n      <td>0.158502</td>\n      <td>0.250313</td>\n      <td>0.371147</td>\n      <td>-0.360305</td>\n      <td>-0.470606</td>\n      <td>-0.543304</td>\n      <td>0.584642</td>\n      <td>0.476153</td>\n      <td>0.307237</td>\n      <td>-0.307237</td>\n    </tr>\n    <tr>\n      <th>length</th>\n      <td>-0.365404</td>\n      <td>0.019424</td>\n      <td>0.876024</td>\n      <td>1.000000</td>\n      <td>0.857170</td>\n      <td>0.492063</td>\n      <td>0.880665</td>\n      <td>0.685025</td>\n      <td>0.608971</td>\n      <td>0.124139</td>\n      <td>0.159733</td>\n      <td>0.579821</td>\n      <td>-0.285970</td>\n      <td>-0.665192</td>\n      <td>-0.698142</td>\n      <td>0.690628</td>\n      <td>0.657373</td>\n      <td>0.211187</td>\n      <td>-0.211187</td>\n    </tr>\n    <tr>\n      <th>width</th>\n      <td>-0.242423</td>\n      <td>0.086802</td>\n      <td>0.814507</td>\n      <td>0.857170</td>\n      <td>1.000000</td>\n      <td>0.306002</td>\n      <td>0.866201</td>\n      <td>0.729436</td>\n      <td>0.544885</td>\n      <td>0.188829</td>\n      <td>0.189867</td>\n      <td>0.615077</td>\n      <td>-0.245800</td>\n      <td>-0.633531</td>\n      <td>-0.680635</td>\n      <td>0.751265</td>\n      <td>0.673363</td>\n      <td>0.244356</td>\n      <td>-0.244356</td>\n    </tr>\n    <tr>\n      <th>height</th>\n      <td>-0.550160</td>\n      <td>-0.373737</td>\n      <td>0.590742</td>\n      <td>0.492063</td>\n      <td>0.306002</td>\n      <td>1.000000</td>\n      <td>0.307581</td>\n      <td>0.074694</td>\n      <td>0.180449</td>\n      <td>-0.062704</td>\n      <td>0.259737</td>\n      <td>-0.087027</td>\n      <td>-0.309974</td>\n      <td>-0.049800</td>\n      <td>-0.104812</td>\n      <td>0.135486</td>\n      <td>0.003811</td>\n      <td>0.281578</td>\n      <td>-0.281578</td>\n    </tr>\n    <tr>\n      <th>curb-weight</th>\n      <td>-0.233118</td>\n      <td>0.099404</td>\n      <td>0.782097</td>\n      <td>0.880665</td>\n      <td>0.866201</td>\n      <td>0.307581</td>\n      <td>1.000000</td>\n      <td>0.849072</td>\n      <td>0.644060</td>\n      <td>0.167562</td>\n      <td>0.156433</td>\n      <td>0.757976</td>\n      <td>-0.279361</td>\n      <td>-0.749543</td>\n      <td>-0.794889</td>\n      <td>0.834415</td>\n      <td>0.785353</td>\n      <td>0.221046</td>\n      <td>-0.221046</td>\n    </tr>\n    <tr>\n      <th>engine-size</th>\n      <td>-0.110581</td>\n      <td>0.112360</td>\n      <td>0.572027</td>\n      <td>0.685025</td>\n      <td>0.729436</td>\n      <td>0.074694</td>\n      <td>0.849072</td>\n      <td>1.000000</td>\n      <td>0.572609</td>\n      <td>0.209523</td>\n      <td>0.028889</td>\n      <td>0.822676</td>\n      <td>-0.256733</td>\n      <td>-0.650546</td>\n      <td>-0.679571</td>\n      <td>0.872335</td>\n      <td>0.745059</td>\n      <td>0.070779</td>\n      <td>-0.070779</td>\n    </tr>\n    <tr>\n      <th>bore</th>\n      <td>-0.140019</td>\n      <td>-0.029862</td>\n      <td>0.493244</td>\n      <td>0.608971</td>\n      <td>0.544885</td>\n      <td>0.180449</td>\n      <td>0.644060</td>\n      <td>0.572609</td>\n      <td>1.000000</td>\n      <td>-0.055390</td>\n      <td>0.001263</td>\n      <td>0.566936</td>\n      <td>-0.267392</td>\n      <td>-0.582027</td>\n      <td>-0.591309</td>\n      <td>0.543155</td>\n      <td>0.554610</td>\n      <td>0.054458</td>\n      <td>-0.054458</td>\n    </tr>\n    <tr>\n      <th>stroke</th>\n      <td>-0.008245</td>\n      <td>0.055563</td>\n      <td>0.158502</td>\n      <td>0.124139</td>\n      <td>0.188829</td>\n      <td>-0.062704</td>\n      <td>0.167562</td>\n      <td>0.209523</td>\n      <td>-0.055390</td>\n      <td>1.000000</td>\n      <td>0.187923</td>\n      <td>0.098462</td>\n      <td>-0.065713</td>\n      <td>-0.034696</td>\n      <td>-0.035201</td>\n      <td>0.082310</td>\n      <td>0.037300</td>\n      <td>0.241303</td>\n      <td>-0.241303</td>\n    </tr>\n    <tr>\n      <th>compression-ratio</th>\n      <td>-0.182196</td>\n      <td>-0.114713</td>\n      <td>0.250313</td>\n      <td>0.159733</td>\n      <td>0.189867</td>\n      <td>0.259737</td>\n      <td>0.156433</td>\n      <td>0.028889</td>\n      <td>0.001263</td>\n      <td>0.187923</td>\n      <td>1.000000</td>\n      <td>-0.214514</td>\n      <td>-0.435780</td>\n      <td>0.331425</td>\n      <td>0.268465</td>\n      <td>0.071107</td>\n      <td>-0.299372</td>\n      <td>0.985231</td>\n      <td>-0.985231</td>\n    </tr>\n    <tr>\n      <th>horsepower</th>\n      <td>0.075819</td>\n      <td>0.217299</td>\n      <td>0.371147</td>\n      <td>0.579821</td>\n      <td>0.615077</td>\n      <td>-0.087027</td>\n      <td>0.757976</td>\n      <td>0.822676</td>\n      <td>0.566936</td>\n      <td>0.098462</td>\n      <td>-0.214514</td>\n      <td>1.000000</td>\n      <td>0.107885</td>\n      <td>-0.822214</td>\n      <td>-0.804575</td>\n      <td>0.809575</td>\n      <td>0.889488</td>\n      <td>-0.169053</td>\n      <td>0.169053</td>\n    </tr>\n    <tr>\n      <th>peak-rpm</th>\n      <td>0.279740</td>\n      <td>0.239543</td>\n      <td>-0.360305</td>\n      <td>-0.285970</td>\n      <td>-0.245800</td>\n      <td>-0.309974</td>\n      <td>-0.279361</td>\n      <td>-0.256733</td>\n      <td>-0.267392</td>\n      <td>-0.065713</td>\n      <td>-0.435780</td>\n      <td>0.107885</td>\n      <td>1.000000</td>\n      <td>-0.115413</td>\n      <td>-0.058598</td>\n      <td>-0.101616</td>\n      <td>0.115830</td>\n      <td>-0.475812</td>\n      <td>0.475812</td>\n    </tr>\n    <tr>\n      <th>city-mpg</th>\n      <td>-0.035527</td>\n      <td>-0.225016</td>\n      <td>-0.470606</td>\n      <td>-0.665192</td>\n      <td>-0.633531</td>\n      <td>-0.049800</td>\n      <td>-0.749543</td>\n      <td>-0.650546</td>\n      <td>-0.582027</td>\n      <td>-0.034696</td>\n      <td>0.331425</td>\n      <td>-0.822214</td>\n      <td>-0.115413</td>\n      <td>1.000000</td>\n      <td>0.972044</td>\n      <td>-0.686571</td>\n      <td>-0.949713</td>\n      <td>0.265676</td>\n      <td>-0.265676</td>\n    </tr>\n    <tr>\n      <th>highway-mpg</th>\n      <td>0.036233</td>\n      <td>-0.181877</td>\n      <td>-0.543304</td>\n      <td>-0.698142</td>\n      <td>-0.680635</td>\n      <td>-0.104812</td>\n      <td>-0.794889</td>\n      <td>-0.679571</td>\n      <td>-0.591309</td>\n      <td>-0.035201</td>\n      <td>0.268465</td>\n      <td>-0.804575</td>\n      <td>-0.058598</td>\n      <td>0.972044</td>\n      <td>1.000000</td>\n      <td>-0.704692</td>\n      <td>-0.930028</td>\n      <td>0.198690</td>\n      <td>-0.198690</td>\n    </tr>\n    <tr>\n      <th>price</th>\n      <td>-0.082391</td>\n      <td>0.133999</td>\n      <td>0.584642</td>\n      <td>0.690628</td>\n      <td>0.751265</td>\n      <td>0.135486</td>\n      <td>0.834415</td>\n      <td>0.872335</td>\n      <td>0.543155</td>\n      <td>0.082310</td>\n      <td>0.071107</td>\n      <td>0.809575</td>\n      <td>-0.101616</td>\n      <td>-0.686571</td>\n      <td>-0.704692</td>\n      <td>1.000000</td>\n      <td>0.789898</td>\n      <td>0.110326</td>\n      <td>-0.110326</td>\n    </tr>\n    <tr>\n      <th>city-L/100km</th>\n      <td>0.066171</td>\n      <td>0.238567</td>\n      <td>0.476153</td>\n      <td>0.657373</td>\n      <td>0.673363</td>\n      <td>0.003811</td>\n      <td>0.785353</td>\n      <td>0.745059</td>\n      <td>0.554610</td>\n      <td>0.037300</td>\n      <td>-0.299372</td>\n      <td>0.889488</td>\n      <td>0.115830</td>\n      <td>-0.949713</td>\n      <td>-0.930028</td>\n      <td>0.789898</td>\n      <td>1.000000</td>\n      <td>-0.241282</td>\n      <td>0.241282</td>\n    </tr>\n    <tr>\n      <th>diesel</th>\n      <td>-0.196735</td>\n      <td>-0.101546</td>\n      <td>0.307237</td>\n      <td>0.211187</td>\n      <td>0.244356</td>\n      <td>0.281578</td>\n      <td>0.221046</td>\n      <td>0.070779</td>\n      <td>0.054458</td>\n      <td>0.241303</td>\n      <td>0.985231</td>\n      <td>-0.169053</td>\n      <td>-0.475812</td>\n      <td>0.265676</td>\n      <td>0.198690</td>\n      <td>0.110326</td>\n      <td>-0.241282</td>\n      <td>1.000000</td>\n      <td>-1.000000</td>\n    </tr>\n    <tr>\n      <th>gas</th>\n      <td>0.196735</td>\n      <td>0.101546</td>\n      <td>-0.307237</td>\n      <td>-0.211187</td>\n      <td>-0.244356</td>\n      <td>-0.281578</td>\n      <td>-0.221046</td>\n      <td>-0.070779</td>\n      <td>-0.054458</td>\n      <td>-0.241303</td>\n      <td>-0.985231</td>\n      <td>0.169053</td>\n      <td>0.475812</td>\n      <td>-0.265676</td>\n      <td>-0.198690</td>\n      <td>-0.110326</td>\n      <td>0.241282</td>\n      <td>-1.000000</td>\n      <td>1.000000</td>\n    </tr>\n  </tbody>\n</table>\n</div>"},"metadata":{}}]},{"metadata":{},"cell_type":"markdown","source":"sometimes we would like to know the significant of the correlation estimate."},{"metadata":{},"cell_type":"markdown","source":"<b> P-value:</b>"},{"metadata":{},"cell_type":"markdown","source":"What is this P-value? The P-value is the probability value that the correlation between these two variables is statistically significant. Normally, we choose a significance level of 0.05, which means that we are 95% confident that the correlation between the variables is significant.\n\nBy convention, when the\n\np-value is $&lt;$ 0.001: we say there is strong evidence that the correlation is significant.\nthe p-value is $&lt;$ 0.05: there is moderate evidence that the correlation is significant.\nthe p-value is $&lt;$ 0.1: there is weak evidence that the correlation is significant.\nthe p-value is $&gt;$ 0.1: there is no evidence that the correlation is significant.\nWe can obtain this information using \"stats\" module in the \"scipy\" library."},{"metadata":{"trusted":true},"cell_type":"code","source":"from scipy import stats","execution_count":7,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<b> Wheel-base vs Price</b>"},{"metadata":{},"cell_type":"markdown","source":"Let's calculate the Pearson Correlation Coefficient and P-value of 'wheel-base' and 'price'."},{"metadata":{"trusted":true},"cell_type":"code","source":"pearson_coef, p_value = stats.pearsonr(df['wheel-base'], df['price'])\nprint(\"The Pearson Correlation Coefficient is\", pearson_coef, \" with a P-value of P =\", p_value)","execution_count":8,"outputs":[{"output_type":"stream","text":"The Pearson Correlation Coefficient is 0.584641822265508  with a P-value of P = 8.076488270733218e-20\n","name":"stdout"}]},{"metadata":{},"cell_type":"markdown","source":"<b> Conclusion:</b>"},{"metadata":{},"cell_type":"markdown","source":"Since the p-value is < 0.001, the correlation between wheel-base and price is statistically significant, although the linear relationship isn't extremely strong (~0.585)"},{"metadata":{},"cell_type":"markdown","source":"<b> Horsepower vs Price</b>"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Write your codes here!","execution_count":9,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<b> Conclusion: </b>"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Write your conclusion here","execution_count":10,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<b> Length vs Price</b>"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Write your codes here!","execution_count":11,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<b> Conclusion: </b>"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Write your conclusion here","execution_count":14,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<b> Width vs Price</b>"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Write your codes here!","execution_count":15,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<b> Conclusion: </b>"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Write your conclusion here","execution_count":16,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"name":"python3","display_name":"Python 3","language":"python"},"language_info":{"name":"python","version":"3.6.13","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":5}