forked from mca91/EconometricsWithR
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy path2-1-random-variables-and-probability-distributions.html
1088 lines (1022 loc) · 108 KB
/
2-1-random-variables-and-probability-distributions.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<!DOCTYPE html>
<html >
<head>
<meta charset="UTF-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<title>2.1 Random Variables and Probability Distributions | Introduction to Econometrics with R</title>
<meta name="description" content="Beginners with little background in statistics and econometrics often have a hard time understanding the benefits of having programming skills for learning and applying Econometrics. ‘Introduction to Econometrics with R’ is an interactive companion to the well-received textbook ‘Introduction to Econometrics’ by James H. Stock and Mark W. Watson (2015). It gives a gentle introduction to the essentials of R programming and guides students in implementing the empirical applications presented throughout the textbook using the newly aquired skills. This is supported by interactive programming exercises generated with DataCamp Light and integration of interactive visualizations of central concepts which are based on the flexible JavaScript library D3.js.">
<meta name="generator" content="bookdown and GitBook 2.6.7">
<meta property="og:title" content="2.1 Random Variables and Probability Distributions | Introduction to Econometrics with R" />
<meta property="og:type" content="book" />
<meta property="og:url" content="https://www.econometrics-with-r.org/" />
<meta property="og:image" content="https://www.econometrics-with-r.org/images/cover.png" />
<meta property="og:description" content="Beginners with little background in statistics and econometrics often have a hard time understanding the benefits of having programming skills for learning and applying Econometrics. ‘Introduction to Econometrics with R’ is an interactive companion to the well-received textbook ‘Introduction to Econometrics’ by James H. Stock and Mark W. Watson (2015). It gives a gentle introduction to the essentials of R programming and guides students in implementing the empirical applications presented throughout the textbook using the newly aquired skills. This is supported by interactive programming exercises generated with DataCamp Light and integration of interactive visualizations of central concepts which are based on the flexible JavaScript library D3.js." />
<meta name="github-repo" content="mca91/EconometricsWithR" />
<meta name="twitter:card" content="summary" />
<meta name="twitter:title" content="2.1 Random Variables and Probability Distributions | Introduction to Econometrics with R" />
<meta name="twitter:description" content="Beginners with little background in statistics and econometrics often have a hard time understanding the benefits of having programming skills for learning and applying Econometrics. ‘Introduction to Econometrics with R’ is an interactive companion to the well-received textbook ‘Introduction to Econometrics’ by James H. Stock and Mark W. Watson (2015). It gives a gentle introduction to the essentials of R programming and guides students in implementing the empirical applications presented throughout the textbook using the newly aquired skills. This is supported by interactive programming exercises generated with DataCamp Light and integration of interactive visualizations of central concepts which are based on the flexible JavaScript library D3.js." />
<meta name="twitter:image" content="https://www.econometrics-with-r.org/images/cover.png" />
<meta name="author" content="Christoph Hanck, Martin Arnold, Alexander Gerber and Martin Schmelzer">
<meta name="date" content="2019-03-12">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="apple-mobile-web-app-capable" content="yes">
<meta name="apple-mobile-web-app-status-bar-style" content="black">
<link rel="prev" href="2-pt.html">
<link rel="next" href="2-2-RSATDOSA.html">
<script src="libs/jquery-2.2.3/jquery.min.js"></script>
<link href="libs/gitbook-2.6.7/css/style.css" rel="stylesheet" />
<link href="libs/gitbook-2.6.7/css/plugin-table.css" rel="stylesheet" />
<link href="libs/gitbook-2.6.7/css/plugin-bookdown.css" rel="stylesheet" />
<link href="libs/gitbook-2.6.7/css/plugin-highlight.css" rel="stylesheet" />
<link href="libs/gitbook-2.6.7/css/plugin-search.css" rel="stylesheet" />
<link href="libs/gitbook-2.6.7/css/plugin-fontsettings.css" rel="stylesheet" />
<script src="libs/htmlwidgets-1.3/htmlwidgets.js"></script>
<script src="libs/plotly-binding-4.8.0/plotly.js"></script>
<script src="libs/typedarray-0.1/typedarray.min.js"></script>
<link href="libs/crosstalk-1.0.0/css/crosstalk.css" rel="stylesheet" />
<script src="libs/crosstalk-1.0.0/js/crosstalk.min.js"></script>
<link href="libs/plotly-htmlwidgets-css-1.39.2/plotly-htmlwidgets.css" rel="stylesheet" />
<script src="libs/plotly-main-1.39.2/plotly-latest.min.js"></script>
<!-- font families -->
<link href="https://fonts.googleapis.com/css?family=PT+Sans|Pacifico|Source+Sans+Pro" rel="stylesheet">
<script src="js/hideOutput.js"></script>
<!-- Mathjax -->
<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/config/default.js"></script>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
extensions: ["tex2jax.js", "TeX/AMSmath.js"],
tex2jax: {inlineMath: [['$','$'], ['\\(','\\)']]},
jax: ["input/TeX","output/CommonHTML"]
});
MathJax.Hub.processSectionDelay = 0;
</script>
<!-- Global site tag (gtag.js) - Google Analytics -->
<script async src="https://www.googletagmanager.com/gtag/js?id=UA-110299877-1"></script>
<script>
window.dataLayer = window.dataLayer || [];
function gtag(){dataLayer.push(arguments);}
gtag('js', new Date());
gtag('config', 'UA-110299877-1');
</script>
<!-- open review block -->
<script async defer src="https://hypothes.is/embed.js"></script>
<style type="text/css">
a.sourceLine { display: inline-block; line-height: 1.25; }
a.sourceLine { pointer-events: none; color: inherit; text-decoration: inherit; }
a.sourceLine:empty { height: 1.2em; }
.sourceCode { overflow: visible; }
code.sourceCode { white-space: pre; position: relative; }
div.sourceCode { margin: 1em 0; }
pre.sourceCode { margin: 0; }
@media screen {
div.sourceCode { overflow: auto; }
}
@media print {
code.sourceCode { white-space: pre-wrap; }
a.sourceLine { text-indent: -1em; padding-left: 1em; }
}
pre.numberSource a.sourceLine
{ position: relative; left: -4em; }
pre.numberSource a.sourceLine::before
{ content: attr(data-line-number);
position: relative; left: -1em; text-align: right; vertical-align: baseline;
border: none; pointer-events: all; display: inline-block;
-webkit-touch-callout: none; -webkit-user-select: none;
-khtml-user-select: none; -moz-user-select: none;
-ms-user-select: none; user-select: none;
padding: 0 4px; width: 4em;
color: #aaaaaa;
}
pre.numberSource { margin-left: 3em; border-left: 1px solid #aaaaaa; padding-left: 4px; }
div.sourceCode
{ background-color: #f8f8f8; }
@media screen {
a.sourceLine::before { text-decoration: underline; }
}
code span.al { color: #ef2929; } /* Alert */
code span.an { color: #8f5902; font-weight: bold; font-style: italic; } /* Annotation */
code span.at { color: #c4a000; } /* Attribute */
code span.bn { color: #0000cf; } /* BaseN */
code span.cf { color: #204a87; font-weight: bold; } /* ControlFlow */
code span.ch { color: #4e9a06; } /* Char */
code span.cn { color: #000000; } /* Constant */
code span.co { color: #8f5902; font-style: italic; } /* Comment */
code span.cv { color: #8f5902; font-weight: bold; font-style: italic; } /* CommentVar */
code span.do { color: #8f5902; font-weight: bold; font-style: italic; } /* Documentation */
code span.dt { color: #204a87; } /* DataType */
code span.dv { color: #0000cf; } /* DecVal */
code span.er { color: #a40000; font-weight: bold; } /* Error */
code span.ex { } /* Extension */
code span.fl { color: #0000cf; } /* Float */
code span.fu { color: #000000; } /* Function */
code span.im { } /* Import */
code span.in { color: #8f5902; font-weight: bold; font-style: italic; } /* Information */
code span.kw { color: #204a87; font-weight: bold; } /* Keyword */
code span.op { color: #ce5c00; font-weight: bold; } /* Operator */
code span.ot { color: #8f5902; } /* Other */
code span.pp { color: #8f5902; font-style: italic; } /* Preprocessor */
code span.sc { color: #000000; } /* SpecialChar */
code span.ss { color: #4e9a06; } /* SpecialString */
code span.st { color: #4e9a06; } /* String */
code span.va { color: #000000; } /* Variable */
code span.vs { color: #4e9a06; } /* VerbatimString */
code span.wa { color: #8f5902; font-weight: bold; font-style: italic; } /* Warning */
</style>
<link rel="stylesheet" href="style.css" type="text/css" />
<link rel="stylesheet" href="toc.css" type="text/css" />
</head>
<body>
<div class="book without-animation with-summary font-size-2 font-family-1" data-basepath=".">
<div class="book-summary">
<nav role="navigation">
<ul class="summary">
<li><center><img src="images/logo.png" alt="logo" width="50%" height="50%"style="margin: 15px 0 0 0"></center></li>
<li class="divider"></li>
<li class="chapter" data-level="" data-path="index.html"><a href="index.html"><i class="fa fa-check"></i>Preface</a></li>
<li class="chapter" data-level="1" data-path="1-introduction.html"><a href="1-introduction.html"><i class="fa fa-check"></i><b>1</b> Introduction</a><ul>
<li class="chapter" data-level="1.1" data-path="1-1-a-very-short-introduction-to-r-and-rstudio.html"><a href="1-1-a-very-short-introduction-to-r-and-rstudio.html"><i class="fa fa-check"></i><b>1.1</b> A Very Short Introduction to <tt>R</tt> and <em>RStudio</em></a></li>
</ul></li>
<li class="chapter" data-level="2" data-path="2-pt.html"><a href="2-pt.html"><i class="fa fa-check"></i><b>2</b> Probability Theory</a><ul>
<li class="chapter" data-level="2.1" data-path="2-1-random-variables-and-probability-distributions.html"><a href="2-1-random-variables-and-probability-distributions.html"><i class="fa fa-check"></i><b>2.1</b> Random Variables and Probability Distributions</a><ul>
<li class="chapter" data-level="" data-path="2-1-random-variables-and-probability-distributions.html"><a href="2-1-random-variables-and-probability-distributions.html#probability-distributions-of-discrete-random-variables"><i class="fa fa-check"></i>Probability Distributions of Discrete Random Variables</a></li>
<li class="chapter" data-level="" data-path="2-1-random-variables-and-probability-distributions.html"><a href="2-1-random-variables-and-probability-distributions.html#bernoulli-trials"><i class="fa fa-check"></i>Bernoulli Trials</a></li>
<li class="chapter" data-level="" data-path="2-1-random-variables-and-probability-distributions.html"><a href="2-1-random-variables-and-probability-distributions.html#expected-value-mean-and-variance"><i class="fa fa-check"></i>Expected Value, Mean and Variance</a></li>
<li class="chapter" data-level="" data-path="2-1-random-variables-and-probability-distributions.html"><a href="2-1-random-variables-and-probability-distributions.html#probability-distributions-of-continuous-random-variables"><i class="fa fa-check"></i>Probability Distributions of Continuous Random Variables</a></li>
<li class="chapter" data-level="" data-path="2-1-random-variables-and-probability-distributions.html"><a href="2-1-random-variables-and-probability-distributions.html#the-normal-distribution"><i class="fa fa-check"></i>The Normal Distribution</a></li>
<li class="chapter" data-level="" data-path="2-1-random-variables-and-probability-distributions.html"><a href="2-1-random-variables-and-probability-distributions.html#the-chi-squared-distribution"><i class="fa fa-check"></i>The Chi-Squared Distribution</a></li>
<li class="chapter" data-level="" data-path="2-1-random-variables-and-probability-distributions.html"><a href="2-1-random-variables-and-probability-distributions.html#thetdist"><i class="fa fa-check"></i>The Student t Distribution</a></li>
<li class="chapter" data-level="" data-path="2-1-random-variables-and-probability-distributions.html"><a href="2-1-random-variables-and-probability-distributions.html#the-f-distribution"><i class="fa fa-check"></i>The F Distribution</a></li>
</ul></li>
<li class="chapter" data-level="2.2" data-path="2-2-RSATDOSA.html"><a href="2-2-RSATDOSA.html"><i class="fa fa-check"></i><b>2.2</b> Random Sampling and the Distribution of Sample Averages</a><ul>
<li class="chapter" data-level="" data-path="2-2-RSATDOSA.html"><a href="2-2-RSATDOSA.html#mean-and-variance-of-the-sample-mean"><i class="fa fa-check"></i>Mean and Variance of the Sample Mean</a></li>
<li class="chapter" data-level="" data-path="2-2-RSATDOSA.html"><a href="2-2-RSATDOSA.html#large-sample-approximations-to-sampling-distributions"><i class="fa fa-check"></i>Large Sample Approximations to Sampling Distributions</a></li>
</ul></li>
<li class="chapter" data-level="2.3" data-path="2-3-exercises.html"><a href="2-3-exercises.html"><i class="fa fa-check"></i><b>2.3</b> Exercises</a></li>
</ul></li>
<li class="chapter" data-level="3" data-path="3-arosur.html"><a href="3-arosur.html"><i class="fa fa-check"></i><b>3</b> A Review of Statistics using R</a><ul>
<li class="chapter" data-level="3.1" data-path="3-1-estimation-of-the-population-mean.html"><a href="3-1-estimation-of-the-population-mean.html"><i class="fa fa-check"></i><b>3.1</b> Estimation of the Population Mean</a></li>
<li class="chapter" data-level="3.2" data-path="3-2-potsm.html"><a href="3-2-potsm.html"><i class="fa fa-check"></i><b>3.2</b> Properties of the Sample Mean</a></li>
<li class="chapter" data-level="3.3" data-path="3-3-hypothesis-tests-concerning-the-population-mean.html"><a href="3-3-hypothesis-tests-concerning-the-population-mean.html"><i class="fa fa-check"></i><b>3.3</b> Hypothesis Tests Concerning the Population Mean</a><ul>
<li class="chapter" data-level="" data-path="3-3-hypothesis-tests-concerning-the-population-mean.html"><a href="3-3-hypothesis-tests-concerning-the-population-mean.html#the-p-value"><i class="fa fa-check"></i>The p-Value</a></li>
<li class="chapter" data-level="" data-path="3-3-hypothesis-tests-concerning-the-population-mean.html"><a href="3-3-hypothesis-tests-concerning-the-population-mean.html#calculating-the-p-value-when-the-standard-deviation-is-known"><i class="fa fa-check"></i>Calculating the p-Value when the Standard Deviation is Known</a></li>
<li class="chapter" data-level="" data-path="3-3-hypothesis-tests-concerning-the-population-mean.html"><a href="3-3-hypothesis-tests-concerning-the-population-mean.html#SVSSDASE"><i class="fa fa-check"></i>Sample Variance, Sample Standard Deviation and Standard Error</a></li>
<li class="chapter" data-level="" data-path="3-3-hypothesis-tests-concerning-the-population-mean.html"><a href="3-3-hypothesis-tests-concerning-the-population-mean.html#calculating-the-p-value-when-the-standard-deviation-is-unknown"><i class="fa fa-check"></i>Calculating the p-value When the Standard Deviation is Unknown</a></li>
<li class="chapter" data-level="" data-path="3-3-hypothesis-tests-concerning-the-population-mean.html"><a href="3-3-hypothesis-tests-concerning-the-population-mean.html#the-t-statistic"><i class="fa fa-check"></i>The t-statistic</a></li>
<li class="chapter" data-level="" data-path="3-3-hypothesis-tests-concerning-the-population-mean.html"><a href="3-3-hypothesis-tests-concerning-the-population-mean.html#hypothesis-testing-with-a-prespecified-significance-level"><i class="fa fa-check"></i>Hypothesis Testing with a Prespecified Significance Level</a></li>
<li class="chapter" data-level="" data-path="3-3-hypothesis-tests-concerning-the-population-mean.html"><a href="3-3-hypothesis-tests-concerning-the-population-mean.html#one-sided-alternatives"><i class="fa fa-check"></i>One-sided Alternatives</a></li>
</ul></li>
<li class="chapter" data-level="3.4" data-path="3-4-confidence-intervals-for-the-population-mean.html"><a href="3-4-confidence-intervals-for-the-population-mean.html"><i class="fa fa-check"></i><b>3.4</b> Confidence Intervals for the Population Mean</a></li>
<li class="chapter" data-level="3.5" data-path="3-5-cmfdp.html"><a href="3-5-cmfdp.html"><i class="fa fa-check"></i><b>3.5</b> Comparing Means from Different Populations</a></li>
<li class="chapter" data-level="3.6" data-path="3-6-aattggoe.html"><a href="3-6-aattggoe.html"><i class="fa fa-check"></i><b>3.6</b> An Application to the Gender Gap of Earnings</a></li>
<li class="chapter" data-level="3.7" data-path="3-7-scatterplots-sample-covariance-and-sample-correlation.html"><a href="3-7-scatterplots-sample-covariance-and-sample-correlation.html"><i class="fa fa-check"></i><b>3.7</b> Scatterplots, Sample Covariance and Sample Correlation</a></li>
<li class="chapter" data-level="3.8" data-path="3-8-exercises-1.html"><a href="3-8-exercises-1.html"><i class="fa fa-check"></i><b>3.8</b> Exercises</a></li>
</ul></li>
<li class="chapter" data-level="4" data-path="4-lrwor.html"><a href="4-lrwor.html"><i class="fa fa-check"></i><b>4</b> Linear Regression with One Regressor</a><ul>
<li class="chapter" data-level="4.1" data-path="4-1-simple-linear-regression.html"><a href="4-1-simple-linear-regression.html"><i class="fa fa-check"></i><b>4.1</b> Simple Linear Regression</a></li>
<li class="chapter" data-level="4.2" data-path="4-2-estimating-the-coefficients-of-the-linear-regression-model.html"><a href="4-2-estimating-the-coefficients-of-the-linear-regression-model.html"><i class="fa fa-check"></i><b>4.2</b> Estimating the Coefficients of the Linear Regression Model</a><ul>
<li class="chapter" data-level="" data-path="4-2-estimating-the-coefficients-of-the-linear-regression-model.html"><a href="4-2-estimating-the-coefficients-of-the-linear-regression-model.html#the-ordinary-least-squares-estimator"><i class="fa fa-check"></i>The Ordinary Least Squares Estimator</a></li>
</ul></li>
<li class="chapter" data-level="4.3" data-path="4-3-measures-of-fit.html"><a href="4-3-measures-of-fit.html"><i class="fa fa-check"></i><b>4.3</b> Measures of Fit</a><ul>
<li class="chapter" data-level="" data-path="4-3-measures-of-fit.html"><a href="4-3-measures-of-fit.html#the-coefficient-of-determination"><i class="fa fa-check"></i>The Coefficient of Determination</a></li>
<li class="chapter" data-level="" data-path="4-3-measures-of-fit.html"><a href="4-3-measures-of-fit.html#the-standard-error-of-the-regression"><i class="fa fa-check"></i>The Standard Error of the Regression</a></li>
<li class="chapter" data-level="" data-path="4-3-measures-of-fit.html"><a href="4-3-measures-of-fit.html#application-to-the-test-score-data"><i class="fa fa-check"></i>Application to the Test Score Data</a></li>
</ul></li>
<li class="chapter" data-level="4.4" data-path="4-4-tlsa.html"><a href="4-4-tlsa.html"><i class="fa fa-check"></i><b>4.4</b> The Least Squares Assumptions</a><ul>
<li class="chapter" data-level="" data-path="4-4-tlsa.html"><a href="4-4-tlsa.html#assumption-1-the-error-term-has-conditional-mean-of-zero"><i class="fa fa-check"></i>Assumption 1: The Error Term has Conditional Mean of Zero</a></li>
<li class="chapter" data-level="" data-path="4-4-tlsa.html"><a href="4-4-tlsa.html#assumption-2-independently-and-identically-distributed-data"><i class="fa fa-check"></i>Assumption 2: Independently and Identically Distributed Data</a></li>
<li class="chapter" data-level="" data-path="4-4-tlsa.html"><a href="4-4-tlsa.html#assumption-3-large-outliers-are-unlikely"><i class="fa fa-check"></i>Assumption 3: Large Outliers are Unlikely</a></li>
</ul></li>
<li class="chapter" data-level="4.5" data-path="4-5-tsdotoe.html"><a href="4-5-tsdotoe.html"><i class="fa fa-check"></i><b>4.5</b> The Sampling Distribution of the OLS Estimator</a><ul>
<li class="chapter" data-level="" data-path="4-5-tsdotoe.html"><a href="4-5-tsdotoe.html#simulation-study-1"><i class="fa fa-check"></i>Simulation Study 1</a></li>
<li class="chapter" data-level="" data-path="4-5-tsdotoe.html"><a href="4-5-tsdotoe.html#simulation-study-2"><i class="fa fa-check"></i>Simulation Study 2</a></li>
<li class="chapter" data-level="" data-path="4-5-tsdotoe.html"><a href="4-5-tsdotoe.html#simulation-study-3"><i class="fa fa-check"></i>Simulation Study 3</a></li>
</ul></li>
<li class="chapter" data-level="4.6" data-path="4-6-exercises-2.html"><a href="4-6-exercises-2.html"><i class="fa fa-check"></i><b>4.6</b> Exercises</a></li>
</ul></li>
<li class="chapter" data-level="5" data-path="5-htaciitslrm.html"><a href="5-htaciitslrm.html"><i class="fa fa-check"></i><b>5</b> Hypothesis Tests and Confidence Intervals in the Simple Linear Regression Model</a><ul>
<li class="chapter" data-level="5.1" data-path="5-1-testing-two-sided-hypotheses-concerning-the-slope-coefficient.html"><a href="5-1-testing-two-sided-hypotheses-concerning-the-slope-coefficient.html"><i class="fa fa-check"></i><b>5.1</b> Testing Two-Sided Hypotheses Concerning the Slope Coefficient</a></li>
<li class="chapter" data-level="5.2" data-path="5-2-cifrc.html"><a href="5-2-cifrc.html"><i class="fa fa-check"></i><b>5.2</b> Confidence Intervals for Regression Coefficients</a><ul>
<li class="chapter" data-level="" data-path="5-2-cifrc.html"><a href="5-2-cifrc.html#simulation-study-confidence-intervals"><i class="fa fa-check"></i>Simulation Study: Confidence Intervals</a></li>
</ul></li>
<li class="chapter" data-level="5.3" data-path="5-3-rwxiabv.html"><a href="5-3-rwxiabv.html"><i class="fa fa-check"></i><b>5.3</b> Regression when X is a Binary Variable</a></li>
<li class="chapter" data-level="5.4" data-path="5-4-hah.html"><a href="5-4-hah.html"><i class="fa fa-check"></i><b>5.4</b> Heteroskedasticity and Homoskedasticity</a><ul>
<li class="chapter" data-level="" data-path="5-4-hah.html"><a href="5-4-hah.html#a-real-world-example-for-heteroskedasticity"><i class="fa fa-check"></i>A Real-World Example for Heteroskedasticity</a></li>
<li class="chapter" data-level="" data-path="5-4-hah.html"><a href="5-4-hah.html#should-we-care-about-heteroskedasticity"><i class="fa fa-check"></i>Should We Care About Heteroskedasticity?</a></li>
<li class="chapter" data-level="" data-path="5-4-hah.html"><a href="5-4-hah.html#computation-of-heteroskedasticity-robust-standard-errors"><i class="fa fa-check"></i>Computation of Heteroskedasticity-Robust Standard Errors</a></li>
</ul></li>
<li class="chapter" data-level="5.5" data-path="5-5-the-gauss-markov-theorem.html"><a href="5-5-the-gauss-markov-theorem.html"><i class="fa fa-check"></i><b>5.5</b> The Gauss-Markov Theorem</a><ul>
<li class="chapter" data-level="" data-path="5-5-the-gauss-markov-theorem.html"><a href="5-5-the-gauss-markov-theorem.html#simulation-study-blue-estimator"><i class="fa fa-check"></i>Simulation Study: BLUE Estimator</a></li>
</ul></li>
<li class="chapter" data-level="5.6" data-path="5-6-using-the-t-statistic-in-regression-when-the-sample-size-is-small.html"><a href="5-6-using-the-t-statistic-in-regression-when-the-sample-size-is-small.html"><i class="fa fa-check"></i><b>5.6</b> Using the t-Statistic in Regression When the Sample Size Is Small</a></li>
<li class="chapter" data-level="5.7" data-path="5-7-exercises-3.html"><a href="5-7-exercises-3.html"><i class="fa fa-check"></i><b>5.7</b> Exercises</a></li>
</ul></li>
<li class="chapter" data-level="6" data-path="6-rmwmr.html"><a href="6-rmwmr.html"><i class="fa fa-check"></i><b>6</b> Regression Models with Multiple Regressors</a><ul>
<li class="chapter" data-level="6.1" data-path="6-1-omitted-variable-bias.html"><a href="6-1-omitted-variable-bias.html"><i class="fa fa-check"></i><b>6.1</b> Omitted Variable Bias</a></li>
<li class="chapter" data-level="6.2" data-path="6-2-tmrm.html"><a href="6-2-tmrm.html"><i class="fa fa-check"></i><b>6.2</b> The Multiple Regression Model</a></li>
<li class="chapter" data-level="6.3" data-path="6-3-mofimr.html"><a href="6-3-mofimr.html"><i class="fa fa-check"></i><b>6.3</b> Measures of Fit in Multiple Regression</a></li>
<li class="chapter" data-level="6.4" data-path="6-4-ols-assumptions-in-multiple-regression.html"><a href="6-4-ols-assumptions-in-multiple-regression.html"><i class="fa fa-check"></i><b>6.4</b> OLS Assumptions in Multiple Regression</a><ul>
<li class="chapter" data-level="" data-path="6-4-ols-assumptions-in-multiple-regression.html"><a href="6-4-ols-assumptions-in-multiple-regression.html#multicollinearity"><i class="fa fa-check"></i>Multicollinearity</a></li>
<li class="chapter" data-level="" data-path="6-4-ols-assumptions-in-multiple-regression.html"><a href="6-4-ols-assumptions-in-multiple-regression.html#simulation-study-imperfect-multicollinearity"><i class="fa fa-check"></i>Simulation Study: Imperfect Multicollinearity</a></li>
</ul></li>
<li class="chapter" data-level="6.5" data-path="6-5-the-distribution-of-the-ols-estimators-in-multiple-regression.html"><a href="6-5-the-distribution-of-the-ols-estimators-in-multiple-regression.html"><i class="fa fa-check"></i><b>6.5</b> The Distribution of the OLS Estimators in Multiple Regression</a></li>
<li class="chapter" data-level="6.6" data-path="6-6-exercises-4.html"><a href="6-6-exercises-4.html"><i class="fa fa-check"></i><b>6.6</b> Exercises</a></li>
</ul></li>
<li class="chapter" data-level="7" data-path="7-htaciimr.html"><a href="7-htaciimr.html"><i class="fa fa-check"></i><b>7</b> Hypothesis Tests and Confidence Intervals in Multiple Regression</a><ul>
<li class="chapter" data-level="7.1" data-path="7-1-hypothesis-tests-and-confidence-intervals-for-a-single-coefficient.html"><a href="7-1-hypothesis-tests-and-confidence-intervals-for-a-single-coefficient.html"><i class="fa fa-check"></i><b>7.1</b> Hypothesis Tests and Confidence Intervals for a Single Coefficient</a></li>
<li class="chapter" data-level="7.2" data-path="7-2-an-application-to-test-scores-and-the-student-teacher-ratio.html"><a href="7-2-an-application-to-test-scores-and-the-student-teacher-ratio.html"><i class="fa fa-check"></i><b>7.2</b> An Application to Test Scores and the Student-Teacher Ratio</a><ul>
<li class="chapter" data-level="" data-path="7-2-an-application-to-test-scores-and-the-student-teacher-ratio.html"><a href="7-2-an-application-to-test-scores-and-the-student-teacher-ratio.html#another-augmentation-of-the-model"><i class="fa fa-check"></i>Another Augmentation of the Model</a></li>
</ul></li>
<li class="chapter" data-level="7.3" data-path="7-3-joint-hypothesis-testing-using-the-f-statistic.html"><a href="7-3-joint-hypothesis-testing-using-the-f-statistic.html"><i class="fa fa-check"></i><b>7.3</b> Joint Hypothesis Testing Using the F-Statistic</a></li>
<li class="chapter" data-level="7.4" data-path="7-4-confidence-sets-for-multiple-coefficients.html"><a href="7-4-confidence-sets-for-multiple-coefficients.html"><i class="fa fa-check"></i><b>7.4</b> Confidence Sets for Multiple Coefficients</a></li>
<li class="chapter" data-level="7.5" data-path="7-5-model-specification-for-multiple-regression.html"><a href="7-5-model-specification-for-multiple-regression.html"><i class="fa fa-check"></i><b>7.5</b> Model Specification for Multiple Regression</a><ul>
<li class="chapter" data-level="" data-path="7-5-model-specification-for-multiple-regression.html"><a href="7-5-model-specification-for-multiple-regression.html#model-specification-in-theory-and-in-practice"><i class="fa fa-check"></i>Model Specification in Theory and in Practice</a></li>
</ul></li>
<li class="chapter" data-level="7.6" data-path="7-6-analysis-of-the-test-score-data-set.html"><a href="7-6-analysis-of-the-test-score-data-set.html"><i class="fa fa-check"></i><b>7.6</b> Analysis of the Test Score Data Set</a></li>
<li class="chapter" data-level="7.7" data-path="7-7-exercises-5.html"><a href="7-7-exercises-5.html"><i class="fa fa-check"></i><b>7.7</b> Exercises</a></li>
</ul></li>
<li class="chapter" data-level="8" data-path="8-nrf.html"><a href="8-nrf.html"><i class="fa fa-check"></i><b>8</b> Nonlinear Regression Functions</a><ul>
<li class="chapter" data-level="8.1" data-path="8-1-a-general-strategy-for-modelling-nonlinear-regression-functions.html"><a href="8-1-a-general-strategy-for-modelling-nonlinear-regression-functions.html"><i class="fa fa-check"></i><b>8.1</b> A General Strategy for Modelling Nonlinear Regression Functions</a></li>
<li class="chapter" data-level="8.2" data-path="8-2-nfoasiv.html"><a href="8-2-nfoasiv.html"><i class="fa fa-check"></i><b>8.2</b> Nonlinear Functions of a Single Independent Variable</a><ul>
<li class="chapter" data-level="" data-path="8-2-nfoasiv.html"><a href="8-2-nfoasiv.html#polynomials"><i class="fa fa-check"></i>Polynomials</a></li>
<li class="chapter" data-level="" data-path="8-2-nfoasiv.html"><a href="8-2-nfoasiv.html#logarithms"><i class="fa fa-check"></i>Logarithms</a></li>
</ul></li>
<li class="chapter" data-level="8.3" data-path="8-3-interactions-between-independent-variables.html"><a href="8-3-interactions-between-independent-variables.html"><i class="fa fa-check"></i><b>8.3</b> Interactions Between Independent Variables</a></li>
<li class="chapter" data-level="8.4" data-path="8-4-nonlinear-effects-on-test-scores-of-the-student-teacher-ratio.html"><a href="8-4-nonlinear-effects-on-test-scores-of-the-student-teacher-ratio.html"><i class="fa fa-check"></i><b>8.4</b> Nonlinear Effects on Test Scores of the Student-Teacher Ratio</a></li>
<li class="chapter" data-level="8.5" data-path="8-5-exercises-6.html"><a href="8-5-exercises-6.html"><i class="fa fa-check"></i><b>8.5</b> Exercises</a></li>
</ul></li>
<li class="chapter" data-level="9" data-path="9-asbomr.html"><a href="9-asbomr.html"><i class="fa fa-check"></i><b>9</b> Assessing Studies Based on Multiple Regression</a><ul>
<li class="chapter" data-level="9.1" data-path="9-1-internal-and-external-validity.html"><a href="9-1-internal-and-external-validity.html"><i class="fa fa-check"></i><b>9.1</b> Internal and External Validity</a></li>
<li class="chapter" data-level="9.2" data-path="9-2-ttivomra.html"><a href="9-2-ttivomra.html"><i class="fa fa-check"></i><b>9.2</b> Threats to Internal Validity of Multiple Regression Analysis</a></li>
<li class="chapter" data-level="9.3" data-path="9-3-internal-and-external-validity-when-the-regression-is-used-for-forecasting.html"><a href="9-3-internal-and-external-validity-when-the-regression-is-used-for-forecasting.html"><i class="fa fa-check"></i><b>9.3</b> Internal and External Validity when the Regression is Used for Forecasting</a></li>
<li class="chapter" data-level="9.4" data-path="9-4-etsacs.html"><a href="9-4-etsacs.html"><i class="fa fa-check"></i><b>9.4</b> Example: Test Scores and Class Size</a></li>
<li class="chapter" data-level="9.5" data-path="9-5-exercises-7.html"><a href="9-5-exercises-7.html"><i class="fa fa-check"></i><b>9.5</b> Exercises</a></li>
</ul></li>
<li class="chapter" data-level="10" data-path="10-rwpd.html"><a href="10-rwpd.html"><i class="fa fa-check"></i><b>10</b> Regression with Panel Data</a><ul>
<li class="chapter" data-level="10.1" data-path="10-1-panel-data.html"><a href="10-1-panel-data.html"><i class="fa fa-check"></i><b>10.1</b> Panel Data</a></li>
<li class="chapter" data-level="10.2" data-path="10-2-PDWTTP.html"><a href="10-2-PDWTTP.html"><i class="fa fa-check"></i><b>10.2</b> Panel Data with Two Time Periods: “Before and After” Comparisons</a></li>
<li class="chapter" data-level="10.3" data-path="10-3-fixed-effects-regression.html"><a href="10-3-fixed-effects-regression.html"><i class="fa fa-check"></i><b>10.3</b> Fixed Effects Regression</a><ul>
<li class="chapter" data-level="" data-path="10-3-fixed-effects-regression.html"><a href="10-3-fixed-effects-regression.html#estimation-and-inference"><i class="fa fa-check"></i>Estimation and Inference</a></li>
<li class="chapter" data-level="" data-path="10-3-fixed-effects-regression.html"><a href="10-3-fixed-effects-regression.html#application-to-traffic-deaths"><i class="fa fa-check"></i>Application to Traffic Deaths</a></li>
</ul></li>
<li class="chapter" data-level="10.4" data-path="10-4-regression-with-time-fixed-effects.html"><a href="10-4-regression-with-time-fixed-effects.html"><i class="fa fa-check"></i><b>10.4</b> Regression with Time Fixed Effects</a></li>
<li class="chapter" data-level="10.5" data-path="10-5-tferaaseffer.html"><a href="10-5-tferaaseffer.html"><i class="fa fa-check"></i><b>10.5</b> The Fixed Effects Regression Assumptions and Standard Errors for Fixed Effects Regression</a></li>
<li class="chapter" data-level="10.6" data-path="10-6-drunk-driving-laws-and-traffic-deaths.html"><a href="10-6-drunk-driving-laws-and-traffic-deaths.html"><i class="fa fa-check"></i><b>10.6</b> Drunk Driving Laws and Traffic Deaths</a></li>
<li class="chapter" data-level="10.7" data-path="10-7-exercises-8.html"><a href="10-7-exercises-8.html"><i class="fa fa-check"></i><b>10.7</b> Exercises</a></li>
</ul></li>
<li class="chapter" data-level="11" data-path="11-rwabdv.html"><a href="11-rwabdv.html"><i class="fa fa-check"></i><b>11</b> Regression with a Binary Dependent Variable</a><ul>
<li class="chapter" data-level="11.1" data-path="11-1-binary-dependent-variables-and-the-linear-probability-model.html"><a href="11-1-binary-dependent-variables-and-the-linear-probability-model.html"><i class="fa fa-check"></i><b>11.1</b> Binary Dependent Variables and the Linear Probability Model</a></li>
<li class="chapter" data-level="11.2" data-path="11-2-palr.html"><a href="11-2-palr.html"><i class="fa fa-check"></i><b>11.2</b> Probit and Logit Regression</a><ul>
<li class="chapter" data-level="" data-path="11-2-palr.html"><a href="11-2-palr.html#probit-regression"><i class="fa fa-check"></i>Probit Regression</a></li>
<li class="chapter" data-level="" data-path="11-2-palr.html"><a href="11-2-palr.html#logit-regression"><i class="fa fa-check"></i>Logit Regression</a></li>
</ul></li>
<li class="chapter" data-level="11.3" data-path="11-3-estimation-and-inference-in-the-logit-and-probit-models.html"><a href="11-3-estimation-and-inference-in-the-logit-and-probit-models.html"><i class="fa fa-check"></i><b>11.3</b> Estimation and Inference in the Logit and Probit Models</a></li>
<li class="chapter" data-level="11.4" data-path="11-4-application-to-the-boston-hmda-data.html"><a href="11-4-application-to-the-boston-hmda-data.html"><i class="fa fa-check"></i><b>11.4</b> Application to the Boston HMDA Data</a></li>
<li class="chapter" data-level="11.5" data-path="11-5-exercises-9.html"><a href="11-5-exercises-9.html"><i class="fa fa-check"></i><b>11.5</b> Exercises</a></li>
</ul></li>
<li class="chapter" data-level="12" data-path="12-ivr.html"><a href="12-ivr.html"><i class="fa fa-check"></i><b>12</b> Instrumental Variables Regression</a><ul>
<li class="chapter" data-level="12.1" data-path="12-1-TIVEWASRAASI.html"><a href="12-1-TIVEWASRAASI.html"><i class="fa fa-check"></i><b>12.1</b> The IV Estimator with a Single Regressor and a Single Instrument</a></li>
<li class="chapter" data-level="12.2" data-path="12-2-TGIVRM.html"><a href="12-2-TGIVRM.html"><i class="fa fa-check"></i><b>12.2</b> The General IV Regression Model</a></li>
<li class="chapter" data-level="12.3" data-path="12-3-civ.html"><a href="12-3-civ.html"><i class="fa fa-check"></i><b>12.3</b> Checking Instrument Validity</a></li>
<li class="chapter" data-level="12.4" data-path="12-4-attdfc.html"><a href="12-4-attdfc.html"><i class="fa fa-check"></i><b>12.4</b> Application to the Demand for Cigarettes</a></li>
<li class="chapter" data-level="12.5" data-path="12-5-where-do-valid-instruments-come-from.html"><a href="12-5-where-do-valid-instruments-come-from.html"><i class="fa fa-check"></i><b>12.5</b> Where Do Valid Instruments Come From?</a></li>
<li class="chapter" data-level="12.6" data-path="12-6-exercises-10.html"><a href="12-6-exercises-10.html"><i class="fa fa-check"></i><b>12.6</b> Exercises</a></li>
</ul></li>
<li class="chapter" data-level="13" data-path="13-eaqe.html"><a href="13-eaqe.html"><i class="fa fa-check"></i><b>13</b> Experiments and Quasi-Experiments</a><ul>
<li class="chapter" data-level="13.1" data-path="13-1-poceaie.html"><a href="13-1-poceaie.html"><i class="fa fa-check"></i><b>13.1</b> Potential Outcomes, Causal Effects and Idealized Experiments</a></li>
<li class="chapter" data-level="13.2" data-path="13-2-threats-to-validity-of-experiments.html"><a href="13-2-threats-to-validity-of-experiments.html"><i class="fa fa-check"></i><b>13.2</b> Threats to Validity of Experiments</a></li>
<li class="chapter" data-level="13.3" data-path="13-3-experimental-estimates-of-the-effect-of-class-size-reductions.html"><a href="13-3-experimental-estimates-of-the-effect-of-class-size-reductions.html"><i class="fa fa-check"></i><b>13.3</b> Experimental Estimates of the Effect of Class Size Reductions</a><ul>
<li class="chapter" data-level="" data-path="13-3-experimental-estimates-of-the-effect-of-class-size-reductions.html"><a href="13-3-experimental-estimates-of-the-effect-of-class-size-reductions.html#experimental-design-and-the-data-set"><i class="fa fa-check"></i>Experimental Design and the Data Set</a></li>
<li class="chapter" data-level="" data-path="13-3-experimental-estimates-of-the-effect-of-class-size-reductions.html"><a href="13-3-experimental-estimates-of-the-effect-of-class-size-reductions.html#analysis-of-the-star-data"><i class="fa fa-check"></i>Analysis of the STAR Data</a></li>
</ul></li>
<li class="chapter" data-level="13.4" data-path="13-4-qe.html"><a href="13-4-qe.html"><i class="fa fa-check"></i><b>13.4</b> Quasi Experiments</a><ul>
<li class="chapter" data-level="" data-path="13-4-qe.html"><a href="13-4-qe.html#the-differences-in-differences-estimator"><i class="fa fa-check"></i>The Differences-in-Differences Estimator</a></li>
<li class="chapter" data-level="" data-path="13-4-qe.html"><a href="13-4-qe.html#regression-discontinuity-estimators"><i class="fa fa-check"></i>Regression Discontinuity Estimators</a></li>
</ul></li>
<li class="chapter" data-level="13.5" data-path="13-5-exercises-11.html"><a href="13-5-exercises-11.html"><i class="fa fa-check"></i><b>13.5</b> Exercises</a></li>
</ul></li>
<li class="chapter" data-level="14" data-path="14-ittsraf.html"><a href="14-ittsraf.html"><i class="fa fa-check"></i><b>14</b> Introduction to Time Series Regression and Forecasting</a><ul>
<li class="chapter" data-level="14.1" data-path="14-1-using-regression-models-for-forecasting.html"><a href="14-1-using-regression-models-for-forecasting.html"><i class="fa fa-check"></i><b>14.1</b> Using Regression Models for Forecasting</a></li>
<li class="chapter" data-level="14.2" data-path="14-2-tsdasc.html"><a href="14-2-tsdasc.html"><i class="fa fa-check"></i><b>14.2</b> Time Series Data and Serial Correlation</a><ul>
<li class="chapter" data-level="" data-path="14-2-tsdasc.html"><a href="14-2-tsdasc.html#notation-lags-differences-logarithms-and-growth-rates"><i class="fa fa-check"></i>Notation, Lags, Differences, Logarithms and Growth Rates</a></li>
</ul></li>
<li class="chapter" data-level="14.3" data-path="14-3-autoregressions.html"><a href="14-3-autoregressions.html"><i class="fa fa-check"></i><b>14.3</b> Autoregressions</a><ul>
<li><a href="14-3-autoregressions.html#autoregressive-models-of-order-p">Autoregressive Models of Order <span class="math inline">\(p\)</span></a></li>
</ul></li>
<li class="chapter" data-level="14.4" data-path="14-4-cybtmpi.html"><a href="14-4-cybtmpi.html"><i class="fa fa-check"></i><b>14.4</b> Can You Beat the Market? (Part I)</a></li>
<li class="chapter" data-level="14.5" data-path="14-5-apatadlm.html"><a href="14-5-apatadlm.html"><i class="fa fa-check"></i><b>14.5</b> Additional Predictors and The ADL Model</a><ul>
<li class="chapter" data-level="" data-path="14-5-apatadlm.html"><a href="14-5-apatadlm.html#forecast-uncertainty-and-forecast-intervals"><i class="fa fa-check"></i>Forecast Uncertainty and Forecast Intervals</a></li>
</ul></li>
<li class="chapter" data-level="14.6" data-path="14-6-llsuic.html"><a href="14-6-llsuic.html"><i class="fa fa-check"></i><b>14.6</b> Lag Length Selection Using Information Criteria</a></li>
<li class="chapter" data-level="14.7" data-path="14-7-nit.html"><a href="14-7-nit.html"><i class="fa fa-check"></i><b>14.7</b> Nonstationarity I: Trends</a></li>
<li class="chapter" data-level="14.8" data-path="14-8-niib.html"><a href="14-8-niib.html"><i class="fa fa-check"></i><b>14.8</b> Nonstationarity II: Breaks</a></li>
<li class="chapter" data-level="14.9" data-path="14-9-can-you-beat-the-market-part-ii.html"><a href="14-9-can-you-beat-the-market-part-ii.html"><i class="fa fa-check"></i><b>14.9</b> Can You Beat the Market? (Part II)</a></li>
</ul></li>
<li class="chapter" data-level="15" data-path="15-eodce.html"><a href="15-eodce.html"><i class="fa fa-check"></i><b>15</b> Estimation of Dynamic Causal Effects</a><ul>
<li class="chapter" data-level="15.1" data-path="15-1-the-orange-juice-data.html"><a href="15-1-the-orange-juice-data.html"><i class="fa fa-check"></i><b>15.1</b> The Orange Juice Data</a></li>
<li class="chapter" data-level="15.2" data-path="15-2-dynamic-causal-effects.html"><a href="15-2-dynamic-causal-effects.html"><i class="fa fa-check"></i><b>15.2</b> Dynamic Causal Effects</a></li>
<li class="chapter" data-level="15.3" data-path="15-3-dynamic-multipliers-and-cumulative-dynamic-multipliers.html"><a href="15-3-dynamic-multipliers-and-cumulative-dynamic-multipliers.html"><i class="fa fa-check"></i><b>15.3</b> Dynamic Multipliers and Cumulative Dynamic Multipliers</a></li>
<li class="chapter" data-level="15.4" data-path="15-4-hac-standard-errors.html"><a href="15-4-hac-standard-errors.html"><i class="fa fa-check"></i><b>15.4</b> HAC Standard Errors</a></li>
<li class="chapter" data-level="15.5" data-path="15-5-estimation-of-dynamic-causal-effects-with-strictly-exogeneous-regressors.html"><a href="15-5-estimation-of-dynamic-causal-effects-with-strictly-exogeneous-regressors.html"><i class="fa fa-check"></i><b>15.5</b> Estimation of Dynamic Causal Effects with Strictly Exogeneous Regressors</a></li>
<li class="chapter" data-level="15.6" data-path="15-6-orange-juice-prices-and-cold-weather.html"><a href="15-6-orange-juice-prices-and-cold-weather.html"><i class="fa fa-check"></i><b>15.6</b> Orange Juice Prices and Cold Weather</a></li>
</ul></li>
<li class="chapter" data-level="16" data-path="16-atitsr.html"><a href="16-atitsr.html"><i class="fa fa-check"></i><b>16</b> Additional Topics in Time Series Regression</a><ul>
<li class="chapter" data-level="16.1" data-path="16-1-vector-autoregressions.html"><a href="16-1-vector-autoregressions.html"><i class="fa fa-check"></i><b>16.1</b> Vector Autoregressions</a></li>
<li class="chapter" data-level="16.2" data-path="16-2-ooiatdfglsurt.html"><a href="16-2-ooiatdfglsurt.html"><i class="fa fa-check"></i><b>16.2</b> Orders of Integration and the DF-GLS Unit Root Test</a></li>
<li class="chapter" data-level="16.3" data-path="16-3-cointegration.html"><a href="16-3-cointegration.html"><i class="fa fa-check"></i><b>16.3</b> Cointegration</a></li>
<li class="chapter" data-level="16.4" data-path="16-4-volatility-clustering-and-autoregressive-conditional-heteroskedasticity.html"><a href="16-4-volatility-clustering-and-autoregressive-conditional-heteroskedasticity.html"><i class="fa fa-check"></i><b>16.4</b> Volatility Clustering and Autoregressive Conditional Heteroskedasticity</a><ul>
<li class="chapter" data-level="" data-path="16-4-volatility-clustering-and-autoregressive-conditional-heteroskedasticity.html"><a href="16-4-volatility-clustering-and-autoregressive-conditional-heteroskedasticity.html#arch-and-garch-models"><i class="fa fa-check"></i>ARCH and GARCH Models</a></li>
<li class="chapter" data-level="" data-path="16-4-volatility-clustering-and-autoregressive-conditional-heteroskedasticity.html"><a href="16-4-volatility-clustering-and-autoregressive-conditional-heteroskedasticity.html#application-to-stock-price-volatility"><i class="fa fa-check"></i>Application to Stock Price Volatility</a></li>
<li class="chapter" data-level="" data-path="16-4-volatility-clustering-and-autoregressive-conditional-heteroskedasticity.html"><a href="16-4-volatility-clustering-and-autoregressive-conditional-heteroskedasticity.html#summary-8"><i class="fa fa-check"></i>Summary</a></li>
</ul></li>
<li class="chapter" data-level="16.5" data-path="16-5-exercises-12.html"><a href="16-5-exercises-12.html"><i class="fa fa-check"></i><b>16.5</b> Exercises</a></li>
</ul></li>
<li class="chapter" data-level="" data-path="references.html"><a href="references.html"><i class="fa fa-check"></i>References</a></li>
<li class="divider"></li>
<li><a href="https://github.com/rstudio/bookdown" target="blank">Published with bookdown</a></li>
</ul>
</nav>
</div>
<div class="book-body">
<div class="body-inner">
<div class="book-header" role="navigation">
<h1>
<i class="fa fa-circle-o-notch fa-spin"></i><a href="./">Introduction to Econometrics with R</a>
</h1>
</div>
<div class="page-wrapper" tabindex="-1" role="main">
<div class="page-inner">
<section class="normal" id="section-">
<div class = rmdreview>
This book is in <b>Open Review</b>. We want your feedback to make the book better for you and other students. You may annotate some text by <span style="background-color: #3297FD; color: white">selecting it with the cursor</span> and then click the <i class="h-icon-annotate"></i> on the pop-up menu. You can also see the annotations of others: click the <i class="h-icon-chevron-left"></i> in the upper right hand corner of the page <i class="fa fa-arrow-circle-right fa-rotate-315" aria-hidden="true"></i>
</div>
<div id="random-variables-and-probability-distributions" class="section level2">
<h2><span class="header-section-number">2.1</span> Random Variables and Probability Distributions</h2>
<p>Let us briefly review some basic concepts of probability theory.</p>
<ul>
<li>The mutually exclusive results of a random process are called the <em>outcomes</em>. ‘Mutually exclusive’ means that only one of the possible outcomes can be observed.</li>
<li>We refer to the <em>probability</em> of an outcome as the proportion that the outcome occurs in the long run, that is, if the experiment is repeated very often.</li>
<li>The set of all possible outcomes of a random variable is called the <em>sample space</em>.</li>
<li>An <em>event</em> is a subset of the sample space and consists of one or more outcomes.</li>
</ul>
<p>These ideas are unified in the concept of a <em>random variable</em> which is a numerical summary of random outcomes. Random variables can be <em>discrete</em> or <em>continuous</em>.</p>
<ul>
<li>Discrete random variables have discrete outcomes, e.g., <span class="math inline">\(0\)</span> and <span class="math inline">\(1\)</span>.</li>
<li>A continuous random variable may take on a continuum of possible values.</li>
</ul>
<div id="probability-distributions-of-discrete-random-variables" class="section level3 unnumbered">
<h3>Probability Distributions of Discrete Random Variables</h3>
<p>A typical example for a discrete random variable <span class="math inline">\(D\)</span> is the result of a dice
roll: in terms of a random experiment this is nothing but randomly selecting a
sample of size <span class="math inline">\(1\)</span> from a set of numbers which are mutually exclusive outcomes.
Here, the sample space is <span class="math inline">\(\{1,2,3,4,5,6\}\)</span> and we can think of many different
events, e.g., ‘the observed outcome lies between <span class="math inline">\(2\)</span> and <span class="math inline">\(5\)</span>’.</p>
<p>A basic function to draw random samples from a specified set of elements is the
the function <tt>sample()</tt>, see <code>?sample</code>. We can use it to simulate the random outcome of a dice roll. Let’s roll the dice!</p>
<pre class="sourceCode r"><code class="sourceCode r"><span class="kw">sample</span>(<span class="dv">1</span><span class="op">:</span><span class="dv">6</span>, <span class="dv">1</span>) </code></pre>
<pre><code>## [1] 4</code></pre>
<p>The probability distribution of a discrete random variable is the list of all possible values of the variable and their probabilities which sum to <span class="math inline">\(1\)</span>. The cumulative probability distribution function gives the probability that the random variable is less than or equal to a particular value.</p>
<p>For the dice roll, the probability distribution and the cumulative probability distribution are summarized in Table <a href="2-1-random-variables-and-probability-distributions.html#tab:pdist">2.1</a>.</p>
<table>
<caption>
<span id="tab:pdist">Table 2.1: </span>PDF and CDF of a Dice Roll
</caption>
<tbody>
<tr>
<td style="text-align:left;">
Outcome
</td>
<td style="text-align:left;">
1
</td>
<td style="text-align:left;">
2
</td>
<td style="text-align:left;">
3
</td>
<td style="text-align:left;">
4
</td>
<td style="text-align:left;">
5
</td>
<td style="text-align:left;">
6
</td>
</tr>
<tr>
<td style="text-align:left;">
Probability
</td>
<td style="text-align:left;">
1/6
</td>
<td style="text-align:left;">
1/6
</td>
<td style="text-align:left;">
1/6
</td>
<td style="text-align:left;">
1/6
</td>
<td style="text-align:left;">
1/6
</td>
<td style="text-align:left;">
1/6
</td>
</tr>
<tr>
<td style="text-align:left;">
Cumulative Probability
</td>
<td style="text-align:left;">
1/6
</td>
<td style="text-align:left;">
2/6
</td>
<td style="text-align:left;">
3/6
</td>
<td style="text-align:left;">
4/6
</td>
<td style="text-align:left;">
5/6
</td>
<td style="text-align:left;">
1
</td>
</tr>
</tbody>
</table>
<p>We can easily plot both functions using <tt>R</tt>. Since the probability equals <span class="math inline">\(1/6\)</span> for each outcome, we set up the vector <tt>probability</tt> by using the function <tt>rep()</tt> which replicates a given value a specified number of times.</p>
<div class="unfolded">
<pre class="sourceCode r"><code class="sourceCode r"><span class="co"># generate the vector of probabilities </span>
probability <-<span class="st"> </span><span class="kw">rep</span>(<span class="dv">1</span><span class="op">/</span><span class="dv">6</span>, <span class="dv">6</span>)
<span class="co"># plot the probabilites </span>
<span class="kw">plot</span>(probability,
<span class="dt">main =</span> <span class="st">"Probability Distribution"</span>,
<span class="dt">xlab =</span> <span class="st">"outcomes"</span>) </code></pre>
<p><img src="ITER_files/figure-html/unnamed-chunk-14-1.png" width="80%" style="display: block; margin: auto;" /></p>
</div>
<p>For the cumulative probability distribution we need the cumulative probabilities, i.e., we need the cumulative sums of the vector <tt>probability</tt>. These sums can be computed using <tt>cumsum()</tt>.</p>
<div class="unfolded">
<pre class="sourceCode r"><code class="sourceCode r"><span class="co"># generate the vector of cumulative probabilities </span>
cum_probability <-<span class="st"> </span><span class="kw">cumsum</span>(probability)
<span class="co"># plot the probabilites </span>
<span class="kw">plot</span>(cum_probability,
<span class="dt">xlab =</span> <span class="st">"outcomes"</span>,
<span class="dt">main =</span> <span class="st">"Cumulative Probability Distribution"</span>) </code></pre>
<p><img src="ITER_files/figure-html/unnamed-chunk-15-1.png" width="80%" style="display: block; margin: auto;" /></p>
</div>
</div>
<div id="bernoulli-trials" class="section level3 unnumbered">
<h3>Bernoulli Trials</h3>
<p>The set of elements from which <tt>sample()</tt> draws outcomes does not have to consist of numbers
only. We might as well simulate coin tossing with outcomes <span class="math inline">\(H\)</span> (heads) and <span class="math inline">\(T\)</span>
(tails).</p>
<pre class="sourceCode r"><code class="sourceCode r"><span class="kw">sample</span>(<span class="kw">c</span>(<span class="st">"H"</span>, <span class="st">"T"</span>), <span class="dv">1</span>) </code></pre>
<pre><code>## [1] "H"</code></pre>
<p>The result of a single coin toss is a <em>Bernoulli</em> distributed random variable, i.e., a variable with two possible distinct outcomes.</p>
<p>Imagine you are about to toss a coin <span class="math inline">\(10\)</span> times in a row and wonder how likely it
is to end up with a <span class="math inline">\(5\)</span> times heads. This is a typical example of what we call a <em>Bernoulli experiment</em> as it consists of <span class="math inline">\(n=10\)</span> Bernoulli trials that are independent of each other and we are interested in the likelihood of observing <span class="math inline">\(k=5\)</span> successes <span class="math inline">\(H\)</span> that occur with probability <span class="math inline">\(p=0.5\)</span> (assuming a fair coin) in each trial. Note that the order of the outcomes does not matter here.</p>
<p>It is a <a href="https://en.wikipedia.org/wiki/Binomial_distribution">well known result</a> that the number of successes <span class="math inline">\(k\)</span> in a Bernoulli experiment follows a binomial distribution. We denote this as</p>
<p><span class="math display">\[k \sim B(n,p).\]</span></p>
<p>The probability of observing <span class="math inline">\(k\)</span> successes in the experiment <span class="math inline">\(B(n,p)\)</span> is given by</p>
<p><span class="math display">\[f(k)=P(k)=\begin{pmatrix}n\\ k \end{pmatrix} \cdot p^k \cdot
(1-p)^{n-k}=\frac{n!}{k!(n-k)!} \cdot p^k \cdot (1-p)^{n-k}\]</span></p>
<p>with <span class="math inline">\(\begin{pmatrix}n\\ k \end{pmatrix}\)</span> the binomial coefficient.</p>
<p>In <tt>R</tt>, we can solve problems like the one stated above by means of the function <tt>dbinom()</tt> which calculates <span class="math inline">\(P(k\vert n, p)\)</span> the probability of the binomial distribution given the parameters <tt>x</tt> (<span class="math inline">\(k\)</span>),
<tt>size</tt> (<span class="math inline">\(n\)</span>), and <tt>prob</tt> (<span class="math inline">\(p\)</span>), see <code>?binom</code>. Let us compute <span class="math inline">\(P(k=5\vert n = 10, p = 0.5)\)</span> (we write this short as <span class="math inline">\(P(k=5)\)</span>.</p>
<pre class="sourceCode r"><code class="sourceCode r"><span class="kw">dbinom</span>(<span class="dt">x =</span> <span class="dv">5</span>,
<span class="dt">size =</span> <span class="dv">10</span>,
<span class="dt">prob =</span> <span class="fl">0.5</span>) </code></pre>
<pre><code>## [1] 0.2460938</code></pre>
<p>We conclude that <span class="math inline">\(P(k=5)\)</span>, the probability of observing Head <span class="math inline">\(k=5\)</span> times when tossing a fair coin <span class="math inline">\(n=10\)</span> times is about <span class="math inline">\(24.6\%\)</span>.</p>
<p>Now assume we are interested in <span class="math inline">\(P(4 \leq k \leq 7)\)</span>, i.e., the probability of
observing <span class="math inline">\(4\)</span>, <span class="math inline">\(5\)</span>, <span class="math inline">\(6\)</span> or <span class="math inline">\(7\)</span> successes for <span class="math inline">\(B(10, 0.5)\)</span>. This may be computed by providing a vector as the argument <tt>x</tt> in our call of <tt>dbinom()</tt> and summing up using <tt>sum()</tt>.</p>
<pre class="sourceCode r"><code class="sourceCode r"><span class="co"># compute P(4 <= k <= 7) using 'dbinom()'</span>
<span class="kw">sum</span>(<span class="kw">dbinom</span>(<span class="dt">x =</span> <span class="dv">4</span><span class="op">:</span><span class="dv">7</span>, <span class="dt">size =</span> <span class="dv">10</span>, <span class="dt">prob =</span> <span class="fl">0.5</span>))</code></pre>
<pre><code>## [1] 0.7734375</code></pre>
<p>An alternative approach is to use <tt>pbinom()</tt>, the distribution function of the binomial distribution to compute <span class="math display">\[P(4 \leq k \leq 7) = P(k \leq 7) - P(k\leq3 ).\]</span></p>
<pre class="sourceCode r"><code class="sourceCode r"><span class="co"># compute P(4 <= k <= 7) using 'pbinom()'</span>
<span class="kw">pbinom</span>(<span class="dt">size =</span> <span class="dv">10</span>, <span class="dt">prob =</span> <span class="fl">0.5</span>, <span class="dt">q =</span> <span class="dv">7</span>) <span class="op">-</span><span class="st"> </span><span class="kw">pbinom</span>(<span class="dt">size =</span> <span class="dv">10</span>, <span class="dt">prob =</span> <span class="fl">0.5</span>, <span class="dt">q =</span> <span class="dv">3</span>) </code></pre>
<pre><code>## [1] 0.7734375</code></pre>
<p>The probability distribution of a discrete random variable is nothing but a list
of all possible outcomes that can occur and their respective probabilities. In the coin tossing example we have <span class="math inline">\(11\)</span> possible outcomes for <span class="math inline">\(k\)</span>.</p>
<pre class="sourceCode r"><code class="sourceCode r"><span class="co"># set up vector of possible outcomes</span>
k <-<span class="st"> </span><span class="dv">0</span><span class="op">:</span><span class="dv">10</span></code></pre>
<p>To visualize the probability distribution function of <span class="math inline">\(k\)</span> we may therefore do the following:</p>
<pre class="sourceCode r"><code class="sourceCode r"><span class="co"># assign the probabilities</span>
probability <-<span class="st"> </span><span class="kw">dbinom</span>(<span class="dt">x =</span> k,
<span class="dt">size =</span> <span class="dv">10</span>,
<span class="dt">prob =</span> <span class="fl">0.5</span>)
<span class="co"># plot the outcomes against their probabilities</span>
<span class="kw">plot</span>(<span class="dt">x =</span> k,
<span class="dt">y =</span> probability,
<span class="dt">main =</span> <span class="st">"Probability Distribution Function"</span>) </code></pre>
<p><img src="ITER_files/figure-html/unnamed-chunk-21-1.png" width="80%" style="display: block; margin: auto;" /></p>
<p>In a similar fashion we may plot the cumulative distribution function of <span class="math inline">\(k\)</span> by
executing the following code chunk:</p>
<pre class="sourceCode r"><code class="sourceCode r"><span class="co"># compute cumulative probabilities</span>
prob <-<span class="st"> </span><span class="kw">pbinom</span>(<span class="dt">q =</span> k,
<span class="dt">size =</span> <span class="dv">10</span>,
<span class="dt">prob =</span> <span class="fl">0.5</span>)
<span class="co"># plot the cumulative probabilities</span>
<span class="kw">plot</span>(<span class="dt">x =</span> k,
<span class="dt">y =</span> prob,
<span class="dt">main =</span> <span class="st">"Cumulative Distribution Function"</span>) </code></pre>
<p><img src="ITER_files/figure-html/unnamed-chunk-22-1.png" width="80%" style="display: block; margin: auto;" /></p>
</div>
<div id="expected-value-mean-and-variance" class="section level3 unnumbered">
<h3>Expected Value, Mean and Variance</h3>
<p>The expected value of a random variable is, loosely, the long-run average value of its outcomes when the number of repeated trials is large. For a discrete random variable, the
expected value is computed as a weighted average of its possible outcomes
whereby the weights are the related probabilities. This is formalized in Key
Concept 2.1.</p>
<div class = "keyconcept" <div class = "keyconcept" id="KC2.1">
<h3 class = "right"> Key Concept 2.1 </h3>
<h3 class= "left"> Expected Value and the Mean </h3>
<p> Suppose the random variable $Y$
takes on $k$ possible values, $y_1, \dots, y_k$, where $y_1$ denotes the first
value, $y_2$ denotes the second value, and so forth, and that the probability
that $Y$ takes on $y_1$ is $p_1$, the probability that $Y$ takes on $y_2$ is
$p_2$ and so forth. The expected value of $Y$, $E(Y)$ is defined as
$$ E(Y) = y_1 p_1 + y_2 p_2 + \cdots + y_k p_k = \sum_{i=1}^k y_i p_i $$
where the notation $\sum_{i=1}^k y_i p_i$ means "the sum of $y_i$ $p_i$ for $i$
running from $1$ to $k$". The expected value of $Y$ is also called the mean of $Y$
or the expectation of $Y$ and is denoted by $\mu_Y$.
</p>
</div>
<p>In the dice example, the random variable, <span class="math inline">\(D\)</span> say, takes on <span class="math inline">\(6\)</span> possible values
<span class="math inline">\(d_1 = 1, d_2 = 2, \dots, d_6 = 6\)</span>. Assuming a fair dice, each of the <span class="math inline">\(6\)</span> outcomes occurs with a probability of <span class="math inline">\(1/6\)</span>. It is therefore easy to calculate the exact value of <span class="math inline">\(E(D)\)</span> by hand:</p>
<p><span class="math display">\[ E(D) = 1/6 \sum_{i=1}^6 d_i = 3.5 \]</span></p>
<p><span class="math inline">\(E(D)\)</span> is simply the average of the natural numbers from <span class="math inline">\(1\)</span> to <span class="math inline">\(6\)</span> since all weights <span class="math inline">\(p_i\)</span> are <span class="math inline">\(1/6\)</span>. This can be easily calculated using the function <tt>mean()</tt> which computes the arithmetic mean of a numeric vector.</p>
<pre class="sourceCode r"><code class="sourceCode r"><span class="co"># compute mean of natural numbers from 1 to 6</span>
<span class="kw">mean</span>(<span class="dv">1</span><span class="op">:</span><span class="dv">6</span>)</code></pre>
<pre><code>## [1] 3.5</code></pre>
<p>An example of sampling with replacement is rolling a dice three times in a row.</p>
<pre class="sourceCode r"><code class="sourceCode r"><span class="co"># set seed for reproducibility</span>
<span class="kw">set.seed</span>(<span class="dv">1</span>)
<span class="co"># rolling a dice three times in a row</span>
<span class="kw">sample</span>(<span class="dv">1</span><span class="op">:</span><span class="dv">6</span>, <span class="dv">3</span>, <span class="dt">replace =</span> T)</code></pre>
<pre><code>## [1] 2 3 4</code></pre>
<p>Note that every call of <code>sample(1:6, 3, replace = T)</code> gives a different outcome since we are we are drawing with replacement at random.</p>
<div class="rmdknit">
<p>Sequences of random numbers generated by R are pseudo-random numbers, i.e., they are not “truly” random but approximate the properties of sequences of random numbers. Since this approximation is good enough for our purposes we refer to pseudo-random numbers as random numbers throughout this book.</p>
<p>In general, sequences of random numbers are generated by functions called “pseudo-random number generators” (PRNGs). The PRNG in R works by performing some operation on a deterministic value. Generally, this value is the previous number generated by the PRNG. However, the first time the PRNG is used, there is no previous value. A “seed” is the first value of a sequence of numbers — it initializes the sequence. Each seed value will correspond to a different sequence of values. In R a seed can be set using <tt>set.seed()</tt>.</p>
<p>This is convenient for us:</p>
If we provide the same seed twice, we get the same sequence of numbers twice. Thus, setting a seed before excecuting R code which invovles random numbers makes the outcome reproducible!
</div>
<p>Of course we could also consider a much bigger number of trials, <span class="math inline">\(10000\)</span> say.
Doing so, it would be pointless to simply print the results to the console: by
default <tt>R</tt> displays up to <span class="math inline">\(1000\)</span> entries of large vectors and omits the
remainder (give it a try). Eyeballing the numbers does not reveal much.
Instead, let us calculate the sample average of the outcomes using <tt>mean()</tt> and see if the result comes close to the expected value <span class="math inline">\(E(D)=3.5\)</span>.</p>
<pre class="sourceCode r"><code class="sourceCode r"><span class="co"># set seed for reproducibility</span>
<span class="kw">set.seed</span>(<span class="dv">1</span>)
<span class="co"># compute the sample mean of 10000 dice rolls</span>
<span class="kw">mean</span>(<span class="kw">sample</span>(<span class="dv">1</span><span class="op">:</span><span class="dv">6</span>,
<span class="dv">10000</span>,
<span class="dt">replace =</span> T))</code></pre>
<pre><code>## [1] 3.5039</code></pre>
<p>We find the sample mean to be fairly close to the expected value. This result will be discussed in Chapter <a href="2-2-RSATDOSA.html#RSATDOSA">2.2</a> in more detail.</p>
<p>Other frequently encountered measures are the variance and the standard deviation. Both are measures of the <em>dispersion</em> of a random variable.</p>
<div id="KC2.2" class="keyconcept">
<h3 class="right">
Key Concept 2.2
</h3>
<h3 class="left">
Variance and Standard Deviation
</h3>
<p>
The variance of the discrete random variable <span class="math inline">\(Y\)</span>, denoted <span class="math inline">\(\sigma^2_Y\)</span>, is
<span class="math display">\[ \sigma^2_Y = \text{Var}(Y) = E\left[(Y-\mu_y)^2\right] = \sum_{i=1}^k (y_i - \mu_y)^2 p_i \]</span>
The standard deviation of <span class="math inline">\(Y\)</span> is <span class="math inline">\(\sigma_Y\)</span>, the square root of the variance. The units of the standard deviation are the same as the units of <span class="math inline">\(Y\)</span>.
</p>
</div>
<p>The variance as defined in Key Concept 2.2, being a population quantity, <em>is not</em> implemented as a function in R. Instead we have the function <tt>var()</tt> which computes the <em>sample variance</em></p>
<p><span class="math display">\[ s^2_Y = \frac{1}{n-1} \sum_{i=1}^n (y_i - \overline{y})^2. \]</span></p>
<p>Remember that <span class="math inline">\(s^2_Y\)</span> is different from the so called <em>population variance</em> of a discrete random variable <span class="math inline">\(Y\)</span>,</p>
<p><span class="math display">\[ \text{Var}(Y) = \frac{1}{N} \sum_{i=1}^N (y_i - \mu_Y)^2. \]</span></p>
<p>since it measures how the <span class="math inline">\(n\)</span> observations in the sample are dispersed around the sample average <span class="math inline">\(\overline{y}\)</span>. Instead, <span class="math inline">\(\text{Var}(Y)\)</span> measures the dispersion of the whole population (<span class="math inline">\(N\)</span> members) around the population mean <span class="math inline">\(\mu_Y\)</span>. The difference becomes clear when we look at our dice rolling example. For <span class="math inline">\(D\)</span> we have</p>
<p><span class="math display">\[ \text{Var}(D) = 1/6 \sum_{i=1}^6 (d_i - 3.5)^2 = 2.92 \]</span>
which is obviously different from the result of <span class="math inline">\(s^2\)</span> as computed by <tt>var()</tt>.</p>
<pre class="sourceCode r"><code class="sourceCode r"><span class="kw">var</span>(<span class="dv">1</span><span class="op">:</span><span class="dv">6</span>)</code></pre>
<pre><code>## [1] 3.5</code></pre>
<p>The sample variance as computed by <tt>var()</tt> is an <em>estimator</em> of the population variance. You may check this using the widget below.</p>
<iframe src="DCL/playground.html" frameborder="0" scrolling="no" style="width:100%;height:340px">
</iframe>
</div>
<div id="probability-distributions-of-continuous-random-variables" class="section level3 unnumbered">
<h3>Probability Distributions of Continuous Random Variables</h3>
<p>Since a continuous random variable takes on a continuum of possible values, we
cannot use the concept of a probability distribution as used for discrete random
variables. Instead, the probability distribution of a continuous random variable
is summarized by its <em>probability density function</em> (PDF).</p>
<p>The cumulative probability distribution function (CDF) for a continuous random variable is defined just as in the discrete case. Hence, the cumulative probability
distribution of a continuous random variables states the probability that the
random variable is less than or equal to a particular value.</p>
<p>For completeness, we present revisions of Key Concepts 2.1 and 2.2 for the continuous case.</p>
<div id="KC2.3" class="keyconcept">
<h3 class="right">
Key Concept 2.3
</h3>
<h3 class="left">
Probabilities, Expected Value and Variance of a Continuous Random Variable
</h3>
<p>
<p>Let <span class="math inline">\(f_Y(y)\)</span> denote the probability density function of <span class="math inline">\(Y\)</span>. The probability that <span class="math inline">\(Y\)</span> falls between <span class="math inline">\(a\)</span> and <span class="math inline">\(b\)</span> where <span class="math inline">\(a < b\)</span> is
<span class="math display">\[ P(a \leq Y \leq b) = \int_a^b f_Y(y) \mathrm{d}y. \]</span>
We further have that <span class="math inline">\(P(-\infty \leq Y \leq \infty) = 1\)</span> and therefore <span class="math inline">\(\int_{-\infty}^{\infty} f_Y(y) \mathrm{d}y = 1\)</span>.</p>
<p>As for the discrete case, the expected value of <span class="math inline">\(Y\)</span> is the probability weighted average of its values. Due to continuity, we use integrals instead of sums. The expected value of <span class="math inline">\(Y\)</span> is defined as</p>
<p><span class="math display">\[ E(Y) = \mu_Y = \int y f_Y(y) \mathrm{d}y. \]</span></p>
<p>The variance is the expected value of <span class="math inline">\((Y - \mu_Y)^2\)</span>. We thus have</p>
<span class="math display">\[\text{Var}(Y) = \sigma_Y^2 = \int (y - \mu_Y)^2 f_Y(y) \mathrm{d}y.\]</span>
</p>
</div>
<p>Let us discuss an example:</p>
<p>Consider the continuous random variable <span class="math inline">\(X\)</span> with probability density function</p>
<p><span class="math display">\[ f_X(x) = \frac{3}{x^4}, x>1. \]</span></p>
<ul>
<li>We can show analytically that the integral of <span class="math inline">\(f_X(x)\)</span> over the real line equals <span class="math inline">\(1\)</span>.</li>
</ul>
<p><span class="math display">\[\begin{align}
\int f_X(x) \mathrm{d}x =& \int_{1}^{\infty} \frac{3}{x^4} \mathrm{d}x \\
=& \lim_{t \rightarrow \infty} \int_{1}^{t} \frac{3}{x^4} \mathrm{d}x \\
=& \lim_{t \rightarrow \infty} -x^{-3} \rvert_{x=1}^t \\
=& -\left(\lim_{t \rightarrow \infty}\frac{1}{t^3} - 1\right) \\
=& 1
\end{align}\]</span></p>
<ul>
<li>The expectation of <span class="math inline">\(X\)</span> can be computed as follows:</li>
</ul>
<p><span class="math display">\[\begin{align}
E(X) = \int x \cdot f_X(x) \mathrm{d}x =& \int_{1}^{\infty} x \cdot \frac{3}{x^4} \mathrm{d}x \\
=& - \frac{3}{2} x^{-2} \rvert_{x=1}^{\infty} \\
=& -\frac{3}{2} \left( \lim_{t \rightarrow \infty} \frac{1}{t^2} - 1 \right) \\
=& \frac{3}{2}
\end{align}\]</span></p>
<ul>
<li>Note that the variance of <span class="math inline">\(X\)</span> can be expressed as <span class="math inline">\(\text{Var}(X) = E(X^2) - E(X)^2\)</span>. Since <span class="math inline">\(E(X)\)</span> has been computed in the previous step, we seek <span class="math inline">\(E(X^2)\)</span>:</li>
</ul>
<p><span class="math display">\[\begin{align}
E(X^2)= \int x^2 \cdot f_X(x) \mathrm{d}x =& \int_{1}^{\infty} x^2 \cdot \frac{3}{x^4} \mathrm{d}x \\
=& -3 x^{-1} \rvert_{x=1}^{\infty} \\
=& -3 \left( \lim_{t \rightarrow \infty} \frac{1}{t} - 1 \right) \\
=& 3
\end{align}\]</span></p>
<p>So we have shown that the area under the curve equals one, that the expectation is <span class="math inline">\(E(X)=\frac{3}{2}\)</span> and we found the variance to be <span class="math inline">\(\text{Var}(X) = \frac{3}{4}\)</span>. However, this was tedious and, as we shall see, an analytic approach is not applicable for some probability density functions, e.g., if integrals have no closed form solutions.</p>
<p>Luckily, <tt>R</tt> also enables us to easily find the results derived above. The tool we use for this is the function <tt>integrate()</tt>. First, we have to define the functions we want to calculate integrals for as <tt>R</tt> functions, i.e., the PDF <span class="math inline">\(f_X(x)\)</span> as well as the expressions <span class="math inline">\(x\cdot f_X(x)\)</span> and <span class="math inline">\(x^2\cdot f_X(x)\)</span>.</p>
<pre class="sourceCode r"><code class="sourceCode r"><span class="co"># define functions</span>
f <-<span class="st"> </span><span class="cf">function</span>(x) <span class="dv">3</span> <span class="op">/</span><span class="st"> </span>x<span class="op">^</span><span class="dv">4</span>
g <-<span class="st"> </span><span class="cf">function</span>(x) x <span class="op">*</span><span class="st"> </span><span class="kw">f</span>(x)
h <-<span class="st"> </span><span class="cf">function</span>(x) x<span class="op">^</span><span class="dv">2</span> <span class="op">*</span><span class="st"> </span><span class="kw">f</span>(x)</code></pre>
<p>Next, we use <tt>integrate()</tt> and set lower and upper limits of integration to <span class="math inline">\(1\)</span> and <span class="math inline">\(\infty\)</span> using arguments <tt>lower</tt> and <tt>upper</tt>. By default, <tt>integrate()</tt> prints the result along with an estimate of the approximation error to the console. However, the outcome is not a numeric value one can readily do further calculation with. In order to get only a numeric value of the integral, we need to use the <tt>$</tt> operator in conjunction with <tt>value</tt>. The <tt>$</tt> operator is used to extract elements by name from an object of type <tt>list</tt>.</p>
<pre class="sourceCode r"><code class="sourceCode r"><span class="co"># compute area under the density curve</span>
area <-<span class="st"> </span><span class="kw">integrate</span>(f,
<span class="dt">lower =</span> <span class="dv">1</span>,
<span class="dt">upper =</span> <span class="ot">Inf</span>)<span class="op">$</span>value
area </code></pre>
<pre><code>## [1] 1</code></pre>
<pre class="sourceCode r"><code class="sourceCode r"><span class="co"># compute E(X)</span>
EX <-<span class="st"> </span><span class="kw">integrate</span>(g,
<span class="dt">lower =</span> <span class="dv">1</span>,
<span class="dt">upper =</span> <span class="ot">Inf</span>)<span class="op">$</span>value
EX</code></pre>
<pre><code>## [1] 1.5</code></pre>
<pre class="sourceCode r"><code class="sourceCode r"><span class="co"># compute Var(X)</span>
VarX <-<span class="st"> </span><span class="kw">integrate</span>(h,
<span class="dt">lower =</span> <span class="dv">1</span>,
<span class="dt">upper =</span> <span class="ot">Inf</span>)<span class="op">$</span>value <span class="op">-</span><span class="st"> </span>EX<span class="op">^</span><span class="dv">2</span>
VarX</code></pre>
<pre><code>## [1] 0.75</code></pre>
<p>Although there is a wide variety of distributions, the ones most often
encountered in econometrics are the normal, chi-squared, Student <span class="math inline">\(t\)</span> and <span class="math inline">\(F\)</span>
distributions. Therefore we will discuss some core <tt>R</tt> functions that allow to do
calculations involving densities, probabilities and quantiles of these
distributions.</p>
<p>Every probability distribution that <tt>R</tt> handles has four basic functions whose names consist of a prefix followed by a root name. As an example, take the normal distribution. The root name of all four functions associated with the normal distribution is <tt>norm</tt>. The four prefixes are</p>
<ul>
<li><tt>d</tt> for “density” - probability function / probability density function</li>
<li><tt>p</tt> for “probability” - cumulative distribution function</li>
<li><tt>q</tt> for “quantile” - quantile function (inverse cumulative distribution function)</li>
<li><tt>r</tt> for “random” - random number generator</li>
</ul>
<p>Thus, for the normal distribution we have the <tt>R</tt> functions <tt>dnorm()</tt>, <tt>pnorm()</tt>, <tt>qnorm()</tt> and <tt>rnorm()</tt>.</p>
</div>
<div id="the-normal-distribution" class="section level3 unnumbered">
<h3>The Normal Distribution</h3>
<p>The probably most important probability distribution considered here is the normal
distribution. This is not least due to the special role of the standard normal distribution and the Central Limit Theorem which is to be treated shortly. Normal distributions are symmetric and bell-shaped. A normal distribution is characterized by its mean <span class="math inline">\(\mu\)</span> and its standard deviation <span class="math inline">\(\sigma\)</span>, concisely expressed by
<span class="math inline">\(\mathcal{N}(\mu,\sigma^2)\)</span>. The normal distribution has the PDF</p>
<p><span class="math display">\[\begin{align}
f(x) = \frac{1}{\sqrt{2 \pi} \sigma} \exp{-(x - \mu)^2/(2 \sigma^2)}.
\end{align}\]</span></p>
<p>For the standard normal distribution we have <span class="math inline">\(\mu=0\)</span> and <span class="math inline">\(\sigma=1\)</span>. Standard normal variates are often denoted by <span class="math inline">\(Z\)</span>. Usually, the standard normal PDF is denoted by <span class="math inline">\(\phi\)</span> and the standard normal CDF is denoted by <span class="math inline">\(\Phi\)</span>. Hence,
<span class="math display">\[ \phi(c) = \Phi'(c) \ \ , \ \ \Phi(c) = P(Z \leq c) \ \ , \ \ Z \sim \mathcal{N}(0,1).\]</span> In <tt>R</tt>, we can conveniently obtain densities of normal distributions using
the function <tt>dnorm()</tt>. Let us draw a plot of the standard normal density function
using <tt>curve()</tt> in conjunction with <tt>dnorm()</tt>.</p>
<div class="unfolded">
<pre class="sourceCode r"><code class="sourceCode r"><span class="co"># draw a plot of the N(0,1) PDF</span>
<span class="kw">curve</span>(<span class="kw">dnorm</span>(x),
<span class="dt">xlim =</span> <span class="kw">c</span>(<span class="op">-</span><span class="fl">3.5</span>, <span class="fl">3.5</span>),
<span class="dt">ylab =</span> <span class="st">"Density"</span>,
<span class="dt">main =</span> <span class="st">"Standard Normal Density Function"</span>) </code></pre>
<p><img src="ITER_files/figure-html/unnamed-chunk-36-1.png" width="80%" style="display: block; margin: auto;" /></p>
</div>
<p>We can obtain the density at different positions by passing a vector to <tt>dnorm()</tt>.</p>
<pre class="sourceCode r"><code class="sourceCode r"><span class="co"># compute denstiy at x=-1.96, x=0 and x=1.96</span>
<span class="kw">dnorm</span>(<span class="dt">x =</span> <span class="kw">c</span>(<span class="op">-</span><span class="fl">1.96</span>, <span class="dv">0</span>, <span class="fl">1.96</span>))</code></pre>
<pre><code>## [1] 0.05844094 0.39894228 0.05844094</code></pre>
<p>Similar to the PDF, we can plot the standard normal CDF using <tt>curve()</tt>. We could use <tt>dnorm()</tt> for this but it is much more convenient to rely on <tt>pnorm()</tt>.</p>
<div class="unfolded">
<pre class="sourceCode r"><code class="sourceCode r"><span class="co"># plot the standard normal CDF</span>
<span class="kw">curve</span>(<span class="kw">pnorm</span>(x),
<span class="dt">xlim =</span> <span class="kw">c</span>(<span class="op">-</span><span class="fl">3.5</span>, <span class="fl">3.5</span>),
<span class="dt">ylab =</span> <span class="st">"Density"</span>,
<span class="dt">main =</span> <span class="st">"Standard Normal Cumulative Distribution Function"</span>)</code></pre>
<p><img src="ITER_files/figure-html/unnamed-chunk-38-1.png" width="80%" style="display: block; margin: auto;" /></p>
</div>
<p>We can also use <tt>R</tt> to calculate the probability of events associated with a standard normal variate.</p>
<p>Let us say we are interested in <span class="math inline">\(P(Z \leq 1.337)\)</span>. For some continuous random variable <span class="math inline">\(Z\)</span> on <span class="math inline">\([-\infty,\infty]\)</span> with density <span class="math inline">\(g(x)\)</span> we would have to determine <span class="math inline">\(G(x)\)</span>, the anti-derivative of <span class="math inline">\(g(x)\)</span> so that</p>
<p><span class="math display">\[ P(Z \leq 1.337 ) = G(1.337) = \int_{-\infty}^{1.337} g(x) \mathrm{d}x. \]</span></p>
<p>If <span class="math inline">\(Z \sim \mathcal{N}(0,1)\)</span>, we have <span class="math inline">\(g(x)=\phi(x)\)</span>. There is no analytic solution to the integral above. Fortunately, <tt>R</tt> offers good approximations. The first approach makes use of the function <tt>integrate()</tt> which allows to solve one-dimensional integration problems using a numerical method. For this, we first define the function we want to compute the integral of as an <tt>R</tt> function <tt>f</tt>. In our example, <tt>f</tt> is the standard normal density function and hence takes a single argument <tt>x</tt>. Following the definition of <span class="math inline">\(\phi(x)\)</span> we define <tt>f</tt> as</p>
<pre class="sourceCode r"><code class="sourceCode r"><span class="co"># define the standard normal PDF as an R function</span>
f <-<span class="st"> </span><span class="cf">function</span>(x) {
<span class="dv">1</span><span class="op">/</span>(<span class="kw">sqrt</span>(<span class="dv">2</span> <span class="op">*</span><span class="st"> </span>pi)) <span class="op">*</span><span class="st"> </span><span class="kw">exp</span>(<span class="op">-</span><span class="fl">0.5</span> <span class="op">*</span><span class="st"> </span>x<span class="op">^</span><span class="dv">2</span>)
}</code></pre>
<p>Let us check if this function computes standard normal densities by passing a vector.</p>
<pre class="sourceCode r"><code class="sourceCode r"><span class="co"># define a vector of reals</span>
quants <-<span class="st"> </span><span class="kw">c</span>(<span class="op">-</span><span class="fl">1.96</span>, <span class="dv">0</span>, <span class="fl">1.96</span>)
<span class="co"># compute densities</span>
<span class="kw">f</span>(quants)</code></pre>
<pre><code>## [1] 0.05844094 0.39894228 0.05844094</code></pre>
<pre class="sourceCode r"><code class="sourceCode r"><span class="co"># compare to the results produced by 'dnorm()'</span>
<span class="kw">f</span>(quants) <span class="op">==</span><span class="st"> </span><span class="kw">dnorm</span>(quants)</code></pre>
<pre><code>## [1] TRUE TRUE TRUE</code></pre>
<p>The results produced by <tt>f()</tt> are indeed equivalent to those given by <tt>dnorm()</tt>.</p>
<p>Next, we call <tt>integrate()</tt> on <tt>f()</tt> and specify the arguments <tt>lower</tt> and <tt>upper</tt>, the lower and upper limits of integration.</p>
<pre class="sourceCode r"><code class="sourceCode r"><span class="co"># integrate f()</span>
<span class="kw">integrate</span>(f,
<span class="dt">lower =</span> <span class="op">-</span><span class="ot">Inf</span>,
<span class="dt">upper =</span> <span class="fl">1.337</span>)</code></pre>
<pre><code>## 0.9093887 with absolute error < 1.7e-07</code></pre>
<p>We find that the probability of observing <span class="math inline">\(Z \leq 1.337\)</span> is about <span class="math inline">\(90.94\%\)</span>.</p>
<p>A second and much more convenient way is to use the function <tt>pnorm()</tt>, the standard normal cumulative distribution function.</p>
<pre class="sourceCode r"><code class="sourceCode r"><span class="co"># compute the probability using pnorm()</span>
<span class="kw">pnorm</span>(<span class="fl">1.337</span>)</code></pre>
<pre><code>## [1] 0.9093887</code></pre>
<p>The result matches the outcome of the approach using <tt>integrate()</tt>.</p>
<p>Let us discuss some further examples:</p>
<p>A commonly known result is that <span class="math inline">\(95\%\)</span> probability mass of a standard normal lies in the interval <span class="math inline">\([-1.96, 1.96]\)</span>, that is, in a distance of about <span class="math inline">\(2\)</span> standard deviations to the mean. We can easily confirm this by calculating <span class="math display">\[ P(-1.96 \leq Z \leq 1.96) = 1-2\times P(Z \leq -1.96) \]</span> due to symmetry of the standard normal PDF. Thanks to <tt>R</tt>, we can abandon the table of the standard normal CDF found in many other textbooks and instead solve this fast by using <tt>pnorm()</tt>.</p>
<pre class="sourceCode r"><code class="sourceCode r"><span class="co"># compute the probability</span>
<span class="dv">1</span> <span class="op">-</span><span class="st"> </span><span class="dv">2</span> <span class="op">*</span><span class="st"> </span>(<span class="kw">pnorm</span>(<span class="op">-</span><span class="fl">1.96</span>)) </code></pre>
<pre><code>## [1] 0.9500042</code></pre>
<p>To make statements about the probability of observing outcomes of <span class="math inline">\(Y\)</span> in some specific range it is more convenient when we standardize first as shown in Key Concept 2.4.</p>
<div id="KC2.4" class="keyconcept">
<h3 class="right">
Key Concept 2.4
</h3>
<h3 class="left">
Computing Probabilities Involving Normal Random Variables
</h3>
<p>
<p>Suppose <span class="math inline">\(Y\)</span> is normally distributed with mean <span class="math inline">\(\mu\)</span> and variance <span class="math inline">\(\sigma^2\)</span>: <span class="math display">\[Y
\sim \mathcal{N}(\mu, \sigma^2)\]</span> Then <span class="math inline">\(Y\)</span> is standardized by subtracting its mean and
dividing by its standard deviation: <span class="math display">\[ Z = \frac{Y -\mu}{\sigma} \]</span> Let <span class="math inline">\(c_1\)</span>
and <span class="math inline">\(c_2\)</span> denote two numbers whereby <span class="math inline">\(c_1 < c_2\)</span> and further <span class="math inline">\(d_1 = (c_1 - \mu) / \sigma\)</span> and <span class="math inline">\(d_2 = (c_2 - \mu)/\sigma\)</span>. Then</p>
<p><span class="math display">\[\begin{align*}
P(Y \leq c_2) =& \, P(Z \leq d_2) = \Phi(d_2), \\
P(Y \geq c_1) =& \, P(Z \geq d_1) = 1 - \Phi(d_1), \\
P(c_1 \leq Y \leq c_2) =& \, P(d_1 \leq Z \leq d_2) = \Phi(d_2) - \Phi(d_1).
\end{align*}\]</span></p>
</p>
</div>
<p>Now consider a random variable <span class="math inline">\(Y\)</span> with <span class="math inline">\(Y \sim \mathcal{N}(5, 25)\)</span>. <tt>R</tt> functions that handle the normal distribution can perform the standardization. If we are interested in <span class="math inline">\(P(3 \leq Y \leq 4)\)</span> we can use <tt>pnorm()</tt> and adjust for a mean and/or a standard deviation that deviate from <span class="math inline">\(\mu=0\)</span> and <span class="math inline">\(\sigma = 1\)</span> by specifying the arguments <tt>mean</tt> and <tt>sd</tt> accordingly. <strong>Attention</strong>: the argument <tt>sd</tt> requires the standard deviation, not the variance!</p>
<pre class="sourceCode r"><code class="sourceCode r"><span class="kw">pnorm</span>(<span class="dv">4</span>, <span class="dt">mean =</span> <span class="dv">5</span>, <span class="dt">sd =</span> <span class="dv">5</span>) <span class="op">-</span><span class="st"> </span><span class="kw">pnorm</span>(<span class="dv">3</span>, <span class="dt">mean =</span> <span class="dv">5</span>, <span class="dt">sd =</span> <span class="dv">5</span>) </code></pre>
<pre><code>## [1] 0.07616203</code></pre>
<p>An extension of the normal distribution in a univariate setting is the multivariate normal distribution. The joint PDF of two random normal variables <span class="math inline">\(X\)</span> and <span class="math inline">\(Y\)</span> is given by</p>
<p><span class="math display" id="eq:bivnorm">\[\begin{align}
\begin{split}
g_{X,Y}(x,y) =& \, \frac{1}{2\pi\sigma_X\sigma_Y\sqrt{1-\rho_{XY}^2}} \\
\cdot & \, \exp \left\{ \frac{1}{-2(1-\rho_{XY}^2)} \left[ \left( \frac{x-\mu_x}{\sigma_X} \right)^2 - 2\rho_{XY}\left( \frac{x-\mu_X}{\sigma_X} \right)\left( \frac{y-\mu_Y}{\sigma_Y} \right) + \left( \frac{y-\mu_Y}{\sigma_Y} \right)^2 \right] \right\}.
\end{split} \tag{2.1}
\end{align}\]</span></p>
<p>Equation <a href="2-1-random-variables-and-probability-distributions.html#eq:bivnorm">(2.1)</a> contains the bivariate normal PDF. It is somewhat hard to gain insights from this complicated expression. Instead, let us consider the special case where <span class="math inline">\(X\)</span> and <span class="math inline">\(Y\)</span> are uncorrelated standard normal random variables with densities <span class="math inline">\(f_X(x)\)</span> and <span class="math inline">\(f_Y(y)\)</span> with joint normal distribution. We then have the parameters <span class="math inline">\(\sigma_X = \sigma_Y = 1\)</span>, <span class="math inline">\(\mu_X=\mu_Y=0\)</span> (due to marginal standard normality) and <span class="math inline">\(\rho_{XY}=0\)</span> (due to independence). The joint density of <span class="math inline">\(X\)</span> and <span class="math inline">\(Y\)</span> then becomes</p>
<p><span class="math display">\[ g_{X,Y}(x,y) = f_X(x) f_Y(y) = \frac{1}{2\pi} \cdot \exp \left\{ -\frac{1}{2} \left[x^2 + y^2 \right] \right\}, \tag{2.2} \]</span></p>
<p>the PDF of the bivariate standard normal distribution. The widget below provides an interactive three-dimensional plot of (<a href="#mjx-eqn-2.2">2.2</a>).</p>
<center>
<iframe scrolling="no" seamless="seamless" style="border:none" src="https://plot.ly/~mca_unidue/22.embed?width=550&height=550?showlink=false/800/1200" width="600" height="400">
</iframe>
</center>
<p>By moving the cursor over the plot you can see that the density is rotationally invariant, i.e., the density at <span class="math inline">\((a, b)\)</span> solely depends on the distance of <span class="math inline">\((a, b)\)</span> to the origin: geometrically, regions of equal density are edges of concentric circles in the <span class="math inline">\(XY\)</span>-plane, centered at <span class="math inline">\((\mu_X = 0, \mu_Y = 0)\)</span>.</p>
<p>The normal distribution has some remarkable characteristics. For example, for two jointly normally distribued variables <span class="math inline">\(X\)</span> and <span class="math inline">\(Y\)</span>, the conditional expectation function is linear: one can show that <span class="math display">\[ E(Y\vert X) = E(Y) + \rho \frac{\sigma_Y}{\sigma_X} (X - E(X)). \]</span> The application below shows standard bivariate normally distributed sample data along with the conditional expectation function <span class="math inline">\(E(Y\vert X)\)</span> and the marginal densities of <span class="math inline">\(X\)</span> and <span class="math inline">\(Y\)</span>. All elements adjust accordingly as you vary the parameters.</p>
<center>
<iframe height="880" width="770" frameborder="0" scrolling="no" src="bivariatenormalv4.html">
</iframe>
</center>
<p><a name="chisquare"></a></p>
</div>
<div id="the-chi-squared-distribution" class="section level3 unnumbered">
<h3>The Chi-Squared Distribution</h3>
<p>The chi-squared distribution is another distribution relevant in econometrics. It is often needed when testing special types of hypotheses frequently encountered when dealing with regression models.</p>
<p>The sum of <span class="math inline">\(M\)</span> squared independent standard normal distributed random variables follows a chi-squared distribution with <span class="math inline">\(M\)</span> degrees of freedom:</p>
<p><span class="math display" id="eq:chisq">\[\begin{align*}
Z_1^2 + \dots + Z_M^2 = \sum_{m=1}^M Z_m^2 \sim \chi^2_M \ \ \text{with} \ \ Z_m \overset{i.i.d.}{\sim} \mathcal{N}(0,1) \tag{2.2}
\end{align*}\]</span></p>
<p>A <span class="math inline">\(\chi^2\)</span> distributed random variable with <span class="math inline">\(M\)</span> degrees of freedom has expectation <span class="math inline">\(M\)</span>, mode at <span class="math inline">\(M-2\)</span> for <span class="math inline">\(M \geq 2\)</span> and variance <span class="math inline">\(2 \cdot M\)</span>. For example, for</p>
<p><span class="math display">\[ Z_1,Z_2,Z_3 \overset{i.i.d.}{\sim} \mathcal{N}(0,1) \]</span></p>
<p>it holds that</p>
<p><span class="math display">\[ Z_1^2+Z_2^2+Z_3^3 \sim \chi^2_3. \tag{2.3} \]</span>
Using the code below, we can display the PDF and the CDF of a <span class="math inline">\(\chi^2_3\)</span> random variable in a single plot. This is achieved by setting the argument <tt>add = TRUE</tt> in the second call of <tt>curve()</tt>. Further we adjust limits of both axes using <tt>xlim</tt> and <tt>ylim</tt> and choose different colors to make both functions better distinguishable. The plot is completed by adding a legend with help of <tt>legend()</tt>.</p>
<div class="unfolded">
<pre class="sourceCode r"><code class="sourceCode r"><span class="co"># plot the PDF</span>
<span class="kw">curve</span>(<span class="kw">dchisq</span>(x, <span class="dt">df =</span> <span class="dv">3</span>),
<span class="dt">xlim =</span> <span class="kw">c</span>(<span class="dv">0</span>, <span class="dv">10</span>),
<span class="dt">ylim =</span> <span class="kw">c</span>(<span class="dv">0</span>, <span class="dv">1</span>),
<span class="dt">col =</span> <span class="st">"blue"</span>,
<span class="dt">ylab =</span> <span class="st">""</span>,
<span class="dt">main =</span> <span class="st">"p.d.f. and c.d.f of Chi-Squared Distribution, M = 3"</span>)
<span class="co"># add the CDF to the plot</span>
<span class="kw">curve</span>(<span class="kw">pchisq</span>(x, <span class="dt">df =</span> <span class="dv">3</span>),
<span class="dt">xlim =</span> <span class="kw">c</span>(<span class="dv">0</span>, <span class="dv">10</span>),
<span class="dt">add =</span> <span class="ot">TRUE</span>,
<span class="dt">col =</span> <span class="st">"red"</span>)
<span class="co"># add a legend to the plot</span>
<span class="kw">legend</span>(<span class="st">"topleft"</span>,
<span class="kw">c</span>(<span class="st">"PDF"</span>, <span class="st">"CDF"</span>),
<span class="dt">col =</span> <span class="kw">c</span>(<span class="st">"blue"</span>, <span class="st">"red"</span>),
<span class="dt">lty =</span> <span class="kw">c</span>(<span class="dv">1</span>, <span class="dv">1</span>))</code></pre>
<p><img src="ITER_files/figure-html/unnamed-chunk-49-1.png" width="80%" style="display: block; margin: auto;" /></p>
</div>
<p>Since the outcomes of a <span class="math inline">\(\chi^2_M\)</span> distributed random variable are always positive, the support of the related PDF and CDF is <span class="math inline">\(\mathbb{R}_{\geq0}\)</span>.</p>
<p>As expectation and variance depend (solely!) on the degrees of freedom, the distribution’s shape changes drastically if we vary the number of squared standard normals that are summed up. This relation is often depicted by overlaying densities for different <span class="math inline">\(M\)</span>, see the <a href="https://en.wikipedia.org/wiki/Chi-squared_distribution">Wikipedia Article</a>.</p>
<p>We reproduce this here by plotting the density of the <span class="math inline">\(\chi_1^2\)</span> distribution on the interval <span class="math inline">\([0,15]\)</span> with <tt>curve()</tt>. In the next step, we loop over degrees of freedom <span class="math inline">\(M=2,...,7\)</span> and add a density curve for each <span class="math inline">\(M\)</span> to the plot. We also adjust the line color for each iteration of the loop by setting <tt>col = M</tt>. At last, we add a legend that displays degrees of freedom and the associated colors.</p>
<div class="unfolded">
<pre class="sourceCode r"><code class="sourceCode r"><span class="co"># plot the density for M=1</span>
<span class="kw">curve</span>(<span class="kw">dchisq</span>(x, <span class="dt">df =</span> <span class="dv">1</span>),
<span class="dt">xlim =</span> <span class="kw">c</span>(<span class="dv">0</span>, <span class="dv">15</span>),
<span class="dt">xlab =</span> <span class="st">"x"</span>,
<span class="dt">ylab =</span> <span class="st">"Density"</span>,
<span class="dt">main =</span> <span class="st">"Chi-Square Distributed Random Variables"</span>)
<span class="co"># add densities for M=2,...,7 to the plot using a 'for()' loop </span>
<span class="cf">for</span> (M <span class="cf">in</span> <span class="dv">2</span><span class="op">:</span><span class="dv">7</span>) {
<span class="kw">curve</span>(<span class="kw">dchisq</span>(x, <span class="dt">df =</span> M),
<span class="dt">xlim =</span> <span class="kw">c</span>(<span class="dv">0</span>, <span class="dv">15</span>),
<span class="dt">add =</span> T,
<span class="dt">col =</span> M)
}
<span class="co"># add a legend</span>
<span class="kw">legend</span>(<span class="st">"topright"</span>,
<span class="kw">as.character</span>(<span class="dv">1</span><span class="op">:</span><span class="dv">7</span>),
<span class="dt">col =</span> <span class="dv">1</span><span class="op">:</span><span class="dv">7</span> ,
<span class="dt">lty =</span> <span class="dv">1</span>,
<span class="dt">title =</span> <span class="st">"D.F."</span>)</code></pre>
<p><img src="ITER_files/figure-html/unnamed-chunk-50-1.png" width="80%" style="display: block; margin: auto;" /></p>
</div>
<p>Increasing the degrees of freedom shifts the distribution to the right (the mode becomes larger) and increases the dispersion (the distribution’s variance grows).</p>
</div>
<div id="thetdist" class="section level3 unnumbered">
<h3>The Student t Distribution</h3>
<p><a name="tdist"></a></p>
<p>Let <span class="math inline">\(Z\)</span> be a standard normal variate, <span class="math inline">\(W\)</span> a <span class="math inline">\(\chi^2_M\)</span> random variable and further assume that <span class="math inline">\(Z\)</span> and <span class="math inline">\(W\)</span> are independent. Then it holds that</p>
<p><span class="math display">\[ \frac{Z}{\sqrt{W/M}} =:X \sim t_M \]</span>
and <span class="math inline">\(X\)</span> follows a <em>Student <span class="math inline">\(t\)</span> distribution</em> (or simply <span class="math inline">\(t\)</span> distribution) with <span class="math inline">\(M\)</span> degrees of freedom.</p>
<p>Similar to the <span class="math inline">\(\chi^2_M\)</span> distribution, the shape of a <span class="math inline">\(t_M\)</span> distribution depends on <span class="math inline">\(M\)</span>. <span class="math inline">\(t\)</span> distributions are symmetric, bell-shaped and look similar to a normal distribution, especially when <span class="math inline">\(M\)</span> is large. This is not a coincidence: for a sufficiently large <span class="math inline">\(M\)</span>, the <span class="math inline">\(t_M\)</span> distribution can be approximated by the standard normal distribution. This approximation works reasonably well for <span class="math inline">\(M\geq 30\)</span>. As we will illustrate later by means of a small simulation study, the <span class="math inline">\(t_{\infty}\)</span> distribution <em>is</em> the standard normal distribution.</p>
<p>A <span class="math inline">\(t_M\)</span> distributed random variable <span class="math inline">\(X\)</span> has an expectation if <span class="math inline">\(M>1\)</span> and it has a variance if <span class="math inline">\(M>2\)</span>.</p>
<p><span class="math display">\[\begin{align}
E(X) =& 0, \ M>1 \\
\text{Var}(X) =& \frac{M}{M-2}, \ M>2
\end{align}\]</span></p>
<p>Let us plot some <span class="math inline">\(t\)</span> distributions with different <span class="math inline">\(M\)</span> and compare them to the standard normal distribution.</p>
<div class="unfolded">
<pre class="sourceCode r"><code class="sourceCode r"><span class="co"># plot the standard normal density</span>
<span class="kw">curve</span>(<span class="kw">dnorm</span>(x),
<span class="dt">xlim =</span> <span class="kw">c</span>(<span class="op">-</span><span class="dv">4</span>, <span class="dv">4</span>),
<span class="dt">xlab =</span> <span class="st">"x"</span>,
<span class="dt">lty =</span> <span class="dv">2</span>,
<span class="dt">ylab =</span> <span class="st">"Density"</span>,
<span class="dt">main =</span> <span class="st">"Densities of t Distributions"</span>)
<span class="co"># plot the t density for M=2</span>
<span class="kw">curve</span>(<span class="kw">dt</span>(x, <span class="dt">df =</span> <span class="dv">2</span>),
<span class="dt">xlim =</span> <span class="kw">c</span>(<span class="op">-</span><span class="dv">4</span>, <span class="dv">4</span>),
<span class="dt">col =</span> <span class="dv">2</span>,
<span class="dt">add =</span> T)
<span class="co"># plot the t density for M=4</span>
<span class="kw">curve</span>(<span class="kw">dt</span>(x, <span class="dt">df =</span> <span class="dv">4</span>),
<span class="dt">xlim =</span> <span class="kw">c</span>(<span class="op">-</span><span class="dv">4</span>, <span class="dv">4</span>),
<span class="dt">col =</span> <span class="dv">3</span>,
<span class="dt">add =</span> T)
<span class="co"># plot the t density for M=25</span>
<span class="kw">curve</span>(<span class="kw">dt</span>(x, <span class="dt">df =</span> <span class="dv">25</span>),
<span class="dt">xlim =</span> <span class="kw">c</span>(<span class="op">-</span><span class="dv">4</span>, <span class="dv">4</span>),
<span class="dt">col =</span> <span class="dv">4</span>,
<span class="dt">add =</span> T)
<span class="co"># add a legend</span>
<span class="kw">legend</span>(<span class="st">"topright"</span>,
<span class="kw">c</span>(<span class="st">"N(0, 1)"</span>, <span class="st">"M=2"</span>, <span class="st">"M=4"</span>, <span class="st">"M=25"</span>),
<span class="dt">col =</span> <span class="dv">1</span><span class="op">:</span><span class="dv">4</span>,
<span class="dt">lty =</span> <span class="kw">c</span>(<span class="dv">2</span>, <span class="dv">1</span>, <span class="dv">1</span>, <span class="dv">1</span>))</code></pre>
<p><img src="ITER_files/figure-html/unnamed-chunk-51-1.png" width="80%" style="display: block; margin: auto;" /></p>
</div>
<p>The plot illustrates what has been said in the previous paragraph: as the degrees of freedom increase, the shape of the <span class="math inline">\(t\)</span> distribution comes closer to that of a standard normal bell curve. Already for <span class="math inline">\(M=25\)</span> we find little difference to the standard normal density. If <span class="math inline">\(M\)</span> is small, we find the distribution to have heavier tails than a standard normal, i.e., it has a “fatter” bell shape.</p>
</div>
<div id="the-f-distribution" class="section level3 unnumbered">
<h3>The F Distribution</h3>
<p>Another ratio of random variables important to econometricians is the ratio of two independent <span class="math inline">\(\chi^2\)</span> distributed random variables that are divided by their degrees of freedom <span class="math inline">\(M\)</span> and <span class="math inline">\(n\)</span>. The quantity</p>
<p><span class="math display">\[ \frac{W/M}{V/n} \sim F_{M,n} \ \ \text{with} \ \ W \sim \chi^2_M \ \ , \ \ V \sim \chi^2_n \]</span>
follows an <span class="math inline">\(F\)</span> distribution with numerator degrees of freedom <span class="math inline">\(M\)</span> and denominator degrees of freedom <span class="math inline">\(n\)</span>, denoted <span class="math inline">\(F_{M,n}\)</span>. The distribution was first derived by George Snedecor but was named in honor of <a href="https://en.wikipedia.org/wiki/Ronald_Fisher">Sir Ronald Fisher</a>.</p>
<p>By definition, the support of both PDF and CDF of an <span class="math inline">\(F_{M,n}\)</span> distributed random variable is <span class="math inline">\(\mathbb{R}_{\geq0}\)</span>.</p>
<p>Say we have an <span class="math inline">\(F\)</span> distributed random variable <span class="math inline">\(Y\)</span> with numerator degrees of freedom <span class="math inline">\(3\)</span> and denominator degrees of freedom <span class="math inline">\(14\)</span> and are interested in <span class="math inline">\(P(Y \geq 2)\)</span>. This can be computed with help of the function <tt>pf()</tt>. By setting the argument <tt>lower.tail</tt> to <tt>TRUE</tt> we ensure that <tt>R</tt> computes <span class="math inline">\(1- P(Y \leq 2)\)</span>, i.e,the probability mass in the tail right of <span class="math inline">\(2\)</span>.</p>
<pre class="sourceCode r"><code class="sourceCode r"><span class="kw">pf</span>(<span class="dv">2</span>, <span class="dv">3</span>, <span class="dv">13</span>, <span class="dt">lower.tail =</span> F)</code></pre>
<pre><code>## [1] 0.1638271</code></pre>
<p>We can visualize this probability by drawing a line plot of the related density and adding a color shading with <tt>polygon()</tt>.</p>
<div class="unfolded">
<pre class="sourceCode r"><code class="sourceCode r"><span class="co"># define coordinate vectors for vertices of the polygon</span>