chapter 5 multiple regression analyst .ppt

materi multiple regression analysis

1. Economics 20 - Prof. Anderson 1
Multiple Regression Analysis
 y = 
0 + 
1x
1 + 
2x
2 + . . . 
kx
k + u
 3. Asymptotic Properties

2. Economics 20 - Prof. Anderson 2
Consistency
Under the Gauss-Markov assumptions OLS
is BLUE, but in other cases it won’t always
be possible to find unbiased estimators
In those cases, we may settle for estimators
that are consistent, meaning as n  ∞, the
distribution of the estimator collapses to the
parameter value

3. Economics 20 - Prof. Anderson 3
Sampling Distributions as n 

1
n
1
n
2
n
3
n
1
< n
2
< n
3

4. Economics 20 - Prof. Anderson 4
Consistency of OLS
Under the Gauss-Markov assumptions, the
OLS estimator is consistent (and unbiased)
Consistency can be proved for the simple
regression case in a manner similar to the
proof of unbiasedness
Will need to take probability limit (plim) to
establish consistency

5. Economics 20 - Prof. Anderson 5
Proving Consistency
    
     

0, because
,
ˆ
plim
ˆ
1
11111
2
11
1
11
1
1
2
11111







uxCov
xVaruxCov
xxnuxxn
xxyxx
iii
iii




6. Economics 20 - Prof. Anderson 6
A Weaker Assumption
For unbiasedness, we assumed a zero
conditional mean – E(u|x
1, x
2,…,x
k) = 0
For consistency, we can have the weaker
assumption of zero mean and zero
correlation – E(u) = 0 and Cov(x
j,u) = 0, for
j = 1, 2, …, k
Without this assumption, OLS will be
biased and inconsistent!

7. Economics 20 - Prof. Anderson 7
Deriving the Inconsistency
Just as we could derive the omitted variable bias
earlier, now we want to think about the
inconsistency, or asymptotic bias, in this case

121
21122
110
22110
, where
~
plim and,
thatso , :You think
:model True
xVarxxCov
vxu
uxy
vxxy









8. Economics 20 - Prof. Anderson 8
Asymptotic Bias (cont)
So, thinking about the direction of the
asymptotic bias is just like thinking about the
direction of bias for an omitted variable
Main difference is that asymptotic bias uses
the population variance and covariance,
while bias uses the sample counterparts
Remember, inconsistency is a large sample
problem – it doesn’t go away as add data

9. Economics 20 - Prof. Anderson 9
Large Sample Inference
Recall that under the CLM assumptions,
the sampling distributions are normal, so we
could derive t and F distributions for testing
This exact normality was due to assuming
the population error distribution was normal
This assumption of normal errors implied
that the distribution of y, given the x’s, was
normal as well

10. Economics 20 - Prof. Anderson 10
Large Sample Inference (cont)
Easy to come up with examples for which
this exact normality assumption will fail
Any clearly skewed variable, like wages,
arrests, savings, etc. can’t be normal, since
a normal distribution is symmetric
Normality assumption not needed to
conclude OLS is BLUE, only for inference

11. Economics 20 - Prof. Anderson 11
Central Limit Theorem
Based on the central limit theorem, we can show
that OLS estimators are asymptotically normal
Asymptotic Normality implies that P(Z<z)(z)
as n , or P(Z<z)  (z)
The central limit theorem states that the
standardized average of any population with mean
 and variance 
2
is asymptotically ~N(0,1), or
1,0~N
n
Y
Z
a
Y




12. Economics 20 - Prof. Anderson 12
Asymptotic Normality
   
 
  1,0Normal ~
ˆˆ
(iii)
ofestimator consistent a is ˆ (ii)
ˆplim where
,,0Normal ~
ˆ
(i)
s,assumption Markov-Gauss Under the
22
212
22
a
jjj
ijj
j
a
jj
se
rna
an









13. Economics 20 - Prof. Anderson 13
Asymptotic Normality (cont)
Because the t distribution approaches the
normal distribution for large df, we can also
say that
 
1
~
ˆˆ


kn
a
jjj
tse
Note that while we no longer need to
assume normality with a large sample, we
do still need homoskedasticity

14. Economics 20 - Prof. Anderson 14
Asymptotic Standard Errors
If u is not normally distributed, we sometimes
will refer to the standard error as an asymptotic
standard error, since


 ncse
RSST
se
jj
jj
j






ˆ
,
1
ˆ
ˆ
2
2
So, we can expect standard errors to shrink at a
rate proportional to the inverse of √n

15. Economics 20 - Prof. Anderson 15
Lagrange Multiplier statistic
Once we are using large samples and relying
on asymptotic normality for inference, we
can use more that t and F stats
The Lagrange multiplier or LM statistic is an
alternative for testing multiple exclusion
restrictions
Because the LM statistic uses an auxiliary
regression it’s sometimes called an nR
2
stat

16. Economics 20 - Prof. Anderson 16
LM Statistic (cont)
Suppose we have a standard model, y = 
0 + 
1x
1 +

2x
2 + . . . 
kx
k + u and our null hypothesis is
H
0
: 
k-q+1
= 0, , 
k
= 0
First, we just run the restricted model
reg thisfrom is where,
) variables the (i.e. ,...,,on
~
regress and ,
~
residuals, the takeNow
~~
...
~~
22
21
110
uu
k
qkqk
RnRLM
allxxxu
u
uxxy




17. Economics 20 - Prof. Anderson 17
LM Statistic (cont)
2
2
2
for value-p a calculatejust
or on,distributi a from , value,
critical a choosecan so ,~
q
q
q
a
c
LM



With a large sample, the result from an F test and
from an LM test should be similar
Unlike the F test and t test for one exclusion, the
LM test and F test will not be identical

18. Economics 20 - Prof. Anderson 18
Asymptotic Efficiency
Estimators besides OLS will be consistent
However, under the Gauss-Markov
assumptions, the OLS estimators will have
the smallest asymptotic variances
We say that OLS is asymptotically efficient
Important to remember our assumptions
though, if not homoskedastic, not true