A brief overview of the classical linear regression model

The elementary condition of linear regression

1. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 1
Chapter 3
A brief overview of the
classical linear regression model

2. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 2
Regression
•Regressionisprobablythesinglemostimportanttoolatthe
econometrician’sdisposal.
Butwhatisregressionanalysis?
•Itisconcernedwithdescribingandevaluatingtherelationshipbetween
agivenvariable(usuallycalledthedependentvariable)andoneor
moreothervariables(usuallyknownastheindependentvariable(s)).

3. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 3
Some Notation
•Denotethedependentvariablebyyandtheindependentvariable(s)byx
1,x
2,
...,x
kwheretherearekindependentvariables.
•Somealternativenamesfortheyandxvariables:
y x
dependentvariable independentvariables
regressand regressors
effectvariable causalvariables
explainedvariable explanatoryvariable
•Notethattherecanbemanyxvariablesbutwewilllimitourselvestothe
casewherethereisonlyonexvariabletostartwith.Inourset-up,thereis
onlyoneyvariable.

4. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 4
Regression is different from Correlation
•Ifwesayyandxarecorrelated,itmeansthatwearetreatingyandxin
acompletelysymmetricalway.
•Inregression,wetreatthedependentvariable(y)andtheindependent
variable(s)(x’s)verydifferently.Theyvariableisassumedtobe
randomor“stochastic”insomeway,i.e.tohaveaprobability
distribution.Thexvariablesare,however,assumedtohavefixed
(“non-stochastic”)valuesinrepeatedsamples.

5. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 5
Simple Regression
•Forsimplicity,sayk=1.Thisisthesituationwhereydependsononlyonex
variable.
•Examplesofthekindofrelationshipthatmaybeofinterestinclude:
–Howassetreturnsvarywiththeirlevelofmarketrisk
–Measuringthelong-termrelationshipbetweenstockpricesand
dividends.
–Constructinganoptimalhedgeratio

6. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 6
Simple Regression: An Example
•Supposethatwehavethefollowingdataontheexcessreturnsonafund
manager’sportfolio(“fundXXX”)togetherwiththeexcessreturnsona
marketindex:
•Wehavesomeintuitionthatthebetaonthisfundispositive,andwe
thereforewanttofindwhetherthereappearstobearelationshipbetween
xandygiventhedatathatwehave.Thefirststagewouldbetoforma
scatterplotofthetwovariables.Year, t Excess return
= rXXX,t – rft
Excess return on market index
= rmt - rft
1 17.8 13.7
2 39.0 23.2
3 12.8 6.9
4 24.2 16.8
5 17.2 12.3

7. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 7
Graph (Scatter Diagram)0
5
10
15
20
25
30
35
40
45
0 5 10 15 20 25
Excess return on market portfolio
Excess return on fund XXX

8. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 8
Finding a Line of Best Fit
•Wecanusethegeneralequationforastraightline,
y=a+bx
togetthelinethatbest“fits”thedata.
•However,thisequation(y=a+bx)iscompletelydeterministic.
•Isthisrealistic?No.Sowhatwedoistoaddarandomdisturbance
term,uintotheequation.
y
t= +x
t+u
t
where t= 1,2,3,4,5

9. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 9
Why do we include a Disturbance term?
•Thedisturbancetermcancaptureanumberoffeatures:
-Wealwaysleaveoutsomedeterminantsofy
t
-Theremaybeerrorsinthemeasurementofy
tthatcannotbe
modelled.
-Randomoutsideinfluencesony
twhichwecannotmodel

10. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 10
Determining the Regression Coefficients
•Sohowdowedeterminewhatandare?
•Chooseandsothatthe(vertical)distancesfromthedatapointstothe
fittedlinesareminimised(sothatthelinefitsthedataascloselyas
possible): y
x

11. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 11
Ordinary Least Squares
•Themostcommonmethodusedtofitalinetothedataisknownas
OLS(ordinaryleastsquares).
•Whatweactuallydoistakeeachdistanceandsquareit(i.e.takethe
areaofeachofthesquaresinthediagram)andminimisethetotalsum
ofthesquares(henceleastsquares).
•Tighteningupthenotation,let
y
tdenotetheactualdatapointt
denotethefittedvaluefromtheregressionline
denotetheresidual,y
t-tyˆ tyˆ tuˆ

12. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 12
Actual and Fitted Value
y





















ix x


i
y
iyˆ

i
uˆ

13. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 13
How OLS Works
•Somin. ,orminimise .Thisisknown
astheresidualsumofsquares.
•Butwhatwas?Itwasthedifferencebetweentheactualpointand
theline,y
t-.
•Sominimising isequivalenttominimising
withrespectto and.$ $
 2
5
2
4
2
3
2
2
2
1
ˆˆˆˆˆ uuuuu  tyˆ tuˆ 

5
1
2
ˆ
t
tu  
2
ˆ
tt
yy 
2
ˆ
t
u

14. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 14
Deriving the OLS Estimator
•But ,solet
•WanttominimiseLwithrespectto(w.r.t.)and,sodifferentiateL
w.r.t.and
(1)
(2)
•From(1),
•But and .$ $
 $ $
 tt
xy 
ˆˆˆ  
t
tt xy
L
0)
ˆˆ(2
ˆ


  
t
ttt xyx
L
0)
ˆˆ(2
ˆ


 0
ˆˆ0)
ˆˆ(    tt
t
tt xTyxy  yTy
t xTx
t   
t i
tttt xyyyL
22
)
ˆˆ()ˆ( 

15. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 15
Deriving the OLS Estimator (cont’d)
•Sowecanwrite or (3)
•From(2), (4)
•From(3), (5)
•Substituteinto(4)forfrom(5),$ 0
ˆˆ  xy  
t
ttt xyx 0)
ˆˆ(  xy
ˆˆ  
 




t
ttt
t
ttttt
t
ttt
xxTxyTyx
xxxxyyx
xxyyx
0
ˆˆ
0
ˆˆ
0)
ˆˆ
(
22
2


 0
ˆˆ  xTTyT 

16. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 16
Deriving the OLS Estimator (cont’d)
•Rearrangingfor,
•Sooverallwehave
•This method of finding the optimum is known as ordinary least squares.$
  
ttt
yxxyTxxT )(
ˆ
22
 xy
xTx
yxTyx
t
tt

ˆˆand
ˆ
22







17. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 17
What do We Use and For?
•In the CAPM example used above, plugging the 5 observations in to make up
the formulae given above would lead to the estimates
= -1.74 and = 1.64. We would write the fitted line as:
•Question: If an analyst tells you that she expects the market to yield a return
20% higher than the risk-free rate next year, what would you expect the return
on fund XXX to be?
•Solution: We can say that the expected value of y= “-1.74 + 1.64 * value of x”,
so plug x= 20 into the equation to get the expected value for y:$ $
 $ $
 06.312064.174.1ˆ 
iy tt xy 64.174.1ˆ 

18. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 18
Accuracy of Intercept Estimate
•Care needs to be exercised when considering the intercept estimate,
particularly if there are no or few observations close to the y-axis:
y
0 x

19. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 19
The Population and the Sample
•Thepopulationisthetotalcollectionofallobjectsorpeopletobestudied,
forexample,
•Interestedin Populationofinterest
predictingoutcome theentireelectorate
ofanelection
•Asampleisaselectionofjustsomeitemsfromthepopulation.
•Arandomsampleisasampleinwhicheachindividualiteminthe
populationisequallylikelytobedrawn.

20. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 20
The DGP and the PRF
•Thepopulationregressionfunction(PRF)isadescriptionofthemodelthat
isthoughttobegeneratingtheactualdataandthetruerelationship
betweenthevariables(i.e.thetruevaluesofand).
•ThePRFis
•TheSRFis
andwealsoknowthat .
•WeusetheSRFtoinferlikelyvaluesofthePRF.
•Wealsowanttoknowhow“good”ourestimatesofandare.tt
xy 
ˆˆˆ ttt uxy  ttt yyu ˆˆ 

21. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 21
Linearity
•InordertouseOLS,weneedamodelwhichislinearintheparameters(
and).Itdoesnotnecessarilyhavetobelinearinthevariables(yandx).
•Linearintheparametersmeansthattheparametersarenotmultiplied
together,divided,squaredorcubedetc.
•Somemodelscanbetransformedtolinearonesbyasuitablesubstitution
ormanipulation,e.g.theexponentialregressionmodel
•Thenlety
t=lnY
tandx
t=lnX
tttt uxy  ttt
u
tt uXYeXeY
t
 lnln 


22. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 22
Linear and Non-linear Models
•Thisisknownastheexponentialregressionmodel.Here,thecoefficients
canbeinterpretedaselasticities.
•Similarly,iftheorysuggeststhatyandxshouldbeinverselyrelated:
thentheregressioncanbeestimatedusingOLSbysubstituting
•Butsomemodelsareintrinsicallynon-linear,e.g.t
t
t u
x
y 

 t
t
x
z
1
 ttt
uxy 



23. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 23
Estimator or Estimate?
•Estimatorsare the formulae used to calculate the coefficients
•Estimatesare the actual numerical values for the coefficients.

24. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 24
The Assumptions Underlying the
Classical Linear Regression Model (CLRM)
•Themodelwhichwehaveusedisknownastheclassicallinearregressionmodel.
•Weobservedataforx
t,butsincey
talsodependsonu
t,wemustbespecificabout
howtheu
taregenerated.
•Weusuallymakethefollowingsetofassumptionsabouttheu
t’s(the
unobservableerrorterms):
•TechnicalNotation Interpretation
1.E(u
t)=0 Theerrorshavezeromean
2.Var(u
t)=
2
Thevarianceoftheerrorsisconstantandfinite
overallvaluesofx
t
3.Cov(u
i,u
j)=0 Theerrorsarestatisticallyindependentof
oneanother
4.Cov(u
t,x
t)=0 Norelationshipbetweentheerrorand
correspondingxvariate

25. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 25
The Assumptions Underlying the
CLRM Again
•Analternativeassumptionto4.,whichisslightlystronger,isthatthe
x
t’sarenon-stochasticorfixedinrepeatedsamples.
•Afifthassumptionisrequiredifwewanttomakeinferencesaboutthe
populationparameters(theactualand)fromthesampleparameters
(and)
•AdditionalAssumption
5.u
tisnormallydistributed$ $


26. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 26
Properties of the OLS Estimator
•Ifassumptions1.through4.hold,thentheestimatorsanddeterminedby
OLSareknownasBestLinearUnbiasedEstimators(BLUE).
Whatdoestheacronymstandfor?
•“Estimator”-isanestimatorofthetruevalueof.
•“Linear”-isalinearestimator
•“Unbiased”-Onaverage,theactualvalueoftheand’swillbeequalto
thetruevalues.
•“Best” -meansthattheOLSestimatorhasminimumvarianceamong
theclassoflinearunbiasedestimators.TheGauss-Markov
theoremprovesthattheOLSestimatorisbest.$ $
 $
 $
 $
 $
 $

27. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 27
Consistency/Unbiasedness/Efficiency
•Consistent
Theleastsquaresestimatorsandareconsistent.Thatis,theestimateswill
convergetotheirtruevaluesasthesamplesizeincreasestoinfinity.Need the
assumptionsE(x
tu
t)=0andVar(u
t)=
2
<toprovethis.Consistencyimpliesthat
•Unbiased
Theleastsquaresestimatesofandareunbiased.ThatisE()=andE()=
Thusonaveragetheestimatedvaluewillbeequaltothetruevalues.Toprove
thisalsorequirestheassumptionthatE(u
t)=0.Unbiasednessisastronger
conditionthanconsistency.
•Efficiency
Anestimatorofparameterissaidtobeefficientifitisunbiasedandnoother
unbiasedestimatorhasasmallervariance.Iftheestimatorisefficient,weare
minimisingtheprobabilitythatitisalongwayofffromthetruevalueof.$
 $
 $ $ $ $
 $
   00
ˆ
Prlim 


T

28. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 28
Precision and Standard Errors
•Anysetofregressionestimatesofandarespecifictothesampleusedin
theirestimation.
•Recallthattheestimatorsofandfromthesampleparameters(and)are
givenby
•Whatweneedissomemeasureofthereliabilityorprecisionoftheestimators
(and).Theprecisionoftheestimateisgivenbyitsstandarderror.Given
assumptions1-4above,thenthestandarderrorscanbeshowntobegivenby
wheresistheestimatedstandarddeviationoftheresiduals.$ $
 $
 $
 $ $ xy
xTx
yxTyx
t
tt

ˆˆand
ˆ
22





 












222
222
2
2
2
1
)(
1
)
ˆ
(
,
)(
)ˆ(
xTx
s
xx
sSE
xTxT
x
s
xxT
x
sSE
tt
t
t
t
t



29. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 29
Estimating the Variance of the Disturbance Term
•Thevarianceoftherandomvariableu
tisgivenby
Var(u
t)=E[(u
t)-E(u
t)]
2
whichreducesto
Var(u
t)=E(u
t
2
)
•Wecouldestimatethisusingtheaverageof:
•Unfortunatelythisisnotworkablesinceu
tisnotobservable.Wecanuse
thesamplecounterparttou
t,whichis:
Butthisestimatorisabiasedestimatorof
2
.2
tu 
221
tu
T
s 
22
ˆ
1
tu
T
s tuˆ

30. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 30
Estimating the Variance of the Disturbance Term
(cont’d)
•An unbiased estimator of is given by
where is the residual sum of squares and Tis the sample size.
SomeCommentsontheStandardErrorEstimators
1.BothSE()andSE()dependons
2
(ors).Thegreaterthevariances
2
,then
themoredispersedtheerrorsareabouttheirmeanvalueandthereforethe
moredispersedywillbeaboutitsmeanvalue.
2.Thesumofthesquaresofxabouttheirmeanappearsinbothformulae.
Thelargerthesumofsquares,thesmallerthecoefficientvariances.$ $
 2
ˆ
2



T
u
s
t 
2
ˆ
t
u

31. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 31
Some Comments on the Standard Error Estimators
Consider what happens if is small or large: y

y
0
x
x y

y
0 x x  
2
xx
t

32. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 32
Some Comments on the Standard Error Estimators
(cont’d)
3.Thelargerthesamplesize,T,thesmallerwillbethecoefficient
variances.TappearsexplicitlyinSE()andimplicitlyinSE().
Tappearsimplicitlysincethesum isfromt=1toT.
4.Theterm appearsintheSE().
Thereasonisthat measureshowfarthepointsareawayfromthe
y-axis.$ $
 $  
2
xx
t 
2
t
x 
2
t
x

33. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 33
Example: How to Calculate the Parameters and
Standard Errors
•Assumewehavethefollowingdatacalculatedfromaregressionofyona
singlevariablexandaconstantover22observations.
•Data:
•Calculations:
•Wewrite$
(*.*.)
*(.)
.



8301022241658665
3919654224165
035
2 $..*. .  866503541655912 6.130,3919654
,65.86,5.416,22,830102
2




RSSx
yxTyx
t
tt tt
xy 
ˆˆˆ tt xy 35.012.59ˆ 

34. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 34
Example (cont’d)
•SE(regression),
•Wenowwritetheresultsas  
 
0079.0
5.416223919654
1
*55.2)(
35.3
5.41622391965422
3919654
*55.2)(
2
2








SE
SE )0079.0(
35.0
)35.3(
12.59ˆ
tt xy  55.2
20
6.130
2
ˆ
2




T
u
s
t

35. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 35
An Introduction to Statistical Inference
•Wewanttomakeinferencesaboutthelikelypopulationvaluesfrom
theregressionparameters.
Example:Supposewehavethefollowingregressionresults:
• isasingle(point)estimateoftheunknownpopulation
parameter,.How“reliable”isthisestimate?
•Thereliabilityofthepointestimateismeasuredbythecoefficient’s
standarderror.$
.05091 )2561.0(
5091.0
)38.14(
3.20ˆ
tt xy 

36. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 36
Hypothesis Testing:Some Concepts
•Wecanusetheinformationinthesampletomakeinferencesaboutthe
population.
•Wewillalwayshavetwohypothesesthatgotogether,thenullhypothesis
(denotedH
0)andthealternativehypothesis(denotedH
1).
•Thenullhypothesisisthestatementorthestatisticalhypothesisthatisactually
beingtested.Thealternativehypothesisrepresentstheremainingoutcomesof
interest.
•Forexample,supposegiventheregressionresultsabove,weareinterestedin
thehypothesisthatthetruevalueofisinfact0.5.Wewouldusethenotation
H
0:=0.5
H
1:0.5
Thiswouldbeknownasatwosidedtest.

37. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 37
One-Sided Hypothesis Tests
•Sometimeswemayhavesomepriorinformationthat,forexample,we
wouldexpect>0.5ratherthan<0.5.Inthiscase,wewoulddoa
one-sidedtest:
H
0:=0.5
H
1:>0.5
orwecouldhavehad
H
0:=0.5
H
1:<0.5
•Therearetwowaystoconductahypothesistest:viathetestof
significanceapproachorviatheconfidenceintervalapproach.

38. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 38
The Probability Distribution of the
Least Squares Estimators
•Weassumethatu
tN(0,
2
)
•Sincetheleastsquaresestimatorsarelinearcombinationsoftherandom
variables
i.e.
•Theweightedsumofnormalrandomvariablesisalsonormallydistributed,so
N(,Var())
N(,Var())
•Whatiftheerrorsarenotnormallydistributed?Willtheparameterestimates
stillbenormallydistributed?
•Yes,iftheotherassumptionsoftheCLRMhold,andthesamplesizeis
sufficientlylarge.$
wy
tt $ $


39. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 39
The Probability Distribution of the
Least Squares Estimators (cont’d)
•Standard normal variates can be constructed from and :
and
•But var() and var() are unknown, so
and$ $
 
1,0~
var
ˆ
N

 
1,0~
var
ˆ
N

 2
~
)ˆ(
ˆ


T
t
SE
 2~
)
ˆ
(
ˆ


Tt
SE


40. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 40
Testing Hypotheses:
The Test of Significance Approach
•Assumetheregressionequationisgivenby,
fort=1,2,...,T
•Thestepsinvolvedindoingatestofsignificanceare:
1.Estimate,and , intheusualway
2.Calculatetheteststatistic.Thisisgivenbytheformula
whereisthevalueofunderthenullhypothesis.teststatistic
SE


$
*
(
$
)

 * SE($) SE(
$
) $ $
 ttt uxy 

41. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 41
The Test of Significance Approach (cont’d)
3.Weneedsometabulateddistributionwithwhichtocomparetheestimated
teststatistics.Teststatisticsderivedinthiswaycanbeshowntofollowat-
distributionwithT-2degreesoffreedom.
Asthenumberofdegreesoffreedomincreases,weneedtobelesscautiousin
ourapproachsincewecanbemoresurethatourresultsarerobust.
4. We need to choose a “significance level”, often denoted . This is also
sometimes called the size of the test and it determines the region where we
will reject or not reject the null hypothesis that we are testing. It is
conventional to use a significance level of 5%.
Intuitiveexplanationisthatwewouldonlyexpectaresultasextremeasthis
ormoreextreme5%ofthetimeasaconsequenceofchancealone.
Conventionaltousea5%sizeoftest,but10%and1%arealsocommonly
used.

42. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 42
Determining the Rejection Region for a Test of
Significance
5.Givenasignificancelevel,wecandeterminearejectionregionandnon-
rejectionregion.Fora2-sidedtest:f(x)
95% non-rejection
region
2.5%
rejection region
2.5%
rejection region

43. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 43
The Rejection Region for a 1-Sided Test (Upper Tail)f(x)
95% non-rejection
region
5% rejection region

44. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 44
The Rejection Region for a 1-Sided Test (Lower Tail)f(x)
95% non-rejection region
5% rejection region

45. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 45
The Test of Significance Approach: Drawing
Conclusions
6.Usethet-tablestoobtainacriticalvalueorvalueswithwhichto
comparetheteststatistic.
7.Finallyperformthetest.Iftheteststatisticliesintherejection
regionthenrejectthenullhypothesis(H
0),elsedonotrejectH
0.

46. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 46
A Note on the tand the Normal Distribution
•Youshouldallbefamiliarwiththenormaldistributionandits
characteristic“bell”shape.
•Wecanscaleanormalvariatetohavezeromeanandunitvarianceby
subtractingitsmeananddividingbyitsstandarddeviation.
•Thereis,however,aspecificrelationshipbetweenthet-andthe
standardnormaldistribution.Botharesymmetricalandcentredon
zero.Thet-distributionhasanotherparameter,itsdegreesoffreedom.
Wewillalwaysknowthis(forthetimebeingfromthenumberof
observations-2).

47. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 47
What Does the t-Distribution Look Like?normal distribution
t-distribution

48. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 48
Comparing the tand the Normal Distribution
•Inthelimit,at-distributionwithaninfinitenumberofdegreesoffreedomis
astandardnormal,i.e.
•Examplesfromstatisticaltables:
Significancelevel N(0,1)t(40)t(4)
50% 0 0 0
5% 1.641.682.13
2.5% 1.962.022.78
0.5% 2.572.704.60
•Thereasonforusingthet-distributionratherthanthestandardnormalisthat
wehadtoestimate,thevarianceofthedisturbances.t N()(,)01 
2

49. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 49
The Confidence Interval Approach
to Hypothesis Testing
•Anexampleofitsusage:Weestimateaparameter,saytobe0.93,and
a“95%confidenceinterval”tobe(0.77,1.09).Thismeansthatweare
95%confidentthattheintervalcontainingthetrue(butunknown)
valueof.
•Confidenceintervalsarealmostinvariablytwo-sided,althoughin
theoryaone-sidedintervalcanbeconstructed.

50. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 50
How to Carry out a Hypothesis Test
Using Confidence Intervals
1.Calculate,and , asbefore.
2.Chooseasignificancelevel,,(againtheconventionis5%).Thisisequivalentto
choosinga(1-)100%confidenceinterval,i.e.5%significancelevel=95%
confidenceinterval
3.Usethet-tablestofindtheappropriatecriticalvalue,whichwillagainhaveT-2
degreesoffreedom.
4.Theconfidenceintervalisgivenby
5.Performthetest:Ifthehypothesisedvalueof(*)liesoutsidetheconfidence
interval,thenrejectthenullhypothesisthat=*,otherwisedonotrejectthenull.$ $
 SE($) SE(
$
) ))
ˆ
(
ˆ
),
ˆ
(
ˆ
(  SEtSEt
critcrit 

51. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 51
Confidence Intervals Versus Tests of Significance
•NotethattheTestofSignificanceandConfidenceIntervalapproaches
alwaysgivethesameanswer.
•Underthetestofsignificanceapproach,wewouldnotrejectH
0that=*
iftheteststatisticlieswithinthenon-rejectionregion,i.e.if
•Rearranging,wewouldnotrejectif
•Butthisisjusttheruleundertheconfidenceintervalapproach.£

£t
SE
t
crit crit
$
*
(
$
)

 )
ˆ
(*
ˆ
)
ˆ
(  SEtSEt
critcrit ££ )
ˆ
(
ˆ
*)
ˆ
(
ˆ
 SEtSEt
critcrit ££

52. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 52
Constructing Tests of Significance and
Confidence Intervals: An Example
•Using the regression results above,
, T=22
•Using both the test of significance and confidence interval approaches,
test the hypothesis that =1 against a two-sided alternative.
•The first step is to obtain the critical value. We want t
crit= t
20;5%)2561.0(
5091.0
)38.14(
3.20ˆ
tt xy 

53. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 53
Determining the Rejection Region-2.086 +2.086
2.5% rejection region
2.5% rejection region
f(x)

54. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 54
Performing the Test
•Thehypothesesare:
H
0:=1
H
1:1
Testofsignificance Confidenceinterval
approach approach
DonotrejectH
0since Since1lieswithinthe
teststatlieswithin confidenceinterval,
non-rejectionregion donotrejectH
0teststat
SE





$
*
(
$
)
.
.
.


050911
02561
1917 )0433.1,0251.0(
2561.0086.25091.0
)
ˆ
(
ˆ


  SEt
crit

55. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 55
Testing other Hypotheses
•WhatifwewantedtotestH
0:=0orH
0:=2?
•Notethatwecantestthesewiththeconfidenceintervalapproach.
Forinterest(!),test
H
0:=0
vs.H
1:0
H
0:=2
vs.H
1:2

56. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 56
Changing the Size of the Test
•Butnotethatwelookedatonlya5%sizeoftest.Inmarginalcases
(e.g.H
0:=1),wemaygetacompletelydifferentanswerifweusea
differentsizeoftest.Thisiswherethetestofsignificanceapproachis
betterthanaconfidenceinterval.
•Forexample,saywewantedtousea10%sizeoftest.Usingthetestof
significanceapproach,
asabove.Theonlythingthatchangesisthecriticalt-value.teststat
SE





$
*
(
$
)
.
.
.


050911
02561
1917

57. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 57
Changing the Size of the Test:
The New Rejection Regions-1.725 +1.725
5% rejection region5% rejection region
f(x)

58. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 58
Changing the Size of the Test:
The Conclusion
•t
20;10%=1.725.Sonow,astheteststatisticliesintherejectionregion,
wewouldrejectH
0.
•Cautionshouldthereforebeusedwhenplacingemphasisonormaking
decisionsinmarginalcases(i.e.incaseswhereweonlyjustrejector
notreject).

59. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 59
Some More Terminology
•Ifwerejectthenullhypothesisatthe5%level,wesaythattheresult
ofthetestisstatisticallysignificant.
•Notethatastatisticallysignificantresultmaybeofnopractical
significance.E.g.ifashipmentofcansofbeansisexpectedtoweigh
450gpertin,buttheactualmeanweightofsometinsis449g,the
resultmaybehighlystatisticallysignificantbutpresumablynobody
wouldcareabout1gofbeans.

60. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 60
The Errors That We Can Make
Using Hypothesis Tests
•WeusuallyrejectH
0iftheteststatisticisstatisticallysignificantata
chosensignificancelevel.
•Therearetwopossibleerrorswecouldmake:
1.RejectingH
0whenitwasreallytrue.ThisiscalledatypeIerror.
2.NotrejectingH
0whenitwasinfactfalse.ThisiscalledatypeIIerror.Reality
H0 is trueH0 is false
Result of
Significant
(reject H0)
Type I error
= 

Test Insignificant
( do not
reject H0)

Type II error
= 

61. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 61
The Trade-off Between Type I and Type II Errors
•TheprobabilityofatypeIerrorisjust,thesignificancelevelorsizeoftestwe
chose.Toseethis,recallwhatwesaidsignificanceatthe5%levelmeant:itisonly
5%likelythataresultasormoreextremeasthiscouldhaveoccurredpurelyby
chance.
•Notethatthereisnochanceforafreelunchhere!Whathappensifwereducethesize
ofthetest(e.g.froma5%testtoa1%test)?Wereducethechancesofmakingatype
Ierror...butwealsoreducetheprobabilitythatwewillrejectthenullhypothesisat
all,soweincreasetheprobabilityofatypeIIerror:
•SothereisalwaysatradeoffbetweentypeIandtypeIIerrorswhenchoosinga
significancelevel.Theonlywaywecanreducethechancesofbothistoincreasethe
samplesize.less likely
to falsely reject
Reduce size more strict reject null
of test criterion forhypothesis more likely to
rejection less often incorrectly not
reject

62. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 62
A Special Type of Hypothesis Test: The t-ratio
•Recall that the formula for a test of significance approach to hypothesis
testing using a t-test was
•If the test is H
0: 
i= 0
H
1: 
i0
i.e. a test that the population coefficient is zero against a two-sided
alternative, this is known as a t-ratio test:
Since 
i* = 0,
•The ratio of the coefficient to its SE is known as the t-ratio or t-statistic.
teststatistic
SE
ii
i


$
$
*

 teststat
SE
i
i

$
(
$
)



63. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 63
The t-ratio: An Example
•Supposethatwehavethefollowingparameterestimates,standarderrors
andt-ratiosforaninterceptandsloperespectively.
Coefficient 1.10 -4.40
SE 1.35 0.96
t-ratio 0.81 -4.63
Comparethiswithat
critwith15-3 = 12d.f.
(2½%ineachtailfora5%test) = 2.1795%
= 3.0551%
•DowerejectH
0: 
1=0? (No)
H
0: 
2=0? (Yes)

64. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 64
What Does the t-ratio tell us?
•If we reject H
0, we say that the result is significant. If the coefficient is not
“significant” (e.g. the intercept coefficient in the last regression above), then
it means that the variable is not helping to explain variations in y. Variables
that are not significant are usually removed from the regression model.
•In practice there are good statistical reasons for always having a constant
even if it is not significant. Look at what happens if no intercept is included:
t
y

tx

65. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 65
An Example of the Use of a Simple t-test to Test a
Theory in Finance
•Testingforthepresenceandsignificanceofabnormalreturns(“Jensen’s
alpha”-Jensen,1968).
•TheData:AnnualReturnsontheportfoliosof115mutualfundsfrom
1945-1964.
•The model: for j= 1, …, 115
•Weareinterestedinthesignificanceof
j.
•ThenullhypothesisisH
0:
j=0.jtftmtjjftjt uRRRR  )(

66. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 66
Frequency Distribution of t-ratios of Mutual Fund
Alphas (gross of transactions costs)
Source Jensen (1968). Reprinted with the permission of Blackwell publishers.

67. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 67
Frequency Distribution of t-ratios of Mutual Fund
Alphas (net of transactions costs)
Source Jensen (1968). Reprinted with the permission of Blackwell publishers.

68. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 68
Can UK Unit Trust Managers “Beat the Market”?
•WenowperformavariantonJensen’stestinthecontextoftheUKmarket,
consideringmonthlyreturnson76equityunittrusts.Thedatacoverthe
periodJanuary1979–May2000(257observationsforeachfund).Some
summarystatisticsforthefundsare:
MeanMinimumMaximumMedian
Average monthly return, 1979-2000 1.0% 0.6%1.4%1.0%
Standard deviation of returns over time 5.1% 4.3%6.9%5.0%
•Jensen Regression Results for UK Unit Trust Returns, January 1979-May
2000RR RR
jt ft j jmt ft jt   ( )

69. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 69
Can UK Unit Trust Managers “Beat the Market”?
: Results
Estimates ofMean Minimum Maximum Median
 -0.02% -0.54% 0.33% -0.03%
 0.91 0.56 1.09 0.91
t-ratio on -0.07 -2.44 3.11 -0.25
•Infact,grossoftransactionscosts,9fundsofthesampleof76were
abletosignificantlyout-performthemarketbyprovidingasignificant
positivealpha,while7fundsyieldedsignificantnegativealphas.

70. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 70
The Overreaction Hypothesis and
the UK Stock Market
•Motivation
TwostudiesbyDeBondtandThaler(1985,1987)showedthatstockswhich
experienceapoorperformanceovera3to5yearperiodtendtooutperform
stockswhichhadpreviouslyperformedrelativelywell.
•HowCanThisbeExplained?
2suggestions
1.Amanifestationofthesizeeffect
DeBondt&Thalerdidnotbelievethisasufficientexplanation,butZarowin
(1990)foundthatallowingforfirmsizedidreducethesubsequentreturnon
thelosers.
2.Reversalsreflectchangesinequilibriumrequiredreturns
Ball&Kothari(1989)findtheCAPMbetaofloserstobeconsiderably
higherthanthatofwinners.

71. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 71
The Overreaction Hypothesis and
the UK Stock Market (cont’d)
•Anotherinterestinganomaly:theJanuaryeffect.
–Anotherpossiblereasonforthesuperiorsubsequentperformance
oflosers.
–Zarowin(1990)findsthat80%oftheextrareturnavailablefrom
holdingthelosersaccruestoinvestorsinJanuary.
•Examplestudy:ClareandThomas(1995)
Data:
MonthlyUKstockreturnsfromJanuary1955to1990onallfirms
tradedontheLondonStockexchange.

72. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 72
Methodology
•Calculatethemonthlyexcessreturnofthestockoverthemarketovera12,
24or36monthperiodforeachstocki:
U
it=R
it-R
mt n=12,24or36months
•Calculatetheaveragemonthlyreturnforthestockioverthefirst12,24,or
36monthperiod:R
n
U
i it
t
n



1
1

73. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 73
Portfolio Formation
•Then rank the stocks from highest average return to lowest and from 5
portfolios:
Portfolio 1: Best performing 20% of firms
Portfolio 2: Next 20%
Portfolio 3: Next 20%
Portfolio 4: Next 20%
Portfolio 5: Worst performing 20% of firms.
•Usethesamesamplelengthntomonitortheperformanceofeach
portfolio.

74. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 74
Portfolio Formation and
Portfolio Tracking Periods
•Howmanysamplesoflengthnhavewegot?
n=1,2,or3years.
•Ifn=1year:
Estimateforyear1
Monitorportfoliosforyear2
Estimateforyear3
...
Monitorportfoliosforyear36
•Soifn=1,wehave18INDEPENDENT(non-overlapping)observation/
trackingperiods.

75. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 75
Constructing Winner and Loser Returns
•Similarly, n= 2 gives 9 independent periods and n= 3 gives 6 independent
periods.
•Calculate monthly portfolio returns assuming an equal weighting of stocks in
each portfolio.
•Denote the mean return for each month over the 18, 9 or 6 periods for the
winner and loser portfolios respectively as and respectively.
•Definethedifferencebetweentheseas = - .
•Thenperformtheregression
=
1+
t (Test1)
•Lookatthesignificanceof
1.R
p
W R
p
L R
Dt R
p
L R
p
W R
Dt

76. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 76
Allowing for Differences in the Riskiness
of the Winner and Loser Portfolios
•Problem: Significant and positive 
1could be due to higher return being
required on loser stocks due to loser stocks being more risky.
•Solution: Allow for risk differences by regressing against the market risk
premium:
= 
2+ (R
mt-R
ft)+
t (Test 2)
where
R
mtis the return on the FTA All-share
R
ftis the return on a UK government 3 month t-bill.R
Dt

77. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 77
Is there an Overreaction Effect in the
UK Stock Market? ResultsPanel A: All Months
n = 12 n = 24 n =36
Return on Loser 0.0033 0.0011 0.0129
Return on Winner 0.0036 -0.0003 0.0115
Implied annualised return difference -0.37% 1.68% 1.56%

Coefficient for (3.47): 1
ˆ
-0.00031
(0.29)
0.0014**
(2.01)
0.0013
(1.55)

Coefficients for (3.48): 2
ˆ
-0.00034
(-0.30)
0.00147**
(2.01)
0.0013*
(1.41)

ˆ

-0.022
(-0.25)
0.010
(0.21)
-0.0025
(-0.06)
Panel B: All Months Except January
Coefficient for (3.47):
1
ˆ
-0.0007
(-0.72)
0.0012*
(1.63)
0.0009
(1.05)
Notes: t-ratios in parentheses; * and ** denote significance at the 10% and 5% levels
respectively. Source: Clare and Thomas (1995). Reprinted with the permission of Blackwell
Publishers.

78. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 78
Testing for Seasonal Effects in Overreactions
•Isthereevidencethatlosersout-performwinnersmoreatonetimeofthe
yearthananother?
•Totestthis,calculatethedifferencebetweenthewinner&loserportfolios
aspreviously,,andregressthison12month-of-the-yeardummies:
•Significantout-performanceoflosersoverwinnersin,
–June(forthe24-monthhorizon),and
–January,AprilandOctober(forthe36-monthhorizon)
–winnersappeartostaysignificantlyaswinnersin
•March(forthe12-monthhorizon).R M
Dt ii t
i
 


1
12 R
Dt

79. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 79
Conclusions
•Evidence of overreactions in stock returns.
•Losers tend to be small so we can attribute most of the overreaction in the
UK to the size effect.
Comments
•Small samples
•No diagnostic checks of model adequacy

80. ‘Introductory Econometrics for Finance’ © Chris Brooks 2013 80
The Exact Significance Level or p-value
•This is equivalent to choosing an infinite number of critical t-values from
tables. It gives us the marginal significance level where we would be
indifferent between rejecting and not rejecting the null hypothesis.
•If the test statistic is large in absolute value, the p-value will be small, and
vice versa. The p-value gives the plausibility of the null hypothesis.
e.g. a test statistic is distributed as a t
62= 1.47.
The p-value = 0.12.
•Do we reject at the 5% level?...........................No
•Do we reject at the 10% level?.........................No
•Do we reject at the 20% level?.........................Yes