Simple linear regression

Simple linear regression

1. ECONOMICS
BASIC STATISTICS
Drrekhachoudhary
Department of Economics
Jai NarainVyas University, Jodhpur
Rajasthan
Department of Economics

2. Introduction
Themeaningofregressionis“goingback”or“returning”.Theterm
regressionwasfirstusedinstatisticsbySirFrancisGaltonafamous
scientistin1877inastudypaperentitled,“RegressiontowardsMediocrity
inHeredityStature”
WallinandRobertshaverightlysaid,“Itisoftenmoreimportanttofind
outwhattherelationactuallyis,inordertoestimateorpredictone
variable(thedependentvariable);andthestatisticaltechniqueappropriate
tosuchacaseiscallregressionanalysis”
Regression analysis is very useful in economic and business world.
Department of Economics

3. Objectives
Aftergoingthroughthisunit,youwillbeableto:
UnderstandtheconceptofSimplelinearRegression;
DefinelinearRegression,typesofRegression,Regressioncoefficients
MeritsandDemeritsofRegression
SomeparticlesproblemofRegressionindifferentserieswithdifferent
methods.
Department of Economics

4. Types Of Regression Analysis
Department of Economics
Regression
Simple&Multiple
Regression
Linear & Non Linear
Regression
Partial & Total
Regression

5. Linear Regression
Definition
Generally,intwomutuallyrelatedstatisticalseries,theregressionanalysisis
basedongraphicmethod.UndergraphicmethodthevaluesofXandY
variablesareplottedonagraphpaperinthefromofscatterdiagram.When
twolinesaredrawnpassingnearesttothedots,theseareknownasregression
lines.Iftheselinesarestraight,theregressioniscalledasLinear
Simple Linear Regression
ThestudyoflinearregressionbetweenthevaluesofXandYvariablesis
calledSimplelinearregression.Thatvariable,outofthetwo,whichis
knowniscalledindependentvariable,whichisthebaseofprediction,and
thatvariablewhichistobepredicatediscalleddependentvariable.
Department of Economics

6. Regression
lines
Regression
Equations
Regression
coefficients
Simple Linear Regression
Department of Economics

7. Regression Lines
Meaning
Theregressionlineshowstheaveragerelationshipbetweentwovariables.It
isalsocalledLineofBestFit.
IftwovariablesX&Yaregiven,thentherearetworegressionlines:
RegressionLineofXonY
RegressionLineofYonX
NatureofRegressionLines
Ifr=±1,thenthetworegressionlinesarecoincident.
Ifr=0,thenthetworegressionlinesintersecteachotherat90°.
Thenearertheregressionlinesaretoeachother,thegreaterwillbethe
degreeofcorrelation.
Ifregressionlinesrisefromlefttorightupward,thencorrelationis
positive.
Department of Economics

8. Functions of Regression Lines
1.ThebestEstimate
2.Degreeanddirectionof
correlation
I.Positive
II.Negative
III.Perfectcorrelationoneline
IV.Absenceofcorrelation
V.Limiteddegreeofcorrelation
Department of Economics

9. Regression Equations
Regression equations are algebraic form of regression lines. There are two
regression equations:
Regression Equation of Y on X
Y = a + bX
Y –�= ??????��(�− �)
Y –�= ??????. σ �(�− �)
σ �
Regression Equation of X on Y
X = a + bY
X –�= ??????��(�− �)
X –�= ??????. σ �(�− �)
σ �
Department of Economics

10. •Regressioncoefficientmeasurestheaveragechangeinthe
valueofonevariableforaunitchangeinthevalueof
anothervariable.
•Theserepresenttheslopeofregressionline
•Therearetworegressioncoefficients:
Regression Coefficients
Department of Economics
Regression coefficient of Y on X:
byx = ??????. σ �
σ �
Regression coefficient of X on Y:
bxy = ??????. σ �
σ �

11. •Coefficientofcorrelationisthegeometricmeanoftheregression
coefficients.i.e.r=√??????��.??????��
•Boththeregressioncoefficientsmusthavethesamealgebraicsign.
•Coefficientofcorrelationmusthavethesamesignasthatofthe
regressioncoefficients.
•Boththeregressioncoefficientscannotbegreaterthanunity.
•Regressioncoefficientisindependentofchangeoforiginbutnotofscale
Properties Of Regression Coefficients
Department of Economics

12. DifferenceBetweenCorrelation&Regression
Degree & Nature of Relationship
CorrelationisameasureofdegreeofrelationshipbetweenX&Y
Regressionstudiesthenatureofrelationshipbetweenthevariablessothatone
maybeabletopredictthevalueofonevariableonthebasisofanother.
Cause & Effect Relationship
Correlationdoesnotalwaysassumecauseandeffectrelationshipbetweentwo
variables.
Regressionclearlyexpressesthecauseandeffectrelationshipbetweentwo
variables.Theindependentvariableisthecauseanddependentvariableiseffect.
IndependentandDependentRelationship
Incorrelationanalysis,thereisnoimportanceofindependentanddependent
variables.
Incaseofregression,therearetwocoefficients.
Non-senseCorrelation
Sometimesmaybenon-sensecorrelationbetweenXandYseries,butregression
isnevernon-sense.
Department of Economics

13. ThismethodisalsocalledasLeastSquareMethod.Underthismethod,
regressionequationscanbecalculatedbysolvingtwonormalequations:
Regression Equations In Individual Series
Using Normal Equations
Department of Economics
Regression Equation of Y on X
Y = a + bX
ƩY = Na + bƩX
ƩXY = a ƩX +bƩX²
Regression Equation of X on Y
X = a + bY
ƩX = Na + bƩY
ƩXY = a ƩY +bƩY²

14. Example: Calculate the regression equation of X on Y and Y on X using method of
least squares:
X 1 2 3 4 5
Y 2 5 3 8 7
X Y X² Y² XY
1 2 1 4 2
2 5 4 25 10
3 3 9 9 9
4 8 16 64 32
5 7 25 49 35
15
ƩX
25
ƩY
55
ƩX²
151
ƩY²
88
ƩXY
Regression Equation of X on Y
ƩX = Na + bƩY
ƩXY = a ƩY +bƩY²
or 15 = 5a + 25b ……(1)
88 = 25a + 151b …..(2)
(i) is multiplied by 5 and then subtracted from eq.
(ii)
Regression Equation of Y on X
ƩY = Na + bƩX
ƩXY = a ƩX +bƩX²
or 25 = 5a + 15b ……(1)
88 = 15a + 55b …..(2)
(i)is multiplied by 3 and then subtracted from eq.
(ii)
88 = 15a + 55b
75 = 15a + 45b
13 = 10b
b = 1.3
25= 5a +15 x 1.3
a = 1.1
Y = a + bX
Y = 1.1 + 1.3X
88 = 25a + 151b
75 = 25a + 125b
13 = 26b
b = 0.5
15= 5a +25x0.5
a = 0.5
X = a + bY
X = 0.5 + 0.5 Y
Department of Economics

15. Regression Equation of Y on X
Y –�= byx (X –�)
where byx = 266 − 40X30
5
340 −(40)²
5
= 1.3
Regression Equation of X on Y
X –�= bxy (Y –�)
where bxy = 266 − 40X30
5
220 −(30)²
5
=0.65
Regression Equations Using Regression
Coefficients (Using Actual Values)
Example :Calculate two regression equations with the help of original values-
X 5 7 9 8 11
Y 2 4 6 8 10
X Y X² Y² XY
5 2 25 4 10
7 4 49 16 28
9 6 81 36 54
8 8 64 64 64
11 10 121 100 110
40
ƩX
30
ƩY
340
ƩX²
220
ƩY²
266
ƩXY
X-8 =0.65(Y-6)
X= 0.65Y + 4.1
Y-6 =1.3(X-8)
Y= 1.3X+ 4.4
Department of Economics

16. Regression Equation of Y on X
Y –�= byx (X –�)
where byx = Σd�d�
Σd²x
Regression Equations Using Coefficients (Using
Deviations From Actual Values)
Regression Equation of Y on X
Y –�= byx (X –�)
where byx = ??????. σ �
σ �
Regression Equation of X on Y
X –�= bxy (Y –�)
where bxy = Σd�d�
Σd²y
Regression Equations Using Coefficients (Using
Standard Deviations)
Regression Equation of X on Y
X –�= bxy (Y –�)
where bxy = ??????. σ �
σ �
Department of Economics

17. Regression Equations Using Coefficients (Using
Deviations From Assumed Mean)
Height of Father 6566676768697173
Height of Sons 6768646872706970
Example:Calculate regression equations by calculating both regression coefficients by
assumed mean method
Height of Father X Height of Son Y Productof dₓ & dy
H in inches Deviation from
67
Square of
Deviation
H in inchesDeviation
from 68
Square of
Deviation
X dₓ d²ₓ Y dy d²y dₓdy
65 -2 4 67 -1 1 2
66 -1 1 68 0 0 0
67 0 0 64 -4 16 0
67 0 0 68 0 0 0
68 1 1 72 4 16 4
69 2 4 70 2 4 4
71 4 16 69 1 1 4
73 6 36 70 2 4 12
N= 8 Σ??????�=10 Σ??????²�=62 Σ??????�=4 Σ??????²y=42Σ??????�??????�=26
Department of Economics

18. Regression Equation of X on Y
X –�= bxy (Y –�)
where bxy = ??????.Σ??????�??????�− Σ??????�Σ??????�
??????.Σ??????²�−(Σ??????�)²
= 8x 26− 10x4 = 0.525
8x 42 −(4)²
Regression Coefficients
Regression Equation of Y on X
Y –�= byx (X –�)
where byx = ??????.Σ??????�??????�− Σ??????�Σ??????�
??????.Σ??????²�−(Σ??????�)²
= 8x 26− 10x4 = 0.424
8x62 −(10)²
Arithmetic Mean
X̅= Aₓ + Ʃdₓ = 67 +10=68.25
N 8
Y̅= Ay+ Ʃdy = 68 +4=68.50
N 8
Regression Equations
X –�= bxy (Y –�)
X –68.25 =0.525 (Y-68.5)
X = 0.525Y + 32.29
Y –�= byx (X –�)
Y –68.5 =0.424 (X-68.25)
Y = 0.424X + 39.56
Department of Economics

19. Theregressionequationissimplyamathematicalequationforaline.Itisthe
equationthatdescribestheregressionline.Inalgebra,werepresenttheequation
foralinewithsomethinglikethis:
y = a + bx
Department of Economics
Ifwewanttodrawalinethatisperfectlythroughthemiddleofthepoints,we
wouldchoosealinethathadthesquareddeviationsfromtheline.Actually,
wewouldusethesmallestsquareddeviations.Thiscriterionforbestlineis
calledthe"LeastSquares"criterionorOrdinaryLeastSquares(OLS).
Weusetheleastsquarescriteriontopicktheregressionline.Theregression
lineissometimescalledthe"lineofbestfit"becauseitisthelinethatfits
bestwhendrawnthroughthepoints.Itisalinethatminimizesthedistanceof
theactualscoresfromthepredictedscores.
Regression Line
Conclusion
Regression Equation

20. MultipleRegression
Multipleregressionisanextensionofasimplelinearregression.
Inmultipleregression,adependentvariableispredictedby
morethanoneindependentvariable
Y=a+b
1x
1+b
2x
2+...+b
kx
k
Department of Economics

21. Unit End Questions
1.ExplainthemeaningandsignificanceoftheconceptofRegression.
2.ExplaintheconceptsofCorrelationandRegression.Howdotheydiffer
fromeachother?WhytherearetwolinesofRegression?
3.WhatisRegressionequations?WritetheRegressionequationsofXon
YandYonXandexplainthesymbolsused.
4.Thetworegressionlinesare:X=2Y+5andY=2X+10
33
Estimatethevalueof(a)YgivenX=4,and(b)XgivenY=6
Department of Economics

22. Required Readings
B.L.Aggrawal(2009).BasicStatistics.NewAgeInternationalPublisher,Delhi.
Gupta,S.C.(1990)FundamentalsofStatistics.HimalayaPublishingHouse,Mumbai
Elhance,D.N:FundamentalofStatistics
Singhal,M.L:ElementsofStatistics
Nagar,A.L.andDas,R.K.:BasicStatistics
CroxtonCowden:AppliedGeneralStatistics
Nagar,K.N.:Sankhyikikemooltatva
Gupta,BN:Sankhyiki
https://www.bmj.com/about-bmj/resources-readers/publications/statistics-square-
one/11-correlation-and-regression
Department of Economics

23. THANKS………
Department of Economics