properties of Ordinary Least Square (OLS )

Ordinary Least Squares (OLS) is a fundamental method in linear regression analysis, used to estimate the parameters of a linear relationship between a dependent variable and one or more independent variables.

1. Properties of Ordinary Least Square

2. Introduction In econometrics, the property of linearity in Ordinary Least Squares (OLS) refers to the assumption that the relationship between the dependent variable and the independent variables is linear. This means that the model is represented by a straight line (in the case of simple linear regression) or a hyperplane (in the case of multiple linear regression) when plotted on a graph.

3. 1. Linearity Mathematically, a linear regression model with one independent variable (simple linear regression) can be represented as:  Y = β₀ + β₁X + ε  where:  Y is the dependent variable (the variable we want to predict or explain). X is the independent variable (the variable used to predict the dependent variable). β₀ is the y-intercept (the value of Y when X is 0). β₁ is the slope coefficient (the change in Y for a one-unit change in X). ε is the error term representing the variability that cannot be explained by the model.

4. Continue.. For multiple linear regression, the equation is extended to include more independent variables:  Y = β₀ + β₁X₁ + β₂X₂ + ... + β ₖ X ₖ + ε  where X₁, X₂, ..., X ₖ are the k independent variables, and β₁ , β₂ , ..., β ₖ are their corresponding slope coefficients. The linearity assumption is crucial because OLS is designed to estimate the slope coefficients that best fit a straight line (or hyperplane) to the observed data points. If the relationship between the variables is not linear, the OLS estimates may be biased, inefficient, or lead to misleading interpretations.  

5. Continue.. If the data suggests that the relationship is not linear, econometricians may consider applying transformations to the variables, introducing interaction terms, or exploring other regression techniques that can handle non-linear relationships, such as polynomial regression or spline regression. To check the linearity assumption in econometrics, researchers often plot the data points and the fitted regression line. A scatter plot of the data points should roughly resemble a straight line for the linearity assumption to hold. Additionally, residual plots are used to assess the randomness and homoscedasticity of the residuals, which helps evaluate if the linearity assumption is met.

6. 2.Unbiasedness The property of unbiasedness refers that they provide estimates that, on average or conditional mean value or estimated values are equal to the true population parameters. In other words, the expected value of the OLS estimates is equal to the true coefficients. That is =  

7. Continue.. Y is the dependent variable (the variable we are trying to explain or predict). X is the independent variable (the variable we use to explain the variation in are the true population parameters (the intercept and slope of the linear relationship between X and Y) ε is the error term, representing the difference between the true value of Y and the predicted value by the model. The OLS estimation process aims to find the values of that minimize the sum of squared differences between the observed values of Y and the predicted values from the model. =  

8. Mathematically, the OLS estimates are given by: where Xi and Yi are the observed values of X and Y respectively, and and are their sample means. Under the assumption that the error term ε has a mean of zero, i.e., E ( є ) =0,the OLS estimators are unbiased. It means that if we were to repeat the process of estimating the coefficients with dif ferent samples from the same population, the average value of the estimated coefficients would be equal to the true population parameters  

9. 3. Minimum Variance Minimum Variance: The OLS estimator has the minimum variance compared to all other unbiased linear estimators. This means that the OLS estimator tends to be more precise and less affected by random sampling fluctuations compared to alternative estimators.

10. Example for Minimum Variance Total Sum of Square (TSS ) Explained Sum of Square (ESS) Residual Sum of Square (RSS)

11. Continue.. The Gauss-Markov theorem further specifies that if the assumptions of the classical linear regression model (CLRM) hold, and the errors (residuals) have constant variance (homoscedasticity) and are uncorrelated with the predictors, then the OLS estimator is not only unbiased but also has the minimum variance among all linear unbiased estimators. The following assumptions are….

12. These assumptions include: Linearity: The relationship between the dependent variable and the independent variables is linear. Independence: The errors are independent of each other. Homoscedasticity: The errors have constant variance. No perfect multicollinearity: The independent variables are not perfectly correlated with each other. Zero conditional mean: The errors have a mean of zero conditional on the independent variables. When the Gauss-Markov assumptions are violated, alternative estimators, such as Generalized Least Squares (GLS), may be more efficient than OLS. However, OLS remains a widely used and valuable estimation method, especially when the violations of the assumptions are not severe.