Lecture 3 Supervised Learning(Machine Learning).pptx

Machine Learning

1. Supervised Learning (Part 2) Md. Shahidul Islam Assistant Professor Dept. of CSE, University of Asia Pacific

2. Linear Regression Linear Regression is a fundamental statistical method for predicting continuous values. It models the relationship between a dependent variable (label) and one or more independent variables (features) using a straight line. A model with two or more dependent variables is called multivariate linear regression A model with two or more independent variables is called multiple linear regression 2

3. Linear Regression Dependent Variable Independent Variable Regression Line 3

4. Linear Regression Now we have to predict the rent of new house size. 4

5. Mathematics Behind Linear Regression Hypothesis function : Cost function: Mean Squared Error ( MSE) =   5

6. Linear Regression 6

7. Initialize weight (W), bias (b), and learning rate ( ). Compute the predicted value, . Compute the loss using Mean Squared Error (MSE), . Update weight and bias using Gradient Descent. Repeat until convergence.   Linear Regression Algorithm 7

8. Real world applications have multiple independent variables Multiple linear regression comes into action The formulation is similar to the simple linear regression Gradient Descent for Multiple Linear Regression   Multiple Linear Regression 8

9. Pros Simple model Computationally efficient Interpretability of the Output Cons Overly-Simplistic Linearity Assumption Severely affected by Outliers Pros and Cons 9

10. Describes the relationship between the independent variable x and the dependent variable y using an n th -degree polynomial in x Characterizes fitting a nonlinear relationship between the x value and the conditional mean of y Types Linear – if degree as 1 Quadratic – if degree as 2 Cubic – if degree as 3 and goes on, on the basis of degree. Polynomial Regression 10

11. Why Polynomial Regression? 11

12. Pros The best approximation of the connection between the dependent and independent variables Can accommodate a wide range of functions Cons One or two outliers in the data might have a significant impact Fewer model validation methods for detecting outliers in nonlinear regression Pros and Cons 12

13. Practice with Data House Size (sq. ft) Rent ($1000s) 800 1.5 1200 2.0 1500 2.5 1800 3.0 Predicting house rent based on size. Fit a linear regression model and derive the best-fit line. 13

14. Questions? 14