Ml1 introduction to-supervised_learning_and_k_nearest_neighbors

Ml1 introduction to-supervised_learning_and_k_nearest_neighbors

1. Introduction to Supervised Learning

2. Legal Notices and Disclaimers This presentation is for informational purposes only. INTEL MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY. Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. Check with your system manufacturer or retailer or learn more at intel.com . This sample source code is released under the Intel Sample Source Code License Agreement . Intel and the Intel logo are trademarks of Intel Corporation in the U.S. and/or other countries. *Other names and brands may be claimed as the property of others. Copyright © 2017, Intel Corporation. All rights reserved.

3. What is Machine Learning? Machine learning allows computers to learn and infer from data.

4. Spam Filtering Web Search Postal Mail Routing Fraud Detection Movie Recommendations Vehicle Driver Assistance Web Advertisements Social Networks Speech Recognition Machine Learning in Our Daily Lives

5. Spam Filtering Web Search Postal Mail Routing Fraud Detection Movie Recommendations Vehicle Driver Assistance Web Advertisements Social Networks Speech Recognition Machine Learning in Our Daily Lives

6. Spam Filtering Web Search Postal Mail Routing Fraud Detection Movie Recommendations Vehicle Driver Assistance Web Advertisements Social Networks Speech Recognition Machine Learning in Our Daily Lives

7. Spam Filtering Web Search Postal Mail Routing Fraud Detection Movie Recommendations Vehicle Driver Assistance Web Advertisements Social Networks Speech Recognition Machine Learning in Our Daily Lives

8. Unsupervised Types of Machine Learning data points have known outcome Supervised data points have unknown outcome

9. Types of Machine Learning data points have known outcome data points have unknown outcome Unsupervised Supervised

10. Types of Machine Learning data points have known outcome Unsupervised data points have unknown outcome Supervised

11. Regression Classification Types of Supervised Learning outcome is continuous (numerical) outcome is a category

12. Regression Classification Types of Supervised Learning outcome is continuous (numerical) outcome is a category

13. Supervised Learning Overview d ata w ith ans w er s model p re di cte d ans w er s d ata w ith o u t ans w er s fit + + predict model model

14. Regression: Numeric Answers movie data with revenue model p re di cte d revenue movie data (unknown revenue) fit + + predict model model

15. Classification: Categorical Answers labeled data model labels unlabeled data fit + + predict model model

16. Classification: Categorical Answers emails labeled as spam/not spam model spam or not spam unlabeled emails fit + + model model predict

17. Target: predicted category or value of the data (column to predict) Features : properties of the data used for prediction (non-target columns) Example: a single data point within the data (one row) Label: the target value for a single data point Machine Learning Vocabulary

18. Machine Learning Vocabulary sepal length sepal width petal length petal width species 6.7 3.0 5.2 2.3 virginica 6.4 2.8 5.6 2.1 virginica 4.6 3.4 1.4 0.3 setosa 6.9 3.1 4.9 1.5 versicolor 4.4 2.9 1.4 0.2 setosa 4.8 3.0 1.4 0.1 setosa 5.9 3.0 5.1 1.8 virginica 5.4 3.9 1.3 0.4 setosa 4.9 3.0 1.4 0.2 setosa 5.4 3.4 1.7 0.2 setosa

19. Machine Learning Vocabulary sepal length sepal width petal length petal width species 6.7 3.0 5.2 2.3 virginica 6.4 2.8 5.6 2.1 virginica 4.6 3.4 1.4 0.3 setosa 6.9 3.1 4.9 1.5 versicolor 4.4 2.9 1.4 0.2 setosa 4.8 3.0 1.4 0.1 setosa 5.9 3.0 5.1 1.8 virginica 5.4 3.9 1.3 0.4 setosa 4.9 3.0 1.4 0.2 setosa 5.4 3.4 1.7 0.2 setosa Target

20. Target: predicted category or value of the data (column to predict) Features : properties of the data used for prediction (non-target columns) Example: a single data point within the data (one row) Label: the target value for a single data point Machine Learning Vocabulary

21. Machine Learning Vocabulary sepal length sepal width petal length petal width species 6.7 3.0 5.2 2.3 virginica 6.4 2.8 5.6 2.1 virginica 4.6 3.4 1.4 0.3 setosa 6.9 3.1 4.9 1.5 versicolor 4.4 2.9 1.4 0.2 setosa 4.8 3.0 1.4 0.1 setosa 5.9 3.0 5.1 1.8 virginica 5.4 3.9 1.3 0.4 setosa 4.9 3.0 1.4 0.2 setosa 5.4 3.4 1.7 0.2 setosa Features

22. Target: predicted category or value of the data (column to predict) Features : properties of the data used for prediction (non-target columns) Example: a single data point within the data (one row) Label: the target value for a single data point Machine Learning Vocabulary

23. Machine Learning Vocabulary sepal length sepal width petal length petal width species 6.7 3.0 5.2 2.3 virginica 6.4 2.8 5.6 2.1 virginica 4.6 3.4 1.4 0.3 setosa 6.9 3.1 4.9 1.5 versicolor 4.4 2.9 1.4 0.2 setosa 4.8 3.0 1.4 0.1 setosa 5.9 3.0 5.1 1.8 virginica 5.4 3.9 1.3 0.4 setosa 4.9 3.0 1.4 0.2 setosa 5.4 3.4 1.7 0.2 setosa Example

24. Machine Learning Vocabulary Target: predicted category or value of the data (column to predict) Features : properties of the data used for prediction (non-target columns) Example: a single data point within the data (one row) Label: the target value for a single data point

25. Machine Learning Vocabulary sepal length sepal width petal length petal width species 6.7 3.0 5.2 2.3 virginica 6.4 2.8 5.6 2.1 virginica 4.6 3.4 1.4 0.3 setosa 6.9 3.1 4.9 1.5 versicolor 4.4 2.9 1.4 0.2 setosa 4.8 3.0 1.4 0.1 setosa 5.9 3.0 5.1 1.8 virginica 5.4 3.9 1.3 0.4 setosa 4.9 3.0 1.4 0.2 setosa 5.4 3.4 1.7 0.2 setosa Label

26. 26

27. K – Nearest Neighbors

28. What is Classification? A flower shop wants to guess a customer's purchase from similarity to most recent purchase.

29. What is Classification? ? Which flower is a customer most likely to purchase based on similarity to previous purchase?

30. What is Classification? ? Which flower is a customer most likely to purchase based on similarity to previous purchase?

31. What is Classification? ? Which flower is a customer most likely to purchase based on similarity to previous purchase?

32. What is Classification? ? Which flower is a customer most likely to purchase based on similarity to previous purchase?

33. What is Needed for Classification? Model data with : Features that can be quantitated Labels that are known Method to measure similarity

34. What is Needed for Classification? Model data with : Features that can be quantitated Labels that are known Method to measure similarity

35. What is Needed for Classification? Model data with : Features that can be quantitated Labels that are known Method to measure similarity

36. K Nearest Neighbors Classification

37. Number of Malignant Nodes Age 60 K Nearest Neighbors Classification 40 20 10 20 Survived Did not survive

38. Number of Malignant Nodes Age 60 Predict K Nearest Neighbors Classification 40 20 10 20

39. K Nearest Neighbors Classification Neighbor Count (K = 1): 1 Number of Malignant Nodes Age 60 Predict 40 20 10 20

40. K Nearest Neighbors Classification Neighbor Count (K = 2): 1 1 Number of Malignant Nodes Age 60 Predict 40 20 10 20

41. K Nearest Neighbors Classification Neighbor Count (K = 3): 2 1 Number of Malignant Nodes Age 60 Predict 40 20 10 20

42. K Nearest Neighbors Classification Neighbor Count (K = 4): 3 1 Number of Malignant Nodes Age 60 Predict 40 20 10 20

43. What is Needed to Select a KNN Model?

44. Correct value for ' K' How to measure closeness of neighbors? What is Needed to Select a KNN Model? Number of Malignant Nodes Age 60 40 20 10 20

45. K = 1 Number of Malignant Nodes Age 60 40 20 10 20 K Nearest Neighbors Decision Boundary

46. K Nearest Neighbors Decision Boundary K = All Number of Malignant Nodes Age 60 40 20 10 20

47. Value of 'K' Affects Decision Boundary Number of Malignant Nodes Age 60 40 20 10 20 Number of Malignant Nodes 60 40 20 10 20 K = 1 K = All

48. Value of 'K' Affects Decision Boundary Number of Malignant Nodes Age 60 40 20 10 20 Number of Malignant Nodes 60 40 20 10 20 K = 1 K = All Methods for determining 'K ' will be discussed in next lesson

49. Number of Malignant Nodes Age 60 40 20 10 20 Measurement of Distance in KNN

50. Number of Malignant Nodes Age 60 40 20 10 20 Measurement of Distance in KNN

51. Number of Malignant Nodes Age Euclidean Distance

52. Number of Malignant Nodes Age Euclidean Distance (L2 Distance) ∆ Age d ∆ Nodes  

53. Number of Malignant Nodes Age ∆ Age ∆ Nodes Manhattan Distance (L1 or City Block Distance)  

54. Number of Surgeries Scale is Important for Distance Measurement Age 60 40 20 1 2 3 4 5

55. Number of Surgeries Age 60 40 20 1 2 3 4 5 24 22 20 18 Scale is Important for Distance Measurement

56. Number of Surgeries Age 60 40 20 1 2 3 4 5 24 22 20 18 Nearest Neighbors! Scale is Important for Distance Measurement

57. Number of Surgeries Age 60 40 20 1 2 4 5 3 "Feature Scaling " Scale is Important for Distance Measurement

58. Number of Surgeries Age 60 40 20 1 2 4 5 3 Scale is Important for Distance Measurement "Feature Scaling "

59. Number of Surgeries Age 60 40 20 1 2 4 5 3 Scale is Important for Distance Measurement Nearest Neighbors! "Feature Scaling "

60. Comparison of Feature Scaling Methods Standard Scaler: mean center data and scale to unit variance Minimum-Maximum Scaler: scale data to fixed range (usually 0–1) Maximum Absolute Value Scaler: scale maximum absolute value

61. Feature Scaling: The Syntax Import the class containing the scaling method from sklearn.preprocessing import StandardScaler Create an instance of the class StdSc = StandardScaler () Fit the scaling parameters and then transform the data StdSc = StdSc . fit ( X_data ) X_scaled = KNN . transform ( X_data ) Other scaling methods exist: MaxAbsScaler , MinMaxScaler .

62. Feature Scaling: The Syntax Import the class containing the scaling method from sklearn.preprocessing import StandardScaler Create an instance of the class StdSc = StandardScaler () Fit the scaling parameters and then transform the data StdSc = StdSc . fit ( X_data ) X_scaled = KNN . transform ( X_data ) Other scaling methods exist: MaxAbsScaler , MinMaxScaler .

63. Feature Scaling: The Syntax Import the class containing the scaling method from sklearn.preprocessing import StandardScaler Create an instance of the class StdSc = StandardScaler () Fit the scaling parameters and then transform the data StdSc = StdSc . fit ( X_data ) X_scaled = StdSc . transform (X_data) Other scaling methods exist: MaxAbsScaler , MinMaxScaler .

64. Feature Scaling: The Syntax Import the class containing the scaling method from sklearn.preprocessing import StandardScaler Create an instance of the class StdSc = StandardScaler () Fit the scaling parameters and then transform the data StdSc = StdSc . fit ( X_data ) X_scaled = StdSc . transform ( X_data ) Other scaling methods exist: MinMaxScaler , MaxAbsScaler .

65. Multiclass KNN Decision Boundary K = 5 Number of Malignant Nodes Age 60 40 20 10 20 Full remission Did not survive Partial remission

66. Regression with KNN K = 1 K = 3 K = 20

67. Characteristics of a KNN Model Fast to create model because it simply stores data Slow to predict because many distance calculations Can require lots of memory if data set is large

68. Import the class containing the classification method from sklearn.neighbors import KNeighborsClassifier Create an instance of the class KNN = KNeighborsClassifier ( n_neighbors =3) Fit the instance on the data and then predict the expected value KNN = KNN . fit ( X_data , y_data ) y_predict = KNN . predict ( X_data ) The fit and predict / transform syntax will show up throughout the course. K Nearest Neighbors: The Syntax

69. Import the class containing the classification method from sklearn.neighbors import KNeighborsClassifier Create an instance of the class KNN = KNeighborsClassifier ( n_neighbors =3) Fit the instance on the data and then predict the expected value KNN = KNN . fit ( X_data , y_data ) y_predict = KNN . predict ( X_data ) The fit and predict / transform syntax will show up throughout the course. K Nearest Neighbors: The Syntax

70. Import the class containing the classification method from sklearn.neighbors import KNeighborsClassifier Create an instance of the class KNN = KNeighborsClassifier ( n_neighbors =3) Fit the instance on the data and then predict the expected value KNN = KNN . fit ( X_data , y_data ) y_predict = KNN . predict ( X_data ) The fit and predict / transform syntax will show up throughout the course. K Nearest Neighbors: The Syntax

71. K Nearest Neighbors: The Syntax Import the class containing the classification method from sklearn.neighbors import KNeighborsClassifier Create an instance of the class KNN = KNeighborsClassifier ( n_neighbors =3) Fit the instance on the data and then predict the expected value KNN = KNN . fit ( X_data , y_data ) y_predict = KNN . predict ( X_data ) The fit and predict / transform syntax will show up throughout the course.

72. K Nearest Neighbors: The Syntax Import the class containing the classification method from sklearn.neighbors import KNeighborsClassifier Create an instance of the class KNN = KNeighborsClassifier ( n_neighbors =3) Fit the instance on the data and then predict the expected value KNN = KNN . fit ( X_data , y_data ) y_predict = KNN . predict ( X_data ) Regression can be done with KNeighborsRegressor .