mathematics for machine learning matrices.pptx

ddbgnhmgfn

1. Matrices Introduction National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

2. Matrices - Introduction Matrix algebra has at least two advantages: Reduces complicated systems of equations to simple expressions Adaptable to systematic method of mathematical treatment and well suited to computers Definition: A matrix is a set or group of numbers arranged in a square or rectangular array enclosed by two brackets National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

3. Matrices - Introduction Properties: A specified number of rows and a specified number of columns Two numbers (rows x columns) describe the dimensions or size of the matrix. Examples: 3x3 matrix 2x4 matrix 1x2 matrix National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

4. Matrices - Introduction A matrix is denoted by a bold capital letter and the elements within the matrix are denoted by lower case letters e.g. matrix [ A ] with elements a ij i goes from 1 to m j goes from 1 to n A mxn = m A n = National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

5. Matrices - Operations ADDITION AND SUBTRACTION OF MATRICES The sum or difference of two matrices, A and B of the same size yields a matrix C of the same size Matrices of different sizes cannot be added or subtracted National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

6. Matrices - Operations MULTIPLICATION OF MATRICES The product of two matrices is another matrix Two matrices A and B must be conformable for multiplication to be possible i.e. the number of columns of A must equal the number of rows of B Example. A x B = C (1x3) (3x1) (1x1) National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

7. Matrices - Operations Successive multiplication of row i of A with column j of B – row by column multiplication National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

8. Matrices - Operations Remember also: IA = A National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

9. Matrices - Operations Assuming that matrices A , B and C are conformable for the operations indicated, the following are true: AI = IA = A A ( BC ) = ( AB ) C = ABC - (associative law) A ( B + C ) = AB + AC - (first distributive law) ( A + B ) C = AC + BC - (second distributive law) Caution! AB not generally equal to BA , BA may not be conformable If AB = , neither A nor B necessarily = If AB = AC , B not necessarily = C National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

10. Matrices - Operations AB not generally equal to BA , BA may not be conformable National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

11. Matrices - Operations If AB = , neither A nor B necessarily = National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

12. Matrices - Operations TRANSPOSE OF A MATRIX If : Then transpose of A, denoted A T is: For all i and j National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

13. Matrices - Operations Properties of transposed matrices: ( A + B ) T = A T + B T ( AB ) T = B T A T (k A ) T = k A T ( A T ) T = A National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

14. Matrices - Operations SYMMETRIC MATRICES A Square matrix is symmetric if it is equal to its transpose: A = A T National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

15. INVERSE OF A MATRIX Consider a scalar k. The inverse is the reciprocal or division of 1 by the scalar. Example: k=7 the inverse of k or k -1 = 1/k = 1/7 Division of matrices is not defined since there may be AB = AC while B = C Instead matrix inversion is used. The inverse of a square matrix, A , if it exists, is the unique matrix A -1 where: AA -1 = A -1 A = I National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

16. Example: Because: National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

17. Properties of the inverse: A square matrix that has an inverse is called a nonsingular matrix A matrix that does not have an inverse is called a singular matrix Square matrices have inverses except when the determinant is zero When the determinant of a matrix is zero the matrix is singular National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

18. Matrices - Operations DETERMINANT OF A MATRIX To compute the inverse of a matrix, the determinant is required Each square matrix A has a unit scalar value called the determinant of A , denoted by det A or |A| If then National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

19. Matrices - Operations If A = [ A ] is a single element (1x1), then the determinant is defined as the value of the element Then | A | =det A = a 11 If A is (n x n), its determinant may be defined in terms of order (n-1) or less. National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

20. Matrices - Operations MINORS If A is an n x n matrix and one row and one column are deleted, the resulting matrix is an (n-1) x (n-1) submatrix of A . The determinant of such a submatrix is called a minor of A and is designated by m ij , where i and j correspond to the deleted row and column, respectively. m ij is the minor of the element a ij in A . National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

21. Each element in A has a minor Delete first row and column from A . The determinant of the remaining 2 x 2 submatrix is the minor of a 11 eg. National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

22. Therefore the minor of a 12 is: And the minor for a 13 is: National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

23. COFACTORS The cofactor C ij of an element a ij is defined as: When the sum of a row number i and column j is even, c ij = m ij and when i + j is odd, c ij =-m ij National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

24. Therefore the 2 x 2 matrix : Has cofactors : And: And the determinant of A is: National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

25. Example 1: National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

26. For a 3 x 3 matrix: The cofactors of the first row are: National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

27. The determinant of a matrix A is: Which by substituting for the cofactors in this case is: National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

28. ADJOINT MATRICES A cofactor matrix C of a matrix A is the square matrix of the same order as A in which each element a ij is replaced by its cofactor c ij . Example: If The cofactor C of A is National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

29. The adjoint matrix of A , denoted by adj A , is the transpose of its cofactor matrix It can be shown that: A (adj A ) = ( adj A ) A = | A | I Example: National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

30. USING THE ADJOINT MATRIX IN MATRIX INVERSION Since AA -1 = A -1 A = I and A (adj A ) = (adj A ) A = | A | I then National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

31. Linear Equations National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

32. Linear Equations Linear equations are common and important for survey problems Matrices can be used to express these linear equations and aid in the computation of unknown values Example n equations in n unknowns, the a ij are numerical coefficients, the b i are constants and the x j are unknowns National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

33. Linear Equations The equations may be expressed in the form AX = B where and n x n n x 1 n x 1 Number of unknowns = number of equations = n National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

34. Linear Equations If the determinant is nonzero, the equation can be solved to produce n numerical values for x that satisfy all the simultaneous equations To solve, premultiply both sides of the equation by A -1 which exists because |A| = A -1 AX = A -1 B Now since A -1 A = I We get X = A -1 B So if the inverse of the coefficient matrix is found, the unknowns, X would be determined National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

35. Linear Equations Example The equations can be expressed as National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

36. Linear Equations When A -1 is computed the equation becomes Therefore National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

37. Linear Equations The values for the unknowns should be checked by substitution back into the initial equations National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

38. Introduction to Statistics and Probability National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

39. STATISTICS It is the science of collecting, organizing, analyzing and interpreting data. There are two types of Statistics: Inferential Statistics : It is about using sample data from a dataset and making inferences and conclusions using probability theory. Descriptive Statistics : It is used to summarize and represent the data in an accurate way using charts, tables and graphs. For example, you might stand in a mall and ask a sample of 100 people if they like shopping at Sears. You could make a bar chart of yes or no answers (that would be descriptive statistics ) or you could use your research (and inferential statistics ) to reason that around 75%-80% of population. National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

40. DESCRIPTIVE STATISTICS The following measures are used to represent the data set : National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

41. MEASURE OF POSITION Also known as measure of Central Tendency . A measure of central tendency is a single value that attempts to describe a set of data by identifying the central position within that set of data. There are three measures of central tendencies: Mean, Median and Mode. National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

42. Median: It is a point that divides the data into two equal halves while being less susceptible to outliers compare to mean. For ungrouped data : middle data point of an ordered data set. For grouped data : Where, L = lower limit of median class n = number of observations cf = cumulative frequency of class preceding the median class f = frequency of median class w = class size Mean: It is a point where mass of distribution of data balances. National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

43. Mode: It refers to the data item that occurs most frequently in a given data set. Mode for ungrouped data : Most frequent observation in the data. Mode for grouped data : National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

44. Example for ungrouped data: Question: National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

45. MEASURE OF DISPERSION It refers to how the data deviates from the position measure i.e. gives an indication of the amount of variation in the process. Dispersion of the data set can be described by: Range: It is the difference between highest and the lowest values. Standard Deviation: It is the measurement of average distance between each quantity and mean i.e. how data is spread out from mean. Higher the standard deviation, more is the data spread from mean. National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

46. In normal distribution, when data is unimodal, z-scores is used to calculate the probability of a score occurring within a standard normal distribution and helps to compare two scores from different samples. Below, calculating the probability of randomly obtaining a score from the distribution . 68% probability for -1 and +1 standard deviation from mean. Similarly, 95% for -1.96 and +1.96 standard deviation. National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

47. MEASURE OF SHAPE It is used to characterize the location and variability of data set. Two common statistics that measure the shape of the data are: Skewness and Kurtosis Skewness : It is the horizontal displacement of the normal curve about the mean position. Skewness for a normal distribution is zero. National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

48. CORRELATION ANALYSIS It is a statistical technique that can show whether and how strongly pairs of variables are related. If correlation coefficient (r) is Positive, then both variables are directly proportional. Zero, there is no relation between them. Negative, then both variables are inversely proportional National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

49. REGRESSION ANALYSIS The statistical technique of estimating the unknown value of one variable(i.e. dependent variable ) from the known value of other variable (i.e. independent variable ) is called regression analysis. The regression equation of X on Y is: X = a +bY The regression equation of Y on X is: Y = a +bX Dependent Variable : The single variable which we wish to estimate/predict by regression model. Independent Variable : The known variable(s) used to predict/estimate the value of dependent variable. X is dependent, Y is independent Y is depedent, X is independent National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

50. PROBABILITY Probability is a numerical description of how likely an event is to occur or how likely it is that a proposition is true. Tossing a coin: When a coin is tossed, there are two possible outcomes: Heads (H) or Tails (T).Thus, probability of the coin landing  H  is ½ and the probability of the coin landing  T  is ½. Rolling a die: When a single die is thrown, there are six possible outcomes:  1, 2, 3, 4, 5, 6 .The probability of any one of them is  1/ 6. Some examples are: National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

51. TERMINOLOGY Experiment :   A process by which an outcome is obtained. Sample space :  The set S of all possible outcomes of an experiment. i.e. the sample space for a dice roll is {1, 2, 3, 4, 5, 6} Event :   Any subset E of the sample space i.e.         Let, E 1  = An even number is rolled.         E 2  = A number less than three is rolled. Outcome : Result of a single trial. Equally likely outcomes : Two outcomes of a random experiment are said to be equally likely, if upon performing the experiment a (very) large number of times, the relative occurrences of the two outcomes turn out to be equal. Trial : Performing a random experiment. National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

52. EVENTS Simple Events : If the event E has only single element of a sample space, it is called as a simple event. Eg : if S = {56 , 78 , 96 , 54 , 89} and E = {78} then E is a simple event. Compound Events : Any event consists of more than one element of the sample space. Eg : if S = {56 ,78 ,96 ,54 ,89}, E1 = {56 ,54 }, E2 = {78 ,56 ,89 } then, E1 and E2 represent two compound events. Independent Events and Dependent Events : If the occurrence of any event is completely unaffected by the occurrence of any other event, such events are   Independent Events . Probability of two independent event is given by, National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

53. The events which are affected by other events are Dependent Events . Probability of dependent event is given by, Exhaustive Events : A set of events is called exhaustive if all the events together consume the entire sample space. Eg : A and B are sets of mutually exclusive events, Mutually Exclusive Events : If the occurrence of one event excludes the occurrence of another event i.e. no two events can occur simultaneously. Where, S = sample space National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

54. Addition Theorem Theorem 1: If A and B are two mutually exclusive events, then P(A ∪ B) = P(A) + P(B) Where, n = Total number of exhaustive cases             n1= Number of cases favorable to A.             n2= Number of cases favorable to B. Theorem2:   If A and B are two events that are not mutually exclusive, then                   P(A ∪ B) = P( A ) + P( B ) - P ( A ∩ B ) Where, P (A ∩ B) = Probability of events favorable to both A and B National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

55. Multiplication Theorem  If A and B are two independent events, then the probability that both will occur is equal to the product of their individual probabilities. Example: The probability of appointing a lecturer who is B.Com , MBA, and PhD, with probabilities 1/20, 1/25 and 1/40 is given by: Using multiplicative theorem for independent events, National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

56. Conditional Probability The  conditional probability  of an event  B  is the probability that the event will occur given the knowledge that an event  A  has already occurred. It is representated  as P( B | A) . P(A | B) = P(A ∩ B) ⁄ P(B) Where A and B are two dependent events. National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

57. Total Probability Theorem Given n mutually exclusive events A1, A2, … Ak such that their probabilities sum is unity and their union is the event space E, then Ai ∩ Aj = NULL, for all i not equal to j A1 U A2 U ... U Ak = E Then Total Probability Theorem or Law of Total Probability is: where B is an arbitrary event, and P(B/Ai) is the conditional probability of B assuming A already occurred. National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

58. Proof of Total Probability Theorem : As intersection and Union are Distributive. Therefore, B = (B ∩ A1) U (B ∩ A2) U ….... U ( B ∩ AK) Since all these partitions are disjoint. So, we have, P (B ∩ A1) = P (B ∩ A1) U P(B ∩ A2) U ….... U P ( B ∩ AK) This is, addition theorem of probabilities for union of disjoint events. Using Conditional Probability: P (B / A) = P(B ∩ A) / P(A) We know, A1 U A2 U A3 U ….. U AK = E(Total) Then, for any event B, we have, B = B ∩ E B = B ∩ (A1 U A2 U A3 U … U AK ) National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

59. As the events are said to be independent here, P(A ∩ B) = P(A) * P(B) where P(B|A) is the conditional probability which gives the probability of occurrence of event B when event A has already occurred. Hence, P( B ∩ Ai ) = P( B | Ai ).P( Ai ) ; i = 1,2,3 . . . k Applying this rule above: This is Law of Total Probability . It is used for evaluation of denominator in Bayes’ Theorem. National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

60. BAYES’ THEOREM It is a mathematical formula for determining conditional probability. In above formula, the posterior probability is equal to the conditional probability of event B given A multiplied by the prior probability of A, all divided by the prior probability of B. Science itself is a special case of Bayes’ theorem because we are revising a prior probability( hypothesis) in the light of observation or experience that confirms our hypothesis( experimental evidence) to develop a posterior probability( conclusion) National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

61. Example Bayes’ Theorem: National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

62. National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

63. BINOMIAL DISTRIBUTION OF PROBABILITY A  binomial distribution  is the probability of a SUCCESS or FAILURE outcome in an experiment or survey that is repeated multiple times. Criteria for binomial distribution: The number of observations or trials is fixed Each observation or trial is independent. The probability of success (tails, heads, fail or pass) is exactly the same from one trial to another. National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

64. Example: Q. A coin is tossed 10 times. What is the probability of getting exactly 6 heads? The number of trials (n) is 10 x = 6 
The odds of success (p) (tossing a heads) is 0.5 Odds of failure (q) = 1- p P(x=6) = 10C6 * 0.5^6 * 0.5^4 = 210 * 0.015625 * 0.0625 = 0.205078125 National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

65. POISSON DISTRIBUTION OF PROBABILITY The Poisson distribution is the discrete probability distribution of the number of events occurring in a given time period, given the average number of times the event occurs over that time period. When the number of trials in a binomial distribution is very large, and the probability of success is very small, then np ~ npq (as q ~ 1), therefore it is possible to change the distribution to a Poisson distribution. Where, x = 0,1,2,3 …. ƛ = mean number of occurrences in the interval e = Euler’s constant National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

66. Example: Q. Twenty sheets of aluminum alloy were examined for surface flaws. The frequency of the number of sheets with a given number of flaws per sheet was as follows The total number of flaws = (0x4)+(1x3)+(2x5)+(3x2+(4x4)+(5x1)+(6x1) = 46 So the average for 20 sheets (ℳ ) = 46/20 = 2.3 Probability = P(X>=3) = 1 – (P(x0) +P(x1)+P(x2)) Using Poisson distribution formula = 0.40396 What is the probability of finding a sheet chosen at random which contains 3 or more surface flaws? National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

67. Continuous Distribution A probability distribution in which the random variable X can take on any value (is continuous) i.e. the probability of X taking on any one specific value is zero. Normal Distribution: A continuous random variable x is said to follow normal distribution, if its probability density function is define as follow, Where, (μ)= means and (σ)= standard deviations. National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center

68. Chi- Squared Test: The Chi-Square statistic is commonly used for testing relationships between categorical variables.  The null hypothesis of the Chi-Square test is that no relationship exists on the categorical variables in the population. They are independent. The calculation of the Chi-Square statistic is quite straight-forward and intuitive. Where, f o  = The observed frequency , f e  = The expected frequency if NO relationship existed between the variables, χ 2 = Degree of freedom. National Institute of Electronics & Information Technology Ministry of Electronics & Information Technology ( MeitY ), Government of India Gorakhpur Center