Multinomial Logistic Regression Analysis

Multinomial Logistic Regression Analysis, Agricultural Economics, Agricultural Statistics

1. WELCOME 1

2. MULTINOMIAL LOGISTIC REGRESSION ANALYSIS – A METHODOLOGICAL REVIEW HARISH KUMAR H.R PALB-9007 II Ph.D.(Agri. Economics ) 9986640586 Second seminar on 2 Seminar Teachers: Dr. K.B Umesh Dr. P.S Srikantha Murthy Major Advisor: Dr. D. Sreenivas Murthy IIHR-Bengaluru

3. Flow of seminar Introduction Odds and log odds Transformation of probability to log odds MLR model Assumptions Model fitting Model validation Analysis and interpretation Case studies Conclusion 3

4. Regression is a functional relationship between dependent variable and one or more independent variable logistic regression , or  logit regression , or  logit model  is a   regression  model where the  dependent variable   is  categorical or nominal . Choosing an appropriate type of regression is mainly based on Type of dependent variable Type and number of independent variables INTRODUCTION 4 Dependent Independent Regression type Quantitative Quantitative (Single variable) Quantitative (>1 variable) Simple linear regression Multiple linear regression Qualitative Dichotomous (Yes/No) > 2 Categories/outcomes Quantitative /Qualitative or both Binary logistic regression Multinomial logistic regression

5. Multinomial logistic regression is a simple extension of binary logistic regression that allows for more than two categories of the dependent or outcome variable . It is used to model nominal outcome variables, in which the log odds of the outcomes are modeled as a linear combination of the predictor variables . The independent (predictor) variables can be either dichotomous (i.e., binary) or continuous (i.e ., interval or ratio in scale). 5

6. Odds are simply a different expression of the probability. The probability of an event occurring relative to the probability of an event not occurring. In terms of probabilities, the equation above is translated into: Where p is the probability of the event occurring . b’s are regression coefficients and x’s are independent variables Odds and log odds   =     6

7. Why do we take all the trouble doing the transformation from probability to log odds One reason is that it is usually difficult to model a variable which has restricted range, such as probability.  Another reason is that among all of the infinitely many choices of transformation, the log of odds is one of the easiest to understand and interpret.  This transformation is called logit transformation.    Probability  ranges from 0 to 1 Odds  range from 0 to  ∞ Log Odds  range from  −∞  to  +∞ It maps probability ranging between 0 and 1 to log odds ranging from negative infinity to positive infinity . That is why the log odds are used to avoid modeling a variable with a restricted range such as probability. ? 7

8. Multinomial Logistic Model Suppose a dependent variable has M categories . One value (typically the first, the last, or the value with the highest frequency) of the Dependent variable is designated as the reference(base) category. The probability of membership in other categories is compared to the probability of membership in the reference (base) category. For a dependent variable with M categories, this requires the calculation of M-1 equations , one for each category relative to the reference category, to describe the relationship between the dependent variable and the independent variables . Examples : 1. Entering high school students make program choices among general program , vocational program and academic program 2. Analysis of Farmers’ participation in agricultural Co- opreatives . ( Non-member, coopreative member, farmer group member ) 3 . Farmers’ perception and adoption to climate change. ( no adoption, Crop rotation, for Cultivate one season , for Mixing irrigation water, for Cultivation of heat resistant varieties , move to another place of cultivation ) 8

9. Hence, if the first category is the reference, then, for m = 2, …, M Where are i th respondent belongs to M category and are regression coefficients x’s are independent variables i =1,2,3,…. n K=1,2,3,….K Hence, for each case, there will be M-1 predicted log odds, one for each category relative to the reference (base) category. 9

10. When there are more than 2 groups, computing probabilities is a little more complicated than it was in logistic regression. For m = 2, …, M , Where = = linear combination of independent variables of all outcomes except m outcome For the reference(base) category, 10

11. Assumption 1 :  Your dependent variable should be measured at the  nominal  level .  Assumption 2 :  You have one or more independent variables that are  continuous, ordinal or nominal (including dichotomous variables). Assumption 3 :  You should have independence of observations and the dependent variable should have mutually exclusive and exhaustive categories. Assumption 4 : There should be no M ulticollinearity .  Assumption 5 :  There needs to be a linear relationship between any continuous independent variables and the logit transformation of the dependent variable. Assumption 6 :  There should be no outliers, high leverage values or highly influential points . Assumption checking 11 Reference: STARKWEATHER, J. AND AMANDA, K. M., 2011, Multinomial Logistic Regression. https://it.unt.edu/sites/default/files/mlr_jds_aug2011.pdf .

12. The obtained model has said to be fit the data based on the chi square value in the model fitting information and goodness of fit tables. In multinomial logistic regression, the proportion of variance that can be explained by the model is measured by Pseudo R-square value. which indicates that how much the independent variables are good to explain the impact on dependent variable in order to make the model adequate. Pseudo R-square value ranges from 0 to 1. zero indicates no variation at all and 1 indicates perfect variation. The model building process is based on step wise regression. Model fitting criteria 12

13. Classification matrix: It is a standard tool for evaluation of statistical model. It compares actual to predicted values for each predicted state. It is an important tool for assessing the results of prediction because it makes it easy to understand and account for the effects of wrong predictions. By viewing the amount and percentages in each cell of this matrix , you can quickly see how often the model predicted accurately. Model validation 13

14. Example Analysis And Interpretation 14

15. W hile entering high school, students make program choices among general program , vocational program and academic program . Their choice might be modeled using their writing score (Write) and their social economic status ( Ses ) . The data set contains variables on 200 students. The outcome variable is  prog , program type. The predictor variables are social economic status,  ses , a three-level categorical variable as low (1), medium (2) and high (3) and writing score,  write , a continuous variable. data.csv 15 Example

16. Analysis procedure in SPSS 1 16

17. 3 2 17

18. 4 5 18

19. N Marginal Percentage prog academic 105 52.5 general 45 22.5 vocation 50 25.0 ses 1.00 47 23.5 2.00 95 47.5 3.00 58 29.0 Valid 200 100.0 Missing Total 200 Table1: Case processing summary 19 Source: Author’s calculations

20. Model Model Fitting Criteria Likelihood Ratio Tests -2 Log Likelihood Chi-Square df Sig. Intercept Only 254.986 Final 206.756 48.230 6 0.000 Table 2: Model Fitting Information H o : There is no significance difference between null model and final model sig. p value < 0.05 , reject null hypothesis. The likelihood ratio chi-square of 48.23 with a p-value < 0.0001 tells us that our model as a whole fits significantly better than an empty model  If it is not significant we will stop the analysis here it self. 20 Source: Author’s calculations

21. Chi-Square df Sig. Pearson 119.766 120 0.489 Deviance 129.875 120 0.254   Table 3: Goodness-of-Fit Cox and Snell 0.214 Nagelkerke 0.246 McFadden 0.118 Table 4: Pseudo R-Square H o : The model is adequately fit the data sig. p value > 0.05 , accept null hypothesis. 21 Source: Author’s calculations

22. Effect Model Fitting Criteria Likelihood Ratio Tests -2 Log Likelihood of Reduced Model Chi-Square df P value Intercept 206.756 a 0.000 . Write 238.203 31.447 2 0.000 Ses 217.815 11.058 4 0.026   Table 5: Likelihood Ratio Tests This table shows which of the independent variables are statistically significant. You can see that write was statistically significant because  p  =0.000 (<0.05) . On the other hand, the ses variable was statistically significant because  p  = .026 (<0.05). There is not usually any interest in the model intercept. This table is mostly useful for nominal independent variables because it is the only table that considers the overall effect of a nominal variable, unlike the  Parameter Estimates  table, as shown in next slide. 22 Source: Author’s calculations

23. Academic as a base category coefficient Std. Error Wald statistic df P value general Intercept 1.689 1.227 1.896 1 0.169 write - 0 .058 0.021 7.320 1 0.007 [ses=1.00] 1.163 0.514 5.114 1 0.024 [ses=2.00] 0.630 0.465 1.833 1 0.176 [ ses =3.00] b . . . vocation Intercept 4.236 1.205 12.361 1 0.000 write - 0 .114 0.022 26.139 1 0.000 [ses=1.00] 0.983 0.596 2.722 1 0.099 [ses=2.00] 1.274 0.511 6.214 1 0.013 [ses=3.00] b . . . Table 6: Parameter estimation b. This parameter is set to zero because it is redundant. 23 Source: Author’s calculations

24. The two equations :   A one-unit increase in the variable  write  is associated with a 0.058 decrease in the relative log odds of being in general program versus academic program . A one-unit increase in the variable  write  is associated with a 0.114 decrease in the relative log odds of being in vocation program versus academic program. The relative log odds of being in general program versus in academic program will increase by 1.163 if moving from the highest level of  ses  ( ses = 3) to the lowest level of  ses  ( ses = 1). 24

25. Observed frequency Predicted academic general vocation Percent Correct academic 92 4 9 87.6 general 27 7 11 15.6 vocation 23 4 23 46.0 Overall Percentage 71.0% 7.5% 21.5% 61.0 Table 7: Classification matrix 25 Source: Author’s calculations

26. 26 CASE STUDIES

27. Case study - 1 An econometric analysis of farmer’s credit issues in Andhra Pradesh, India (with reference to south coastal Andhra – a multinomial logit regression model ) Srinivasa R.P Methodology Study area: Andhra Pradesh (Guntur and Prakasam district) Sample size: 50 Dependent variables The dependent variable of the model is the households’ choice of approaches for borrowing from different sources . 1. Institutional Sources 2. Both Institutional and Non-institutional Sources 3. Friend and Relatives 4. Borrowing from money lender (Non institutional source) alone = Reference category I ndependent variable X 1 = Age of the head of the household X 2 = Sex as binary (Male-1, Female-0) X 3 = Literacy status as binary (Illiterate-1, literate-0) X 4 = Type of Ownership as binary (Tenancy-1, Own-0) X 5 = Income from other than Agriculture X 6 = Gross Agriculture Income X 7 = Farm size X 8 = Family Size 27

28. 28 Table 9 : Factors influencing the sources of borrowing with outcome of institutional sources Note: *indicates five percent level of significance. Reference/base category: Borrowing from money lender (Non institutional source) alone

29. 29 Table 8 : Factors influencing the sources of borrowing with outcome of institutional and non-institutional sources Note: *indicates five percent level of significance Reference/base category: Borrowing from money lender (Non institutional source) alone

30. 30 Table 10: Factors influencing the sources of borrowing with outcome of relatives and friends Note: *indicates one percent level of significance ** indicates five percent level of significance *** indicates ten percent level of significance Reference/base category : Borrowing from money lender (Non institutional source) alone Number of observation = 100 LR chi2 (24) = 83.77 Prob > chi2 = 0.0000 Psedo R 2 = 0.3373 Log Likelihood = -82.2961

31. Multinomial Logistic Regression Model in Identifying Factors of m4agriNEI in CSA Innovations SINGH, S.P., SING, R.J., CHAUHAN, J.K., RAM SINGH AND HEMOCHANDRA, L Methodology The study was conducted in four project districts viz . Ri-bhoi , East Khasi Hills, West Khasi Hills and West Jaintia Hills districts of Meghalaya. Sample size: 65 farmers Independent and Dependent variables The study includes a set of independent variables ( Timeliness’, ‘Accuracy’, ‘Relevancy’, ‘Economy’ and ‘Completeness’ of information of AAS (Agro Advisory Services) of m4agriNEI to understand the extent and differentials in the level of adaptation intention in enhancing CSA (Climate Smart Agriculture) innovation by the registered farmers. The study embraces ‘Adaptation Intension in enhancing CSA (Climate Smart Agriculture) innovation by the registered farmers’ as dependent variable (Low, medium and High adoption intensions) . Case study - 2 31

32. Results Table 11 : M odel fitting information Model Model Fitting Criteria -2 Log Likelihood Chi- Square df Sig. Intercept Only 108.907       Final 56.007 52.901*** 22 .001 (*** p <0.01) Table 12: Pseudo R square Cox and Snell R2 Nagelkerke R2 0.557 0.633 32 H0: There was no significant difference between null model and the final model

33. Results Table 13: Relationship of independent variables and competency level of farmers using Likelihood Ratio Tests Effects 2 Log Likelihood of Reduced Mode Chi- Square df Sig. Intercept 56.007 0.00 .00 Timeliness 74.009*** 18.003 4 .001 Economy 70.708** 14.702 6 .023 Relevancy 64.224* 8.217 4 .084 Accuracy 72.229*** 16.292 4 .003 Completeness 63.343 7.337 4 .119 (*** p <0.01, **p < 0.05 and *p < 0.10) 33

34. If the number of observations is lesser than the number of features, MLR should not be used, otherwise, it may lead to over fitting Non linear problems can't be solved with logistic regression since it has a linear decision surface The major limitation of MLR is assumption of linearity between the dependent and independent variables Limitations of Multinomial Logistic Regression 34

35. The usage of the MLR model gives the opportunity to deal with a response categorical variable with more than two levels and variety of explanatory variables . MLR indicates the effect of each of explanatory variables as well as its additive effect by used in the analysis The logistic regression model is a suitable model to many types of data when the response variable with more than two categories. MLR has no any restrictions about the explanatory variables; this model is most common in the categorical data analysis. MLR can be used in many areas of social, educational, health, behavioral and even scientific experiments. 35 Conclusion

36. 36 Suggestions: Dr . P.S Srikantha Murthy Can this model be used to solve the problems affecting the agriculture? Any examples? Yes, explained in slide number 8 Are there any studies by students/faculties of UAS-Bengaluru has been used model? To analyze the influence of different factors on decision pattern of decision making while adopting new innovations by the farmers (Naveen Kumar G.S., 2018) Limitations of Multinomial Logistic Regression? Explained in Slide no 34 2 . Dr . K.B Umesh Include Economic content in the topic? With the help of case studies, I tried to explained how multinomial logistic regression used in agriculture sector

37. Reference: SAMWEL, N., MWENDA, ANTHONY, K. W. AND ANTHONY, G. W., 2015, Analysis of Tobacco Smoking Patterns in Kenya Using the Multinomial Logit Model. American Journal of Theoretical and Applied Statistics, 4 (3):89-98. TAMURA, K. A. AND GIAMPAOLI, V., 2010, Prediction in multilevel logistic regression. Communications in Statistics - Simulation and Computation, 39 : 1083-1096. GRILLI, L. AND RAMPICHINI, C., 2007, A multilevel multinomial logit model for the analysis of graduates’ skills. Statistical Methods and Applications. 16: 381-393. DIAZ, M. M. AND ONES, V. G., 2005, Estimating multilevel models for categorical data via generalized least squares. Revista Colombiana de Estadística . 28 : 63-76. Data source: Institute for Digital Research and Education 37

38. 38