Discriminant analysis using spss

Linear & Multiple discriminant analysis using spss

1. MULTIVARIATE
ANALYSIS
-Dr Nisha Arora

2. About Me Concepts
How it Works?
Q/A Session
Agenda

3. •Dr. Nisha Arora is a proficient educator, passionate trainer,
You Tuber, occasional writer, and a learner forever.
✓ PhD in Mathematics.
✓ Works in the area of Data Science, Statistical
Research, Data Visualization & Storytelling
✓ Creator of various courses
✓ Contributor to various research communities and
Q/A forums
✓ Mentor for women in Tech Global
3
About Me
An educator by heart & a
trainer by profession.

4. http://stats.stackexchange.com/users/79100/learner
https://stackoverflow.com/users/5114585/dr-nisha-arora
https://www.quora.com/profile/Nisha-Arora-9
https://www.researchgate.net/profile/Nisha_Arora2/contributions
http://learnerworld.tumblr.com/
https://www.slideshare.net/NishaArora1
https://scholar.google.com/citations?user=JgCRWh4AAAAJ&hl=en&authuser=
1
https://www.youtube.com/channel/UCniyhvrD_8AM2jXki3eEErw
https://groups.google.com/g/dataanalysistraining/search?q=nisha%20arora
https://www.linkedin.com/in/drnishaarora/detail/recent-activity/posts/
✓Research Queries
✓Coding Queries
✓Blog Posts
✓Slide Decks
✓My Talks
✓Publications
✓Lectures
✓Layman’s Term
Explanation
✓Mentoring
✓Articles & Much More
My Contribution to the Community

5. ❖ Statistics
❖ Data Analysis
❖ Machine Learning
❖ Analytics & Data Science
❖ Data Visualization & Storytelling
❖ Mathematics & Operations Research
❖ Online Teaching
❖ Excel/SPSS/R/Python/Shiny
❖ Tableau/PowerBI
My Expertise

6. Connect With Me
HTTPS://WWW.LINKEDIN.COM/IN/DRNISHAARORA /
[email protected] .

7. Discriminant Analysis
USING SPSS

8. My answer to ‘classification of multiple outcomes
with categorical and continuous predictors’:
https://stats.stackexchange.com/a/513616/79100

9. When to use LDA?
✓Non-Ordinalresponsevariable
✓Metricpredictors
✓Workswellforlowsamplesize
✓Workswellwhencasesarewellseparable
✓Morerestrictivethanlogisticregression

10. Assumptions of LDA
✓BothLDAandQDAassumethepredictorvariablesXare
drawnfromamultivariateGaussiandistribution.
✓LDAassumesequalityofcovariancesamongthepredictor
variablesXacrosseachalllevelsofY
✓LDAandQDArequirethenumberofpredictorvariables(p)to
belessthenthesamplesize(n).Asimpleruleofthumbisto
useLDA&QDAondatasetswheren≥5p

11. Default Prediction
✓Theinformationon700pastcustomersiscontainedin
bankloan.sav
✓Thesearethecustomerswhowerepreviouslygivenloans.
✓Usearandomsampleofthese80%customerstocreatea
discriminantanalysismodel,settingtheremainingcustomers
asidetovalidatetheanalysis.
✓Thenusethemodeltoclassifytheremaining20%prospective
customersasgoodorbadcreditrisks.

12. Data Preparation for LDA
Toreplicatemy
results

13. Data Preparation for LDA
Creating anew
variablefortraining
andvalidationset

14. Discriminant Analysis
Analyze→Classify→
Discriminant

15. Discriminant Analysis
✓Groupingvariable–
Categoricalresponse
variable
✓DefineRange–Asper
numberofcategories
[seecodinginvariable
view]
✓Independents–Metric
predictors
✓How to choose
predictors–
✓Domainknowledge
✓Previousresearch
✓EDA
✓Step-wise

16. Discriminant Analysis
Forhold-out/validation
set
Useselectionvariable
Selectforvalue‘1’

17. Discriminant Analysis
Statisticssub-dialogbox
UnivariateANOVAs
Box’sMand
Fisher’sstandardizescore
willbeusedforreportingthe
results.

18. Discriminant Analysis
Classifysub-dialogbox
Almostalwayscheck
‘Computefromgroupsizes’

19. Discriminant Analysis
Savesub-dialogbox

20. Caseprocessingsummary
✓Nomissingvalues
✓Heremodelistrainedon566observations
&134areunselectedcases(hold-outset)

21. GroupStatistics
Observeifvariablesarediscriminatingthe
response
Largerstdindicatesissueswithpredictors,
specificallyincome&debttoincomeratio.
Youmaywanttotransformthesepredictors

22. Testofequalityofgroupmeans
✓Allpredictorsarecontributingtothemodel
excepthouseholdincome
✓Wilk’slambdavalues(unexplainedvarationin
eachpredictorbygroupsofresponsevariable)
ishigher
✓ThetablesuggeststhatDebttoincomeratio
(x100)isbest,followedbyYearswithcurrent
employer,Creditcarddebtinthousands,and
Yearsatcurrentaddress,andthenotherdebts

23. Pooled within-group matrices
✓Multi-colinearitymaybe
anissue
✓Look fordifferences
betweenthestructure
matrixanddiscriminant
functioncoefficientstobe
sure.

24. Box Test
Box'sMtests
NullHypothesis:Equalityofcovariances
acrossgroups
P-value<alpha(0.05)
NullRejected
Useseparatematricestoseeifitgives
radicallydifferentclassificationresults.
We will see using separate groups covariance
matrices later

25. Summary of Canonical Discriminant Functions
✓EigenValue-Higherthebetter
✓CanonicalCorrelation-Pearson'scorrelationbetween
thediscriminantscoresandthegroups.Higherthe
better
✓Wilks'lambda-Itmeasureshowwelleachfunction
separatescasesintogroups.
SmallervaluesofWilks'lambdaindicategreater
discriminatoryabilityofthefunction.
✓Associatedchi-squaretests-Null:themeansofthe
functionslistedareequalacrossgroups
P-value<0.05thediscriminantfunctiondoesbetterthan
chanceatseparatingthegroups.

26. Standardized canonical DF coefficient
Coefficientswith large
absolutevaluescorrespondto
variableswith greater
discriminatingability
Differentorderinbothtables
indicatescollinearityor
presenceofoutlier
Insuchcase,it’ssafetouse
structurematrix

27. Standardized canonical DF coefficient
SameOrder

28. Canonical Discriminant Function Coefficients
Used for writing equation & computing
discriminant function for each predictor

29. Functions at Group Centroids
Used for determining cut-off value

30. Discriminant Analysis_ Outputs
Classificationfunctions
✓Theclassificationfunctionsareusedtoassigncasestogroups.
✓Thereisaseparatefunctionforeachgroup.Foreachcase,a
classificationscoreiscomputedforeachfunction.
✓Thediscriminantmodelassignsthecasetothegroupwhose
classificationfunctionobtainedthehighestscore.

31. Discriminant Analysis _Output
Thewithin-groupscorrelationmatrix
showsthecorrelationsbetweenthe
predictors.Thelargestcorrelations
occurbetweenCreditcarddebtin
thousandsandtheothervariables,but
itisdifficulttotelliftheyarelarge
enoughtobeaconcern.Lookfor
differencesbetweenthestructure
matrixanddiscriminantfunction
coefficientstobesure.

32. Discriminant Analysis _Output
Box'sMteststheassumptionofequalityofcovariances
acrossgroups.Logdeterminantsareameasureofthe
variabilityofthegroups.Largerlogdeterminants
correspondtomorevariablegroups.Largedifferencesin
logdeterminantsindicategroupsthathavedifferent
covariancematrices.
SinceBox'sMissignificant,youshouldrequestseparate
matricestoseeifitgivesradicallydifferentclassification
results.Seethesectiononspecifyingseparate-groups
covariancematricesformoreinformation.

33. Discriminant Analysis _Output
Thereareseveraltablesthatassess
thecontributionofeachvariabletothe
model,includingthetestsofequalityof
groupmeans,thediscriminantfunction
coefficients,andthestructurematrix

34. Discriminant Analysis _Output
Thestandardizedcoefficientsallowyou
tocomparevariablesmeasuredon
differentscales.Coefficientswithlarge
absolutevaluescorrespondtovariables
withgreaterdiscriminatingability.
Thistabledowngradestheimportance
ofDebttoincomeratio(x100),butthe
orderisotherwisethesame.

35. Prior Probabilities for Groups
Apriorprobabilityisanestimateofthe
likelihoodthatacasebelongstoa
particulargroupwhennoother
informationaboutitisavailable

36. Classification Function Coefficients
These areusedtocompute
probabilitiesforgroupmembership.

37. Classification Results
Trainingsetaccuracy=82.2%
Validationsetaccuracy=78.4%

38. How to improve model
Usevariableselection
InSPSS,step-wisemethod
Useseparatecasecovariancematrix

39. Discriminant Analysis
SinceBox'sMissignificant,it'sworth
runningasecondanalysistosee
whetherusingaseparate-groups
covariancematrixchangesthe
classification.

40. Discriminant Analysis _Output
Thestructurematrixshowsthecorrelationofeachpredictorvariablewiththe
discriminantfunction.Theorderinginthestructurematrixisthesameasthatsuggested
bythetestsofequalityofgroupmeansandisdifferentfromthatinthestandardized
coefficientstable.ThisdisagreementislikelyduetothecollinearitybetweenYearswith
currentemployerandCreditcarddebtinthousandsnotedinthecorrelationmatrix.
Sincethestructurematrixisunaffectedbycollinearity,it'ssafetosaythatthis
collinearityhasinflatedtheimportanceofYearswithcurrentemployerandCreditcard
debtinthousandsinthestandardizedcoefficientstable.Thus,Debttoincomeratio
(x100)bestdiscriminatesbetweendefaultersandnondefaulters.

41. Discriminant Analysis _Output
Inadditiontomeasuresforchecking
thecontributionofindividual
predictorstoyourdiscriminantmodel,
theDiscriminantAnalysisprocedure
providestheeigenvaluesandWilks'
lambdatablesforseeinghowwellthe
discriminantmodelasawholefitsthe
data.

42. Discriminant Analysis _Output
Theeigenvaluestableprovides
informationabouttherelativeefficacy
ofeachdiscriminantfunction.When
therearetwogroups,thecanonical
correlationisthemostusefulmeasure
inthetable,anditisequivalentto
Pearson'scorrelationbetweenthe
discriminantscoresandthegroups.

43. Discriminant Analysis _Output
Wilks'lambdaisameasureofhowwelleachfunction
separatescasesintogroups.Itisequaltothe
proportionofthetotalvarianceinthediscriminant
scoresnotexplainedbydifferencesamongthegroups.
SmallervaluesofWilks'lambdaindicategreater
discriminatoryabilityofthefunction.
Theassociatedchi-squarestatisticteststhehypothesis
thatthemeansofthefunctionslistedareequalacross
groups.Thesmallsignificancevalueindicatesthatthe
discriminantfunctiondoesbetterthanchanceat
separatingthegroups.

44. Discriminant Analysis _Output
The classification table shows the practical
results of using the discriminant model. Of
the cases used to create the model, 94 of
the 124 people who previously defaulted are
classified correctly. 281 of the 375
nondefaultersare classified correctly.
Overall, 75.2% of the cases are classified
correctly.
Classifications based upon the cases used to
create the model tend to be too "optimistic"
in the sense that their classification rate is
inflated. The cross-validated section of the
table attempts to correct this by classifying
each case while leaving it out from the
model calculations; however, this method is
generally still more "optimistic" than subset
validation.
Subset validation is obtained by classifying

45. Discriminant Analysis _Output
Restalltablesaresame.
Theclassificationresultshavenot
changedmuch,soit'sprobablynot
worthusingseparatecovariance
matrices.Box'sMcanbeoverly
sensitivetolargedatafiles,whichis
likelywhathappenedhere.

46. Using z-scores

47. Get your hands dirty!
Playaroundwithdifferentmodels&seewhatworksbestforyourproblem

48. How to report the results?
1.ANOVATable[univariateanovainstatisticssubdialogbox)
relationofindividualpredictor
2.BOXM(Assumptionchecking)
it'snotverystrongmeasure...forlargesample,mostly,itgivespvalue>
0.05
3.Performance(EigenValue,Wilkslambda,Classificationtable)
4.Discriminantequations&centroidscores
5.Relativeimportance

49. Multiple Discriminant Analysis
Dataset:Iris.sav
Responsevariable=Species_3categories(storedasstringvariable)
Iris-setosa,Iris-versicolor,andIris-virginica
Alternatetechnique:Multonominallogisticregression
ForMDA,weneedtoconvertstringvariabletocategoricalvariable
STEPS:ClickTransform>AutomaticRecode.
Double-clickvariableStateintheleftcolumntomoveittotheVariable->NewNamebox.
Enteranameforthenew,recodedvariableintheNewNamefield,thenclickAddNewName.
ChecktheboxforTreatblankstringvaluesasuser-missing.
ClickOKtofinish.

50. Group Statistics

51. Box M
No need for separate group

52. Summary of Canonical Discriminant Functions

53. Variable Importance

54. Canonical Discriminant Functions

55. Functions at Group Centroids

56. Prior Probabilities for Groups

57. Classification Function Coefficients

58. Classification Results
Using leave-one-out cross validation

59. Combined Group Plot

60. Thank You