Lecture 5 Variable Selection, Categorical Input Consolidation, Adjusting for Separate Sampling for handling Imbalanced Data.pdf

Lecture 5 Variable Selection, Categorical Input Consolidation, Adjusting for Separate Sampling for handling Imbalanced Data.p

1. Variable Selection

2. ❑Redundantorirrelevantinputsareremovedfromthetrainingdatasettoreduce
overfittingandimprovespredictionperformance.
❑Somepredictionmodeldonotincludemethodsforselectinginputs.Therefore,
separateSASEnterpriseMinertoolisusedforinputselection.
❑Alternativesforinput(variable)selection:
1)RegressionNode-Sequentialselection
2)VariableSelectionNode-UnivariateandForwardselection(R-square)
-Tree-likeselection(Chi-Square)
3)PartialLeastSquaresNode–VariableImportanceintheProjection
4)DecisionTreeNode–SplitSearchselection

3. ❑UsingRegressionInputSelection:
❖TheRegressionNodeinSASE-Minerprovidesthreesequentialselectionmethods.
❖ForwardSelection–createsasequenceofmodelsofincreasingcomplexity.Itbegins
withanemptymodelandaddsinvariablesonebyone.Ineachforwardstep,a
variable(withsmallp-value)thatgivesthesinglebestimprovementisaddedtothe
model.
❖BackwardElimination–createsasequenceofmodelsofdecreasingcomplexity.
Backwardeliminationbeginswithallvariablesselectedandeliminatesvariablesone
atatimeuntilastoppingcriterionisreached(Stopwhenallp-values<??????
????????????????????????).
❖StepwiseSelection–combinestheelementsfromboththeforwardandbackward
selectionprocedures.Themethodbeginsinthesamewayastheforwardprocedure,
sequentiallyaddinginputs.Allcandidatevariablesinthemodelarecheckedtoseeif
theirsignificancehasbeenreducedbelowthespecifiedtolerancelevel.Ifa
nonsignificantvariableisfound,itisremovedfromthemodel.
❖Stepwiseregressionrequirestwosignificancelevels:oneforaddingvariablesandone
forremovingvariables.Thecutoffprobabilityforaddingvariablesshouldbelessthan
thecutoffprobabilityforremovingvariablessothattheproceduredoesnotgetinto
aninfiniteloop.

4. ❑UsingVariableSelectionforInputSelection:
❖TheVariableSelectiontoolprovidesaselectionbasedononeofthetwocriteria.
❖R-SquaredCriterion–isatwo-stepprocess.
Step1:Thesquaredcorrelationforeachvariableiscomputed.Variableswithvalueless
thanthesquaredcorrelationcriterionisrejected.(DefaultR-squares=0.005).
Step2:SASE-MinerevaluatestheremainingvariablesusingforwardstepwiseR-squared
regression.VariablewithstepwiseR-squaredimprovementlessthanthresholdcriterion
(default=0.0005)isrejected.
❖Chi-SquaredSelectionCriterion–variableselectionisperformedusingbinarysplitsfor
maximizingthechi-squaredvalue.Theapproachisquitesimilartodecisiontree
algorithmwithitsabilitytodetectnonlinearrelationshipbetweeninputsandtarget.
However,themethodforhandlingcategoricalinputsmakesitsensitivetospurious
input/targetcorrelation.

5. ❑UsingPartialLeastSquares(PLS)InputSelection:
❖Thegoalistohavelinearcombinationoftheinputsthataccountforvariationinboth
theinputsandtarget.AusefulfeaturesofthePLSprocedureistheinclusionof
variableimportanceintheprojection(VIP).VIPquantifiestherelativeimportanceof
theoriginalinputvariablestothelatentvectors(linearcombinationofinputs).

6. ❑UsingDecisionTreeInputSelection:
❖Decisiontreescanbeusedtoselectinputsforflexiblepredictivemodels.
❖The Importance column quantifies the
overall variability in the target each
input explains.
❖The variable Importance not only
consider inputs appear in tree but also
accounts for surrogate inputs.
Notes:
Surrogate inputs are correlated with the
selected split input. Advantage of
adding surrogates in the variable
selection is to enable inclusion of inputs
that do not appear in tree but still
important predictors for the target.

7. Categorical Input
Consolidation

8. ❑CategoricalInputConsolidationistechniqueofcombiningcategoricalinputlevelsthat
havesimilarprimaryoutcomeproportions.
❑DecisionTreemethodcaneasilygroupedthedistinctlevelsofthecategoricalvariable
togetherandproducegoodpredictions.
❑InSASE-Miner,theDecisionTreenodeisconnectedtotheImputenode.Now,the
DecisionTreenodeisknownasConsolidationTreenode.
❑ThegroupingcanbedonebysimplyrunningtheConsolidationTreenodeoryoumay
changethefeaturesofthenode.

9. Adjusting for Separate
Sampling for handling
Imbalanced Data

10. ❑Imbalancedclassesareacommonprobleminmachinelearningclassificationwhere
thereareadisproportionateratioofobservationsineachclass.
❑Classimbalancecanbefoundinmanydifferentareasincludingmedicaldiagnosis,
spamfiltering,andfrauddetection.Example:IndatasetCreditCardFraudDetection
BankA,only0.17%ofthecustomerarecategoriesasFraud.
❑Proportionatesamplingcouldnotbeusedbecausethenumberofobservations
betweenoutcomesarebiased.Separatesamplingisapplied.
❑Inseparatesampling,samplesaredrawnseparatelybasedonthetargetoutcome.
❑Separatesamplingenablesustoobtainamodelofsimilarpredictivepowerwitha
smalleroverallcasecount.