Image segmentation

Image Segmentation
Types of Image Segmentation
Semantic Segmentation
Instance Segmentation
Types of Image Segmentation Techniques based on the image properties:
Threshold Method.
Edge Based Segmentation.
Region-Based Segmentation.
Clustering Based Segmentation.
Watershed Based Method.
Artificial Neural Network Based Segmentation.

1. Image Segmentation
P. Kuppusamy

2. Content
•ImageSegmentation
•TypesofImageSegmentation
•SemanticSegmentation
•InstanceSegmentation
•TypesofImageSegmentationTechniquesbasedontheimage
properties:
•ThresholdMethod.
•EdgeBasedSegmentation.
•RegionBasedSegmentation.
•ClusteringBasedSegmentation.
•WatershedBasedMethod.
•ArtificialNeuralNetworkBasedSegmentation.

3. ImageSegmentation
•ImageSegmentationisfindingthegroupofpixelsthataresimilar.
•Itispartitioningtheimageintovarioussubgroups(ofpixels)calledImage
Objects.
•Assigninglabelstopixels,andthepixelswiththesamelabelfallundersingle
category.
•Itcanreducethecomplexityoftheimagetoanalysetheimagebecomessimpler.
•Usingtheselabels,Usercanspecifyboundaries,drawlines,andseparatethe
mostrequiredobjectsinanimagefromunnecessarypixelslabel.
•Algorithmdetectsaninstances,whichprovidesaboutindividualobjects,and
hencetheallthepersonshavedifferentcolors.

4. TypesofImageSegmentation
1.SemanticSegmentation-Inimage1,everypixelbelongstoaparticularclass
(eitherbackgroundorperson).Also,allthepixelsbelongingtoaparticular
classarerepresentedbythesamecolor(backgroundasblackandpersonas
pink).
2.InstanceSegmentation-Image2hasalsoassignedaparticularclasstoeach
pixeloftheimage.However,differentobjectsofthesameclasshavedifferent
colors(Person1asred,Person2asgreen,backgroundasblack,etc.).

5. WhyImageSegmentationisNeeded?
1.Identifytheobjectsfromimage.
2.Notonlyidentifying,AlsoProvidesmoreinsightinformationaboutthe
objects.
Applications
•facialrecognition-identifyinganemployeetomarkattendance
automatically.
•Medicalindustry-fasterdiagnosis,detectingdiseases,tumors,celland
tissuepatternsfromradiography,MRI,endoscopy,thermography,
ultrasonography,etc.
•Satelliteimages-identifyvariouspatterns,objects,geographicalcontours,
soilinformationetc.,whichcanbeusedforagriculture,mining,geo-sensing,
etc.
•Robotics,likeRoboticprocessautomation(RPA),self-drivingcars,etc.

6. TypesofImageSegmentationTechniquesbasedontheimage
properties:
Twoapproaches:
1.SimilarityDetection(RegionbasedApproach)
1.Detectingsimilarpixelsinanimagebasedonathreshold,regiongrowing,region
spreading,andregionmerging.
2.Machinelearningalgorithmslikeclusteringreliesonthisapproachofsimilarity
detectiononanunknownsetoffeatures.Itdetectssimilaritybasedonapre-defined
(known)setoffeatures.
2.DiscontinuityDetection(Boundary/EdgebasedApproach)
1.Searchesfordiscontinuity.
2.Here,edgesdetectedbasedonvariousmetricsofdiscontinuitylikeintensity,color
etc.
3.E.g.ImageSegmentationAlgorithmslikeEdgeDetection,PointDetection,Line
Detection

7. TypesofImageSegmentationTechniquesbasedontheimage
properties:
1.Threshold Method.
2.Edge Based Segmentation.
3.Region Based Segmentation.
4.Clustering Based Segmentation.
5.Watershed Based Method.
6.Artificial Neural Network Based Segmentation.

8. Threshold based Segmentation
•ThresholdvalueTisconsideredasaconstant.Butthatapproachmaybeineffective
consideringtheamountofnoiseoftheimage.So,wecaneitherkeepitconstantor
changeitdynamicallybasedontheimagepropertiestoobtainbetterresults.
•GlobalThreshold-Ifwewanttodividetheimageintotworegions(objectand
background),wedefineasinglethresholdvalue.
•LocalThreshold-Ifwehavemultipleobjectsalongwiththebackground,wemust
definemultiplethresholds.Thesethresholdsarecollectivelyknownaslocalthreshold.
Basedonthat,thresholdingisofthefollowingtypes:
1.SimpleThresholding
•Thistechniquereplacesthepixelsinanimagewitheitherblackorwhite.
•Iftheintensityofapixel(I
i,j)atposition(i,j)islessthanthethreshold(T),thenreplace
thatwithblackandifitismore,thenwereplacethatpixelwithwhite.Thisisabinary
approachtothresholding.

9. Threshold based Segmentation
2.Otsu’sBinarization
•Inglobalthresholding,wehadusedanarbitraryvalueforthresholdvalueandit
remainsaconstant.
•Howcanwedefineanddeterminethecorrectnessoftheselectedthreshold?A
simplerbutratherineptmethodistotrialandseetheerror.
•Takeanimagehistogramthathastwopeaks(bimodalimage),oneforthe
backgroundandonefortheforeground.
•AccordingtoOtsubinarization,forthatimage,wecanapproximatelytakea
valueinthemiddleofthosepeaksasthethresholdvalue.
•So,itautomaticallycalculatesathresholdvaluefromimagehistogramfora
bimodalimage.
•Thedisadvantageisforimagesthatarenotbimodal,theimagehistogramhas
multiplepeaks,oroneoftheclasses(peaks)presenthashighvariance.
•However,Otsu’sBinarizationiswidelyusedindocumentscans,removing
unwantedcolorsfromadocument,patternrecognitionetc.

10. Threshold based Segmentation
3.AdaptiveThresholding
•Aglobalvalueasthresholdvaluemaynotbegoodinalltheconditionswherean
imagehasdifferentbackgroundandforegroundlightingconditionsindifferent
actionableareas.
•Weneedanadaptiveapproachthatcanchangethethresholdforvarious
componentsoftheimage.
•Inthis,thealgorithmdividestheimageintovarioussmallerportionsand
calculatesthethresholdforthoseportionsoftheimage.
•Hence,weobtaindifferentthresholdsfordifferentregionsofthesameimage.
•Thisinturngivesusbetterresultsforimageswithvaryingillumination.
•Thealgorithmcanautomaticallycalculatethethresholdvalue.
•Thethresholdvaluecanbethemeanofneighborhoodareaoritcanbethe
weightedsumofneighborhoodvalueswhereweightsareaGaussianwindow(a
windowfunctiontodefineregions).

11. Edge based Segmentation
•Edgedetectionistheprocessoflocatingedgesinanimagebasedonvarious
discontinuitiesingreylevel,colour,texture,brightness,saturation,contrastetc.
•Tofurtherenhancetheresults,supplementaryprocessingstepsmustfollowto
concatenatealltheedgesintoedgechainsthatcorrespondbetterwithbordersin
theimage.
•Edgesconsistofmeaningfulfeaturesandcontainssignificantinformation.
•Itsignificantlyreducesthesizeoftheimageandfiltersoutinformationthatmay
beregardedaslessrelevant,preservingandfocusingsolelyontheimportant
structuralpropertiesofanimageforabusinessproblem.

12. Edge based Segmentation
Edgedetectionalgorithmsaretwocategories:
•GradientbasedmethodsandGrayHistograms.
•Basicedgedetectionoperatorslikesobeloperator,canny,Robert’svariableetc.
•Theseoperatorsaidindetectingtheedgediscontinuitiesandhencemarktheedge
boundaries.
•Theendgoalistoreachatleastapartialsegmentationusingthisprocess,wherewegroup
allthelocaledgesintoanewbinaryimagewhereonlyedgechainsthatmatchthe
requiredexistingobjectsorimagepartsarepresent.

13. Region based Segmentation
•Creatingsegmentsbydividingtheimageintovariouscomponentshaving
similarcharacteristics.
•Region-basedimagesegmentationtechniquesinitiallysearchforsomeseed
points–eithersmallerpartsorconsiderablybiggerchunksintheinputimage.
•Next,Eitheraddmorepixelstotheseedpointsorfurtherdiminishorshrink
theseedpointtosmallersegments,andmergewithothersmallerseedpoints.
Hence,therearetwobasictechniquesbasedonthismethod.
1.RegionGrowing
•It’sabottomtoupmethodwherebeginwithasmallersetofpixelandstart
accumulatingoriterativelymergingitbasedoncertainpre-determinedsimilarity
constraints.
•Regiongrowthalgorithmstartswithchoosinganarbitraryseedpixelinthe
imageandcompareitwithitsneighboringpixels.

14. Region based Segmentation
•Ifthereisamatchorsimilarityinneighboringpixels,thentheyareaddedtothe
initialseedpixel,thusincreasingthesizeoftheregion.
•Whenwereachthesaturation,thegrowthofthatregioncannotproceedfurther.
•So,thealgorithmnowchoosesanotherseedpixel,whichnecessarilydoesnot
belongtoanyregion(s)thatcurrentlyexistsandstarttheprocessagain.
•RegiongrowingmethodsoftenachieveeffectiveSegmentationthatcorresponds
welltotheobservededges.
•Butsometimes,whenthealgorithmletsaregiongrowcompletelybeforetrying
otherseeds,thatusuallybiasesthesegmentationinfavouroftheregionswhich
aresegmentedfirst.
•Tocounterthiseffect,mostofthealgorithmsbeginwiththeuserinputsof
similaritiesfirst,nosingleregionisallowedtodominateandgrowcompletely
andmultipleregionsareallowedtogrowsimultaneously.

15. Region based Segmentation
•Regiongrowth,alsoapixelbasedalgorithmlikethresholdingbutthemajor
differenceisthresholdingextractsalargeregionbasedoutofsimilarpixels,
fromanywhereintheimagewhereasregion-growthextractsonlytheadjacent
pixels.
•Regiongrowingtechniquesarepreferablefornoisyimages,whereitishighly
difficulttodetecttheedges.
2.RegionSplittingandMerging
•Thesplittingandmergingbasedsegmentationmethodsusetwobasictechniques
donetogetherinconjunction–regionsplittingandregionmerging–for
segmentinganimage.
•Splittinginvolvesiterativelydividinganimageintoregionshavingsimilar
characteristicsandmergingemployscombiningtheadjacentregionsthatare
somewhatsimilartoeachother.

16. Region based Segmentation
•Aregionsplit,unliketheregiongrowth,considerstheentireinputimageasthe
areaofbusinessinterest.
•Then,itwouldtrymatchingaknownsetofparametersorpre-definedsimilarity
constraintsandpicksupallthepixelareasmatchingthecriteria.
•Thisisadivideandconquersmethodasopposedtotheregiongrowthalgorithm.

17. Clustering based Segmentation
•Clusteringisdividingthepopulation(datapoints)intoanumberofgroups,suchthatdata
pointsinthesamegroupsaremoresimilartootherdatapointsinthatsamegroupthan
thoseinothergroups(clusters).
k-meansclustering
•Thekrepresentsthenumberofclusters.
1.First,randomlyselectkinitialclusters
2.Randomlyassigneachdatapointtoanyoneofthekclusters
3.Calculatethecentersoftheseclusters
4.Calculatethedistanceofallthepointsfromthecenterofeachcluster
5.Dependingonthisdistance,thepointsarereassignedtothenearestcluster
6.Calculatethecenterofthenewlyformedclusters
7.Finally,repeatsteps(4),(5)and(6)untileitherthecenteroftheclustersdoesnot
changeorwereachthesetnumberofiterations
•Thekeyadvantageofusingk-meansalgorithmisthatitissimpleandeasytounderstand.

18. Watershedbased Segmentation
•Watershedisaridgeapproach,alsoaregion-basedmethod,whichfollowstheconceptof
topologicalinterpretation.
•Itconsiderstheanalogyofgeographiclandscapewithridgesandvalleysforvarious
componentsofanimage.Theslopeandelevationofthetopographyaredistinctly
quantifiedbythegrayvaluesoftherespectivepixels–calledthegradientmagnitude.
•Basedonthis3DrepresentationwhichisusuallyfollowedforEarthlandscapes,the
watershedtransformdecomposesanimageintoregionsthatarecalled“catchmentbasins”.
•Foreachlocalminimum,acatchmentbasincomprisesallpixelswhosepathofsteepest
descentofgrayvaluesterminatesatthisminimum.Thealgorithmconsidersthepixelsasa
“localtopography”(elevation),ofteninitializingitselffromuser-definedmarkers.Then,
thealgorithmdefines“basins”whicharetheminimapointsandhence,basinsareflooded
fromthemarkersuntilbasinsmeetonwatershedlines.
•Thewatershedsareformedthatseparatebasinsfromeachother.Hencethepicturegets
decomposedbecausewehavepixelsassignedtoeachsuchregionorwatershed

19. Artificial Neural Network based Segmentation
•ANNusesAItoautomaticallyprocessandidentifythecomponentsofanimagelike
objects,faces,text,hand-writtentextetc.ConvolutionalNeuralNetworksarespecifically
usedtoidentifyandprocesshigh-definitionimagedata.
•Animageisconsideredeitherasasetofvectors(colourannotatedpolygons)oraraster(a
tableofpixelswithnumericalvaluesforcolors).Thevectororrasteristurnedintosimpler
componentsthatrepresenttheconstituentphysicalobjectsandfeaturesinanimage.

20. Morphological based Segmentation
•Itisforanalysingthegeometricstructureinherentwithinanimage.
•Inthis,theoutputimagepixelvaluesarebasedonsimilarpixelsofinputimage
withisneighboursandproducesanewbinaryimage.
•Thismethodisalsousedinforegroundbackgroundseparation.
•Thebaseofthemorphologicaloperationisdilation,erosion,opening,closing
expressedinlogicalAND,OR.
•Thistechniqueismainlyusedinshapeanalysisandnoiseremovalafter
thresholdinganimage.Example:watershedalgorithm.

21. References
•Richard Szeliski, Computer Vision: Algorithms and Applications,
Springer 2010