Panoptic Segmentation was introduces at ECCV 2018 by FAIR (Facebook AI Research). It's gaining the popularity and there were a few papers presented at one of the biggest computer vision conference, CVPR 2019. This slide contains the descriptions of panoptic segmentation networks presented at CVP...
Panoptic Segmentation was introduces at ECCV 2018 by FAIR (Facebook AI Research). It's gaining the popularity and there were a few papers presented at one of the biggest computer vision conference, CVPR 2019. This slide contains the descriptions of panoptic segmentation networks presented at CVPR 2019, as well as the description of Panoptic Segmentation.
Panoptic Segmentation
Semantic Segmentation can:
-Segment instances
without boundaries
-Segment every pixel in the
input image
Instance Segmentation can:
-Segment instance class
with boundaries
-Segment object in the RoI
(Region of Interest)
A. Kirillov, K. He, R. Girshick, C. Rother, and P. Dolla ́r. Panoptic segmentation
Panoptic Segmentation
Every instance that belongs to things (people,
cars, etc.) needs to be identified (instance
segmentation), while every class that belongs to
stuff class (sky, road, etc.) needs to be correctly
classified (semantic segmentation)
A. Kirillov, K. He, R. Girshick, C. Rother, and P. Dolla ́r. Panoptic segmentation
Panoptic Segmentation
●FCN (Fully Convolutional Network) and DC
(Dilated Convolution) are widely used in high
precision semantic segmentation networks
●Each pixel is classified by producing the output
feature map with the same image shape, except
the depth channel
●The first part of the network produces class
agnostic boxes (RoIs), which then will be
classified by the second part of the network
●Box refinement and pixel classification will be
applied for each RoI produced by the RPN
(Region Proposal Network)
J. Long, E. Shelhamer, and T. Darrell. Fully convolutional
networks for semantic segmentation
K. He, G. Gkioxari, P. Dolla ́r, and R. Girshick. Mask R- CNN
Panoptic Segmentation
●FCN (Fully Convolutional Network) and DC
(Dilated Convolution) are widely used in high
precision semantic segmentation networks
●Each pixel is classified by producing the output
feature map with the same image shape, except
the depth channel
●The first part of the network produces class
agnostic boxes (RoIs), which then will be
classified by the second part of the network
●Box refinement and pixel classification will be
applied for each RoI produced by the RPN
(Region Proposal Network)
J. Long, E. Shelhamer, and T. Darrell. Fully convolutional
networks for semantic segmentation
K. He, G. Gkioxari, P. Dolla ́r, and R. Girshick. Mask R- CNN
Panoptic segmentation network is difficult
to design, because the architectures differ!
Panoptic Feature Pyramid Network
Any prediction that
has an IoU with a GT
object greater than
0.5 is considered a
TP
Class prediction needs to
be the same as the GT
class, hence it’s an FP
RQ (Recognition Quality) is the F1
score for the instance segmentation
network, while SQ (Semantic
Quality) is the mIoU of the TP
segments.
A. Kirillov, K. He, R. Girshick, C. Rother, and P. Dolla ́r. Panoptic segmentation