【14-C-7】コンピュータビジョンを支える深層学習技術の新潮流

Developers Summit 2019【14-C-7】鮫島様の講演資料です。

1. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Masaki Samejima
Machine Learning Solutions Architect, Amazon Web Services Japan.
2019.2.14
]>Of)="Q]?OD??
^>??=?~?J??m?P`?]'
Developers Summit 2019

2. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Agenda
•]>Of)="Q]?.[0b
•]>Of)="Q]ڧ?m?J??m
•]>Of)="Q]ڞ?k=J??m
•]>Of)="Q]?OD??9f3?????

3. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
]>Of)="Q]?.[0b

4. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
]>Of)="Q]??
•'????kW*???8???G?7 ?E????????
•]>Of)??0ZW+?'???ڒԒ??Ko???????
J??m?]>Of)="Q]
Demographic Data
Facial Landmarks
Sentiment Expressed
Image QualityGeneral Attributes
??-S??(??
'???۬????Ґ?G?
??????0??

5. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
]>Of)="Q]????^>??=?~??Z07
2012
kW*????N
SuperVision[1]
ILSVRC2012+?.g
[1] A. Krizhevsky, et al., Imagenetclassification with deep convolutional neural networks, NIPS 2012.
[2] R Girshick, et al., Rich feature hierarchies for accurate object detection and semantic segmentation, CVPR 2014.
[3] I.J. Goodfellow, et al., Generative Adversarial Nets, NIPS 2014.
[4] V. Badrinarayanan, et al, SegNet: A Deep Convolutional Encoder-Decoder Architecturefor Image Segmentation. PAMI 2017
2014
f?(dU?,?
R-CNN[2]?Pascal
VOC?y5C?0b&s
kW*?k0I?
O??m?k0I?7-
2YfGAN[3]
%K]0f!Q]
>%U/C(X?SC
T]SegNet[4]
2015

6. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
?r?(m??F??I0b&skW*????N
https://gluon-cv.mxnet.io/model_zoo/classification.html
senet_154resnet_v1d
resnet_v1cresnet_v1b
resnet_v1
densenet
darknet
VGG
resnet_v2
mobilenet
mobilenetv2
0.80
0.75
0.70
Accuracy
10002000
L?7?C?
#sample/sec.30004000
•ImageNet????80%?y5C?
•V100 GPU?O?/*SP?L?7?C?
,<?8????KLT$

7. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
?r?(m??F??I0b&sf?(dU?,?
https://gluon-cv.mxnet.io/model_zoo/detection.html
mAP
10 100
L?7?C?
#sample/sec.
,<?8????
KLT$
40
35
30
yolo3
faster_rcnn
ssd
•U?,??D6??Y??QԝZ?(IoU?8???)f?(d
????nmAPy5C??30-40%s?C?
•?Cݫ?ۭr?(m??0b&s

8. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
?r?(m??F??I0b&s%K]0f!Q]
https://gluon-cv.mxnet.io/model_zoo/segmentation.html
0
10
20
30
40
50
60
70
80
90
100
fcn_resnet101psp_resnet101deeplab_resnet101fcn_resnet101psp_resnet101deeplab_resnet101deeplab_resnet152
COCO 1f)%-2VOC 1f)%-2IoU
f?(d???Y????N-=??ˬD6??-?0U?0b&s

9. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
???C?֔??NX#?I
3Y?+??f?(dU?,?[1]
[1] B. Tekin, et al., Real-Time Seamless Single Shot 6D Object Pose Prediction, CVPR 2018.
[2] R. Girdhar, et al., Detect-and-Track: Efficient Pose Estimation in Videos, CVPR 2018.
[3] L. Chen, et al., MaskLab: Instance Segmentation by Refining Object Detection with
Semantic and Direction Features, CVPR 2018.
.[kW?f?(dU?,?e?Ě[2]f?(d???
%K]0f!Q][3]

10. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
GANڞ?k=?D6??KA8?
kW*??k0I?
Noise
kW*??8?L?Text-to-image[3]
(and Image-to-text)8$??kW*??yP?kW*???5?53?[1]
kW*?ڱ?Q*?C?.?[2]
[1] P. Isola, et al., Image-to-Image Translation with Conditional Adversarial Nets, CVPR 2017.
[2] C. Ledig, et al., Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network, CVPR 2017.
[3] S. Reed, et al., Generative Adversarial Text to Image Synthesis, ICML 2016.

11. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
kW*???W+r?]0#2ڔ??N
Saliency (\?na?D6?) ?U?,?[1]
??{?P/0b?U?,????2~'?
(rI???1f)%-2?-Ck=
[1] N. Liu, et al., PiCANet: Learning Pixel-wise Contextual Attention for Saliency Detection, CVPR 2018.
[2] Z. Li, et al., MegaDepth: Learning Single-View Depth Prediction from Internet Photos, CVPR 2018.
kW*??94?dL>@[2]
???֓,C???NFEf??kW*??
94?d??L>@??1f)%-2?(rI?

12. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
?0??1f)????K?J
0
2
4
6
8
10
12
14
16
18
20
1234567891011121314151617181920
??kW*?1f)%-2SPO?
kW*?ID
-??
???
-??????
??(B???
[1] O. Vinyals, et al., Matching Networks for One Shot Learning, arXiv:1606.04080
1f)?C"?/'-?
????????
•)?r??N-=??L1U?(r????5-?
•(B????????-;-=??[1]
Z? ?? ? ??

13. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
O??m?kW*??k0I?Ԩ?E?
•Deep Learning?L1U?H8??8$?D??
•;]*?kW*??8$?.???04 U?-;>@???(?
X. Yuan, et al., Adversarial Examples: Attacks and Defenses for Deep Learning, IEEE Trans Neural NetwLearn Syst.2019.

14. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
]>Of)="Q]ڧ?m?J??m

15. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
]>Of)="Q]ڧ?m?ڔ쬗
•?F??I?UT$J?>K?2ڪ??
•^>??=?~??VfJYf?' u'
•.,hfm??f3?>K?2e1:-?5-??
•1f),?ie.B?nL1U?O?(rV_ژ>K
9VCU>K?2?>Ki
???VfJYf
ONNX????VfJ
Yf???L1U8?L?
AutoML
Define-by-run ???
AWSI]

16. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
9VCU>K?2?>Ki???VfJYf
•RQ+?um?L1U?>K?2?=?~?^e?L1U???B?
•Pf ?L1U?Wf3??n,D=?~?nL?7?03?I
TensorFlow models
TF slim
GluonCVChainerCVPyTorchCV

17. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ResNet>K?2?Z??(Gluon vs MXNet)
num_unit= len(units)
assert(num_unit== num_stages)
data = mx.sym.Variable(name='data')
if dtype== 'float32':
data = mx.sym.identity(data=data, name='id')
else:
if dtype== 'float16':
data = mx.sym.Cast(data=data, dtype=np.float16)
data = mx.sym.BatchNorm(data=data, fix_gamma=True, eps=2e-5, momentum=bn_mom, name='bn_data')
(nchannel, height, width) = image_shape
if height <= 32: # such as cifar10
body = mx.sym.Convolution(data=data, num_filter=filter_list[0], kernel=(3, 3), stride=(1,1), pad=(1, 1),
no_bias=True, name="conv0", workspace=workspace)
else: # often expected to be 224 such as imagenet
body = mx.sym.Convolution(data=data, num_filter=filter_list[0], kernel=(7, 7), stride=(2,2), pad=(3, 3),
no_bias=True, name="conv0", workspace=workspace)
body = mx.sym.BatchNorm(data=body, fix_gamma=False, eps=2e-5, momentum=bn_mom, name='bn0')
body = mx.sym.Activation(data=body, act_type='relu', name='relu0')
body = mx.sym.Pooling(data=body, kernel=(3, 3), stride=(2,2), pad=(1,1), pool_type='max')
for iin range(num_stages):
body = residual_unit(body, filter_list[i+1], (1 if i==0 else 2, 1 if i==0 else 2), False,
name='stage%d_unit%d' % (i+ 1, 1), bottle_neck=bottle_neck, workspace=workspace,
memonger=memonger)
for j in range(units[i]-1):
body = residual_unit(body, filter_list[i+1], (1,1), True, name='stage%d_unit%d' % (i+ 1, j + 2),
bottle_neck=bottle_neck, workspace=workspace, memonger=memonger)
bn1 = mx.sym.BatchNorm(data=body, fix_gamma=False, eps=2e-5, momentum=bn_mom, name='bn1')
relu1 = mx.sym.Activation(data=bn1, act_type='relu', name='relu1')
MXNet>K?2?w&g???x
frommxnet.gluon.model_zooimportvision
resnet18 = vision.resnet18_v1()Gluon >K?2

18. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ONNX (Open Neural Network Exchange)
MXNet
Caffe2
PyTorch
TF
CNTKCoreML
Tensor
RT
NGraph
SNPE•0Ps??VfJYf?Ww ??
L1U?ONNX?8?L?nONNX
??-=??VfJYf?8?L?
03?I
•fA]'f#???nIO
50???ҧ?m??????

19. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ONNX ?,;??WI?
Protocol Buffers ?:4T?U
•];2?.,hfm?
•????AS-2?fJ?.[(r
•API??????%?eRE??%??03
?I
Protocol Buffers
GraphOperatorTensor, …
Operator Definitions
ONNX Python API

20. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Define-and-run ?Define-by-run
•Define-and-run
•7-2Yf?>@~?????1f)?,.??P/D?
•TensorFlow, MXNet?-1R????*,
•Define-by-run
•7-2Yf?>@~?????1f)?,.??P/D?
•Chainer?Lpk=n??E?nPyTorch, TensorFlow, MXNet??\?/?

21. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Define-and-run ?Define-by-run ?(?
Define-and-runDefine-by-run
defour_function(A, B):
C = A + B
returnC
A = Load_Data_A()
B = Load_Data_B()
result = our_function(A, B)
A = placeholder()
B = placeholder()
C = A + B
our_function=
compile(inputs=[A, B], outputs =[C])
A = Load_Data_A()
B = Load_Data_B()
result = our_function(A, B)
https://gluon.mxnet.io/chapter07_distributed-learning/hybridize.html
Define
Run
Define, Run

22. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Define-by-run ?KT-2
Define-and-runDefine-by-run
defour_function(A, B):
C = A + B
returnC
A = Load_Data_A()
B = Load_Data_B()
result = our_function(A, B)
A = placeholder()
B = placeholder()
C = A + B
our_function=
compile(inputs=[A, B], outputs =[C])
A = Load_Data_A()
B = Load_Data_B()
result = our_function(A, B)
1f)?G?>@?˓㓘
?E??w8?.???1f
)?J??????x
8?.??0U??˓㓘e1:-
?03?I

23. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AutoML??
•X#U=?~??>Kk=.???????W+r?)#?=?5?
•1f)?-=ek0I??wf?E?K,??xL1U?J?, etc.
D. Bayor, et al., TFX: A TensorFlow-Based Production-Scale Machine Learning Platform, KDD 2017.

24. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AutoML??
•????>z????E??)#??H.[.???AutoML?pGtT??L?
•ICML 2014??AutoML?Yf!Q-A*?#)f2
•&????pGtT2>-
•9;f;SKf)?RQ??.?
X#U=?~?????n'?J[?????>@?E??
9;f;SKf)?.,hf??Lqz
•Meta-Learning, Learning to learn
X#U=?~??-Ck=???RQ??.?J[\????1f)
??=?~???P/\??w=?~???P/\??=?~????x
* https://sites.google.com/site/automlwsicml14/

25. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AutoML?????X#U=?~?!#0J
?H.[?1f)0??eO?*??H.[?RQ???X#U=?~??>K?d
Pf ?X#U=?~??UT$J?1f)?O?*?P/\??
[a???E????

26. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AutoML?(?Amazon Forecast
User
CSV file
1. S3?1f)-AWf3
2. Forecast ?1f)%-2
?'.^?UT$Jړ?>@
3. Forecast ?'.^?L1UWw
4. L1U???Qy?-(1f)
?'.^?????03??.?
Pf ????RQ(Z??E?????n1f)?-AWf3?-1R???>@

27. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
]>Of)="Q]ڞ?k=J??m

28. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
]>Of)="Q]ڞ?k=ڔ쬗
•=?~???L1U?.,hf??L?7?????
jF7??>?P??Ww ???
•L1U???L?7z`SS?9?E2F???0(???
??
Model Server
L1U];S
Interpretable ML

29. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Model Server
L1U1AWڔ쬗
•X#U=?~?L1U????nL?7T#2?,?i??jF7??E?
•L?7jF7??Ww Y?u???nV0]!?2O??????????
Model Server ?Eu-?
•.,hfڄz?L?7jF7??>?P??Ww 03
•REST/RPC???])f?f#?S]2??-Ck=????
X#U=?~?L1UModel ServerMobile client
DeployREST/RPC

30. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
TensorFlow Serving
[1] C. Olston, et al., TensorFlow-Serving: Flexible, High-Performance ML Serving, NIPS 2017.
•Controller, Synchronizer?zW?nL?7k=?Serving job ???O?k0I?
•Router?nL?7T#2?0???nServing job ם?)K

31. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
MXNetModel Server
https://aws.amazon.com/jp/blogs/news/model-server-for-apache-mxnet-v1-0-released/
•L1Um??.nL?7T
#2ڝ?)K?REST
APIEND?Ӑd??
•MMS 1.0?F??I?0b&s
?n0ZQ1,000%#
??,?,?i03?I
MMS 1.0
MMS 0.4

32. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
^>??=?~?L1U];S
•^>??=?~??L1U?>K?d??9f3
?F??L1U?8?L??nL
1U?L?7?C??0b&s
•];UE??L1Uۛꡀ?S]
)J?>K?d03?I?n$?8??
?^>??=?~??VfJYf?&v??
•];S?(?
•AWS, SageMaker Neo
•Nvidia, TensorRT
Raspberry Pi?L?7Q??
ResNet18 Mobilenet
11.5x2.2x

33. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
SageMaker Neo / TVM ???RQ??.?
•Operator Fusion
`4v?????????ұ?.?
•Data Layout Transformation
4x4ڐd-(`4v???d?7%0U?n1f)?4x4 ?)Ug???
•Tensor Expression and Schedule Space
.,hf.??????(ZVCU֐?i?8?L?n`4v??#"OfT]
•Nested Parallelism with Cooperation
0P#V-3?1f)?/?.??0?אd?n,RcKLT??????
•etc…
T. Chen, et al., TVM: An Automated End-to-End Optimizing Compiler for Deep Learning, OSDI 2018.

34. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
TensorRT???RQ??.?
•Layer & Tensor Fusion
??O??VN???,?i?1??VN?,?i?????
•FP16 and INT8 Precision Calibration
FP32?'????FP16?INT8?-Ck=??];2??n`4v????.?
•Kernel Auto-Tuning
+Of5]???f7U??H.[?J??n(???nk??%??`4v??RQ??.?
•Dynamic Tensor Memory
KLT?-Ck=?^~?????????nKLT-??E2??f:B-3?-t?
•Multi Stream Execution
??O??,.#2TfJ?&?-(?,?i??L?703?I
https://devblogs.nvidia.com/tensorrt-3-faster-tensorflow-inference/

35. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Interpretable ML: /Cy??L1U
!]AU?X#U=?~?L1U??nL1U??QS???n??1f)?'.
^??-???Rc.,??o?????????w(?: [?>@R?nSVMnGBT?xo
C. Molnar, Interpretable Machine Learning, https://christophm.github.io/interpretable-ml-book/
'?0!>900&p'?0!< 900&p
?t< 2000 km2
S'h??
8Ш?C?
?t> 2000 km2
'h??C?
(???n[?>@R??7%0Un
&s?8f3?????~??
?n??7%0Un'?0!?-
??ڡ~???J[???

36. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Interpretable ML for computer vision
•^>??=?~????ב???.???L1U??nL1U?nkLxm?֓QS??
5-?o]>Of)="Q]?7%0U?nkW*??,.??K?.[??QS?o
•kW*??&g???Y??E????Y??????N????nZM?˟?-۔??N
???ҡ~???v?J=?w]f?x?????o
+?kW*?]f]f'?8??OU8????Y??????N
M.T. Ribeiro, et al., Anchors: High-Precision Model-Agnostic Explanations, AAAI 2018.

37. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
]>Of)="Q]?OD??
9f3?????

38. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
L?7k=+-A
•(Z#2ӱ??.[(r??L?7+-Aڧ?m?
•=?~?k=+-A??k=???k???
•=?~?k=?O?/'SP?kW*?????:-+???=?~?
•L?7k=?1SP1SP?Pf ?T#2
•m????????+-A?(?
•AWS Inferentia
•Intel Nervana

39. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Machine Learning on FPGA
•?.,hf?FPGA?(?k=??^>??=?~?
•AWS F1 instance??n??m?.fU?0???Amazon Machine
Image?-Ck=???
•k??%?5OfSU7-2Yf????|???dÓ?v??E?
????Loop tiling ???RQ??.??E?[1]
[1] C. Zhang, et al., Optimizing FPGA-based Accelerator Design for Deep Convolutional Neural Networks, FPGA 2015.

40. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
X#U=?~??z-"1:#
•7-2Yf&v???n(ZV
0]!?L?7?03?I
•?-6?,;?GPU??^>??=?~?
?L?7?03?I?

41. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
X#U=?~??z-"1:#

42. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
???

43. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
???
•]>Of)="Q]?pGtT?ny5C?e?C??0b&s?????n
ATf!Q]?C4?C??????o
•^>??=?~?ATf!Q]ڧ?m?e??k=?.,hf.??\?na????
?oAutoML???˃H.[.?ק???pGtT?d??n'?E??AI?
[V&?.???L?o
•^>??=?~????]>Of)="Q]?nL?7?>K&?kr?Fk=?
??f$?oL?7+-An-"1:#?'?E?]?(o

44. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
%-!Q]??H??n?H,G??????????
https://amzn.to/aws_dev