Making your app see with Mobile Vision API

Talk presented during GDG DevFest Cerrado 2016, on October 15th, at Goiânia.

1. RAMON RABELLO
CEO @ VISYO
MAKING YOUR APP SEE WITH
MOBILE VISION API

2. RAMON RABELLO
about.me/ramonrabello
Agile Community
Active Member
GDG Brasília
Organizer
Senior Developer
~10 years

3. HOW DO WE SEE?

5. HOW DO
MACHINES SEE?

7. WHAT IS
COMPUTER VISION?

8. HOW MACHINES UNDERSTAND THE WORLD

10. COMPUTER VISION
IS EVERYWHERE!

11. ON ROBOTICS.

12. ON SECURITY.
http://www.eecs.qmul.ac.uk/assets/img/uploads/page-image/behaviour.png

13. ON SOCIAL NETWORKS.

14. ON MEDICINE.

15. ON GAMING.

16. ON CARS.

17. AND ALSO ON MOBILE.

18. MOBILE VISION API
OVERVIEW

19. OVERVIEW
COMMON FUNCTIONALITY
FACE API
BARCODE API
TEXT API (OCR)

20. COMMON FUNCTIONALITY
EVENT-DRIVEN PIPELINE
FACE DETECTOR
BARCODE DETECTOR
TEXT DETECTOR

21. FACE API
WHAT CAN WE DO WITH FACES?

22. FACE TRACKING

23. LANDMARKS DETECTION

24. FACE CLASSIFICATIONS
EYES OPENED
0.0 1.0

25. FACE CLASSIFICATIONS
SMILLING
0.0 1.0

26. FACE ORIENTATION
POSE ANGLE ESTIMATION
Y
Z
X

27. BARCODE API
WHAT TO DO WITH BARCODES?

28. BARCODE TRACKING
FORMATS SUPPORTED

29. TEXT API
READING TEXT FROM IMAGES

30. READING TEXT FROM IMAGES
18+ LANGUAGES SUPPORTED

31. TEXT STRUCTURE

32. MOBILE VISION API USAGE
1. ADD DEPENDENCIES
2. ADD META-DATA ON MANIFEST
3. CREATE DETECTOR
4. CHECK IF DETECTOR IS OPERATIONAL
5. CREATE FRAME
6. DETECT FACES

33. ADD DEPENDENCIES
build.gradle
dependencies {
…
compile 'com.google.android.gms:play-services-vision:{version}'
…
}

34. ADD META-DATA ON MANIFEST
AndroidManifest.xml
<application…>



<meta-data

android:name="com.google.android.gms.vision.DEPENDENCIES" 

android:value="face" />
</application>

35. CREATE DETECTOR
onCreate()
FaceDetector faceDetector = new FaceDetector.Builder(context) 

.setTrackingEnabled( false)

.setProminentFaceOnly( false)

.setMode(FaceDetector. FAST_MODE)

.setClassificationType(FaceDetector. ALL_CLASSIFICATIONS)

.setLandmarkType(FaceDetector. NO_LANDMARKS)

.build();

36. CHECK IF DETECTOR IS OPERATIONAL
onCreate()
faceDetector.isOperational(){
// code to detect faces here
}
downloads a native library,
when run for the first time.

37. CREATE FRAME
Bitmap bitmap = …; // read Bitmap
Frame frame = new Frame.Builder()
.setBitmap(bitmap)
.build();
onCreate()

38. DETECT FACES
SparseArray<Face> faces
= detector.detect(frame);
// face(s) detected
if (faces.size() > 0){

}
onCreate()

39. DEMO
THE FACEMINATOR
/visyo/faceminator/

40. WHAT’S NEXT?
MOBILE VISION GUIDE
https://developers.google.com/vision/

41. WHAT’S NEXT?
VISION SAMPLES
https://github.com/googlesamples/android-vision

42. WHAT’S NEXT?
DEVBYTES
https://www.youtube.com/androiddevelopers

43. THANKS!
goo.gl/v5D0mg
@visyosolutions
/visyo
goo.gl/VBJTIh
@gdgbrasilia
/gdgbrasilia