3D scanner using kinect

How to make a 3D Scanner using kinect and PCL

1. 3D Scanner Using kinect Mehdi Saman Booy Supervisor : Dr. Karimi

2. Contents Introduction 3D importance Technologies Kinect PCL Drivers Implementation Filtering Registration 1

3. Introduction 2

4. 3D Importance 2D images Easily interpreted by human eyes Difficultly analyzing by computer 3

5. 3D Importance (cont’d) So, Third dimension is needed We call it “ depth ” Depth can be get from 3D scanner 4

6. 3D Scanner Applications Motion capture Animations Video games Industrial Reverse engineering Quality check Virtual reality … 5

7. Technologies 6

8. Kinect A webcam-style add-on peripheral (Wikipedia) Enables users to control and interact with their console/computer without a controller Originally for Xbox 360 Also released for Windows and Xbox One Kinetic + Connect = Kinect 7

9. Kinect (history) 8

10. Kinect (history) 8 Kinect for Xbox 360 release

11. Kinect (history) 8 LibFreenect (Open source library) release

12. Kinect (history) 8 OpenNI (Open source driver) release

13. Kinect (history) 8 Announced to release Non-commercial Kinect SDK in spring 2011

14. Kinect (history) 8 Non-commercial Kinect SDK release

15. Kinect (history) 8 Announced to launching commercial version of Kinect (Kinect for Windows) with SDK

16. Kinect (history) 8 Kinect for windows release

17. Kinect Hardware 9

18. Kinect Hardware 10

19. Kinect Hardware 10 Multi-array microphone

20. Kinect Hardware 10 IR laser emitter

21. Kinect Hardware 10 IR camera

22. Kinect Hardware 10 Motorized tilt

23. Kinect Hardware 10 USB cable

24. Kinect Hardware 10 RGB camera

25. PCL (Point Cloud Library) Point Cloud: A set of data points in some coordinate system PCL: A library to works with point clouds Has a lots of functions for registration , filtering , … 11

26. Other Technologies Geomagic Studio Good to works with 3D scanner outputs OpenNI (Open Natural Interaction) Used to interact between Kinect and OS Other drivers such as Microsoft official driver and Libfreenect 12

27. Use Kinect for Xbox 360 As before said, Kinect is originally for Xbox 360 Connect Kinect for Xbox 360 to PC is not easy USB cable Additional Driver A lots of properties 13

28. Implementation 14

29. Our environment 15

30. Implementation Flowchart 16

31. Read data Kinect depth image resolution : 640 x 480 307500 points (x, y, z for each point) Shadow problem 17

32. Depth filtering Kinect depth range : 50cm – 15m Use passthrough filter On z-axis between 1.5 , 15 18

33. Depth filtering Model after depth filtering between 1.5 and 15 m 19

34. Down sampling Voxel: such as pixel in 2D image, each element of 3D Use voxelGrid function Change voxel size -> change down sample ratio 20

35. Down sampling Voxel grid leaf size is 0.5 cm 21

36. Plane removal With RANSAC algorithm we can fit a mathematical model to set of points Models : circle, sphere, plane, … Fit a plane and delete that points 22

37. Cluster extraction Still other objects or noises are exist Points with distance lower than a threshold is a cluster Delete all clusters except the biggest 23

38. Cluster extraction Clear object after filtering 24

39. Registration Definition Aligning two or more images of the same scene One reference image Transform other images The most important step 25

40. Angle of rotation is known We can rotate object manually a) Original b) Expected c) happened Registration (first try) 26 a b c

41. Registration (second try) 27 Translate object to center Rotation Translate object to its location 1 2 3

42. Registration (cont’d) Another down sampling for each iteration is required Number of iterations: Note: Environment variables is so important. E.g. camera with center of object distance and angle of rotation   28

43. Point cloud result Result after registration process 29

44. Work with Geomagic Better point cloud with Geomagic studio Reduce noise Refinement Polygon mesh Fill holes Smoothness 30

45. Final result Result after work with Geomagic studio 31

46. Specially thanks to Dr. Karimi 32 Thank you