Robot Control using 20 lines of C++ of Code and Classical Vision Tools

The system uses OpenCV and an Intel Neurostick on a Raspberry Pi4 to do all the localization and motion planning.
I use the Neurostick (shown as blue USB in the image) for yolo.

1. Darius Burschka
Machine Vision and Perception Group (MVP)
Department of Computer Science
Technische Universität München
Hardware Fault Tolerant distributed
Scene Analysis for Manipulation Task

2. System Overview
2Franka’s Panda manipulators will be used (just ordered)
Global camera
Intel Realsense D435
Wrist camera
with IR
Edge system with
Myriad (camera+
robot control)
Cuda enabled system
(object, human pose
recognition, 3D)

3. 3

4. 4

5. PointCloud based Recognition using Realsense
5

6. Appearance-based Recognition
6
Hierarchical object recognition (new for embedded applications)
1.Yolo for attention
2a. Appearance based human pose (multi-skeleton) estimation
2b. Clipping of object attention regions
3. Depth augmentation from 3D pointcloud on ”scene server”