Projections

Model

We used YOLOv3 for object detection and augmented reality makers (ArUco) for marking the coordinate system to the scene.
You only look once (YOLO) is a state-of-the-art, real-time object detection system. YOLOv3 is extremely fast and accurate. YOLO applies a single neural network to the full image. This network divides the image into regions and predicts bounding boxes and probabilities for each region. These bounding boxes are weighted by the predicted probabilities.

System overview

Examples

This is the example of the scene captured by a camera and run through YoloV3 object detection network. We can see AruCo markers which are used as a relative reference to calculate the position of projected labels.

Also we use an additional deep neural network to detect the written text in the scene.

In the end, we calculate the absolute location of projected objects and text labels and project them via laser projector back to the scene. Pretty high precision, on the spot!

See it in action...