High-speed optical sensing of pose, position, and surface normal for dynamic projection mapping

In this paper, we propose a new optical system to simultaneously measure the pose, position, and surface normals of a target at high speed. In the computer vision field, it is a fundamental task to measure a motion of a target, and a high-speed motion measurement system has been researched for real-time applications. In most cases, a target is idealized as a rigid body, and the motion is described as six-DoF pose and position. However, general objects are not entirely rigid, and a non-rigid motion component is significant in some applications. We focus on dynamic projection mapping as an application example and propose an integration system of rigid and non-rigid motion sensing. In order to measure the non-rigid motion, non-rigid registration between 3D point clouds is often used, but it requires high computation cost, especially in three-dimensional nearest neighbor searches. Then, we optically measure surface normal vectors of a target from 2D images as a non-rigid motion component. To achieve a high-speed and non-interference measurement, we introduce a three-band infrared optical system and the lighting setup and evaluate the coupling efficiency by demonstrating the simultaneous measurement of a pose, position, and surface normal.