Share Email Print

Proceedings Paper

Real-time face and gesture analysis for human-robot interaction
Author(s): Frank Wallhoff; Tobias Rehrl; Christoph Mayer; Bernd Radig
Format Member Price Non-Member Price
PDF $17.00 $21.00

Paper Abstract

Human communication relies on a large number of different communication mechanisms like spoken language, facial expressions, or gestures. Facial expressions and gestures are one of the main nonverbal communication mechanisms and pass large amounts of information between human dialog partners. Therefore, to allow for intuitive human-machine interaction, a real-time capable processing and recognition of facial expressions, hand and head gestures are of great importance. We present a system that is tackling these challenges. The input features for the dynamic head gestures and facial expressions are obtained from a sophisticated three-dimensional model, which is fitted to the user in a real-time capable manner. Applying this model different kinds of information are extracted from the image data and afterwards handed over to a real-time capable data-transferring framework, the so-called Real-Time DataBase (RTDB). In addition to the head and facial-related features, also low-level image features regarding the human hand - optical flow, Hu-moments are stored into the RTDB for the evaluation process of hand gestures. In general, the input of a single camera is sufficient for the parallel evaluation of the different gestures and facial expressions. The real-time capable recognition of the dynamic hand and head gestures are performed via different Hidden Markov Models, which have proven to be a quick and real-time capable classification method. On the other hand, for the facial expressions classical decision trees or more sophisticated support vector machines are used for the classification process. These obtained results of the classification processes are again handed over to the RTDB, where other processes (like a Dialog Management Unit) can easily access them without any blocking effects. In addition, an adjustable amount of history can be stored by the RTDB buffer unit.

Paper Details

Date Published: 4 May 2010
PDF: 10 pages
Proc. SPIE 7724, Real-Time Image and Video Processing 2010, 77240A (4 May 2010); doi: 10.1117/12.855963
Show Author Affiliations
Frank Wallhoff, Technische Univ. München (Germany)
Tobias Rehrl, Technische Univ. München (Germany)
Christoph Mayer, Technische Univ. München (Germany)
Bernd Radig, Technische Univ. München (Germany)

Published in SPIE Proceedings Vol. 7724:
Real-Time Image and Video Processing 2010
Nasser Kehtarnavaz; Matthias F. Carlsohn, Editor(s)

© SPIE. Terms of Use
Back to Top
Sign in to read the full article
Create a free SPIE account to get access to
premium articles and original research
Forgot your username?