Metrological Evaluation of Multimodal 3D Camera System for Reliable Dynamic Facial Motion Analysis

Wang Liao, Chen Zhang, Shiyao Gao, Hongyu Chen, Hao Chen, Maik Rosenberger, Gunther Notni

Abstract

Traditional 2D-based dynamic facial motion analysis methods often rely on texture features and lack depth information, raising the fundamental question: can facial motions be measured rather than inferred? This work proposes a multimodal 3D measurement-based concept for dynamic facial motion analysis, leveraging a camera system that combines a GOBO-based active stereo 3D unit with a synchronized RGB camera. A pretrained neural network extracts 2D facial landmarks, which are mapped into 3D-metric space. As proof-of-concept, 19 geometric features (e.g., distances and angles) are defined based on 3D metric facial landmarks to correspond to selected Action Units (AUs). The concept is evaluated from a metrological perspective. Under static condition, where facial expressions remain unchanged, 90 frames were captured per video across 30 recordings for each of six participants. The average standard deviations of the defined features measurements were 0.85° (angles) and 0.37 mm (distances), yielding expanded uncertainties of 2.6° and 1.1 mm (99.73% confidence). Feasible measurement capability of the camera system is further supported by low variation coefficients (1.33% and 1.61%) of the measured features values. To further evaluate dynamic capability, facial motions were induced, producing feature changes much greater than the uncertainty, confirming statistical significance. The proposed concept and camera system provide a robust and precise foundation for dynamic facial motion analysis, with potential applications in scenarios like micro-expression recognition, lie detection, and healthcare monitoring.

Download

IMEKO-TC2-2025-007.pdf

DOI

10.21014/tc2-2025.007

IMEKO TC

TC2 - Photonics

Back to the proceedings