Behavior Prediction InThe-Wild

Christos Georgakis,M. Pantic
Abstract:In this paper, the problem of audio-visual behavior prediction in-the-wild is addressed. In this context, both audiovisual descriptors of behavioral cues (features) and continuoustime real-valued characterizations of behavior (annotations) are (possibly) corrupted by non-Gaussian noise of large magnitude. The modeling assumption behind the proposed framework is that naturalistic affect and behavior captured in audiovisual episodes are smoothly-varying dynamic phenomena and thus the hidden temporal dynamics can be modeled as a generative auto-regressive process. Consequently, continuoustime real-valued characterizations of behavior (annotations) are postulated to be outputs of a low-complexity (i.e., loworder) time-invariant Linear Dynamical System (LDS) when descriptors of behavioral cues (features) act as inputs. To learn the parameters of the LDS, a recently proposed spectral method that relies on Hankel-rank minimization is adopted. Experimental evaluation on a challenging database recorded in the wild demonstrate the effectiveness of the proposed approach in behavior prediction.
What problem does this paper attempt to address?