Measures of Emotion: How Feelings are Expressed in the Body and Face During Walking.

Abstract

The goal of this dissertation was to assess multimodal behavioral patterns associated with emotion expression in the body and face during walking. Motion capture data were collected from 42 individuals walking while experiencing five target emotions. Facial expressions were captured from half of the walkers with a head-mounted video camera. A set of 60 observer subjects viewed videos of the walking trials to determine whether the experienced emotions were observable in the walkers’ body movements. An additional set of 60 observers evaluated the qualitative characteristics of the emotionally expressive movement trials using a scale derived from Effort-Shape analysis. The body movements were also analyzed quantitatively using kinematic analysis. The facial expressions were assessed using the Facial Action Coding System (FACS). Finally, statistical analyses were conducted to determine if: (1) emotion elicitation, recognition, qualitative movement characteristics, and kinematics were similar in walkers with and without the head-mounted camera, (2) Effort-Shape scores were different among target emotions, (3) kinematic outcomes were different among the target emotions, and (4) multi-modal behavior patterns could be identified for each target emotion.

Results showed that the head-mounted camera did not interfere with a walker’s ability to feel an emotion, an observer’s ability to recognize an emotion, or with qualitative or quantitative aspects of movement characteristics. The qualitative assessment indicated that each target emotion was defined by a unique combination of characteristics related to the six Effort-Shape qualities. The quantitative analysis showed that specific kinematic parameters were associated with emotion recognition. The multimodal analysis demonstrated that emotion-related patterns were most consistent in the face, followed by the face and body combined, and lastly the body. While facial expressions tended to be distinct for each emotion, bodily expressions tended to cluster into one of two categories that clearly separated the sad portrayals from the other emotions.

These studies demonstrated that emotions affected facial expressions and whole body movement characteristics in measurable ways during walking. These findings can be used to guide development of nonverbal behavior models used to detect, predict, and synthesize expressive behaviors.