To identify the AUs producing early confusions and thosesupporting later accurate discrimination, we used a leave-one-out method that removed each AU independently fromall models and time points before recomputing the Bayesianclassiﬁer performance (see the Supplemental ExperimentalProcedures, Confusing and Diagnostic Face Signals). Figure 3(Confusing and Diagnostic Face Signals) shows the AUs—presented as color-coded deviation maps—that produceearly confusions (outlined in magenta) and support later discrimination between emotions (outlined in green) for twoconfusions (surprise/fear and disgust/anger). As shown, early confusions between surprise and fear arise due to the common transmission of Upper Lid Raiser and Jaw Drop, (
2) then Upper Lid Raiser (
5), withaccurate discrimination arising due to the later availabilityof Eyebrow Raiser (
6). Similarly, disgust and anger areconfused early in the signaling dynamics due to the commontransmission of Nose Wrinkler (
5), then Lip Funneler (
Figure 2. Expected Dynamic Signaling of FacialExpressions of Emotion over TimeTo quantify the dynamic signaling of facialexpression signals (i.e., AUs) expected over time, we mapped the distribution of expectedtimes of all AUs comprising all models pooled(‘‘All Facial Expression Models,’’ n = 720 models)and also split by emotion (‘‘Models Split byEmotion,’’ n = 120 models).(Top) All facial expression models. In each row,color-coded circles represent the distribution of expected times for each AU, where brightnessindicates the median expected time and dark-ness indicates distance from the median,weighted by the proportion of models with that AU. As shown by the white line, signal complexity(measured byShannonentropy,inbits) increasesbefore later decreasing over the signaling dy-namics, where low entropy reﬂects systematicsignaling of few AUs. As represented by magentacircles, AUs systematically expected early in thesignaling dynamics (e.g., Upper Lid Raiser,Nose Wrinker; p < 0.05) comprise biologicallyadaptive AUs . As represented by green cir-cles, AUs systematically expected later (e.g.,BrowRaiser,UpperLipRaiser;p<0.05)comprise AUs diagnostic for categorizing the six classicemotions .(Bottom) Models split by emotion. Note that ob-serversexpectUpperLidRaisertobetransmittedearly in both surprise and fear, and Nose Wrinkler to be transmitted early in disgust and anger.Together, these data show that dynamic facialexpressions transmit signals that evolve over time from simpler, biologically rooted signals tosocially speciﬁc signals.See also Table S2.
with accurate discrimination occurr-ing due to the later transmission of Upper Lip Raiser Left (
7). Based onsystematic early confusions betweenspeciﬁc emotion categories, these re-sults reﬂect that expected early facesignals enable discrimination of onlyfour emotion categories – i.e., (1) happy,(2) sad, (3) fear/surprise, and (4) disgust/ anger—whereas the later availability of diagnostic information supports discrimination of all sixemotion categories.
Using perceptual expectation modeling, we derived thedynamic signaling of the six classic facial expressions of emotion—happy, surprise, fear, disgust, anger, and sad—in60 Western white Caucasian observers. Information-theoreticanalysis showed that the dynamics transmit informationevolving from simpler, biologically rooted signals (e.g., Upper Lid Raiser and Nose Wrinkler) to more-complex signals. UsingBayesianclassiﬁers,weshowthatearlysignalingischaracter-ized by the common transmission of speciﬁc AUs (e.g., Upper Lid Raiser) between emotion categories (e.g., surprise andfear),thereby giving rise tosystematic confusions. Incontrast,later signaling comprises the availability of diagnosticinformation (e.g., Eyebrow Raiser), supporting the accurate
Facial Expressions Transmit Signals Hierarchically
Please cite this article in press as: Jack et al., Dynamic Facial Expressions of Emotion Transmit an Evolving Hierarchy of Signals over Time, Current Biology (2014), http://dx.doi.org/10.1016/j.cub.2013.11.064