The Notion Of Ideational Triggers For Affect Reconsidered

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 126-7):

AFFECT in verbal and visual texts is always triggered by ideational phenomena. These can be entities or occurrences of any kind. Ideational triggers for expressions of FACIAL AFFECT in Coraline may be sourced via a diversity of perceptual channels that are interpreted as available to the character in particular instances. A taxonomy of types of perceptual channel is presented in Figure 5.5.

The triggering information may be sourced externally through an auditory perceptual channel (as sound or silence) or a visual, olfactory, gustatory or tactile one (Feng and O’Halloran, 2013). Alternatively, it can be sourced internally through reflection, memory or imagination. In interpreting the trigger for a particular facial expression of emotion in multimodal discourse such as that in Coraline more than one perceptual channel is likely to play a part.

Blogger Comments:

[1] To be clear, with the notion of ideational triggers for emotion, the authors have left the interpersonal domain of appraisal by emotional attitude and entered the ideational domain of cause and effect (reason and result). The ideational trigger, here, is the Phenomenon/Agent of an impinging mental Process of emotion (Music pleases me). This notion, however, ignores the distinction with the Phenomenon/Range of an emanating mental Process of emotion (I like music).

[2] To be clear, the taxonomy of types of perceptual channel corresponds to a taxonomy of types perceptual Phenomenon/Agent ('triggering information') of an impinging mental Process of emotion.

[3] To be clear, internal triggering information corresponds to a cognitive Phenomenon/Agent of an impinging mental Process of emotion.

A Problematic Analysis Of Facial Affect

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 125-6):

Expressions of relative FORCE in FACIAL AFFECT are additionally realised through the relative duration over which an expression is held. …

In example (6), Coraline, having sensed danger, tells her Other Parents that she wants to go to bed. Her intention is to escape from the Other world in her sleep. However, the Other Parents follow closely behind her, the Other Mother even offering to tuck her into bed. 

Coraline’s anxiety is not revealed in the spoken exchange with the Other Mother but rather in her expression of FACIAL AFFECT as [spirit:down] realised through eyebrows raised and drawn together and downcast eyes. The expression is extended in duration, sustained over the three tone groups of the exchange (marked as //…//…//…).

Blogger Comments:

As previously argued, from the perspective of SFL Theory, these graded epilinguistic images are of the personal microfunction of protolanguage depicted on clay puppets by animators using the emotion-face code devised by Ekman.

[1] To be clear, since the authors claim (p123-4) that expressions of surprise typically have the briefest duration, the claim here is that surprise is typically has weaker force than other emotions.

[2] To be clear, the Coraline character is here concealing her anxiety from the other characters, as the spoken language demonstrates, so as not to raise suspicion, so the interpretation of this facial configuration — in which the eyes are not downcast — expressing any anxiety at all, let alone stronger anxiety, would seem to be the opposite of what is true.

The Praat waveforms are irrelevant here, since they just represent the articulation of consonants and vowels.

The Depiction Of Muscle Tension On A Clay Puppet

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 125):

Illustrated in (5) are two instances of FACIAL AFFECT as [spirit:up]. Image 2 is graded up in PARALINGUISTIC FORCE through increased muscle tension in the face as evident in the curled up corners of the mouth.

Blogger Comments:

As previously argued, from the perspective of SFL Theory, these graded epilinguistic images are of the personal microfunction of protolanguage depicted on clay puppets by animators using the emotion-face code devised by Ekman. What varies is the intensity of the emotional state depicted on the puppet.

The Paralinguistic Force Of Facial Affect Reconsidered

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 124):

Linguistic GRADUATION comprises two principal subsystems – FORCE and FOCUS. FORCE can function to adjust the relative intensity or quantity of inscribed attitude or to invoke an attitudinal meaning by grading ideational phenomena. FOCUS has to do with adjusting the categorical boundaries of phenomena as more or less sharply or softly defined (Hood, 2010, 2021; Hood and Zhang, 2020). However, unlike its linguistic counterpart and unlike the PARALINGUISTIC GRADUATION of body gestures (Hao and Hood, 2019; Hood and Zhang, 2020), FACIAL AFFECT can only be graded in FORCE. Features of [strong] to [weak] are shown as positions on a cline in Figure 5.4, realised through variations in muscle tension and/or the duration for which an expression is held.

Blogger Comments:

As previously demonstrated, the facial expression of emotion is not an interpersonal system of language, so the linguistic systems of APPRAISAL, such as ATTITUDE and GRADUATION, do not apply. Instead, from the perspective of SFL Theory, the facial expression of emotion serves the personal microfunction of protolanguage, and so what is presented here as graduated force is more consistently understood as a graduation of emotional intensity within the personal microfunction.

Misunderstanding Semiosis As Somasis

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 124):

Our focus is on the semiosis of facial expression realising emotion in human interaction, but it is important to note that the face can also manifest non-emotional (somatic) states. A frown, for example, might manifest concentrated thinking (Fasel and Luettin, 2003: 260) or physiological states of pain or fatigue (see Chapter 1). Instances of somatic facial expression can of course index purposeful feelings, which remains a challenge for analysts as discussed in Chapter 1. The approach taken in this book is that behaviours can be treated as paralinguistic (i.e. semiotic) depending on whether or not they are negotiated as meaningful in interaction.

Blogger Comments:

This is a very serious misunderstanding. If a facial configuration means ('manifests', indexes') something other than itself, then it is the signifier of a signified, and so semiotic, not "somatic". At the social level, in terms of Halliday's linear taxonomy of complex systems, some facial configurations select a positive value (e.g. 'approach') in the other, whereas some select a negative value (e.g. 'avoid').

Simultaneous Emotion

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 123-4):

A further consideration in analysing and interpreting facial expressions is the potential for one feature of facial affect to transition very quickly into another in an animated expression. An instance in example (4) expresses both [surprise] and [spirit:up]. 

From a systemic functional perspective, rather than describing this as a blending or merging of emotions it is considered as the co-instantiation of two different emotions with each realised through particular parts of the face (e.g. eyes, eyebrows, mouth) and often in very quick succession. In (4) the raised curved eyebrows realise [surprise] and the upturned lips realise [spirit:up]. 

A facial expression of [surprise], interpreted as a perturbance (Martin, 2017a) typically has the briefest duration and often transitions quickly to the expression of another emotion, one which responds to the specific trigger of the perturbance.

Blogger Comments:

[1] From a systemic functional perspective, this blurs the axial distinction between simultaneous systems ('both', 'co-instantiation', 'and') and syntagmatic order ('transition', 'succession'). Moreover, if two emotions can be realised in the same facial expression, the system network needs to be redrawn to represent simultaneous (conjunct) systems. This the authors have not done.

[2] On the one hand, the claim that a facial expression of surprise typically has the briefest duration is an instance of the logical fallacy known as ipse dixit: a bare assertion unsupported by evidence, and is belied by synonyms for 'surprised' such as 'stupefied' and 'dumbfounded'. On the other hand, surprise is the emotion that is the response to what triggered it as a perturbance.

Why Facial Affect Is More Limited Than Language

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 121):

As noted earlier, ATTITUDE in language can be expressed through systems of AFFECT, APPRECIATION or JUDGEMENT while paralinguistic expressions of ATTITUDE are restricted to FACIAL AFFECT (see, e.g. Tian, 2011). This means that the paralinguistic meaning potential for expressing emotion is relatively limited with respect to language. 

For example, an array of finely distinguished lexical instantiations of the feature [realis: happiness; mood; positive] (Table 5.1) are possible, as, for instance, in happy/joyful/delighted/thrilled and so on, such fine distinctions are not available in FACIAL AFFECT. 

In analyses of intermodal resonance in Coraline, fine distinctions in verbal instances (e.g. happy vs joyful) may be inferred for resonant facial expressions but cannot be attributed to specific variations in the facial expression. In other words a given expression of FACIAL AFFECT might couple with a diverse array of lexical realisations of [realis: happiness; mood; positive].

Blogger Comments:

To be clear, this relative limitation with respect to language is simply explained by the fact that the facial expression of emotion is a protolinguistic semiotic system, which means it lacks a grammatical stratum.

Neither Affect Nor Body Language

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 121, 122):

In summary, the network in Figure 5.3 models FACIAL EFFECT with six features of emotion: [spirit:up], [spirit:down], [fear], [anger], [disdain] and [surprise]. Each of these six features is illustrated and described in (2). Facial resources for expression are in bold.

Blogger Comments:

[1] As previously demonstrated, in terms of SFL Theory, the facial expression of emotion is protolanguage, serving the personal metafunction, and so not an interpersonal system (AFFECT) of the discourse semantics of language.

[2] As previously observed, the facial expressions constructed on clay puppets are not body language, but epilinguistic representations of body language created by animators using the emotion-face coding of Ekman.

Misrepresenting 'Threat' As An Emotion

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 120-1, 239n):

An additional opposition proposed by Darwin (1872) is between facial movements interpreted as ‘fear’ and ‘anger’. For Darwin, ‘fear’ is a feeling caused by the anticipation that one could be harmed (which we interpret as a response to what might happen, i.e. an irrealis trigger) and ‘anger’ is a feeling that might result in one harming others (which we interpret as a response to something real happening, i.e. a realis trigger). In the network of FACIAL AFFECT in Figure 5.3 [fear] and [anger] are opposing features of [threat]. Each feature is realised through a different set of facial expressions shown in italics. In the intersemiosis of facial expression and the unfolding storyline in language and action, the facial feature [fear] is interpretable as negative and irrealis, that is, it is a negative emotional response to what might happen. In contrast the feature [anger] is interpretable as negative and realis, an emotional response to what is happening or has happened.

Blogger Comments;

[1] As previously explained, the title Darwin (1872), The expression of the emotions in man and animals, demonstrates that the meanings here are protolinguistic, since other animals do not express the meanings of language. So, to model protolanguage as language, as FACIAL AFFECT, is theoretically invalid. On Halliday's model, the expression of emotion serves the personal microfunction of protolanguage. 

[2] To be clear, here the authors misrepresent the result of anger (harming others) as the reason for it (trigger).

[3] To be clear, in Figure 5.3, the authors misrepresent 'threat' as an emotion, with its result (fear) and cause (anger) as its subtypes.

[4] Again, the system in Figure 5.3 confirms the fact that here the authors model paralanguage as a bi-stratal semiotic system, and although this is consistent with the notion of a semiotic system, it is inconsistent with the preceding chapters in which paralanguage is misunderstood as an expression-only semiotic system. Where in previous chapters it was just paralinguistic expression that was semovergent with language, in this chapter it is both paralinguistic content and expression that is semovergent with language.

Problems With The System Of Facial Affect

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 120, 121):

The systemic functional semiotic system of FACIAL AFFECT presented in Figure 5.3 takes into account these important contributions in a number of areas, including descriptive terminology. For example, the naming of features in the model of FACIAL AFFECT avoids the use of Ekman’s terms of ‘happiness’ and ‘sadness’ as [happiness] is already a feature in linguistic ATTITUDE. Instead emotion terminology is sourced to Darwin’s (1872) opposition in facial movements of ‘high spirit’ and ‘low spirit’. Darwin’s influence is seen in Figure 5.3 in the naming of the feature [spirit] and its opposing features as [up] and [down].

Blogger Comments:

[1] To be clear, the system in Figure 5.3 confirms the fact that here the authors model paralanguage as a bi-stratal semiotic system. As previously noted, although this is consistent with the notion of a semiotic system, it is inconsistent with the preceding chapters in which paralanguage is misunderstood as an expression-only semiotic system. Where in previous chapters it was just paralinguistic expression that was semovergent with language, in this chapter it is both paralinguistic content and expression that is semovergent with language.

[2] To be clear, the title of Darwin (1872), The expression of the emotions in man and animals, acknowledges that the expression of emotion does not require the evolution and development of language. As such, the facial expression of emotion is protolanguage, not language.

Where AFFECT is a system of the interpersonal metafunction in the tri-stratal semiotic of language, protolanguage is a bi-stratal system that is pre-metafunctional. On Halliday's model, the expression of emotion serves the personal microfunction. 

In short, to model the personal microfunction of protolanguage as the interpersonal metafunction of language, as FACIAL AFFECT, is theoretically invalid.

An Inconsistent Use Of 'Semovergent'

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 119-20):

The linguistic system of AFFECT does not constitute a blueprint for the development of a system of PARALINGUISTIC AFFECT, its features or oppositions; the systems in the two modalities are named differently to reflect this (as in PARALINGUISTIC ENGAGEMENT and PARALINGUISTIC GRADUATION). PARALINGUISTIC AFFECT models expressions of emotion in FACIAL AFFECT with features realised through muscle movement of the face, and in VOICE AFFECT with features realised through qualities of the voice.

 

Blogger Comments:

To be clear, here the authors model paralanguage as a bi-stratal semiotic system. Although this is consistent with the notion of a semiotic system, it is inconsistent with the preceding chapters in which paralanguage is misunderstood as an expression-only semiotic system. Where in previous chapters it was just paralinguistic expression that was semovergent with language, in this chapter it is both paralinguistic content and expression that is semovergent with language.

Paralinguistic Expressions Of Attitude

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 117-8):

The discourse system of ATTITUDE (Figure 5.1) differentiates AFFECT as the expression of feelings or emotions from JUDGEMENT as the evaluation of people and behaviour and APPRECIATION as the assessment of phenomena (Martin and Rose, [2003] 2007; Martin, 2020). Of these three kinds of ATTITUDE only AFFECT is relevant to a discussion of interpersonal semovergence as paralinguistic expressions of ATTITUDE are restricted to those of emotion (e.g. Tian, 2011; Welch, 2005; Painter et al., 2013: 31–2).

Blogger Comments:

To be clear, the claim here is that, in the authors' model, paralanguage cannot enact judgement or appreciation.

Moreover, in the authors' terms, there is an unrecognised instance of a representation of judgement in the authors' data (p147):

If you point the finger at someone or point an accusing finger at someone, you blame them or accuse them of doing wrong.

Problems With A Phonological Analysis

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 117):

The example in (1) shows paralinguistic expressions sonovergent with each major tone choice (other than the level-rise tone 3 which offers minimal phonological scope for convergent body-part movement). A phonological transcription records the tone (as, e.g. //1), and the intonation contour describes and interprets it (as, e.g. falling – ‘certain’). The paralanguage which is sonovergent with each intonation contour – which visualises it – is then described. The resonance of the visual and phonological contours adds further salience to the tonic and hence the given tone choice.

Blogger Comments:

[1] To be clear, tone 3 is a level or low rise in pitch, so it can be realised facially by a maintained level or low rise in the eyebrows.

[2] To be clear, adding salience to the tonic is adding salience to what the tonic realises: the focus of New information, not to the tone choice that realises the system of KEY.

[3] To be clear, based on the system of KEY, the most likely tone for // I didn't / break it // is tone 2 ('protest'), not tone 4 ('reservation'), and this analysis is supported by the raised eyebrows on the clay puppet.

The Problem With Interpersonal Sonovergent Paralanguage

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 116):

Sonovergent paralanguage is only meaningful in its relation to the prosodic phonology of co-expressed speech (Halliday, 1967, 1970a; Halliday and Greaves, 2008; Smith and Greaves, 2015) (see Chapters 1 and 3). … Where interpersonal sonovergent paralanguage resonates with tone choices it is frequently expressed in up or down movements of the head, eyebrows or arms in tune with pitch movements in co-articulated speech.

Blogger Comments:

As previously explained, sonovergent paralanguage is neither sonovergent nor paralanguage. It is not sonovergent because the bodily expressions diverge from the phonological expressions, and it is not paralanguage because it is language, since the expressions realise the grammatical system of KEY. Again, this is why it is termed 'linguistic' in Cléirigh's model, which the authors in this book rebrand as their own.

The Designed Representations Of Body Language On Clay Puppets

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 116):

In stop-motion animations such as Coraline story characters are portrayed by puppets. Their movements, body language and facial expressions are designed by animators to match the pre-recorded speech of voice actors. The animations are developed frame-by-frame as animators make very small changes to the puppet’s face or body. Each change is captured as a ‘stop-motion’ or static frame before being collated and camera recorded to create movements in film (Laika Studios, 2017). In films, the characterisation of the main characters conveys the thematic message; accordingly, every facial or body movement of the main puppet characters (including the way they walk) is designed to express meaning – and so everything they do can be interpreted as semiotic rather than somatic behaviour (see Chapter 1; Mohamed and Nor, 2015; van Leeuwen, 2005). That said, the body movements of secondary characters arguably include both semiotic and somatic behaviour in order to progress the storyline. In this chapter we concentrate on the meaningful behaviour of the main characters.

Blogger Comments:

[1] To be clear, epilinguistic representations on clay puppets designed by animators are not instances of the body language of a member of socio-semiotic species.

[2] To be clear, the facial postures and movements of all the puppets designed by the animators represent both social and semiotic systems, in terms of Halliday's linear taxonomy of evolutionary complexity.