Most Pageviews by Country

Accepting The Biological Nature Of Vocal And Facial Expressions Of Emotion

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 139):

If we accept that natural (i.e. not performed) vocal and facial expressions of emotion are biological in nature (Darwin, 1872; Barlow, 2002), this would suggest resonance across the systems of FACIAL AFFECT and VOICE AFFECT (in the absence of intentional divergence such as in expressions of sarcasm).

Blogger Comments:

[1] To be clear, the biological nature of vocal and facial expressions of emotion, whether natural or performed, lies in the fact that the organs of organisms are their material basis. Moreover, the fact that the vocal and facial expressions of emotion are not restricted to humans (Darwin 1872) demonstrates that these semiotic systems do not require the prior evolution and development of language, and so are (personal) protolinguistic systems, rather than (interpersonal) AFFECT systems of language.

[2] For amusement, the following illustrates the divergence between the expressed content of protolanguage and the unexpressed content of language.

Problems With Irrealis vs Realis Affect

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 119, 136, 137):

An important distinction in the AFFECT system in language (Table 5.1) is between realis (an emotional response triggered by a present or past happening) and [ir]realis (an emotional response triggered by what might happen). Where the response is irrealis positive this is glossed as ‘desire’ and where it is negative as ‘fear’. However, in the VOICE AFFECT system [fear] is a feature (not simply a gloss) and its realisations are restricted to qualities of voice. Nonetheless the intersemiotic convergence of voiced [fear] with the language and action of the unfolding storyline in Coraline can support an interpretation of the voiced negative emotion as a response to what might happen, or in the case of (13) to whom the voices might belong. …

In contrast to voiced [fear], the intersemiotic convergence of voiced [anxiety] with the language and action of the unfolding storyline in Coraline can support an interpretation of the voiced emotion as a response to seeing the Ghost Children, that is, a realis happening.

Blogger Comments:

[1] To be clear, this confuses interpersonal meaning (AFFECT) with experiential meaning (cause, happening). The distinction here is between mental processes of emotion ('realis') and mental processes of desideration ('irrealis').

However, the exclusive association of desire and fear with irrealis is invalid, since both can be triggered by a present or past happening, as demonstrated by He desired her from the moment he saw her and She feared the non-venomous snake the moment she saw it.

[2] Importantly, here the intersemiotic convergence is of the content of paralanguage with the content of language. This is inconsistent with the authors' model of ideational paralanguage, where it is the expression of paralanguage that converges with the content of language.

[3] To be clear, here the authors are anxious to justify their categorisation of 'fear' as irrealis (desiderative), in contrast to 'anxiety', which they categorise as realis (emotive). Their anxiety, however, is unjustified, because 'fear' can be realis, as in She feared the Ghost Children, as well as irrealis, as in She feared that the Ghost Children might harm her.

Problems With The System Of The 'Emotion' Threat

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 135):

The sets of voice qualities which differentiate VOICE AFFECT features of [fear], [anxiety] and [anger] are shown in Figure 5.9.

 

Blogger Comments:

The system in Figure 5.9 models paralanguage as a bi-stratal semiotic system, and although this is consistent with the notion of a semiotic system, it is inconsistent with the preceding chapters in which paralanguage is misunderstood as an expression-only semiotic system. Where in previous chapters it was just paralinguistic expression that was semovergent with language, in this chapter it is both paralinguistic content and expression that is semovergent with language.

Further, in Figure 5.9, the authors again misrepresent 'threat' as an emotion, with its result (fear, anxiety) and reason (anger) as its subtypes.

Problems With The System Of The 'Emotion' Spirit

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 133):

VOICE AFFECT as [spirit:down] has opposing features of [misery] and [ennui]. The voice quality contours which realise these features are shown in Figure 5.8.

Blogger Comments:

The system in Figure 5.8 models paralanguage as a bi-stratal semiotic system, and although this is consistent with the notion of a semiotic system, it is inconsistent with the preceding chapters in which paralanguage is misunderstood as an expression-only semiotic system. Where in previous chapters it was just paralinguistic expression that was semovergent with language, in this chapter it is both paralinguistic content and expression that is semovergent with language.

Problems With The System Of Voice Affect

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 129, 131):

It is the particular ‘mixture’ of options [from the system of VOICE QUALITY] which realise one feature or another in the system of VOICE AFFECT (see Figure 5.7).

Blogger Comments:

The system in Figure 5.7 models paralanguage as a bi-stratal semiotic system, and although this is consistent with the notion of a semiotic system, it is inconsistent with the preceding chapters in which paralanguage is misunderstood as an expression-only semiotic system. Where in previous chapters it was just paralinguistic expression that was semovergent with language, in this chapter it is both paralinguistic content and expression that is semovergent with language.

Further, in Figure 5.7, the authors again misrepresent 'threat' as an emotion, with its result (fear, anxiety) and reason (anger) as its subtypes. Moreover, within the emotion of threat, it groups one result (anxiety) with a reason (anger) instead of with a result (fear).

Inconsistency In The Authors' Analysis

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 128):

Just as an expression of FACIAL AFFECT supports the identification of the trigger, so available ideational information supports the interpretation of FACIAL AFFECT. A sequence of triggers is interpreted as prompting the sequence of emotions in (8). 

We interpret Coraline’s expression of [fear] in image 1 of example (8) as triggered by the potential consequences of accumulated information sourced visually in the falling rock and auditorily in the cry of pain and the loud, angry ‘meow’.  

We interpret the expression of [surprise] in image 2 as triggered visually by Coraline’s first sight of the cat.  

The trigger for [anger] in image 3 is interpreted not as a response to seeing the cat but to an internal realisation that it was the cat who had instigated her fear.

Blogger Comments:

To be clear, the authors' interpretations here are inconsistent with their own previous account:

A pained cry is heard. Extremely alarmed by this, she runs as fast as possible, sensing something is pursuing her. Startled by a loud ‘meow’ from behind, she turns to look. Seeing that it is only a cat, her facial expression of [fear] swiftly changes to [surprise], but then to [anger], as in the three images in (8).

That is:

• hearing a cry of pain triggered her fear;
• hearing a loud meow triggered her surprise; and
• seeing a cat, rather than a threat, triggered her anger.

Misapplying A Confusion Of Ideational And Interpersonal Meaning To A Representation Of Protolanguage [2]

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 127-8):

A further example in (8) shows how information acquired from past events can trigger a response in FACIAL AFFECT. The instance involves Coraline’s first encounter with the Cat in the orientation stage of the film’s narrative. The episode begins with Coraline exploring the neighbourhood along a steep hillside path. A rock falls onto her path from on high. She calls out but gets no response, then throws the rock in the direction from which it fell. A pained cry is heard. Extremely alarmed by this, she runs as fast as possible, sensing something is pursuing her. Startled by a loud ‘meow’ from behind, she turns to look. Seeing that it is only a cat, her facial expression of [fear] swiftly changes to [surprise], but then to [anger], as in the three images in (8).

Blogger Comments:

[1] That is:

• a Phenomenon of auditory perception (pained cry) is the Agent (trigger) of the mental Process of emotion (alarm);
• a Phenomenon of auditory perception (a loud meow) is the Agent (trigger) of the mental Process of emotion (surprise); and
• a Phenomenon of visual perception (a cat) is the Agent (trigger) of the mental Process of emotion (anger).

Again this confuses ideational with interpersonal meaning, and misapplies the confusion to an epilinguistic representation of pre-metafunctional protolanguage on a clay puppet.

[2] To be clear, here the expression of emotion does not accompany speech, so it is not functioning as paralanguage, and is not semovergent.

Misapplying A Confusion Of Ideational And Interpersonal Meaning To A Representation Of Protolanguage [1]

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 127):

In (7), the trigger for facial expressions of emotion is apparently sourced internally. 

In the resolution stage of the narrative storyline in the film, Coraline meets the Cat, a good friend whom she has not seen since she threw him at the Other Mother in attempting her escape from the Other World.

In image 1 in (7) Coraline expresses both mild [surprise] and [spirit:up]. There is no immediately convergent speech, and the trigger is not interpretable at this point by the viewer. 

However, in image 2 more visual information is made available. The Cat is now revealed as standing outside Coraline’s bedroom window, and his presence retrospectively explains the trigger for her facial [surprise] and [spirit:up] in image 1. 

In image 2, convergent with her spoken language, Coraline’s expression of FACIAL AFFECT changes from [spirit:up] to [spirit:down]. Again there is no apparent trigger in the visually available information. The resonant spoken language I’m really sorry I threw you out at the Other Mother suggests that the trigger at this point is sourced internally through her reflection on past events. The broader co-text of the story supports this interpretation.

Blogger Comments:

[1] That is, a cognitive Phenomenon is the Agent (trigger) of the mental Process of emotion. This confuses ideational with interpersonal meaning, and misapplies the confusion to an epilinguistic representation of pre-metafunctional protolanguage on a clay puppet.

[2] To be clear, in SFL terms, this is an expression of the personal microfunction of protolanguage, epilinguistically represented on a clay puppet.

[3] To be clear, if the expression of emotion does not accompany speech, then it is not functioning as paralanguage, and is not semovergent.

[4] That is, a Phenomenon of visual perception is the Agent (trigger) of the mental Process of emotion. This again confuses ideational with interpersonal meaning, and misapplies the confusion to an epilinguistic representation of pre-metafunctional protolanguage on a clay puppet.

The Notion Of Ideational Triggers For Affect Reconsidered

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 126-7):

AFFECT in verbal and visual texts is always triggered by ideational phenomena. These can be entities or occurrences of any kind. Ideational triggers for expressions of FACIAL AFFECT in Coraline may be sourced via a diversity of perceptual channels that are interpreted as available to the character in particular instances. A taxonomy of types of perceptual channel is presented in Figure 5.5.

The triggering information may be sourced externally through an auditory perceptual channel (as sound or silence) or a visual, olfactory, gustatory or tactile one (Feng and O’Halloran, 2013). Alternatively, it can be sourced internally through reflection, memory or imagination. In interpreting the trigger for a particular facial expression of emotion in multimodal discourse such as that in Coraline more than one perceptual channel is likely to play a part.

Blogger Comments:

[1] To be clear, with the notion of ideational triggers for emotion, the authors have left the interpersonal domain of appraisal by emotional attitude and entered the ideational domain of cause and effect (reason and result). The ideational trigger, here, is the Phenomenon/Agent of an impinging mental Process of emotion (Music pleases me). This notion, however, ignores the distinction with the Phenomenon/Range of an emanating mental Process of emotion (I like music).

[2] To be clear, the taxonomy of types of perceptual channel corresponds to a taxonomy of types perceptual Phenomenon/Agent ('triggering information') of an impinging mental Process of emotion.

[3] To be clear, internal triggering information corresponds to a cognitive Phenomenon/Agent of an impinging mental Process of emotion.

A Problematic Analysis Of Facial Affect

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 125-6):

Expressions of relative FORCE in FACIAL AFFECT are additionally realised through the relative duration over which an expression is held. …

In example (6), Coraline, having sensed danger, tells her Other Parents that she wants to go to bed. Her intention is to escape from the Other world in her sleep. However, the Other Parents follow closely behind her, the Other Mother even offering to tuck her into bed. 

Coraline’s anxiety is not revealed in the spoken exchange with the Other Mother but rather in her expression of FACIAL AFFECT as [spirit:down] realised through eyebrows raised and drawn together and downcast eyes. The expression is extended in duration, sustained over the three tone groups of the exchange (marked as //…//…//…).

Blogger Comments:

As previously argued, from the perspective of SFL Theory, these graded epilinguistic images are of the personal microfunction of protolanguage depicted on clay puppets by animators using the emotion-face code devised by Ekman.

[1] To be clear, since the authors claim (p123-4) that expressions of surprise typically have the briefest duration, the claim here is that surprise is typically has weaker force than other emotions.

[2] To be clear, the Coraline character is here concealing her anxiety from the other characters, as the spoken language demonstrates, so as not to raise suspicion, so the interpretation of this facial configuration — in which the eyes are not downcast — expressing any anxiety at all, let alone stronger anxiety, would seem to be the opposite of what is true.

The Praat waveforms are irrelevant here, since they just represent the articulation of consonants and vowels.

The Depiction Of Muscle Tension On A Clay Puppet

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 125):

Illustrated in (5) are two instances of FACIAL AFFECT as [spirit:up]. Image 2 is graded up in PARALINGUISTIC FORCE through increased muscle tension in the face as evident in the curled up corners of the mouth.

Blogger Comments:

As previously argued, from the perspective of SFL Theory, these graded epilinguistic images are of the personal microfunction of protolanguage depicted on clay puppets by animators using the emotion-face code devised by Ekman. What varies is the intensity of the emotional state depicted on the puppet.

The Paralinguistic Force Of Facial Affect Reconsidered

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 124):

Linguistic GRADUATION comprises two principal subsystems – FORCE and FOCUS. FORCE can function to adjust the relative intensity or quantity of inscribed attitude or to invoke an attitudinal meaning by grading ideational phenomena. FOCUS has to do with adjusting the categorical boundaries of phenomena as more or less sharply or softly defined (Hood, 2010, 2021; Hood and Zhang, 2020). However, unlike its linguistic counterpart and unlike the PARALINGUISTIC GRADUATION of body gestures (Hao and Hood, 2019; Hood and Zhang, 2020), FACIAL AFFECT can only be graded in FORCE. Features of [strong] to [weak] are shown as positions on a cline in Figure 5.4, realised through variations in muscle tension and/or the duration for which an expression is held.

Blogger Comments:

As previously demonstrated, the facial expression of emotion is not an interpersonal system of language, so the linguistic systems of APPRAISAL, such as ATTITUDE and GRADUATION, do not apply. Instead, from the perspective of SFL Theory, the facial expression of emotion serves the personal microfunction of protolanguage, and so what is presented here as graduated force is more consistently understood as a graduation of emotional intensity within the personal microfunction.

Misunderstanding Semiosis As Somasis

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 124):

Our focus is on the semiosis of facial expression realising emotion in human interaction, but it is important to note that the face can also manifest non-emotional (somatic) states. A frown, for example, might manifest concentrated thinking (Fasel and Luettin, 2003: 260) or physiological states of pain or fatigue (see Chapter 1). Instances of somatic facial expression can of course index purposeful feelings, which remains a challenge for analysts as discussed in Chapter 1. The approach taken in this book is that behaviours can be treated as paralinguistic (i.e. semiotic) depending on whether or not they are negotiated as meaningful in interaction.

Blogger Comments:

This is a very serious misunderstanding. If a facial configuration means ('manifests', indexes') something other than itself, then it is the signifier of a signified, and so semiotic, not "somatic". At the social level, in terms of Halliday's linear taxonomy of complex systems, some facial configurations select a positive value (e.g. 'approach') in the other, whereas some select a negative value (e.g. 'avoid').

Simultaneous Emotion

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 123-4):

A further consideration in analysing and interpreting facial expressions is the potential for one feature of facial affect to transition very quickly into another in an animated expression. An instance in example (4) expresses both [surprise] and [spirit:up]. 

From a systemic functional perspective, rather than describing this as a blending or merging of emotions it is considered as the co-instantiation of two different emotions with each realised through particular parts of the face (e.g. eyes, eyebrows, mouth) and often in very quick succession. In (4) the raised curved eyebrows realise [surprise] and the upturned lips realise [spirit:up]. 

A facial expression of [surprise], interpreted as a perturbance (Martin, 2017a) typically has the briefest duration and often transitions quickly to the expression of another emotion, one which responds to the specific trigger of the perturbance.

Blogger Comments:

[1] From a systemic functional perspective, this blurs the axial distinction between simultaneous systems ('both', 'co-instantiation', 'and') and syntagmatic order ('transition', 'succession'). Moreover, if two emotions can be realised in the same facial expression, the system network needs to be redrawn to represent simultaneous (conjunct) systems. This the authors have not done.

[2] On the one hand, the claim that a facial expression of surprise typically has the briefest duration is an instance of the logical fallacy known as ipse dixit: a bare assertion unsupported by evidence, and is belied by synonyms for 'surprised' such as 'stupefied' and 'dumbfounded'. On the other hand, surprise is the emotion that is the response to what triggered it as a perturbance.

Why Facial Affect Is More Limited Than Language

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 121):

As noted earlier, ATTITUDE in language can be expressed through systems of AFFECT, APPRECIATION or JUDGEMENT while paralinguistic expressions of ATTITUDE are restricted to FACIAL AFFECT (see, e.g. Tian, 2011). This means that the paralinguistic meaning potential for expressing emotion is relatively limited with respect to language. 

For example, an array of finely distinguished lexical instantiations of the feature [realis: happiness; mood; positive] (Table 5.1) are possible, as, for instance, in happy/joyful/delighted/thrilled and so on, such fine distinctions are not available in FACIAL AFFECT. 

In analyses of intermodal resonance in Coraline, fine distinctions in verbal instances (e.g. happy vs joyful) may be inferred for resonant facial expressions but cannot be attributed to specific variations in the facial expression. In other words a given expression of FACIAL AFFECT might couple with a diverse array of lexical realisations of [realis: happiness; mood; positive].

Blogger Comments:

To be clear, this relative limitation with respect to language is simply explained by the fact that the facial expression of emotion is a protolinguistic semiotic system, which means it lacks a grammatical stratum.

Neither Affect Nor Body Language

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 121, 122):

In summary, the network in Figure 5.3 models FACIAL EFFECT with six features of emotion: [spirit:up], [spirit:down], [fear], [anger], [disdain] and [surprise]. Each of these six features is illustrated and described in (2). Facial resources for expression are in bold.

Blogger Comments:

[1] As previously demonstrated, in terms of SFL Theory, the facial expression of emotion is protolanguage, serving the personal metafunction, and so not an interpersonal system (AFFECT) of the discourse semantics of language.

[2] As previously observed, the facial expressions constructed on clay puppets are not body language, but epilinguistic representations of body language created by animators using the emotion-face coding of Ekman.

Misrepresenting 'Threat' As An Emotion

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 120-1, 239n):

An additional opposition proposed by Darwin (1872) is between facial movements interpreted as ‘fear’ and ‘anger’. For Darwin, ‘fear’ is a feeling caused by the anticipation that one could be harmed (which we interpret as a response to what might happen, i.e. an irrealis trigger) and ‘anger’ is a feeling that might result in one harming others (which we interpret as a response to something real happening, i.e. a realis trigger). In the network of FACIAL AFFECT in Figure 5.3 [fear] and [anger] are opposing features of [threat]. Each feature is realised through a different set of facial expressions shown in italics. In the intersemiosis of facial expression and the unfolding storyline in language and action, the facial feature [fear] is interpretable as negative and irrealis, that is, it is a negative emotional response to what might happen. In contrast the feature [anger] is interpretable as negative and realis, an emotional response to what is happening or has happened.

Blogger Comments;

[1] As previously explained, the title Darwin (1872), The expression of the emotions in man and animals, demonstrates that the meanings here are protolinguistic, since other animals do not express the meanings of language. So, to model protolanguage as language, as FACIAL AFFECT, is theoretically invalid. On Halliday's model, the expression of emotion serves the personal microfunction of protolanguage. 

[2] To be clear, here the authors misrepresent the result of anger (harming others) as the reason for it (trigger).

[3] To be clear, in Figure 5.3, the authors misrepresent 'threat' as an emotion, with its result (fear) and cause (anger) as its subtypes.

[4] Again, the system in Figure 5.3 confirms the fact that here the authors model paralanguage as a bi-stratal semiotic system, and although this is consistent with the notion of a semiotic system, it is inconsistent with the preceding chapters in which paralanguage is misunderstood as an expression-only semiotic system. Where in previous chapters it was just paralinguistic expression that was semovergent with language, in this chapter it is both paralinguistic content and expression that is semovergent with language.

Problems With The System Of Facial Affect

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 120, 121):

The systemic functional semiotic system of FACIAL AFFECT presented in Figure 5.3 takes into account these important contributions in a number of areas, including descriptive terminology. For example, the naming of features in the model of FACIAL AFFECT avoids the use of Ekman’s terms of ‘happiness’ and ‘sadness’ as [happiness] is already a feature in linguistic ATTITUDE. Instead emotion terminology is sourced to Darwin’s (1872) opposition in facial movements of ‘high spirit’ and ‘low spirit’. Darwin’s influence is seen in Figure 5.3 in the naming of the feature [spirit] and its opposing features as [up] and [down].

Blogger Comments:

[1] To be clear, the system in Figure 5.3 confirms the fact that here the authors model paralanguage as a bi-stratal semiotic system. As previously noted, although this is consistent with the notion of a semiotic system, it is inconsistent with the preceding chapters in which paralanguage is misunderstood as an expression-only semiotic system. Where in previous chapters it was just paralinguistic expression that was semovergent with language, in this chapter it is both paralinguistic content and expression that is semovergent with language.

[2] To be clear, the title of Darwin (1872), The expression of the emotions in man and animals, acknowledges that the expression of emotion does not require the evolution and development of language. As such, the facial expression of emotion is protolanguage, not language.

Where AFFECT is a system of the interpersonal metafunction in the tri-stratal semiotic of language, protolanguage is a bi-stratal system that is pre-metafunctional. On Halliday's model, the expression of emotion serves the personal microfunction. 

In short, to model the personal microfunction of protolanguage as the interpersonal metafunction of language, as FACIAL AFFECT, is theoretically invalid.

An Inconsistent Use Of 'Semovergent'

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 119-20):

The linguistic system of AFFECT does not constitute a blueprint for the development of a system of PARALINGUISTIC AFFECT, its features or oppositions; the systems in the two modalities are named differently to reflect this (as in PARALINGUISTIC ENGAGEMENT and PARALINGUISTIC GRADUATION). PARALINGUISTIC AFFECT models expressions of emotion in FACIAL AFFECT with features realised through muscle movement of the face, and in VOICE AFFECT with features realised through qualities of the voice.

 

Blogger Comments:

To be clear, here the authors model paralanguage as a bi-stratal semiotic system. Although this is consistent with the notion of a semiotic system, it is inconsistent with the preceding chapters in which paralanguage is misunderstood as an expression-only semiotic system. Where in previous chapters it was just paralinguistic expression that was semovergent with language, in this chapter it is both paralinguistic content and expression that is semovergent with language.

Paralinguistic Expressions Of Attitude

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 117-8):

The discourse system of ATTITUDE (Figure 5.1) differentiates AFFECT as the expression of feelings or emotions from JUDGEMENT as the evaluation of people and behaviour and APPRECIATION as the assessment of phenomena (Martin and Rose, [2003] 2007; Martin, 2020). Of these three kinds of ATTITUDE only AFFECT is relevant to a discussion of interpersonal semovergence as paralinguistic expressions of ATTITUDE are restricted to those of emotion (e.g. Tian, 2011; Welch, 2005; Painter et al., 2013: 31–2).

Blogger Comments:

To be clear, the claim here is that, in the authors' model, paralanguage cannot enact judgement or appreciation.

Moreover, in the authors' terms, there is an unrecognised instance of a representation of judgement in the authors' data (p147):

If you point the finger at someone or point an accusing finger at someone, you blame them or accuse them of doing wrong.

Problems With A Phonological Analysis

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 117):

The example in (1) shows paralinguistic expressions sonovergent with each major tone choice (other than the level-rise tone 3 which offers minimal phonological scope for convergent body-part movement). A phonological transcription records the tone (as, e.g. //1), and the intonation contour describes and interprets it (as, e.g. falling – ‘certain’). The paralanguage which is sonovergent with each intonation contour – which visualises it – is then described. The resonance of the visual and phonological contours adds further salience to the tonic and hence the given tone choice.

Blogger Comments:

[1] To be clear, tone 3 is a level or low rise in pitch, so it can be realised facially by a maintained level or low rise in the eyebrows.

[2] To be clear, adding salience to the tonic is adding salience to what the tonic realises: the focus of New information, not to the tone choice that realises the system of KEY.

[3] To be clear, based on the system of KEY, the most likely tone for // I didn't / break it // is tone 2 ('protest'), not tone 4 ('reservation'), and this analysis is supported by the raised eyebrows on the clay puppet.

The Problem With Interpersonal Sonovergent Paralanguage

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 116):

Sonovergent paralanguage is only meaningful in its relation to the prosodic phonology of co-expressed speech (Halliday, 1967, 1970a; Halliday and Greaves, 2008; Smith and Greaves, 2015) (see Chapters 1 and 3). … Where interpersonal sonovergent paralanguage resonates with tone choices it is frequently expressed in up or down movements of the head, eyebrows or arms in tune with pitch movements in co-articulated speech.

Blogger Comments:

As previously explained, sonovergent paralanguage is neither sonovergent nor paralanguage. It is not sonovergent because the bodily expressions diverge from the phonological expressions, and it is not paralanguage because it is language, since the expressions realise the grammatical system of KEY. Again, this is why it is termed 'linguistic' in Cléirigh's model, which the authors in this book rebrand as their own.

The Designed Representations Of Body Language On Clay Puppets

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 116):

In stop-motion animations such as Coraline story characters are portrayed by puppets. Their movements, body language and facial expressions are designed by animators to match the pre-recorded speech of voice actors. The animations are developed frame-by-frame as animators make very small changes to the puppet’s face or body. Each change is captured as a ‘stop-motion’ or static frame before being collated and camera recorded to create movements in film (Laika Studios, 2017). In films, the characterisation of the main characters conveys the thematic message; accordingly, every facial or body movement of the main puppet characters (including the way they walk) is designed to express meaning – and so everything they do can be interpreted as semiotic rather than somatic behaviour (see Chapter 1; Mohamed and Nor, 2015; van Leeuwen, 2005). That said, the body movements of secondary characters arguably include both semiotic and somatic behaviour in order to progress the storyline. In this chapter we concentrate on the meaningful behaviour of the main characters.

Blogger Comments:

[1] To be clear, epilinguistic representations on clay puppets designed by animators are not instances of the body language of a member of socio-semiotic species.

[2] To be clear, the facial postures and movements of all the puppets designed by the animators represent both social and semiotic systems, in terms of Halliday's linear taxonomy of evolutionary complexity.

Foreshadowing Problems With Chapter 5 Analyses

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 115):

Analyses explore the paralinguistic systems of interpersonal sonovergence in which movements of parts of the body or face rise and fall in tune with the intonation contours of the prosodic phonology and interpersonal semovergence in which paralinguistic expressions converge with interpersonal meanings in spoken discourse. … System choices are illustrated in instances from Coraline and discussion focuses on intermodal convergences in expressions of emotion and the enactment of inter-character relations.

Blogger Comments:

[1] As previously explained, sonovergent paralanguage is neither sonovergent nor paralanguage. It is not sonovergent because the bodily expressions diverge from the phonological expressions, and it is not paralanguage because it is language, since the expressions realise the grammatical system of KEY. Again, this is why it is termed 'linguistic' in Cléirigh's model, which the authors in this book rebrand as their own.

[2] As previously explained, the notion of convergence misunderstands paralanguage as an expression-only semiotic system, and the notion of semovergence entails that these expressions realise the content of language. However, since the expression of these meanings (emotions) does not require the evolution and development of language, these systems are protolinguistic, not epilinguistic, which means that the meanings that are expressed are not metafunctional (interpersonal) but microfunctional (personal).

[3] As previously observed, the data used by the authors is not the body language of humans, but representations of body language on clay puppets, as constructed by animators, using the emotion-face coding proposed by Ekman. The data are thus epilinguistic depictions of a protolinguistic system.

The Representation Of Emotion On Clay Puppets As Paralanguage

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 115):

This chapter focuses on interpersonal meaning in paralanguage – on the ways in which the paralanguage of facial expression, voice quality, body gestures and positioning express feelings and enact social relations in cooperation with spoken language. The data are drawn from an award-winning stop-motion puppet animation film Coraline, directed by Henry Selick (2009) and based on a novella of the same name written by Neil Gaiman (2002).

Blogger Comments:

[1] To be clear, in modelling the facial expression of emotion in terms of a metafunction, the interpersonal, the authors present this semiotic system as one that requires the evolution and development of language. Clearly, the meaningful expression of emotion is not restricted to the one species in whom language has evolved, Homo sapiens, but is common to all social semiotic species. On this basis, the facial expression of emotion is a protolinguistic semiotic system, and in Halliday's model, is understood in terms of the personal microfunction.

[2] To be clear, the data for this chapter on paralanguage are not actually paralanguage, but representations of paralanguage constructed on clay puppets by animators, using  the emotion-face coding system of Ekman. The representation of paralanguage on clay puppets does require the prior development of language in the animators, and so is an epilinguistic semiotic system, in the terminology of Cléirigh's model.

Why All The Authors' Ideational Semovergent Systems Are Invalid

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 112-3):

This chapter has described how semovergent systems construe ideational meaning and has explored entities and figures as resources for embodied ideational meaning across language and paralanguage. These systems have been formalised in system networks that can be used by an analyst as they consider how gestures interact through a relationship of concurrence or divergence with the ideational meanings made in spoken discourse. …

A robust analytical framework for investigating ideational meaning offers a key resource for understanding human experience in social life. The ideational paralinguistic systems presented in this chapter have important potential in applied linguistics where adopting a multimodal approach to studying communication involving multiple modalities is becoming increasingly important. … We look forward to seeing how the systems explored in this chapter are taken up in disciplines such as the humanities and in studies of different semiotic modes (including face-to-face communication and communication in digital environments).

 

Blogger Comments:

[1] As previously demonstrated, the authors' notion of semovergent systems, where gestures realise ("interact through a relationship of concurrence") the ideational meanings of language, derives from their misunderstanding of paralanguage as an expression-only semiotic system.

[2] As previously observed, all eight of the system networks in this chapter confuse discourse semantics with expression plane systems and features.

[3] As the review of this chapter has demonstrated, the framework presented here is not even theoretically valid, let alone "robust".

Confusing Discourse Semantics And Expression In A System Network [7]

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 111):

A system network bringing together the various choices we have covered in the previous sections is provided in Figure 4.8.

 Blogger Comments:

As previously explained, the term 'paralinguistic figure' confuses discourse semantics (figure) with paralanguage misunderstood as an expression-only semiotic system. The system network in Figure 4.8 further demonstrates this confusion by presenting a discourse semantic network (figure) with both discourse semantic features (e.g. state figure, occurrence figure) and expression plane systems (e.g. RECURRENCE, FLOW, DIRECTION).

Confusing Discourse Semantics And Expression In A System Network [6]

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 109-10):

The final two dimensions to consider when analysing an occurrence figure are flow and direction – as outlined in Figure 4.7.

 Blogger Comments:

As previously explained, the authors' notion of an 'occurrence figure' confuses discourse semantics (figure) with paralanguage misunderstood as an expression-only semiotic system. The system network in Figure 4.7 further demonstrates this confusion by presenting a discourse semantic network (occurrence figure) with expression plane systems of gestural motion (FLOW, DIRECTION).

Confusing Discourse Semantics And Expression In A System Network [5]

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 107-8):

Another dimension of occurrence figures has to do with whether or not they incorporate gestures that repeat – [iterated] versus [isolated], and if so, in what manner – [ordered] versus [unordered], and if [ordered], then [to-and-fro] or [stepped]. These options are outlined in Figure 4.6.

 Blogger Comments:

As previously explained, the authors' notion of an 'occurrence figure' confuses discourse semantics (figure) with paralanguage misunderstood as an expression-only semiotic system. The system network in Figure 4.6 further demonstrates this confusion by presenting a discourse semantic network (occurrence figure) with the expression plane features (iterated, ordered, to-and-fro, stepped, unordered, isolated) of an expression plane system of gestural motion (RECURRENCE).

Confusing Discourse Semantics And Expression In A System Network [4]

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 104, 105):

Where an entity is present in the paralinguistic realisation of an occurrence figure this entity may change, [transformative] versus [non-transformative] in either size, [increase] versus [decrease] or [shape]. These options are outlined in Figure 4.5. If it remains a constant size or shape, it may impact another entity in the gestural space, [impacting] versus [non-impacting].

 Blogger Comments:

As previously explained, the terms 'paralinguistic entity' and 'paralinguistic figure' confuse discourse semantics (entity, figure) with paralanguage misunderstood as an expression-only semiotic system. The system network in Figure 4.5 further demonstrates this confusion by presenting a discourse semantic network (entitied occurrence figure) with both discourse semantic features (entitied, non-entitied) and expression plane features (transformative, size, increase, decrease, shape, non-transformative, impacting, non-impacting).

Moreover, realisation statements like 'insert entity' specify a constraint on structural configuration — cf. insert Agent — but no structural configuration for occurrence figures has been identified.

Confusing Discourse Semantics And Expression In A System Network [3]

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 104, 105):

Paralinguistic occurrence figures incorporate motion to construe a happening or activity. Unlike paralinguistic state figures which always visually incorporate a paralinguistic entity, a paralinguistic occurrence figure can occur both with or without committing a definable entity. There are three other dimensions along which such figures vary: whether or not the motion repeats (iterated/isolated), the speed of the motion (constant/adjusted) and the direction of the motion (omni/linear) – as shown in the system network in Figure 4.4.

Blogger Comments:

[1] As previously explained, the terms 'paralinguistic entity' and 'paralinguistic figure' confuse discourse semantics (entity, figure) with paralanguage misunderstood as an expression-only semiotic system. The system network in Figure 4.4 further demonstrates this confusion by presenting a discourse semantic network (occurrence figure) with expression plane systems of gestural motion (RECURRENCE, FLOW, DIRECTION).

[2] To be clear, this use of 'committing' misunderstands the authors' own notion of commitment. The authors' notion of commitment is misunderstood as the degree of delicacy selected in the process of instantiation. Here the term is used, not for delicacy, but for the relation between a figure and one of its constituents (entity).

Confusing Discourse Semantics And Expression In A System Network [2]

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 102):

State figures involve either single paralinguistic entities that are simply manifested or more than one entity that enters into an association with another one (the [presentational] vs [relational] options in Figure 4.3). These paralinguistic entities are not involved in a paralinguistic occurrence. For relational state figures, the association may be represented via variations in either the relative size or relative position of the entities, or both, within the gestural space.

Blogger Comments:

As previously explained, the terms 'paralinguistic entity' and 'paralinguistic (occurrence) figure' confuse discourse semantics (entity, figure) with paralanguage misunderstood as an expression-only semiotic system. The system network in Figure 4.3 further demonstrates this confusion by presenting a discourse semantic network (state figure) with an expression plane system of features (size, position).

Confusing Discourse Semantics And Expression In A System Network [1]

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 101-2):

The system network in Figure 4.2 outlines the paralinguistic figures which can concur with these kinds of meanings. It distinguishes between paralinguistic occurrence figures, in which a paralinguistic entity is involved in an activity, and paralinguistic state figures, where a paralinguistic entity is manifested. Each type of paralinguistic figure can be positioned in space, relative to the neutral position adopted by a speaker where most of their gestures occur (in front of the speaker’s solar plexus with elbows slightly bent). Paralinguistic state figures necessarily involve an entity; paralinguistic occurrence figures necessarily involve motion (as specified by the realisation statements following the downward slanting arrows in the network).

 Blogger Comments:

As previously explained, the terms 'paralinguistic entity' 'paralinguistic figure' confuse discourse semantics (entity, figure) with paralanguage misunderstood as an expression-only semiotic system. The system network in Figure 4.2 further demonstrates this confusion by presenting a discourse semantic network (figure) with expression plane systems and features (positioned, neutral). This confusion is compounded by including one discourse semantic feature (state figure) realised by the insertion of a constituent discourse semantic feature (entity), and the other discourse semantic feature (occurrence figure) realised by an expression plane feature (motion).

Confusing Semantic Figures With The Gestures That Realise Them

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 100):

In order to explore paralinguistic entities in more detail it is necessary to consider how they enter into paralinguistic figures in discourse. This will enable us to account for how paralinguistic entities are variously manifested, or presented as relating to other paralinguistic entities, and/or involved in actions or happenings in the discourse.

Blogger Comments:

As previously explained, the term 'paralinguistic entity' confuses discourse semantics (entity) with paralanguage misunderstood as an expression-only semiotic system. Here the authors further introduce the term 'paralinguistic figure' which again confuses discourse semantics (figure) with paralanguage misunderstood as an expression-only semiotic system

Confusing Content With Expression, Semiosis With Somasis, And Paralanguage With Language

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 97):

DEPICTION considers whether the paralinguistic entity has a defined contour that is visually ‘drawn’ in space (by, e.g. drawing the outline of an entity with a pointed finger in the air) or ‘sculpted’ (by, e.g. cupping a hand as in the example in (3)). The features sculpted and drawn correspond to two of Müller’s (1998) four modes of expression used in representational gestures: drawing (tracing the silhouette of an object in the air with a finger or hand) and moulding (sculpting or shaping the form of an object with the hands). 

Müller’s (1998) two other modes, imitating/acting (‘acting out’ an action) and representing/portraying (where the hands represent an object, e.g. a ‘V’ shape made with middle fingers to represent scissors) are dealt with in Chapter 1 in terms of somasis and emblems, respectively.

Blogger Comments:

[1] As previously explained, the term 'paralinguistic entity' confuses discourse semantics (entity) with paralanguage misunderstood as an expression-only semiotic system. To be clear, gestures are expressions that realise semantic entities; they are not the entities — just as phonemes that realise semantic entities are not the entities.

[2] To be clear, the system of DEPICTION confuses content (entity) with its expression (drawn, sculpted).

[3] To be clear, here the authors misunderstand semiosis as nonsemiosis (somasis). A gesture that imitates (mimes) an action represents that action, and as such, is semiotic, since it means something other than the gesture itself.

[4] As seen in Chapter 1, the authors treat emblems as part of language, rather than paralanguage, so here they are claiming that a hand shape that means 'scissors' is language. To be clear, even in their own model, this hand shape semovergently realises the entity 'scissors'. Here again the authors have become confused by taking the view 'from below' (expression) rather than the view 'from above' (content). That is, because the V-shape meaning 'scissors' resembles the V-shape meaning 'two', they have classified it in the same way: as an emblem.

Confusing Semantic Entities With The Gestures That Realise Them

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 96-7):

Paralinguistic entities may also vary in terms of SIZE relative to the amount of gestural space taken up by the prosodic unfolding of gestures in a stretch of discourse. In other words they may be, for instance, bigger or smaller than other entities that have occurred up to a given point in the speaker’s discourse. For example, the ‘heaping bowl of Chex Mix’ gesture in (3) is large relative to the ‘applesauce squeeze’ gesture in (1) that it precedes.

Blogger Comments:

As previously explained, terms such as 'paralinguistic entity' confuse discourse semantics (entity) with paralanguage misunderstood as an expression-only semiotic system (size, gestural space, gestures, bigger, smaller). To be clear, the gestures are the expressions that realise the semantic entities; they are not the entities — just as the phonemes that realise semantic entities are not the entities.

Confusing Semantic Content With Its Paralinguistic Expression

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 96):

Alternatively, the paralinguistic entity may be shaped as either two- or three-dimensional, with rounded or straightened hands and fingers. For example, the vlogger gestures defined entities when referring to the bump formation of the granuloma on her foot (2).

 

Blogger Comments:

[1] As previously explained, terms such as 'paralinguistic entity' confuse discourse semantics (entity) with paralanguage misunderstood as an expression-only semiotic system (shape, gesture).

[2] To be clear, the terms 'shaped', 'gestures' and 'referring to' function here to express the realisation relation between paralinguistic expression and linguistic content:

• a linguistic entity may be realised as either two- or three-dimensional, with rounded or straightened hands and fingers;
• the vlogger's gestures realise defined entities;
• the vlogger's gestures realise the bump formation (but see [3] below).

[3] Again, this hand shape realises the 'bubbling up' of the granuloma after the injection of the steroid.

Confusing Semantic Entities With Gestural Expressions

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 96):

The SHAPE of a paralinguistic entity may be default, which means that it is rendered simply as a thing held in one or two hands in front of the body – with hand and fingers in a relaxed naturally cupped configuration. At the end of the ‘Hair Dye’ phase, for example, the vlogger is interrupted by her hungry children, and when filming resumes she explains that she has already given them a heaping bowl of ‘Chex Mix’ with applesauce squeeze – and she uses a default entity gesture for the applesauce.

Blogger Comments:

[1] As previously explained, terms such as 'paralinguistic entity' and 'entity gesture' confuse discourse semantics (entity) with paralanguage misunderstood as an expression-only semiotic system (shape, gesture).

[2] Strictly speaking, the hand shape does not realise the meaning 'applesauce'. Instead, it realises (mimes) the manner of holding a pouch of applesauce Squeez.