Most Pageviews by Country

The Authors' Intention In Chapter 5

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 159):

Researchers from both social semiotics and linguistics (Thibault, 2004; Feng and O’Halloran, 2013; Mondada, 2016; Lim, 2019) have recently raised the challenge of developing a holistic approach to the study of social interaction. They argue persuasively that it is not sufficient to single out just one or two semiotic modes for examination (e.g. language and facial expression or language and gesture) if we are to understand the meaning of social interactions. 

The intention in this chapter is to respond to the challenge by providing a systemic functional social semiotic account of a number of paralinguistic systems as a framework for studying the orchestration of multiple semiotic modes in interaction in the expression of interpersonal meaning and the enactment of social relations in the context of animated film. 

We look forward to reports of research adapting our framework to the study of interpersonal relations in other modalities of interaction, in film, theatre, clinical, educational and forensic contexts, casual conversation and beyond, and additionally to its application in educational contexts. In relation to the latter, as an exemplary animation of its kind, Coraline offers a significant educational resource.

Blogger Comments:

[1] To be clear, the extent to which the authors have succeeded in responding to this challenge can be gauged by carefully considering all the posts that examine this chapter (here).

[2] To be clear, the representations of body language on clay puppets by animators using the emotion-face coding system of Ekman are not instances of the system of human body language, and so do not constitute valid data for theorising.

[3] To be clear, the authors' framework is their misunderstanding of Cléirigh's framework.

Problems With The Authors' Analysis Of Paralinguistic Engagement

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 157, 158):

A final analytical step involves an exploration of PARALINGUISTIC ENGAGEMENT and features of [heteroglossic:contraction] and [heteroglossic:expansion]. Relevant here are shifts in body posture in the sequence of four images. In the first image (the angry stomp), the upper body and head are in a prone (closed) position. The torso remains prone in the second and third images, at the same time as the head/face is progressively raised from prone to neutral in the final image. These realisations of [contraction] are accompanied in the second and third images with realisations of [heteroglossic:expansion] – as Coraline moves her arms and hands to an open supine position by her sides, and in addition raises her eyes in the third image. 

In the final image in (30), Coraline’s face continues to express [anger]. Although she stands upright, her posture is prone in certain respects – her shoulders are rounded, her arms are close to her body and her hands are clasped, closing off her torso (enacting [contraction]). At the same time, however, her head and face are decentred – an expression of [heteroglossic:expansion] (Hao and Hood, 2019).

The apparently disjunctive concatenation of options in PARALINGUISTIC ENGAGEMENT needs to be interpreted in relation to the attendant semovergence. From the perspective of affiliation, as discussed earlier, the rhetorical question in the spoken text functions as a firm rejection of Wybie’s tendered coupling; it is this discourse move that resonates with the prone features of Coraline’s posture – those realising [contraction]. At the same time the supine features of her posture – those realising [expansion] – open up space for ongoing interaction and the negotiation of other potential bonds. The semiotic resources of the body negotiate relations on two fronts simultaneously – retrospective [contraction] and [prospective expansion].

 

Blogger Comments:

To be clear, in terms of ENGAGEMENT, the question If I'm a water witch, then where's my secret well is an instance of [heteroglossic: contraction: disclaim]. Martin & White (2003: 118):

Under disclaim we cover those formulations by which some prior utterance or some alternative position is invoked so as to be directly rejected, replaced or held to be unsustainable.

Given that there is no heteroglossic expansion instantiated in language, the authors are faced with the contradiction that the representation of body language on the clay puppet also realises [expansion], according to their model. The authors' solution is to interpret the expansion gestures as opening up space for further dialogue.

However, this interpretation is clearly inconsistent with the meanings being enacted at this point in the text, especially in light of the fact that, of the two types of expansion [entertain] and [attribute], the only option open to body language here would be [entertain] . Martin & White (2003: 104):

We begin with what we term ‘entertain’ – those wordings by which the authorial voice indicates that its position is but one of a number of possible positions and thereby, to greater or lesser degrees, makes dialogic space for those possibilities. …

Under the heading of ‘attribution’, we deal with those formulations which disassociate the proposition from the text’s internal authorial voice by attributing it so some external source.

Clearly, then where's my secret well is not making dialogic space for other possibilities. This suggests that there are serious problems with the authors' model of PARALINGUISTIC ENGAGEMENT.

Problems With The Authors' Analysis Of A Discourse Move

 Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 156, 157-8):

In Coraline’s first encounter with Wybie, a boy of her own age from the same neighbourhood, he accuses her of being a water witch to which she responds: //3 ^ and if / I’m a / water / witch //1 ^ then / where’s the secret / well //. The focus in (30) is on the second tone group of this utterance, that is, //1 ^ then / where’s the secret / well //. … 

In the spoken language of this tone group there is apparently no resonant inscribed or invoked linguistic AFFECT. However, before we assume a divergent semovergent relation, there is more to be considered in the verbal and imagic co-text. 

The spoken language in (30) configures a question through a wh- interrogative on a falling tone 1 (signalling ‘certainty’). Taken in conjunction with the PARALINGUISTIC expressions [anger], this discourse move (then where’s the secret well) can be interpreted as a rhetorical question, one that challenges Wybie’s judgemental accusation that she is a water witch. From the perspective of affiliation and the negotiation of bonds (Section 5.3.4), Coraline is forcefully rejecting the coupling proposed by Wybie.

Blogger Comments:

[1] To be clear, the 'certainty' realised by tone 1 is 'polarity known'. Halliday (1994: 302):

[2] This misunderstands both the text and the notion of a rhetorical question. A rhetorical question is one that does not demand information from an addressee. The question then where’s the secret well is not rhetorical, because demands from the addressee the information that would validate the proposition that she is a water witch.

Problems With The Authors' Analysis Of The Resonance Of Affect And Force

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 156, 157):

In Coraline’s first encounter with Wybie, a boy of her own age from the same neighbourhood, he accuses her of being a water witch to which she responds: //3 ^ and if / I’m a / water / witch //1 ^ then / where’s the secret / well //. The focus in (30) is on the second tone group of this utterance, that is, //1 ^ then / where’s the secret / well //. … 

The first image in (30) captures Coraline forcefully stomping her right foot and punching down with her arms and clenched hands in an expression of PARALINGUISTIC AFFECT – [anger] with [strong] FORCE. The voice quality on where realises VOICE AFFECT as [anger] – through high intensity, tension and roughness (shown as the grey area in the spectrogram in (30)). Coraline’s face is not visible in the first image; but a prosody of FACIAL AFFECT [anger] is additionally realised more or less intensively in the remaining three images – as the eyebrows are drawn down and together. These expressions of PARALINGUISTIC FORCE in realisations of [anger] resonate with and amplify one another.

Blogger Comments:

To be clear, the depiction of body language on an animated clay puppet is not body language, but an epilinguistic construal of body language, because the representation of body language on an animated clay puppet requires the prior development of language in the animator; it is not something that a dog or cat, for example, could do.

In considering the body language that is thus epilinguistically depicted:

[1] As previously explained, the beating of the foot and arms realises textual salience, and is linguistic, like the beats of speech. Here the authors again misconstrue this textual salience as interpersonal force (and as epilinguistic instead of linguistic).

[2] As previously explained, the bodily expression of emotion is protolinguistic, and so pre-metafunctional, because it does not require the prior evolution and development of language, as demonstrated by Darwin's work on the expression of emotions in other animal species. Here, however, the authors misconstrue the expression of emotion as requiring the prior evolution and development of language (epilinguistic) and locate it within the interpersonal metafunction, regardless of whether or not it is used to evaluate.

[3] To be clear, as the above demonstrates, the "resonance" here is between the protolinguistic expression of emotion (which other animals can do) and the linguistic expression of salience (which other animals cannot do).

The Claim That The Face Does Not Express Desire

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 152, 156):

The possibility of different paralinguistic resources being instantiated simultaneously allows us to infer meanings not necessarily interpretable from an expression in a single paralinguistic mode. For example, FACIAL AFFECT has no distinct option for the expression of desire. However, when raised eyebrows and wide-opened eyes (realising FACIAL AFFECT as [surprise]) are expressed convergently with PARALINGUISTIC PROXIMITY as [personal] and PARALINGUISTIC ORIENTATION as [involved], the emotion of desire is strongly invoked. Two such instances are described in (29).

Blogger Comments:

[1] To be clear, this is a serious omission in the authors' model. The bodily expression of desire is enshrined in Shakespeare's Julius Cæsar:

Yond Cassius has a lean and hungry look

and at least one of the following facial expressions might reasonably be construed as expressing desire:

[2] To be clear, the authors' claim here is that a personal, involved expression of surprise "invokes" (i.e. evokes) an expression of desire. In contrast, none of the desiring faces above look at all surprised.

Problems With The System Of Paralinguistic Power

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 150-1):

Social relations of relative POWER in images relate to the vertical angle of viewing in Kress and van Leeuwen (2006). In Painter et al. (2013) it relates to the vertical positioning of one character’s body in relation to another. In van Leeuwen (1999), POWER is also discussed as an aspect of interpersonal meaning afforded by the voice; the higher in pitch and the louder the voice is, the more dominant the speaker. The system of PARALINGUISTIC POWER in Figure 5.17 opposes features of equal and unequal on a cline and realised through the vertical positioning of bodies in relation to each other. The features of relative pitch and loudness are not identified as realisations in Figure 5.17.

Blogger Comments:

[1] To be clear, the claim here is that the loud, high-pitched cry of a depicted child is dominant, whereas the soft, low pitch of the depicted child's adult male teacher is subordinated. This is also indirectly at odds with the fact that female newsreaders are trained to lower the pitch of their voice in presenting authoritative stories of events.

[2] To be clear, the claim here is that a depicted head of a tall person is dominant, whereas a depicted head of a short person is subordinated, and a depicted teacher and student of the same height are equal in power.

[3] On the contrary.

[4] Like previous systems in this chapter, the system in Figure 5.17 models paralanguage as a bi-stratal semiotic system, and although this is consistent with the notion of a semiotic system, it is inconsistent with the preceding chapters in which paralanguage is misunderstood as an expression-only semiotic system. Where in previous chapters it was just paralinguistic expression that was semovergent with language, in this chapter it is both paralinguistic content and expression that is semovergent with language.

The Contextual Dependence Of The Meaning Of Body Orientation

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 149):

Greater or lesser involvement is realised through the horizontal angle between the characters and the presence or absence of accompanying gaze. At one end the features [involved] indicates maximum involvement though face-to-face orientation accompanied by mutual gaze. At the other end of the continuum there is an absence of involvement; the interlocutors share no gaze and have a widely oblique or even back-to-back orientation in relation to each other. Between these endpoints, body (and head) angle varies and involvement with the other may be enhanced by direct gaze or weakened by a lack of it. An oblique angle to another realises a [less involved] or relatively detached orientation, while face-to-back indicates [involvement sought] or a desire to engage and back-to-back indicates [uninvolved] or thorough disengagement. A side-by-side orientation on the other hand realises a solidarity relation but yet [less involved]. Note that the head and the body can be angled relatively independently so there are more points on the continuum than actually specified here.

Blogger Comments:

To be clear, the 'involvement' meanings of these body orientations depend crucially on context. For example, consider the "solidarity" of two strangers sitting side-by-side on public transport or in a waiting room, or the "involvement sought" of two strangers face-to-back in a queue, or in former times, of "uninvolved" orientation of two duellists standing back-to-back before trying to kill each other.

A Problem With The System Of Paralinguistic Orientation

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 149, 150):

Analogising from Kress and van Leeuwen’s account of viewer/depiction relations referred to as INVOLVEMENT, Painter et al. (2013) propose a system of body ORIENTATION as an additional means for interpreting relations between depicted characters in images. Figure 5.16 shows options in a system of PARALINGUISTIC ORIENTATION and how they are relatively positioned as degrees of involvement.

Blogger Comments:

[1] To be clear, Kress and van Leeuwen’s original system is construed in terms of content (INVOLVEMENT), whereas the authors' rebranding of it is construed in terms of expression (ORIENTATION).

[2] To be clear, the system in Figure 5.16 models paralanguage as a bi-stratal semiotic system, and although this is consistent with the notion of a semiotic system, it is inconsistent with the preceding chapters in which paralanguage is misunderstood as an expression-only semiotic system. Where in previous chapters it was just paralinguistic expression that was semovergent with language, in this chapter it is both paralinguistic content and expression that is semovergent with language.

Not Acknowledging The Intellectual Source Of 'Paralinguistic Proximity'

 Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 147-8):

Opposing features in the system of PARALINGUISTIC PROXIMITY are: [personal], realised through close body positioning of characters vis-à-vis one another; [social] as realised through greater separation of the characters within a picture frame; and [impersonal] through distanced separation of the characters. These features are presented along a cline of PARALINGUISTIC PROXIMITY in Figure 5.15.

Blogger Comments:

[1] To be clear, the unacknowledged intellectual source of the features of the authors' system of PARALINGUISTIC PROXIMITY is Edward T. Hall (1963):

Hall described the interpersonal distances of humans (the relative distances between people) in four distinct zones: 

• Intimate distance for embracing, touching or whispering
  • Close phase – less than one inch (0.01 to 0.02 m)
  • Far phase – 6 to 18 inches (0.15 to 0.46 m)
• Personal distance for interactions among good friends or family
  • Close phase – 1.5 to 2.5 feet (0.46 to 0.76 m)
  • Far phase – 2.5 to 4 feet (0.76 to 1.22 m)
• Social distance for interactions among acquaintances
  • Close phase – 4 to 7 feet (1.2 to 2.1 m)
  • Far phase – 7 to 12 feet (2.1 to 3.7 m)
• Public distance used for public speaking
  • Close phase – 12 to 25 feet (3.7 to 7.6 m)
  • Far phase – 25 feet (7.6 m) or more.

A chart depicting Edward T. Hall's interpersonal distances

[2] To be clear, the system in Figure 5.15 models paralanguage as a bi-stratal semiotic system, and although this is consistent with the notion of a semiotic system, it is inconsistent with the preceding chapters in which paralanguage is misunderstood as an expression-only semiotic system. Where in previous chapters it was just paralinguistic expression that was semovergent with language, in this chapter it is both paralinguistic content and expression that is semovergent with language.

Not Acknowledging The Intellectual Source Of 'Social Distance' And 'Proximity'

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 147):

Kress and van Leeuwen’s system of SOCIAL DISTANCE (2006) relates to the constructed social relation between viewer and depicted person and is realised through shot size (e.g. close-up versus long shot). Painter et al. (2013) adapt this notion of relative distance to refer to the constructed social relation between depicted characters within images as PROXIMITY.  

Blogger Comments:

To be clear, unacknowledged by the authors, the intellectual source of both Kress and van Leeuwen’s system of SOCIAL DISTANCE (2006) and the adaptation of their notion by Painter et al. (2013) as PROXIMITY is the work on proxemics by Edward T. Hall, published in 1963):

Edward T. Hall, the cultural anthropologist who coined the term in 1963, defined proxemics as "the interrelated observations and theories of humans' use of space as a specialised elaboration of culture". In his foundational work on proxemics, The Hidden Dimension, Hall emphasised the impact of proxemic behavior (the use of space) on interpersonal communication.

More specifically, both derive from the application of Edward T. Hall's proxemics to cinema. The work of Painter et al. (2013) derives from character proxemics, and the work of Kress and van Leeuwen derives from camera proxemics:

Proxemics is an essential component of cinematic mise-en-scène, the placement of characters, props and scenery within a frame, creating visual weight and movement. There are two aspects to the consideration of proxemics in this context, the first being character proxemics, which addresses such questions as: How much space is there between the characters?, What is suggested by characters who are close to (or, conversely, far away from) each other?, Do distances change as the film progresses? and, Do distances depend on the film's other content?

The other consideration is camera proxemics, which answers the single question: How far away is the camera from the characters/action? Analysis of camera proxemics typically relates Hall's system of proxemic patterns to the camera angle used to create a specific shot, with the long shot or extreme long shot becoming the public proxemic, a full shot (sometimes called a figure shot, complete view, or medium long shot) becoming the social proxemic, the medium shot becoming the personal proxemic, and the close up or extreme close up becoming the intimate proxemic.

Problems With The Authors' Analysis Of Paralinguistic Focus

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 146-7):

In (24), PARALINGUISTIC FOCUS as [sharpen] is expressed in the narrowly targeted index-finger point that zooms in towards the Other Mother. This expression of sharpened FOCUS functions to identify the target (Other Mother) in a highly specifying manner (see Chapter 6). This together with the expression of negative judgement in the spoken language in stole serves to amplify the expression of [disdain] in FACIAL AFFECT.

Blogger Comments:

[1] To be clear, GRADUATION is the scaling of an interpersonal APPRAISAL (Martin & White 2005: 135). The evaluation here is made through the process stole, so any upscaling of the evaluation must be an upscaling of stole not of the evaluated you. So there is no sharpening of the FOCUS of evaluation here (nor a quantifying of the FORCE of evaluation expressed by the extended arm). The interpersonal function of the orientation of the index-finger of the clay puppet here is simply deictic: it points to the addressee you from the speaker I .

[2] To be clear, this is an instance of 'pointing the finger' which is to accuse or blame (someone) — a 'j'accuse' — a gesture which the authors might easily have classified, in their own terms, as an emblem, since they write (p38):

Gestures treated as playing a speech functional role in dialogue in other models are treated as emblems in our framework.

In terms of Cléirigh's original model, this is a genuine example of epilinguistic body language — not to be found in protolinguistic species — in which the gesture expresses an appraisal of judgement, an accusation of wrongdoing, which the authors claimed (pp118, 121) that body language can not do.

[3] To be clear, the judgement instantiated as body language is consistent with the judgement instantiated as language. Whether or not the face of the clay puppet here specifically represents disdain, a feeling of contempt for someone or something regarded as unworthy or inferior is arguable, at the very least.

Problems With The Authors' Analysis Of Paralinguistic Force

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 146):

In (23), Coraline is arguing with her mother about why she has locked a tiny door. Convergent with dreams aren’t dangerous, her left hand depicts the proposition (dreams aren’t dangerous) as a semiotic entity (see Chapter 4) at the same time as her left arm is extended out front of her body. The expression realises PARALINGUISTIC FORCE as [quantify:size:extent]. In this instance FORCE is expressed in the embodied paralanguage but not in convergent spoken language.

Blogger Comments:

[1] This is a bare assertion since it is unsupported by argument: the ipse dixit fallacy. Moreover, it is demonstrably false. In terms of practicability, the reader is invited to use one hand to represent dreams aren’t dangerous as an entity. In terms of theory, if this were possible, it would be an instance of grammatical metaphor — a figure realised as an element (Halliday & Matthiessen (1999: 250) — in a semiotic system without a grammar: a contradiction in terms.

[2] To be clear, the claim here is that the extending of an arm to represent the extent of an entity is an instance of GRADUATION, the scaling of an interpersonal APPRAISAL (Martin & White 2005: 135). This is demonstrably false. Firstly, the representation of the extent of an entity is an ideational construal, not an interpersonal appraisal. Secondly, the only entity here is dreams, and this mental 'process thing' (Halliday & Matthiessen 1999: 244) is clearly not represented by the hand, and the extent of the arm does not represent the extent (duration) of dreams. Thirdly, the evaluation here is made through the quality dangerous, so any upscaling of the evaluation must be an upscaling of dangerous not of dreams, and this the extending of the arm does not represent.

Moreover, this image contradicts the authors' model of PARALINGUISTIC ENGAGEMENT, because here a supine hand is used to represent the [monogloss] of dreams aren’t dangerous, whereas on the authors' model a supine hand represents not only [heterogloss], but [heterogloss: expansion], which is 'allowing space for other voices' (p143).

Misunderstanding Textual Language As Interpersonal Paralanguage

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 146):

Embodied paralanguage can adjust the FORCE of verbally expressed meanings through options of [intensify] and [quantify]. The feature [intensify] can be realised in a number of ways: through increased muscle tension in a hand-beat; in a very rapid frequency of such beats (syncing with syllable-timed rhythm in the prosodic phonology of English – see Chapter 6); or in the holding of the completed position of a beat for an extended time.

Blogger Comments:

[1] As Halliday (1967) pointed out, the prosodic phonology of English is foot-timed, not syllable-timed (e.g. Italian).

[2] To be clear, the beating of the hands with the rhythm of speech is the use of the hands to realise the same content as those realised by the rhythm of speech. That is, the function of the hand-beats is linguistic, to realise textual salience, which is why such gestures are classified as linguistic in Cléirigh's model. So here the authors have misunderstood the textual salience as interpersonal force. Put in their own terms, the authors have misunderstood textual sonovergent paralanguage as interpersonal semovergent paralanguage.

A Problem With The System Of Paralinguistic Graduation (Body)

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 145):

The system of PARALINGUISTIC GRADUATION (body) (Hood and Zhang, 2020) is shown in Figure 5.14.

Blogger Comments:

The system in Figure 5.14 models paralanguage as a bi-stratal semiotic system, and although this is consistent with the notion of a semiotic system, it is inconsistent with the preceding chapters in which paralanguage is misunderstood as an expression-only semiotic system. Where in previous chapters it was just paralinguistic expression that was semovergent with language, in this chapter it is both paralinguistic content and expression that is semovergent with language.

Problems With The Authors' Analysis Of Paralinguistic Engagement

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 143, 144-5):

The primary opposing features are [monogloss] in which other voices are ignored, and [heterogloss] in which other voices are implicitly or explicitly allowed into the discourse. If [heterogloss] is selected, the opposing features are [heterogloss: expansion], allowing space for other voices, or [heterogloss: contraction], closing down space for other voices. …

Instances of gesture and posture realising PARALINGUISTIC ENGAGEMENT options are shown in (22).

 

Blogger Comments:

[1] To be clear, the wording Be strong Coraline is [monogloss], not [heterogloss], because it does not allow other voices into the discourse. So, in this instance, the compressed face actually expresses [monogloss], not heterogloss: contraction], contradicting the authors' system.

[2] To be clear, the wordings

Dad I'm not five anymore

Don't believe me

I didn't break it

are all [heterogloss: contraction], not [heterogloss: expansion], because they all close down the space for other voices. So, in these instances, the

prone head; closed body
open face; decentred head
supine head and hands; open body

all express [heterogloss: contraction], not [heterogloss: expansion], contradicting the authors' system.

Problems With The System Of Paralinguistic Engagement

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 144):

From a social semiotic approach, Hood (2011) considers how embodied paralanguage resonates with linguistic ENGAGEMENT resources (see Figure 5.13).

 

For example, a prone (palm down) hand gesture realises [contraction] and functions to close down space for the negotiation of propositions or proposals. A supine (palm up) hand gesture realises [expansion] and functions to open up space for negotiation. Hao and Hood (2019) and Hood and Zhang (2020) also discuss an oscillating movement of the hand as softening focus in relation to the fulfilment or actualisation of a propositional figure, while additionally realising [heteroglossic: expansion]. Heteroglossic [expansion] and [contraction] are frequently realised through the positioning of the hands but can also be expressed through a more general open or closed posture of the body torso or the positioning of the head. An open face (tilted upwards) realising [expansion] will also display relaxed rather than compressed facial muscles.

Blogger Comments:

[1] To be clear ,the system in Figure 5.13 models paralanguage as a bi-stratal semiotic system, and although this is consistent with the notion of a semiotic system, it is inconsistent with the preceding chapters in which paralanguage is misunderstood as an expression-only semiotic system. Where in previous chapters it was just paralinguistic expression that was semovergent with language, in this chapter it is both paralinguistic content and expression that is semovergent with language.

[2] Importantly, this use of body language requires the prior evolution of language — it is not found in pre-linguistic species — and so is epilinguistic, in terms of Cléirigh's model. This contrasts with the prior discussion in this chapter of the bodily expression of emotion, which is found in pre-linguistic species, and so is protolinguistic, in terms of Cléirigh's model.

Social Bonding Through Reciprocated Expressions Of Emotion

 Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 140-1): 

When such couplings are tendered in interaction with others and reciprocated they are said to constitute bonds, and it is through the sharing of multiple bonds that we build affiliating communities (Knight, 2013; Zappavigna, 2018, 2019).  Here we are concerned with how selected features of FACIAL AFFECT and VOICE AFFECT couple with their ideational triggers in the negotiation of bonds in the service of affiliation. Figure 5.11 presents options and realisations in a system of BONDING adapted from Zappavigna (2018, 2019) with realisations for PARALINGUISTIC AFFECT.

Blogger Comments:

[1] To be clear, the claim here is simply that reciprocated evaluations bond interlocutors socially.

[2] To be clear, the claim here is that reciprocated emotional evaluations, expressed by the face and voice, bond people socially.

The problem here is that facial and vocal expressions of emotion are not systems of the interpersonal metafunction of language, but are systems of pre-metafunctional protolanguage, as demonstrated by all the members of other socio-semiotic species whose facial and vocal configurations express emotion.

Given this, the claim is that reciprocated emotional responses to the same environmental stimulus bond people socially. One example of this would be when two strangers, both afraid of being eaten by same approaching crocodile, are bonded socially through their reciprocated facial expressions.

Problems With The Notion Of "Coupling" An Evaluative Meaning With An Ideational Trigger

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 140):

From a systemic functional perspective the co-instantiation of an evaluative meaning with an ideational one constitutes a kind of ‘coupling’, one that can be applied to expressions of AFFECT with accompanying ideational triggers. 

 

Blogger Comments:

[1] To be clear, Halliday (2008: 179) characterises the APPRAISAL system of ATTITUDE as follows:

This is a grammatical system that is realised by a selection of lexical items.

That is, AFFECT is realised by lexical item, and the "coupling" is with some wording of its co-text, both of which function both interpersonally and ideationally, at least.

[2] To be clear, 'ideational trigger' is the textual/interpersonal/ideational wording that is "coupled" with the lexical item that expresses the attitude. Since a 'trigger' is a cause of a process, the term only applies in the case of impinging mental clauses (Adele pleases her) where the Phenomenon is the Agent. In emanating mental clauses (She likes Adele), the Phenomenon is not the Agent ('trigger') but the Range that 'delimits the boundaries of the sensing' (Halliday & Matthiessen 2014: 347).

Accepting The Biological Nature Of Vocal And Facial Expressions Of Emotion

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 139):

If we accept that natural (i.e. not performed) vocal and facial expressions of emotion are biological in nature (Darwin, 1872; Barlow, 2002), this would suggest resonance across the systems of FACIAL AFFECT and VOICE AFFECT (in the absence of intentional divergence such as in expressions of sarcasm).

Blogger Comments:

[1] To be clear, the biological nature of vocal and facial expressions of emotion, whether natural or performed, lies in the fact that the organs of organisms are their material basis. Moreover, the fact that the vocal and facial expressions of emotion are not restricted to humans (Darwin 1872) demonstrates that these semiotic systems do not require the prior evolution and development of language, and so are (personal) protolinguistic systems, rather than (interpersonal) AFFECT systems of language.

[2] For amusement, the following illustrates the divergence between the expressed content of protolanguage and the unexpressed content of language.

Problems With Irrealis vs Realis Affect

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 119, 136, 137):

An important distinction in the AFFECT system in language (Table 5.1) is between realis (an emotional response triggered by a present or past happening) and [ir]realis (an emotional response triggered by what might happen). Where the response is irrealis positive this is glossed as ‘desire’ and where it is negative as ‘fear’. However, in the VOICE AFFECT system [fear] is a feature (not simply a gloss) and its realisations are restricted to qualities of voice. Nonetheless the intersemiotic convergence of voiced [fear] with the language and action of the unfolding storyline in Coraline can support an interpretation of the voiced negative emotion as a response to what might happen, or in the case of (13) to whom the voices might belong. …

In contrast to voiced [fear], the intersemiotic convergence of voiced [anxiety] with the language and action of the unfolding storyline in Coraline can support an interpretation of the voiced emotion as a response to seeing the Ghost Children, that is, a realis happening.

Blogger Comments:

[1] To be clear, this confuses interpersonal meaning (AFFECT) with experiential meaning (cause, happening). The distinction here is between mental processes of emotion ('realis') and mental processes of desideration ('irrealis').

However, the exclusive association of desire and fear with irrealis is invalid, since both can be triggered by a present or past happening, as demonstrated by He desired her from the moment he saw her and She feared the non-venomous snake the moment she saw it.

[2] Importantly, here the intersemiotic convergence is of the content of paralanguage with the content of language. This is inconsistent with the authors' model of ideational paralanguage, where it is the expression of paralanguage that converges with the content of language.

[3] To be clear, here the authors are anxious to justify their categorisation of 'fear' as irrealis (desiderative), in contrast to 'anxiety', which they categorise as realis (emotive). Their anxiety, however, is unjustified, because 'fear' can be realis, as in She feared the Ghost Children, as well as irrealis, as in She feared that the Ghost Children might harm her.

Problems With The System Of The 'Emotion' Threat

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 135):

The sets of voice qualities which differentiate VOICE AFFECT features of [fear], [anxiety] and [anger] are shown in Figure 5.9.

 

Blogger Comments:

The system in Figure 5.9 models paralanguage as a bi-stratal semiotic system, and although this is consistent with the notion of a semiotic system, it is inconsistent with the preceding chapters in which paralanguage is misunderstood as an expression-only semiotic system. Where in previous chapters it was just paralinguistic expression that was semovergent with language, in this chapter it is both paralinguistic content and expression that is semovergent with language.

Further, in Figure 5.9, the authors again misrepresent 'threat' as an emotion, with its result (fear, anxiety) and reason (anger) as its subtypes.

Problems With The System Of The 'Emotion' Spirit

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 133):

VOICE AFFECT as [spirit:down] has opposing features of [misery] and [ennui]. The voice quality contours which realise these features are shown in Figure 5.8.

Blogger Comments:

The system in Figure 5.8 models paralanguage as a bi-stratal semiotic system, and although this is consistent with the notion of a semiotic system, it is inconsistent with the preceding chapters in which paralanguage is misunderstood as an expression-only semiotic system. Where in previous chapters it was just paralinguistic expression that was semovergent with language, in this chapter it is both paralinguistic content and expression that is semovergent with language.

Problems With The System Of Voice Affect

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 129, 131):

It is the particular ‘mixture’ of options [from the system of VOICE QUALITY] which realise one feature or another in the system of VOICE AFFECT (see Figure 5.7).

Blogger Comments:

The system in Figure 5.7 models paralanguage as a bi-stratal semiotic system, and although this is consistent with the notion of a semiotic system, it is inconsistent with the preceding chapters in which paralanguage is misunderstood as an expression-only semiotic system. Where in previous chapters it was just paralinguistic expression that was semovergent with language, in this chapter it is both paralinguistic content and expression that is semovergent with language.

Further, in Figure 5.7, the authors again misrepresent 'threat' as an emotion, with its result (fear, anxiety) and reason (anger) as its subtypes. Moreover, within the emotion of threat, it groups one result (anxiety) with a reason (anger) instead of with a result (fear).

Inconsistency In The Authors' Analysis

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 128):

Just as an expression of FACIAL AFFECT supports the identification of the trigger, so available ideational information supports the interpretation of FACIAL AFFECT. A sequence of triggers is interpreted as prompting the sequence of emotions in (8). 

We interpret Coraline’s expression of [fear] in image 1 of example (8) as triggered by the potential consequences of accumulated information sourced visually in the falling rock and auditorily in the cry of pain and the loud, angry ‘meow’.  

We interpret the expression of [surprise] in image 2 as triggered visually by Coraline’s first sight of the cat.  

The trigger for [anger] in image 3 is interpreted not as a response to seeing the cat but to an internal realisation that it was the cat who had instigated her fear.

Blogger Comments:

To be clear, the authors' interpretations here are inconsistent with their own previous account:

A pained cry is heard. Extremely alarmed by this, she runs as fast as possible, sensing something is pursuing her. Startled by a loud ‘meow’ from behind, she turns to look. Seeing that it is only a cat, her facial expression of [fear] swiftly changes to [surprise], but then to [anger], as in the three images in (8).

That is:

• hearing a cry of pain triggered her fear;
• hearing a loud meow triggered her surprise; and
• seeing a cat, rather than a threat, triggered her anger.

Misapplying A Confusion Of Ideational And Interpersonal Meaning To A Representation Of Protolanguage [2]

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 127-8):

A further example in (8) shows how information acquired from past events can trigger a response in FACIAL AFFECT. The instance involves Coraline’s first encounter with the Cat in the orientation stage of the film’s narrative. The episode begins with Coraline exploring the neighbourhood along a steep hillside path. A rock falls onto her path from on high. She calls out but gets no response, then throws the rock in the direction from which it fell. A pained cry is heard. Extremely alarmed by this, she runs as fast as possible, sensing something is pursuing her. Startled by a loud ‘meow’ from behind, she turns to look. Seeing that it is only a cat, her facial expression of [fear] swiftly changes to [surprise], but then to [anger], as in the three images in (8).

Blogger Comments:

[1] That is:

• a Phenomenon of auditory perception (pained cry) is the Agent (trigger) of the mental Process of emotion (alarm);
• a Phenomenon of auditory perception (a loud meow) is the Agent (trigger) of the mental Process of emotion (surprise); and
• a Phenomenon of visual perception (a cat) is the Agent (trigger) of the mental Process of emotion (anger).

Again this confuses ideational with interpersonal meaning, and misapplies the confusion to an epilinguistic representation of pre-metafunctional protolanguage on a clay puppet.

[2] To be clear, here the expression of emotion does not accompany speech, so it is not functioning as paralanguage, and is not semovergent.

Misapplying A Confusion Of Ideational And Interpersonal Meaning To A Representation Of Protolanguage [1]

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 127):

In (7), the trigger for facial expressions of emotion is apparently sourced internally. 

In the resolution stage of the narrative storyline in the film, Coraline meets the Cat, a good friend whom she has not seen since she threw him at the Other Mother in attempting her escape from the Other World.

In image 1 in (7) Coraline expresses both mild [surprise] and [spirit:up]. There is no immediately convergent speech, and the trigger is not interpretable at this point by the viewer. 

However, in image 2 more visual information is made available. The Cat is now revealed as standing outside Coraline’s bedroom window, and his presence retrospectively explains the trigger for her facial [surprise] and [spirit:up] in image 1. 

In image 2, convergent with her spoken language, Coraline’s expression of FACIAL AFFECT changes from [spirit:up] to [spirit:down]. Again there is no apparent trigger in the visually available information. The resonant spoken language I’m really sorry I threw you out at the Other Mother suggests that the trigger at this point is sourced internally through her reflection on past events. The broader co-text of the story supports this interpretation.

Blogger Comments:

[1] That is, a cognitive Phenomenon is the Agent (trigger) of the mental Process of emotion. This confuses ideational with interpersonal meaning, and misapplies the confusion to an epilinguistic representation of pre-metafunctional protolanguage on a clay puppet.

[2] To be clear, in SFL terms, this is an expression of the personal microfunction of protolanguage, epilinguistically represented on a clay puppet.

[3] To be clear, if the expression of emotion does not accompany speech, then it is not functioning as paralanguage, and is not semovergent.

[4] That is, a Phenomenon of visual perception is the Agent (trigger) of the mental Process of emotion. This again confuses ideational with interpersonal meaning, and misapplies the confusion to an epilinguistic representation of pre-metafunctional protolanguage on a clay puppet.

The Notion Of Ideational Triggers For Affect Reconsidered

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 126-7):

AFFECT in verbal and visual texts is always triggered by ideational phenomena. These can be entities or occurrences of any kind. Ideational triggers for expressions of FACIAL AFFECT in Coraline may be sourced via a diversity of perceptual channels that are interpreted as available to the character in particular instances. A taxonomy of types of perceptual channel is presented in Figure 5.5.

The triggering information may be sourced externally through an auditory perceptual channel (as sound or silence) or a visual, olfactory, gustatory or tactile one (Feng and O’Halloran, 2013). Alternatively, it can be sourced internally through reflection, memory or imagination. In interpreting the trigger for a particular facial expression of emotion in multimodal discourse such as that in Coraline more than one perceptual channel is likely to play a part.

Blogger Comments:

[1] To be clear, with the notion of ideational triggers for emotion, the authors have left the interpersonal domain of appraisal by emotional attitude and entered the ideational domain of cause and effect (reason and result). The ideational trigger, here, is the Phenomenon/Agent of an impinging mental Process of emotion (Music pleases me). This notion, however, ignores the distinction with the Phenomenon/Range of an emanating mental Process of emotion (I like music).

[2] To be clear, the taxonomy of types of perceptual channel corresponds to a taxonomy of types perceptual Phenomenon/Agent ('triggering information') of an impinging mental Process of emotion.

[3] To be clear, internal triggering information corresponds to a cognitive Phenomenon/Agent of an impinging mental Process of emotion.

A Problematic Analysis Of Facial Affect

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 125-6):

Expressions of relative FORCE in FACIAL AFFECT are additionally realised through the relative duration over which an expression is held. …

In example (6), Coraline, having sensed danger, tells her Other Parents that she wants to go to bed. Her intention is to escape from the Other world in her sleep. However, the Other Parents follow closely behind her, the Other Mother even offering to tuck her into bed. 

Coraline’s anxiety is not revealed in the spoken exchange with the Other Mother but rather in her expression of FACIAL AFFECT as [spirit:down] realised through eyebrows raised and drawn together and downcast eyes. The expression is extended in duration, sustained over the three tone groups of the exchange (marked as //…//…//…).

Blogger Comments:

As previously argued, from the perspective of SFL Theory, these graded epilinguistic images are of the personal microfunction of protolanguage depicted on clay puppets by animators using the emotion-face code devised by Ekman.

[1] To be clear, since the authors claim (p123-4) that expressions of surprise typically have the briefest duration, the claim here is that surprise is typically has weaker force than other emotions.

[2] To be clear, the Coraline character is here concealing her anxiety from the other characters, as the spoken language demonstrates, so as not to raise suspicion, so the interpretation of this facial configuration — in which the eyes are not downcast — expressing any anxiety at all, let alone stronger anxiety, would seem to be the opposite of what is true.

The Praat waveforms are irrelevant here, since they just represent the articulation of consonants and vowels.

The Depiction Of Muscle Tension On A Clay Puppet

Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 125):

Illustrated in (5) are two instances of FACIAL AFFECT as [spirit:up]. Image 2 is graded up in PARALINGUISTIC FORCE through increased muscle tension in the face as evident in the curled up corners of the mouth.

Blogger Comments:

As previously argued, from the perspective of SFL Theory, these graded epilinguistic images are of the personal microfunction of protolanguage depicted on clay puppets by animators using the emotion-face code devised by Ekman. What varies is the intensity of the emotional state depicted on the puppet.