Modelling Paralanguage Using Systemic Functional Semiotics
A Meticulous Review Of Ngo, Hood, Martin, Painter, Smith and Zappavigna (2022)
27 April 2024
26 April 2024
Motion Used To Support Direction
Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 33):
Motion can also be used to support direction in space or time. In Section 1.5.1 we illustrated two examples of hands sweeping right to left towards the past, concurring with the tone groups //2 bought / previously when I // (57) and // loved the / first time // (58). These contrast with left-to-right movement towards the future, concurrent with // hopefully next time I will //. This motion to the right is reinforced by a pointing gesture, which we discuss in Section 1.5.2.3 (as textual semovergence).
Blogger Comments:
This is recycled verbatim from Martin & Zappavigna (2019). Here are the comments from the review of Martin & Zappavigna (2019): Gestural Motion "Supporting" Direction In Space Or Time.
In examples (2) and (3) the vlogger makes a sweeping right-to-left gesture referencing past time;
If next time is interpreted as a circumstantial Adjunct, then, as a circumstance of Location, it signifies 'rest' not 'motion'. Halliday & Matthiessen (2014: 317):
However, Halliday & Matthiessen (2014: 612-3) list next time as an example of a conjunctive Adjunct (enhancement: spatiotemporal: complex). On this reading, the meaning of next time is textual in metafunction, rather than ideational.
In Martin (1992), however, cohesive conjunction in the grammar is misunderstood as a logical system of discourse semantics (now termed CONNEXION). That is, in Martin's terms, this gesture "concurs" with a logical relation between message parts in a message (here relabelled as figure and sequence, after Halliday & Matthiessen 1999). However, the authors failed to recognise it as an instance of Martin's CONNEXION.
24 April 2024
Motion On Its Own
Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 32-3):
Motion can also occur on its own, without a hand shape concurring with an entity. For example, the vlogger uses a circular hand motion (two rotations) concurrent with the tone group //1 tried washing it / out it’s //.
Blogger Comments:
This is recycled verbatim from Martin & Zappavigna (2019). Here are the comments from the review of Martin & Zappavigna (2019): Failing To Account For Body Language Meaning.
22 April 2024
Hand Shapes
Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 32):
As noted earlier, for this paralinguistic sequence hand shape and motion are combined. In other cases hand shapes occur on their own. In the following sequence our vlogger concentrates on the size of the snack she has given her children, without setting the bowl in motion:
(70) //3 then they had a / snack I(71) //4 gave them / each a / bowl - like a heaping / bowl(72) //3 full of / Chex Mix and an(73) //4 applesauce / squeeze and they //
Blogger Comments:
This is recycled almost verbatim from Martin & Zappavigna (2019). Here are the comments from the review of Martin & Zappavigna (2019): Gestures Realising Elements Rather Than Figures.
- then they had a snack
- I gave them each a bowl like a heaping bowl full of Chex Mix and applesauce squeeze
20 April 2024
Gesture Sequence
Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 31-2):
As with imagic sequences in film, animations, graphic novels, comics, cartoons and picture books, the gesture sequence does not make explicit the conjunctive relations between events (and so cannot support discourse semantic connexion). These relations have to be abduced (Bateman, 2007) from the sequence and concurring language. In the case of the sequence in (66)–(69), conjunctive relations of time and cause are not made explicit in language either; only the additive linker and is used. A defeasible reading of the sequence is offered in (66'')–(69'').(66'') // and so the dermatologist um took like this needle(temporal sequential)(67'') // and under each like bump(temporal overlapping)(68'') // and injected this like steroid(causal)(69'') // and like it all bubbled up //
Blogger Comments:
This is recycled almost verbatim from Martin & Zappavigna (2019). Here are the comments from the review of Martin & Zappavigna (2019): Abducing Defeasible Conjunctive Relations.
Abductive reasoning allows inferring a as an explanation of b. As a result of this inference, abduction allows the precondition a to be abduced from the consequence b. Deductive reasoning and abductive reasoning thus differ in the direction in which a rule like "a entails b" is used for inference. As such, abduction is formally equivalent to the logical fallacy of affirming the consequent (or Post hoc ergo propter hoc) because of multiple possible explanations for b.
In logic, defeasible reasoning is a kind of reasoning that is rationally compelling, though not deductively valid. … Defeasible reasoning is a particular kind of non-demonstrative reasoning, where the reasoning does not produce a full, complete, or final demonstration of a claim, i.e., where fallibility and corrigibility of a conclusion are acknowledged. In other words, defeasible reasoning produces a contingent statement or claim.
- and so the dermatologist um took like this needle
- and under each like bump (and) injected this like steroid
- and like it all bubbled up
18 April 2024
Occurrences
Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 30-1):
Turning from a static to a dynamic perspective, the language of this sequence makes explicit three occurrences (took, injected, bubbled). The paralanguage concurs with these and in addition uses six rapid piercing gestures to make explicit the events implied by the second tone group (67').
In each case the entity indicated by the hand shape is in motion, as the dermatologist picks the needle up, pierces the bumps, injects the steroid and the bump bubbles up.
Blogger Comments:
// and so the dermatologist um took like this needle
// and under each like bump
// and injected this like steroid
// and like it all bubbled up //
Turning from participant elements ("entities") to process elements ("occurrences"), the language of this sequence construes three processes (took, injected, bubbled).
16 April 2024
Commitment
Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 30):
In terms of commitment (i.e. the amount of meaning specified across semiotic modes; Martin, 2010; Painter et al., 2013), the ‘dermatologist’ and ‘steroid’ are committed in the language but not the paralanguage; but the ‘needle’ is more delicately committed in the paralanguage as a tiny pointed entity and then as a syringe. And the paralinguistic commitment of the ‘bump’ convergent with (69) in fact takes place two tone groups after it is committed verbally in (67).
Blogger Comments:
This is recycled verbatim from Martin & Zappavigna (2019). Here are the comments from the review of Martin & Zappavigna (2019): Martin's Notion Of Commitment.
Instantiation also opens up theoretical and descriptive space for considering commitment (Martin 2008, 2010), which refers to the amount of meaning instantiated as the text unfolds. This depends on the number of optional systems taken up and the degree of delicacy pursued in those that are, so that the more systems entered, and the more options chosen, the greater the semantic weight of a text (Hood 2008).
- assuming that handshape is the only bodily expression of ideational meaning here, and
- analysing at the level of element ("entity") instead of figure (while claiming the latter).
14 April 2024
Entity Concurrence
Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 29-30):
By way of illustration we now move to the fifth phase in the vlog, which concerns a visit to the vlogger’s dermatologist (for treatment of granuloma). The sequence of events we are interested in unfolds verbally in tone groups as follows (for the complete text of this phase of the vlog, see Appendix B5):
(66) //3 and so the / dermatologist um / took like this / needle and(67) //3 under / each like / bump and in-(68) //3 jected this like / steroid and it would like(69) //3 all / bubble up… //From the perspective of language, the verbiage in this sequence makes explicit four entities (dermatologist, needle, bump, steroid). The paralanguage uses hand shape to concur with two of these (needle and bump). The ‘needle’ is first rendered as a tiny pointed entity the vlogger holds between thumb and index finger and then with the hand shape used for holding a syringe; the ‘bump’ is not actually visualised until the fourth tone group, where it renders the shape of the steroid bubbling up (Table 1.7). As we can see, the meanings construed in language and paralanguage can either correspond with or complement one another.
Blogger Comments:
This is recycled almost verbatim from Martin & Zappavigna (2019). Here are the comments from the review of Martin & Zappavigna (2019): Misinterpreting The Data.
[3] To be clear, the claim here is that the meaning realised by the handshape "concurs" with the meaning realised by the wordings needle and bump. However, neither of the two handshapes realises needle, since neither handshape depicts a sharply pointed metal stick; see further in [5] below.
- Firstly, the paralanguage gloss confuses content (holding needle, holding syringe) with expression (cupped hands).
- Secondly, the glosses correlate elements ("entities") of language (needle, bump) with figures for paralanguage (holding needle, holding syringe).
- Thirdly, the glosses of the paralanguage content are not motivated by the data. On the basis of both the gestures and the accompanying language, the glosses are more consistently construed along the lines of taking needle and injecting steroid; moreover, the word syringe was not used by the speaker.
[5] To be clear, this handshape does not depict a needle. Instead, the handshape realises the same meaning as the wording took this needle in the figure so the dermatologist took this needle; that is, it realises the nucleus of the figure, Process and Medium. Halliday & Matthiessen (1999: 156):
Semantically, the nucleus construes the centre of gravity of a figure, the focal point around which the system of figures is organised. When we describe the Medium as "actualising" the Process, we are really saying that the unfolding is constituted by the fusion of the two together — there can be no Process without an element through which this process is translated from the virtual to the actual.
[6] To be clear, this handshape does not depict a needle. Instead, the handshape realises the same meaning as the wording injected steroid in the figure and under each bump injected this steroid; that is, this again realises the the nucleus of the figure. Again the (ellipsed) Agent of this figure, the dermatologist, is represented by the speaker herself.
Note that, on the authors' interpretation, the meaning of the second tone group does not "concur" with the meaning of the co-occurring body language.
[8] To be clear, the handshape depicts the shape of a granuloma as it rises after the injection of the steroid.
- the meanings construed in language and paralanguage neither correspond nor complement one another.
12 April 2024
Representation (Ideational Semovergent Paralanguage)
Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 29):
Representation (ideational semovergent paralanguage)From an ideational perspective we need to take into account how spoken language combines entities, occurrences and qualities as figures (ideation). Semovergent paralanguage supports these resources with hand shapes, which potentially concur with entities, and hand/arm motion, which potentially concurs with occurrences (Hood and Hao, 2021); the hand/arm motion is optionally directed, potentially concurring with spatiotemporal direction (i.e. to/from here and there in space, to/from now and then in time). We say ‘potentially concurring’ because ideational paralanguage can be used on its own, without accompanying spoken language; see the discussion of mime in Chapter 7.
Blogger Comments:
This is recycled almost verbatim from Martin & Zappavigna (2019). Here are the comments from the review of Martin & Zappavigna (2019): Ideational Semovergent Paralanguage.
[2] As previously explained, and argued here, Martin's ideational discourse semantic systems of IDEATION and CONNEXION are neither ideational nor semantic, since they are misunderstood rebrandings of Halliday & Hasan's (1976) lexical cohesion and cohesive conjunction, which are lexicogrammatical systems of the textual metafunction.
[3] To be clear, this is a matter of language, regardless of whether it is spoken, written or signed.
[4] To be clear, in the discourse semantic system of IDEATION (Martin 1992: 314-9; Martin & Rose 2007: 96), 'entity' refers only to a subtype of Range.
[5] To be clear, in the discourse semantic system of IDEATION (Martin 1992: 314-9; Martin & Rose 2007: 90ff), these are termed 'processes', not 'occurrences'.
[8] The word 'support' here is potentially misleading, since epilinguistic body language makes meaning in its own right.
[9] Here the authors propose 1-to-1 relationships between the expression of body language and the content of language — instead of the content of body language. This confusion leads the authors to the false conclusion at the end of the [2022] paper that body language is just another expression mode of language itself.
Even so, the validity of proposed 1-to-1 relationships will be examined in upcoming posts.
[11] See the upcoming critique of the authors' discussion of 'mime'.
10 April 2024
Semovergent Paralanguage Converging With Discourse Semantics
Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 28-9, 233):
Semovergent paralanguage is convergent with the lexicogrammar and discourse semantics of spoken language (its content plane). We adopt a discourse semantic perspective on these meaning-making resources here (Martin and Rose, [2003] 2007). Drawing on terms from Painter et al. (2013) we can position ideational paralanguage as concurring with IDEATION systems (but not CONNEXION, as will be discussed later), interpersonal paralanguage as resonating with APPRAISAL systems (but not NEGOTIATION, as will be discussed later) and textual body language as coordinating information flow alongside IDENTIFICATION and PERIODICITY²⁸ systems. These convergences are outlined in Table 1.6.
²⁸ Semovergent synchronicity is concerned with the syncing of paralanguage with periodic structure composed above and beyond prosodic phonology.
Blogger Comments:
[1] This is recycled verbatim from Martin & Zappavigna (2019). Here are the comments from the review of Martin & Zappavigna (2019): Seriously Misunderstanding Cléirigh's Epilinguistic Body Language.
[3] It will be seen that the "convergences" that the authors propose are between the expression plane of semovergent paralanguage [epilinguistic body language] and the content plane of language. Here the authors reveal their serious misunderstanding of paralanguage as just an expression plane system. See also from the review of Martin & Zappavigna (2019):
08 April 2024
Prosodic Phonology Demarcating Somatic From Semiotic Behaviour
Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 26-8):
The contribution of sonovergent paralanguage to the vlog is interrupted in tone group 19 of Appendix B3, suspended for tone groups 20–24 and resumes for tone group 25 (examples (61)–(65)) – to allow for a somatic phase during which the vlogger uses her left hand to scratch her right arm. This phase unfolds as follows:(61) //3 lighter than it / was a few / days ago(62) //1 ^ but / yeah it’s(63) //1 such a / bummer and then I(64) //2 went to / Target(65) //3 ^ like / two days / later and there was a //The vlogger stops looking at her followers and begins scratching in the final foot of (61'). The scratching and absence of gaze continues for two tone groups ((62')–(63')). Gaze resumes in the final foot of (64'). And the vlogger then resumes gesturing in (65').
It is interesting to note that the vlogger does not scratch in sync with the RHYTHM, TONICITY and TONALITY of the text; the scratching lasts for two and a half tone groups and does not match the timing of salient and tonic syllables. But the paralanguage remains in sync, stopping precisely at the tonic syllable of (61') (/ days ago //), resuming with a smile precisely at the tonic syllable of (64') (/ Target //) and resuming with gesture precisely at the beginning of (65'). This indicates that synchronicity with prosodic phonology can function as a demarcating criterion for distinguishing somatic from semiotic behaviour.
Blogger Comments:
This is recycled verbatim from Martin & Zappavigna (2019). Here are the comments from the review of Martin & Zappavigna (2019): Interpreting Averted Gaze As Non-Semiotic.
06 April 2024
Super-Salience
Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 25-6):
Salient syllables other than the tonic syllable can be given additional prominence (super-salience) through various means. In (14') the vlogger’s pitch on the first tone group is unusually high, and contrasts with the descending lower pitch of the following tone group (a sing/song effect). We use upward and downward arrows, ‘↑’ and ‘↓’, to signal pronounced salience of this kind.
(14''')
//3 ↑ hopefully next / time I will//1 get my / ↓hair colour / back //
And the vlogger’s eyebrows move up in tune and in sync with the higher pitch on / hopefully /, before lowering again by the end of the following tone group.
The same sing/song effect follows on and culminates this section of the vlog, with a high pitch on the tonic syllable / now // contrasting with the low pitch on / do //. The vlogger’s eyebrows once again move up and down in tune and in sync with the contrasting pitch salience (this time on contrasting tonic syllables).//3 [handclap] / um /but for / ↑now
//3 This will / ↓do //
These rhythmic in-tune gestures reinforce the attitudinal import of the RHYTHM and TONICITY.
Blogger Comments:
This is recycled verbatim from Martin & Zappavigna (2019). Here are the comments from the review of Martin & Zappavigna (2019): Misunderstanding Rhythm And Tonicity.
[2] Here the authors confuse the textual function of phonological prominence with the interpersonal function of pitch movement.
[3] This misunderstands the data. The "sing/song" effect is a result of the tone sequence 3^13; see [4].
[4] This analysis misrepresents the data. What the speaker actually intones can be phonologically represented as:
//3 hopefully / next time I willRegarding the first of these, contrary to the authors' claims, even on their own analysis, time is not a salient syllable, and listening to the data reveals that the "unusually high pitch" extends throughout the tone group, rather than just for the word time.
//13 get my / hair colour / back //
With regard to the second tone group, contrary to the authors' claims, hair is a tonic syllable, not a non-tonic salient syllable. This is because hair is the first tonic in a compound tone group.
[5] This claim is manifestly untrue, since if the eyebrows stay raised for two tone groups, it is neither "in sync" with one tone group (TONALITY) nor "in tune" with the major pitch movements (TONE) of the two tone groups: level/low rise – fall – level/low rise.
This is a case of the authors misrepresenting the data in order to make them fit their misunderstandings of Cléirigh's model.
[6] This "same sing/song" effect is this time simply a result of the tone sequence 3^1-. What the speaker actually intones can be phonologically represented as:
//3 um /but for / now //1- this will / do //That is, the handclap co-occurs with the tonic of the previous tone group, back, and the tone of the second tone group here is a narrow fall (1-), not a level/low-rise (3).
[7] Here the authors make a brave stab at guessing what these "rhythmic in-tune" gestures might mean. But the truth lies elsewhere.
Firstly, this is potentially misleading. On Cléirigh's original model, it is only the rhythmic dimension or aspect of a gesture that functions textually like the rhythm of speech, and it is only the rise/fall dimension or aspect of a gesture that functions interpersonally like the pitch movement of speech. Other dimensions or aspects of a gesture may serve additional functions.
Secondly, the notion of 'import' here derives from the work of Martinec (and possibly van Leeuwen), but the authors present it as their own.
Thirdly, the notion of attitudinal import is inappropriate here for two reasons:
- attitude is concerned with interpersonal meaning whereas rhythm and tonicity are concerned with textual meaning, and
- there are no instances of attitude in the instances of text under discussion.
Fourthly, as previously explained, the tonic marks the focus of New information, and the non-tonic salient syllables identify the potential foci of New information that the speaker chose not to instantiate.
04 April 2024
The Phonological System Of Rhythm
Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 25):
The phonological system of RHYTHM is realised in English through the timing of the salient syllables beginning each foot (relatively equal timing between salient syllables in a stress-timed language like English). In (60), the vlogger beats with her hands in time with the salient syllables of the feet / not /, / find the / and / hair dye that I /. The last of these beats in fact syncs with the tonic syllable hair.
Blogger Comments:
This is recycled verbatim from Martin & Zappavigna (2019). Here are the comments from the review of Martin & Zappavigna (2019): Not Recognising The Function Of Gestural Rhythm.
02 April 2024
The Phonological System Of Tonicity
Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 24-5):
The phonological system of TONICITY highlights a peak of informational prominence by positioning the major pitch movement of a tone group (its tone) on one or another of its salient syllables (its culminative salient syllable in the unmarked case). In example (59) the vlogger claps on the syllable realising the tone group’s major pitch movement – hair.
Blogger Comments:
This is recycled verbatim from Martin & Zappavigna (2019). Here are the comments from the review of Martin & Zappavigna (2019): Misrepresenting Tonicity.
31 March 2024
The Phonological System Of Tonality
Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 24):
The phonological system of TONALITY organises spoken language into waves of information called tone groups, with one salient syllable carrying this tone movement. Gestures may be coextensive with this periodic unit. In examples (57) and (58) the vlogger makes a sweeping right-to-left gesture referencing past time; the gestures unfold in sync with the temporal extent of the tone group.
Blogger Comments:
This is recycled verbatim from Martin & Zappavigna (2019). Here are the comments from the review of Martin & Zappavigna (2019): Blurring The Distinction Between Linguistic And Epilinguistic Body Language.
29 March 2024
The Phonological System Of Tone
Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 23):
The phonological system of TONE is realised through pitch movement. In example (56) the vlogger’s eyebrows move up in tune with the rising tone (tone 2) on the syllable prev.
Blogger Comments:
This is recycled verbatim from Martin & Zappavigna (2019). Here are the comments from the review of Martin & Zappavigna (2019): Ignoring Content And Getting The Phonology Wrong.
Since this is linguistic body language, the interpersonal meaning that the pitch and eyebrow movement both realise depends on the grammar, the choice of MOOD, which in this instance is declarative:
//4 but /^ I could / not / find the /hair dye that I //2 bought / previously when I //3 dyed my / hair which I //3 loved I //3 loved the/ first timeOn Halliday's model, the combination of tone 2 with declarative mood realises a protesting or contradicting statement (Halliday 1994: 305), as in
//2 that / can't be / true // ('so don't try and tell me!')
//2 ^ it / didn't / hurt you ('so don't make a fuss')
The basic meaning of ('low-rising') tone 3, on the other hand, is that the information being realised is dependent on something else (op. cit.: 303). In this monologic instance, it could be take to mean 'hold on, there's more to come'. (In dialogue, tone 3 can function as a as a turn-keeping device: 'I'm not finished yet, so don't interrupt!'.) On this basis, the instance in question is better analysed as an emphatic variant of tone 3, just like the three tones that follow (and also the preceding "tone 4").
- the tonic syllable is pre, not prev, and
- a 'rise-fall' eyebrow movement corresponds to tone 5, not tone 2.
27 March 2024
Paralanguage Converging With Sound: Sonovergent Paralanguage
Ngo, Hood, Martin, Painter, Smith & Zappavigna (2022: 23, 233):
Sonovergent paralanguage converges with the prosodic phonology of spoken language (Halliday, 1967, 1970a; Halliday and Greaves, 2008; Smith and Greaves, 2015). From an interpersonal perspective, it resonates with tone and involves a body part (e.g. eyebrows or arms) moving up and down in tune with pitch movement in a tone group (TONE and marked salience). From a textual perspective, it involves a body part²³ (e.g. hands, head) moving in sync with the periodicity of speech – which might involve beats aligned with a salient syllable of a foot (which might also be the tonic syllable of a tone group) or a gesture coextensive with a tone group (i.e. in sync with TONALITY, TONICITY or RHYTHM systems). An outline of this sonovergent paralanguage is presented in Table 1.5. …
²³ For wavelengths longer than a tone group, whole body motion is involved.
Blogger Comments:
[1] To be clear, "sonovergent paralanguage" realises the same lexicogrammatical distinctions as prosodic phonology, which is why Cléirigh termed it linguistic body language, and why it is invalid to model it as paralanguage.
Linguistic Body Language (Body Language)
This is body language that only occurs during speech. Its kinology involves visible body movements that are in sync with the rhythm or in tune with the (defining) pitch movement of spoken language. In doing so, the function of such movements is precisely that of the prosodic phonology: rhythm and intonation.
As linguistic, ‘prosodic’ body language is thus:
v tri-stratal: its kinology realises the lexicogrammar of (adult) language, and
v metafunctional (textual and interpersonal) in terms of Halliday’s modes of meaning.
lexicogrammar
prosodic expression
phonology
kinetic
lexical salience°
rhythm
gesture (hand, head) in sync with the speech rhythm
textual
focus of new information
tonicity
gesture (hand, head) in sync with the tonic placement
information distribution
tonality
gesture (hand, head) co-extensive with tone group
interpersonal
key
tone
gesture (eyebrow*, hand) in tune with the tone choice
* also: rolling of the eyes for tone 5.
°Halliday (1985: 60):
The function of rhythm in discourse is to highlight content words (lexical items).
[3] To be clear, this is a bare assertion, unsupported by evidence: the logical fallacy known as ipse dixit.
∞