The tone atlas of perceptual discriminability and perceptual distance: Four tone languages and five language groups

Brain & Language 229 (2022) 105106 Available online 4 April 2022 0093-934X/© 2022 The Authors. Published by Elsevier Inc. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/). The tone atlas of perceptual discriminability and perceptual distance: Four tone languages and five language groups Liquan Liu a , b , c , d , * , Regine Lai e , Leher Singh f , Marina Kalashnikova b , g , h , Patrick C.M. Wong e , i , Benjawan Kasisopa b , Ao Chen j , Chutamanee Onsuwan k , Denis Burnham b , * a School of Psychology, Western Sydney University, Australia b The MARCS Institute for Brain, Behaviour and Development, Western Sydney University, Australia c Center for Multilingualism in Society Across the Lifespan, University of Oslo, Norway d Centre of Excellence for the Dynamics of Language, Australian Research Council, Australia e Department of Linguistics and Modern Languages, The Chinese University of Hong Kong, Hong Kong f Department of Psychology, National University of Singapore, Singapore g Basque Center on Cognition, Brain and Language, Spain h Ikerbasque, Basque Foundation for Science, Spain i Brain and Mind Institute, The Chinese University of Hong Kong, Hong Kong j School of Communication Sciences, Beijing Language and Culture University, China k Department of Linguistics, Faculty of Liberal Arts and Center of Excellence in Intelligent Informatics, Speech and Language Technology, and Service Innovation (CILS), Thammasat University, Thailand ARTICLE INFO Keywords: Tone Cross-linguistic perception Tone system Contrast type Perceptual asymmetry Cue-weighting Multi-dimensional analysis ABSTRACT Some prior investigations suggest that tone perception is flexible, reasonably independent of native phonology, whereas others suggest it is constrained by native phonology. We address this issue in a systematic and comprehensive investigation of adult tone perception. Sampling from diverse tone and non-tone speaking communities, we tested discrimination of the three major tone systems (Cantonese, Thai, Mandarin) that dominate the tone perception literature, in relation to native language and language experience as well as stimulus variation (tone properties, presentation order, pitch cues) using linear mixed effect modelling and multidimensional scaling. There was an overall discrimination advantage for tone language speakers and for native tones. However, languageand tone-specific effects, and presentation order effects also emerged. Thus, over and above native phonology, stimulus variation exerts a powerful influence on tone discrimination. This study provides a tone atlas, a reference guide to inform empirical studies of tone sensitivity, both retrospectively and prospectively. 1. Introduction Around 60–70% of the world’s languages are tone (or pitch-accent) languages – they use pitch to differentiate word meanings (Yip, 2002). Over half of the world’s population speaks a tone language (Fromkin, 1978). Despite this ubiquity, the determinants of tone perception are not well understood. Some suggest tone perception is flexible, reasonably independent of native phonology, whereas others suggest it is constrained by native phonology or perceptual asymmetries. This study aims to investigate the relative roles of the perceiver, the stimulus and the context in lexical tone perception. To this end, the study involves a comprehensive investigation of how different tones in different tone languages are perceived by listeners from different language backgrounds in different task contexts. In terms of the classification of lexical tones, unlike other phonetic units (consonants, vowels), which are often defined by articulatory gestures, tones are defined mostly by their pitch characteristics (Burnham, Attina, Xu, & Kasisopa, 2011). Fundamental frequency (f 0 ), the acoustic basis of pitch perception, is a widely recognized index of tone differences (Barry & Blamey, 2004), even though tones are also differentiated, to a lesser extent, on other dimensions, such as duration, amplitude, and voice register (Feng, Wu, & Nissenbaum, 2020). Given the predominance of pitch in the perception of tone, Chao (1930) introduced a system to describe individual tones, whereby each tone is * Corresponding authors. E-mail addresses: [email protected] (L. Liu), [email protected] (D. Burnham). Contents lists available at ScienceDirect Brain and Language journal homepage: www.elsevier.com/locate/b&l https://doi.org/10.1016/j.bandl.2022.105106 Received 15 March 2021; Received in revised form 2 March 2022; Accepted 8 March 2022 Brain and Language 229 (2022) 105106 2 described on a normalized pitch scale from 1 (low) to 5 (high), one value at the start, another at the end, and a third medial value when there is an inflexion point within the tone. As this index is normalized, it is independent of the language and speaker and is used here to describe and compare tones within and between languages. The core issue in studying perceptual discriminability and perceptual distance lies in the understanding of the determinants of tone discrimination. Listeners’ tone perception has been shown to be influenced by multiple factors. Here, we discuss the effects of the perceiver (tone language experiences); the stimulus (including tone systems, contrast type) and the stimulus itself (tone properties and cues); and the task context (order of stimulus presentation) on tone perception. Like the perception of other phonetic units, tone perception is shaped by linguistic experience from the beginning of life (Fikkert, Liu, & Ota, 2020; Werker, 2018). Unlike consonants and vowels for which a clear developmental trajectory has been reported involving perceptual narrowing, i.e., from flexible (cross-language, phonetic) to constrained (language-specific, phonological) perception, the picture is mixed for tones. Some studies have demonstrated perceptual narrowing, indexed by decreased tonal sensitivity over age in non-native tone learners (Mattock & Burnham, 2006; Mattock, Molnar, Polka, & Burnham, 2008; Quam & Swingley, 2010). Others have reported facilitation, an agebased increased sensitivity by both tone and non-tone language learning infants (Chen & Kager, 2016; Chen, Stevens, & Kager, 2017; Ramachers, Brouwer, & Fikkert, 2018; Singh et al. 2018; Tsao, 2017). Yet other studies report results that attest to the flexibility of tone perception – a renewed sensitivity to tone by non-tone language learning infants in their second year (G¨ otz, Yeung, Krasotkina, Schwarzer, & H¨ ohle, 2018; Liu & Kager, 2014, 2017). Burnham and Singh (2018) point out that the mixed results are the product of studies using different tone pairs and different tone languages and could well be due to the psychophysics of tone salience. Accordingly, more acoustically salient tones may be more resilient to listeners across language backgrounds in the course of perceptual narrowing. However, the measure of inherent tonal salience has been difficult and is an issue of controversy, exacerbated by the absence of tone perception metrics solidly grounded on articulatory gestures. The effect of language background is evident in studies of adult listeners’ tone perception, in which tone language speaking adults show distinct and enhanced tone discrimination compared with their nontone language-speaking peers (Burnham & Francis, 1997; Kaan, Wayland, Bao, & Barkley, 2007; Malins & Joanisse, 2012; also see Maggu, Zong, Law, & Wong, 2018 and Tong, Lee, Lee, & Burnham, 2015 for the effect of monolingual versus bilingual tone and non-tone languages). In a study measuring Thai tone perception by speakers of Thai, Cantonese, Mandarin (tone languages), Swedish (pitch-accent) and English, better perception was evident in native speakers of a tone/pitchaccent language over native non-tone (English) speakers (Burnham et al., 2015). Direct comparisons of tone language speakers’ native versus nonnative tone perception are rare, but indirect evidence suggests differences in the perception of native and non-native tones. As early as 4 months after birth, Cantoneseand Mandarin-learning infants display language-specific perception of Cantonese tones (Yeung, Chen, & Werker, 2013). Such an effect also appears to carry over into adulthood: While Thai speakers have no difficulty identifying Thai tones, Mandarin and Cantonese speakers’ Thai tone perception appears to be modulated by their own tonal inventory (Reid et al., 2015). It has been argued that listeners perform better when non-native tones can be assimilated to their native tone categories (Chen, Best, & Antoniou, 2020), thus adhering to assimilation accounts of native versus non-native speech perception (e.g., Perceptual Assimilation Model, Best, 1995). These combined results strongly suggest, but do not directly test the hypothesis, that tone language speakers’ perception of native tones should be superior to their perception versus non-native tones In this paper, we will directly test the hypothesis that tone language speakers’ perception of native tones will be better than their perception of non-native tones. Tone language speakers’ processing of tones involves not only the bilateral frontoparietal brain regions used for acoustic analysis and abstraction (Gandour, Dzemidzic, et al., 2003; Gandour, Wong, et al., 2003; Gandour et al., 2004; Li et al., 2010), but also frontotemporal regions used for processing phonological and semantic information (Kwok, Dan, Yakpo, Matthews, & Tan, 2016; Kwok et al., 2017; Liang & Du, 2018; Wong et al., 2004). In addition, tone processing in listeners from tone language backgrounds becomes more left-lateralised when language comprehension tasks evoke more semantic processing (Gandour, Dzemidzic, et al., 2003; Gandour, Wong, et al., 2003; Gandour, Wong, & Hutchins, 1998; Gandour et al., 2004). Moreover, Chen, Peter, et al. (2018) found that while tone (Mandarin Chinese) and non-tone (Dutch) language adults show similar right-lateralised mismatch negativity (MMN) for 3-note musical melodies based on Mandarin tones, for the lexical tones on which the f 0 of these 3-note melodies were based, Chinese adults showed a later MMN peak to lexical tone oddballs than did non-tone Dutch adults. These results imply that, consistent with findings that tone language speakers process tones categorically (Feng, Gan, Wang, Wong, & Chandrasekaran, 2018; Gandour & Krishnan, 2016; Peng et al., 2010; Xi, Zhang, Shu, Zhang, & Li, 2010), a larger f 0 difference is necessary for Chinese than for Dutch adults to detect the lexical tone change. Moreover, Chen and colleagues (Chen, et al., 2018; Chen, Liu, & Kager, 2016; Liu, Chen, & Kager, 2020) found significant correlations between lexical tone and music pitch for non-tone, but not tone language speakers. It is known that adult-like right-hemispheric lateralisation to piano tones develops early at 2 months after birth (He, Hotson, & Trainor, 2007), and therefore, given this and the above findings, it appears that at the neurophysiological level non-tone language adults perceive lexical tones in a similar fashion as they do nonlexical (musical) pitch variations, whereas adult tone language speakers shift the manner of processing of lexical (but not musical) tones sometime after early infancy, and this facilitates the categorisation of f 0 modulations that signal phonological distinctions. At the behavioural level, both the absence and presence of tonerelevant experience affects non-tone language speakers’ lexical tone perception. On the one hand, the absence of tone-relevant experience limits tone discrimination (Burnham & Mattock, 2007). Burnham et al. (2015) showed that lexical tones presented in a speech context were less well discriminated by non-tone (English) language listeners than by tone or pitch-accent listeners, but once the very same f 0 pitch contours were presented as in a non-linguistic context, as hums or violin glides, differences in the perceptual performance between toneand non-tone language listeners were diminished. On the other hand, the presence of tone-relevant experience shows that tone discrimination is flexible; adult non-tone language listeners’ tone perception is improved by both tone training (Antoniou & Wong, 2015; 2016; Chandrasekaran, Sampath, & Wong, 2010; Francis, Ciocca, Ma, & Fenn, 2008; Ingvalson, Barr, & Wong, 2013) and musical training (Burnham, Brooker, & Reid, 2015; Cooper & Wang, 2012; Wong & Perrachione, 2007). Thus, while there is developmental divergence in lexical tone perception as a function of language environment, experience with a non-tone versus a tone language does not result in complete tone insensitivity in infants. In adults, non-tone language listeners’ tone discrimination is a function of the acoustic continuum of f 0 , and can be improved as a function of tonerelevant training and experience. With respect to the tone language system, it is unclear whether certain systems are easier to perceive than others. From a phonological density perspective, it is reasonable to predict that tone systems with less density (i.e., fewer tones) would be easier to perceive. There is an additional factor that is related to the tone language system: the type and mix of tone types in a tone language. Lexical tones have been classed as static or dynamic (Abramson, 1978). Static tones (S) are those with a relatively level contour (e.g., Cantonese 55) or a mild slope (e.g., Thai 45), differing from one another mainly in pitch height; dynamic tones (D) involve distinct pitch contours (changes in pitch height over time), L. Liu et al. Brain and Language 229 (2022) 105106 3 such as dynamic rising, (Dr, e.g., Cantonese 25), dynamic falling (Df, e. g., Mandarin 51), or more complex contours (e.g., rise-fall, Thai 241). In terms of contrast type, there is evidence from studies examining multiple tone contrasts that listeners’ tone discrimination is related to the type of tone contrast they hear. For instance, in Huang and Johnson (2010), discrimination accuracy of the Mandarin rising-falling 35–51 tone contrast was the best, suggesting that dynamic rising versus dynamic falling tones may be the easiest to discriminate. Similarly, in a comparison across tone contrast types, Burnham and colleagues found that Dr-Df tone pairs are more easily discriminated than S-S or S-D tone pairs (Burnham, Kirkwood, Luksaneeyanawin, & Pansottee, 1992; Burnham, Kasisopa, Reid, Luksaneeyanawin, Lacerda, Attina, Xu Rattanasone, Schwarz, & Webster, 2015). Moreover, it appears that the static versus dynamic distinction has psychological reality, for it has been reported that listeners’ native static tone experience facilitates their perception of static tones in another tone language (Chiao, Kabak, & Braun, 2011; Qin & Mok, 2011). More comprehensive empirical studies investigating more contrast types across languages are needed. Another factor that can influence perception is the order in which stimuli are presented. “Perceptual asymmetry” occurs when the order of presentation leads to asymmetrical perceptual patterns. This phenomenon has been reported for segmental features, especially vowels (Polka, Ruan, & Masapollo, 2018; Zhao, Masapollo, Polka, M´ enard, & Kuhl, 2019). With respect to lexical tones, Chen, Liu and Kager (2015) have shown that Mandarin tone sandhi (e.g., articulation of dipping-dipping 214–214 tone sequences as rising-dipping, 35–214) may rest upon a perceptual effect – directionally-specific acoustic masking. In line with this, it has been found that Mandarin rising-to-dipping, 35–214 and rising-to-falling, 35–51 tone pairs elicit larger MMN responses than the same tones in the opposite order (214–35 and 51–35), among native but not non-native Mandarin speakers, suggesting a perceptual constraint related to stimulus order on native listeners’ phonological representations (Politzer-Ahles, Schluter, Wu, & Almeida, 2016). Similarly, Wayland and Chen (2018) found changes from Mandarin flat (55) to other tones quite challenging, and from Mandarin falling (51) to other tones easier than the reverse order (other tones to 51) for both tone (Mandarin) and non-tone (English) speakers. However, a follow-up study reported the opposite pattern for Mandarin flat tones, and a change from Mandarin dipping to other tones is more difficult than the reverse direction (Wayland, Chen, Zhou, & Hong, 2019). Moreover, processing load (Liu, Ong, Tuninetti, & Escudero, 2018) and acoustic cues such as spectral dynamicity (Masapollo, Zhao, Franklin, & Morgan, 2019) or breathiness (Yang & Sundara, 2019) may all play a role in whether perceptual asymmetry occurs in tone discrimination. Theoretical models have been proposed to account for the constraints imposed on tone perception by the asymmetry of the order effects, including the under-specification hypothesis (e.g., Politzer-Ahles et al., 2016), prototypicality theory (Best, 1994), natural referent vowel framework (Polka & Bohn, 2003) and psychophysics of tone salience (Burnham & Singh, 2018). Further study is required to investigate the relative merit of these theories and potential constraints through empirical research. Finally, with respect to tone properties, pitch height and pitch direction have consistently been argued to be the two dominant cues in tone perception (Gandour, 1983; Gandour & Harshman, 1978; Lin, 1987; Vance, 1977) and recognition (Howie, 1976; Xu, 1997). While this is the case for both tone and non-tone language listeners, functional magnetic resonance imaging (fMRI) (Feng et al., 2018) and multidimensional scaling (MDS) studies (Chandrasekaran et al., 2010; Francis et al., 2008) have shown that tone language speakers attend more to directional cues and non-tone language speakers more to height cues, indicating a native tonological/phonological constraint (Chandrasekaran et al., 2010; Francis et al., 2008). Here, in order to extract general principles and specific drivers of tone perception and to investigate flexibility and constraint in tone processing, listeners from four different tone language backgrounds (Chinese Mandarin, Singaporean Mandarin, Hong Kong Cantonese, and Bangkok Thai) and one non-tone language (Australian English) were tested on tone discrimination in four different tone systems (Chinese Mandarin, Singaporean Mandarin, Cantonese, and Thai), all of which have both static and dynamic tones but differ in the number of tones. We expected superior discrimination of lexical tones by toneover non-tone language speakers and, among tone language speakers, of native over non-native tones. We also expected acoustic properties of tone would modulate listeners’ perceptual ability for tones across language systems and tone types (Choi, Tong, & Singh, 2017). As mixed findings have been reported regarding perceptual asymmetry, we keep the prediction open for whether and in what direction tonal asymmetries may occur. Finally, we focused on the two primary dimensions of tone perception, pitch height and pitch direction, exploring the role of 23 acoustic measurements/cues in these two dimensions for tone perception (see Appendix III) and predicted that listeners would use the cues and/or pitch-related knowledge that are relevant in their native language to non-native tone perception. To increase the validity and generalisability of the study, two discrimination tasks, AX and AXB, were used in two separate groups of participants. The results for the AX task are presented in the body of the paper, with the methods of the AXB task reported in Supplementary material along with any AXB results that showed meaningful divergent patterns from the AX task results. 2. Methods 2.1. Participants One hundred and twenty adults from five language backgrounds participated in an AX discrimination task (N =24 per language background): Cantonese speakers (M =21.00 years, SD =1.64) tested at the Chinese University of Hong Kong, Thai speakers (M =27.33 years, SD = 5.41) tested at the Thammasat University, Mandarin Chinese speakers (M =21.25 years, SD =2.44) tested at the Beijing Language and Culture University, Singaporean Mandarin speakers (M =21.70 years, SD = 2.18) tested at the National University of Singapore, and Australian English speakers (M =28.38 years, SD =10.43) tested at the Western Sydney University (see Appendix I for details including a separate group of 120 participants for AXB). Tone language speaker groups included those who knew other languages including tone/pitch-accent languages, but all used their native tone language in their daily lives. Australian English speakers had no prior tone language experience. 2.2. Stimuli All tones from the four tone systems, Cantonese (6 tones, 15 pairwise contrasts), Thai (5 tones, 10 contrasts), Chinese Mandarin (4 tones, 6 contrasts) and Singaporean Mandarin (4 tones, 6 contrasts), were used as stimuli, resulting in 37 within-language tone contrasts (Appendix II). Tones were categorized into three types: static (S) tones, dynamic rising (Dr) tones, and dynamic falling (Df) tones, resulting in five contrast types: S-S, S-Df, S-Dr, Df-Dr, Dr-Dr. To increase generalisability, each tone was presented on six different CV syllables (two initials (/p, pʰ/) ×three vowels (/i, a, u/)). All stimuli (N =90) words or phonologically legal non-words in their respective languages, were spoken by a native speaker of each language (all young females: Cantonese, 24 years; Chinese Mandarin, 30 years; Singaporean Mandarin, 28 years; Thai, 27 years). The speaker produced six to eight tokens of each tone word. Two most representative tokens (not the first or the last to avoid ‘list’ intonation, or productions without halts, voice breaking or other imperfections) were chosen for the study, with the rider that overall f 0 was roughly equated between languages. The chosen stimuli were manipulated to equate amplitude (at 65 dB) within and between language in PRAAT (Boersma & Weenink, 2013), but not duration to keep the stimuli sound natural. Among the four language systems, only Thai tones share similar durations across tones (due to vowel length being phonological); durations differ across tones in each L. Liu et al. Brain and Language 229 (2022) 105106 4 of the other three languages. 2.3. Procedure The task was run using the DMDX experimental platform (Forster & Forster, 2003). AX trials consisted of two syllables from the same language differing only in tone. The A-X inter-stimulus interval was 1000msec, and there were four types of tone pairs (Same: AA, BB; Different: AB, BA). To encourage phonemic rather than acoustic processing, in the same trials the two instances of the same tone were different exemplars of that tone (i.e., (A 1 A 2 , B 1 B 2 ); and in the different trials (AB, BA) the tone exemplars in A and B positions differed between any particular set of trials. The total number of AX trials was 888, given by: [37 different tone pair trials] * [2 identity (same/different) * 2 order (AA vs BB or AB vs BA orders)] * [2C (/p, p h /) ×3 V (/a, i, u/) =6 syllable contexts]. To avoid fatigue effects, half the participants in each language group (12 out of 24) were tested on the 444 trials in which /p/ was the initial consonant and the other half on the 444 trials on which /p h / was the initial consonant. Participants were required to press the left/right shift key on a computer keyboard if they perceived the two sounds (AX) to be the same/different as quickly and accurately as possible. 2.4. Data treatment and analyses CV contexts (/p, p h / x /a, i, u/) were included simply to enhance generalizability and, as they are of no specific interest, data were collapsed across CV contexts to provide greater power. To control for response biases, the dependent variable was d’, defined as: d’ =Z(hit rate, where hit =‘different’ response on an AB or BA trial) – Z (false alarm rate, where false alarm =‘different’ response on an AA or BB trial). 3. Results The data were submitted to linear mixed-effects modelling (LME) analyses using the lme4 (Bates et al., 2015) package’s lmer function in R (R Core Team, 2018; R Studio Team, 2015). There were three key factors: listeners’ language background (5 levels), tone system (4 levels) and contrast type (5 levels) and the random factor was participant. Results indicated that all three key factors played a role in listeners’ overall performance. Post-hoc effects were obtained through pairwise comparisons using emmeans. Tone pair order across the various contrast types was characterized as ‘high-to-low’ versus ‘low-to-high’ as follows: For S-S trials, ‘low-to-high’ was defined as lower pitch height first and higher pitch height second, and vice versa for ‘high-to-low’; for S-Dr/SDf trials ‘low-to-high’ was defined as S as the first sound in the pair and D the second and vice versa for ‘high-to-low’ trials; for Df-Dr trials ‘low-tohigh’ was defined as Df (decreasing pitch) as the first sound and Dr (increasing pitch) as the second and vice versa for ‘high-to-low’ trials; and for Dr-Dr trials ‘low-to-high’ was defined as the rising tone with the lesser dynamic contour as the first sound and that with the greater dynamic contour as the second and vice versa for ‘high-to-low’ trials. See Supplementary Materials for tables reporting the full model output. 3.1. Discrimination Performance: Cross-language tone perception Fig. 1 shows the d’ results for each of the four tone systems for each of the five participant groups. Linear mixed-effects modelling analyses revealed that tone language speakers performed significantly better than the non-tone language speakers (estimate =0.518, standard error (SE) = 0.0946, z =5.473, p <.0001). This effect was evident for contrasts in Cantonese, Thai and Chinese Mandarin tone systems (estimates >0.351, SEs <0.142, zs >2.908, ps <0.0037), but not for the Singaporean Mandarin tone system (estimate =0.241, SE =0.141, z =1.713, p =.0868) in which scores across participants were high, a ‘ceiling’ effect. Tone language listeners generally discriminated native tone contrasts significantly better than non-native contrasts (estimate =0.374, SE =0.0521, z =7.178, p <.0001) (Fig. 2). This was evident for contrasts in the Cantonese, Thai and Chinese Mandarin (estimates >0.2991, SEs <0.1024, zs >2.922, ps <0.0036), but not the Singaporean Mandarin tone system (estimate =0.0258, SE =0.1255, z =0.205, p =.8374) where participants across tone language background exhibited a ceiling effect. Over and above modulations due to listeners’ language background, discriminability of the four tone systems (Fig. 1A-1D) differed significantly (estimates >0.257, SEs <0.0648, zs >4.457, ps <0.0001). Cantonese tones (1A) were the most difficult to discriminate (emmean = 2.61, SE =0.0477), with best discrimination by the native (dotted-line bar) Cantonese background listeners, (ps <0.0066), and the worst by the non-tone language Australian English listeners (leftmost bar) (ps < 0.0180). This is followed by the Thai tones (1B) (emmean =3.04, SE = 0.0519) with the Australian English listeners performing significantly worse than the native Thai (p =.0025) and the non-native Cantonese (p =.0134) listeners, although there was no difference between Thai and Fig. 1. Tone Language System and Language Background. For the four tone systems (a. Cantonese, b. Thai, c. Chinese Mandarin, d. Singaporean Mandarin), mean d’ scores (y-axis) represent language background listeners’ performances – across the x-axes, from left-to-right: the non-tone Australian English listeners, the mean performance across tone language speakers, then native speakers of that particular tone system (bar enclosed by a discontinuous line), then each of the tone language speakers listening to non-native tones. Error bars = ±1 SE. L. Liu et al. Brain and Language 229 (2022) 105106 5 Cantonese listeners (p =.9896). Discrimination of Chinese Mandarin tones (1C) was easier than either the Cantonese (p <.0001) or the Thai (<0.0001) tones (emmean =3.30, SE =0.0595), with the best performance by the native Chinese Mandarin listeners (ps <0.0467) and the worst by the non-tone Australian English listeners (ps <0.0211). Singaporean Mandarin tones (1D) were the easiest to discriminate (emmean =4.11, SE =0.0595) with no significant differences between listener groups (ps >0.1805). Across listener groups, discrimination performance differed between tone contrast types (Fig. 2) (estimates >0.2761, SEs <0.0836, zs > 4.739, ps <0.0001). Df-Dr (2D) contrasts were the easiest to discriminate (emmean =3.62, SE =0.0583), then S-Df (2B) (emmean =3.11, SE =0.0561) and S-Dr (2C) (emmean =3.07, SE =0.0498) (which did not differ from each other (estimate =0.0426, SE =0.0531, z =0.802, p =.9301), then S-S (2A) (emmean =2.80, SE =0.0611) with the Dr-Dr (2E) contrasts, being the most difficult (emmean =2.37, SE =0.0778). In terms of the specific native language influence, Cantonese listeners marginally outperformed Thai (p =.0685), Chinese (p =.0540), and significantly outperformed Singaporean (p =.0350) and Australian listeners (p =.0001) in S-S discrimination, and Australian English listeners underperformed other listeners (ps <0.0227) in S-Dr and Df-Dr discrimination. 3.2. Discrimination Performance: Perceptual asymmetry among contrast types Fig. 2 shows the d’ scores for each of the four Tone Contrast Types and the two Tone Pair Orders for each of the five Language Background groups. Tests for orientational asymmetries for contrast type revealed that there were no overall cross-language background differences in the effect of order for the Df-Dr or S-S tone contrast types (estimates < 0.0509, SEs >0.0746, zs <0.681, ps >0.4967). There were, however, significant overall cross-language background differences for: (i) Dr-Dr (more-to-less rising (lighter bars) better than less-to-more rising (darker bars), (ii) S-Df (Df → S pairs (lighter bars) better than S → Df pairs (darker bars); and (iii) S-Dr (Dr → S pairs (lighter bars) better than S → Dr pairs (darker bars) (estimates >0.2521, SEs <0.1142, zs >2.209, ps <0.0272). Within these three contrast types showing significant directional differences: (i) for Dr-Dr, there were no significant interactions with language background, i.e., the more-to-less rising (lighter bars) >less-to-more rising (darker bars) effect was consistent across language background groups, (ii) for S-Df, the Df → S pairs (lighter bars) >S → Df pairs (darker bars) effect was greater for Cantonese and Thai than the for other participants (estimates >0.4382, SEs <0.157, zs >2.804, ps <0.0051), and (iii) for S-Dr, the Dr → S pairs (lighter bars) >S → Dr pairs (darker bars) effect was greater for Australian English and Thai than other participants (estimates >0.5520, SEs <0.122, zs >4.503, ps <0.0001). Taken together, in contrast types where differences occur, there is a general trend that discrimination of tone pairs is better when the first tone is more dynamic (higher pitch, greater pitch contour, or greater increase in pitch contour) than the second tone compared to when the first tone is less dynamic than the second tone. 3.3. Modelling For MDS analyses, the d’ scores were further fitted by the Individual Differences Scaling INDSCAL model (Carroll & Chang, 1970), a 3-way extension of the classical MDS using the smacofIndDiff function in the smacof R package (Borg, Groenen, & Mair, 2013; De Leeuw & Mair, 2009). A two-dimensional solution (pitch height, pitch direction) was opted for based on previous tone perception literature, visual inspection of the screen plots, and the interpretability of the dimensions. Both overall and language-specific cue weighting by listeners is reported using Kruskal’s stress (Stress-1) to evaluate the model fit (Kruskal & Wish, 1978). Results with randomstress() baseline comparisons indicated Fig. 2. Tone Contrast Types and Tone Contrast Orientation. For the five contrast types (a. Static-Static, b. Static-Falling, c. Static-Rising, d. Falling Rising, e. RisingRising), mean d’ scores (y-axis) represent listeners’ performances – across the x-axes, from left-to-right: the non-tone Australian English listeners, then mean performance across tone language speakers, then tone language speakers. Bar colours represent whether tone pairs were presented in the order from low-to-high (darker) or high-to-low (lighter) dynamic orientation. Error bar = ±1 SE. L. Liu et al. Brain and Language 229 (2022) 105106 6 a satisfactory model fit for four models corresponding to each language system (Mair, de Leeuw, & Groenen, 2015). The dimensions of each tone language were interpreted by matching each tone’s f 0 and its derivatives (slope, acceleration etc., see Appendix III for detailed acoustic measurements) to the two dimensions of each tone system as closely as possible as follows: The acoustic measurements that were the most similar to the dimension coordinates were first selected (similarity measured by the Pearson’s r correlation coefficient), and then confirmed by a set of regression analyses to ensure they were significant factors in predicting each dimension (height, direction). These height and direction weights were interpreted to index the relative importance a participant placed on the two cues in tone perception in a tone system and were compared within each INDSCAL model/tone system. Depending on the norming of the common space (of a tone system), they were considered as ranked data (Borg et al., 2013) and analysed by an ordinal regression model: Cumulative Link Mixed Model in R. Model assumptions were checked with the ordinal packages nominal_test() and scale_test() functions (Christensen, 2015). If any independent variable failed these tests (i.e., a significant p-value was returned), that variable was handled differently in the model using the nominal and scale options in the clm() function. 3.3.1. Modelling: Dimensionality The INDSCAL model (Carroll & Chang, 1970) revealed pitch height and direction cues for the four tone systems (Fig. 3). The most salient height and direction cues for listeners were (i) for Thai tones (stress-1 = 0.134): average pitch (r =0.99, p =.001) and mean slope after the first inflection (r =0.90, p =.038); (ii) for Cantonese tones (stress-1 =0.168): start pitch (r =-0.97, p =.001) and end slope (r =0.96, p =.002); (iii) for Chinese Mandarin tones (stress-1 =0.090): end pitch (r =0.97, p =.030) and mean magnitude of slope change rate over the last two time points (r =-0.99, p =.010); and (iv) for Singaporean Mandarin tones (stress-1 = 0.077): min pitch (r =0.99, p =.010) and end slope (r =-0.94, p =.060). Fig. 3. Dimensionality modelling (INDSCAL schematics) of the four tone systems: (a. Cantonese, b. Thai, c. Chinese Mandarin, d. Singaporean Mandarin). The connected points represent acoustic space across listeners (black line) and interpreted dimensional solutions (red line) along the best fit height (x-axis) and direction (y-axis) cues most correlated to the two perceptual dimensions in each of the languages. The greater the overlap of the black and red spaces, the higher the likelihood that the interpreted perceptual dimensions are correct. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.) L. Liu et al. Brain and Language 229 (2022) 105106 7 3.3.2. Modelling: Cue weighting The main effects of language background and cue dimensions in each of the four tone systems were examined (Fig. 4 and Table 1, see Supplementary material for detailed weighting magnitudes across tone systems and language backgrounds). Overall, when tone background participants listen to tones in their native languages (dotted-line bar) Cantonese listeners favoured direction (End Slope) over height (Start Pitch) (D >H); Chinese Mandarin listeners favoured height (End Pitch) over direction (End Acc (abs) (H >D); Singaporean Mandarin listeners favoured direction (End Slope) over height (Min Pitch), and for Thai listeners, AvePitch and SlopeAfterInf were the most salient, but there was no bias for one or the other (D =H). These native language biases did not transfer directly to non-native tone languages, indeed there were many instances in which listeners adapted to the target language, e.g. when Cantonese speakers with their native bias of D >H listened to Chinese Mandarin, their bias changed to that of the native Chinese Mandarin speakers, H >D. There is, in fact, more adaptation to the L2 than there is L1 → L2 carryover bias. Most notable is that the non-tone language speakers did not adapt at all, they maintained an H >D bias, suggesting that experience with a particular tone language facilitates further facility with other tone languages. 4. Discussion This study of the perception of four tone systems (Cantonese, Thai, Chinese Mandarin, Singaporean Mandarin) by listeners from five language backgrounds (Australian English, Cantonese, Thai, Chinese, Singaporean) showed that tone perception is constrained by (i) language background, (ii) tone system, (iii) tone contrast type, (iv) order of presentation, and (v) salience weighting of height and direction cues. Details of these findings are set out below. 4.1. Language background Language background is a driving factor in tone perception. Consistent with previous literature (Burnham & Francis, 1997; Burnham, et al., 2015; Kaan, Wayland, Bao, & Barkley, 2007; Maggu, Zong, Law, & Wong, 2018), tone language adults show enhanced discrimination of tone compared with non-tone language adults. This facilitation, also evident in infancy research (Mattock & Burnham, 2006), was consistent across four of the five tone systems but was diminished in Singaporean Mandarin, which was found to be highly discriminable (see tone systems below). While previous studies have indirectly shown support for the hypothesis that tone language listeners discriminate their native tones better than the non-native ones by comparing the perceptual outcomes of the same tones/tonal contrasts from different tone language speakers (e.g., Reid et al., 2015), our results show directly that tone language speakers’ perception of native tones is better than their perception of non-native tones and do so quite convincingly – for three of the four tone language speaker groups here – Cantonese, Chinese Mandarin, and Thai (with the lack of effect for Singaporean Mandarin speakers being possibly due to all tone language speakers doing well on Singaporean tones). These observations likely point to the effect of specific linguistic experiences, such as perceptual assimilation be it on a phonetic (So & Best, 2014) or phonological (Bohn, Avesani, Best, & Vayra, 2011) level. These findings also dovetail with existing neurophysiological evidence for tone perception. In a positron emission tomography study comparing Thai and non-speech tones, Thai listeners showed significant activation in the left frontal operculum, suggesting phonological processing, whereas Chinese and English listeners exhibited sensitivity in the anterior insular region, indicating phonetic or non-linguistic processing. Hence, tone processing is constrained by language (tone versus nontone; native versus non-native) experiences (Gandour et al., 2000). 4.2. Tone system Discriminability of the four tone systems differed, with an ordinal ranking in discrimination ease, of Singaporean Mandarin (4 tones) > Chinese Mandarin (4 tones) >Thai (5 tones) >Cantonese (6 tones). At first glance, such ranking may be related to the complexity/density of tone space. However, that explanation cannot account for the difference between the two Mandarin tone systems which should share similar complexity. A recent study has proposed that Singaporean Mandarin tones may have a unique articulatory origin. Using ultrasound to track laryngeal movement, Yun and Moisik (2019) found that compared to Chinese Mandarin speakers, Singaporean Mandarin speakers’ articulation of tones involves the additional use of laryngeal medialization and approximated thyroid lamina. It needs to be investigated whether these mechanisms also facilitate tone perception. The advantage for Singaporean Mandarin over Chinese Mandarin tones was evident in Australian English, Cantonese and Thai speakers, but the relative superiority was not so great for the Cantonese and Thai speakers. This may reflect similar static and dynamic tonal distributions/spaces in Chinese Fig. 4. Cue weighting scores for the four tone systems (a. Cantonese, b. Thai, c. Chinese Mandarin, d. Singaporean Mandarin). Mean weight scores (y-axis) represent listeners’ performances (x-axis, left-to-right: Australian English listeners, then mean performance across tone language speakers, then native speakers of that particular tone system, then tone language speakers listening to non-native tones. Bar colours represent direction (darker) versus height (lighter) cues. Error bar = ±1 SE. L. Liu et al. Brain and Language 229 (2022) 105106 8 Mandarin and Singaporean Mandarin tone systems, and is also in line with findings showing the number of static tones in listeners’ L1 can predict their discrimination ability at least for static tones (Chiao et al., 2011; Qin & Mok, 2011). Thus, there is evidence for a general constraint that the density/diversity of tone space in the native language affects tone discriminability irrespective of the listeners’ language background. While this might be a key constraint affecting tone discriminability, it is not the only factor. 4.3. Tone contrast type In order from easier to more difficult, the discrimination ease of the five contrast types was: Dynamic-Dynamic (Df-Dr) >Static-Static (S-S) >Static-Dynamic (S-Df =S-Dr) >Dynamic-Dynamic (Dr-Dr), which is in line with previous studies (Burnham et al., 1992; Burnham & Francis, 1997; Huang & Johnson, 2010; Liu & Kager, 2018). Turning to the effect of language background on contrast type, for Df-Dr, Australian English listeners performed more poorly than all tone language listeners, whose performance did not differ. For the most difficult Dr-Dr (rising versus. rising) contrast type, there was no influence of language background on performance, which could be interpreted as being due to the acoustic/ phonetic similarity of the two rising tones. However, it is equally plausible that for non-tone language speakers the difficulty is at the acoustic/phonetic level (So & Best, 2014), but for the tone language speakers, the difficulty may be more phonological (Reid et al., 2015). In support of such a conclusion, (i) the non-tone (Australian English) language listeners perform much better on the high-to-low order of presentation of Dr-Dr contrasts, whereas for the tone language background speakers this is not so apparent (greater acoustic effect for the non-tone language speakers), and (ii) Thai is the only tone language system which does not have a Dr-Dr contrast (see Supplementary material I) and Thai language background speakers perform more poorly on Dr-Dr contrasts than do the other three tone language background groups. 4.4. Tone order The order in which tones are presented affected discriminability. Better discrimination of tone pairs was observed when the first tone is more dynamic (higher pitch, greater pitch contour, or greater increase in pitch contour) than the second tone compared to the other way around. Five explanations that have been proposed to account for such perceptual asymmetries can be considered. First, the under-specification hypothesis (Cornell, Lahiri, & Eulitz, 2013; Lahiri & Reetz, 2010) has been used to explain asymmetries in consonant (Gaskell, 2003; Hestvik & Durvasula, 2016), vowel (De Jonge & Boersma, 2015; Scharinger et al., 2012), and tone perception (Politzer-Ahles et al., 2016). This hypothesis argues that underspecified (and therefore more inclusive, flexible) features are better discriminated than specified features (Shafer, Schwartz, & Kurtzberg, 2004; Schluter et al., 2016; Schluter, Politzer-Ahles, Al Kaabi, & Almeida, 2017), resulting in the prediction that a presentation order with high-to-low dynamicity would be more underspecified. Second, in prototypicality theories, it is considered that non-native phonemes that are more deviant from prototypical members of a phonological category should be more salient and more easily discriminated from the prototype, than those closer to the prototypical members (Best, 1994; Best & Tyler, 2007; Ikeda et al., 2002; Kriengwatana & Escudero, 2017). This would imply that the more dynamic a tone the more it is prototypical. Third, in the Natural Referent Vowel framework Table 1 Summary of the weighting of relative height (H) and direction (D) cues of the 4 tone systems and the 5 language experience groups plus means for tone and nonnative tone groups, along with the specific significant height and direction cues with estimates (est), standard errors (SE) and z scores (* p <.015; ** p <.001; *** p <.0005) and gloss, i.e., whether D >H, H >D or D =H. Tone System Language Background Cantonese Chinese Mandarin Singaporean Mandarin Thai Australian (Non-Tone) All Tone Languages All Non-Nat Tone Langs Cantonese D>H End Slope > Start Pitch est= -2.59 SE = 0.300, z = -8.652** D=H NSDs D>H End Slope > Start Pitch est= -2.59 SE = 0.300, z = -8.652** D>H End Slope > Start Pitch est= -2.59 SE = 0.300 z = -8.652** H>D Start Pitch > End Slope est= 1.7810 SE = 0.509 z = 3.497*** D>HD>H Ch-Mand H>D H>D H>D H>D H>D --- H>D Ch-Mand vs rest est= -1.787 SE = 0.493 z = -3.624*** All 5: End Pitch > End Acc (abs), est > -1.786, SEs<0.579 zs>-3.624** L. Liu et al. Brain and Language 229 (2022) 105106 9 (Polka & Bohn, 2003, 2011), vowels that are more peripheral in F1-F2 space are discriminated better from more central vowels than vice versa, leading to the prediction that tones with higher dynamicity are considered more central in this model. The fourth explanation targets the psychophysics of tone salience (Burnham & Singh, 2018; Chandrasekaran, Krishnan, & Gandour, 2007, 2009; Krishnan, Gandour, & Bidelman, 2010) and is grounded in empirical findings demonstrating relationships between acoustic–phonetic properties and phonological development (Polka & Bohn, 2011; Tsuji, Mazuka, Cristia, & Fikkert, 2015). This would indicate an initial higher salience would be more distinguishable than if the initial tone were less salient. Finally, as pitch is a salient linguistic feature humans are exposed to since and before birth, we hypothesize that attentional factors may also play a role, such that orders leading to higher attention and arousal will be better discriminated, in our case the order of high-to-low dynamicity. Table 2 summarizes these explanations in the current context. These data that we report are the first systematic demonstration of tone asymmetry; contrasts presented from a high-to-low dynamic direction are better discriminated than those presented from a low-to-high dynamic direction. We further hypothesize this as a potential general constraint in tone discrimination and learning. 4.5. Cue dimensionality and weighting The results regarding pitch height and direction cues reflect both flexibility and constraint, illustrating the tight bond between listeners’ tone perception and the relative weight of height and direction cues in their native tone language (Francis et al., 2008). Of great importance are (i) differences in cue weighting between languages and (ii) how those differences may influence perceptual cue weighting in L2 tone perception. First, there were systematic differences in how listeners from different language backgrounds weigh perceptual cues (Table 1). In the perception of their native tone systems, Chinese Mandarin listeners weighted height over direction cues whereas Singaporean Mandarin listeners weighted direction over height. Neither of these Mandarin varieties has two tones that differ only in f 0 height, leaving the unexplained finding of Chinese Mandarin listeners’ greater reliance on height cues a subject for further exploration. Unlike Mandarin, in addition to contour tones, Cantonese and Thai each has three level tones contrasting in f 0 height with little f 0 modulation across time (Abramson, 1962; Mor´ en & Zsiga, 2006), so the equal weighting of height and direction by native Thai speakers for Thai, and of direction over height by native Cantonese speakers for Cantonese are relatively explicable (although given the number of static: dynamic tones in each, 3:3 for Cantonese and 3:2 for Thai, the opposite could have been expected). Second, regarding cues for L2 tone perception, previous studies indicate that Cantonese listeners show more reliance on direction than height cues (Gandour 1983; Li & Shuai, 2011), and that tone and nontone language speakers attend to both height and direction cues when listening to Thai (Chandrasekaran et al., 2010). Our results depict a somewhat different picture (Table 1). Chinese Mandarin listeners, when listening to Singaporean Mandarin, shifted their native tone H >D reliance to align with that favoured by native Singaporean listeners, D > H; but while their reliance also changed for Thai (to D >H) and Cantonese (D =H), neither aligned with the native speakers’ reliance. Singaporean Mandarin listeners exhibited perfect adaptability; given their D >H for native tones, they remained with D >H for Cantonese, shifted to H >D for Chinese Mandarin, and to H =D for Thai, weightings that might be considered optimal as they are the weightings that native listeners use for those languages (shaded cells in Table 1). Cantonese listeners were more similar to Chinese Mandarin listeners; they changed their native D >H weightings for all three of the other languages, but this only accorded with the native weighing in the case of Chinese Mandarin (H >D). Finally, Thai listeners showed some degree of adaptation: from their native D =H, they shifted to native languageappropriate weightings for Cantonese (D >H) and Chinese Mandarin (H >D), but not for Singaporean Mandarin (D >H rather than D =H). The non-tone Australian English listeners showed strikingly different results. They were insensitive to the optimal cue weighting in different tone languages and consistently relied on H >D across the tone systems (except for Singaporean Mandarin – H =D). This is in line with previous studies on English listeners’ reliance on height cues in their perception of Cantonese (Francis et al., 2008; Gandour, 1983; Gandour & Harshmen, 1978) and Thai (Burnham & Francis, 1997). Overall, these results depict a slightly different picture from previous cue weighting studies which suggest that listeners’ cross-language tone perception is determined by the cue weighting in their native language (e.g., Francis et al., 2008). Rather, it appears that tone language speakers are relatively flexible, shifting cues to a strategy that resembles the cue weighting of native speakers. Singaporean Mandarin and Thai listeners show the greatest adaptability, and their Cantonese and Chinese Mandarin counterparts do so with varying success. In contrast, the non-tone language listeners appeared insensitive to the differences between tone languages; they maintained an H >D strategy (H =D for Singaporean Mandarin) across tone languages. Therefore, rather than a transfer of cue weighting priority from tone language speakers’ L1 to L2, our results suggest that tone language speakers are sensitive to differences in tone systems, and that it is the nature of the tone system rather than listeners’ native tone language background that governs cue weighting in tone perception. That said, having a tone language background is crucial for such adaptability. Australian English (non-tone) listeners were resistant and nonadaptive; they consistently weighted height over direction cues. This can be viewed in two ways, as (i) insensitivity to the nuances of tone languages resulting in non-optimal performance, or (ii) maintaining a preferred approach that, as a general strategy, works reasonably well across all tone languages, and specifically, selective attention to the general height cues that are present in most tone contrasts and selectively restricted attention to cues that are not universally useful across tone contrasts. In sum, tone language listeners do not necessarily use the same cues in native and non-native tone perception. In fact, they tend to adapt flexibly to the acoustics of the target language rather than fixedly transferring their L1 priorities. Non-tone language listeners, on the other hand, do not show such adaptation, for them it appears that all tone languages are equal, as in equally foreign. 5. Conclusion Some previous research supports the notion that tone perception is flexible, independent of native phonology, whereas other research suggests it is constrained by native phonology or perceptual asymmetries. Given that such mixed results could be due to stimulus and/or task Table 2 A summary of perceptual asymmetry explanations against the current finding. Under these explanations, the presentation orders from a high to low dynamicity would be considered as being more specified, more typical and central representation, with higher salience and arousal than the other way around. Results/ Hypotheses Tone type A Tone type B Discrimination performance Empirical Results (here) High dynamicity Low dynamicity Hi → Lo >Lo → Hi (dynamicity) Underspecification Specified Underspecified Spec → Under >Under → Spec Prototypes Typical Atypical Typical → Atypical > Atypical → Typical NRV Central Peripheral Central → Periph > Periph → Central Salience High Salience Low Salience Hi → Lo >Lo → Hi (salience) Attention High Attention Low Attention Hi → Lo >Lo → Hi (attention) L. Liu et al.