Spectral Measurements of Vowels for Speaker Identification in Czech

Spectral Measurements of Vowels for Speaker Identification in Czech Lenka Weingartová— jan Volín AbSTRACT: The expansion of telecommunication increased the availability of speech recordings which can be used in criminal investigations. Forensic science is amultidisciplinary approach that provides scientific grounds for assessing the evidence in such investigations. Forensic phonetics explores segmental (vocalic, consonantal) and suprasegmental (prosodic) speech parameters that are discriminant among speakers. There is, however, agap between technical data ‑driven and linguistically informed approaches, which we attempt to bridge in this study by examining Czech vowels through rigorous computational means. Seven different methods of quantifying vocalic spectral slope were compared for the purposes of speaker identification. In forensics, the use of spectral slope is mainly limited to the long ‑term average spectra, which are easy to obtain, but have some serious drawbacks. Therefore, in this study, short ‑term spectra of Czech vowels were used: although their extraction is more laborious, they provide more speaker ‑specific information. Of the seven methods tested, two software predefined functions performed unsatisfactorily, while acombination of modified band density difference and band density ratio was able to differentiate among all of our speakers. The effect of vowel quality on these measures was also investigated. KEY wORDS: forensic phonetics, speaker identification, speaker recognition, spectral slope, spectral til 1. INTRODUCTION When afamiliar person starts talking on the phone without introducing him or her‑ self, the recipients are usually able to determine the identity of the caller after afew seconds— possibly just after asingle word of greeting. Our perceptual mechanisms instantly and automatically match the acoustics of the incoming signal to astored pattern of the caller’svoice. Speech characteristics that are idiosyncratic or unique to aspeaker are explored in the field of speaker identification. One of the important practical applications lies in the area of forensic phonetics, adynamically developing discipline, which among other things makes use of speaker identification in solving criminal cases. Forensic phoneticians are often asked to provide expert judgements in court re‑ garding the identity of asuspect on the basis of his or her speech sample. As the num‑ ber of communication possibilities increases, audio recordings as forensic evidence are becoming more and more frequent and the need for qualified experts is growing. Even lay listeners are usually able to determine the gender or approximate age of an unfamiliar speaker from his or her voice and notice some distinct peculiarities. In addition to that, linguistically trained experts can extract information about the 22 STUDIE ZAPLIKOVANÉ LINGVISTIKY 1/2013 speaker’ssocial background, education level, geographical origin, etc., as well as as‑ sess some idiosyncrasies of the voice. The fundamental questions lying at the core of the area of speaker identification are the following: 1) What information are listeners using to pass their judgement concerning the identity of the speaker? 2) Which extractable acoustic measures of speech can be employed to perform asuc‑ cessful computer ‑aided speaker identification? These two questions might be related— if we determine the information in the speech crucial for identification by listeners, we can employ it for an automatic iden‑ tification. And vice versa— finding acoustic features that perform well in computer‑ ‑aided identification experiments may enhance our knowledge about human identi‑ fication abilities. From this point of view, speech descriptors can be roughly divided into two cat‑ egories— high ‑level and low ‑level features (Doddington, 1985, p. 1653). The former consist of linguistic information such as speaking style, word choice, syntactical or morphological peculiarities, etc., not to mention the content of the utterance. While these features can be successfully used by aforensic linguist, they are very difficult or nearly impossible to be extracted and analyzed by acomputer. However, forensic specialists currently agree that purely auditory analysis (or “aural ‑perceptual” in Hollien’s(2002, p. 11) terminology) is insufficient to form an expert opinion to present before court, and ajoint auditory ‑acoustic approach is strongly recommended (see e.g. Nolan, 1990, p. 461, or French, 1994, p. 173). Therefore, considerable attention is paid to the low ‑level acoustic features, e.g., the fundamental frequency, segmental parameters or descriptors of coarticulation, temporal features such as pauses, articulation rate, timing metrics, or anumber of spectral features. All these characteristics are assumed to contain speaker ‑specific information— however, the main challenge lies in their mechanical extraction and quantification. Our study would like to contribute in this respect by testing seven general methods of spectral analysis with the aim to establish their sensitivity to speaker ‑specific differences. Spectral characteristics are tied to the notion of voice quality, commonly known as timbre or colour— the combination of the source signal and its resonances in the speaker’svocal tract. Voice quality has been evaluated purely auditorily for along time, using descriptive frameworks such as those of Laver (1980) or Hammarberg et al. (1980). Nevertheless, emergent techniques of spectral analysis made acoustic assessments possible as well. It is reasonable to expect that spectral characteristics— since they reflect speak‑ er’sindividual vocal tract physiology— are able to convey speaker ‑idiosyncratic in‑ formation. Indeed, spectral features are used in forensics, in particular in the form of long ‑term average spectrum (LTAS) parameters. This approach averages the spectra over longer stretches of utterances (see, e.g., Nolan, 2009 [1983], pp. 130ff., or Dod‑ dington, 1985) and is computationally relatively straightforward. LENKA WEINGArTOVá— jAN VOLíN 23 Several other acoustic features are also used to describe spectral properties of speech— either with the goal of characterizing abnormal voices or phonation types (as in Hammarberg et al., 1980), or for the purposes of speaker identification (see for example Nolan, 2009 [1983]). The parameters used were at first mainly formant means or formant ratios. Newer approaches employ the so ‑called spectral slope. This term (also spectral tilt or spectral balance) refers to the gradual decay of energy to‑ wards higher frequencies in the frequency spectrum of avoice. Achange in spectral slope is associated with achange in voice quality (Hammar‑ berg et al., 1980), in vocal effort (Doddington, 1985; Sluijter— van Heuven, 1996) or with linguistic prominence (Sluijter— van Heuven, 1996; Sluijter— van Heuven— Pacilly, 1997; Campbell— Beckman, 1997, or Heldner, 2001). In many of these studies the authors need to compensate for inter ‑speaker differences and normalize the data; for example, in the paper concerned with spectral slope as acorrelate of emotions in speech, Tamarit, Goudbeek and Scherer (2008) have proposed computing aspeaker‑ ‑specific pivot, otherwise the differences between speakers might obscure the re‑ sults. Our goal, on the other hand, is to exploit this information to differentiate one speaker from another. It should be noted that in order to characterize persons’ overall vocal quality, the spectral slope of along ‑term spectrum needs to be measured. However, when de‑ scribing syllabic prominences, short ‑term spectra of individual vowels are exploited. The former approach, although widely used, has some disadvantages— first, the experimental method has to deal somehow with unvoiced regions of speech or pauses (usually eliminating them from the LTAS computations, as in Hammarberg et al., 1980; Kitzing, 1986; Löfqvist, 1986, or Tamarit— Goudbeek— Scherer, 2008). Moreo‑ ver, the results can be affected by content and length of the speaker’sutterance— in forensic cases, the speech samples can be completely incomparable in this regard, specifically their length is often insufficient to be reliable. Rodman et al. (2002) or Master et al. (2006), explicitly state that long ‑term averages of spectral characteris‑ tics require agreat amount of speech data. It could be therefore useful for practical purposes to measure comparable “chunks” of the material and to compute the spectral slope from short ‑term spectra of the same linguistic elements— these comparable elements are often nasals, fricatives or vow‑ els (see, e.g., French, 1994, p. 176; Nolan, 1983, pp. 75–77, or Jackson et al., 1985). In our study, we can benefit from the fact that Czech has relatively few vowel qual‑ ities and these are not systematically reduced. Therefore, we can compare the same category of sounds for every speaker, whether or not the utterances are the same. Even shorter speech samples can be used provided that the number of usable tokens is sufficient. In this study, we plan to compare several methods for measuring spectral slope of Czech vowels to see which of them are sensitive enough to capture differences among speakers. At this point, we use laboratory, high ‑quality speech data with the aim to prevent introduction of any artifacts caused by signal distortions. If some of the outcomes are promising, the course for future research is to try them under more natural conditions (more spontaneous speech, noise, degraded signal, etc.). 24 STUDIE ZAPLIKOVANÉ LINGVISTIKY 1/2013 2. METHODS Of MEASURING SPECTRAL SLOPE The spectral envelope is an inherently two ‑dimensional curve. To quantify its slope with asingle value the computational procedure has to be determined. Perhaps the most influential method was used by Hammarberg and her colleagues (1980) and was consequently termed the Hammarberg index. The original principle was to measure the difference between sound pressure level peaks in given frequency bands. This basis (measuring the difference between energies above and below agiven frequency value) was developed further by Sluijter and van Heuven (1996), Sluijter, van Heuven and Pacilly (1997), Sundberg and Nordenberg (2006), or Boersma and Kovacic (2006). Adifferent approach is to fit aregression line to the spectrum. It was explored for example in the work of Kochanski et al. (2005). Athird group of methods quantifies ratios of certain peaks in the spectrum, e.g., the amplitude of the first harmonic to the amplitude of the second harmonic, the first harmonic and the third formant (as in Hanson, 1997; Hanson— Chuang, 1999), or the amplitudes of the first two formants (Fulop— Kari— Ladefoged, 1998; Mills, 2009). This approach was with moderate success used for speaker discrimination in Czech by Skarnitzl, Vaňková and Weingartová (2012), although the manual extrac‑ tion of these parameters was with current technological background found to be difficult and time ‑consuming. Volín and Zimmermann (2011) used amethod similar to the Hammarberg index measurements on Czech vowels. Although their goal was to quantify linguistic stress and not to identify speakers, they reported asignificant inter ‑speaker variability in the spectral slope behaviour. One of our objectives is to compare different methods with respect to their practi‑ cal applicability: they should be easy to employ without extensive technical training and should be dependent only on commonly available technology and software. 3. ExPERIMENT mEThOD The material used in this study consisted of recordings of four male native Czech speakers at the age of 20–30 years, with university education, coming from Central Bohemia. They were recorded individually in asound ‑treated studio of the Institute of Phonetics in Prague. The men were selected to represent four distinctive voice types (i.e., they were deliberately chosen not to sound alike). The speakers were asked to read alist of pseudowords in the form CVCVCV (con‑ sonants and vowels being the same in one pseudoword), where the vowel was one of the Czech five short vowels /i ε aou/ and the onset consonant either /m/, /t/ or /h/. These consonants were selected to represent three frequent classes: nasals, voiceless stops and fricatives. We also considered their ease of articulation and distinctiveness in spectrograms (for pre ‑analysis signal processing). Moreover, /h/ was also chosen LENKA WEINGArTOVá— jAN VOLíN 25 as aconsonant with no transient effects. Each pseudoword was recorded three times with different placement of stress— either on the first, second or third syllable. We decided to use pseudowords in order to obtain the results as clear as possible. The effects we were looking for should not be distorted by any extraneous variables. The recordings were automatically labelled (using the Prague Labeller software, see Pollák— Volín— Skarnitzl, 2007) and the boundaries of the phones were manu‑ ally corrected in the open ‑source software Praat (Boersma— Weenink, 2012). Atotal number of 180 pseudowords and 540 tokens of individual vowels was ob‑ tained, 135 vowels per speaker. Six of the tokens (all from the same speaker) had to be discarded due to background noise or non ‑standard quality of the vowel. Thus, for further analyses, 534 vowels were used. Aspectral slice was extracted from the middle third of each vowel in the Praat program. For measurements of the spectral slope, we used functions available in this software. The investigated methods are listed in Table 1. No. Name Interpretation Formula Praat function 1 Band energy difference 1 difference between the sums of energies in the given frequency bands 10 × log10 hbenergy— 10 × log10 lbenergy Spectrum > Get energy difference 2 Band density difference 1 difference between the averages of energies in the given frequency bands 10 × log10 hbdensity— 10 × log10 lbdensity Spectrum > Get density difference 3 Band energy difference 2 difference between the sums of energies in the given frequency bands 10 × log10 (lbenergy— hbenergy)see text 4 Band density difference 2 difference between the averages of energies in the given frequency bands 10 × log10 (Ibdensity— hbdensity)see text 5 Band energy ratio ratio of the sums of energies in the given frequency bands 10 × log10 lbenergy / 10 × log10 hbenergy see text 6 Band density ratio ratio of the averages of energies in the given frequency bands 10 × log10 lbdensity / 10 × log10 hbdensity see text 7 Skewness the shape of the spectrum above and below its centre of gravity see text Spectrum > Get skewness table 1: Overview of the methods used to calculate spectral slope. Hbenergy and lbenergy refers to high band energy and low band energy, respectively. The same applies for hbdensity and lbdensity. 26 STUDIE ZAPLIKOVANÉ LINGVISTIKY 1/2013 The first six methods are based on measuring energies in given frequency bands— the bands being the same across all methods: the low band spans the frequencies from 350 to 1100 Hz and the high band from 2300 to 5500 Hz. It is the best frequency band setting from those examined by Volín and Zimmermann (2011). Note that F0 and, more importantly, F2 range (or at least agreat part of it) are excluded in this set‑ ting— we hypothesize that F0, being very variable within one speaker, would with its energy obscure the spectral measurements; F2 on the other hand marks the linguis‑ tic category of the vowel and therefore might contribute less to speaker identity. For more information about average formant ranges of Czech speakers the reader should refer to Skarnitzland Volín (2012). Figure 1 provides an illustration of avowel spectrum with the measured frequency bands highlighted. The first six methods are arranged in pairs. The first of the pair takes as input the sum of energy in the given band, the second uses the average energy (called density in this context) in the same band. Even though energy and density are dependent on each other, their interpretation is not the same and they might behave differently when using different formulae. One of our goals is to see whether it is more useful to employ the sum rather than average of spectral energies, or vice versa. The seventh method computes the skewness of the spectrum, which is ameasure of asymmetricity of the spectral shape. It is quantified as the difference between the shape of the spectrum below and above its centre of gravity (afrequency which di‑ vides the spectrum into two parts so that the energy in the lower half is equal to the energy in the upper half). An illustration is provided in Figure 2. Methods 3 to 6 were not computed directly with apredefined function from the spectrum, but with the help of aPraat script— the formulae are given in the fourth column of Table 1. Band energies and densities are calculated with the function Get band energy/density. figure 1: Spectrum of avowel [ε] with highlighted energy in measured high and low frequency bands. LENKA WEINGArTOVá— jAN VOLíN 27 The formula for computing skewness will not be stated here for the sake of sim‑ plicity, but it is the standard formula for calculating the third central moment of aspectrum. For assessing the statistical significance of the results, one ‑way ANOVA was used with the values of spectral slope as adependent variable and SPEAKER as factor. For evaluating individual inter ‑speaker differences, Tukey HSD post ‑hoc test was used. To see whether identity of the vowel influenced the results, we also used two ‑way ANOVA with SPEAKER and VOWEL as factors. RESULTS All of the methods resulted in statistically significant ANOVA differences (at the level of α = 0.05), in both of the tests for SPEAKER and SPEAKER*VOWEL. This means that each of the method was able to differentiate at least one speaker from the others. The magnitude of the test criterion F was considerable for all main effects (30 < F < 85), thus the effect size was not calculated. We found out that the results for energy vs. density measures were very similar, they differed only marginally and yielded very similar p values. The advantage we found was that when using methods 3 and 4, it is theoretically possible for the differ‑ ence between high band and low band energies to be negative— it is then impossible to calculate the logarithm. In our data, this happened only in two cases and only when measuring energy; density seems to be more robust. For the sake of simplicity, only graphs with density measures will be presented further. Note that throughout the paper we keep the convention for steeper spectral slope values to be represented lower in the graphs, which in some cases required reversion of the y ‑scale. figure 2: Three functions with different skewness. The points mark the centre of gravity of each function. Function Ais symmetric, therefore its skewness equals 0. Functions B and C are skewed to the right, which results in positive values of skewness. Function C is more asymmetric than B, therefore its skewness will be higher. 28 STUDIE ZAPLIKOVANÉ LINGVISTIKY 1/2013 mEThODS 1 + 2: bAND ENErGY/DENSITY DIffErENcE 1 (bED1/bDD1) Figure 3 shows values of band density difference 1 (the standard function from Praat) for each of the four speakers (one ‑way ANOVA results: F(3, 530) = 30.85, p < 0.001). Higher negative values of BDD1 mean bigger difference between low band and high band energies and therefore asteeper spectral slope. M1 has, using these methods, therefore the steepest spectral slope in his vowels. BED1/BDD1 was able to discriminate only speaker M1, which was confirmed by Tukey HSD post ‑hoc test (p < 0.001 for all differences between M1 and the others). Speakers M2, M3 and M4 were not statistically different from each other (p > 0.05). mEThODS 3 + 4: bAND ENErGY/DENSITY DIffErENcE 2 (bED2/bDD2) Figure 4 shows values of the fourth method, the modified band density difference (one ‑way ANOVA results: F(3, 530) = 84.04, p < 0.001). Note that the scale of the graph is reversed due to adifference in calculation (high band and low band values are in‑ terchanged). As in Figure 3, the values lower in the graph (but in this case higher val‑ ues) mean asteeper spectral slope. The drawback of methods 3 and 4, as mentioned above, is the possibility for the difference between bands to be negative (the case of aflat or reversed spectral slope)— the calculation then fails because of the position of the logarithm in the formula. However, in our corpus of 534 vowels this happened only twice and only for method 3, i.e., energy. Nevertheless, this should be remembered for further research when measuring spectral slope of degraded or noisy signals. Contrary to methods 1 and 2, we now have two speakers (M2 and M3) highly sta‑ tistically different from each other as well as from the two remaining individuals (confirmed by Tukey HSD post ‑hoc test: p < 0.001). The values of M1 and M4 are very nearly the same. In conclusion, these methods clustered the four speakers into three groups, which are clearly distinguishable mEThODS 5 + 6: bAND ENErGY/DENSITY rATIO (bEr/bDr) The results of method 5 are shown in Figure 5 (one ‑way ANOVA results: F(3, 530) = 31.8, p < 0.001). Again, asteeper spectral slope is represented lower in the graph. Al‑ though the picture is quite similar to Figure 3, the speakers are more spread out from each other— M1 is highly significantly different from all the others (Tukey HSD post‑ ‑hoc test: p < 0.001), M2 and M3 are also significantly distinct (Tukey HSD post ‑hoc test: p = 0.008) and M3 and M4 are marginally different (Tukey HSD post ‑hoc test: p=0.059). The only difference this method failed to recognize was between speak‑ ers M2 and M4. This makes BDR and BDD2 successful to the same extent. Moreover, calculating the ratio eliminates the mathematical problem that was identified as the disadvantage of BDD2. LENKA WEINGArTOVá— jAN VOLíN 29 figure 3: Mean values of the band density difference 1 for four speakers. Whiskers denote 0.95 confidence interval. figure 4: Mean values of the band density difference 2 for four speakers. Whiskers denote 0.95 confidence interval. figure 5: Mean values of the band density ratio for four speakers. Whiskers denote 0.95 confidence interval. 36 STUDIE ZAPLIKOVANÉ LINGVISTIKY 1/2013 AbSTRAKT: vzestup telekomunikačních technologií vsoučasné době umožňuje častější využití řečových nahrávek při vyšetřování trestných činů. Forenzní věda je multidisciplinární obor, který poskytuje vědeckou bázi pro posuzování důkazního materiálu během těchto vyšetřování. Forenzní fonetika se zabývá segmentálními (vokalickými akonsonantickými) asuprasegmentálními (prozodickými) řečovými rysy, které mohou odlišovat jednotlivé mluvčí. vtomto ohledu se nicméně rozšiřuje propast mezi technicky alingvisticky orientovanými přístupy— tato studie je pokusem ojejí překlenutí zkoumáním českých vokálů rigorózními komputačními přístupy: pro účely rozpoznávání mluvčího ve forenzní praxi je zde porovnáno sedm metod stanovení vokalického spektrálního sklonu. ve forenzní fonetice byl dosud spektrální sklon používán zejména při měření dlouhodobých průměrných spekter. Tato spektra se snadno získávají, avšak vykazují několik podstatných omezení. Zde jsou tedy využita krátkodobá spektra českých krátkých vokálů, jež přinášejí větší množství charakteristik specifických pro mluvčího, ale jejich extrakce je pracnější. Ze sedmi testovaných metod se softwarem předdefinované funkce ukázaly jako nevyhovující, zatímco kombinace modifikovaného rozdílu hustot pásem apoměru hustot pásem od sebe dokázala odlišit všechny mluvčí. Dále byl také prozkoumán vliv kvality vokálů na výsledky jednotlivých měření. Lenka Weingartová | Institute of Phonetics, Faculty of Arts, Charles university in Prague [email protected] Jan Volín | Institute of Phonetics, Faculty of Arts, Charles university in Prague [email protected]