A comparison of several recent methods of fundamental frequency and voicing decision estimation

A COMPARISON OF SEVERAL RECENT METHODS OF FUNDAMENTAL FREQUENCY AND VOICING DECISION ESTIMATION ‘*2Eric Mousset, ’ Wlliam A. Ainsworrh, 2J~~e‘ A.R. Fonollosa ‘Department of Communication and Neuroscience, Keele University, Keele, Staffordshire ST5 5BG, U.K. 2Universitat Politknica de Catalunya, E.T.S.E. Telecomunicacib, Barcelona 08080, SPAIN ABSTRACT In this paper, we an interested in the comparison of several idnds of methods for fundamental frrquency estimation and GCI (Glottal Closure Instant) detection. These methods operatc in various doperformances have been compand for both fundamental frequency estimation and voicing decision tasks as well as GCI detection, when applicable. This comparison was designed to be as unbiased BS possible, SO as to nRect the infrinsic propaties of each method. A “d based on a “Bom-Jordan” kernel bilinear ti-frequency representation of speech signals achieves the best performance in terms of GQ. deuction accuracybut is not as robust to inter-speaker variability as the SIFT algorithm. An auditory model, which has been applied on the same data in a previous smdy, has been shown to compare favourably to other methods (such as SIFT) in adverse noisy conditions only. 1. INTRODUCTION The work presented here should be seen as a drst step towads the preparation of an experimental framework devoted to systematic evaluation and comparison of FO (fuadamental frequency) estimation methods, including GCI (Glottal Closure Instant) detection mahods. A common, low ambient noise. continuous speech databasehasalrcadybeen~cordedand~~forthispurpose[8]. This study consists of a comparison between several kinds of methods for GCI detection (a SIFT-based method, a Frobenius Norm based method and two bilinear time-frequency based methods) and for FO estimation (a modulated gaussian wavelets based algorithm and an AMPEX based algorithm). The performance of the various methods have been evaluated for both fundamental frequency estimation and voicing decision tasks as well as GCI dewtion, when applicable. This evaluation is not @od for only one configuration of parameters but for a range of values of the most influencialparametcrs. in other words, for each tested algorithm. such values have been varied between each database processing session. A brief presentation of the methods to be compared is given in section 2. Section 3 is devoted to the comparison framework itself and pnsents the database for evaluation and the evaluation critnia. Results an pnsented and discussed in section 4. mains (*. frequmcy- OT joint tim~-fnquenCy domains). Th& 2. PRESENTATION OF THE COMPARED METHODS 2.1. GCI estimation methods The methods as they an pnsented below do not actually pmduce series of GCIs, but a onedimensional temporal signal the peaks of which an expected to indicate GUS. This resulting signal can then be used in several ways, to ex- GUS or to locate the voiced frames for example. For each of the techniques prcsentcdbelow, the GCI detection process can be seen as a chain of four successive pm cessing steps: 1. Acoustic speech signal pmmphasis (optional). 2. Transformation aiming at pmducing peaks at GCIs. 3. Post-processing aiming at incnasmg contrasts in the resulting 4. Peak picking operation. signal (optional). These four steps are f” now on nfemd to by their item number. In this section, we focus mostly on step 2 and ignore step 4, which will be addnssed below (see section 3.2). A SIFIrbpsCdmetbod. This method. proposed by Plante et al. [2], operates in the time domain. It consists of a Sm based filtering of the speech signal extacting the socallcd residualsignal. Steps 1 to 3 can be arbitxarily sequenced the following way: The signal is first passed to a plecmphasis module improving the accuraq of the LFC (Linear Redictive Coding) analysis @aformed on 25.6 ms asynchronous windows, overlapping by 12.8 ms). The filter comsponding to the vocal tract is calculated from the LPC coefficients and the nsiddsignul is obtained by inverse 6ltering. In order to increase the nsidual amplitude for voiced frames, the nsidual signal is weighted by the energy ratio between the original and the pn-emphasised versions. In practice, the rendual sigtwl generally contains some noise corresponding to vocal tract charaaeristics. To remve some of this, the signal is clamped, low-passed lilted and its envelope is calculated using a Hilbert -form. 1273 A Frobenius norm based method. This method was proposed by Ma et al. [l] as an altemative to more conventional methods based on LPC. It relies on the computation of the Frobenius nom of a matrix LW(,,~~), the rows of which arc formed with sequences of speech signal samples, using a simple rectangular sliding window of length p + 1 samples (shifted sample by sample between two successive rows). It has been demonstrated in [l] that the Frobenius norm of matrix M can be also expressed in terms of its p + 1 singular values e; (assuming that m 2 p + 1 and that m has full column rank, i.e. p + 1). The following expression C (whose computation does not require any eigenvalue decomposition) is expected to produce peaks at GCIs: It should be mentionned that steps 1 and 3 are absent in the original version of this method. Two bfiear timefrequency repreentation based metiiods. In this study we only consider one bilinear TFR (Time-Frequency Rep resentation) based method of epoch detection. The e&a of using two different kemels for this method is investigated: 0 the Born-Jordan kemel, 0 the cone-shclpcd kemel (or cone krmel). The principle of TFR based methods, reported by Navm and Esquerra, is inspired by the WO& of Flandrin, who has proposed a nonparametric time-frequency formulation of a general ciass of receives VI. Given an observation f(t) of a signal and Ctr(t, f; U) its &hen’s class TFR using kemel 9, an adaptation of Flandrin’s nceiver has bcen proposed in p73 in orda to make it suitable for practical issues of GCI detection (where (T) is some shortinrtgration time interval): According to the four step processing scheme described above, step 1 is ignored. step 2 is achieved by evaluating expression (1) for each signal sampk and step 3 includes a conat enhancemeat oprration followed by a compression of the dynamic range (by application of a logarithm). 2.2. F’O estimation methods A modulated gaussian wavelets based plsoritam. The algorithm proposed by Janer is based on a family of 17 gaussian wavelets, whose mother wavelet dilation parameter has been tuned so that tbe whole family behaves like a Bark scale lilter-bank [4]. The first step of this algorithm consists in picking peaks in each of the 17 bands. Then, for each single band, eachnew peak mark is either validated or rejected according to a criterion based on the time interval betmen consecutive marks. The rest of the algorithm then relies on the following twofold general assumption. For each glottal cycle, at least one of the 17 detectos will always produce a mark and such marks will always fall in a common small time inmal (with respect to the current glottal cycle length). All marks which are produced during the first phase are stacked. This operation results in a time series of clusters of marks ; the time interval between two consecutiveclusters being expectedto provide an estimate of the comsponding local glottal cycle length. In a last step, this series of marks is processedin order to select only one mark per cluster. M) estimation based on an auditory model. In a previous study, FO estimation accuracy of an auditory model was evaluated using the same speech database as in the current study [2]. This model con” the key elements of the AMPEX algorithm but its last step consists in using the cochlear nucleus onset units. The latter selectively enhancepitch periodicities by summing cochlearncrve activity over wide frequency bands (7 barks) and perfodng a sort of peak picking operation. 3. COMPARISONSCHEME 3.1. Database for evaluation We used a database created at Keek University D]. which aims at providing a common general framework for the evaluation of FO and GCI estidon methods. It includes two kinds of signals: Iraditional acoustic speech signals and laryngograph signals (single spcakcrrrcordiug). Five adult female speakers and five adult male speakm were recorded m low ambient noise conditions using a sound-proof mom. Each utterance consisted of the same phonetically balanced English furt In each case, the acoustic and laryngograph signals an timGsynchconised (Le. start and end at the same iustants) and shan the same sampling rate value of 20,000 Hz. For GCI detection pafomance evaluation, we designed our own GCI xefcrmcc label files. as advised m [31. using a technique which consists m estimating the GCIs by looking for the minima of the first daivative of the laryngogmn signal [SI. As far as the evaluations related to VN (Voiced/Unvoiced) decision and FO estimation wen conccmcd, we used the reference files provided in the &le database, which contain a VN decision and a pitch estimate for each lOmt block of speech. Segments where no consistent and obvious decision could be made by visual inspection arc labeled as uncertain and ignored during the evaluation. 3.2. Pre- and post.processings for GCI production The n~ults presented m section 4 were obtained through two scries of evaluations, the fixst of which involves the original GCI detection methods as they arc describedin section 2.1 above. This means that they can differ in step 1 (preprocessing), step 3 (post-processing beforepeakpicking)andofcoursein step2,embodyingeachmethod‘s peculiarities. In the second series of evaluations, step 1 was added when absent. This has been achieved by equalising the energy level of the original 1274 acoustic speech signal over time. Step 3 has been skippaifor all the tested methods. In both series of evaluations, a common module of peak picking (step 4). based on morphological filtering has been ap plied (see [71 for more detail about this module). Parameter se-. Like any other method, the ones described above an sensitive to parameter values such as analysis window lengrhs, thresholds, etc. We have chosen to hezc these parametus in the evaluation sessions. Only one of them has been tuned for each method and each uttered sentence, i.e. the eventual bias in GCI estimation (see next section). 33. Evaluation scheme Correcting the bm in the GCI estimation. A fist series of tests, the results of which not reported here, has nvealedbiases in the GCI estimates which are speaker dependent [6]. If bmin denotes the optimal length of time interval, which, when added to each GCI estimate (for a given “cl) minimises the global em, then values of b,,,,, measured for each speaker and each uttemcc suggest that this bias is stationary at the speaker level but may vary significantly from one speaker to another (and from one method to another). Then, before starting any further performance evaluation, we decided to comct this bias for each method, each speaker and each uttered sentence. In other words, what is evaluated hereafter is the ability to produce accurate detection of some event in the glottal cycle. The problem of how this event is related to the glottis’ closure has ken ignored. Error types (GCI detection). We distinguished between the four following types of error (for a given RM R3): 0 fie error ve.), if a given AM Ai is such thar the error Eij (see below) is below some threshold (set to 0.1). 0 gross emr (g.e.), if Eij is above this thnshold, 0 non-&&crion (ad), when thm is no AM in V( j), 0 f&e insertion 6%) (or redundant inrcnion), when there are nnnethanoneAMinV(j). Wedistinguishthenbetweenthne and other ones don’t, or none of them do (reM below as case 1. case 2 and case 3, respectively). sub.caSes: all these AMs fall in the gross error case, some do Eij = lei> I CM be seen BP the gb& cycle synchroniedand normaliied em?: (Ai - Rj)/(Rj+l - Rj) if Ai > Rj 0thmKise (Ai - Rj)/(Rj - Rj-1) Error trpcs (FO estimation). Voicedlunvoiceddecision and FO estimation were simultaneously obtained from GCI detection methods by pufoming an autocomlation of their resulting signal (output of step 3). Apart from the voicing mr types W (voiced-to- Unvoiced) and W (Unvoicsd-to-Voiced), we only looked at the fine- and grosserror types, which again illc defined according to the FO refermce value of the corresponding local lOms frame (with a threshold set to OJ), so as to be pitch independent: (3) where FOA(~) (resp. FOR(n)) is the algorithm (nsp. reference) FO estimate for frame of index n. 3.4. Algorithm operating characteristics In the case of GCI detection, the morphological filtering based peak picking process is mostly sensitive to one of its parameten, namely the size of the saucturing element As a consqucncc, when the database is successively processed for different values of this parameter, diffuexu sms arc obtained. Performances related to somecriterion (e.g. nondetection error) improve whilst others are get worse (e.g. false insertion percentage). The same phenomenon occuls with the autocomlation when one is varying the voicing decuwn threshold. Perfoimance results discussed in the next section were obtained by varying these parameters between each database processing session. 4. COMPARISON RESULTS AND DISCUSSION 4.1. Evaluation results Perfonnance results take the form of twodimensional cross-plots, The legend associated with the last figure also applies to the pnvious ones. The results obtained with the “origina” methods. i.e. as they are presented in section 2.1 illc plotted with solid lines whereas mults obtained with “hom0geniscd”pre- and post-processings (see section 3.2) are plotted with dashed lines2. 4.2. Discusion The graphs presented in this paper do not show the bigh &gm of inter-speaker variability existing in the results. Hence, the distance between curves that one can visually obsuve should not be interpreted as a statistically significant difference. As far as GCI detection is conccmed (figure I), the best results were obtained by the bilinear TFR based method when associated with the Bom-Jordan time-wuency kemel. However, this method seems patticulsrty sensitive to the choice of its assochted kuneL The Sm-bascd method pun& out to be the most robust to speaker . tics as far as the bias in GCI estimation was conctmcd (see section 33). The modulated gaussian wavelets based metbod realises the best performance relative to the voicing decision ability tests. Considaing FO estimation (figures 2 and 3). the bat results were obtained by the SIFT-based method in its “homogenised form”, i.e. without any application of a Hilbert tansform before autoconelation. This changing in score indicates that the Comsponding resulting signal is less contrasted than the one nsulting from the Bom-Jordan kne1 mahod. The auditory model bas ben shown in previous studies to compare favourably to the SIFT-based method in adverse noisy The nmon included m this definition frees the perfonnauce m- ’Ona dqlayed from tbe PIoCetdings CD-ROM. pphs should appear ulu f” any dependency on Fo. m color. 1275 conditions [2] while achieving poorer performances when applied to the database used in this study. UI WI a5% &h lao%: e ari ao I 5 51, 0%. 5. CONCLUSION ; itory model V~ISUS inverse filtering. Pm. LOA., 16(5):81-88, 3. G. E Meyer E Plank and W. A. Ainsworth. A pitch extracton nfaene database. Proc. EUROSPEECH’95, pages 837-840, 1995. 4. L. Janer. A modulated gaussian wavelet transform based speech analyser pitch determinaton algorithm. Roc. - 1994. - - - EUROSPEECH9S, pages 401-404.1995. - 5. A. K Krishnamunhy and D. G. Chillden. Wo-channel speech - analysis. IEEE Trans. ASSP, 34(4):73&743,1986. aoos We have presented a comparison of methods for FO estimation and GCI deteaion. Although none of them was originally designed with the intention to be used for voicing decision, we also evaluated their peiformance in this task. The Born-Jordan kernel time-frequency representation based method achieves the best global results and suggests that joint time-frquency analysis is a promising technique for GCI detection. Nevertheless, none of the methods investigated is significantly better than the others, neither globally nor if the evaluation criteria are considered individually. - BTFR ’B.-J.’ (’original’) - BTFR ’c--IC’ (’original’) 35% -, 30% - Frob. Norm (’original’) - SD?T (’with Hilb. tram&’) c - - - BTFR ’B.-J.’ (’homogenised’) ---- BTFR ’c-k.’ (%omogenised’) -. - -..I Frob. Norm (’homogenised’) Sm (’without Hilb. ansf.’) L. ls% - 3 E- 10 % 5% - --- OS0% a0.k W error 40% percentage 60% 80% Figure% Wemrpct vsUVmorpct. 6. ACKNOWLEDGMENTS This work was supported by tfie HCM contract EFtBCHRX-CT93- 0098. We would like to thank Ignasi Esquma, Leonard Janu, Brian ao % WZ 30s 40% Karlsen, Juan-Luis Navm and &thin Williams for their Eriendly and great help. We are also very grateful to Fabrice Plante for several enlightening discussions. I -”_ -- OI 0% Non-detnstSon percclnlllr (56- Figm 1: f.i. pct (case 1) vs ad. pn (GCI detection). 7. REFERENCES 1276