Demisyllable based Spanish Number Recognition Experiments

European Conference on Speech Technology Edinburgh, Scotland, UK September 1987 ISCA Archive http://www.iscaĆspeech.org/archive European Conference on Speech Technology, Edinburgh, Scotland, UK, September 1987 2321 DEMISYLLABLE BASED SPANTSH NUMBER RECOGNITION EXPERIMENTS Jose B. Marifio, c. Nadeu, E. Lleida * ABSTRACT The main features of our demisyllable based continuous speech recognition system (RAMSES) are showed. Special attention is paid to demisyllable definition and the syntactic constraints used with the dynamic programming algorithm; particularly, the grammatical inference method is described. Recognition scores are not included but they could be provided on request. INTRODUCTION Nowadays, syllables and demisyllables are under active investigation as phonetic unit for continous speech recognition systems; reasons for that are well known. As far as the Spanish language is concerned, the large nurober of syllables and the relatively small inventory of Spanish demisyllables (about 500 initial ones and 160 final ones) are two good reasons to prefer the demisyllable as recognition unit. By using tools from connected speech recognition it has been built in our laboratory two alternative versions for a continous speech recognition system. The first one uses both templates and the time-warping approach and it is essentially speaker dependent; the second one is based on hidden Markov Models, and the decision is made from the maximum 1 ikel ihood path. In both vers ions, the one-stage algorithm (ref. 1) performs the dynamic programming optimization, and the set of possible sentences is specified to the recognition system by syntactic constraints. To test the performance of this recognition system, two vocabulary sets have been chosen. The first one is the Spanish nurober set from zero to one thousand; the telephone numbers pronounced in a Spanish natural way (3/21/69/20, three/twenty one/sixty nine/twenty) constitute the second set. DEMISYLLABLE DEFINITION AND REFERENCE EXTRACTION Every possible syllable is divided by the strong vowel into an initial demisyllable (/ids/-) and a final demisyllable; the cutting is made in an asymmetric way, like in ref. 2. * Dpto. Teoria de la Sefial y Comunicaciones. Universidad Politecnica de Catalunya. Apdo. 30.002, 08080 Barcelona, Spain. This work is supported by the CAICYT grant number 21096/84. European Conference on Speech Technology, Edinburgh, Scotland, UK, September 1987 2322 This cutting allows the final demisyllable to account for the stress of the syllable; accordingly, we distinguish between stressed final demisyllables (-/sfds/) and unstressed final demisyllables (-/fds/). The references for the demisyllables are taken from nonsense words, defined trying to provide a neutral phonetic context to each demisyllable. In general, initial demisyllables are collect from words like /ids/-pa (i.e. the word "cuapa" is used to obtain the demisyllable /cua/-); however, when the demisyllable can not be at the begining of the ward, the following alternative construction is considered: pa/ids/-pa ( i.e. paropa); in these nonsense words the stress is always made in the demisyllable part. Final demisyllables are taken from the word tap-/sfds/ or tap-/fds/ and the stress is made a c c o r d in g t o t he s t r e s s e d o r n o n s t r e s s e d c h a r a c t e r o f t h e demisyllable. In several special cases, the sound of the final demisyllable depends on the next syllable; in a such situation, the word used to extract the demisyllable is properly modified ( i.e. we use "posba" when the realization of the phoneme s is voiced). In the recognition process, the demisyllable reference will always be ident ical indepently from the word o.r phase where i t a p p e a r s • Pos s i b 1 e a 1 o p ho n i c o r p r o n o u n c in g a 1 t e r n a t i v es for words or sentences are incorporated as different entries of the vocabulary set. The whole of demisyllables references is intended to cover every sound all over the vocabulary. For the nurober recognition experiments we use 39 initial demisyllables, 8 unstressed and 15 stressed final demisyllables. GRAMMATICAL INFERENCE Every word or phrase (number for our experiments) constitutes a string of demisyllables; legal strings can be described in termsofafinite state grammar (ref. 1), where each state corresponds to a demisyllable. In general, it could be necessary to use several different states for the same demisyllable in order to account for the different contexts in which the demisyllable may appear. Hereafter, we will call "copies" to these alternative states. A finite state grammar can be determined by the set of predecessors of every state. In the following paragraphs the algorithm we have used to inference the grammar, is outlined. STEP 1: The legal strings of the vocabulary are processed sequentially in order to establish the necessary copies for every demisyllable and the corresponding sets of predecessors P and successors S; the latter is used only fo.r purposes of the inference algorithm. The processing rules a re: rule 1: If a demisyllable appears several times in a string, every occurrence must be associated to a different copy. European Conference on Speech Technology, Edinburgh, Scotland, UK, September 1987 2323 rule 2: A copy may be shared by several strings, only if every string exhibits the same substring from the demisyllable copy to a specified ending of the string (i.e. the shared substring must always be predecessor or always be successor of the demisyllable). Once the copies of every demisyllable are determined, we can inmediately obtain the sets of predecessor and successors. STEP 2: In general, the application of step 1 overestimates the number of necessary copies for each demisyllable; the following lemma bounds this number. lemma: Let be n the number of different predecessors of a demisyllable ancf n the number of different sucessors. If n is the lowest of n8 and n , it is possible to redefine the copies in such a wa~ that ~heir number will not exceed n. To achieve this goal it is enough to use the following: rule 3: Let be n equal to n (n ); each new copy is defined with only one predecessorp(su~cessor) and with a set of successors (predecessors) determined as the union of the successor (predecessor) sets of every old copy that exhibits the predecessor (successor) defining the new copy. In several cases, further reduction of achieved by using the following: cop ie s can be rule 4: Two copies that exhibit the same set of predecessors (successors) may be merged in only one copy, which successor (predecessor) set will be the union of sets of the original copies. The modification of copy definition make us to update the P and S sets of all demisyllable copies of the grammar. The algorithm ends when no reduction can be accomplished. Figure 1 illustrates the exposed grammatical inference procedure. DEMISYLLABLE LENGTH VARIABILITY CONSIDERATION One of the main sources of recognition errors is the variability of syllable duration in natural or continuous speech. Although the dynamic programming algorithm provides a reasonable treatement of this problem, we have implemented in our system two aditional strategies to account for it: a) considering various references for every final demisyll~ ble; b) allowing the optimum path of the dynamic programming algorithm to skip several frames at the end of an initial demisyllable and at the begining of the following final demisyllable. European Conference on Speech Technology, Edinburgh, Scotland, UK, September 1987 2324 RESULTS Unexpected problems have precluded the generation of the data base necessary to evaluate the performance of our recognition system. As a consequence, at present we can n6t afford significant results. At the conference we will sho'w the recognition scores for one speaker provided by the version with templates; the results of our preliminary experiments are encou~aging, because testing 20 numbers from zero to one hundred no error has taken place. The final results will be provided on request. RE FE RENCES /1/ H. Ney. "The use of a One-stage dynamic programming algorithm for connected word recognition". IEEE Trans. ASSP-32, no. 2, pp. 263-271 (1984). /2/ A.E. Rosenberg et al. "Demisyllable-based isolated word recognition system". IEEE Trans. ASSP-31, no. 3, PP• 713-725 (1983). VOCABULARY LEGAL STRINGS: abc, abd, eabc, fabc, gabd. STEP 1 p copy s 1 b1 a 1 b1 c1/ 1 2 b2 . a e a 1 b1 dl/ 1 3 b3 . a g a 1 2 b2 c1/ 1 b1 1 d1 . e a a c , f 1 2 b2 c1/ 2 b2 1 . a a c 1 3 b3 dl/ 3 b3 dl . g a a STEP 2 p copy s p copy s 1 b 1 b2 1 1 b1 a ' • 'e a 1 2 b1 1 2 b2 e a • 'g a 1 3 b2 1 b1 1 g a a c 1 2 b1 1 2 b2 d1 a ,a c a 1 3 b2 dl a ,a f irst iteration second iteration Figure 1. Example of grammatical inference, (.=begin, /=end, P=predecessor set, S=successor set).