Perceived feasibility of computer-generated auralization in concert halls

International Seminar on Virtual Acoustics VALENCIA, November 24-25, 2011 153 PERCEIVED FEASIBILITY OF COMPUTER-GENERATED AURALIZATION IN CONCERT HALLS Reference PACS: 43.55.Gx , 43.55.Hy, 43.55.Ka Garrido, José A.1; Zamarreño, Teófilo2; Girón, Sara2; Ruiz-Padillo, Diego P.1 1Dpto. de Física Aplicada, Universidad de Granada, Facultad de Ciencias, Avda. Fuente Nueva s/n, 18071-Granada (SPAIN) Tel.: +34 958244161. Fax: +34 958243214. 2Dpto. de Física Aplicada II, Universidad de Sevilla, ETS de Arquitectura, IUACC, Avda. de la Reina Mercedes 2, 41012-Sevilla (SPAIN) Tel.: + 34 954556612. Fax: +34 954557892 [email protected], [email protected], [email protected], [email protected] ABSTRACT Over the years auralization has become a useful tool for simulating and evaluating the listening experience in virtual environments. Psychoacoustic phenomena, embodied by the human hearing system, highly determine the accuracy of sound-field recreation that is required for seemingly feasible auralization. In this scenario, the main aim of this study is to assess the suitability of computer-generated room impulse responses when used for auralizing spaces. To this purpose, simulated and experimentally measured binaural room impulse responses (BRIRs) have been convolved with iconic musical excerpts for various representative seats within a medium-sized concert hall. Listening tests have been thereby conducted and their results will be shown, analyzed and discussed. RESUMEN En estos últimos años, la auralización se ha convertido en una herramienta útil para la simulación y evaluación de la experiencia del oyente en entornos virtuales. En este ámbito, son de especial importancia los aspectos psicológicos y perceptivos, derivados del funcionamiento del sistema auditivo y del cerebro humanos, que determinan la precisión que se requiere para una recreación aparentemente creíble del campo sonoro. En este contexto, el principal objetivo de este estudio es evaluar la validez de respuestas al impulso de la sala generadas por ordenador para su utilización en la auralización de espacios. Con este propósito, para algunas localidades representativas de una sala de conciertos de tamaño medio, se han convolucionado varias respuestas al impulso binaurales (BRIRs), simuladas y medidas experimentalmente, con fragmentos musicales conocidos. Con los datos obtenidos, se han llevado a cabo pruebas de audición, cuyos resultados se presentan, analizan y discuten en esta comunicación. 1. INTRODUCTION Although auralization has been proved an important tool for multiple applications in fields as diverse as Architecture, Telecommunications, Psychology or Virtual Reality, very little is still known about the accuracy of the output audio files generated by the commercial computer programmes featuring this ability that are currently available in the market. Trying to evaluate the quality of the performance achieved by some of these pieces of prediction software when confronted with in situ measurements, three round-robin tests have been held in history [1][2][3][4]. However, none of them embraced auralization within its scope. When it comes to auralization, physical accuracy does not constitute the only cornerstone for this kind of software, but achieving correctness in the listening experience eventually created International Seminar on Virtual Acoustics VALENCIA, November 24-25, 2011 154 also plays a decisive part [5]. Therefore, Psychoacoustics ought to take over the task of drawing the fuzzy line that determines whether the sound field created with this software is precise enough to generate auditory events deemed correct by a majority of listeners. In order to shed some light on the feasibility of computer-aided auralization, a three-dimension geometric model of a 1,311-seat concert hall, whose acoustic performance had already been validated for the most common parameters, was simulated to obtain impulse responses at various locations spread over the audience area. Experimental measurements previously done at analogous positions in the actual room had already gathered equivalent data. Finally, auralized audio files corresponding to impulse responses from both sources have been produced and subjected to comparison with the assessment of a panel of untrained individuals, who will ultimately look for differences between stimuli throughout listening tests. Likewise, auralizations coming from simulating the concert hall under changing acoustical conditions because of refurbishment works have been judged against those attained in the former state. In this case, an eventual statement will go in the necessity of the undertaking. 2. MANUEL DE FALLA AUDITORIUM First inaugurated on June 10, 1978, Manuel de Falla Auditorium in Granada (Spain) could be proudly counted amongst a selected group of Spanish concert halls with remarkable acoustic qualities. The auditorium is located at La Sabika hill beside the carmen once inhabited by the Gaditan composer after whom the edifice was named, and within the area of influence of La Alhambra, a world-wide admired landmark. Sevillian architect José María de Paredes along with renowned German acoustician Lothar Cremer account respectively for the architectural design and acoustical consultancy towards the executed blueprints of the building6. As illustrated by the photograph in Figure 1(a), the room features a rotund bilateral symmetry on whose axis the stage is disposed at nearly two-thirds of the way, providing an uncommon arrangement of the audience into two main areas, A and B –labelling seats in front of and behind the stage in that order-. One of the main advantages derived from this unusual plan lays on the adaptability of the venue to spectacles of various kinds. Changeable capacity and volume allow for several configurations and, thereof, variable acoustics. As shown in the ground plan of the auditorium displayed in Figure 1(b), three seats within the room have been carefully chosen to carry out this work. It must be said that receiver locations close to either the source or large reflecting surfaces have been ruled out since experimental measurements at such intricate positions usually yield defective results. To properly sample the room, two of them -noted A-1 and A-2- have been placed in front of the stage whereas only one –namely B- has been located in the backstage. In recent times, refurbishment works involving roof renovation, heating, ventilation and air conditioning system remodelling, restitution of deteriorated finishes and replacement of Figure 11. Manuel de Falla Auditorium, in Granada (Spain): (a) general view of the room after the recent completion of refurbishment works –on the left-, and (b) ground plan showing the placement of the source (in blue) and three receivers (in red) considered in this study –on the right-. Architect: J. M. García de Paredes. International Seminar on Virtual Acoustics VALENCIA, November 24-25, 2011 155 audience seating have been undertaken in the grand concert hall. After their completion, the Manuel de Falla Auditorium reopened to the general public in September, 2010. 3. EXPERIMENTAL MEASUREMENTS Extensive on-site measurements were performed in the auditorium under empty-room and cleared-stage conditions, in attendance to what established in the ISO 3382-1 [7] international standard. Temperature and humidity were monitored during the measurement campaign with a digital thermo-hygrometer. Temperature ranged from 19.6 °C to 20 °C, whereas relative humidity exhibited values within the 49% to 50% interval. Binaural room impulse responses (BRIRs) for each receiver location have been obtained by means of 30-second sine sweep signals covering the entire audible frequency range, generated and subsequently analyzed in WinMLS 2004, through the Digigram VX Pocket v2 soundcard. The audio signal so-produced was sent –via an INTER-1000 amplifier- to a 01-dB Stell AVM DO12 dodecahedral loudspeaker –omnidirectional sound sourceplaced onstage at a central position, 1.5 metres high above the stage level. Head Acoustics head and torso simulator Type HSU III (Code 1323) has been used for recording BRIRs at the three seats of choice, allocating both entrances to the ear canals at a 1.2-metre height above the floor. Monaural room impulse responses (RIRs) have been measured as well, allowing for the calculation of widely-known acoustic parameters. Regarding the previously-described set-up, a couple of additional microphones –namely, B&K 4190 ½” omnidirectional and Audio-Technica AT4050/CM5 multi-pattern condenser microphoneswith suitable 01dB-Stell pre-amplifiers have been utilized instead of the head and torso simulator to this purpose. 4. COMPUTER SIMULATION CATT TUCT v1.0f was the piece of room acoustics prediction software chosen to carry out the simulations whose results are hereby presented. TUCT –acronym for The Universal Cone Tracerwas recently born as a CATT-Acoustic-dependent bare calculation engine capable of performing acoustic simulation and auralization of virtual environments. However, CATT- Acoustic keeps on being necessary to build up the three-dimension geometrical model of the room and later define the acoustic features of enclosing surfaces. For the time being, three distinct algorithms have been implemented in TUCT for the generation of echograms and impulse responses for any given source-receiver combination specified in the digital model [8]. Differences among themselves mainly affect the way diffuse reflections are handled. The three of them deal with direct sound and 1st order specular reflections deterministically. So do they with 1st order diffuse reflection by means of the socalled random split-up, a brand-new technique made gradually extensive to higher orders of diffuse reflection in algorithms 2 and 3. It is for that reason that run-to-run variability traditionally associated to computer-aided acoustic simulation due to the usage of Lambert’s law [9] accounting for scattering phenomena has been significantly reduced. TUCT’s algorithm 2 has been the one of choice for attaining the data utilized in this work. It must be noticed that TUCT predictions of the most common acoustic parameters are based on the previous calculation of either echograms or impulse responses, which gives rise to slightly different results coming from energy (E) or pressure (h) analyses respectively. Since the Figure 12. Geometrical model for the acoustic simulation of the Manuel de Falla Auditorium. Audience area is shaded in grey colour. International Seminar on Virtual Acoustics VALENCIA, November 24-25, 2011 156 outcome of the latter follows a procedure that is identical to that instituted by the international standard ISO 3382-1 [7] for experimental measurements –successive octave frequency-band filtering, squaring and pressure-function time integration-, a good agreement of in situ measured descriptors and their IR-based simulated counterparts is to be expected, providing as well clear evidence of similarity between recorded and predicted impulse responses. Additionally, BRIRs at the specified locations can be generated and exported into suitable formats making it possible their later usage for auralization. In order to simulate the acoustical behaviour of the Manuel de Falla Auditorium, a threedimension geometric model of the room –illustrated in Figure 2- has been implemented, in accordance to the guidelines supplied by the developers of prediction software [10] and stated in the literature [11]. Acoustic characteristics for the materials in the model were first estimated and defined with scattering and absorption coefficients [11][12], which have been subsequently fine-tuned by means of an iterative calibration process based on achieving a match in the comparison between measured and simulated reverberation times at various inner locations [13]. Finally, the model resulting from such a procedure has been validated for the rest of conventionally-accepted room acoustics parameters in terms of just noticeable differences (JNDs) as proposed by Bork [3][4]. Moreover, starting from this former model, another one has been developed in an attempt to mimic the changed acoustic features within the concert hall due to a recently finished refurbishment. Both models have been acoustically simulated, main descriptors calculated, and BRIRs produced. As for the initial prediction settings adopted in the computer software, 26,611,576 cones –in application of the formula suggested by Vorländer [11]- have been traced. Truncation time for the calculation of impulse responses and echograms was set at 3 seconds counted from the arrival of direct sound –thus, beyond experimentally-measured reverberation times, as recommended by some authors [10][11]. 5. AURALIZATION Auralization is the technique that allows for the creation of sound files suitable for reproduction from mere numerical data obtained by simulation, measure or synthesis [11]. Physically speaking, such a task requires the definition of the so-called binaural room impulse response, h(t), temporal function based on a three-dimension sound propagation model whose calculation involves the computational technique previously explained. BRIRs can be also depicted as actual filters, f(t), incorporating additional components, to wit: source directivity, head-related transfer function (HRTF) and exact equalization for the sound-reproduction system of choice. The convolution of input sound signals recorded in anechoic environments, s(t), with filter impulse responses, f(t), to produce output audio files as heard by the listeners at any given location, g(t), is the basis of signal processing for auralization, and can be mathematically written as the following linear-time invariant system: !"#$ %&!'#(!")'#*' + ,+ A little stand-alone piece of software called MultiVolver WCP included in CATT-Acoustic package has been used for carrying out the auralizations to be later on presented to the assessors. MultiVolver is intended as a flexible multichannel convolver for multiple applications whose purpose is to convolve input WAV-files into output WAV-files [14]. A mono to binaural configuration –one to two channels-, making use of previously measured and simulated finite binaural impulse response filters, has been taken up in this work. Three short, dry, soloist classical-music samples differing from one another in rhythmical pattern, frequency spectrum and, above all, timbre have been chosen as input signal for auralization: International Seminar on Virtual Acoustics VALENCIA, November 24-25, 2011 157 C. Weber: Theme; for cello [15]. F. Chopin: Prelude, Op. 28, No. 12 in G-sharp minor; for piano [16]. Bruckner: Symphony No. 8 in C minor, I. Allegro moderato; for timbales [17]. It must be noted that, in order to have auralizations from impulse response filters of diverse nature suitable for comparison, the gain in dB applied to each of the three input audio files in either case was the result of a prior calibration process aimed to yield a match in terms of average loudness monitored at the closest location to the sound source. 6. LISTENING TESTS Twenty-seven people have been subjected to listening tests in order to discriminate whether there are sensory differences between audio stimuli. A triangle-test structure [18] has been deemed well fitted to determine both whether: A significant difference exists between auralizations obtained from measured and simulated BRIRs in the Manuel de Falla Auditorium. Such a difference can be stated between auralizations from simulated BRIRs before and after the completion of some refurbishment works within the concert hall. To this purpose, a panel of untrained assessors consisting of Spanish university students with normal-hearing between ages eighteen to twenty-eight was gathered. The campaign of listening tests was carried out in a quiet, windowless room, free from bias. In this environment, acoustical stimuli were administered to the panellists via an ASUS Xonar Essence ST internal sound card and Sennheiser HD 600 over-ear headphones. A picture of one of the individuals undergoing the listening tests immerse in this setup is shown for better illustration in Figure 3(a). The tests have been designed to figure out answers to two questions: Is there a difference between the pair of stimuli under comparison? And were that the case, what is the nature of this difference? To find out, assessors were presented with three of them and told that two are identical and one is different. According to a triangle-test design, six possible orders of sample presentation have been considered: AAB, ABA, BAA, BBA, BAB, and ABB. The use of naïve assessors made it advisory to use a forced-choice mode in the experiments. Panellists were asked to carefully listen to the samples in the order of supply and single the odd one out. After hearing each set of three stimuli for the first time, assessors were not permitted to re-listen to the samples. To do so, panellists could use all available information without any restraints. In addition to this, they have been questioned about the nature of the difference noticed. To facilitate for a panel of untrained individuals to answer to such an inquiry, four key acoustical features –coloration, spaciousness, reverberance and loudnesswere suggested. Having been easily defined to every assessor at the beginning of the experimental session, from none to all of them could be marked in the pop-up window displayed in Figure 3(b), which turned up after the presentation of each set. Figure 13. On the left, one of the voluntary assessors undergoing the tests is being subjected to the experiments (a). On the right, the short questionnaire -in Spanish- to be answered in a window environment after listening to every train of stimuli is shown (b). International Seminar on Virtual Acoustics VALENCIA, November 24-25, 2011 158 Up to eighteen so-conformed sets of three stimuli have been administered in random arrangement to every assessor, regarding three receiver locations, three musical excerpts, and two distinct experiments with the same structure. Prior to the commencement of the actual listening tests, a trial run with two different samples was presented twice to the panellists, as to provide a brief explanation on the sort of variations that might be encountered between stimuli throughout the experiments. 7. RESULTS AND DISCUSSION According to plan, the campaign of listening tests aimed to unearth significant differences between the auralized samples in the cases studied has been conducted. The total number of responses correctly identifying the uneven stimulus has been counted, and the collected data regarding each experiment has been summarized in the vertical bar charts shown in Figure 4. For a panel conformed of twenty-seven assessors and at a discretionary significance level of 5%, the minimum number of correct answers required before concluding a significant difference from either experiment is 14 –marked by the horizontal line plotted with dashed trace in the diagrams-. At that level of significance, the amount of right responses must hence exceed this critical value for a positive statement. For the sake of clarity, the analysis of the results for the two experiments presented in this work will be reported separately. 7.1. Measured-BRIRs Auralization versus Simulated-BRIRs Auralization. In view of the data coming from the triangle tests for the first experiment, it can be stated that a significant difference exists between auralizations from field-measured BRIRs and their simulated equivalents in the concert hall under investigation. Only one out of nine possible combinations of receiver placement and musical excerpt does not yield such a conclusion, to wit, piano sample at the seat located behind the stage. Nevertheless, it is also worth pointing out that the difference between the pairs of audio signals under comparison turned out to be less noticeable in auralizations of the piano anechoic recording than in the rest of assessed cases. By contrast, using a percussion sample –i.e. timbalesseems to have made it the easiest for the assessors to tell the difference between auralized files from measured IRs and their simulated counterparts. Looking at acoustic spectrograms of the original reverberation-free musical excerpts to be auralized illustrated in Figure 5, it is of no wonder the striking dependence found between the degree of difficulty encountered by the assessors when making their judgments and the nature of the musical sample that was being administered in this first experiment. Frequency spectra Figure 14. Number of correct identifications of the odd sample collected from the two experiments conducted, classified by receiver location and nature of the input signal. On the left: measured vs. simulated-IR auralization (a). On the right: beforevs. after-refurbish auralization (b). Receiver A-1 A-2 B Number of correct responses 0 5 10 15 20 25 30 Cello Piano Timbals Receiver A-1 A-2 B Number of correct responses 0 5 10 15 20 25 30 Cello Piano Timbals International Seminar on Virtual Acoustics VALENCIA, November 24-25, 2011 159 (FFT) versus time in terms of A-weighed sound pressure level show an irregular energetic distribution in the frequency range of analysis for the anechoic input signals in use. Whereas the piece of piano music goes with fundamental frequencies falling within the 500-hertz-centred octave band, cello and timbales reveal greater amounts of energy within the 250 and 125-hertz octave bands correspondingly. On the other hand, it is well known that some acoustic parameters are closely related to certain subjective features. Likewise, sound strength (G) is a good indicator of the subjective level of sound. So applies to perceived reverberance and early decay time (EDT), perceived clarity and clarity itself (C80), and spatial impression and interaural cross-correlation coefficients (IACC). As a consequence, salient mismatches in the comparison between measured and simulated figures for these descriptors must have been translated into noticeable differences between their subsequent auralizations. In consideration of the exemplary data plotted in Figure 6, corresponding to the values of the above-mentioned acoustic parameters at the receiver labelled as A-1, the most outstanding deviances between measured and simulated ciphers concentrate in the 125 and 250-Hz- centred octave frequency bands, mainly affecting strength (G) and clarity (C80) respectively – counting up to 10 JND and 5 JND in either case-. That is the reason why differences happen to be fairly more perceptible when timbales and cello excerpts, in that order, are administered to the assessors within the experimental sessions. Asked about the nature of the difference, an overwhelming majority of the panellists was inclined towards coloration. As illustrated by the polar plot in Figure 7, the choices of the individuals undergoing the listening tests favoured coloration and reverberance over loudness and spaciousness. Going back to the descriptors whose ciphers have been spectrally depicted in Figure 6, the assessors’ responses to this question correlate fairly well to the findings in the graphs, regarding significant deviations between measured and simulated numerical data in strength (G), early decay time (EDT), and clarity (C80). No noteworthy dependence upon the location of the receiver has been found in the experiment. Figure 15. Frequency spectra (FFT) versus time for A-weighed sound pressure level (SPL) in dBA of the original fragments of anechoic recordings auralized for the experiments. From top to bottom: cello, piano and timbales sample spectrograms. International Seminar on Virtual Acoustics VALENCIA, November 24-25, 2011 160 Figure 16. Measured -in black- and simulated -in blue and red, regarding before and after refurbishdata for some room acoustics parameters at receiver A-1. From top to bottom, and left to right: sound strength (G), early decay time (EDT), clarity (C80), and interaural cross-correlation coefficient (IACC). For the sake of comparison, simulated values from energetic analysis (E) –in contrast to those from IRs (h)- are depicted with dashed trace. 7.2. Simulated-BRIRs Auralization: Before versus After Refurbishment. As for the second of the experiments conducted, whose results are broken down in Figure 4(b), a significant difference between auralizations from impulse responses respectively referring to the conditions before and after the refurbishment of the auditorium cannot be concluded. Just in the scenario of the cello anechoic recording being convoluted with the binaural room impulse response obtained at the backstage seat included in this study, a statement of significant difference would be statistically justified. Surprisingly enough, prominent deviances in terms of spatially-averaged reverberation time (T30) between the states in comparison are manifest in the data plotted in Figure 8. Moreover, such mismatches get to a peak value beyond 5 JND falling at the 250-hertz-centred octave band, which should have yielded an easier recognition, above all, with cello music. Nonetheless, it must also be said that none of the rest of acoustic parameters herein analyzed put on display disparities surpassing 2 JND –with the exception of early decay time (EDT), connected to the subjective discernment of reverberance as wellas shown in Figure 6. On the whole, however, the number of correct responses does not seem to have been linked to either the receiver’s location or the nature of the stimulus administered in this case. To the question on how the odd sample was different, fewer answers were collected from this experiment than the preceding one. Among the assessors who did provide a response, reverberance and loudness were the most popular. But, given the fact that a low percentage of the panellists were truly able to tell the difference between the stimuli being presented for f [Hz] 125 250 500 1000 2000 4000 G [dB] 5 10 15 20 25 Measured Simulated h Simulated E Refurbished h Refurbished E f [Hz] 125 250 500 1000 2000 4000 EDT [s] 0.0 0.5 1.0 1.5 2.0 2.5 3.0 Measured Simulated h Simulated E Refurbished h Refurbished E f [Hz] 125 250 500 1000 2000 4000 C 80 [dB] -10 -8 -6 -4 -2 0 2 4 6 Measured Simulated h Simulated E Refurbished h Refurbished E f [Hz] 125 250 500 1000 2000 4000 IACC 0.0 0.2 0.4 0.6 0.8 1.0 Measured Simulated h Refurbished h International Seminar on Virtual Acoustics VALENCIA, November 24-25, 2011 161 comparison, these answers must have been the result of a guesswork, and are, therefore, inappropriate for further analysis. 8. CONCLUSIONS AND FUTURE WORK It has been proven that there is still a significant difference between auralizations from experimentally recorded binaural room impulse responses and their simulated equivalents obtained by means of a numerically-validated acoustic model of the enclosure. Consequently, slightly different approaches should be attempted in the near future in order to reach room acoustics digital models specifically suited for feasible auralization. To do so, a strategy based on calibrating the geometric model by using figures for the best-known descriptors coming out of post-processed impulse responses –following standard procedures analogous to those considered when handling in-situ registers-, instead of resorting to data from energetic echograms might be advisable. Interesting conclusions associated to the use of stimuli of varied nature can be reached from the listening tests conducted. The highly frequency-dependent behaviour usually demonstrated by most of the conventional room acoustics parameters makes it crucial to carry out specific spectral analysis of those anechoic input signals to be utilized in auralization prior to opting for one of convenience. It has been demonstrated that such a choice does play a fundamental role in disclosing dissimilarities in central acoustic features between auralizations eventually subjected to comparison. Furthermore, considerable divergences found in descriptors correlating with certain subjective aspects of sound have been revealed to be capable of correctly foreseeing latter noticeable differences from eventual panellists’ assessments in this sort of sensory experiments. That is the case for early decay time (EDT) and clarity (C80) -conversely connected to reverberance- and sound strength (G) -in close relation to perceived sound level-. Therefore, analyzing numerical data for some acoustic parameters could help warn of juiceless and unnecessary campaigns of listening tests before being undertaken. However, such a translation from numerical into perceptual differences turns out not to be so straight when it comes to reverberation time (T30) being the only descriptor showing noteworthy variations. Given the fact that the seats removed from the concert hall were still in seemingly sound condition at the time of replacement, the outcome from the second experiment would have made it acoustically pointless. Nevertheless, not only is Acoustics in the architect’s mind Number of responses 0 25 50 75 100 125 150 Coloration Spaciousness Reverberance Loudness Measured vs. Simulated Before vs. After Refurbish Figure 17. Combined polar diagram of panellists’ preferences a propos of the nature of the difference between stimuli compared in the experiments. In blue: measured vs. simulated-IR auralization. In red: beforevs. after-refurbish auralization. f [Hz] 125 250 500 1000 2000 4000 T 30 [s] 0.0 0.5 1.0 1.5 2.0 2.5 3.0 Measured Simulated h Simulated E Refurbished h Refurbished E Figure 18. Spatially-averaged reverberation time (T30) assessed at the three receivers in this work. Measured –in black- and simulated records –distinctively using blue and red colours for before- and after-refurbish statesare shown. Energy (E) and pressure analyses (h) have been also differentiated by dashed plot for the latter.