Database of Gaze, Movement and Communication Behaviour in Free Triadic Conversations
Hinrichs, Paula; Hohmann, Volker; Grimm, Giso
- Publisher
- Zenodo
- Language
- en
Abstract
General Information The data was collected between 16 and 28 January 2025 at the Carl von Ossietzky Universität Oldenburg, Germany. Reference: For more detailed information about the lab setup, see also: Hohmann, V., Paluch, R., Krueger, M., Meis, M. & Grimm, G. (2020). The Virtual Reality Lab: Realization and Application of Virtual Sound Environments. Ear & Hearing, 41(Supplement 1), 31S–38S. doi:10.1097/AUD.0000000000000945 Data and File Overview The data is structured in folders as follows: group/condition/variable.csv Condition Noise Type Noise Level Noise Location warmup babble noise 54 dB SPL(C) diffuse quiet no background noise 39 dB SPL(C) - SSN62 speech-shaped noise 62 dB SPL(C) diffuse SSN78 speech-shaped noise 78 dB SPL(C) diffuse babble62 babble noise 62 dB SPL(C) diffuse babble78 babble noise 77 dB SPL(C) diffuse triad babble noise, triadic conversation 68 dB SPL(C) diffuse / head level ads radio advertisements 67 dB SPL(C) top centre The noise level was measured with a measurement microphone on the table. In the 'triad' condition, the noise level was the combined level of diffuse noise (62 dB SPL) and the concurrent triadic conversation. The virtual speakers were seated in an equilateral triangle which is shifted by 60 degree relatively to the triangle formed by the participants. Overview of variables (see below for details on coordinate system and dimensions): Variable Data Size Sensor HeadPos head translation / m 30001 x 9 optical head tracking HeadRot head rotation / deg 30001 x 9 optical head tracking EOG horizontal and vertical EOG / V 30001 x 6 EOG Sensor Gaze gaze direction, global coordinate system / deg 30001 x 3 derived from HeadRot and EOG GazeLocal gaze direction, local coordinate system / deg 30001 x 3 derived from HeadRot and EOG Levels10 Leq over 10 ms / dB SPL(C) 30001 x 3 head-mounted mic. Levels50 Leq over 50 ms / dB SPL(C) 30001 x 3 head-mounted mic. LevelsTable10 Leq over 10 ms / dB SPL(C) 30001 x 1 table mic. LevelsTable50 Leq over 50 ms / dB SPL(C) 30001 x 1 table mic. VAD voice activity 30001 x 3 derived from Levels50 T sample time / s 30001 x 1 time since begin of measurement Success conversation success (*) 10 x 3 questionnaire (*) In group 4, conditions 'babble62' and 'triad', and in group 5, condition 'babble62', it is not possible to correctly assign the questionnaire data to the individuals due to missing data. Methodological Information This study presents a dataset comprising triadic face-to-face conversations conducted under various noise conditions. Head movements, electrooculography (EOG) and short-term speech levels were recorded, as was self-perceived conversation success, as measured by a questionnaire developed by Nicoras et al. (2022) [4]. Twenty-seven participants with normal hearing, aged 19–29, and fluent in German took part in this study. Participants registered in groups of three friends, with mixed genders. They were seated in an equilateral triangle with a side length of 1.5 metres around a table within a 45-channel, semi-spherical loudspeaker setup. Each participant wore an EOG sensor to measure eye blinks and movements, a head-mounted microphone to measure their speech's sound pressure level, and a head-tracking device to measure translation and rotation of the head. An additional microphone was placed in the centre of the table to measure the overall sound pressure level. Each participant was given a tablet computer to complete the questionnaires on. TASCAR was used to interface with all sensors and data streams, render background noise and log data [2]. The measurements consisted of seven conditions ('quiet', 'SSN62', 'SSN78', 'babble62', 'babble78', 'triad' and 'ads') and one training condition ('warm up'). Each condition was measured once and consisted of a five-minute-long free triadic conversation with a certain background noise. The measurement started with the training condition, after which the remaining conditions were randomised. After each condition, participants were asked to complete a questionnaire developed by Nicoras et al. (2022) [4] to assess their perceived success in the conversation. The babble noise was recorded in the canteen at the Carl von Ossietzky Universität Oldenburg, with comprehensible speech signals removed by Grimm et al. (2019) [5]. The speech-shaped noise has the frequency spectrum of a speech signal and was created from the babble noise recording. The triadic conversation, used in the condition 'triad', was scripted and recorded with background noise in order to create a Lombard effect by Gerken et al. (2020) [6]. In the 'ads' condition, radio advertisements for local businesses from the 'Mein Spot im Radio' website were used [7]. Post processing: The receiving time stamps of EOG data were de-jittered based on hardware sensor time stamps using tascar_dl_dejitter.m from the TASCAR toolbox [1, 2]. All data except the 'Success' were resampled to 100 Hz and time-aligned using tascar_dl_resample.m. Voice activity (VAD) was calculated from 'Levels50' with the function levels2vad.m from the communication behaviour toolbox [3]. For versions of these files see file gitversions. Coordinate system: The global coordinate system is a right-handed coordinate system, i.e., x is pointing to the front, y to the left, and z upwards. The origin was in the centre of the setup on floor level. The three subjects were seated at -120 degree (right), 0 degree (centre) and 120 degree (left), facing a table in the centre of the setup. Therefore their average orientation around the z-axis in global coordinates was 60 degree, 180 degree and -60 degree, respectively. Data and dimensions: HeadPos: head position in global cartesian coordinate system, for each subject x, y, z: x1, y1, z1, x2, y2, z2, x3, y3, z3 HeadRot: head rotation in euler angles in global coordinate system, for each subject Rz, Ry, Rx Rz1, Ry1, Rx1, Rz2, Ry2, Rx2, Rz3, Ry3, Rx3 EOG: for each subject EOG-horizontal Uh, EOG-vertical Uv Uh1, Uv1, Uh2, Uv2, Uh3, Uv3 All variables with 3 columns contain data for the three subjects, one column per subject. Sensors: optical head tracking: infrared based head tracking device: Qualysis Miqus M3, 6 cameras, tracking of marker crowns EOG sensor: TI ADS1115 analog-to-digital converter (res=16 Bit, fs=860 Hz), with ESP32 WiFi microcontroller board head-mounted microphone: AKG C520 table microphone: NTI Audio M2211 questionnaire: Conversation Success Questionnaire based on items by Nicoras et al. (2022) [4], except for question 5, because here triadic conversations were used. Example Scripts Gaze direction as a function of time: load('group2/quiet/T.csv'); load('group2/quiet/GazeLocal.csv'); plot( T, GazeLocal ); xlabel('experiment time / s'); ylabel('gaze direction / deg'); Gaze direction histograms: plot_gaze_histogram.m: This script plots the gaze histograms and median head positions for a given condition and group number. To generate the data plot, type plot_gaze_histogram( 2, 'quiet' ); Speech levels: plot_speech_levels.m: Create box plot of speech levels for all subjects and conditions. This function takes no options. To generate the data plot, type plot_speech_levels(); Noise levels: plot_noise_levels.m: Create a box plot of noise levels for all conditions, i.e., the median level at the table microphone position while none of the subjects was speaking. plot_noise_levels(); References [1] Grimm, G. et al., TASCAR. https://github.com/gisogrimm/tascar [2] Grimm, G., Luberadzka, J., & Hohmann V. (2019). A toolbox for rendering virtual acoustic environments in the context of audiology. Acta Acustica united with Acustica, 105(3), 566-578. https://doi.org/10.3813/AAA.919337 [3] Grimm, G., Communication Behaviour Toolbox, https://github.com/gisogrimm/communication-behaviour-toolbox [4] Nicoras, R., Buck, B., Fischer, R. L., Godfrey, M., Hadley, L. V., Smeds, K., & Naylor, G. (2025). Effective Design for Experiments on Small-Group Conversation: Insights From an Example Study. American Journal of Audiology, 34(2), 305–320. https://doi.org/10.1044/2025_AJA-24-00226 [5] Grimm, G., & Hohmann, V. (2019, December 20). First Order Ambisonics field recordings for use in virtual acoustic environments in the context of audiology. Zenodo. https://doi.org/10.5281/zenodo.3588303 [6] Gerken, M., Hendrikse, M. M. E., Hohmann, V., & Grimm, G. (2020, November 24). German Lombard conversation recordings. Zenodo. https://doi.org/10.5281/zenodo.4160499 [7] reflexmedia GmbH (2025). Mein Spot im Radio. Accessed: 24.04.2025. https://www.meinspotimradio.de/referenzen/
Full text
Da t aba se of Ga ze , M ovement a nd C ommuni ca tion B eh a viour in F ree T ri a di c C onvers a tions G ener a l I nform a tion T he d a t a w a s c olle c ted b etween 16 a nd 28 Ja nu a ry 2025 a t the Ca rl von O ssietzky U niversit ä t O lden b urg , G erm a ny . K eywords : tri a di c c onvers a tion , f ac e - to - f ac e c onvers a tion , free c onvers a tion , he a d movements , ele c tro - o c ulogr a phy ( E O G ), spee c h level , c onvers a tion su cc ess , bac kground noise , babb le noise , spee c h - sh a ped noise F unding : F unded b y the D euts c he F ors c hungsgemeins c h a ft ( DFG , G erm a n R ese a r c h F ound a tion ) – P roje c t - ID 352015383 – SFB 1330 . L i c ense : A ttri b ution - N on C ommer c i a l - S h a re A like 4 . 0 I ntern a tion a l CC BY - NC - SA 4 . 0 R eferen c e : F or more det a iled inform a tion ab out the l ab setup , see a lso : H ohm a nn , V ., Pa lu c h , R ., K rueger , M ., M eis , M . & G rimm , G . ( 2020 ). T he V irtu a l R e a lity Lab : R e a liz a tion a nd A ppli ca tion of V irtu a l S ound E nvironments . Ea r & H e a ring , 41 ( S upplement 1 ), 31 S – 38 S . doi : 10 . 1097 / AUD . 0000000000000945 Da t a a nd F ile O verview T he d a t a is stru c tured in folders a s follows : group/condition/variable.csv C ondition N oise T ype N oise L evel N oise L o ca tion w a rmup babb le noise 54 d B SPL ( C ) diuse quiet no bac kground noise 39 d B SPL ( C ) - SSN 62 spee c h - sh a ped noise 62 d B SPL ( C ) diuse SSN 78 spee c h - sh a ped noise 78 d B SPL ( C ) diuse babb le 62 babb le noise 62 d B SPL ( C ) diuse babb le 78 babb le noise 77 d B SPL ( C ) diuse tri a d babb le noise , tri a di c c onvers a tion 68 d B SPL ( C ) diuse / he a d level a ds r a dio a dvertisements 67 d B SPL ( C ) top c entre
T he noise level w a s me a sured with a me a surement mi c rophone on the t ab le . I n the ' tri a d ' c ondition , the noise level w a s the c om b ined level of diuse noise ( 62 d B SPL ) a nd the c on c urrent tri a di c c onvers a tion . T he virtu a l spe a kers were se a ted in a n equil a ter a l tri a ngle whi c h is shifted b y 60 degree rel a tively to the tri a ngle formed b y the p a rti c ip a nts . O verview of v a ri ab les ( see b elow for det a ils on c oordin a te system a nd dimensions ) : Va ri ab le Da t a S ize S ensor H e a d P os he a d tr a nsl a tion / m 30001 x 9 opti ca l he a d tr ac king H e a d R ot he a d rot a tion / deg 30001 x 9 opti ca l he a d tr ac king E O G horizont a l a nd verti ca l E O G / V 30001 x 6 E O G S ensor Ga ze g a ze dire c tion , glo ba l c oordin a te system / deg 30001 x 3 derived from H e a d R ot a nd E O G Ga ze L o ca lg a ze dire c tion , lo ca l c oordin a te system / deg 30001 x 3 derived from H e a d R ot a nd E O G L evels 10 L eq over 10 ms / d B SPL ( C ) 30001 x 3 he a d - mounted mi c . L evels 50 L eq over 50 ms / d B SPL ( C ) 30001 x 3 he a d - mounted mi c . L evels Tab le 10 L eq over 10 ms / d B SPL ( C ) 30001 x 1 t ab le mi c . L evels Tab le 50 L eq over 50 ms / d B SPL ( C ) 30001 x 1 t ab le mi c . VAD voi c e ac tivity 30001 x 3 derived from L evels 50 T s a mple time / s 30001 x 1 time sin c e b egin of me a surement S u cc ess c onvers a tion su cc ess ( * ) 10 x 3 questionn a ire ( * ) I n group 4 , c onditions ' babb le 62 ' a nd ' tri a d ' , a nd in group 5 , c ondition ' babb le 62 ' , it is not possi b le to c orre c tly a ssign the questionn a ire d a t a to the individu a ls due to missing d a t a . M ethodologi ca l I nform a tion T his study presents a d a t a set c omprising tri a di c f ac e - to - f ac e c onvers a tions c ondu c ted under v a rious noise c onditions . H e a d movements , ele c troo c ulogr a phy ( E O G ) a nd short - term spee c h levels were re c orded , a s w a s self - per c eived c onvers a tion su cc ess , a s me a sured b y a questionn a ire developed b y N i c or a s et a l . ( 2022 ) [ 4 ]. T wenty - seven p a rti c ip a nts with norm a l he a ring , a ged 19 – 29 , a nd uent in G erm a n took p a rt in this study . Pa rti c ip a nts registered in groups of three friends , with mixed genders . T hey were se a ted in a n equil a ter a l tri a ngle with a side length of 1 . 5 metres a round a t ab le within a 45 - c h a nnel , semi - spheri ca l loudspe a ker setup . Eac h p a rti c ip a nt wore a n E O G sensor to me a sure eye b links a nd movements , a he a d - mounted mi c rophone to me a sure their spee c h ' s sound pressure level , a nd a he a d - tr ac king devi c e to me a sure tr a nsl a tion a nd rot a tion of the he a d . A n a ddition a l mi c rophone w a s pl ac ed in the c entre of the t ab le to me a sure the over a ll sound pressure level . Eac h p a rti c ip a nt w a s given a t ab let c omputer to c omplete the questionn a ires on . TASCAR w a s used to interf ac e with a ll sensors a nd d a t a stre a ms , render bac kground noise a nd log d a t a [ 2 ].
T he me a surements c onsisted of seven c onditions ( ' quiet ' , ' SSN 62 ' , ' SSN 78 ' , ' babb le 62 ' , ' babb le 78 ' , ' tri a d ' a nd ' a ds ' ) a nd one tr a ining c ondition ( ' w a rm up ' ). Eac h c ondition w a s me a sured on c e a nd c onsisted of a ve - minute - long free tri a di c c onvers a tion with a c ert a in bac kground noise . T he me a surement st a rted with the tr a ining c ondition , a fter whi c h the rem a ining c onditions were r a ndomised . A fter e ac h c ondition , p a rti c ip a nts were a sked to c omplete a questionn a ire developed b y N i c or a s et a l . ( 2022 ) [ 4 ] to a ssess their per c eived su cc ess in the c onvers a tion . T he babb le noise w a s re c orded in the ca nteen a t the Ca rl von O ssietzky U niversit ä t O lden b urg , with c omprehensi b le spee c h sign a ls removed b y G rimm et a l . ( 2019 ) [ 5 ]. T he spee c h - sh a ped noise h a s the frequen c y spe c trum of a spee c h sign a l a nd w a s c re a ted from the babb le noise re c ording . T he tri a di c c onvers a tion , used in the c ondition ' tri a d ' , w a s s c ripted a nd re c orded with bac kground noise in order to c re a te a L om ba rd ee c t b y G erken et a l . ( 2020 ) [ 6 ]. I n the ' a ds ' c ondition , r a dio a dvertisements for lo ca l b usinesses from the ' M ein S pot im Ra dio ' we b site were used [ 7 ]. P ost pro c essing : T he re c eiving time st a mps of E O G d a t a were de - jittered ba sed on h a rdw a re sensor time st a mps using tascar_dl_dejitter.m from the TASCAR tool b ox [ 1 , 2 ]. A ll d a t a ex c ept the ' S u cc ess ' were res a mpled to 100 H z a nd time - a ligned using tascar_dl_resample.m . V oi c e ac tivity ( VAD ) w a s ca l c ul a ted from ' L evels 50 ' with the fun c tion levels2vad.m from the c ommuni ca tion b eh a viour tool b ox [ 3 ]. F or versions of these les see le gitversions . C oordin a te system : T he glo ba l c oordin a te system is a right - h a nded c oordin a te system , i . e ., x is pointing to the front , y to the left , a nd z upw a rds . T he origin w a s in the c entre of the setup on oor level . T he three su b je c ts were se a ted a t - 120 degree ( right ), 0 degree ( c entre ) a nd 120 degree ( left ), f ac ing a t ab le in the c entre of the setup . T herefore their a ver a ge orient a tion a round the z - a xis in glo ba l c oordin a tes w a s 60 degree , 180 degree a nd - 60 degree , respe c tively . Da t a a nd dimensions : HeadPos : he a d position in glo ba l ca rtesi a n c oordin a te system , for e ac h su b je c t x , y , z : x1, y1, z1, x2, y2, z2, x3, y3, z3 HeadRot : he a d rot a tion in euler a ngles in glo ba l c oordin a te system , for e ac h su b je c t Rz , Ry , Rx Rz1, Ry1, Rx1, Rz2, Ry2, Rx2, Rz3, Ry3, Rx3 EOG : for e ac h su b je c t E O G - horizont a l Uh , E O G - verti ca l Uv Uh1, Uv1, Uh2, Uv2, Uh3, Uv3 A ll v a ri ab les with 3 c olumns c ont a in d a t a for the three su b je c ts , one c olumn per su b je c t . S ensors : optical head tracking : infr a red ba sed he a d tr ac king devi c e : Q u a lysis M iqus M 3 , 6 ca mer a s , tr ac king of m a rker c rowns EOG sensor : TI ADS 1115 a n a log - to - digit a l c onverter ( res = 16 B it , fs = 860 H z ), with ESP 32 W i F i mi c ro c ontroller b o a rd
head-mounted microphone : AKG C 520 table microphone : NTI A udio M 2211 questionnaire : C onvers a tion S u cc ess Q uestionn a ire ba sed on items b y N i c or a s et a l . ( 2022 ) [ 4 ], ex c ept for question 5 , b e ca use here tri a di c c onvers a tions were used . E x a mple Sc ripts Ga ze dire c tion a s a fun c tion of time : load('group2/quiet/T.csv'); load('group2/quiet/GazeLocal.csv'); plot( T, GazeLocal ); xlabel('experiment time / s'); ylabel('gaze direction / deg'); Ga ze dire c tion histogr a ms : plot_gaze_histogram.m : T his s c ript plots the g a ze histogr a ms a nd medi a n he a d positions for a given c ondition a nd group num b er . T o gener a te the following gure , type plot_gaze_histogram( 2, 'quiet' );
S pee c h levels : plot_speech_levels.m : C re a te b ox plot of spee c h levels for a ll su b je c ts a nd c onditions . T his fun c tion t a kes no options . T o gener a te the following gure , type plot_speech_levels(); N oise levels : plot_noise_levels.m : C re a te a b ox plot of noise levels for a ll c onditions , i . e ., the medi a n level a t the t ab le mi c rophone position while none of the su b je c ts w a s spe a king . plot_noise_levels(); R eferen c es [ 1 ] G rimm , G . et a l ., TASCAR . https : // githu b . c om / gisogrimm / t a s ca r [ 2 ] G rimm , G ., L u b er a dzk a , J ., & H ohm a nn V . ( 2019 ). A tool b ox for rendering virtu a l ac ousti c environments in the c ontext of a udiology . Ac t a Ac usti ca united with Ac usti ca , 105 ( 3 ), 566 - 578 . https : // doi . org / 10 . 3813 / AAA . 919337 [ 3 ] G rimm , G ., C ommuni ca tion B eh a viour T ool b ox , https : // githu b . c om / gisogrimm / c ommuni ca tion - b eh a viour - tool b ox
[ 4 ] N i c or a s , R ., B u c k , B ., F is c her , R . L ., G odfrey , M ., Ha dley , L . V ., S meds , K ., & Na ylor , G . ( 2025 ). E e c tive D esign for E xperiments on S m a ll - G roup C onvers a tion : I nsights F rom a n E x a mple S tudy . A meri ca n J ourn a l of A udiology , 34 ( 2 ), 305 – 320 . https : // doi . org / 10 . 1044 / 2025 _ AJA - 24 - 00226 [ 5 ] G rimm , G ., & H ohm a nn , V . ( 2019 , D e c em b er 20 ). F irst O rder A m b isoni c s eld re c ordings for use in virtu a l ac ousti c environments in the c ontext of a udiology . Z enodo . https : // doi . org / 10 . 5281 / zenodo . 3588303 [ 6 ] G erken , M ., H endrikse , M . M . E ., H ohm a nn , V ., & G rimm , G . ( 2020 , N ovem b er 24 ). G erm a n L om ba rd c onvers a tion re c ordings . Z enodo . https : // doi . org / 10 . 5281 / zenodo . 4160499 [ 7 ] reexmedi a G m bH ( 2025 ). M ein S pot im Ra dio . Acc essed : 24 . 04 . 2025 . https : // www . meinspotimr a dio . de / referenzen /
