Finze, Nikola; Jechle, Deine a; Fauße , S e an; Gewald, Heiko
A icle — Published Ve sion
How a e We Doing Today? Using Na u al Speech Analysis
o Assess Olde Adul s’ Subjec i e Well-Being
Business & In o ma ion Sys ems Enginee ing
P o ided in Coope a ion wi h:
Sp inge Na u e
Sugges ed Ci a ion: Finze, Nikola; Jechle, Deine a; Fauße , S e an; Gewald, Heiko (2024) : How a e We
Doing Today? Using Na u al Speech Analysis o Assess Olde Adul s’ Subjec i e Well-Being, Business
& In o ma ion Sys ems Enginee ing, ISSN 1867-0202, Sp inge Fachmedien Wiesbaden GmbH,
Wiesbaden, Vol. 66, Iss. 3, pp. 321-334,
h ps://doi.o g/10.1007/s12599-024-00877-4
This Ve sion is a ailable a :
h ps://hdl.handle.ne /10419/315765
S anda d-Nu zungsbedingungen:
Die Dokumen e au EconS o dü en zu eigenen wissenscha lichen
Zwecken und zum P i a geb auch gespeiche und kopie we den.
Sie dü en die Dokumen e nich ü ö en liche ode komme zielle
Zwecke e iel äl igen, ö en lich auss ellen, ö en lich zugänglich
machen, e eiben ode ande wei ig nu zen.
So e n die Ve asse die Dokumen e un e Open-Con en -Lizenzen
(insbesonde e CC-Lizenzen) zu Ve ügung ges ell haben soll en,
gel en abweichend on diesen Nu zungsbedingungen die in de do
genann en Lizenz gewäh en Nu zungs ech e.
Te ms o use:
Documen s in EconS o may be sa ed and copied o you pe sonal
and schola ly pu poses.
You a e no o copy documen s o public o comme cial pu poses, o
exhibi he documen s publicly, o make hem publicly a ailable on he
in e ne , o o dis ibu e o o he wise use he documen s in public.
I he documen s ha e been made a ailable unde an Open Con en
Licence (especially C ea i e Commons Licences), you may exe cise
u he usage igh s as speci ied in he indica ed licence.
h p://c ea i ecommons.o g/licenses/by/4.0/
RESEARCH PAPER
How a e We Doing Today? Using Na u al Speech Analysis
o Assess Olde Adul s’ Subjec i e Well-Being
Nikola Finze •Deine a Jechle •S e an Fauße •Heiko Gewald
Recei ed: 23 June 2023 / Accep ed: 17 Ap il 2024 / Published online: 8 June 2024
ÓThe Au ho (s) 2024
Abs ac The esea ch p esen s he de elopmen and es
o a machine lea ning (ML) model o assess he subjec i e
well-being o olde adul s based solely on na u al speech.
The use o such echnologies can ha e a posi i e impac on
heal hca e deli e y: he p oposed ML model is pa ien -
cen ic and secu ely uses use -gene a ed da a o p o ide
sus ainable alue no only in he heal hca e con ex bu also
o add ess he global challenge o demog aphic change,
especially wi h espec o heal hy aging. The de eloped
model unob usi ely analyzes he ocal cha ac e is ics o
olde adul s by u ilizing na u al language p ocessing bu
wi hou using speech ecogni ion capabili ies and adhe ing
o he highes p i acy s anda ds. I is based on heo ies o
subjec i e well-being, acous ic phone ics, and p osodic
heo ies. The ML models we e ained wi h oice da a om
olun ee pa icipan s and calib a ed h ough he Wo ld
Heal h O ganiza ion Quali y o Li e Ques ionnai e
(WHOQOL), a widely accep ed ool o assessing he
subjec i e well-being o human beings. Using WHOQOL
sco es as a p oxy, he de eloped model p o ides accu a e
nume ical es ima es o indi iduals’ subjec i e well-being.
Di e en models we e es ed and compa ed. The
eg ession model p o es bene icial o de ec ing unex-
pec ed shi s in subjec i e well-being, whe eas he suppo
ec o eg ession model pe o med bes and achie ed a
mean absolu e e o o 10.90 wi h a s anda d de ia ion o
2.17. The esul s enhance he unde s anding o he sub-
conscious in o ma ion con eyed h ough na u al speech.
This o e s mul iple applica ions in heal hca e and aging,
as well as new ways o collec , analyze, and in e p e sel -
epo ed use da a. P ac i ione s can use hese insigh s o
de elop a weal h o inno a i e p oduc s and se ices o
help senio s main ain hei independence longe , and
physicians can gain much g ea e insigh in o changes in
hei pa ien s’ subjec i e well-being.
Keywo ds A i icial in elligence Machine lea ning
Na u al language p ocessing Olde adul Subjec i e
well-being assessmen WHOQOL-OLD, WHOQOL-
BREF Voice analysis
1 In oduc ion
Taking ca e o he aging gene a ion is a socie al ask ha
pu s na ions a ound he wo ld unde p essu e. Olde adul s
(as pe he WHO de ined as people 60 ?yea s o age) who
p o usely su e om ch onic diseases and declining heal h
in gene al need ca e and a en ion, which collides wi h he
end owa ds smalle and geog aphically dispe sed ami-
lies, speci ically in de eloped economies. The esul ing
si ua ion is ala ming: in Ge many, a ound one- hi d o all
pe sons o e 65 li e alone, and his le el is expec ed no o
dec ease (S a is isches Bundesam 2023). Simul aneously,
he numbe o p o essional ca e ake s is con inuously
declining (Flake e al. 2018). This is a wo ldwide
Accep ed a e h ee e isions by he edi o s o he Special Issue.
N. Finze (&)D. Jechle S. Fauße H. Gewald
Cen e o Resea ch on Se ice Sciences (CROSS), Facul y o
In o ma ion Managemen , Neu-Ulm Uni e si y o Applied
Sciences, Wileys . 1, 89231 Neu-Ulm, Ge many
e-mail: [email p o ec ed]
D. Jechle
e-mail: [email p o ec ed]
S. Fauße
e-mail: [email p o ec ed]
H. Gewald
e-mail: [email p o ec ed]
123
Bus In Sys Eng 66(3):321–334 (2024)
h ps://doi.o g/10.1007/s12599-024-00877-4
phenomenon ha is no es ic ed o Ge many. As he
global popula ion ages, he subjec i e well-being o olde
adul s has gained subs an ial a en ion om heal hca e
p o ide s, policymake s, and socie y (Uni ed Na ions 2019;
Na ional Ins i u e on Aging 2021). This global challenge is
unde sco ed by he Uni ed Na ions (UN) commi men o
he UN Decade o Heal hy Aging objec i es, a global ini-
ia i e o p omo e he heal h and quali y o li e o olde
indi iduals wo ldwide, inaugu a ed in 2021 (Wo ld Heal h
O ganiza ion 2023). An indi idual’s subjec i e well-being
is a complex mul idimensional cons uc encompassing
physical and men al heal h, social suppo , and li e sa is-
ac ion (Diene 1984; Cooke e al. 2016).
Fo mos senio s, a key well-being ac o is o li e a sel -
sus ained li e in hei amilia en i onmen o as long as
possible (Die x 2019). A challenge o his demand is he
gene al decline o physical and men al abili ies as a na u al
consequence o aging (Gae ne e al. 2023). Nega i e
de elopmen s migh go unno iced i he adi ional weekly
isi o phone call – including he s anda d ques ion ‘‘How
a e you doing oday?’’ – is he only ex e nal measu e o an
indi idual’s well-being. Indeed, assessing a pe son’s ‘ eal’
well-being is di icul as his cons uc is en i ely subjec-
i e. To add ess his issue, psychologis s ha e de eloped
and es ed nume ous measu emen s based on s uc u ed
ques ionnai es (Cooke e al. 2016). The Wo ld Heal h
O ganiza ion (WHO) made a g ea e o o de elop an
in e na ionally alida ed ins umen o measu e indi idu-
als’ pe cei ed quali y o li e, he WHOQOL (The WHO-
QOL G oup 1998b). I was a join de elopmen o 15
in e na ional esea ch cen e s, es ed wi h mo e han 4,500
pa icipan s, and ansla ed in o mo e han 70 languages. I
is gene ally conside ed a alid ins umen p oducing ade-
qua e esul s (The WHOQOL G oup 1998a; Cooke e al.
2016). The WHO also de eloped a specialized ins umen
o assess he pe cep ions o senio ci izens, he WHOQOL-
OLD (Powe e al. 2005), which supplemen s he gene al
WHOQOL.
Howe e , al hough he WHOQOL ins umen s p o ide a
comp ehensi e and dependable assessmen o an indi id-
ual’s subjec i e well-being, hey a e no sui able o e-
quen use due o he la ge numbe o ques ions (100) he
pa icipan s need o answe . Thus, i his assessmen could
be done au oma ically, unob usi ely, and con inuously,
ela i es and p o essional ca egi e s would be able o eac
mo e imely o p e en ad e se ou comes in case o
declines in subjec i e well-being (A ola e al. 2021).
Addi ionally, p o essional ca egi e s a e becoming sca ce
(Ribei o e al. 2021; Flake e al. 2018), and he op imal
deploymen o hese skilled wo ke s is becoming inc eas-
ingly impo an o na ional heal hca e sys ems. The
inc easing numbe o olde people in combina ion wi h he
dec ease in ca egi e s calls o echnological solu ions o
ease he nega i e impac s o his inc easing imbalance
(Ma inho e al. 2019; Czaja and Ce uso 2022).
One way o add ess he a o emen ioned challenges
could be he con inuous au oma ed assessmen o senio s’
subjec i e well-being h ough na u al speech analysis.
Technological ad ancemen s in machine lea ning (ML)
and na u al language p ocessing (NLP) ha e demons a ed
p omising po en ial in de i ing subconscious in o ma ion
om he human oice (Zunic e al. 2020). In he heal hca e
con ex , o example, NLP has been used o analyze pa ien
eedback, gauge sen imen , and e en de ec ea ly signs o
diseases om pa ien s’ speech pa e ns and use o language
(DeSouza e al. 2021; Khanbhai e al. 2021; Bel ami e al.
2018; Pe ez e al. 2018). Fu he mo e, wi hin linguis ic
esea ch, acous ic phone ics heo y, which ocuses on he
sounds o speech (Ve e idis and Ko opoulos 2006), and
p osody heo y, which ocuses on elemen s such as pi ch,
du a ion, and in ensi y o speech (Hubba d e al. 2017;
Ba nes and Sha uck-Hu nagel 2022; Ladd 2008), a e
c i ical o he exp ession o emo ions (Co ales-As o gano
e al. 2019), and he iden i ica ion o an indi idual’s
emo ional s a e (Bha an e al. 2019; Rusz e al. 2011;
Godino-Llo en e and Gomez-Vilda 2004).
Howe e , his esea ch s eam has p ima ily ocused on
speci ic aspec s o emo ions, such as iden i ying dep ession
(Rejaibi e al. 2022; Lin e al. 2020; An e al. 2019), ea ly
signs o diseases om pa ien s’ speech pa e ns (DeSouza
e al. 2021; Khanbhai e al. 2021; Bel ami e al. 2018;
Pe ez e al. 2018), o challenges in emo ion ecogni ion
among olde adul s (Schulle e al. 2020), wi hou ully
ha nessing he po en ial o hose echnologies o he
au oma ed assessmen and moni o ing o subjec i e well-
being. By u ilizing esea ch on NLP analy ic capabili ies,
we aim o add ess he global challenge o heal hy aging by
assessing subjec i e well-being h ough speech analysis.
The e o e, we o mula e he ollowing esea ch ques ion:
How can olde adul ’s subjec i e well-being be assessed
om na u al speech?
Ou wo k con ibu es o esea ch on ad ances in he
au oma ic assessmen o subjec i e well-being h ough
ML. We p esen an inno a i e, pa ien -cen e ed model ha
uses NLP o enable da a-d i en ca e and suppo o he
aging gene a ion. The model is a no el way o assess a
human’s subjec i e well-being unob usi ely, con inu-
ously, and au oma ically and could p o ide amily and
p o essional ca e ake s wi h imely and comp ehensi e
o e iews o a senio ’s subjec i e well-being. I could
igge in e en ions as needed, hus suppo ing he op imal
u iliza ion o he sca ce esou ce o p o essional ca e ake s.
I would also p o ide ease-o -mind o amily membe s i
equen in e ac ion wi h he senio is no possible, o
ins ance, due o geog aphic dis ance.
123
322 N. Finze e al.: How a e We Doing Today?, Bus In Sys Eng 66(3):321–334 (2024)
The ML model de eloped o answe he esea ch ques-
ion is heo e ically oo ed in subjec i e well-being heo y,
as well as heo ies o p osody and acous ic phone ics. This
ounda ion is combined wi h NLP echniques, including
ea u e ex ac ion om speech and eg ession models.
Wo kshops wi h s akeholde s ha e been conduc ed o
de i e equi emen s o he echnical solu ion and ensu e
he use ulness o he p ac ical con ibu ion. The aining
da ase o he model was c ea ed by eco ding he oices
o Ge man-speaking senio ci izens while hey comple ed
he WHOQOL ques ionnai es. The ML aining o he
pa icipan s’ oices was calib a ed wi h he esul s o hei
WHOQOL assessmen .
In he ollowing, his pape elabo a es on he heo e ical
basis o his esea ch, p o ides an o e iew o he cu en
s a e o echnological de elopmen , and desc ibes he da a
collec ion p ocedu es as well as he me hodological
app oach o he ML algo i hm. Based on he e alua ion o
he esul s, he pe o mance o he ML model is discussed,
and he implica ions o heo y and p ac ice a e p esen ed.
The pape closes by explica ing i s limi a ions, p o iding
a enues o u u e esea ch, and i s conclusion.
2 Theo e ical Backg ound and Rela ed Wo k
Ou esea ch is posi ioned a he in e sec ion o subjec i e
well-being measu emen and he compu a ional analysis o
emo ion ecogni ion om na u al speech. We add ess ou
esea ch ques ion based on h ee heo e ical ounda ions:
subjec i e well-being, p osody, and acous ic phone ics.
The in eg a ion o hese heo ies poses ha he pe cei ed
human well-being is closely connec ed o a pe son’s eel-
ings and emo ions. These a e subconsciously ansmi ed
h ough he human oice, which allows NLP me hods o
de ec and quan i y hem (Lin e al. 2020). This sec ion
discusses he heo e ical ounda ions and p o ides an
o e iew o ela ed esea ch.
2.1 Subjec i e Well-being
His o ically, well-being was de ined as he absence o
disease and disabili y (Cooke e al. 2016). Howe e , in
1948, he WHO s a ed, ‘‘heal h is a s a e o comple e
physical, men al, and social well-being and no me ely he
absence o disease and in i mi y’’ (Wo ld Heal h O gani-
za ion 2020). Since hen, esea ch on subjec i e well-being
has ecei ed conside able a en ion, pa icula ly in psy-
chology, and is now mo ing owa d a mul idimensional
concep ha encompasses an indi idual’s op imal unc-
ioning and sa is ac ion in a ious aspec s o li e (The
WHOQOL G oup 1998b). Among he di e en app oaches
o assess subjec i e well-being, hedonic app oaches
emphasize pleasu e and happiness, wi h subjec i e well-
being as a p ominen model consis ing o li e sa is ac ion,
absence o nega i e a ec , and p esence o posi i e a ec
(Cooke e al. 2016). Fu he mo e, eudaimonic app oaches
o well-being p opose ha psychological heal h is achie ed
‘‘by ul illing one’s po en ial, unc ioning a an op imal
le el’’ (Cooke e al. 2016), which includes li ing in a way
ha ul ills one’s po en ial and leads o pe sonal g ow h
and de elopmen (Len 2004).
While in li e a u e, he e ms quali y o li e and sub-
jec i e well-being a e o en used in e changeably, quali y
o li e esea che s concep ualize subjec i e well-being
mo e b oadly o include hedonic (p esence o pleasu e,
absence o pain) and eudaimonic pe spec i es (li ing a
meaning ul and pu pose ul li e) and a e based on physical,
psychological, and social aspec s (Vik and Ca lquis 2018;
Cooke e al. 2016). Following hese a gumen s, we b oadly
concep ualized subjec i e well-being o his s udy o
include physical, men al, and social dimensions (Diene
e al. 2009; Cooke e al. 2016).
As subjec i e well-being s ongly in luences physical
and men al heal h, esilience, and many mo e essen ial
aspec s o human li e (Ry 2014; Cooke e al. 2016;Yıl-
dı ım and C¸ elik Tan ı e di 2020), he WHO pu g ea e o
in o de eloping a s anda dized measu e o i (Huppe and
So 2013; Keyes 2005). In 1998, he WHO p esen ed he
quali y o li e ins umen s (WHOQOL) de eloped in an
in e na ional s udy p og am, including ocus g oups wi h
pa ien s and medical expe s wo ldwide (Wo ld Heal h
O ganiza ion 2012). The WHOQOL has he ad an age ha
i was de eloped c oss-cul u ally and, he e o e, can be
used in in e na ional s udies wi hou addi ional adap a ions.
This opens a enues o u he esea ch, which can enhance
he gene alizabili y o he indings o o he languages and
cul u al con ex s (Cooke e al. 2016). I is widely ega ded
as a high-quali y pa ien -cen e ed ool, success ully
assessing indi iduals’ subjec i e well-being in di e en
esea ch a eas (Ske ing on and McC a e 2012).
The ins umen comp ises ou domains: physical, psy-
chological, social ela ionships, and en i onmen . The
physical domain ocuses on pain and ene gy, he psycho-
logical domain on sel -es eem, he social ela ionships on
pe sonal ela ionships, and he en i onmen al domain on
sa e y and inancial esou ces. In his way, he WHOQOL
ins umen ecognizes ha well-being is in luenced by
mul iple in e connec ed ac o s and shaped by he collec-
i e impac o a ious dimensions (The WHOQOL G oup
1998b; Wo ld Heal h O ganiza ion 2012).
The WHOQOL is supplemen ed by a se o speci ic
ins umen s like he WHOQOL-HIV, which is calib a ed
o people in ec ed wi h HIV, o he WHOQOL-OLD,
which is con ex -speci ic o he subjec i e well-being o
people o e 60 yea s o age (Cen e s o Disease Con ol
123
N. Finze e al.: How a e We Doing Today?, Bus In Sys Eng 66(3):321–334 (2024) 323
P e en ion 2012; Wo ld Heal h O ganiza ion 2012). The
WHOQOL-OLD e lec s he unique si ua ion and chal-
lenges he olde popula ion aces (Diene 1984). The
ques ionnai e comp ises 24 ques ions ac oss six dimen-
sions: senso y abili ies, au onomy, pas , p esen , and u u e,
social pa icipa ion, dea h, and in imacy (Powe e al.
2005). I is conside ed a obus and eliable ins umen used
in a ious coun ies and con ex s, making i an essen ial
ool in ge on ological esea ch (Lucas-Ca asco 2012;
Chachamo ich e al. 2008; Con ad e al. 2014).
The WHOQOL is comp ised o 100 ques ions, neces-
si a ing conside able ime o hough ul and accu a e
esponses. Consequen ly, an abb e ia ed e sion known as
he WHOQOL-BREF was de eloped o add ess his issue.
I is educed o 26 ques ions and gene a es esul s wi h
eliabili y compa able o he longe WHOQOL (The
WHOQOL G oup 1998a). To unde s and olde adul s’
well-being, bo h ins umen s (WHOQOL-BREF and
WHOQOL-OLD) mus be combined (i.e., he pa icipan s
mus answe bo h ques ionnai es). To enhance eadabili y,
we abb e ia e he combined use o bo h ins umen s as
‘QOLs’ in he emainde o his pape .
2.2 T ansmission o Human Feelings and Emo ions
ia Na u al Speech
The human oice exp esses bo h conscious and subcon-
scious in o ma ion like eelings and emo ions (Li e al.
2019). The e o e, he abili y o analyze he human oice o
de i e he speake ’s subconscious eelings can be a solu-
ion o assess an indi idual’s subjec i e well-being in an
unin usi e way h ough an au oma ed ML algo i hm.
Acous ic phone ics and p osody p o ide he ounda ion o
ex ac ing he ele an ea u es om speech da a ia NLP
me hods.
2.2.1 Acous ic Phone ic Theo y
Acous ic phone ics is he s udy o he physical p ope ies o
speech sounds, including hei p oduc ion, ansmission,
and pe cep ion, apa om he ac ual wo ds spoken
(Lade oged and Johnson 2014). I ocuses on how speech
sounds a e o med as ai is o ced ou o he lungs and
examines phonemes’ a icula o y and audi o y cha ac e -
is ics, ocusing on o man s, pi ch, in ensi y, spec al
cha ac e is ics, and du a ion (Ve e idis and Ko opoulos
2006). Acco ding o p e ious esea ch, a ia ions in hese
ea u es indica e changes in a pe son’s emo ional s a e,
heal h condi ion, o well-being (Godino-Llo en e and
Gomez-Vilda 2004; Rusz e al. 2011). Fo ins ance, s ess
o emo ional dis ess o en mani es s as changes in he
pi ch, in ensi y, o hy hm o speech (Reine 2013).
Resea ch has also demons a ed he ole o acous ic
phone ics in iden i ying emo ional s a es such as sadness,
ange , happiness, and ea (Bha an e al. 2019; Ta iq e al.
2019).
2.2.2 P osodic Theo y
The p osodic heo y ocuses on cen al aspec s o speech,
including s ess, in ona ion, hy hm, and ph asing (Ladd
2008; Ba nes and Sha uck-Hu nagel 2022). I explo es
how pi ch, du a ion, and in ensi y a ia ions shape speech’s
melody, hy hm, and emphasis, con eying linguis ic and
a ec i e in o ma ion (Hubba d e al. 2017). P osody is
c ucial in con eying meaning, exp essing emo ions, and
s uc u ing discou se (Co ales-As o gano e al. 2019). Fo
ins ance, a change in in ona ion can u n a s a emen in o a
ques ion. By s udying p osodic ea u es, esea che s gain
insigh s in o speech’s exp essi e and p agma ic aspec s
(Weed and Fusa oli 2020).
Gi en he in ica e na u e o speech and i s emo ional
con en , app oaches ha inco po a e bo h acous ic phone ic
and p osodic ea u es o e a mo e comp ehensi e and
accu a e analysis. This mul imodal app oach, which com-
bines he physical p ope ies o speech sounds (acous ic
ea u es) and he hy hm, s ess, and in ona ion o speech
(p osodic ea u es), is conside ed mo e e ec i e and close
o he na u al complexi y o human speech (Byun e al.
2021). A mul imodal app oach allows o a nuanced
analysis o speech, cap u ing bo h he linguis ic and
a ec i e in o ma ion con eyed in speech, he eby p o id-
ing a mo e ealis ic ep esen a ion o human emo ional
exp ession (Schulle e al. 2020; An e al. 2019).
2.3 Applying NLP o Assess Feelings and Emo ions
om Na u al Speech
The use o NLP o de ec human well-being is a compa -
a i ely new a ea o esea ch. This new a ea p esen s sig-
ni ican in e disciplina y challenges, combining he
complex heo ies o emo ion exp ession in human speech
(i.e., acous ic phone ics and p osody) wi h ad anced
compu a ional echniques. While he li e a u e has no ed
con e sa ional agen s and cha bo s (e.g., Ahmad e al.
2022) o assessing subjec i e well-being, human oice
analysis o e s a pa icula ly in iguing a enue o esea ch.
Published esea ch ocuses on de ec ing eelings, emo ions,
and men al heal h issues h ough oice analysis, using
a ious app oaches o cap u e he complex signals con-
eyed by he speake ’s oice. While hese s udies di e in
hei speci ic esea ch goals, hey collec i ely con ibu e o
a b oade unde s anding o how subjec i e well-being can
be measu ed non-in asi ely h ough na u al speech. Th ee
main esea ch s eams ha e eme ged: dep ession de ec ion,
123
324 N. Finze e al.: How a e We Doing Today?, Bus In Sys Eng 66(3):321–334 (2024)
speech emo ion ecogni ion, and subjec i e well-being
measu emen om speech.
2.3.1 Dep ession De ec ion
NLP echniques a e used in dep ession de ec ion due o he
measu able di e ences in speech signals o dep essed
pa ien s, as well as he non-in usi e, emo e, and low-cos
na u e o speech de ec ion (Rejaibi e al. 2022; Lin e al.
2020). Ea ly s udies ha e ocused on elemen s such as
p osody and spec al analysis, ypically pai ed wi h con-
en ional ML echniques (Wu e al. 2023). Sanchez e al.
(2011) used p osodic and spec al speech ea u es o
iden i y di e ences be ween heal hy and dep essed indi-
iduals using a suppo ec o machine (SVM), yielding a
p edic ion accu acy o o e 81%. E en wi h ad anced
echniques, SVMs emained e ec i e in modeling and
classi ying acous ic signals ha indica e dep ession in eal-
ime wi h an accu acy o 90% (Yalamanchili e al. 2020).
Howe e , mo e complex ML algo i hms, speci ically deep
lea ning, a e now used o cap u e mo e nuanced ea u es in
speech (Wu e al. 2023). Lin e al. (2020) in eg a ed a
bidi ec ional LSTM and a one-dimensional con olu ional
neu al ne wo k (1D-CNN) o cap u e empo al and spec al
ea u es om speech da a, leading o g ea e accu acy in
de ec ing dep ession. Thei p oposed model achie ed up o
84% accu acy by employing pu ely acous ic ea u es. In a
simila ein, long sho - e m memo y (LSTM) models ha
u ilized mel- equency ceps um coe icien (MFCC) ea-
u es we e able o ecognize pa e ns indica i e o di e en
le els o dep ession o e ime wi h app oxima ely 76%
accu acy (Rejaibi e al. 2022).
2.3.2 Emo ion De ec ion
Emo ion de ec ion om speech, also e e ed o as speech
emo ion ecogni ion (SER), ocuses on ep esen ing emo-
ions using nume ical alue ea u e se s ex ac ed om
speech da a (Pen a i e al. 2024). ML algo i hms, like
SVMs, Decision T ees, o Random Fo es , ha e been used
o de ec emo ional s a es om na u al speech based on
analysis o he ocal ea u es (Su esh e al. 2023). While
ea u e se s yield good ou comes, nuances in speech o en
lack su icien ep esen a ion in such se s (Wang e al.
2015; Pen a i e al. 2024). Recen ad ances in SER ha e
shown he powe o deep lea ning algo i hms o de ec ing
emo ions om speech. Simila ly, Ta iq e al. (2019)
de eloped a 2D-CNN o de ec se en basic emo ions (calm,
happy, sad, ang y, ea ul, disgus , and su p ise) in he
speech o elde ly pa ien s, wi h an o e all accu acy o up o
95%. In ela ed esea ch, mul imodal app oaches using
cascaded LSTM ecu en neu al ne wo ks (RNN) we e
used by Gup a e al. (2022) o de ec s essed and
uns essed s a es o es subjec s wi h an accu acy o 91%.
Schulle e al. (2020) demons a ed he e ec i eness o
employing acous ic and p osodic ea u es in a CNN and
LSTM RNN app oach o de ec a ousal and alence om
olde adul s’ speech samples wi h 72% accu acy.
Resea ch on emo ion de ec ion om speech is s ill
compa a i ely young and he p oposed ea u e se s using
s a is ical and s uc u al in o ma ion ex ac ed om speech
signals o de ec emo ional s a es con inuously g ows
(Pen a i e al. 2024).
2.3.3 Measu emen o Subjec i e Well-Being
T adi ionally, ques ionnai es a e used o assess a pe son’s
subjec i e well-being. Ad ances in ML/NLP ha e led o
inc eased esea ch ac i i y on he au oma ic de ec ion o
subjec i e well-being om speech. Howe e , he numbe
o published esea ch pape s combining complex speech-
emo ion heo ies wi h compu a ional echniques is com-
pa a i ely small.
Kim e al. (2019) epo on an NLP me hod o assess
human subjec i e well-being om na u al language by
measu ing h ee dis inc cons uc s: anxie y, mood, and
sleep quali y. Thei algo i hm uses a 41-dimensional
supe ec o o a ious ea u es such as MFCCs, pe cep ual
linea p edic ion (PLP), p osody, and oice quali y- ela ed
ea u es. I shows a p edic i e capaci y o 41% o anxie y,
44% o sleep quali y, and 38% o mood. Nakagawa e al.
(2020) p oposed a p omising app oach based on a 3D-CNN
o es ima e he quali y o li e o a speake wi h an accu acy
o 71%. Un o una ely, no mo e p o ound elabo a ion on
he measu emen i ems is p o ided in he a icle, and no
pee - e iewed ollow-up wo k could be loca ed.
3 Requi emen s o he Technical Solu ion
Paying a en ion o he subjec i e well-being o olde
people, especially hose who li e alone, is essen ial o
enable hem o li e independen ly longe . While i is na -
u al o e e yone o ha e good days and bad days, close
ela i es and designa ed ca egi e s o senio ci izens need
o no ice a s eady o sudden decline in subjec i e well-
being o p o ide he necessa y assis ance. T adi ionally, he
well-being o ano he pe son is assessed h ough ace- o-
ace con e sa ions. In oday’s socie ies, howe e , people
gene ally mee wi h each o he less han hey would like.
Technological solu ions can ale amily membe s and
ca egi e s when a p oblem a ises and igge an
in e en ion.
To p o ide a sus ainable and p ac ical impac solu ion, i
mus mee se e al equi emen s de i ed h ough wo kshops
wi h senio s, p o essional ca egi e s, and gene al
123
N. Finze e al.: How a e We Doing Today?, Bus In Sys Eng 66(3):321–334 (2024) 325
p ac i ione s who ea elde ly pa ien s. These equi emen s
wo kshops we e conduc ed in a ee discussion mode a ed
by he p ojec eam, wi h no es ic ions o bounda ies on
he inal solu ion. A e collec ing hese basic equi emen s,
di e en echnological op ions we e discussed o ul ill he
s a ed demands. I was concluded ha an ML model based
on oice analysis would be he mos p omising app oach—
acco dingly, some co esponding equi emen s needed o
be o mula ed o his speci ic echnology by he p ojec
eam. Table 1p o ides an o e iew o he equi emen s
based on he di e en pe spec i es. These equi emen s
guided he de elopmen o his esea ch’s ML model. A
mo e p o ound elabo a ion on conc e e design p inciples
and he sys em enginee ing pa o he esea ch is no he
subjec o his publica ion.
4 S udy Design and Da a Collec ion
No da a se o add ess he esea ch ques ion is cu en ly
publicly a ailable. The e o e, a new da a se had o be
c ea ed. Senio ci izens we e asked o pa icipa e in a da a
collec ion exe cise o collec he necessa y aining da a.
The asks we e o comple e he wo QOL ques ionnai es
(WHOQOL-BREF and WHOQOL-OLD) and pa icipa e in
a eco ded in e iew. The inclusion c i e ia we e: (1) o e
60 yea s o age; (2) being able o li e independen ly; (3) no
men al/cogni i e challenges.
Be o e s a ing he in e iews, he s udy was explained,
ques ions we e answe ed, and he pa icipan signed he
da a p i acy ag eemen . The in e iews ook be ween 20
and 86 min (Ø = 42). The in e iewe s asked some wa m-
up ques ions abou he in e iewees’ li es in gene al and
hei cu en subjec i e well-being, ollowed by he p e-
de ined ques ions o he QOLs. The audio eco dings we e
cu and labeled wi h he co esponding QOL sco es cal-
cula ed om he espec i e QOL ques ionnai es.
Da a collec ion was conduc ed om Janua y o June
2023. Volun ee s we e ec ui ed om local senio ci cles,
ins i u ions whe e senio s mee weekly o unde ake cha i y
p ojec s, alk o each o he , lis en o lec u es, and play
games. 32 in e iewees, comp ising 20 emales and 12
males be ween 60 and 88, pa icipa ed in his s udy.
Acco ding o he applicable e hics commission’s egula-
ions, no app o al was necessa y as he isk assessmen o
his s udy indica ed no isk o downside po en ial o he
pa icipan s.
5 Model De elopmen
This esea ch discusses he de elopmen o an ML model
o de ec olde adul s’ subjec i e well-being -calib a ed o
he QOLs- om na u al language. The model analyzes an
audio ile o ee speech om an indi idual and p edic s he
co esponding pe cei ed well-being on a scale om 0 o
100 as pe he QOLs. This sec ion p o ides de ails
ega ding da a labeling and ea u e ex ac ion.
5.1 Da a Labeling
The in e iewees’ esponses o he QOLs we e used o label
he da a. Each pa icipan answe ed he p ede ined ques ions
ac oss he six domains. The answe s wi hin each domain
we e sco ed and agg ega ed in o an o e all domain alue
be ween 0 and 100, acco ding o he WHOQOL manual,
whe eas highe sco es indica e be e well-being (Con ad
e al. 2016). In he nex s ep, he domain esul s we e
agg ega ed in o one o e a ching sco e ep esen ing he
indi iduals’ gene al subjec i e well-being. This calcula ed
subjec i e well-being sco e was used o label each pa ici-
pan ’s oice eco ding. This app oach enables a me hod-
ologically sound e alua ion o he indi idual’s subjec i e
well-being and allows o co esponding compa isons.
5.2 Fea u e Ex ac ion
The a e age leng h o he ee-speech audio iles used o
analysis was 30 s, wi h a sampling a e o 44.100 Hz.
O e all, 18 acous ic phone ic and p osodic ea u es ha
heo e ically should indica e he speake ’s subjec i e well-
being we e ex ac ed by employing he lib osa lib a y
(McFee e al. 2015). This includes one empo al ea u e, he
di e ences in speech pauses, and mul iple spec al ea u es
(e.g., MFCCs). P osodic ea u es, such as he speech pauses,
cap u e speech’s hy hm, s ess, and in ona ion. These ea-
u es p o ide insigh s in o he empo al dynamics o speech
and can be used o de ec emo ional s a es in speech
(Khodabakhsh e al. 2015; Ra hina e al. 2012; Sanchez e al.
2011).
The ea u e co ela ion plo o he 18 ex ac ed ea u es
can be seen in Fig. 1. Those a e: he mean o ou MFCCs
(0–3 in he co ela ion plo ), he mean and s anda d de i-
a ion o he spec al oll-o (Klapu i and Da y 2006) o
es ima ing he minimum spec al mass (4 and 5), he mean
o he spec al cen oid o es ima ing he mean o he
spec al mass (6), he mean and s anda d de ia ion o ou
ch omag ams, co esponding o ou pi ch classes (7–10;
11–14), he numbe o speech pauses (McFee e al. 2015)
la ge han 1.5 imes o he median o i (15) and he i s
and hi d quan iles o he undamen al equencies (de
Che eigne and Kawaha a 2002) o he speake s (16–17). A
comple e desc ip ion o he ea u es can be ound in he
Appendix (a ailable online ia h p://link.sp inge .com).
Analyzing he co ela ions, some o he ex ac ed ea u es
a e mode a ely co ela ed bu wi hin he ea u e classes
123
326 N. Finze e al.: How a e We Doing Today?, Bus In Sys Eng 66(3):321–334 (2024)
only, e.g., he s anda d de ia ions o he ch omag ams
(11–14). None o he ex ac ed ea u es a e s ongly co -
ela ed. Hence, all ea u es add in o ma ion, none a e
duplica es.
The con inuous speech signal was di ided in o disc e e
ames o equal leng h based on he p inciple o sho - ime
spec al analysis. Since speech is conside ed s a ic o 5 o
25 ms, a window shi o 23 ms was used, employing a hop
size o 1024 (Logan 2000).
5.3 Model T aining
Reg ession models o o ecas ing he subjec i e well-be-
ing om speech da a we e e alua ed: Fou classes o
Table 1 O e iew o equi emen s
Use expe ience and in e ace Moni o ing and ca egi e usabili y Da a p o ec ion and eliabili y Technology-speci ic
equi emen s
E o less use expe ience o
senio s
No addi ional aining
equi ed
Consis en use in e ace
independen o upda es
Non-in usi e design
Real- ime o highly equen moni o ing o
senio s’ subjec i e well-being
P omp / imely ale s in c i ical si ua ions
Time-se ies unc ion o his o ical analysis
Easy usabili y o ca egi e s
Seamless communica ion be ween ca egi e s
and senio s
Compliance wi h da a
p o ec ion egula ions
P inciple o olun a y use
Use anonymi y p ese a ion
Consis en deli e y o accu a e
and eliable esul s
Func ionali y wi h limi ed
audio quali y
Main enance o accu acy and
cos -e ec i eness
Implemen a ion and
main enance easibili y
Fig. 1 Fea u e co ela ion plo
123
N. Finze e al.: How a e We Doing Today?, Bus In Sys Eng 66(3):321–334 (2024) 327
eg ession models we e conside ed o he quan i a i e
es ima ions o he QOL sco e. The e alua ed eg ession
models a e: Suppo ec o eg ession (SVR) (D ucke
e al. 1996), Lasso eg ession (LR) (Tibshi ani
1996,2011), andom o es eg ession (RFR) (B eiman
2001) and XGBoos eg ession (XGBR) (Chen and
Gues in 2016). Two o hose (RFR and XGBR) a e ee-
based ensemble me hods ha o en achie e high pe o -
mances in machine lea ning compe i ions, such as hose
o e ed ia Kaggle. Howe e , when he da a is small
enough, he suppo ec o machines, including SVR,
pe o m be e han he ee-based me hods (D ucke e al.
1996; Bose e al. 1992). Las ly, LR is a a ian o linea
eg ession ha con ains ea u e selec ion. Also, SVR and
XGBR implici ly make ea u e selec ions bu allow o
nonlinea ela ions be ween he a ge , he subjec i e well-
being sco e, and inpu a iables.
The models we e ained and es ed using a g oup-
s a i ied shu le spli c oss- alida ion wi h 60% o he
aining da a, 40% o he es da a, and 100 spli s, whe e
he g oups a e he pa icipan s. This ensu es ha he
models we e ained and es ed wi h di e en , non-o e -
lapping pa icipan s.
The hype pa ame e s o he models we e ound wi h a
andomized g id sea ch. Because he ou classes o
eg ession models e alua ed mos ly ha e a di e en se o
hype pa ame e s, we will no lis ex ensi e ables o he
e alua ions in his pape bu ins ead ocus on desc ibing
which hype pa ame e s a e he bes models o each class
(see Sec . 6).
5.4 Bias
A small numbe o eco dings may cause a bias in he
machine lea ning models, such as assigning iden ical sub-
jec i e well-being sco es o pe sons o he same gende .
To a oid his, we es ed ou da a o de e mine whe he he
collec ed demog aphic ac o s (gende , age, ela ionship
s a us, and educa ion le el) a ec he p oposed subjec i e
well-being. Fo all hese ac o s, we ound no s a is ically
signi ican dependency.
6 Resul s
The pe o mance o he eg ession models was e alua ed
using he me ics mean absolu e e o (MAE) and oo
mean squa ed e o (RMSE). While bo h me ics can be
oughly desc ibed as an a e age de ia ion om he eal
subjec i e well-being sco es, he RMSE is sensi i e o
ou lie s because o he squa e in he e m, while he MAE
is less sensi i e o ou lie s (Chicco e al. 2021). The esul s
o he eg ession models a e p o ided in Table 2.
All he e alua ed models pe o m be e han andom
(‘educa ed’) guesses. Those guesses we e andomly dis-
ibu ed acco ding o he subjec i e well-being sco e p o-
ided by he da a. The bes -pe o ming model, suppo
ec o eg ession (SVR), has an inc ease o abou 86%, and
he second-bes , he andom o es eg ession (RFR), has an
inc ease o abou 52% in MAE o he andom guesses. The
bes SVR model employed a adial basis unc ion (RBF)
ke nel wi h a scaled gamma and a egula iza ion alue
( ypically called C) o 0.5 as hype pa ame e s, while he
bes RFR model had a maximum ee dep h o six and a en
samples minimum ee spli . Ou o he ained models, he
Lasso eg ession pe o med wo s bu s ill be e han he
andom guesses. F om his, i can be in e ed ha he sub-
jec i e well-being sco e is no linea ly dependen on he
ex ac ed ea u es. This aligns wi h he inding ha SVR and
RFR, bo h able o cap u e non-linea i ies, had highe MAEs.
Du ing he aining o he models, he ea u e impo -
ances we e collec ed u ilizing he ea u e pe mu a ion
impo ances echnique (B eiman 2001). The ea u e
impo ances o he SVR model can be seen in Fig. 2.
In he plo , 14 o he 18 ex ac ed ea u es a e isible and
anked acco ding o a change in sco e (he e: MAE) when
he ea u es a e andomly pe mu a ed. The emaining ou
ea u es a e omi ed because hei sco e change was close
o ze o. The ea u e impo ances show ha he bes model
used all classes o ex ac ed ea u es (MFCCs, ch oma,
speech pauses, spec al cen oid, oll-o , and undamen al
equency) bu no all ea u es (e.g., he mean o MFCC 9).
Acco ding o he impo ances, he highes anks ha e he
phone ic ea u es (spec al oll-o , se en h and eigh h
MFCC, and he second ch omag am), ollowed by p osodic
ea u es (speech pauses, undamen al equency). The
ea u e impo ances o he second-bes model can be ound
in he Appendix.
Summa ized, he eg ession model p o es bene icial o
de ec ing unexpec ed shi s in subjec i e well-being,
yielding p ecise nume ical es ima es o QOL sco es, and
excelling in quan i a i e e alua ions. Gi en he eg es-
sion’s p edic i e powe , mode a e o small shi s in he
subjec i e well-being sco es can be de ec ed, allowing he
ini ia ion o necessa y in e en ions mo e quickly in case
Table 2 Reg ession model esul s
Model Ø MAE Ø RMSE
Random (educa ed) guesses 20.30 ±1.60 24.80 ±1.90
Suppo ec o eg ession 10.90 –2.17 14.29 –2.27
Random o es eg ession 13.33 ±2.84 16.64 ±3.53
XGBoos eg ession 13.70 ±2.55 17.02 ±3.14
Lasso eg ession 17.44 ±2.59 21.50 ±3.00
Bold alues indica e he bes eg ession model esul s
123
328 N. Finze e al.: How a e We Doing Today?, Bus In Sys Eng 66(3):321–334 (2024)
