1 o 17
Expe Sys ems, 2025; 42:e13832
h ps://doi.o g/10.1111/exsy.13832
Expe Sys ems
ORIGINAL ARTICLE OPEN ACCESS
Machine Lea ning Algo i hms o Add ess he Pola i y and
S igma o Men al Heal h Disclosu es on Ins ag am
NoemíMe ayo1 | AlbaAyuso- Lancha es2 | Cla aGonzález- Sanguino3
1Signal and Communica ions Theo y and Telema ics Enginee ing, School o Telecommunica ions Enginee ing, Uni e sidad de Valladolid, Valladolid,
Spain | 2Depa men o Pedagogy, Facul y o Medicine, Uni e sidad de Valladolid, Valladolid, Spain | 3Depa men o Psychology, Facul y Educa ion and
Social, Uni e sidad de Valladolid, Valladolid, Spain
Co espondence: Noemí Me ayo (noeme @ el.u a.es)
Recei ed: 9 July 2024 | Re ised: 16 Decembe 2024 | Accep ed: 23 Decembe 2024
Funding: This wo k was suppo ed by Uni e sidad de Valladolid.
Keywo ds: Ins ag am| machine lea ning| men al heal h| na u al language p ocessing| sen imen analysis| social ne wo ks| s igma
ABSTRACT
This esea ch explo es he social esponse o disclosu es and con e sa ions abou men al heal h on social media, which is a pi-
onee ing and inno a i e app oach. Unlike p e ious s udies, which ocused p edominan ly on psychopa hological aspec s, his
s udy explo es how communi ies eac o con e sa ions abou men al heal h on Ins ag am, one o he a ou i e social media
pla o ms among young people, b eaking new g ound no only in he Spanish con ex , bu also on a global scale, illing a gap
in in e na ional esea ch. The s udy c ea ed a no el co pus by collec ing and labelling commen s on Ins ag am pos s ela ed o
celeb i y men al heal h disclosu es, ca ego ising hem by pola i y (posi i e, nega i e, neu al) and s igma. Addi ionally, he e-
sea ch implemen s machine lea ning algo i hms o de ec s igma and pola i y in men al heal h disclosu es on Ins ag am. While
adi ional echniques like Suppo Vec o Machine (SVM) and RF (Random Fo es ) displayed decen pe o mance wi h lowe
compu a ional loads, ad anced deep lea ning and BERT (Bidi ec ional Encode Rep esen a ion om T ans o me s) algo i hms
achie ed ou s anding esul s. In ac , BERT models achie e a ound 96% accu acy in pola i y and s igma de ec ion, while deep
lea ning models achie e 80% o pola i y and 87% o s igma, e y high accu acy me ics. This esea ch con ibu es signi ican ly
o unde s anding he impac o men al heal h discussions on social media, o e ing insigh s ha can educe s igma and aise
awa eness. A i icial in elligence can be used o mo e esponsible use o social media and e ec i e managemen o men al
heal h p oblems in digi al en i onmen s.
1 | In oduc ion
Social ne wo ks ha e become one o he mos widesp ead com-
munica ion channels oday and allow o a cons an low o
in o ma ion ha e lec s he a i udes, ends and opinions o
ou socie y in eal ime. In ac , he e a e cu en ly 4.76 billion
social ne wo k use s wo ldwide (Da a epo al 2023), equi a-
len o app oxima ely 60% o he wo ld's popula ion. Looking
a social ne wo ks in e ms o mon hly ac i e use s, he la es
da a sugges s ha Facebook emains he wo ld's numbe one
social ne wo k wi h nea ly 3000 million use s. In his con ex ,
Ins ag am has also consolida ed i s posi ion among he op so-
cial media pla o ms, anking ou h wi h 2 million use s be-
hind Facebook, You ube and Wha sapp, wi h an a e age ime
pe use pe mon h o 12 h. When examining social media
p e e ences based on age and gende , indi iduals aged 16–24
and young women aged 25–34 p e e Ins ag am as hei op
social pla o m. Indeed, in Janua y 2023, nea ly wo- hi ds o
Ins ag am's o al audience we e 34 yea s old o younge (51% o
he o al audience we e be ween 13 and 17 yea s old; 33.7% be-
ween 18 and 24 yea s old, and 31.3% be ween 25 and 34 yea s
old) (S a is 2023). Fu he mo e, some ecen s udies show ha
This is an open access a icle unde he e ms o he C ea i e Commons A ibu ion-NonComme cial License, which pe mi s use, dis ibu ion and ep oduc ion in any medium, p o ided he
o iginal wo k is p ope ly ci ed and is no used o comme cial pu poses.
© 2025 The Au ho (s). Expe Sys ems published by John Wiley & Sons L d.
2 o 17 Expe Sys ems, 2025
Ins ag am is he SN mos used by young people (Oden and
Po e 2023).
Simila ly, he signi icance o men al heal h in socie y has
g own in ecen yea s, as he Wo ld Heal h O ganisa ion e-
po s a global inc ease in men al heal h issues. In 2019, 970
millions o people wo ldwide we e li ing wi h a men al dis-
o de , wi h anxie y and dep ession being he mos p e alen
condi ions (Wo ld Heal h O ganiza ion 2024). Mo eo e , an
Eu opean s udy om Oc obe 2023 e ealed ha 46% o EU
ci izens had expe ienced an emo ional o psychosocial issue
in he las 12 mon hs, such as eelings o dep ession o anxi-
e y (Eu opean Commission2023). When analysing he men-
al heal h landscape o younge popula ions, he si ua ion
becomes pa icula ly conce ning. Acco ding o UNICEF, in
2019, app oxima ely one in se en adolescen s globally, ep e-
sen ing 166 million indi iduals (89 million boys and 77 mil-
lion gi ls), we e es ima ed o be a ec ed by men al illness
(UNICEF2021). Hal o all men al diso de s in young people
de elop be o e age 14, and 75% by hei mid- wen ies. In ac ,
3% o 12- o 17- yea - olds a i m expe iencing dep ession and
32% epo anxie y. This issue ex ends o young adul s, wi h
33.7% o hose aged 18 o 25 epo ing some o m o men al ill-
ness. Pa icula ly, in 2023, app oxima ely 1 in 5 child en and
young people aged 8 o 25 in he UK we e es ima ed o ha e a
p obable men al diso de , wi h p e alences o 20.3% in he 8
o 16 age g oup, 23.3% among hose aged 17 o 19, and 21.7%
in he 20 o 25 age g oup (NHS Digi al 2023). In he same
yea (2023), 20.17% o 12–17 yea olds in he USA epo ed a
leas one dep essi e episode du ing ha yea (Men al Heal h
Ame ica2023). A ecen s udy conduc ed in Spain claims ha
42.1% o indi iduals ha e expe ienced dep ession, and 14.5%
ha e had suicidal idea ion o a emp ed suicide, wi h an a e -
age age o diagnosis a 26 yea s (Fundación Mu ua2023).
Howe e , despi e he commonali y and impo ance o men al
heal h, i is possible o say ha men al heal h s igma s ill exis s
in ou socie y, and despi e he p og ess s ill is an issue necessa y
o add ess (G onholm and Tho nic o 2022). This cons uc e-
e s o hough s (belie s, my hs, a ibu ions), emo ions (such as
eac ions o ea o pi y) and nega i e beha iou s (usually dis-
c imina ion, o desi e o dis ance onesel ), sha ed by he socie y
owa ds a speci ic g oup, in his case people wi h men al heal h
p oblems (Co igan and Wa son2002). In Spain, s igma owa ds
men al heal h is p esen in he gene al popula ion (González
Sanguino e al.2023), and ce ain s igma ising belie s and a -
ibu ions ha e been simila ly ound in adolescen s (González
Sanguino e al.2024).
On he ace o i , le e aging social media o alk abou men al
heal h should p omo e accep ance and educe s igma, as some
s udies show how high- impac pos s by celeb i ies can p o-
mo e awa eness and help- seeking by eaching la ge audiences
(G onholm and Tho nic o 2022; Jain, Pandey, and Roy2017;
Lee2019; Lee, Yuan, and Wohn2021). Howe e , due o he
shee olume and immediacy o opinions exp essed on hese
pla o ms, i has become a complex phenomenon. As a esul ,
i is unclea whe he hese social media pos s a e ac ually
os e ing accep ance and good quali y knowledge o inad e -
en ly pe pe ua ing u he s igma isa ion and/o i ializa-
ion o men al heal h (Pa lo a and Be ke s 2022; Robinson
e al.2018). In his social en i onmen o widesp ead adop-
ion o social ne wo ks, oge he wi h he cons an inc ease o
in o ma ion and opinions in eal ime, he applica ion o au-
oma ed echniques becomes highly ele an . Thus, A i icial
In elligence (AI) allows hese ac ions o be ca ied ou join ly,
speci ically he b anch o Na u al Language P ocessing (NLP)
(Män ylä, G azio in, and Kuu ila 2018) h ough wha is
known as Sen imen Analysis. Sen imen analysis combines
na u al language p ocessing and compu a ional linguis ics
o explo e he meanings o wo ds and hei con ex , wi h he
aim o unde s anding he unde lying emo ional ones. This
echnology can be applied o disce n emo ional eac ions
in commen s pos ed on social ne wo ks, enabling eal- ime
end acking, unde s anding cu en o u u e beha iou s.
Howe e , he linguis ic na u e o commen s pos ed on social
media pla o ms exhibi s signi ican dispa i ies compa ed o
con en ional language use (Ma ínez- Cáma a e al. 2014),
o en posing subs an ial challenges. These unique ea u es,
including b ie messages, missing con ex , g amma ical e -
o s, and a casual w i ing s yle, complica e he applica ion
o e ec i e sen imen analysis echniques. Fu he mo e, in
sen imen analysis, wo main app oaches a e p ima ily used:
lexicon- based echniques and supe ised lea ning- based ech-
niques. Lexicon- based echniques equi e dic iona ies whe e
wo ds a e labelled wi h emo ional esponses, such as pola i y
o associa ed emo ions. Howe e , in Spanish, he numbe o
dic iona ies is limi ed, and i is necessa y o de elop speci ic
dic iona ies o each a ea o applica ion (Redondo e al.2007).
Addi ionally, his me hod should be supplemen ed wi h ech-
niques o iden i ying nega ion o language ambigui y, which
adds complexi y o hese s a egies, especially on social ne -
wo ks (Taboada e al.2011). In con as , supe ised lea ning
echniques equi e labelled co po a, which a e examples o
opinions o commen s ha ha e been p e iously anno a ed.
This allows machine lea ning algo i hms o lea n om his
da a and make e icien p edic ions (A co e al.2021; Shah
e al.2023). This me hod p o ides he bene i o allowing a -
ious machine lea ning algo i hms o be lexibly used on he
same da ase . Fu he mo e, ce ain s udies, like he one men-
ioned in (S i as a a, Bha i, and Ve ma2022), ha e shown
ha supe ised me hods can ou pe o m lexical- based ech-
niques in speci ic scena ios, so ou esea ch will ollow he
machine lea ning app oach.
Despi e he mul iple ad an ages ha AI could p o ide, e-
sea ch in eg a ing AI o add ess men al heal h and s igma in
social ne wo ks is limi ed. NLP b ings impo an bene i s o
men al heal h esea ch by au onomously unco e ing impo -
an insigh s and pa e ns in he da a ha migh go unno iced
o una ailable o men al heal h expe s, and migh e en be
o e looked in manual e iews. In addi ion, exp essions ela ed
o men al heal h o en exhibi g ea e emo ional complexi y
and sub le y, as indi iduals in hese con ex s end o use am-
bi alen o me apho ical language o desc ibe hei emo ional
s a e. This cha ac e is ic equi es he de elopmen o mo e ad-
anced NLP models o accu a ely in e p e hese exp essions.
In addi ion, discussions abou men al heal h a e o en ma ked
by social s igma. This in luence can lead people o hide hei
ue eelings o use language ha does no accu a ely e lec
hei expe ience, adding an addi ional le el o complexi y o
he analysis o hese con e sa ions. E en mo e, he exis ing
14680394, 2025, 2, Downloaded om h ps://onlinelib a y.wiley.com/doi/10.1111/exsy.13832 by Uni e sidad De Valladolid, Wiley Online Lib a y on [26/02/2025]. See he Te ms and Condi ions (h ps://onlinelib a y.wiley.com/ e ms-and-condi ions) on Wiley Online Lib a y o ules o use; OA a icles a e go e ned by he applicable C ea i e Commons License
3 o 17
s udies ha assess s igma and pola i y in social ne wo ks
commen s ela e o speci ic diagnoses such as schizoph enia,
bipola diso de and Alzheime 's disease (Budenz e al.2019;
Jilka e al.2022; Osca e al.2017). O he s udies ha e ocused
on eac ions o an ipsycho ic medica ion (Mon e al. 2021),
and o he s udies a e indi ec ly ela ed o men al heal h is-
sues such as obesi y (Bog ad, Chen, and Ka ulu u2022) o
Co id- 19 (Xue e al.2020). In his way, o he esea ch (Budenz
e al.2019) uses supe ised lea ning o analyse he exis ence
o s igma in social ne wo ks owa ds bipola diso de , one o
he mos s igma ised men al illnesses. Speci ically, he s udy
allows us o cha ac e ise s igma om suppo i e messages
abou bipola diso de and hei epe cussion and impac on
Twi e . Rega ding psychosis, he conduc ed esea ch o (Jilka
e al.2022) p oposes o iden i y s igma ising wee s on Twi e
ela ed o schizoph enia by applying algo i hms such as SVM
(Suppo Vec o Machine), RF (Random Fo es ) among o h-
e s. Finally, he au ho s o (Osca e al. 2017) use machine
lea ning o analyse he exis ence o s igma on Twi e ega d-
ing Alzheime . All hese s udies ha e been ca ied ou on he
Twi e social ne wo k and only one o hem was de eloped
in he Spanish con ex (Mon e al.2021). In ha Spanish e-
sea ch, he au ho s aimed o in es iga e Twi e con e sa ions
conce ning an ipsycho ic d ugs in o de o gain insigh s in o
public eac ions and iden i y he mos equen ly discussed
a eas o clinical in e es ela ed o hei usage. Consequen ly,
his con ex depic s a scena io whe e he men al heal h and
social ne wo ks a e e ealed as an excep ional en i onmen ,
in e ms o hei ela ionship and impac , and allows us o un-
de s and public opinion o men al heal h, he emo ional e-
sponses i elici s and he p esence o s igma. Thus, he main
con ibu ions o ou esea ch a e: (a) Design a no el co pus
labelled wi h pola i y (posi i e, nega i e, neu al) and s igma
om Ins ag am pos commen s on celeb i y men al heal h
disclosu es. This da ase can be accessed on Gi Hub (Me ayo,
Ayuso, and González- Sanguino2023), allowing esea che s o
use i . (b) Modelling machine lea ning algo i hms o p edic
pola i y and s igma in social ne wo ks in case o disclosu e
o men al heal h p oblems, speci ically anxie y o dep ession.
Thus, his esea ch is inno a i e on se e al le els. I p oposes
machine lea ning analysis o men al heal h on Ins ag am o
which he e is ba ely any p eceden , since S udies on AI ha e
mainly concen a ed on Twi e . Secondly, his is a no el ap-
p oach, as p e ious esea ch has ocused p ima ily on iden i-
ying psychopa hology a he han examining he communi y
esponse o i (Ahmed e al.2022; Bi nbaum e al.2017; Fodeh
e al.2019; Gun uku e al.2017; Joshi and Kanoongo2022;
Lejeune e al.2022). Thi dly, his esea ch holds signi ican
social ele ance as i examines he eme ging end o public
eac ions o celeb i ies discussing hei men al heal h issues,
in luencing millions o indi iduals, a he han ocusing on
hash ags o gene al commen s on a opic. This can acili a e
mo e e ec i e campaigns and ac ions, leading o inc eased
awa eness and a ou able e ec on socie y as a whole, along
wi h pa icula g oups, including men al heal h p o essionals,
and o ganisa ions, p omo ing esponsible use o social media
and making in o med decisions o add ess men al heal h on
hese pla o ms.
This documen is s uc u ed as ollows. Sec ion2 desc ibes he
me hodology o c ea ing he da ase om Ins ag am social
media pos s. Sec ion3 de ails he classi ica ion models imple-
men ed. Sec ion4 e eals he esul s o hese models. Finally,
Sec ion5 explains he main conclusions.
2 | Design o he Men al Heal h Co pus in Social
Ne wo ks
2.1 | Selec ion o Pos s on Ins ag am
A sea ch was conduc ed o p ima y pos s (made di ec ly by
he au ho ) con aining disclosu es o con e sa ions abou hei
men al heal h p oblems by Spanish in luence s on Ins ag am
(wi h o e 100,000 ollowe s). To ca y ou his p ocess, we
ha e sea ched o publica ions om di e en p o iles o he
people wi h he mos ollowe s in Spain, as well as e iewing
p ess and ele ision news ha usually announce his ype o
publica ion. This sea ch co e ed publica ions om Sep embe
2020 o Decembe 2022. A e an ini ial e iew, mos o he
pos s we e made by women, and as we we e unable o ha e
a gende balance in he pos s, we decided o include he
male gende in he Ins ag am pos s as an exclusion c i e ion
o a oid possible bias in he analysis. We ound a ound 20
pos s by high- impac emale in luence s on Ins ag am and
selec ed he 10 wi h mo e esponses o commen s. All pos s
had a simila o ma , wi h one o mo e pho os accompanied
by ex alking abou men al heal h p oblems, such as dep es-
sion o anxie y. A couple o pos s announced ha hey we e
wi hd awing om he social ne wo k due o hei men al
heal h p oblems, and in ano he couple o pos s, he image
showed he pe son c ying. Unusually, one o he pos s con-
sis ed o a p omo ional ideo in which an in luence alked
abou he men al heal h issues o p omo e a p oduc . Once
he Ins ag am pos s we e selec ed, all commen s in esponse
o hem we e collec ed using IGCommen Expo (“One Click
Commen Ex ac o o IG”) (Ch ome Web S o e2023), a ool
o expo Ins ag am commen s o CSV (Comma Sepa a ed
Values) o ma . A o al o N = 21,151 commen s we e collec ed.
Rega ding e hical conside a ions and da a p i acy, he s udy
has been app o ed by he e hics and deon ology commi ee o
he Uni e si y o Valladolid (PI 23- 3365) and we anonymised
all Ins ag am commen s (disca ding @men ions, use names
and URLs). The inal selec ion o Ins ag am pos s, oge he
wi h he name o he in luence s, he numbe o ollowe s and
he esponses associa ed wi h each pos a e in he Suppo ing
In o ma ion.
2.2 | Desc ip ion o he Labels in he Co pus:
Pola i y and S igma
Following a manual obse a ion o he da ase , and in line
wi h p e ious li e a u e on manual labelling o commen s
(Budenz e al. 2019; Mon e al. 2021; Bog ad, Chen, and
Ka ulu u 2022; Toma , Ma hu , and Suman 2022; Delanys
e al.2020) we se up guidelines o he di e en labelling ca -
ego ies: pola i y and s igma. Rega ding he pola i y associa ed
wi h a commen , i consis s o gi ing a posi i e, nega i e o
neu al/unde ined alue o he commen s in esponse o he
disclosu e o desc ip ion o he symp oma ology in he pos .
Posi i e pola i y e lec s unde s anding, encou agemen o
14680394, 2025, 2, Downloaded om h ps://onlinelib a y.wiley.com/doi/10.1111/exsy.13832 by Uni e sidad De Valladolid, Wiley Online Lib a y on [26/02/2025]. See he Te ms and Condi ions (h ps://onlinelib a y.wiley.com/ e ms-and-condi ions) on Wiley Online Lib a y o ules o use; OA a icles a e go e ned by he applicable C ea i e Commons License
4 o 17 Expe Sys ems, 2025
e en admi a ion o he pos . Fo example, “Chee up, we lo e
you”. Nega i e pola i y is assigned when he pe son exp esses
nega i e opinions, usually by ques ioning he pos wi h i onic,
sa cas ic o e en de isi e and de oga o y commen s. Fo ex-
ample, “how you show ha you don' know wha dep ession
o anxie y is, shame on you!”. Neu al o unde ined pola i y is
assigned in cases whe e no clea opinion is de ec ed o can be
in e p e ed in bo h di ec ions. Fo example, “ ake medica ion,
i will help you” “and you pa ne ?”
Abou s igma isa ion, s igma ising esponses o commen s a e
beha iou s in which nega i e belie s and emo ions owa ds
men al heal h p oblems a e exp essed. S igma mani es s in a a-
ie y o o ms including ejec ion and ange agains he pe son,
which may ex end o con emp o mocke y, beli ling hei p ob-
lem. Fo example, “Wha a desi e o d aw a en ion o you sel ”;
“you' e so inconsis en and seeking he limeligh ”. Because so-
cially we know ha “s igma is w ong” many ejec ion commen s
a e made in an i onic o sa cas ic way. Fo example, and how
do you w i e on ins a?. Addi ionally, ange is shown by a guing
ha such pos s “ i ialise o comme cialise” men al heal h. Fo
example, “don' come and ell me you alse s o ies o o e com-
ing, wi hou e en knowing wha i is o wo k…”. O he imes he
s igma mani es s i sel as pi y o so ow o he pe son. Fo ex-
ample, “I b eaks my hea ”.
2.3 | P ocess o Ca ego ising he Co pus
The labelling p ocess was di ided in o h ee phases: an ini ial
phase wi h a pilo co pus (N = 787 commen s), a second phase
ocused on he de elopmen o he co pus wi h all he commen s
o he selec ed pos s (N = 21,151) and a hi d phase wi h he inal
co pus (N = 2287). The same me hodology was ollowed in he
i s wo phases: once he commen s we e collec ed, he co pus
was cleaned, and hen wo independen expe s we e esponsi-
ble o labelling each ca ego y. A hi d expe hen e iewed he
commen s o esol e disc epancies. In he hi d and inal phase,
a inal co pus is buil om he la ge co pus o apply machine
lea ning algo i hms (N = 2287).
In his way, he labelling p ocess in he pilo co pus ep e-
sen ed a i s s age ca ied ou on a andom subse o commen s
(N = 787, including emo icons). The p ocess was di ided in o he
ollowing s ages:
(a). Ini ial da a cleaning: commen s in o he languages, wi h
ac onyms only (e.g., “TQ”, “I lo e you” in English) and hose
lacking cohe ence, e.g., “cui- de- se- BR” o “gus a e u” (“like
see you”) in English, we e dele ed o main ain he sample's ele-
ance. Addi ionally, commen s labelling o he people who ha e
eplied o he same pos ha e been emo ed, excep when he au-
ho o he pos is labelled and ele an in o ma ion is gi e (e.g.,
“@dulceida I hope all is well”). This esul s in a inal sample o
N = 573 commen s.
(b). Handling Emo icons: emo icons ha e been excluded o ocus
solely on he linguis ic e ec s.
(c). Expe labelling: wo specialis s independen ly labelled he
clean sample wi hou knowing each o he 's assessmen s, while
a hi d expe examined he inconsis encies ha eme ged in he
clean sample.
In his pilo co pus, we iden i ied disc epancies in only 2.43%
(N = 14) o he labelled commen s, which showed a e y sa is-
ac o y in e - a e eliabili y among expe s who ca ego ised
(Hallg en2012). These disc epancies occu ed p edominan ly
in he neu al pola i y ca ego ies, as some commen s con ained
messages wi h bo h posi i e and nega i e pola i ies (“Wha a
pi y! I'm so so y abou wha happened o you, lo s o encou -
agemen ”). The hi d e iewe ound ha he message o sym-
pa hy p e ailed in hese cases, so i was ca ego ised as posi i e
pola i y. Sa is ac o y esul s om his ini ial p ocess (pilo co -
pus) p o ide a solid basis o eplica ing he esul s in a la ge
sample, ensu ing consis ency o labelling and main aining da a
quali y o u u e analysis. Rega ding he second phase, which
co esponds o he whole co pus, he co pus was labelled wi h
all commen s (N = 21,151). The same p ocedu e as in he pilo
s udy was ollowed:
a. Ini ial da a cleaning: his p ocess educed he whole co -
pus o a inal sample o N = 15,213 commen s. To gua an-
ee he sui abili y o he sample, commen s ha consis ed
solely o ac onyms (e.g., LOL), ha we e no w i en in
English o ha we e incomp ehensible we e disca ded
in his p ocess. In addi ion, commen s ha only se ed
o name o he use s wi hou u he con en we e also
emo ed.
b. Handling Emo icons: emo icons ha e been emo ed.
c. Expe labelling: he inal sample was labelled by wo
expe s sepa a ely, while a hi d expe e alua ed any in-
consis encies. I he wo ini ial expe s could no ag ee
on he assigned label, a hi d e iewe was equi ed.
I his hi d e iewe also could no esol e he ie, he
commen was excluded om he co pus o a oid pos-
sible e o s. To ca y ou he labelling p ocess, he ex-
pe s we e psychologis s o ained pe sons who used a
labelling guide, elabo a ed wi h examples and desc ip-
ions o he di e en ca ego ies o ou co pus (pola i y,
s igma). This labelling guide is also eely accessible and
a ailable in a Gi hub eposi o y (Me ayo, Ayuso, and
González- Sanguino2023).
The pe cen age o disc epancy in his second phase was a ound
2.3% (489 commen s). The g ea es dispa i y was obse ed be-
ween neu al and posi i e pola i y in hose commen s whe e
ad ice was gi en (“Ayyy, ake as much ime as you need, heal h
comes i s ”). In hese cases, he hi d e iewe ca ego ise hese
commen s wi h neu al pola i y. Finally, he hi d phase was as-
socia ed wi h he inal co pus. Once he ca ego isa ion o he
ull co pus was comple ed, a ho ough selec ion p ocess was un-
de aken o de elop a ep esen a i e co pus ha would be app o-
p ia e o ou IA algo i hms. A key conside a ion in da a co po a
is class balance; when add essing a classi ica ion p oblem, an
imbalance whe e one class has signi ican ly mo e da a han an-
o he can lead classi ica ion models o a ou p edic ions o he
majo i y class. This imbalance ad e sely a ec s he algo i hm's
pe o mance and p edic i e capabili y. As a esul , he o iginal
se o 15,213 commen s om he ex ensi e co pus was sys em-
a ically condensed in o 2287 commen s in he ollowing s eps:
14680394, 2025, 2, Downloaded om h ps://onlinelib a y.wiley.com/doi/10.1111/exsy.13832 by Uni e sidad De Valladolid, Wiley Online Lib a y on [26/02/2025]. See he Te ms and Condi ions (h ps://onlinelib a y.wiley.com/ e ms-and-condi ions) on Wiley Online Lib a y o ules o use; OA a icles a e go e ned by he applicable C ea i e Commons License
5 o 17
a. Remo al o edundan commen s: messages wi h iden ical
con en we e emo ed, keeping only a single example o
each ecu ing opic.
b. Dis ibu ion: we e alua ed he dis ibu ion o commen s
among di e en ca ego ies o ensu e a mo e equi able ep-
esen a ion o pola i y and s igma in he da ase , ensu ing
ha models would make accu a e p edic ions in all classes.
Al hough he e a e mo e commen s o posi i e pola i y
han nega i e o neu al, and mo e non- s igma ising han
s igma ising commen s, he dis ibu ion is signi ican ly
mo e balanced compa ed o he whole co pus.
c. Commen s andomly chosen: a e de e mining he a ge
dis ibu ion pe cen ages, commen s om all pos s we e
andomly picked o be pa o he inal da ase .
2.4 | Co pus S a is ics
Table1 displays he equency- based desc ip i e s a is ics o each
ca ego y o he co pus, and Figu e1 p esen s he desc ip i e s a-
is ics by pe cen age. The mos p edominan pola i y is posi i e
(66.7%), as well as non- s igma ising esponses (80.1%). Besides, i
is obse ed ha he e a e mo e nega i e pola i y commen s han
s igma ising ones. This is because many commen s ha include
disclosu es o men al heal h p oblems a e no s igma ising bu
hei emo ional pola i y is nega i e (e.g., “I c ied wi h you when I
saw you ea s… e en hough I don' know you in pe son, ell you
ha my hand will always be wi h you”). Likewise, he e a e also
neu al messages ha include ad ice o eac ions wi hou a spe-
ci ic emo ional one (e.g., “Good bless you”; “ eal li e”) ha also do
no mee he equi emen s o be ca ego ised as s igma ising.
3 | Classi ica ion Models
3.1 | Suppo Vec o Machine
The Suppo Vec o Machines (SVM) algo i hm is used in bo h
classi ica ion and eg ession p oblems. I s goal is o ind an op i-
mal sepa a ion hype plane ha maximises he dis ance be ween
da a classes, in ou case pola i y and s igma ca ego ies, in a ea-
u e space (Cholle 2021; Noble2006). The mos impo an con-
igu able hype pa ame e s in SVM include he Ke nel, which
de e mines he ype o ans o ma ion used o sepa a e da a in
a high- dimensional space, and he egula isa ion pa ame e C,
which con ols he ade- o be ween achie ing a wide ma gin
and minimising e o s in classi ica ion. P ope uning o hese
hype pa ame e s is essen ial o achie e op imal pe o mance in
SVM models and ensu e accu a e classi ica ion o he da a.
3.2 | Random Fo es
The Random Fo es (RF) algo i hm elies on building mul i-
ple decision ees du ing aining and combining hei esul s
o make mo e accu a e and obus decisions (P obs , W igh ,
and Boules eix2019). Each ee in he o es is ained on a an-
dom sample o he da a and uses a andom selec ion o ea u es
o make decisions. Typical con igu able hype pa ame e s in
Random Fo es include he numbe o ees and he maximum
numbe o ea u es o conside in each spli (max_ ea u es).
P ope uning o hese hype pa ame e s can signi ican ly in lu-
ence he pe o mance and gene alisabili y o he model.
TABLE 1 | Desc ip i e s a is ics by equencies.
Va iable F equency
Pola i y
Nega i e (P) 588
Posi i e (N) 1526
Neu al (NEU) 173
To al 2287
Es igma
No 1833
Yes 454
To al 2287
FIGURE 1 | Desc ip i e s a is ics by pe cen age o he da ase o pola i y and s igma ca ego ies (a) Pola i y (b) S igma.
14680394, 2025, 2, Downloaded om h ps://onlinelib a y.wiley.com/doi/10.1111/exsy.13832 by Uni e sidad De Valladolid, Wiley Online Lib a y on [26/02/2025]. See he Te ms and Condi ions (h ps://onlinelib a y.wiley.com/ e ms-and-condi ions) on Wiley Online Lib a y o ules o use; OA a icles a e go e ned by he applicable C ea i e Commons License
6 o 17 Expe Sys ems, 2025
3.3 | Hyb id CNN- LSTM Model
The p oposed hyb id deep lea ning a chi ec u e consis s o he ol-
lowing laye s (Figu e2): 1 Embedding laye ⟶ 1 Dimension con-
olu ional laye (Con 1D) ⟶ 1 MaxPooling Laye ⟶ 1 LSTM
(Long Sho - Te m Memo y) laye ⟶ 1 Dense Laye . Ou model
bene i s om he po en ial o e ed by he combina ion o ecu -
en (Recu en Neu al Ne wo k, RRN) and con olu ional (CNN,
Con olu ional Neu al Ne wo k) laye s. As he con olu ional laye
assumes ce ain asks, i educes he p ocessing load on he LSTM
( ecu en laye ), imp o ing he e ec i eness o ha laye . This
model will classi y he men al heal h commen s in o h ee pola -
i ies (P, N, NEU) and in wo s igma ca ego ies (Yes, No). In he
ollowing, each laye o he model will be desc ibed in de ail:
1. Embedding laye : This laye ans o ms ex s, Ins ag am
commen s in ou case, in o nume ical ec o s so ha hey
can be in e p e ed by neu al ne wo ks. This p ocess is ca -
ied ou using wo d embedding, in which each wo d is de-
pic ed as a ec o . The goal is o assign simila alues o
wo ds ha sha e a ce ain seman ic ela ionship. Thus, wo
echniques can be applied: lea ning wo d embeddings in
conjunc ion wi h he p oblem o be sol ed (using he p ob-
lem co pus) o loading embedding ec o s om p ecom-
pu ed da abases o wo d embeddings. The i s op ion was
chosen because p e- calcula ed dic iona ies a e designed
in a gene al way and hei e ec i eness depends o a la ge
ex en on hei simila i y o he wo ds in ou speci ic co -
pus. In con as , lea ning di ec ly om ou co pus will be
op imal as i p o ides a ele an sou ce o in o ma ion. The
embedding laye unc ions like a dic iona y ha maps in-
ege indices, which co espond o speci ic wo ds, o dense
ec o s. I accep s in ege s as inpu , sea ches hem in i s
in e nal dic iona y, and e u ns he associa ed ec o s,
much like a dic iona y lookup. Ini ially, when he embed-
ding laye is se up, i s in e nal wo d ec o s, o weigh s,
a e andomised. Th oughou he aining p ocess, hese
ec o s a e adjus ed using backp opaga ion, esul ing in
a dis inc s uc u e ha is ailo ed o he speci ic ask by
he end o aining. The e o e, he embedding laye akes
a wo- dimensional enso as inpu and e u ns a h ee-
dimensional enso ha can be u he p ocessed by a con-
olu ional laye .
2. 1D (Dimension) Con olu ion Laye (Con 1D): This laye
employs il e s on he da a o iden i y local ea u es wi hin
he inpu . I s unc ion is o iden i y impo an pa e ns
using con olu ion ope a ions, educing he wo kload o
he subsequen RNN laye . In essence, i s eamlines he
p ocessing o he RNN by emo ing in e media e s ages
h ough ex pa e n de ec ion. In addi ion, he ReLU
(Rec i ied Linea Uni ) unc ion shall be used as an ac i-
a ion unc ion, which shall be applied a e con olu ion.
Addi ional key pa ame e s include he numbe o il e s
(wi h each il e designed o cap u e a speci ic pa e n in
he inpu da a) and he il e size (ke nel size), which de e -
mines he dimensions o he il e s based on he leng h o
he window.
3. MaxPooling laye : This laye is used a e he con olu ion
laye o educe he dimensionali y o he ea u es ex ac ed
by he p e ious laye s o he ne wo k, while p ese ing he
mos ele an in o ma ion. The MaxPooling laye ans-
o ms a da a ma ix in o a smalle ma ix, e aining he key
elemen s o he o iginal.
4. LSTM laye : This RNN laye is employed o iden i y long-
ange pa e ns in he inpu da a. In pa icula , LSTM laye s
imp o e he unc ionali y o adi ional ecu en ne wo ks
by combining bo h long- e m and sho - e m memo y ca-
pabili ies. The pa ame e s o be adjus ed a e he numbe
o neu ons, a d opou a e and a ecu en d opou a e.
Bo h pa ame e s a e adjus ed o p e en o e i ing and
enhance he model's gene alisa ion du ing aining. These
pa ame e s indica e he p opo ion o neu on uni s ha a e
andomly “deac i a ed” du ing each aining s ep, o p e-
en hem om becoming oo dependen on neighbou ing
neu ons.
5. Dense Laye : This laye akes he ea u es lea ned by he
p e ious laye s and p oduces he inal ou pu o he ne wo k,
c ucial o p oducing he inal model p edic ions. Since we
use he model o classi ica ion, he numbe o neu ons in
he ou pu laye co esponds o he ca ego ies in o which we
wan o classi y he da a, ha is in mul i- class classi ica ion
p oblems he e will be one neu on pe class. The e o e, in ou
case we will pu h ee neu ons o ca ego ise pola i y (P, N,
NEU) o wo neu ons o ca ego ise s igma (Yes, No). Finally,
he so max ac i a ion unc ion will be u ilised o ans o m
FIGURE 2 | Con igu a ion o laye s in he p oposed hyb id CNN- LSTM ne wo k model.
14680394, 2025, 2, Downloaded om h ps://onlinelib a y.wiley.com/doi/10.1111/exsy.13832 by Uni e sidad De Valladolid, Wiley Online Lib a y on [26/02/2025]. See he Te ms and Condi ions (h ps://onlinelib a y.wiley.com/ e ms-and-condi ions) on Wiley Online Lib a y o ules o use; OA a icles a e go e ned by he applicable C ea i e Commons License
7 o 17
he ou pu s in o a p obabili y dis ibu ion, as i is equen ly
used in mul i- class classi ica ion asks.
Finally, in he con ex o classi ica ion p oblems, he e a e
o he c ucial hype pa ame e s in luencing model con e gence,
including op imize s and loss unc ions. In ou pa icula sce-
na io, we chose he ca ego ical c oss- en opy loss unc ion,
which is a commonly used op ion o classi ica ion asks, espe-
cially when dealing wi h h ee o mo e labels as in ou scena io.
Addi ionally, he ADAM op imize was selec ed due o i s e sa-
ili y, s ong pe o mance, and widesp ead u ilisa ion in simila
p oblem domains. As men ioned abo e, he wo d embedding
p ocess can be ca ied ou in wo ways: lea ning wo d embed-
dings join ly wi h he p oblem you wan o sol e (using he
co pus o he p oblem) o by loading embedding ec o s om
p ecompu ed da abases. The i s op ion was chosen because
o he ad an ages ha his echnique p o ides. To accomplish
his, a ocabula y is gene a ed om he co pus by employing he
In o ma ion Gain (IG) echnique, which emphasises e ms ha
occu mos equen ly. IG is a ou ed o e absolu e equency
since i e alua es how o en a wo d appea s in a pa icula class
compa ed o i s occu ence in o he classes, while absolu e e-
quency only quan i ies o al occu ences wi hou conside ing
class di e ences. To de e mine he IG o a wo d, we i s com-
pu e i s en opy (La ose and La ose2014; Wi en, F ank, and
Hall2011), shown in 1:
whe e
pi
is he p obabili y ha he wo d
wi
appea s in he class
ci
. Then, we calcula e he IG ollowing 2:
In his equa ion,
C
ep esen s he se o classes and
X
is he sub-
se o ex s in which he wo d
wi
is ound. To calcula e
H(C)
,
he p obabili ies o each ca ego y in he co pus a e de e mined,
while o calcula e
H(C,X)
he likelihood o a wo d occu ing o
no occu ing in he co pus needs o be calcula ed, along wi h
i s p obabili ies o occu ence and non- occu ence in each
ca ego y.
3.4 | BERT Model
BERT is a ans o me - based language model ha unde s ands
he con ex o wo ds in bo h di ec ions, enhancing asks like
na u al language p ocessing and comp ehension. Employing
BERT o sen imen analysis equi es ini ially aining he
model on a subs an ial da ase be o e ine- uning i on a dedi-
ca ed da ase . This p ocess enables he model o gain a b oad
unde s anding o language, which i can hen e ine o cap u e
he nuances o sen imen analysis wi hin a speci ic ield, such
as men al heal h in social media con ex s. In ou case, we ha e
used a p e- ained linguis ic model o social ne wo king ex
in Spanish, called RoBERTui o, ained ollowing he RoBERTa
guidelines on 500 million wee s (Pé ez e al.2021). This model
su passes o he p e- ained language models o Spanish.
Mo eo e , he T ans o me s lib a y o e s he T aine class o
ine- une any o he p e- ained models on a pa icula da ase .
This app oach allows us o iden i y he bes aining pa ame e s,
such as he lea ning a e (which accele a es model con e gence),
ba ch size ( he numbe o samples be o e upda ing weigh s), and
epochs (which indica e he numbe o i e a ions o e he ain-
ing da ase ).
4 | Expe imen s and Resul s
This sec ion desc ibes he main esul s o he expe imen s. To
iden i y he ideal con igu a ion, we will sea ch o he mos sui -
able hype pa ame e alues o each model in o de o op imise
a ious pe o mance me ics. Al hough accu acy e lec s he
o e all p opo ion o ins ances ha he model has co ec ly clas-
si ied, i is essen ial o conside addi ional me ics ha assess
he model's pe o mance o each indi idual class. Me ics like
p ecision, ecall, and F1- sco e a e pa icula ly aluable when
dealing wi h imbalanced da ase s. A key aspec o model e al-
ua ion is ha ing wo sepa a e da ase s: he aining se , used o
aining he model, and he alida ion se , used o e alua ing
i s pe o mance. In ou case his has been spli in a ypical a io
o 70%–30%, espec i ely (Onei os2023). On he o he hand, we
used he c oss- alida ion echnique o iden i y he op imal pa-
ame e se s o he model being ained. Speci ically, We u i-
lised k- old c oss- alida ion, a me hod ha en ails pe o ming
k i e a ions in which he model is ained and assessed k imes.
Addi ionally, we implemen ed he Ea lyS opping echnique o
main ain he model's gene alisa ion abili y. This me hod hal s
aining once he alida ion loss eaches i s minimum, p e-
en ing u he o e i ing. We implemen ed ou classi ica ion
models in Py hon ( e sion 3.1) using Ke as (Onei os2023) and
Tenso Flow (Google B ain Team2023), and execu ed all models
on he Google Colab pla o m.
4.1 | Da a P e- P ocessing and Encoding
Tex p e- p ocessing consis s o cleaning and p epa ing ex ual
da a o ob ain a seman ically iche ep esen a ion ha acili-
a es i s compu a ional ep esen a ion. Thus, he ollowing se-
ies o p e- p ocessing echniques ha e been implemen ed:
• To con e o lowe case: o educe duplica e wo ds.
• To elimina e men ions (), hash ags (#), and URLs: as hey
do no con ibu e any aluable in o ma ion.
• To dele e punc ua ion ma ks.
• To minimise he epe i ion o cha ac e s: o example,
change “Siiiiii” o “Si” (“yes” in English).
• To s anda dise slang/ja gon: o example, change “ b” o
“ ambién” (“ u he mo e” in English).
The ollowing s ep in ol es na owing down he ocabula y
used by he classi ica ion models ( ea u e educ ion) by applying
he echniques ou lined below:
• Remo e s opwo ds: wo ds ha ca y no signi ican mean-
ing on hei own, such as a icles, ad e bs, p eposi ions,
(1)
H
=
N−1
∑
i=1
pilogi(pi)
.
(2)
IG(C,X)=H(C)−H(C,X).
14680394, 2025, 2, Downloaded om h ps://onlinelib a y.wiley.com/doi/10.1111/exsy.13832 by Uni e sidad De Valladolid, Wiley Online Lib a y on [26/02/2025]. See he Te ms and Condi ions (h ps://onlinelib a y.wiley.com/ e ms-and-condi ions) on Wiley Online Lib a y o ules o use; OA a icles a e go e ned by he applicable C ea i e Commons License
8 o 17 Expe Sys ems, 2025
conjunc ions, and ce ain e bs. They a e usually e y
equen wo ds in na u al language and depend on he
language.
• Apply s emming: a ex no malisa ion echnique ha e-
duces wo ds o hei oo . This echnique emo es a ixes
om wo ds, which can lead o in alid wo ds. Fo in-
s ance, he Spanish wo ds “pensando” (meaning “ hink-
ing” in English) and “pensamien o” (meaning “ hough ”
in English) will be sho ened o “pensa.” We will employ
he SnowballS emme om NLTK (Py hon So wa e
Founda ion2021) o his pu pose in Spanish.
In addi ion, he BERT model (RoBERTui o) (Pé ez e al. 2021)
includes a speci ic da a p e- p ocessing consis ing o : cha ac e
epe i ions a e capped a h ee, use names a e eplaced wi h a des-
igna ed oken, hash ags a e subs i u ed wi h ano he oken, and
emojis a e con e ed in o hei ex ual desc ip ions using a spe-
cialised lib a y. Howe e , RoBERTui o was e alua ed using bo h
da a p ep ocessing me hods, and he pe o mance esul s we e
qui e simila . The subsequen s ep in ol es okeniza ion, a unda-
men al s ep in ex p ocessing, ha consis s o b eaking ex in o
disc e e uni s called “ okens”, in ou case wo ds. Tokens p o ide
disc e e uni s ha compu e s can wo k wi h o unde s and and
pa se ex mo e e ec i ely. In his case we use he Twee Tokenize
okenize (NLTK P ojec 2023) o he SVM, RF and hyb id CNN-
LSTM models. On he con a y, BERT algo i hms use hei own
okenize . The goal is o ind he mos meaning ul bu smalles
ep esen a ion. The nex s ep is o ans o m he ex s in o numbe
( ea u e ex ac ion p ocess), since machine lea ning models and
hei inpu s ha e o be numbe s. In ou case we ha e o ans o m
wo pa s: on he one hand he okenised and no malised messages
(Ins ag am pos s), and on he o he hand he labels ha co espond
o he ca ego ies (pola i y and s igma in his case). To con e he
messages in o nume ical o ma , a dic iona y has been es ablished
whe e each wo d is assigned a speci ic index ec o (as desc ibed
in he p e ious sec ion on wo d embedding). Fo he labels, One
Ho Encoding was used, which encodes a ious classes as a ma-
ix. In his ma ix, a “1” is placed in he column co esponding o
he class o he ex (Ins ag am message), while “0” is used o all
o he classes. The e o e, o he pola i ies (P, N, and NEU), we will
c ea e a ma ix wi h h ee columns. A simila app oach is used o
ca ego ising s igma labels (Yes and No), esul ing in a wo- column
ma ix. Fo he SVM and RF models, ins ead o using indi idual
columns o each a iable wi h bina y alues (0 o 1), we use a sin-
gle global a iable o ep esen one ou pu , as hese models gene -
a e only a single esul . He e, pola i y is indica ed as P, N, o NEU,
unlike he ea lie bina y encoding o “0” o “1”.
4.2 | Pola i y Resul s in he Men al Heal h Co pus
4.2.1 | Resul s SVM Model
The op imal hype pa ame e s in SVM will be sea ched in he nex
o de : ke nel ype and egula isa ion pa ame e (C). The op imisa-
ion p ocess begins wi h a ke nel scan, which e eals ha he RBF
ke nel achie es he highes accu acy a 63%. In con as , he Poly
and Sigmoid ke nel ypes yield lowe esul s o 62%. We hen p o-
ceeded o op imise he C egula isa ion pa ame e , whe e he bes
alue is 1.4, achie ing an accu acy o 65%. Howe e , as we inc ease
he alue o C, s a ing a 10, he model exhibi s he wo s accu a-
cies, eaching alues o 59% and 57% o 100 and 200, espec i ely.
In con as , i we con inue dec easing he alue o C below 1, he
model's accu acies emain mo e s able bu lowe , eaching le els
o 63% om C = 0.01 o C = 0.001.
In summa y, he alues o he hype pa ame e s ha op imise
he SVM model a e RBF o he ke nel ype and 1.4 o C ( eg-
ula isa ion pa ame e ), eaching a inal accu acy o 65%. In ad-
di ion o accu acy, i is impo an o e alua e he model in each
class sepa a ely h ough o he me ics such as p ecision, ecall
and F1 sco e (Table2). The esul s show ha he P class is he
bes p edic ed, eaching a p ecision o 68%. In con as , he NEU
and N classes show lowe pe o mance in all me ics.
4.2.2 | Resul s o he RF Model
In he case o RF, he hype pa ame e sea ch will ocus on de-
e mining he ideal numbe o ees o employ. Addi ionally, RF
selec s andom ea u e subse s o op imise spli s, making he
hype pa ame e max_ ea u es c ucial in deciding how many
ea u es should be conside ed. The e o e, he hype pa ame e s
will be sea ched in he ollowing o de : max_ ea u es and he
numbe o ees. Fi s , he accu acy o he model was e alua ed
wi h di e en alues o max_ ea u es. Using Log2, he accu acy
achie ed was 66%. Using he squa e oo (Sq ), he accu acy im-
p o ed o 69%, so Sq was selec ed. Then, we p oceed o ind he
bes numbe o decision ees o add ess he p oblem. Figu e3
shows ha he op imal alue is 600 decision ees, ob aining an
accu acy o 71%.
In addi ion o global accu acy, Table3 shows he esul s o p eci-
sion, ecall and F1- sco e o each class sepa a ely. I is obse ed
ha he P class is he class bes p edic ed by he RF model, as all
me ics show e y good esul s. As o he N class, a p ecision
close o 60% is achie ed (highe han he SVM model). Finally,
he NEU class has a low p ecision o 40%, which is easonable
since i is qui e di icul o de ec neu al commen s when we
exp ess opinions on social ne wo ks. As expec ed, he RF clas-
si ica ion model imp o es SVM pe o mance o all pola i ies.
4.2.3 | Resul s o he Hyb id CNN- LSTM Model
To ain and e alua e he pe o mance o he model, di e -
en ypes o es s ha e been pe o med o see he a ia ions in
he accu acy me ics in ol ed. These es s a e: adjus men o
he numbe o il e s and neu ons, he d opou a es and he
lea ning a e o he Adam op imize . Nex , es s we e made
wi h he educ ion o he o al numbe o unique wo ds chosen
TABLE 2 | Summa y o he SVM model esul s o p ecision, ecall
and F1- sco e me ics conside ing h ee pola i y classes (P, N, NEU).
Label P ecision Recall F1- sco e
P68% 90% 77%
N42% 19% 26%
NEU 50% 2% 4%
14680394, 2025, 2, Downloaded om h ps://onlinelib a y.wiley.com/doi/10.1111/exsy.13832 by Uni e sidad De Valladolid, Wiley Online Lib a y on [26/02/2025]. See he Te ms and Condi ions (h ps://onlinelib a y.wiley.com/ e ms-and-condi ions) on Wiley Online Lib a y o ules o use; OA a icles a e go e ned by he applicable C ea i e Commons License
9 o 17
om he co pus and inally an adjus men o he ba ch size
pa ame e . All his o adjus he hype pa ame e s and a oid
o e i ing he de eloped model. The es ing phase was ca -
ied ou using se e al examples o pe o mance me ics: p eci-
sion, ecall and F1- sco e.
As a i s s age o he aining phase, he aining p ocess will
be epea ed by a ying he numbe o neu ons and il e s in
he laye s o ind he combina ion ha gi es he bes accu acy
esul s, he alues o which a e shown in Table4. Fo hese
es s, he ke nel size has been se o 8, he d opou pa ame-
e o 0.2 and he ecu en pa ame e o 0.3. The esul s in
Table4 show ha he model is qui e sensi i e o a ia ions in
hese hype pa ame e s, achie ing a highe accu acy a e o
a numbe o 180 in he con olu ion laye and 256 neu ons in
he LSTM laye .
The second s ep in aining he model consis s o a ying he
d opou a es (d opou and ecu en d opou a e) o he LSTM
laye in he ange o 0.2 o 0.8. Acco ding o he Ke as documen-
a ion (Onei os2023), he LSTM laye s ha e wo di e en ypes
o d opou a es, which a e ep esen ed as a loa ing poin num-
be be ween 0 and 1. In addi ion, o hese es s, he ke nel size
was se o 8, he numbe o il e s in he con olu ional laye o
180 and he numbe o neu ons in he LSTM laye o 256. Table5
shows ha a ying he alues gene a es signi ican changes in
he accu acy o he model, and he bes pe o mance, 85.02%, is
achie ed wi h alues 0.2 and 0.3, o he d opou and ecu en
d opou a e pa ame e s espec i ely.
The nex s ep in he model is a sweep o di e en alues o
he lea ning a e. This is a undamen al ac o when aining
machine lea ning models, since i will de e mine he deg ee o
magni ude wi h which adjus men s o he di e en pa ame e s
o he model will be made, which in u n a ec s he con e gence
o he model. The e a e wo main easons why i is in e es ing
o con ol he lea ning a e: o con ol he speed o con e gence
and o o e come local minima ha may occu . I can be seen
in Table6 ha depending on he alue, he e is no conside able
a ia ion in he accu acy me ic o he model when changing he
alue o his pa ame e , wi h he bes alue, 79.01% achie ed o
a lea ning a e o 0.01.
The nex aining s ep is o educe he o al numbe o unique
wo ds in he co pus used in he model o inc ease compu a-
ional e iciency. Howe e , he e is a ade- o be ween he
educ ion o he ocabula y used and he possibili y o los-
ing impo an in o ma ion, so i is necessa y o pe o m he
educ ions in a s epwise manne (in his case in 100- wo d
jumps). Using all wo ds, he model achie ed an accu acy o
FIGURE 3 | Op imisa ion o he numbe o ees.
TABLE 3 | Summa y o he RF model esul s o p ecision, ecall
and F1- sco e me ics conside ing h ee pola i y classes (P, N, NEU).
Label P ecision Recall F1- sco e
P74% 92% 82%
N59% 37% 46%
NEU 40% 8% 14%
TABLE 4 | Resul s o he e alua ion and op imisa ion o he
hype pa ame e s numbe o neu ons and il e s in he hyb id CNN-
LSTM model.
Numbe il e s
con olu ional
laye
Numbe neu ons
LSMT laye
C oss-
alida ion
accu acy
192 256 76.97%
192 128 65.74%
192 96 76.82%
192 64 77.11%
180 256 79.15%
180 96 65.74%
160 128 77.84%
160 64 77.55%
150 256 76.24%
128 128 65.74%
128 64 65.74%
96 64 65.74%
192 150 76.8%
14680394, 2025, 2, Downloaded om h ps://onlinelib a y.wiley.com/doi/10.1111/exsy.13832 by Uni e sidad De Valladolid, Wiley Online Lib a y on [26/02/2025]. See he Te ms and Condi ions (h ps://onlinelib a y.wiley.com/ e ms-and-condi ions) on Wiley Online Lib a y o ules o use; OA a icles a e go e ned by he applicable C ea i e Commons License
16 o 17 Expe Sys ems, 2025
d a , W i ing – e iew and edi ing, Visualisa ion, Supe ision, P ojec
adminis a ion. Cla a González- Sanguino: Concep ualiza ion,
In es iga ion, Da a cu a ion, W i ing – o iginal d a , W i ing – e iew
and edi ing, Visualisa ion. Alba Ayuso- Lancha es: Concep ualiza ion,
Fo mal analysis, In es iga ion, Da a cu a ion, W i ing – o iginal d a ,
W i ing – e iew and edi ing, Visualisa ion.
Acknowledgemen s
This esea ch has been suppo ed by he Uni e si y o Valladolid.
Con lic s o In e es
The au ho s decla e no con lic s o in e es .
Da a A ailabili y S a emen
The de eloped co pus, he labelling decalogue and he machine lea n-
ing algo i hms will be a ailable on a Gi hub eposi o y, h ps:// gi hub.
com/ GCOde elop e / Men a l- Heal h- Da ase , o esea che s o use
hem in con ex s ela ed o men al heal h in social ne wo ks. O he da a
and models will be made a ailable on eques .
Re e ences
Ahmed, A., S. Aziz, C. To o, e al. 2022. “Machine Lea ning Models
o De ec Anxie y and Dep ession Th ough Social Media: A Scoping
Re iew.” Compu e Me hods and P og ams in Biomedicine Upda e 2:
1–9. h ps:// doi. o g/10.1016/j.cmpbup.2022.100066.
A co, d F M P., M. D. Molina- González, L. A. U. López, and M. T.
Ma ín- Valdi ia. 2021. “A Mul i- Task Lea ning App oach o Ha e
Speech De ec ion Le e aging Sen imen Analysis.” IEEE Access 9:
112478–112489. h ps:// doi. o g/ 10. 1109/ ACCESS. 2021. 3103697.
Bi nbaum, M., S. K. E nala, A. Riz i, M. Choudhu y, and J. Kane.
2017. “A Collabo a i e App oach o Iden i ying Social Media Ma ke s
o Schizoph enia by Employing Machine Lea ning and Clinical
App aisals.” Jou nal o Medical In e ne Resea ch 19: e289. h ps:// doi.
o g/ 10. 2196/ jmi . 7956.
Bog ad, S., B. Chen, and R. Ka ulu u. 2022. “T acking Sen imen s
Towa d Fa Accep ance O e a Decade on Twi e .” Heal h In o ma ics
Jou nal 28, no. 1: 14604582211065702. h ps:// doi. o g/ 10. 1177/ 14604
58221 1065702.
Budenz, A., A. Klassen, J. Pu le, E. To , M. Yudell, and P. Massey.
2019. “Men al Illness and Bipola Diso de on Twi e : Implica ions o
S igma and Social Suppo .” Jou nal o Men al Heal h 29: 1–9. h ps://
doi. o g/ 10. 1080/ 09638 237. 2019. 1677878.
Cholle , F. 2021. Deep Lea ning Wi h Py hon. Shel e Island, New Yo k:
Simon and Schus e .
Ch ome Web S o e. 2023. “One Click Commen Ex ac o o IG
- Ch ome Web S o e.” h ps:ch ome.google.com/webs o e/de ail/
commen - expo e /cckachhlpdnncmhlhaep cmmhadmpbgp Accessed
Janua y 26, 2023.
Co igan, P. W., and A. C. Wa son. 2002. “The Pa adox o Sel - S igma
and Men al Illness.” Clinical Psychology: Science and P ac ice 9, no. 1:
35–53. h ps:// doi. o g/ 10. 1093/ clipsy.9. 1. 35.
Da a epo al. 2023. Accessed May 22, 2024. “Digi al 2023: Global
O e iew Repo .” h ps:// da a epo al. com/ epo s/ digi al- 2023- globa
l- o e iew- epo .
Delanys, S., F. Benama a, V. Mo iceau, F. Oli ie , and J. Mo he. 2020.
“Psychia y on Twi e : A Con en Analysis o he Use o Psychia ic
Te ms in F ench.” JMIR Fo ma i e Resea ch 6, no. 2: 1–13. h ps:// doi.
o g/ 10. 2196/ 18539 .
Eu opean Commission. 2023. “Eu oba ome e Su ey 3032.” Accessed
July 20, 2024.
Fodeh, S., T. Li, K. Menczynski, e al. 2019. “Using Machine Lea ning
Algo i hms o De ec Suicide Risk Fac o s on Twi e .” In In e na ional
Con e ence on Da a Mining Wo kshops (ICDMW), 941–948.
Fundación Mu ua. 2023. “In o me de Salud Men al en España 2023.”
Accessed July 12, 2024.
González Sanguino, C., J. Medina, J. Redondo, E. Be egón, L. Valdi ieso-
León, and M. I u ia. 2024. “An Explo a o y C oss- Sec ional S udy on
Men al Heal h Li e acy o Spanish Adolescen s.” BMC Public Heal h 24,
no. 1: 1–10. h ps:// doi. o g/ 10. 1186/ s1288 9- 024- 18933 - 9.
González Sanguino, C., A. B. San os- Olmo, S. Zamo ano, I. Sánchez-
Iglesias, and L. M. Muñoz. 2023. “The S igma o Men al Heal h
P oblems: A C oss- Sec ional S udy in a Rep esen a i e Sample o
Spain.” In e na ional Jou nal o Social Psychia y 69, no. 8: 1928–1937.
h ps:// doi. o g/ 10. 1177/ 00207 64023 1180124.
Google B ain Team. 2023. “Tenso Flow Lib a y.” h ps:// www. enso
low. o g Accessed May 10, 2024.
G onholm, P. C., and G. Tho nic o . 2022. “Impac o Celeb i y Disclosu e
on Men al Heal h- Rela ed S igma.” Epidemiology and Psychia ic Sciences
31, no. e62: 1–5. h ps:// doi. o g/ 10. 1017/ S2045 79602 2000488.
Gun uku, S. C., D. B. Yaden, M. L. Ke n, L. H. Unga , and J. C.
Eichs aed . 2017. “De ec ing Dep ession and Men al Illness on Social
Media: An In eg a i e Re iew.” Cu en Opinion in Beha io al Sciences
18: 43–49. h ps:// doi. o g/ 10. 1016/j. cobeha. 2017. 07. 005.
Hallg en, K. 2012. “Compu ing In e - Ra e Reliabili y o Obse a ional
Da a: An O e iew and Tu o ial.” Tu o ial in Quan i a i e Me hods o
Psychology 8: 23–34. h ps:// doi. o g/ 10. 20982/ qmp. 08.1. p023.
S a is a. 2023. Accessed May 13, 2024. “Ins ag am: dis ibución mun-
dial de usua ios po edad en 2023.” h ps:// es. s a i s a. com/ es ad is ic
a s/ 87525 8/ d is ibuc i on- p o - e da d- de - lo s- u sua ios - mund i a les - de -
ins a g am.
Jain, P., U. Pandey, and E. Roy. 2017. “Pe cei ed E icacy and In en ions
Rega ding Seeking Men al Heal hca e: Impac o Deepika Padukone,
A Bollywood Celeb i y's Public Announcemen o S uggle Wi h
Dep ession.” Jou nal o Heal h Communica ion 22, no. 8: 713–720.
h ps:// doi. o g/ 10. 1080/ 10810 730. 2017. 1343878.
Jilka, S., C. Odoi, J. Bilsen, e al. 2022. “Iden i ying Schizoph enia
S igma on Twi e : A P oo o P inciple Model Using Se ice Use
Supe ised Machine Lea ning.” NPJ Schizoph enia 8: 1. h ps:// doi. o g/
10. 1038/ s4153 7- 021- 00197 - 6.
Joshi, M. L., and N. Kanoongo. 2022. “Dep ession De ec ion Using
Emo ional A i icial In elligence and Machine Lea ning: A Close
Re iew.” Ma e ials Today P oceedings 58: 217–226. h ps:// doi. o g/ 10.
1016/j. ma p . 2022. 01. 467.
La ose, D. T., and C. D. La ose. 2014. Decision T eesch. Vol. 8, 165–186.
Ox o d, UK: John Wiley & Sons, L d.
Lee, S. Y. 2019. “Media Co e age o Celeb i y Suicide Caused by
Dep ession and Inc ease in he Numbe o People Who Seek Dep ession
T ea men .” Psychia y Resea ch 271: 598–603. h ps:// doi. o g/ 10. 1016/j.
psych es. 2018. 12. 055.
Lee, Y., C. Yuan, and D. Wohn. 2021. “How Video S eame s' Men al
Heal h Disclosu es A ec Viewe s' Risk Pe cep ions.” Heal h
Communica ion 36, no. 14: 1931–1941. h ps:// doi. o g/ 10. 1080/ 10410
236. 2020. 1808405.
Lejeune, A., B. M. Robaglia, M. Wal e , S. Be ouigue , and C. Lemey.
2022. “Use o Social Media Da a o Diagnose and Moni o Psycho ic
Diso de s: Sys ema ic Re iew and Pe spec i es (P ep in ).” Jou nal o
Medical In e ne Resea ch 24, no. 9: 1–15. h ps:// doi. o g/ 10. 2196/ 36986 .
Män ylä, M. V., D. G azio in, and M. Kuu ila. 2018. “The E olu ion o
Sen imen Analysis—A Re iew o Resea ch Topics, Venues, and Top
Ci ed Pape s.” Compu e Science Re iew 27: 16–32. h ps:// doi. o g/ 10.
1016/j. cos e . 2017. 10. 002.
14680394, 2025, 2, Downloaded om h ps://onlinelib a y.wiley.com/doi/10.1111/exsy.13832 by Uni e sidad De Valladolid, Wiley Online Lib a y on [26/02/2025]. See he Te ms and Condi ions (h ps://onlinelib a y.wiley.com/ e ms-and-condi ions) on Wiley Online Lib a y o ules o use; OA a icles a e go e ned by he applicable C ea i e Commons License
17 o 17
Ma ínez- Cáma a, E., M. T. Ma ín- Valdi ia, L. A. U eña- López, and
A. R. Mon ejo- Ráez. 2014. “Sen imen Analysis in Twi e .” Na u al
Language Enginee ing 20, no. 1: 1–28. h ps:// doi. o g/ 10. 1017/ S1351
32491 2000332.
Men al Heal h Ame ica. 2023. “Men al Heal h Ame ica: You h Da a
2023.” Accessed July 15, 2024.
Me ayo, N., A. Ayuso, and C. González- Sanguino. 2023. “Reposi o y o
Co pus o Men al Heal h in Spanish Social Ne wo ks.” h ps:// gi hub.
com/ GCOde elop e / Men a l- Heal h- Da ase Accessed June 19, 2024.
Mon, A. d M., C. Dona - Va gas, J. San oma- Vilacla a, e al. 2021.
“Assessmen o An ipsycho ic Medica ions on Social Media: Machine
Lea ning S udy.” F on ie s in Psychia y 12, no. 1: 1–13. h ps:// doi. o g/
10. 3389/ psy . 2021. 737684.
NHS Digi al. 2023. “Men al Heal h o Child en and Young People in
England, 2023 - Wa e 4 Follow Up.” Accessed July 15, 2024.
NLTK P ojec . 2023. “Na u al Language Toolki (NLTK) Lib a y.”
h ps:// www. nl k. o g/ Accessed Ap il 1, 2024.
Noble, W. S. 2006. “Wha Is a Suppo Vec o Machine?” Na u e
Bio echnology 24, no. 12: 1565–1567. h ps:// doi. o g/ 10. 1038/ NBT12
06- 1565.
Oden, A., and L. Po e . 2023. “The Kids A e Online: Teen Social Media
Use, Ci ic Engagemen , and A ec i e Pola iza ion.” Social Media +
Socie y 9, no. 3: 20563051231186364. h ps:// doi. o g/ 10. 1177/ 20563
05123 1186364.
Onei os. 2023. “Ke as Lib a y.” h ps:// ke as. io/ Accessed June 26,
2024.
Osca , N., P. A. Fox, R. C ouche , R. We nick, J. Keune, and K. Hooke .
2017. “Machine Lea ning, Sen imen Analysis, and Twee s: An
Examina ion o Alzheime 's Disease S igma on Twi e .” Jou nals o
Ge on ology: Se ies B 72, no. 5: 742–751. h ps:// doi. o g/ 10. 1093/ ge onb/
gbx014.
Pa lo a, A., and P. Be ke s. 2022. ““Men al Heal h” as De ined by
Twi e : F ames, Emo ions, S igma.” Heal h Communica ion 37, no. 5:
637–647. h ps:// doi. o g/ 10. 1080/ 10410 236. 2020. 1862396.
Pé ez, J. M., D. A. Fu man, L. A. Alemany, and F. M. Luque. 2021.
RoBERTui o: A P e- T ained Language Model o Social Media Tex in
Spanish. P ep in a ailable a a Xi . h ps:// a xi . o g/ abs/ 2111. 09453 .
P obs , P., M. N. W igh , and A. L. Boules eix. 2019. “Hype pa ame e s
and Tuning S a egies o Random Fo es .” Wiley In e disciplina y
Re iews: Da a Mining and Knowledge Disco e y 9, no. 3: e1301. h ps://
doi. o g/ 10. 1002/ widm. 1301.
Py hon So wa e Founda ion. 2021. “Snowball Algo i hms o
S emming.” h ps:// pypi. o g/ p oje c / snowb alls emme Accessed Ap il
15, 2024.
Redondo, J., I. F aga, I. Pad ón, and M. Comesaña. 2007. “The Spanish
Adap a ion o ANEW (A ec i e No ms o English Wo ds).” Beha io
Resea ch Me hods 39: 600–605.
Robinson, P., D. Tu k, S. R. Jilka, and M. Cella. 2018. “Measu ing
A i udes Towa ds Men al Heal h Using Social Media: In es iga ing
S igma and T i ialisa ion.” Social Psychia y and Psychia ic
Epidemiology 54: 51–58. h ps:// doi. o g/ 10. 1007/ s0012 7- 018- 1571- 5.
Shah, S., H. Ghomeshi, E. Vakaj, E. Coope , and R. Mohammad. 2023.
“An Ensemble- Lea ning- Based Technique o Bimodal Sen imen
Analysis.” Big Da a and Cogni i e Compu ing 7, no. 2: 1–20. h ps:// doi.
o g/ 10. 3390/ bdcc7 020085.
S i as a a, R., P. K. Bha i, and P. Ve ma. 2022. “Compa a i e Analysis
o Lexicon and Machine Lea ning App oach o Sen imen Analysis.”
In e na ional Jou nal o Ad anced Compu e Science and Applica ions
13, no. 3: 71–77. h ps:// doi. o g/ 10. 14569/ IJACSA. 2022. 0130312.
Taboada, M., J. B ooke, M. To iloski, K. D. Voll, and M. S ede. 2011.
“Lexicon- Based Me hods o Sen imen Analysis.” Compu a ional
Linguis ics 37: 267–307. h ps:// doi. o g/ 10. 1162/ COLIa 00049 .
Toma , P., K. Ma hu , and U. Suman. 2022. “Unimodal App oaches
o Emo ion Recogni ion: A Sys ema ic Re iew.” Cogni i e Sys ems
Resea ch 77: 94–109. h ps:// doi. o g/ 10. 1016/j. cogsys. 2022. 10. 012.
UNICEF. 2021. “Men al Heal h.” Accessed July 20, 2024.
Wi en, I. H., E. F ank, and M. A. Hall. 2011. Da a Mining: P ac ical
Machine Lea ning Tools and Techniques. The Mo gan Kau mann Se ies
in Da a Managemen Sys ems. Bos on: Mo gan Kau mann.
Wo ld Heal h O ganiza ion. 2024. “Men al Heal h.” Accessed June 10,
2024.
Xue, J., J. Chen, R. Hu, e al. 2020. “Twi e Discussions and Emo ions
Abou he COVID- 19 Pandemic: Machine Lea ning App oach.” Jou nal
o Medical In e ne Resea ch 22, no. 11: e20550. h ps:// doi. o g/ 10. 2196/
20550 .
Suppo ing In o ma ion
Addi ional suppo ing in o ma ion can be ound online in he
Suppo ing In o ma ion sec ion.
14680394, 2025, 2, Downloaded om h ps://onlinelib a y.wiley.com/doi/10.1111/exsy.13832 by Uni e sidad De Valladolid, Wiley Online Lib a y on [26/02/2025]. See he Te ms and Condi ions (h ps://onlinelib a y.wiley.com/ e ms-and-condi ions) on Wiley Online Lib a y o ules o use; OA a icles a e go e ned by he applicable C ea i e Commons License
