Eu opean Sociological Re iew, 2023, 39, 449–463
h ps://doi.o g/10.1093/es /jcac056
Ad ance access publica ion 9 No embe 2022
O iginal A icle
Recei ed: Decembe 2021; e ised: Sep embe 2022; accep ed: Oc obe 2022
© The Au ho (s) 2022. Published by Ox o d Uni e si y P ess.
This is an Open Access a icle dis ibu ed unde he e ms o he C ea i e Commons A ibu ion License (h ps://c ea i ecommons.o g/
licenses/by/4.0/), which pe mi s un es ic ed euse, dis ibu ion, and ep oduc ion in any medium, p o ided he o iginal wo k is p ope ly
ci ed.
Unco e ing hidden opinions: social no ms and he
exp ession o xenophobic a i udes
AmaliaÁl a ez-Benjumea1,2,*,
1Max Planck Ins i u e o Resea ch on Collec i e Goods, Bonn, Ge many
2Chai o Sociological Me hodology, Uni e si y o Mannheim, Mannheim, Ge many
*Co esponding au ho . Email: [email p o ec ed]
Abs ac
Social no ms agains p ejudice a e widesp ead and gene ally suppo ed by socie y, ye examples o bigo y a e o en ound. I p opose
ha an i-p ejudice no ms can quickly e ode when indi iduals a e exposed o ha e con en , he e o e, acili a ing he exp ession o
p ejudice. To es his, pa icipan s we e in i ed o pa icipa e in an expe imen al online o um discussing immig a ion. I compa e he
commen s o pa icipan s exposed o xenophobic con en o hose no exposed. The empi ical esul s show ha exposu e o ha e ul
con en e odes no m compliance: he mo e ha e ul con en pa icipan s could obse e, he mo e ha e ul hei subsequen commen s
we e. The e ec is p ima ily d i en by hose mo e likely o hold an i-immig an iews. This poin s o an ‘emboldening e ec ’ whe eby
p ejudiced indi iduals e ain om exp essing p ejudiced opinions in he absence o o ensi e speech. O he wise, hidden a i udes a e
e ealed. The esul s con i m ha he exp ession o p ejudice is con ex dependen and ha p ese ing a ‘no m en i onmen ’ equi es
sus ained ein o cemen o he no m. Fu he mo e, he esul s show ha he composi ion o he popula ion in e ms o indi idual
belie s is pa amoun o he dynamics o e osion o he social no m. On he whole, hese indings can in o m e ec i e public s a egies
agains he sp ead o ha e speech and o e a new me hodological app oach o s udying ha e speech in online con ex s.
In oduc ion
In he pas yea s, he inc ease in ha e speech, such as ac-
is , misogynis ic, o o he wise disc imina o y messages,
has become a ma e o conce n. This phenomenon has
spa ked a conside able amoun o esea ch, om i s
oo s in populis poli icians (e.g. Giani and Méon 2019;
Newman e al., 2021; Scha ne , 2020) o i s conse-
quences o public opinion (C andall, Mille , and Whi e,
2018; Bu sz yn, Ego o and Fio in, 2020). In gene al,
he e is an inc eased awa eness o he ubiqui y o ha e
and i s ha m ul consequences. A he cen e o he dis-
cussion, he e is online ha e speech, as exposu e o i is
a majo ac o in he inc ease in exp essions o ha e and
p ejudice (So al, Bilewicz and Winiewski, 2018) and a
p ecu so o ha e c imes (e.g. Chan, Ghose and Seamans
2016; Williams e al., 2020; Mülle and Schwa z 2021).
Fa om a inge phenomenon, online ha e speech
is mo e and mo e isible on mains eam social media
pla o ms (Siegel, 2020), including Facebook, YouTube,
and Twi e (Ben-Da id and Fe nández, 2016; Fo una
and Nunes, 2018; Munge and Phillips, 2022). No
only does he hos ili y o commen s on Twi e no
s op hem om being sha ed (Klubička and Fe nández,
2018), bu ha e con en , such as e hnic insul s, migh
sp ead as e h ough online discussions (Spö lein and
Schlue e , 2020). As a consequence, an inc easing num-
be o people a e exposed o ha e ul online con en
(Hawdon, Oksanen and Räsänen, 2017; Kaakinen e
al., 2018; Siegel, 2020).1 The ising online ha e con en
exis s despi e he gene al ag eemen ha ha e ul speech
is undesi able beha iou .
Social no ms p osc ibing he exp ession o p eju-
dice, such as acis o xenophobic speech, a e wide-
sp ead and gene ally heeded in socie y (Mendelbe g,
2001; C andall and S ango , 2005; Paluck and G een,
2009; Ba , Lane and Nosenzo, 2018) In mos cases,
hese no ms a e also s ong p edic o s o exp essed
p ejudice (C andall, Eshleman and O’B ien, 2002).
Howe e , social no ms e ode. This expe imen exam-
ines how he social no m agains he exp ession o
p ejudice weakens when people can obse e ha e ul
commen s om o he s.
Downloaded om h ps://academic.oup.com/es /a icle/39/3/449/6815669 by Cen o de Ciencias Humanas y Sociales use on 16 Ap il 2024
450 ÁLVAREZ-BENJUMEA
A social no m is a sha ed unde s anding o wha is
he app op ia e beha iou in a speci ic si ua ion. The
p e e ence o con o mi y o he no m is condi ional
on ha ing ce ain social expec a ions, an e ec called
condi ional compliance (Bicchie i and Xiao, 2009;
Diekmann, P zepio ka and Rauhu , 2015). I social
expec a ions change, he pull o he no m also changes.
I one can see o he s b eaking he no m, expec a ions
abou how o he s beha e will change. In he con ex o
a social no m agains he exp ession o p ejudice, his
implies ha people ha obse e o he s p oducing ha e
speech, such as acis o xenophobic commen s, will
dec ease hei adhe ence o he no m, and mo e indi-
iduals will oice hei p ejudice. The e is e idence ha
his dynamic migh go e n ha e speech in social media.
Fo example, pas esea ch has ound ha obse -
ing e hnic insul s in commen s sec ions inc eases he
numbe o insul s in subsequen commen s (Spö lein
and Schlue e , 2020), o ha censo ing ha e con en
dec eases he use o ha e in online o ums (Ál a ez-
Benjumea and Win e , 2018).
These s udies emain agnos ic as o he ac o s d i -
ing he obse ed condi ionali y o beha iou . I expec
ha exposu e o ha e con en leads o mo e ha e com-
men s by ac ing as a igge o indi iduals who a e
p edisposed owa ds exp essing p ejudice. These indi-
iduals who p i a ely disag ee wi h he no m migh
quickly eac o exposu e and disclose hei p e iously
hidden opinion once hey eel hei commen s will be
ole a ed. The e o e, whe he a social no m agains he
exp ession o ha e weakens would, o a la ge ex en ,
depend on he composi ion o he popula ion (Cen ola,
Wille and Macy, 2005).
I p esen an expe imen ha explo es he e ec o
exposu e o ha e commen s in an online o um whe e
pa icipan s discuss issues on immig a ion and e ugees
in Ge many, a coun y whe e e ugees and mig a ion
ha e been cen al issues in i s public discou se o he
pas yea s (A l , Schumann and Wolling, 2020), and
whe e p e ious esea ch has linked aising online
an i- e ugee speech wi h an inc emen o iolen c imes
(Mülle and Schwa z, 2021). I in i ed pa icipan s (N
= 2,283) o ake pa in a pu pose-buil online o um
discussing e ugees and immig a ion issues, which
consis ed o i e consecu i e di e en discussions. The
pa icipan s could obse e a selec ion o commen s
om p e ious use s. They we e assigned o one o wo
ea men g oups. In he exposed g oup, pa icipan s
could obse e xenophobic commen s le by p e ious
use s, ha is, no m iola ions. In he unexposed g oup,
he e was no ha e con en . In he second pa o he
expe imen , pa icipan s we e classi ied wi h ega d
o whe he hey we e likely o hold an i-immig an
o p o-immig an iews as a p oxy o hei p i a e
a i ude owa ds he social no m. I es how p i a e
a i udes a ec hei esponse o exposu e o he xen-
ophobic commen s. The commen s le by he pa ici-
pan s in he di e en discussion h eads a e he main
ou come o in e es .
A key challenge is dis inguishing he con agious e ec
o obse ing ha e con en om homophily, whe eby sim-
ila people jus ‘come oge he ’, o example, xenopho-
bic use s seeking ou xenophobic social pla o ms.2 In
his s udy, andomiza ion occu s a wo di e en le els.
Besides andomizing pa icipan s in o ea men s, I an-
domly picked he commen s pa icipan s could obse e
in he o um. The double andomiza ion ensu es ha
homophily is no d i ing he esul s. In addi ion, he
anonymous se ing o he online o um minimizes expec-
a ions o sanc ions om o he use s, which helps o con-
i m ha obse ing xenophobic commen s om p e ious
use s alone d i es he e ec s e en i he an icipa ion o
punishmen emains cons an (Diekmann, P zepio ka
and Rauhu , 2015).
The empi ical esul s show ha pa icipan s exposed
o no m iola ions we e mo e p one o exp essing
xenophobic iews: he la ge he p opo ion o ha e-
ul commen s hey could obse e, he mo e likely hey
we e o iola e he no m and he mo e hos ile hei
commen s. Fu he mo e, al hough exposu e o xeno-
phobic speech seems o exe an e ec on all pa ic-
ipan s, he e ec is la ge amongs hose who we e
deemed mo e likely o hold an i-immig an iews. This
indica es an ‘emboldening e ec ’, ha is, in he absence
o p ejudiced speech, p ejudiced ci izens cons ain he
exp ession o hei p ejudice, bu hey disclose hei
opinions i hey a e exposed o xenophobic con en .
The esul s imply ha no ms can easily weaken and
ha he p ese a ion o a ‘no m en i onmen ’ equi es
sus ained ein o cemen o he no m.
Theo e ical amewo k
Social no ms egula e he exp ession o
p ejudice
Social no ms a e he accep ed s anda ds o beha iou
o a social g oup in a gi en si ua ion. Social no ms
may p esc ibe ce ain ypes o beha iou o cons ain
beha iou ha is conside ed an isocial. Wha people
say (o don’ say) is also egula ed by social no ms
and, he e o e, public opinions do no always mi o
p i a e belie s (Noelle-Neumann, 1974; Ku an, 1995).
The cos o a icula ing a posi ion ha o he s may
ind inapp op ia e cu s ac oss se e al di e en opics,
such as abo ion (Cowan, 2014), poli ical opinions
(Ku an, 1995; Mo ales, 2020), o e en ea ing habi s
(Ki s 2003). Social no ms lead people o selec i ely
disclose con o e sial opinions (Ki s, 2003; Cowan
and Baldassa i, 2018), e aining om doing so when
hey belie e hei iews o be unbecoming in o de o
Downloaded om h ps://academic.oup.com/es /a icle/39/3/449/6815669 by Cen o de Ciencias Humanas y Sociales use on 16 Ap il 2024
451SOCIAL NORMS AND THE EXPRESSION OF XENOPHOBIC ATTITUDES
a oid punishmen (Bicchie i, 2006, p. 11), o ob ain
he app o al o o he s (Cialdini and Golds ein, 2004),
o bo h.
The exp ession o p ejudice is also egula ed by
social no ms. Explici ly acis , xenophobic, misogynis-
ic, o o he wise ha e ul language is conside ed an iso-
cial by a la ge pa o socie y (Beye and Liebe, 2015);
he e o e, social no ms end o p osc ibe hese ypes o
messages in public (Mendelbe g, 2001; Paluck, 2009;
Paluck and G een, 2009; Beye and Liebe, 2015). The
an i-ha e no m can be ound in he li e a u e unde di -
e en names and in e e ence o a a ie y o a ge s,
such as an egali a ian no m (C andall, Eshleman and
O’B ien, 2002), a no m o acial ole ance (Webe e
al., 2014), an an i- acis no m (I a s la en, Blinde and
Fo d, 2010), o an an i-p ejudice no m (Blinde , Fo d
and I a s la en, 2013). The no m agains he exp es-
sion o p ejudice is a social no m ha es ains lan-
guage conside ed socially unaccep able because o i s
ha e ul con en agains a ce ain social g oup.
Some social g oups a e conside ed accep able a ge s
o p ejudice, while o he s a e no . Mos o he ime,
people could somewha eely a icula e hei iews on
Canadians, d ug deale s, o ich people, bu mos people
would supp ess hei p ejudice agains sexual o e hnic
mino i ies. Wha is conside ed an inapp op ia e a ge o
p ejudice is con es ed, changes, and depends on he con-
ex (C andall, Eshleman and O’B ien, 2002; C andall,
Fe guson and Bahns, 2013; C andall, Mille and Whi e,
2018). Fo example, in he Uni ed S a es, he dominan
no m abou acism in he la e 1950s was one o accep -
ance (Ducki , 1992). Since hen, he public exp ession
o acial p ejudice has declined in wes e n socie ies
(Schuman e al., 1997), and su ey esponden s became
less willing o endo se o e acial p ejudices (Huddy
and Feldman, 2009), ul ima ely shaping he way ace is
discussed in public.
Di e ences be ween con ex s a e also b oad.
Eu opean coun ies adi ionally had s ic e iews o
wha is conside ed ha e speech, whe eas in he Uni ed
S a es, mo e weigh is gi en o ee speech (Pe ig ew,
1958; Do idio and Gae ne , 1986; Ducki , 1992). Fo
ins ance, Ge many, in an e o o dis ance i sel om
he nazi pas , ins i u ionalized a ‘cul u e o con i ion’
(A , 2005), which esul s in a widesp ead an i-p ejudice
no m (Blinde , Fo d and I a s la en, 2013). Ge many
also has some o he s ic es laws agains online ha e
speech in he wo ld o cing social media si es o dele e
o ensi e con en (‘Ne zwe kdu chse zungsgese z’).
This s udy deals wi h he exp ession o xenophobia,
unde s ood as p ejudice agains immig an s and e u-
gees. The exp ession o xenophobic iews is nowadays
gene ally conside ed no no ma i ely accep able in
mos coun ies (Blinde , Fo d and I a s la en, 2013).
People, he e o e, end o dis ance hemsel es om
xenophobic labels (Schwa z e al., 2021) and e ain
om exp essing hei an i-immig an iews, especially
when an i-p ejudice no ms a e emphasized (Ál a ez-
Benjumea and Win e , 2020).
Mechanisms o no ma i e change: obse ing
ha e igge s i s sp ead
Social no ms ha e wo di e en componen s. Fi s ,
expec a ions abou wha mos o he s ind app op i-
a e. Second, expec a ions abou wha mos o he s
do, pa icula ly whe he hey will o will no beha e
in con o mi y wi h he no m (Bicchie i, 2006, 2016;
Bicchie i and Diman , 2019). People p e e o ollow
he no m, condi ional on hese wo expec a ions, and
inconsis encies be ween he wo can weaken he pull
o he no m.
Mos o he ime, e en i people ag ee on wha is
app op ia e, hey will only comply wi h he no m i
hey expec o he s o do so as well (Bicchie i and Xiao,
2009), which has been called condi ional compliance
(Bicchie i and Xiao, 2009; Diekmann, P zepio ka and
Rauhu , 2015; Bicchie i and Diman , 2019). I could
be because obse ing o he s misbeha ing educes he
social cos s o no complying o because i changes he
belie s abou he app op ia eness o he beha iou ;
ul ima ely, i we expec o he s o misbeha e, we eel
licensed o do so as well (Gino, Ayal and A iely, 2009;
Diekmann, P zepio ka and Rauhu , 2015). Imagine
inding a sign in you new building elling you e e y-
body ecycles, bu you ind mixed was e in he ecy-
cling con aine s. You will su ely be less likely o ecycle
you sel because he social cos s o no doing so a e
educed: you do no expec you neighbou s o eac
nega i ely. The no m may s ill exis , bu you will eel
less compelled o comply. The e o e, me ely obse -
ing no m iola ions igge s hei sp ead (Diekmann,
P zepio ka and Rauhu , 2015).3
In his s udy, I ocus on he idea ha he beha iou o
o he s se es as a powe ul cue ha signals he s a us
o he no m in a pa icula con ex (Cialdini, Kallg en
and Reno, 1991; Cialdini and Golds ein, 2004). I
one can obse e o he s eely exp essing xenophobic
opinions, one migh eel less p essu e o inhibi hei
xenophobia; he e o e, exposu e o ha e speech will
os e u he ha e. Exis ing e idence co obo a es he
idea ha obse ing ha e speech inc eases he likelihood
o obse e s doing so. Fo example, p e ious expe i-
men s ound ha hea ing o he s endo sing acis iews
p omp ed pa icipan s o do so as well (Blancha d e
al., 1994; S ango , Sech is and Jos , 2001; C andall,
Eshleman and O’B ien, 2002; C andall and S ango ,
2005). The e e se also holds: p o iding people wi h
he expec a ion ha o he s will ollow he no m makes
hem a oid oicing p ejudiced opinions (Paluck and
G een, 2009).
Downloaded om h ps://academic.oup.com/es /a icle/39/3/449/6815669 by Cen o de Ciencias Humanas y Sociales use on 16 Ap il 2024
452 ÁLVAREZ-BENJUMEA
The e is some e idence ha his dynamic may
play ou on social media. Fo example, people who
spend mo e ime on Reddi and Tumbl epo ha
hey dissemina e mo e ha e speech online, especially
hose who spend ime in communi ies whe e ha e
speech is p e alen (Cos ello and Hawdon, 2018).
Fu he mo e, use s who c ea e online ha e con en
end o adicalize, p obably because hey a e exposed
o ha e con en p oduced by o he s and, e en ually,
his beha iou eels less s igma ized (Siegel, 2020).
The shee p esence o e hnic insul s in commen s sec-
ions inc eases he equency o subsequen insul ing
commen s (Spö lein and Schlue e , 2020). Howe e ,
mos o he e idence comes om obse a ional da a,
which makes isola ing he e ec o exposu e o
ha e con en om o he ac o s impossible. The e is
expe imen al e idence ha censo ing ha e commen s
in an online o um dec eases he use o ha e speech
(Ál a ez-Benjumea and Win e , 2018), which poin s
o he idea ha he decision o ollow he no m pa -
ially depends on wha one can obse e o he s com-
monly doing. Ye , his does no necessa ily imply ha
he e e se is ue. I aim speci ically o es whe he
use s on an online o um ha obse e xenophobic
commen s om o he s will be mo e likely o p oduce
xenophobic commen s (H1). This i s hypo hesis
es s he idea ha obse a ion o no m iola ions
educes one’s no m compliance.
One impo an aspec o social no ms is ha hey
in luence beha iou ega dless o indi idual a i udes
(Paluck, 2009; Paluck and G een, 2009), which allows
o dispa i ies be ween p i a e belie s and he p e ail-
ing no m (Diekmann, P zepio ka and Rauhu , 2015).
I he pull o he social no m is s ong enough, e en
hose who p i a ely disag ee wi h he no m comply
wi h i . Howe e , indi iduals whose pe sonal a i udes
disag ee wi h he no ms a e mo e likely o ollow sui
when hey obse e a beha iou al shi in he di ec ion
o hei p i a e belie s. Dinas, Ma ínez and Valen im
(2022) p o ide a compelling example: hey show ha
he public display o a his o ically s igma ized igh -
wing symbol led mo e sympa hize s o use i and peo-
ple on a e age o unde s and i as mo e app op ia e,
despi e poli ical a i udes no changing.
In he case o he social no m agains he exp ession
o p ejudice, I expec ha indi iduals mo e likely o
hold xenophobic a i udes—whose p i a e a i udes
do no align wi h he no m—will be mo e likely han
hose who ag ee wi h he no m o p oduce xenophobic
commen s when hey can obse e p e ious use s’ xen-
ophobic con en (H2). In o he wo ds, I expec mainly
people wi h an i-mino i y sen imen who p o i om
he loosening o he social no m o exp ess hei opin-
ions in an ‘emboldening e ec ’ o he al eady p eju-
diced (Newman e al., 2021).
Unde s anding he in e play be ween indi idual a i-
udes and social no ms can shed ligh on he unde ly-
ing mechanisms o he di usion o ha e online. I only
p ejudiced people ake ad an age o he loosening o
social no ms, hen he sp ead o ha e will ma kedly
depend on he p opo ion o people whose pe sonal
a i udes de ia e om he social no m.
Me hods
Expe imen al design
The expe imen ook place in an online o um de el-
oped speci ically o he expe imen . Fo um pages con-
sis o one image depic ing immig an s and e ugees
ollowed by h ee commen s discussing he pic u e, as
shown in Figu e 1. The h ee commen s on each o um
discussion we e andomly selec ed om a pool o com-
men s (N = 307) collec ed du ing a p e-expe imen al
phase. The e o e, pa icipan s do no in e ac in eal
ime, bu a e only exposed o commen s p e iously
selec ed by he expe imen e . These ini ial commen s
we e ca ego ized by h ee independen a e s in o
h ee ca ego ies: xenophobic, neu al, and iendly. The
collec ion and selec ion o commen s a e explained
in Appendix A. All pa icipan s we e p esen ed wi h
i e consecu i e o um discussions and asked o lea e
a commen on each o he pages. Commen ing was
compulso y. The o de o he pic u es is andomized
be ween pa icipan s.
A he beginning o he expe imen , pa icipan s
we e assigned a use name and a a a and andomly
assigned o one o he wo expe imen al condi ions.
Figu e 2 depic s he ea men s and hei con en . In
he unexposed condi ion, pa icipan s do no obse e
ha e ul commen s, only posi i e and neu al ones. In
he exposed condi ion, pa icipan s a e exposed o
xenophobic commen s. S a ing om o um 2, he
xenophobic commen s inc ease as pa icipan s mo e
h ough he o a as depic ed in Figu e 2. A he end o
he expe imen , e e y pa icipan has seen a o al num-
be o 15 commen s. In he exposed ea men , 66 pe
cen o he o al displayed commen s a e xenophobic.
Pa icipan s in his condi ion a e, he e o e, exposed o
online ha e in an inc emen al manne .
This design allows o es ing H1, ha is, use s on
an online o um ha obse e xenophobic commen s
om o he s will be mo e likely o p oduce xenophobic
commen s, a di e en obse able pe cen ages o no m
iola ions. The equency o he beha iou inc eases
i s pe cei ed app op ia eness and educes he cos o
non-con o ming. As in a h eshold model o beha -
iou (G ano e e , 1978; And eoni, Niki o akis and
Siegen hale , 2021), one should expec he p obabili y
o p oducing a ha e commen o inc ease pas a ce ain
ipping poin . Howe e , I do no make any heo e ical
Downloaded om h ps://academic.oup.com/es /a icle/39/3/449/6815669 by Cen o de Ciencias Humanas y Sociales use on 16 Ap il 2024
453SOCIAL NORMS AND THE EXPRESSION OF XENOPHOBIC ATTITUDES
Figu e 1. Sc eensho o he online o um (ou ansla ion; in Ge man in he o iginal). Image c edi : Shu e s ock/Punghi.
Downloaded om h ps://academic.oup.com/es /a icle/39/3/449/6815669 by Cen o de Ciencias Humanas y Sociales use on 16 Ap il 2024

454 ÁLVAREZ-BENJUMEA
assump ion o wha he ipping poin o a su icien
amoun o no m iola ions migh be.
No o he no ma i e in o ma ion o use ules we e
p o ided. The ea men , he o de o he images, and
he speci ic commen s in he commen h eads a e an-
domized amongs pa icipan s. A e he expe imen ,
pa icipan s comple ed a b ie demog aphic ques-
ionnai e and ecei ed a code o claim hei paymen .
IRB app o al o he s udy was g an ed by he E hics
Commission o he Uni e si y o Bonn: IRB app o al
numbe 140/18.
Pa icipan s we e ec ui ed om a c owd-sou cing
In e ne ma ke place.4 The expe imen was conduc ed
en i ely in Ge man and he sample was es ic ed o
esiden s in Ge many. The demog aphic cha ac e is ics
o he pa icipan s a e desc ibed in Appendix A.5 Da a
we e collec ed online be ween he 9 h and he 20 h o
No embe 2018. The ea men s an simul aneously.
The o al numbe o pa icipan s is 2,283, wi h 1,170
in he Unexposed ea men and 1113 in he Exposed
ea men . Table A4 (Appendix B in Supplemen a y
Ma e ial) shows ha indi idual cha ac e is ics a e bal-
anced ac oss he wo expe imen al condi ions. A o al
o 11,320 commen s we e collec ed (see Table 1).6,7
Classi ica ion o he commen s: a c owd-
sou ced analysis
A c owd-sou ced ex anno a ion me hod was used
o classi y he commen s (Benoi e al., 2016). A o al
o 2,748 non-expe a e s we e ec ui ed online and
asked o label a se o commen s. The a e s used a sim-
ple scheme o labelling. Fi s , he scheme asks a e s
o classi y each commen ollowing a 9-poin scale,
whe e 1 is e y iendly language and 9 is e y hos ile
language owa ds he p o ec ed g oup, namely e ugees
and immig an s. The e o e, he highe he sco e, he
mo e xenophobic he commen . The classi ica ion seeks
o delimi ha e speech as p ejudice agains a pa icula
g oup and di e en ia e om ela ed concep s, such as
gene ally agg essi e language o nega i e emo ional
con en , which has been shown o sp ead in social ne -
wo ks (K ame , Guillo y and Hancock, 2014). Second,
a e s we e also asked whe he hey hough he com-
men should be banned om he online o um.
Each a e classi ied 30 andomly selec ed commen s
and hen was gi en he op ion o label 30 mo e. Ra e s
used a a ing sc een like he one depic ed in Figu e 3
and comple ed he ask wi hin 24 hou s. Each a e
labelled an a e age o 40.4 commen s, which yields a
o al o 112,472 a ings. The c owd-sou ced analysis
has wo ad an ages: he da a-gene a ion p ocess is
as e en o la ge amoun s o da a, and i migh be
easie o eplica e and ep oduce (Benoi e al., 2016).
Fu he mo e, eliabili y does no g ea ly di e be ween
manual code s and c owd-sou ced code s (Benoi
e al., 2016; Lind, G ube and Boomgaa den, 2017;
Ho n, 2019).
The indi idual sco es assigned o he commen s by
he a e s a e a e aged o cons uc a mean ha e sco e
o each commen . Commen s ha mo e han 50 pe
cen o he a e s s a ed ha ‘i should be banned’
we e ca ego ized as banned, a bina y a iable. The
Figu e 2. Di e en expe imen al ea men s and hei con en . The e a e wo expe imen al condi ions, exposed and unexposed, each wi h
i e consecu i e o a. In he unexposed condi ion, each o um ea u es one posi i e and wo neu al commen s. In he exposed condi ion,
xenophobic commen s a e added a e Fo um 2. As a esul , exposed pa icipan s obse e 33 pe cen o xenophobic commen s in Fo um
2, 66 pe cen o xenophobic commen s in Fo um 3, and 100 pe cen o xenophobic commen s in Fo a 4 and 5. Exposed pa icipan s
obse e a cumula i e pe cen age o xenophobic commen s o 16 pe cen in Fo um 2, 3 pe cen in Fo um 3, 50 pe cen in Fo um 4,
and inally 66 pe cen o commen s in Fo um 5.
Table 1 Dis ibu ion o he o al numbe o commen s (N = 11,320)
in he o um pages in he Unexposed and Exposed g oups. The
expe imen has a o al o 10 di e en possible combina ions
(2×5).
T ea men Fo um page numbe To al
Fo um 1 Fo um 2 Fo um 3 Fo um 4 Fo um 5
Unexposed 1,185 1,144 1,179 1,157 1,150 5,815
Exposed 1,151 1,128 1,070 1,127 1,029 5,505
N2,336 2,272 2,249 2,284 2,179 11,320
Downloaded om h ps://academic.oup.com/es /a icle/39/3/449/6815669 by Cen o de Ciencias Humanas y Sociales use on 16 Ap il 2024
455SOCIAL NORMS AND THE EXPRESSION OF XENOPHOBIC ATTITUDES
banned commen s ac as a p oxy o no m iola ions.
Commen s ecei ed 9.94 a ings on a e age (SD =
3.18).8 Commen s wi h a sco e o one end o be ol-
e an and sympa he ic (e.g. ‘Hope ully e e yone will
be sa ed and no one will be ha med’ o ‘You ha e o
accep and espec e e y eligion. No ma e whe e!’),
while he highes sco es and commen s ma ked as ‘i
should be banned’ no mally con ain abusi e o big-
o ed language (e.g. ‘I hink he ence is no bad because
we some imes ha e hese wild boa s in ou ga den’
o ‘D own he e min’).9 The e is a signi ican ela-
ionship be ween he banned commen s and he ha e
sco e. Banned commen s ha e, on a e age, a sco e o
7.30 (SD = 1.12) while non-banned commen s ha e a
mean sco e o 3.86 (SD = 1.49). A logis ic model o
p edic banned commen s shows a s a is ically signi -
ican e ec o sco e in banned commen s (b = 1.75,
95% CI [1.59, 1.92], P < 0.001. The model’s explan-
a o y powe is subs an ial (Tju ’s
R2=0.31
). Based
on his model, he p edic ed p obabili y o a commen
wi h a sco e o 9 being banned is 0.84 (95% CI [0.79,
0.89]).
I es ed whe he he leng h o he commen s di e s by
ea men , as di e ences in leng h may indica e di e -
ences in con en and s yle. Commen s ha e on a e age
158 cha ac e s (
SD =103.5
) wi h no signi ican di -
e ences be ween he g oups (
b=0.04
,
SE =0.03
,
P=0.19
). The leng h o he commen s dec eases
ac oss o um pages wi h no di e ences be ween he
exposed and he unexposed pa icipan s (see Table A16
in Supplemen a y Ma e ial). Finally, commen s ha
ecei ed 50 pe cen o mo e o No Applicable (NAs)
a ings (e.g. gibbe ish, s ings o cha ac e s, o copy-
pas ed messages) a e no included in he analyses (4.9
pe cen ), which lea es a o al o 10,771 commen s.
The e a e no signi ica i e di e ences in he dis ibu-
ion o he NA commen s amongs he ea men s (
χ2
=
0.32, d = 1, N = 11,320,
P<0.57
, wo-sided es ).
No m-elici a ion ask
I use a no m-elici a ion p ocedu e o es ablish
whe he he xenophobic commen s a e unde s ood
as no m iola ions in he online o um. In he ask, I
ask pa icipan s o desc ibe hei pe sonal no ma i e
belie s and hei no ma i e expec a ions, o iden-
i y whe he xenophobic commen s a e conside ed
socially inapp op ia e. The no m-elici a ion ask was
un wi h pa icipan s ec ui ed om he same popu-
la ion as he pa icipan s in he expe imen al online
o um.
Figu e 3. Sc eensho o he a ing scheme used by he a e s (ou ansla ion; in Ge man in he o iginal). The sc eensho only shows he
a ing scheme o he sco e. A e his, a e s we e also asked whe he hey hough he commen should be banned om he online
o um, o he cha ac e is ics o he commen s, and comple ed a sho demog aphic ques ionnai e. Image c edi : Shu e s ock/Ve e idis
Vasilis.
Downloaded om h ps://academic.oup.com/es /a icle/39/3/449/6815669 by Cen o de Ciencias Humanas y Sociales use on 16 Ap il 2024
456 ÁLVAREZ-BENJUMEA
The ask, adap ed om Bicchie i and Cha ez (2010),
has wo di e en pa s. The i s expe imen elici s he
pa icipan s’ pe sonal no ma i e belie s, and he sec-
ond measu ed hei no ma i e expec a ions abou
beha iou in he o um. I selec ed a se o i e com-
men s: h ee xenophobic/banned commen s, one neu-
al commen , and one iendly commen . To elici
pe sonal no ma i e belie s and no ma i e expec a-
ions, I ec ui ed wo di e en g oups o subjec s and
desc ibed he online o um o hem. I asked he i s
g oup o subjec s (N = 105) o epo how socially
app op ia e hey would conside each o he commen s
o he selec ion o be le in he o um (i.e. hei pe -
sonal no ma i e belie s). The commen s could be ma k
as Ex emely socially inapp op ia e, Ve y socially
inapp op ia e, Mode a ely socially inapp op ia e, and
No a all socially inapp op ia e. I hen asked he sec-
ond g oup o subjec s (N = 100) o p edic wha hey
hough he majo i y o pa icipan s in he i s g oup
had s a ed (i.e. no ma i e expec a ions). In he second
ask, pa icipan s we e paid a ewa d i hei no ma i e
expec a ions ma ched he pe sonal no ma i e belie s o
he i s g oup.
In he second ask, 92.7 pe cen o pa ici-
pan s ma ked he xenophobic/banned commen s as
Ex emely socially inapp op ia e o Ve y socially inap-
p op ia e.10 The ag eemen o he no ma i e expec a-
ions is an indica o o he exis ence o a social no m
and ha pa icipan s pe cei e he xenophobic/banned
commen s as socially inapp op ia e. The esul s o he
elici a ion ask a e depic ed in Figu es A2 and A3.
Resul s
Exposu e o online ha e con en dec eases
one’s own no m compliance
I exposu e o ha e ul commen s igge s he sp ead
o ha e, we should expec he ha e sco e o be la ge
amongs he exposed pa icipan s. To es his hypo h-
esis, I es ima e a linea eg ession model wi h andom
in e cep s o pa icipan s o he ollowing o m:
Yij =β0+β1Exposu eij +uj+ij,
whe e
Yij
is he ha e sco e o commen
i
, o pa icipan
j
.
Consis en wi h H1, pa icipan s exposed o online
ha e con en had an o e all ha e sco e 0.37 poin s
la ge han hose no exposed o i (
SE =0.05
,
P<0.001
).11 The ull esul s a e depic ed in Table A7
in Supplemen a y Ma e ial. Nex , I examine whe he
he e ec depends on he numbe o xenophobic com-
men s pa icipan s can see. In he exposed condi ion,
pa icipan s a e exposed o xenophobic commen s
s a ing om Fo um 2. The numbe o xenophobic
commen s inc eases as pa icipan s mo e h ough he
o a. Exposed pa icipan s obse e a cumula i e pe -
cen age o xenophobic commen s o 16 pe cen in
Fo um 2, 33 pe cen in Fo um 3, 50 pe cen in Fo um
4, and 66 pe cen in Fo um 5.
The aincloud plo (Allen e al., 2019) in Panel A
in Figu e 4 shows he mean es ima es (e o ba s)
o he ha e sco e ac oss o um pages in he exposed
and unexposed g oups, in ed and g een, espec i ely.
Figu e 4. The plo p o ides an o e iew o he aw ha e sco e (do s), i s p obabili y dis ibu ion, median and in e -quan ile ange, and
he mean es ima es wi h he 95 pe cen con idence in e als o he Exposed and he Unexposed g oups ac oss all o um pages. The
aw da a and cen al endency measu ed a e plo ed on (A) he le plo ; he di e ences in di e ences es ima es a e plo ed on (B) he
igh plo . Di -in-Di (Fo um × Exposu e) coe icien s a e depic ed as do s; 95 pe cen con idence in e als a e indica ed by he ends o
he e ical e o ba s. The igh plo shows he Di -in-Di Es ima es om a linea eg ession andom-e ec s model wi h a pa icipan -
speci ic in e cep (N = 10,771). The unexposed ea men se es as he e e ence ca ego y. S a s indica e signi ican di e ences a he
con en ional le els,
∗∗∗ P<0.001
,
∗∗ P<0.01
, and
∗P<0.05
Downloaded om h ps://academic.oup.com/es /a icle/39/3/449/6815669 by Cen o de Ciencias Humanas y Sociales use on 16 Ap il 2024
457SOCIAL NORMS AND THE EXPRESSION OF XENOPHOBIC ATTITUDES
Pa icipan s in bo h g oups begin wi h simila hos il-
i y measu emen s in Fo um 1 when no ha e con en is
shown, bu he di e ence be ween he g oups g ows
as xenophobic commen s inc ease in he exposed con-
di ion. I hen compa e he beha iou o exposed and
unexposed pa icipan s ac oss o um pages (Fo um
Page × Exposu e) and use he unexposed g oup as he
coun e ac ual. Fo um page used as a p edic o is hus
a p oxy o he p opo ion o xenophobic commen s
obse ed by pa icipan s. Again, I es ima e a linea
eg ession model wi h a andom in e cep o pa ic-
ipan s. The model has he ollowing o m:
Y
ij
=β
0
+β
1
Exposu e
ij
+β
2
Type
ij
+β3

Exposu eij
×
Typeij

+uj+ij
,
whe e
Yij
is he ha e sco e o commen
i
, o pa icipan
j
. Panel B in Figu e 4 shows he di e ence-in-di e ence
es ima es o he linea eg ession wi h Fo um Page ×
Exposu e as he p edic o . Exposed pa icipan s p o-
duced signi ican ly mo e hos ile commen s in Fo um
3 and a e wa ds. The slope o he di -in-di es ima-
o is only signi ican ly la ge o he changes be ween
Fo a 2 and 3 (
b=0.27
,
SE =0.07
,
P<0.001
) com-
pa ed o he changes be ween Fo a 1 and 2. The di e -
ences be ween unexposed and exposed o ums emain
cons an a e his (be ween Fo a 3 and 4 [
b=0.06
,
SE =0.07
,
P=0.46
], and be ween Fo a 4 and 5
[
b=0.04
,
SE =0.07
,
P=0.59
]). So, he di e ences
be ween he unexposed and exposed g oups do no
signi ican ly g ow la ge a e Fo um 3. This means
ha pa icipan s need o obse e a minimum cumula-
i e pe cen age o 33 pe cen o xenophobic commen s
be o e he e ec becomes signi ican .
Do hese changes o igina e in an inc ease in non-com-
pliance? I exposu e o ha e con en weakens he pull
o he social no m, i should igge u he no m
iola ions. I analyse he commen s classi ied as banned.
The bina y na u e o his a iable p o ides a cleane
es o his e ec . Figu e 5 shows he aw equency o
banned commen s o exposed and unexposed pa ici-
pan s. To es H1, I es ima e a linea p obabili y model
wi h a andom in e cep o pa icipan s wi h he same
o m as Equa ion 1, bu whe e
Yij
is he p obabili y
ha commen
i
o pa icipan
j
is a banned commen .
The ela i e equency o banned commen s mo e han
doubles in he exposed g oup as i inc eases om 1.3
o 3.1 pe cen (
b=0.02, SE =0.004, P<0.001
) as
shown in model 1 in Table 2, which is consis en wi h
H1.12
Nex , I es ima e he same model as in Equa ion 2
wi h Fo um Page × Exposu e as he main p edic o ,
bu whe e
Yij
is he p obabili y ha commen
i
o pa -
icipan
j
is a banned commen . The esul s a e shown
in Model 2 in Table 2.13 The di e ence be ween he
exposed and unexposed pa icipan s is signi ican in
Fo um 3 (
b=0.02
,
SE =0.01
,
P=0.03
) and hen
again in Fo um 5 (
b=0.03
,
SE =0.01
,
P<0.001
). In
Fo um 4, he di e ence is signi ican only a he 10
pe cen le el (
b=0.01
,
SE =0.01
,
P<0.09
). As wi h
he ha e sco e, pa icipan s needed o obse e a leas
wo xenophobic commen s o a cumula i e 33 pe cen
be o e hei p obabili y o p oducing a banned com-
men inc ease. Obse ing u he commen s does no
seem o p oduce a la ge e ec .
The e ec condi ioned on indi idual a i udes
Pa icipan s in he expe imen a e classi ied based on
how likely hey a e o hold an i-immig an belie s. Fo
his classi ica ion, hal o he pa icipan s we e asked
whe he o no hey would ag ee o dona e a small
amoun o money o a well-known an i-immig an
o ganiza ion, and he o he hal we e asked whe he
o no hey would ag ee o dona e o a well-known
p o-immig a ion o ganiza ion.14 Each pa icipan was
asked only once a a andomly selec ed ime du ing
hei pa icipa ion in he online o um. They we e old
in ad ance ha hey would be asked his ques ion
abou a andomly chosen o ganiza ion. The o gani-
za ions we e p esen ed in sepa a e ques ions o a oid
a demand e ec , as p esen ing bo h op ions oge he
could make pa icipan s dis us ul o he esea che .
The pa icipan s we e classi ied in o wo g oups:
hose who dona ed o he p o-immig a ion o ganiza-
ion we e classi ied as mo e likely o hold p o-immi-
g a ion iews (N = 3,478, 31 pe cen ), while hose
who dona ed o he an i-immig a ion o ganiza ion
we e classi ied as mo e likely o hold an i-immig a ion
iews (N = 927, 8 pe cen ). Those who decided no o
dona e a e unclassi ied, as no dona ing does no e eal
any in o ma ion abou he a i udes owa ds immi-
g a ion (N = 6,915, 61 pe cen ). The idea behind he
0.0%
2.5%
5.0%
7.5%
10.0%
12345
Fo um Page
F equency o
Banned Commen s
Unexposed
Exposed
Figu e 5. F equency o commen s classi ied as ‘banned’ om he
online o um by he a e s in he Exposed and Unexposed g oups.
Figu es a e shown in pe cen ages o e he o al. Nine y- i e pe
cen con idence in e als a e indica ed by he ends o he e ical
e o ba s (N = 10,771).
Downloaded om h ps://academic.oup.com/es /a icle/39/3/449/6815669 by Cen o de Ciencias Humanas y Sociales use on 16 Ap il 2024