P oceedings o he 1s Wo kshop on Gende Bias in Na u al Language P ocessing, pages 33–39
Flo ence, I aly, Augus 2, 2019. c
2019 Associa ion o Compu a ional Linguis ics
33
E alua ing he Unde lying Gende Bias in
Con ex ualized Wo d Embeddings
Ch is ine Bas a Ma a R. Cos a-juss`
a Noe Casas
Uni e si a Poli `
ecnica de Ca alunya
{ch is ine. aou .saad.bas a,ma a. uiz,noe.casas}@upc.edu
Abs ac
Gende bias is highly impac ing na u al lan-
guage p ocessing applica ions. Wo d embed-
dings ha e clea ly been p o en bo h o keep
and ampli y gende biases ha a e p esen in
cu en da a sou ces. Recen ly, con ex ual-
ized wo d embeddings ha e enhanced p e i-
ous wo d embedding echniques by compu ing
wo d ec o ep esen a ions dependen on he
sen ence hey appea in.
In his pape , we s udy he impac o his con-
cep ual change in he wo d embedding compu-
a ion in ela ion wi h gende bias. Ou analy-
sis includes di e en measu es p e iously ap-
plied in he li e a u e o s anda d wo d em-
beddings. Ou indings sugges ha con ex u-
alized wo d embeddings a e less biased han
s anda d ones e en when he la e a e debi-
ased.
1 In oduc ion
Social biases in machine lea ning, in gene al and
in na u al language p ocessing (NLP) applica ions
in pa icula , a e aising he ala m o he scien-
i ic communi y. Examples o hese biases a e
e idences such ha ace ecogni ion sys ems o
speech ecogni ion sys ems wo k be e o whi e
men han o e hnic mino i ies (Buolamwini and
Geb u,2018). Examples in he a ea o NLP a e
he case o machine ansla ion ha sys ems end
o igno e he co e e ence in o ma ion in bene i o
a s e eo ype (Fon and Cos a-juss`
a,2019) o sen-
imen analysis whe e highe sen imen in ensi y
p edic ion is biased o a pa icula gende (Ki -
i chenko and Mohammad,2018).
In his wo k we ocus on he pa icula NLP a ea
o wo d embeddings (Mikolo e al.,2010), which
ep esen wo ds in a nume ical ec o space. Wo d
embeddings ep esen a ion spaces a e known o
p esen geome ical phenomena mimicking ela-
ions and analogies be ween wo ds (e.g. man is o
woman as king is o queen). Following his p op-
e y o inding ela ions o analogies, one popula
example o gende bias is he wo d associa ion be-
ween man o compu e p og amme as woman o
homemake (Bolukbasi e al.,2016). P e- ained
wo d embeddings a e used in many NLP down-
s eam asks, such as na u al language in e ence
(NLI), machine ansla ion (MT) o ques ion an-
swe ing (QA). Recen p og ess in wo d embed-
ding echniques has been achie ed wi h con ex-
ualized wo d embeddings (Pe e s e al.,2018)
which p o ide di e en ec o ep esen a ions o
he same wo d in di e en con ex s.
While gende bias has been s udied, de ec ed
and pa ially add essed o s anda d wo d embed-
dings echniques (Bolukbasi e al.,2016;Zhao
e al.,2018a;Gonen and Goldbe g,2019), i is no
he case o he la es echniques o con ex ualized
wo d embeddings. Only jus ecen ly, Zhao e al.
(2019) p esen a i s analysis on he opic based on
he p oposed me hods in Bolukbasi e al. (2016).
In his pape , we u he analyse he p esence o
gende biases in con ex ualized wo d embeddings
by means o he p oposed me hods in Gonen and
Goldbe g (2019). Fo his, in sec ion 2we p o-
ide an o e iew o he ele an wo k on which
we build ou analysis; in sec ion 3we s a e he
speci ic eques ques ions add essed in his wo k,
while in sec ion 4we desc ibe he expe imen al
amewo k p oposed o add ess hem and in sec-
ion 5we p esen he ob ained and discuss he e-
sul s; inally, in sec ion 6we d aw he conclusions
o ou wo k and p opose some u he esea ch.
2 Backg ound
In his sec ion we desc ibe he ele an NLP ech-
niques used along he pape , including wo d em-
beddings, hei debiased e sion and con ex ual-
ized wo d ep esen a ions.
34
2.1 Wo ds Embeddings
Wo d embeddings a e dis ibu ed ep esen a ions
in a ec o space. These ec o s a e no mally
lea ned om la ge co po a and a e hen used in
downs eam asks like NLI, MT, e c. Se e al ap-
p oaches ha e been p oposed o compu e hose
ec o ep esen a ions, wi h wo d2 ec (Mikolo
e al.,2013) being one o he dominan op ions.
Wo d2 ec p oposes wo a ian s: con inuous bag
o wo ds (CBoW) and skipg am, bo h consis ing
o a single hidden laye neu al ne wo k ained on
p edic ing a a ge wo d om i s con ex wo ds o
CBoW, and he opposi e o he skipg am a ian .
The ou come o wo d2 ec is an embedding able,
whe e a nume ic ec o is associa ed o each o he
wo ds included in he ocabula y.
These ec o ep esen a ions, which in he end
a e compu ed on co-occu ence s a is ics, exhibi
geome ic p ope ies esembling he seman ics o
he ela ions be ween wo ds. This way, sub ac -
ing he ec o ep esen a ions o wo ela ed wo ds
and adding he esul o a hi d wo d, esul s in a
ep esen a ion ha is close o he applica ion o he
seman ic ela ionship be ween he wo i s wo ds
o he hi d one. This applica ion o analogical e-
la ionships ha e been used o showcase he bias
p esen in wo d embeddings, wi h he p o o ypical
example ha when sub ac ing he ec o ep e-
sen a ion o man om ha o compu e and adding
i o woman, we ob ain homemake .
2.2 Debiased Wo d Embeddings
Human-gene a ed co po a su e om social bi-
ases. Those biases a e e lec ed in he co-
occu ence s a is ics, and he e o e lea ned in o
wo d embeddings ained in hose co po a, ampli-
ying hem (Bolukbasi e al.,2016;Caliskan e al.,
2017).
Bolukbasi e al. (2016) s udied om a geome -
ical poin o iew he p esence o gende bias in
wo d embeddings. Fo his, hey compu e he sub-
space whe e he gende in o ma ion concen a es
by compu ing he p incipal componen s o he di -
e ence o ec o ep esen a ions o male and e-
male gende -de ining wo d pai s. Wi h he gende
subspace, he au ho s iden i y di ec and indi ec
biases in p o ession wo ds. Finally, hey mi iga e
he bias by nulli ying he in o ma ion in he gen-
de subspace o wo ds ha should no be associ-
a ed o gende , and also equalize hei dis ance o
bo h elemen s o gende -de ining wo d pai s.
Zhao e al. (2018b) p oposed an ex ension
o GloVe embeddings (Penning on e al.,2014)
whe e he loss unc ion used o ain he embed-
dings is en iched wi h e ms ha con ine he gen-
de in o ma ion o a speci ic po ion o he embed-
ded ec o . The au ho s e e o hese pieces o
in o ma ion as p o ec ed a ibu es. Once he em-
beddings a e ained, he gende p o ec ed a ibu e
can be simply emo ed om he ec o ep esen a-
ion, he e o e elimina ing any gende bias p esen
in i .
The ans o ma ions p oposed by bo h Boluk-
basi e al. (2016) and Zhao e al. (2018b) a e down-
s eam ask-agnos ic. This ac is used in he wo k
o Gonen and Goldbe g (2019) o showcase ha ,
while appa en ly he embedding in o ma ion is e-
mo ed, he e is s ill gende in o ma ion emaining
in he ec o ep esen a ions.
2.3 Con ex ualized Wo d Embeddings
P e ained Language Models (LM) like ULM i
(Howa d and Rude ,2018), ELMo (Pe e s e al.,
2018), OpenAI GPT (Rad o d,2018;Rad o d
e al.,2019) and BERT (De lin e al.,2018), p o-
posed di e en neu al language model a chi ec-
u es and made hei p e- ained weigh s a ail-
able o ease he applica ion o ans e lea ning
o downs eam asks, whe e hey ha e pushed he
s a e-o - he-a o se e al benchma ks including
ques ion answe ing on SQuAD, NLI, c oss-lingual
NLI and named iden i y ecogni ion (NER).
While some o hese p e- ained LMs, like
BERT, use subwo d le el okens, ELMo p o ides
wo d-le el ep esen a ions. Pe e s e al. (2019)
and Liu e al. (2019) con i med he iabili y o
using ELMo ep esen a ions di ec ly as ea u es
o downs eam asks wi hou e- aining he ull
model on he a ge ask.
Unlike wo d2 ec ec o ep esen a ions, which
a e cons an ega dless o hei con ex , ELMo
ep esen a ions depend on he sen ence whe e he
wo d appea s, and he e o e he ull model has o
be ed wi h each whole sen ence o ge he wo d
ep esen a ions.
The neu al a chi ec u e p oposed in ELMo (Pe-
e s e al.,2018) consis s o a cha ac e -le el con-
olu ional laye p ocessing he cha ac e s o each
wo d and c ea ing a wo d ep esen a ion ha is
hen ed o a 2-laye bidi ec ional LSTM (Hoch e-
i e and Schmidhube ,1997), ained on language
modeling ask on a la ge co pus.
35
3 Resea ch ques ions
Gi en he high impac o con ex ualized wo d em-
beddings in he a ea o NLP and he social con-
sequences o ha ing biases in such embeddings,
in his wo k we analyse he p esence o bias in
hese con ex ualized wo d embeddings. In pa ic-
ula , we ocus on gende biases, and speci ically
on he ollowing ques ions:
•Do con ex ualized wo d embeddings exhibi
gende bias and how does his bias compa e
o s anda d and debiased wo d embeddings?
•Do di e en e alua ion echniques iden i y
bias simila ly and wha would be he bes
measu e o use o gende bias de ec ion in
con ex ualized embeddings?
To add ess hese ques ions, we adap and con-
as wi h he e alua ion measu es p oposed by
Bolukbasi e al. (2016) and Gonen and Goldbe g
(2019).
4 Expe imen al F amewo k
As ollows, we de ine he da a and esou ces ha
we use o pe o ming ou expe imen s. The ap-
p oach mo i a ion is applying he expe imen s on
con ex ualized wo d embeddings.
We wo ked wi h he English-Ge man news co -
pus om he WMT181. We used he English side
wi h 464,947 lines and 1,004,6125 okens.
To pe o m ou analysis, we used a se o lis s
om p e ious wo k (Bolukbasi e al.,2016;Go-
nen and Goldbe g,2019). We e e o he lis o
de ini ional pai s 2as ‘De ini onal Lis ’ (e.g. she-
he,gi l-boy). We e e o he lis o emale and
male p o essions 3as ‘P o essional Lis ’ (e.g. ac-
coun an ,su geon). The ‘Biased Lis ’ is he lis
used in he clus e ing expe imen and i consis s o
biased male and emale wo ds (500 emale biased
okens and 500 male biased oken). This lis is
gene a ed by aking he mos biased wo ds, whe e
he bias o a wo d is compu ed by aking i s p ojec-
ion on he gende di ec ion (−→
he-−→
she) (e.g. b eas -
eeding,b idal and die o emale and he o,ciga
and eamma es o male). The ‘Ex ended Biased
1h p://da a.s a m .o g/
wm 18/ ansla ion- ask/
aining-pa allel-nc- 13. gz
2h ps://gi hub.com/ olga-b/debiaswe/
blob/mas e /da a/de ini ional_pai s.json
3h ps://gi hub.com/ olga-b/debiaswe/
blob/mas e /da a/p o essions.json
Lis ’ is he lis used in classi ica ion expe imen ,
which con ains 5000 male and emale biased o-
kens, 2500 o each gende , gene a ed in he same
way o he Biased Lis 4. A no e o be conside ed,
is ha he lis s we used in ou expe imen s (and
ob ained om Bolukbasi e al. (2016) and Gonen
and Goldbe g (2019)) may con ain wo ds ha a e
missing in ou co pus and so we canno ob ain
con ex ualized embeddings o hem.
Among di e en app oaches o con ex ualized
wo d embeddings (men ioned in sec ion 2), we
choose ELMo (Pe e s e al.,2018) as con ex ual-
ized wo d embedding app oach. The mo i a ion
o using ELMo ins ead o o he app oaches like
BERT (De lin e al.,2018) is ha ELMo p o ides
wo d-le el ep esen a ions, as opposed o BERT’s
subwo ds. This makes i possible o s udy he
wo d-le el seman ic ai s di ec ly, wi hou eso -
ing o ex a s eps o compose wo d-le el in o ma-
ion om he subwo ds ha could in e e e wi h
ou analyses.
5 E alua ion measu es and esul s
The e is no s anda d measu e o gende bias, and
e en less o such he ecen ly p oposed con ex u-
alized wo d embeddings. In his sec ion, we adap
gende bias measu es o wo d embedding me h-
ods om p e ious wo k (Bolukbasi e al.,2016)
and (Gonen and Goldbe g,2019) o be applicable
o con ex ualized wo d embeddings.
We s a by compu ing he gende subspace
om he ELMo ec o ep esen a ions o gende -
de ining wo ds, hen iden i y he p esence o di ec
bias in he con ex ualized ep esen a ions. We hen
p oceed o iden i y gende in o ma ion by means
o clus e ing and classi ica ions echniques. We
compa e ou esul s o p e ious esul s om debi-
ased and non-debiased wo d embeddings (Boluk-
basi e al.,2016) .
De ec ing he Gende Space Bolukbasi e al.
(2016) p opose o iden i y gende bias in wo d ep-
esen a ions by compu ing he di ec ion be ween
ep esen a ions o male and emale wo d pai s
om he De ini ional Lis (−→
he-−→
she,−−→
man-−−−−−→
woman)
and compu ing hei p incipal componen s.
In he case o con ex ualized embeddings, he e
is no jus a single ep esen a ion o each wo d,
bu i s ep esen a ion depends on he sen ence i
4Bo h ‘Biased Lis ’ and ‘Ex ended Biased Lis ’ we e
kindly p o ided by Hila Gonen o ep oduce expe imen s
om he s udy (Gonen and Goldbe g,2019)
36
Figu e 1: (Le ) he pe cen age o a iance explained in he PC o de ini ional ec o di e ences. (Righ ) The
co esponding pe cen ages o andom ec o s.
appea s in. Hence, in o de o compu e he gende
subspace we ake he ep esen a ion o wo ds by
andomly sampling sen ences ha con ain wo ds
om he De ini ional Lis and, o each o hem,
we swap he de ini ional wo d wi h i s pai -wise
equi alen om he opposi e gende . We hen ob-
ain he ELMo ep esen a ion o he de inin ional
wo d in each sen ence pai , compu ing hei di e -
ence. On he se o di e ence ec o s, we compu e
hei p incipal componen s o e i y he p esence
o bias. In o de o ha e a e e ence, we compu ed
he p incipal componen s o ep esen a ion o an-
dom wo ds.
Simila ly o Bolukbasi e al. (2016), igu e 1
shows ha he i s eigen alue is signi ican ly
la ge han he es and ha he e is also a single di-
ec ion desc ibing he majo i y o a iance in hese
ec o s, s ill he di e ence be ween he pe cen age
o a iances is less in case o con ex ualized em-
beddings, which may e e ha he e is less bias
in such embeddings. In he igh g aph o he ig-
u e, we can easily no e he di e ence in he case
o andom, whe e he da a is no concen a ed in
a speci ic di ec ion, as he weigh is sp ead among
all componen s.
A simila conclusion was s a ed in he ecen
wo k (Zhao e al.,2019) whe e he au ho s ap-
plied he same app oach, bu o gende swapped
a ian s o sen ences wi h p o essions. They com-
pu ed he di e ence be ween he ec o s o occu-
pa ion wo ds in co esponding sen ences and go a
skewed g aph whe e he i s componen ep esen
he gende in o ma ion while he second compo-
nen g oups he male and emale ela ed wo ds.
Di ec Bias Di ec Bias is a measu e o how
close a ce ain se o wo ds a e o he gende ec-
o . To compu e i , we ex ac ed om he aining
da a he sen ences ha con ain wo ds in he P o es-
sional Lis . We excluded he sen ences ha ha e
bo h a p o essional oken and de ini ional gende
wo d o a oid he in luence o he la e o e he
p esence o bias in he o me . We applied he de -
ini ion o di ec bias om Bolukbasi e al. (2016)
on he ELMo ep esen a ions o he p o essional
wo ds in hese sen ences.
1
|N|X
wN
|cos(~w, g)|(1)
whe e N is he amoun o gende neu al wo ds,
g he gende di ec ion, and ~w he wo d ec o o
each p o ession. We go di ec bias o 0.03, com-
pa ed o 0.08 om s anda d wo d2 ec embeddings
desc ibed in Bolukbasi e al. (2016). This educ-
ion on he di ec bias con i ms ha he subs an-
ial componen along he gende di ec ion ha is
p esen in s anda d wo d embeddings is less o he
con ex ualized wo d embeddings. P obably, his
educ ion comes om he ac ha we a e using
di e en wo d embeddings o he same p o es-
sion depending on he sen ence which is a di ec
consequence and ad an age o using con ex ual-
ized embeddings.
Male and emale-biased wo ds clus e ing. In
o de o s udy i biased male and emale wo ds
clus e oge he when applying con ex ualized em-
beddings, we used k-means o gene a e 2 clus e s
o he embeddings o okens om he Biased lis .
No e ha we canno use se e al ep esen a ions o
each wo d, since i would no make any sense o
clus e one wo d as male and emale a he same
ime. The e o e, in o de o make use o he ad-
an ages o he con ex ualized embeddings, we e-
pea ed 10 independen expe imen s, each wi h a
di e en andom sen ence o each wo d om he
lis o biased male and emale wo ds.
37
Figu e 2: K-means clus e ing, he yellow colo ep e-
sen s he emale and he iole ep esen s he male
Among hese 10 expe imen s, we go a min-
imum accu acy o 69.1% and a maximum o
71.3%, wi h a e age accu acy o 70.1%, much
lowe han in he case o biased and debiased wo d
embeddings which we e 99.9 and 92.5, espec-
i ely, as s a ed in Gonen and Goldbe g (2019).
Based on his c i e ion, e en i he e is s ill bias in-
o ma ion o be emo ed om con ex ualized em-
beddings, i is much less han in case o s anda d
wo d embeddings, e en i debiased.
The clus e s ( o one pa icula expe imen ou
o he 10 o hem) a e shown in Figu e 2a e
applying UMAP (McInnes e al.,2018;McInnes
e al.,2018) o he con ex ualized embeddings.
Classi ica ion App oach In o de o s udy i
con ex ualized embeddings lea n o gene alize
bias, we ained a Radial Basis Func ion-ke nel
Suppo Vec o Machine classi ie on he embed-
dings o andom 1000 biased wo ds om he Ex-
ended Biased Lis . A e ha , we e alua ed he
gene aliza ion on he o he andom 4000 biased o-
kens. Again, we pe o med 10 independen expe -
imen s, o gua an ee andomiza ion o wo d ep e-
sen a ions. Among hese 10 expe imen s, we go a
minimum accu acy o 83.33% and a maximum o
88.43%, wi h a e age accu acy o 85.56%. This
numbe shows ha he bias is lea ned in hese em-
beddings wi h high a e. Howe e , i lea ns in
a lowe a e han he no mal embeddings, whose
classi ica ion eached 88.88% and 98.25% o de-
biased and biased e sions, espec i ely.
K-Nea es Neighbo App oach To unde s and
mo e abou he bias in con ex ualized embeddings,
i is impo an o analyze he bias in he p o es-
sions. The ques ion is whe he hese embeddings
54321012
11.5
12.0
12.5
13.0
s uden
ma hema ician
minis e
a chi ec
his o ian maid
boss
physician
philosophe
manage
physicis
se an
eenage
nun
nanny
sociali e
housewi e
hai d esse
lib a ian
ecep ionis
ba ende
compose
Figu e 3: Visualiza ion o con ex ualized embeddings
o p o essions.
s e eo ype he p o essions as he no mal embed-
dings. This can be shown by he nea es neighbo s
o he emale and male s e eo yped p o essions,
o example ‘ ecep ionis ’ and ‘lib a ian’ o e-
male and ‘a chi ec ’ and ‘philosophe ’ o male.
We applied he k nea es neighbo s on he P o es-
sional Lis , o ge he nea es k neighbo o each
p o ession. We used a andom ep esen a ion o
each oken o he p o ession lis , a e applying
he k nea es neighbo algo i hm on each p o es-
sion, we compu ed he pe cen age o emale and
male s e eo yped p o essions among he k nea es
neighbo o each p o ession oken. A e wa ds,
we compu ed he Pea son co ela ion o his pe -
cen age wi h he o iginal bias o each p o ession.
Once again, o assu e andomiza ion o okens,
we pe o med 10 expe imen s, each wi h di e -
en andom sen ences o each p o ession, he e-
o e wi h di e en wo d ep esen a ions. The min-
imum Pea son co ela ion is 0.801 and he max-
imum is 0.961, wi h a e age o 0.89. All hese
co ela ions a e signi ican wi h p- alues smalle
han 1×10−40. This expe imen showed he high-
es in luence o bias compa ed o 0.606 o de-
biased embeddings and 0.774 o biased. Figu e
3demons a es his in luence o bias by showing
ha emale biased wo ds (e.g. nanny) has highe
pe cen o emale wo ds han male ones and ice-
e sa o male biased wo ds (e.g. philosophe ).
6 Conclusions and u he wo k
While ou s udy canno d aw clea conclusions
on whe he con ex ualized wo d embeddings aug-
men o educe he gende bias, ou esul s show
mo e insigh s in o which aspec s o he inal con-
ex ualized wo d ec o s ge a ec ed by such phe-

38
nomena, wi h a endency mo e owa ds educing
he gende bias a he han he con a y.
Con ex ualized wo d embeddings mi iga e gen-
de bias when measu ing in he ollowing aspec s:
1. Gende space, which is cap u ing he gende
di ec ion om wo d ec o s, is educed o
gende speci ic con ex ualized wo d ec o s
compa ed o s anda d wo d ec o s.
2. Di ec bias, which is measu ing how close se
o wo ds a e o he gende ec o , is lowe
o con ex ualized wo d embeddings han o
s anda d ones.
3. Male/ emale clus e ing, which is p oduced
be ween wo ds wi h s ong gende bias, is
less s ong han in debiased and non-debiased
s anda d wo d embeddings.
Howe e , con ex ualized wo d embeddings p e-
se e and e en ampli y gende bias when aking
in o accoun o he aspec s:
1. The implici gende o wo ds can be p e-
dic ed wi h accu acies highe han 80% based
on con ex ualized wo d ec o s which is only
a sligh ly lowe accu acy han when using
ec o s om debiased and non-debiased s an-
da d wo d embeddings.
2. The s e eo yped wo ds g oup wi h implici -
gende wo ds o he same gende mo e han
in he case o debiased and non-debiased
s anda d wo d embeddings.
While all measu es ha we p esen exhibi ce -
ain gende bias, when e alua ing u u e debiasing
me hods o con ex ualized wo d embeddings i
would be wo h pu ing emphasis on he la e wo
e alua ion measu es ha show highe bias han he
i s h ee.
Hope ully, ou analysis will p o ide a g ain o
sand owa ds de ining s anda d e alua ion me h-
ods o gende bias, p oposing e ec i e debiasing
me hods o e en di ec ly designing equi able algo-
i hms which au oma ically lea n o igno e biased
da a.
As u he wo k, we plan o ex end ou s udy o
mul iple domains and mul iple languages o ana-
lyze and measu e he impac o gende bias p esen
in con ex ualized embeddings in hese di e en
scena ios.
Acknowledgemen s
We wan o hank Hila Gonen o he suppo du -
ing ou esea ch.
This wo k is suppo ed in pa by he Ca a-
lan Agency o Managemen o Uni e si y and
Resea ch G an s (AGAUR) h ough he FI PhD
Schola ship and he Indus ial PhD G an . This
wo k is also suppo ed in pa by he Span-
ish Minis e io de Economa y Compe i i idad,
he Eu opean Regional De elopmen Fund and
he Agencia Es a al de In es igacin, h ough he
pos doc o al senio g an Ramn y Cajal, con-
ac TEC2015-69266-P (MINECO/FEDER,EU)
and con ac PCIN-2017-079 (AEI/MINECO).
Re e ences
Tolga Bolukbasi, Kai-Wei Chang, James Y Zou,
Venka esh Salig ama, and Adam T Kalai. 2016.
Man is o compu e p og amme as woman is o
homemake ? debiasing wo d embeddings. In D. D.
Lee, M. Sugiyama, U. V. Luxbu g, I. Guyon, and
R. Ga ne , edi o s, Ad ances in Neu al In o ma ion
P ocessing Sys ems 29, pages 4349–4357. Cu an
Associa es, Inc.
Joy Buolamwini and Timni Geb u. 2018. Gende
shades: In e sec ional accu acy dispa i ies in com-
me cial gende classi ica ion. In Con e ence on
Fai ness, Accoun abili y and T anspa ency, FAT
2018, 23-24 Feb ua y 2018, New Yo k, NY, USA,
pages 77–91.
Aylin Caliskan, Joanna J. B yson, and A ind
Na ayanan. 2017. Seman ics de i ed au oma ically
om language co po a necessa ily con ain human
biases. Science, 356:183–186.
Jacob De lin, Ming-Wei Chang, Ken on Lee, and
K is ina Tou ano a. 2018. Be : P e- aining o deep
bidi ec ional ans o me s o language unde s and-
ing. a Xi p ep in a Xi :1810.04805.
Joel Escud´
e Fon and Ma a R. Cos a-juss`
a. 2019.
Equalizing gende biases in neu al machine ans-
la ion wi h wo d embeddings echniques. CoRR,
abs/1901.03116.
Hila Gonen and Yoa Goldbe g. 2019. Lips ick on a
pig: Debiasing me hods co e up sys ema ic gende
biases in wo d embeddings bu do no emo e hem.
CoRR, abs/1903.03862.
Sepp Hoch ei e and J¨
u gen Schmidhube . 1997.
Long sho - e m memo y. Neu al Compu a ion,
9(8):1735–1780.
Je emy Howa d and Sebas ian Rude . 2018. Uni e sal
language model ine- uning o ex classi ica ion.
In P oceedings o he 56 h Annual Mee ing o he
39
Associa ion o Compu a ional Linguis ics (Volume
1: Long Pape s), pages 328–339, Melbou ne, Aus-
alia. Associa ion o Compu a ional Linguis ics.
S e lana Ki i chenko and Sai Mohammad. 2018. Ex-
amining gende and ace bias in wo hund ed sen-
imen analysis sys ems. In P oceedings o he
Se en h Join Con e ence on Lexical and Com-
pu a ional Seman ics, pages 43–53, New O leans,
Louisiana. Associa ion o Compu a ional Linguis-
ics.
Nelson F. Liu, Ma Ga dne , Yona an Belinko ,
Ma hew E. Pe e s, and Noah A. Smi h. 2019. Lin-
guis ic knowledge and ans e abili y o con ex ual
ep esen a ions. In P oceedings o he Con e ence o
he No h Ame ican Chap e o he Associa ion o
Compu a ional Linguis ics: Human Language Tech-
nologies.
Leland McInnes, John Healy, and James Mel ille.
2018. UMAP: Uni o m Mani old App oxima ion
and P ojec ion o Dimension Reduc ion. A Xi e-
p in s.
Leland McInnes, John Healy, Na haniel Saul, and
Lukas G ossbe ge . 2018. Umap: Uni o m mani-
old app oxima ion and p ojec ion. The Jou nal o
Open Sou ce So wa e, 3(29):861.
Tomas Mikolo , Ma in Ka a i , Luks Bu ge , Jan Ce -
nock, and Sanjee Khudanpu . 2010. Recu en
neu al ne wo k based language model. In INTER-
SPEECH, pages 1045–1048. ISCA.
Tomas Mikolo , Ilya Su ske e , Kai Chen, G eg S Co -
ado, and Je Dean. 2013. Dis ibu ed ep esen a-
ions o wo ds and ph ases and hei composi ion-
ali y. In C. J. C. Bu ges, L. Bo ou, M. Welling,
Z. Ghah amani, and K. Q. Weinbe ge , edi o s, Ad-
ances in Neu al In o ma ion P ocessing Sys ems
26, pages 3111–3119. Cu an Associa es, Inc.
Je ey Penning on, Richa d Soche , and Ch is ophe
Manning. 2014. Glo e: Global ec o s o wo d
ep esen a ion. In P oceedings o he 2014 Con-
e ence on Empi ical Me hods in Na u al Language
P ocessing (EMNLP), pages 1532–1543, Doha,
Qa a . Associa ion o Compu a ional Linguis ics.
Ma hew Pe e s, Ma k Neumann, Mohi Iyye , Ma
Ga dne , Ch is ophe Cla k, Ken on Lee, and Luke
Ze lemoye . 2018. Deep con ex ualized wo d ep-
esen a ions. In P oceedings o he 2018 Con e -
ence o he No h Ame ican Chap e o he Associ-
a ion o Compu a ional Linguis ics: Human Lan-
guage Technologies, Volume 1 (Long Pape s), pages
2227–2237, New O leans, Louisiana. Associa ion
o Compu a ional Linguis ics.
Ma hew Pe e s, Sebas ian Rude , and Noah A Smi h.
2019. To une o no o une? adap ing p e ained
ep esen a ions o di e se asks. a Xi p ep in
a Xi :1903.05987.
Alec Rad o d. 2018. Imp o ing language unde s and-
ing by gene a i e p e- aining.
Alec Rad o d, Je Wu, Rewon Child, Da id Luan,
Da io Amodei, and Ilya Su ske e . 2019. Language
models a e unsupe ised mul i ask lea ne s.
Jieyu Zhao, Tianlu Wang, Ma k Ya ska , Ryan Co -
e ell, Vicen e O donez, and Kai-Wei Chang. 2019.
Gende bias in con ex ualized wo d embeddings. In
Fo hcoming in NAACL.
Jieyu Zhao, Tianlu Wang, Ma k Ya ska , Vicen e O -
donez, and Kai-Wei Chang. 2018a. Gende bias
in co e e ence esolu ion: E alua ion and debiasing
me hods. a Xi p ep in a Xi :1804.06876.
Jieyu Zhao, Yichao Zhou, Zeyu Li, Wei Wang, and Kai-
Wei Chang. 2018b. Lea ning gende -neu al wo d
embeddings. a Xi p ep in a Xi :1809.01496.