KnowNe : Building a La ge Ne o Knowledge om he Web
Mon se Cuad os
TALP Resea ch Cen e , UPC
Ba celona, Spain
[email p o ec ed]
Ge man Rigau
IXA NLP G oup, UPV/EHU
Donos ia, Spain
[email p o ec ed]
Abs ac
This pape p esen s a new ully au o-
ma ic me hod o building highly dense
and accu a e knowledge bases om ex-
is ing seman ic esou ces. Basically, he
me hod uses a wide-co e age and accu-
a e knowledge-based Wo d Sense Dis-
ambigua ion algo i hm o assign he mos
app op ia e senses o la ge se s o opi-
cally ela ed wo ds acqui ed om he web.
KnowNe , he esul ing knowledge-base
which connec s la ge se s o seman ically-
ela ed concep s is a majo s ep owa ds
he au onomous acquisi ion o knowledge
om aw co po a. In ac , KnowNe is se -
e al imes la ge han any a ailable knowl-
edge esou ce encoding ela ions be ween
synse s, and he knowledge KnowNe con-
ains ou pe o m any o he esou ce when
is empi ically e alua ed in a common
amewo k.
1 In oduc ion
Using la ge-scale knowledge bases, such as Wo d-
Ne (Fellbaum, 1998), has become a usual, o -
en necessa y, p ac ice o mos cu en Na u al
Language P ocessing (NLP) sys ems. E en now,
building la ge and ich enough knowledge bases
o b oad–co e age seman ic p ocessing akes a
g ea deal o expensi e manual e o in ol ing
la ge esea ch g oups du ing long pe iods o de-
elopmen . In ac , hund eds o pe son-yea s ha e
been in es ed in he de elopmen o wo dne s o
a ious languages (Vossen, 1998). Fo example, in
c
2008. Licensed unde he C ea i e Commons
A ibu ion-Noncomme cial-Sha e Alike 3.0 Unpo ed li-
cense (h p://c ea i ecommons.o g/licenses/by-nc-sa/3.0/).
Some igh s ese ed.
mo e han en yea s o manual cons uc ion ( om
1995 o 2006, ha is om e sion 1.5 o 3.0),
Wo dNe g ew om 103,445 o 235,402 seman ic
ela ions1. Bu his da a does no seem o be ich
enough o suppo ad anced concep -based NLP
applica ions di ec ly. I seems ha applica ions
will no scale up o wo k in open domains wi hou
mo e de ailed and ich gene al-pu pose (and also
domain-speci ic) seman ic knowledge buil by au-
oma ic means. Ob iously, his ac has se e ely
hampe ed he s a e-o - he-a o ad anced NLP ap-
plica ions.
Howe e , he P ince on Wo dNe (WN) is by a
he mos widely-used knowledge base (Fellbaum,
1998). In ac , Wo dNe is being used wo ld-wide
o ancho ing di e en ypes o seman ic knowl-
edge including wo dne s o languages o he han
English (A se ias e al., 2004), domain knowledge
(Magnini and Ca agli`
a, 2000) o on ologies like
SUMO (Niles and Pease, 2001) o he Eu oWo d-
Ne Top Concep On ology ( ´
Al ez e al., 2008).
I con ains manually coded in o ma ion abou En-
glish nouns, e bs, adjec i es and ad e bs and is
o ganized a ound he no ion o a synse . A synse
is a se o wo ds wi h he same pa -o -speech ha
can be in e changed in a ce ain con ex . Fo ex-
ample, <pa y,poli ical pa y> o m a synse be-
cause hey can be used o e e o he same concep .
A synse is o en u he desc ibed by a gloss, in
his case: ”an o ganiza ion o gain poli ical powe ”
and by explici seman ic ela ions o o he synse s.
Fo una ely, du ing he las yea s he esea ch
communi y has de ised a la ge se o inno a i e
me hods and ools o la ge-scale au oma ic acqui-
si ion o lexical knowledge om s uc u ed and un-
s uc u ed co po a. Among o he s we can men-
1Symme ic ela ions a e coun ed only once.
ion eX ended Wo dNe (Mihalcea and Moldo an,
2001), la ge collec ions o seman ic p e e ences
acqui ed om SemCo (Agi e and Ma inez,
2001; Agi e and Ma inez, 2002) o acqui ed om
B i ish Na ional Co pus (BNC) (McCa hy, 2001),
la ge-scale Topic Signa u es o each synse ac-
qui ed om he web (Agi e and de Lacalle, 2004)
o knowledge abou indi iduals om Wikipedia
(Suchanek e al., 2007). Ob iously, all hese se-
man ic esou ces ha e been acqui ed using a e y
di e en me hods, ools and co po a. As expec ed,
each seman ic esou ce has di e en olume and
accu acy igu es when e alua ed in a common and
con olled amewo k (Cuad os and Rigau, 2006).
Howe e , no all hese la ge-scale esou ces en-
code seman ic ela ions be ween synse s. In some
cases, only ela ions be ween synse s and wo ds
ha e been acqui ed. This is he case o he Topic
Signa u es acqui ed om he web (Agi e and de
Lacalle, 2004). This is one o he la ges seman-
ic esou ces e e buil wi h a ound one hund ed
million ela ions be ween synse s and seman ically
ela ed wo ds 2.
A knowledge ne o KnowNe (KN), is an ex en-
sible, la ge and accu a e knowledge base, which
has been de i ed by seman ically disambigua ing
small po ions o he Topic Signa u es acqui ed
om he web. Basically, he me hod uses a o-
bus and accu a e knowledge-based Wo d Sense
Disambigua ion algo i hm o assign he mos ap-
p op ia e senses o he opic wo ds associa ed o
a pa icula synse . The esul ing knowledge-base
which connec s la ge se s o opically- ela ed con-
cep s is a majo s ep owa ds he au onomous ac-
quisi ion o knowledge om aw ex .
Table 1 compa es he di e en olumes o se-
man ic ela ions be ween synse pai s o a ail-
able knowledge bases and he newly c ea ed
KnowNe s3.
Va ying om i e o wen y he numbe o p o-
cessed wo ds om each Topic Signa u e, we c e-
a ed au oma ically ou di e en KnowNe e -
sions wi h millions o new seman ic ela ions be-
ween synse s. In ac , KnowNe is se e al imes
la ge han Wo dNe , and when e alua ed empi -
ically in a common amewo k, he knowledge i
con ains ou pe o ms any o he seman ic esou ce.
A e his in oduc ion, sec ion 2 desc ibes he
Topic Signa u es acqui ed om he web. Sec ion
2A ailable a h p://ixa.si.ehu.es/Ixa/ esou ces/senseco pus
3These KnowNe e sions a e a ailable a
h p://adimen.si.ehu.es
Sou ce # ela ions
P ince on WN3.0 235,402
Selec ional P e e ences om SemCo 203,546
eX ended WN 550,922
Co-occu ing ela ions om SemCo 932,008
New KnowNe -5 231,163
New KnowNe -10 689,610
New KnowNe -15 1,378,286
New KnowNe -20 2,358,927
Table 1: Numbe o synse ela ions
3 p esen s he app oach we ollowed o building
highly dense and accu a e knowledge bases om
he Topic Signa u es. In sec ion 4, we p esen he
e alua ion amewo k used in his s udy. Sec ion 5
desc ibes he esul s when e alua ing di e en e -
sions o KnowNe and inally, sec ion 6 p esen s
some concluding ema ks and u u e wo k.
2 Topic Signa u es
Topic Signa u es (TS) a e wo d ec o s ela ed o a
pa icula opic (Lin and Ho y, 2000). Topic Sig-
na u es a e buil by e ie ing con ex wo ds o a
a ge opic om a la ge co po a. This s udy con-
side s wo d senses as opics. Basically, he acqui-
si ion o TS consis s o :
•acqui ing he bes possible co pus examples
o a pa icula wo d sense (usually cha ac e -
izing each wo d sense as a que y and pe o m-
ing a sea ch on he co pus o hose examples
ha bes ma ch he que ies)
•building he TS by selec ing he con ex
wo ds ha bes ep esen he wo d sense om
he selec ed co po a.
The Topic Signa u es acqui ed om he web
(he eina e TSWEB) cons i u es one o he la ges
seman ic esou ce a ailable wi h a ound 100 mil-
lion ela ions (be ween synse s and wo ds) (Agi e
and de Lacalle, 2004). Inspi ed by he wo k o
(Leacock e al., 1998), TSWEB was cons uc ed
using monosemous ela i es om WN (synonyms,
hype nyms, di ec and indi ec hyponyms, and sib-
lings), que ying Google and e ie ing up o one
housand snippe s pe que y ( ha is, a wo d sense),
ex ac ing he salien wo ds wi h dis inc i e e-
quency using TFIDF. Thus, TSWEB consis o
la ge o de ed lis s o wo ds wi h weigh s associ-
a ed o he polysemous nouns o WN1.6. The
numbe o cons uc ed opic signa u es is 35,250
wi h an a e age size pe signa u e o 6,877 wo ds.
ammany#n 0.0319
ede alis #n 0.0315
whig#n 0.0300
missiona y#j 0.0229
Democ a ic#n 0.0218
nazi#j 0.0202
epublican#n 0.0189
cons i u ional#n 0.0186
conse a i e#j 0.0148
socialis #n 0.0140
Table 2: TS o pa y#n#1 ( i s 10 ou o 12,890
o al wo ds)
When e alua ing TSWEB, we used a maximum
he i s 700 wo ds while o building KnowNe we
used a maximum he i s 20 wo ds.
Fo example, able 2 p esen s he i s wo ds
(lemmas and pa -o -speech) and weigh s o he
Topic Signa u e acqui ed o pa y#n#14.
3 Building highly connec ed and dense
knowledge bases
We acqui ed by ully au oma ic means highly
connec ed and dense knowledge bases by disam-
bigua ing small po ions o he Topic Signa u es
ob ained om he web, inc easing he o al num-
be o seman ic ela ions om less han one mil-
lion ( he cu en numbe o a ailable ela ions) o
millions o new and accu a e seman ic ela ions
be ween synse s. We applied a knowledge–based
all–wo ds Wo d Sense Disambigua ion algo i hm
o he Topic Signa u es o de i ing a sense ec o
om each wo d ec o .
3.1 SSI-Dijks a
We ha e implemen ed a e sion o he S uc-
u al Seman ic In e connec ions algo i hm (SSI), a
knowledge-based i e a i e app oach o Wo d Sense
Disambigua ion (Na igli and Vela di, 2005). The
SSI algo i hm is e y simple and consis s o an ini-
ializa ion s ep and a se o i e a i e s eps (see al-
go i hm 1).
Gi en W, an o de ed lis o wo ds o be dis-
ambigua ed, he SSI algo i hm pe o ms as ol-
lows. Du ing he ini ializa ion s ep, all monose-
mous wo ds a e included in o he se I o al eady
in e p e ed wo ds, and he polysemous wo ds a e
included in P (all o hem pending o be disam-
bigua ed). A each s ep, he se I is used o disam-
bigua e one wo d o P, selec ing he wo d sense
which is close o he se I o al eady disam-
4This o ma s ands o wo d#pos#sense.
bigua ed wo ds. Once a sense is selec ed, he wo d
sense is emo ed om P and included in o I. The
algo i hm inishes when no mo e pending wo ds
emain in P.
Algo i hm 1 SSI-Dijks a Algo i hm
SSI (T: lis o e ms)
o each { ∈T}do
I[ ] = ∅
i is monosemous hen
I[ ] := he only sense o
else
P:= P∪ { }
end i
end o
epea
P0:= P
o each { ∈P}do
Bes Sense := ∅
MaxV alue := 0
o each {sense s o }do
W[s] := 0
N[s] := 0
o each {sense s0∈I}do
w:= Dijsk aSho es Pa h(s, s0)
i w > 0 hen
W[s] := W[s] + (1/w)
N[s] := N[s]+1
end i
end o
i N[s]>0 hen
NewV alue := W[s]/N[s]
i NewV alue > MaxV alue hen
MaxV alue := NewV alue
Bes Sense := s
end i
end i
end o
i MaxV alue > 0 hen
I[ ] := Bes Sense
P:= P { }
end i
end o
un il P6=P0
e u n (I, P);
Ini ially, he lis I o in e p e ed wo ds should in-
clude he senses o he monosemous wo ds in W,
o a ixed se o wo d senses5. Howe e , when dis-
5I no monosemous wo ds a e ound o i no ini ial senses
a e p o ided, he algo i hm could make an ini ial guess based
on he mos p obable sense o he less ambiguous wo d o W.
ambigua ing a TS o a wo d sense s( o ins ance
pa y#n#1), he lis I al eady includes s.
In o de o measu e he p oximi y o one synse
o he es o synse s o I, we use pa o he
knowledge al eady a ailable o build a e y la ge
connec ed g aph wi h 99,635 nodes (synse s) and
636,077 edges. This g aph includes he se o
di ec ela ions be ween synse s ga he ed om
Wo dNe and eX ended Wo dNe . On ha g aph,
we used a e y e icien g aph lib a y, Boos -
G aph6 o compu e he Dijks a algo i hm. The
Dijks a algo i hm is a g eedy algo i hm o com-
pu ing he sho es pa h dis ance be ween one node
an he es o nodes o a g aph. In ha way, we can
compu e e y e icien ly he sho es dis ance be-
ween any wo gi en nodes o a g aph. We call his
e sion o he SSI algo i hm, SSI-Dijks a.
SSI-Dijks a has e y in e es ing p ope ies. Fo
ins ance, i always p o ides he minimum dis ance
be ween wo synse s. Tha is, he algo i hm always
p o ides an answe being he minimum dis ance
close o a . In con as , he o iginal SSI algo i hm
no always p o ides a pa h dis ance because i de-
pends on a p ede ined g amma o seman ic ela-
ions. In ac , he SSI-Dijks a algo i hm compa es
he dis ances be ween he synse s o a wo d and all
he synse s al eady in e p e ed in I. A each s ep,
he SSI-Dijks a algo i hm selec s he synse which
is close o I ( he se o al eady in e p e ed wo ds).
Fu he mo e, his app oach is comple ely lan-
guage independen . The same g aph can be used
o any language ha ing wo ds connec ed o Wo d-
Ne .
3.2 Building KnowNe
We de eloped KnowNe (KN), a la ge-scale and
ex ensible knowledge base, by applying SSI-
Dijks a o each opic signa u e om TSWEB.
We ha e gene a ed ou di e en e sions o
KnowNe applying SSI-Dijks a o only he i s
5, 10, 15 and 20 wo ds o each TS. SSI-Dijks a
used only he knowledge p esen in Wo dNe and
eX ended Wo dNe which consis o a e y la ge
connec ed g aph wi h 99,635 nodes (synse s) and
636,077 edges (seman ic ela ions).
We gene a ed each KnowNe by applying he
SSI-Dijks a algo i hm o he whole TSWEB (p o-
cessing he i s wo ds o each o he 35,250
opic signa u es). Fo each TS, we ob ained he
di ec ela ions om he opic (a wo d sense)
6h p://www.boos .o g
KB WN+XWN # ela ions #synse s
KN-5 3,1% 231,163 39,864
KN-10 5,0% 689,610 45,817
KN-15 6,9% 1,378,286 48,521
KN-20 8,5% 2,358,927 50,789
Table 3: Size and pe cen age o o e lapping ela-
ions be ween KnowNe e sions and WN+XWN
o he disambigua ed wo d senses o he TS
( o ins ance, pa y#n#1–> ede alis #n#1), bu
also he indi ec ela ions be ween disambigua ed
wo ds om he TS ( o ins ance, ede alis #n#1–
> epublican#n#1). Finally, we emo ed symme -
ic and epea ed ela ions.
Table 3 shows he o e laping pe cen age be-
ween each KnowNe and he knowledge con-
ained in o Wo dNe and eX ended Wo dNe , and
he o al numbe o ela ions and synse s o each
esou ce. Fo ins ance, only 8,5% o he o al di-
ec ela ions included in o WN+XWN a e also
p esen in KnowNe -20. This means ha he es
o ela ions om KnowNe -20 a e new. As ex-
pec ed, each KnowNe is e y la ge, anging om
hund eds o housands o millions o new seman ic
ela ions be ween synse s among inc easing se s o
synse s.
4 E alua ion amewo k
In o de o empi ically es ablish he ela i e qual-
i y o hese new seman ic esou ces, we used he
e alua ion amewo k o ask 16 o SemE al-2007:
E alua ion o wide co e age knowledge esou ces
(Cuad os and Rigau, 2007).
In his amewo k all knowledge esou ces a e
e alua ed on a common WSD ask. In pa icu-
la , we used he noun-se s o he English Lexi-
cal Sample ask o Sense al-3 and SemE al-2007
exe cises which consis s o 20 and 35 nouns e-
spec i ely. All pe o mances a e e alua ed on he
es da a using he ine-g ained sco ing sys em p o-
ided by he o ganize s.
Fu he mo e, ying o be as neu al as possible
wi h espec o he esou ces s udied, we applied
sys ema ically he same disambigua ion me hod o
all o hem. Recall ha ou main goal is o es-
ablish a ai compa ison o he knowledge e-
sou ces a he han p o iding he bes disambigua-
ion echnique o a pa icula knowledge base. All
knowledge bases a e e alua ed as opic signa u es.
Tha is, wo d ec o s wi h weigh s associa ed o a
pa icula synse which a e ob ained by collec ing
hose wo d senses appea ing in he synse s di ec ly
ela ed o he opics. This simple ep esen a ion
ies o be as neu al as possible wi h espec o he
esou ces used.
A common WSD me hod has been applied o all
knowledge esou ces. A simple wo d o e lapping
coun ing is pe o med be ween he opic signa u e
ep esen ing a wo d sense and he es example7.
The synse ha ing highe o e lapping wo d coun s
is selec ed. In ac , his is a e y simple WSD
me hod which only conside s he opical in o ma-
ion a ound he wo d o be disambigua ed. Finally,
we should ema k ha he esul s a e no skewed
( o ins ance, o esol ing ies) by he mos e-
quen sense in WN o any o he s a is ically p e-
dic ed knowledge.
4.1 Baselines
We ha e designed a numbe o baselines in o de
o es ablish a comple e e alua ion amewo k o
compa ing he pe o mance o each seman ic e-
sou ce on he English WSD asks.
RANDOM: Fo each a ge wo d, his me hod
selec s a andom sense. This baseline can be con-
side ed as a lowe -bound.
SEMCOR-MFS: This baseline selec s he mos
equen sense o he a ge wo d in SemCo .
WN-MFS: This baseline is ob ained by se-
lec ing he mos equen sense ( he i s sense
in WN1.6) o he a ge wo d. Wo dNe wo d-
senses we e anked using SemCo and o he sense-
anno a ed co po a. Thus, WN-MFS and SemCo -
MFS a e simila , bu no equal.
TRAIN-MFS: This baseline selec s he mos
equen sense in he aining co pus o he a ge
wo d.
TRAIN: This baseline uses he aining co pus
o di ec ly build a Topic Signa u e using TFIDF
measu e o each wo d sense and selec ing a max-
imum he i s 450 wo ds. No e ha in WSD e al-
ua ion amewo ks, his is a e y basic baseline.
Howe e , in ou e alua ion amewo k, his ”WSD
baseline” could be conside ed as an uppe -bound.
We do no expec o ob ain be e opic signa u es
o a pa icula sense han om i s own anno a ed
co pus.
4.2 O he La ge-scale Knowledge Resou ces
In o de o measu e he ela i e quali y o he new
esou ces, we include in he e alua ion a wide
7We also conside hose mul iwo d e ms appea ing in
WN.
ange o la ge-scale knowledge esou ces con-
nec ed o Wo dNe .
WN (Fellbaum, 1998): This esou ce uses he
di e en di ec ela ions encoded in WN1.6 and
WN2.0. We also es ed WN2using ela ions a dis-
ance 1 and 2, WN3using ela ions a dis ances 1
o 3 and WN4using ela ions a dis ances 1 o 4.
XWN (Mihalcea and Moldo an, 2001): This
esou ce uses he di ec ela ions encoded in eX-
ended WN.
spBNC (McCa hy, 2001): This esou ce con-
ains 707,618 selec ional p e e ences acqui ed o
subjec s and objec s om BNC.
spSemCo (Agi e and Ma inez, 2002): This
esou ce con ains he selec ional p e e ences ac-
qui ed o subjec s and objec s om SemCo .
MCR (A se ias e al., 2004): This esou ce in-
eg a es he di ec ela ions o WN, XWN and
spSemCo .
TSSEM (Cuad os e al., 2007): These Topic
Signa u es ha e been cons uc ed using Sem-
Co .Fo each wo d-sense appea ing in SemCo , we
ga he all sen ences o ha wo d sense, building a
TS using TFIDF o all wo d-senses co-occu ing
in hose sen ences.
4.3 In eg a ed Knowledge Resou ces
We also e alua ed he pe o mance o he in eg a-
ion ( emo ing duplica ed ela ions) o some o
hese esou ces.
WN+XWN: This esou ce in eg a es he di-
ec ela ions o WN and XWN. We also es ed
(WN+XWN)2(using ei he WN o XWN ela-
ions a dis ances 1 and 2).
MCR (A se ias e al., 2004): This esou ce in-
eg a es he di ec ela ions o WN, XWN and
spSemCo .
WN+XWN+KN-20: This esou ce in eg a es
he di ec ela ions o WN, XWN and KnowNe -
20.
5 KnowNe E alua ion
We e alua ed KnowNe using he same amewo k
explained in sec ion 4. Tha is, he noun pa o he
es se om he English Sense al-3 and SemE al-
2007 English lexical sample asks.
5.1 Sense al-3 e alua ion
Table 4 p esen s o de ed by F1 measu e, he pe -
o mance in e ms o p ecision (P), ecall (R) and
KB P R F1 A . Size
TRAIN 65.1 65.1 65.1 450
TRAIN-MFS 54.5 54.5 54.5
WN-MFS 53.0 53.0 53.0
TSSEM 52.5 52.4 52.4 103
SEMCOR-MFS 49.0 49.1 49.0
MCR245.1 45.1 45.1 26,429
WN+XWN+KN-20 44.8 44.8 44.8 671
MCR 45.3 43.7 44.5 129
KnowNe -20 44.1 44.1 44.1 610
KnowNe -15 43.9 43.9 43.9 339
spSemCo 43.1 38.7 40.8 56
KnowNe -10 40.1 40.0 40.0 154
(WN+XWN)238.5 38.0 38.3 5,730
WN+XWN 40.0 34.2 36.8 74
TSWEB 36.1 35.9 36.0 1,721
XWN 38.8 32.5 35.4 69
KnowNe -5 35.0 35.0 35.0 44
WN335.0 34.7 34.8 503
WN433.2 33.1 33.2 2,346
WN233.1 27.5 30.0 105
spBNC 36.3 25.4 29.9 128
WN 44.9 18.4 26.1 14
RANDOM 19.1 19.1 19.1
Table 4: P, R and F1 ine-g ained esul s o he
esou ces e alua ed a Sense al-3, English Lexical
Sample Task.
F1 measu e (F1, ha monic mean o ecall and p e-
cision) o each knowledge esou ce on Sense al-3
and he a e age size o he TS pe wo d-sense. The
di e en KnowNe e sions appea ma ked in bold
and he baselines appea in i alics.
As expec ed, RANDOM ob ains he poo es e-
sul . The mos equen senses ob ained om Sem-
Co (SEMCOR-MFS) and WN (WN-MFS) a e
bo h below he mos equen sense o he aining
co pus (TRAIN-MFS). Howe e , all o hem a e
a below o he Topic Signa u es acqui ed using
he aining co pus (TRAIN).
The bes esul s a e ob ained by TSSEM (wi h
F1 o 52.4). The lowes esul is ob ained by he
knowledge di ec ly ga he ed om WN mainly be-
cause o i s poo co e age (R o 18.4 and F1 o
26.1). In e es ingly, he knowledge in eg a ed in
he MCR al hough pa ly de i ed by au oma ic
means pe o ms much be e in e ms o p ecision,
ecall and F1 measu es han using hem sepa a ely
(F1 wi h 18.4 poin s highe han WN, 9.1 han
XWN and 3.7 han spSemCo ).
Despi e i s small size, he esou ces de i ed
om SemCo ob ain be e esul s han i s coun-
e pa s using much la ge co po a (TSSEM s.
TSWEB and spSemCo s. spBNC).
Rega ding he baselines, all knowledge e-
sou ces su pass RANDOM, bu none achie es nei-
he WN-MFS, TRAIN-MFS no TRAIN. Only
TSSEM ob ains be e esul s han SEMCOR-MFS
and is e y close o he mos equen sense o WN
(WN-MFS) and he aining (TRAIN-MFS).
Rega ding he expansions and combina ions, he
pe o mance o WN is imp o ed using wo ds a
dis ances up o 2, and up o 3, bu i dec eases using
dis ances up o 4. In e es ingly, none o hese WN
expansions achie e he esul s o XWN. Finally,
(WN+XWN)2pe o ms be e han WN+XWN
and MCR2sligh ly be e han MCR8.
The di e en e sions o KnowNe consis en ly
ob ain be e pe o mances as hey inc ease he
window size o p ocessed wo ds o TSWEB. As
expec ed, KnowNe -5 ob ain he lowe esul s.
Howe e , i pe o ms be e han WN (and all
i s ex ensions) and spBNC. In e es ingly, om
KnowNe -10, all KnowNe e sions su pass he
knowledge esou ces used o hei cons uc ion
(WN, XWN, TSWEB and WN+XWN). In ac ,
KnowNe -10 also ou pe o ms (WN+XWN)2wi h
much mo e ela ions pe sense. Also in e es ing
is ha KnowNe -10 and KnowNe -20 ob ain be -
e pe o mance han spSemCo which was de i ed
om anno a ed co po a. Howe e , KnowNe -20
only pe o ms sligh ly be e han KnowNe -15
while almos doubling he numbe o ela ions.
These ini ial esul s seem o be e y p omis-
ing. I we do no conside he esou ces de i ed
om manually sense anno a ed da a (spSemCo ,
MCR, TSSEM, e c.), KnowNe -10 pe o ms be -
e ha any knowledge esou ce de i ed by man-
ual o au oma ic means. In ac , KnowNe -15 and
KnowNe -20 ou pe o ms spSemCo which was
de i ed om manually anno a ed co po a. This is
a e y in e es ing esul since hese KnowNe e -
sions ha e been de i ed only wi h he knowledge
coming om WN and he web ( ha is, TSWEB),
and WN and XWN as a knowledge sou ce o SSI-
Dijks a9.
Rega ding he in eg a ion o esou ces,
WN+XWN+KN-20 pe o ms be e han MCR
and simila ly o MCR2(ha ing less han 50 imes
i s size). Also in e es ing is ha WN+XWN+KN-
20 ha e be e pe o mance han hei indi idual
esou ces, indica ing a complemen a y knowledge.
In ac , WN+XWN+KN-20 pe o ms much be e
han he esou ces om which i de i es (WN,
XWN and TSWEB).
8No u he dis ances ha e been es ed
9eX ended Wo dNe only has 17,185 manually labeled
senses.
KB P R F1 A . Size
TRAIN 87.6 87.6 87.6 450
TRAIN-MFS 81.2 79.6 80.4
WN-MFS 66.2 59.9 62.9
WN+XWN+KN-20 53.0 53.0 53.0 627
(WN+XWN)254.9 51.1 52.9 5,153
TSWEB 54.8 47.8 51.0 700
KnowNe -20 49.5 46.1 47.7 561
KnowNe -15 47.0 43.5 45.2 308
XWN 50.1 39.8 44.4 96
KnowNe -10 44.0 39.8 41.8 139
WN+XWN 45.4 36.8 40.7 101
SEMCOR-MFS 42.4 38.4 40.3
MCR 40.2 35.5 37.7 149
TSSEM 35.1 32.7 33.9 428
KnowNe -5 35.5 26.5 30.3 41
MCR232.4 29.5 30.9 24,896
WN329.3 26.3 27.7 584
RANDOM 27.4 27.4 27.4
WN225.9 27.4 26.6 72
spSemCo 31.4 23.0 26.5 51.0
WN426.1 23.9 24.9 2,710
WN 36.8 16.1 22.4 13
spBNC 24.4 18.1 20.8 290
Table 5: P, R and F1 ine-g ained esul s o he e-
sou ces e alua ed a SemE al-2007, English Lexi-
cal Sample Task.
5.2 SemE al-2007 e alua ion
Table 5 p esen s o de ed by F1 measu e, he pe -
o mance in e ms o p ecision (P), ecall (R) and
F1 measu e (F1) o each knowledge esou ce on
SemE al-2007 and i s a e age size o he TS pe
wo d-sense10. Again, he di e en KnowNe e -
sions appea ma ked in bold and he baselines ap-
pea in i alics.
As in he p e ious e alua ion, RANDOM ob-
ains he poo es esul . The mos equen senses
ob ained om SemCo (SEMCOR-MFS) and WN
(WN-MFS) a e bo h a below he mos equen
sense o he aining co pus (TRAIN-MFS), and all
o hem a e below he Topic Signa u es acqui ed
using he aining co pus (TRAIN).
In e es ingly, on SemE al-2007, all he knowl-
edge esou ces beha e di e en ly. Now, he bes
indi idual esul s a e ob ained by TSWEB, while
in his case TSSEM ob ains e y modes esul s.
The lowes esul is ob ained by he knowledge en-
coded in spBNC.
Rega ding he baselines, spBNC, WN (and also
WN2and WN4) and spSemCo do no su pass
RANDOM, and none achie es nei he WN-MFS,
TRAIN-MFS no TRAIN. Now, WN+XWN,
XWN, TSWEB and (WN+XWN)2ob ain be e
10The a e age size is di e en wi h espec Sense al-3 be-
cause he wo ds selec ed o his ask a e di e en
esul s han SEMCOR-MFS bu a below he mos
equen sense o WN (WN-MFS) and he aining
(TRAIN-MFS).
Rega ding o he expansions and combina ions,
he pe o mance o WN is imp o ed using wo ds
a dis ances up o 2, and up o 3, bu i dec eases
using dis ances up o 4. Again, none o hese WN
expansions achie e he esul s o XWN. Finally,
(WN+XWN)2pe o ms be e han WN+XWN
and MCR2sligh ly be e han MCR11.
On SemE al-2007, he di e en e sions o
KnowNe consis en ly ob ain be e pe o mances
as hey incease he window size o p ocessed
wo ds o TSWEB. As expec ed, KnowNe -5 ob-
ain he lowe esul s. Howe e , i pe o ms be e
han spBNC, WN (and all i s ex ensions), spSem-
Co and MCR2. This ime, all KnowNe e -
sions pe o m wo se han TSWEB. Howe e , as in
he p e ious e alua ion, KnowNe -10 ou pe o ms
WN+XWN, and his ime, also TSSEM and he
MCR, wi h much mo e ela ions pe sense. Also
in e es ing is ha om KnowNe -10, all KnowNe
e sions pe o m be e han he esou ces de i ed
om manually sense anno a ed co po a (spSem-
Co , MCR, TSSEM, e c.).
Rega ding he in eg a ion o esou ces,
WN+XWN+KN-20 pe o ms be e han any
knowledge esou ce de i ed by manual o au o-
ma ic means. Again, i is in e es ing o no e ha
WN+XWN+KN-20 ha e be e pe o mance han
hei indi idual esou ces, indica ing a comple-
men a y knowledge. In ac , WN+XWN+KN-20
pe o ms much be e han he esou ces om
which i de i es (WN, XWN and TSWEB).
5.3 Discussion
When compa ing he anking o he di e en
knowledge esou ces, he di e en e sions o
KnowNe seem o be mo e obus and s able
ac oss co po a changes. Fo ins ance, in bo h
e alua ion amewo ks (Sense al-3 and SemE al-
2007), KnowNe -20 anks 5 h and 4 h, espec-
i ely ((WN+XWN)2 anks 8 h and 2nd, TSSEM
anks 1s and 10 h, MCR anks 4 h and 9 h,
TSWEB anks 11 h and 3 d, e c.). In ac ,
WN+XWN+KN-20 anks 3 d and 1s , espec-
i ely.
11No u he dis ances ha e been es ed
6 Conclusions and u u e esea ch
I is ou belie , ha accu a e seman ic p ocessing
(such as WSD) would ely no only on sophis i-
ca ed algo i hms bu on knowledge in ensi e ap-
p oaches. The esul s p esen ed in his pape sug-
ges s ha much mo e esea ch on acqui ing and
using la ge-scale seman ic esou ces should be ad-
d essed.
The knowledge acquisi ion bo leneck p oblem
is pa icula ly acu e o open domain (and also
domain speci ic) seman ic p ocessing. The ini-
ial esul s ob ained o he di e en e sions o
KnowNe seem o be a majo s ep owa ds he au-
onomous acquisi ion o knowledge om aw co -
po a, since hey a e se e al imes la ge han he
a ailable knowledge esou ces which encode e-
la ions be ween synse s, and he knowledge hey
con ain ou pe o m any o he esou ce when is em-
pi ically e alua ed in a common amewo k.
I emains o u u e esea ch he e alua ion o
hese KnowNe e sions in combina ion wi h o he
la ge-scale seman ic esou ces o in a c oss-lingual
se ing.
Acknowledgmen s
We wan o hank Ai o So oa o his echnical
suppo and he anonymous e iewe s o hei
commen s. This wo k has been suppo ed by
KNOW (TIN2006-15049-C03-01) and KYOTO
(ICT-2007-211423).
Re e ences
Agi e, E. and O. Lopez de Lacalle. 2004. Publicly
a ailable opic signa u es o all wo dne nominal
senses. In P oceedings o LREC, Lisbon, Po ugal.
Agi e, E. and D. Ma inez. 2001. Lea ning class-
o-class selec ional p e e ences. In P oceedings o
CoNLL, Toulouse, F ance.
Agi e, E. and D. Ma inez. 2002. In eg a ing selec-
ional p e e ences in wo dne . In P oceedings o
GWC, Myso e, India.
´
Al ez, J., J. A se ias, J. Ca e a, S. Climen , A. Oli e ,
and G. Rigau. 2008. Consis en anno a ion o eu-
owo dne wi h he op concep on ology. In P o-
ceedings o Fou h In e na ional Wo dNe Con e -
ence (GWC’08).
A se ias, J., L. Villa ejo, G. Rigau, E. Agi e, J. Ca -
oll, B. Magnini, and Piek Vossen. 2004. The mean-
ing mul ilingual cen al eposi o y. In P oceedings
o GWC, B no, Czech Republic.
Cuad os, M. and G. Rigau. 2006. Quali y assessmen
o la ge scale knowledge esou ces. In P oceedings
o he EMNLP.
Cuad os, M. and G. Rigau. 2007. Seme al-2007
ask 16: E alua ion o wide co e age knowledge e-
sou ces. In P oceedings o he Fou h In e na ional
Wo kshop on Seman ic E alua ions (SemE al-2007).
Cuad os, M., G. Rigau, and M. Cas illo. 2007. E al-
ua ing la ge-scale knowledge esou ces ac oss lan-
guages. In P oceedings o RANLP.
Fellbaum, C., edi o . 1998. Wo dNe . An Elec onic
Lexical Da abase. The MIT P ess.
Leacock, C., M. Chodo ow, and G. Mille . 1998.
Using Co pus S a is ics and Wo dNe Rela ions o
Sense Iden i ica ion. Compu a ional Linguis ics,
24(1):147–166.
Lin, C. and E. Ho y. 2000. The au oma ed acquisi-
ion o opic signa u es o ex summa iza ion. In
P oceedings o COLING. S asbou g, F ance.
Magnini, B. and G. Ca agli`
a. 2000. In eg a ing subjec
ield codes in o wo dne . In P oceedings o LREC,
A hens. G eece.
McCa hy, D. 2001. Lexical Acquisi ion a he Syn ax-
Seman ics In e ace: Dia hesis A e na ions, Sub-
ca ego iza ion F ames and Selec ional P e e ences.
Ph.D. hesis, Uni e si y o Sussex.
Mihalcea, R. and D. Moldo an. 2001. ex ended wo d-
ne : P og ess epo . In P oceedings o NAACL
Wo kshop on Wo dNe and O he Lexical Resou ces,
Pi sbu gh, PA.
Na igli, R. and P. Vela di. 2005. S uc u al seman-
ic in e connec ions: a knowledge-based app oach o
wo d sense disambigua ion. IEEE T ansac ions on
Pa e n Analysis and Machine In elligence (PAMI),
27(7):1063–1074.
Niles, I. and A. Pease. 2001. Towa ds a s anda d up-
pe on ology. In P oceedings o he 2nd In e na-
ional Con e ence on Fo mal On ology in In o ma-
ion Sys ems (FOIS-2001), pages 17–19. Ch is Wel y
and Ba y Smi h, eds.
Suchanek, Fabian M., Gje gji Kasneci, and Ge ha d
Weikum. 2007. Yago: A Co e o Seman ic Knowl-
edge. In 16 h in e na ional Wo ld Wide Web con-
e ence (WWW 2007), New Yo k, NY, USA. ACM
P ess.
Vossen, P., edi o . 1998. Eu oWo dNe : A Mul ilingual
Da abase wi h Lexical Seman ic Ne wo ks . Kluwe
Academic Publishe s .
