KnowNet: building a large net of knowledge from the web

KnowNe : Building a La ge Ne o Knowledge om he Web Mon se Cuad os TALP Resea ch Cen e , UPC Ba celona, Spain [email p o ec ed] Ge man Rigau IXA NLP G oup, UPV/EHU Donos ia, Spain [email p o ec ed] Abs ac This pape p esen s a new ully au o- ma ic me hod o building highly dense and accu a e knowledge bases om ex- is ing seman ic esou ces. Basically, he me hod uses a wide-co e age and accu- a e knowledge-based Wo d Sense Dis- ambigua ion algo i hm o assign he mos app op ia e senses o la ge se s o opi- cally ela ed wo ds acqui ed om he web. KnowNe , he esul ing knowledge-base which connec s la ge se s o seman ically- ela ed concep s is a majo s ep owa ds he au onomous acquisi ion o knowledge om aw co po a. In ac , KnowNe is se - e al imes la ge han any a ailable knowl- edge esou ce encoding ela ions be ween synse s, and he knowledge KnowNe con- ains ou pe o m any o he esou ce when is empi ically e alua ed in a common amewo k. 1 In oduc ion Using la ge-scale knowledge bases, such as Wo d- Ne (Fellbaum, 1998), has become a usual, o - en necessa y, p ac ice o mos cu en Na u al Language P ocessing (NLP) sys ems. E en now, building la ge and ich enough knowledge bases o b oad–co e age seman ic p ocessing akes a g ea deal o expensi e manual e o in ol ing la ge esea ch g oups du ing long pe iods o de- elopmen . In ac , hund eds o pe son-yea s ha e been in es ed in he de elopmen o wo dne s o a ious languages (Vossen, 1998). Fo example, in c 2008. Licensed unde he C ea i e Commons A ibu ion-Noncomme cial-Sha e Alike 3.0 Unpo ed li- cense (h p://c ea i ecommons.o g/licenses/by-nc-sa/3.0/). Some igh s ese ed. mo e han en yea s o manual cons uc ion ( om 1995 o 2006, ha is om e sion 1.5 o 3.0), Wo dNe g ew om 103,445 o 235,402 seman ic ela ions1. Bu his da a does no seem o be ich enough o suppo ad anced concep -based NLP applica ions di ec ly. I seems ha applica ions will no scale up o wo k in open domains wi hou mo e de ailed and ich gene al-pu pose (and also domain-speci ic) seman ic knowledge buil by au- oma ic means. Ob iously, his ac has se e ely hampe ed he s a e-o - he-a o ad anced NLP ap- plica ions. Howe e , he P ince on Wo dNe (WN) is by a he mos widely-used knowledge base (Fellbaum, 1998). In ac , Wo dNe is being used wo ld-wide o ancho ing di e en ypes o seman ic knowl- edge including wo dne s o languages o he han English (A se ias e al., 2004), domain knowledge (Magnini and Ca agli` a, 2000) o on ologies like SUMO (Niles and Pease, 2001) o he Eu oWo d- Ne Top Concep On ology ( ´ Al ez e al., 2008). I con ains manually coded in o ma ion abou En- glish nouns, e bs, adjec i es and ad e bs and is o ganized a ound he no ion o a synse . A synse is a se o wo ds wi h he same pa -o -speech ha can be in e changed in a ce ain con ex . Fo ex- ample, <pa y,poli ical pa y> o m a synse be- cause hey can be used o e e o he same concep . A synse is o en u he desc ibed by a gloss, in his case: ”an o ganiza ion o gain poli ical powe ” and by explici seman ic ela ions o o he synse s. Fo una ely, du ing he las yea s he esea ch communi y has de ised a la ge se o inno a i e me hods and ools o la ge-scale au oma ic acqui- si ion o lexical knowledge om s uc u ed and un- s uc u ed co po a. Among o he s we can men- 1Symme ic ela ions a e coun ed only once. ion eX ended Wo dNe (Mihalcea and Moldo an, 2001), la ge collec ions o seman ic p e e ences acqui ed om SemCo (Agi e and Ma inez, 2001; Agi e and Ma inez, 2002) o acqui ed om B i ish Na ional Co pus (BNC) (McCa hy, 2001), la ge-scale Topic Signa u es o each synse ac- qui ed om he web (Agi e and de Lacalle, 2004) o knowledge abou indi iduals om Wikipedia (Suchanek e al., 2007). Ob iously, all hese se- man ic esou ces ha e been acqui ed using a e y di e en me hods, ools and co po a. As expec ed, each seman ic esou ce has di e en olume and accu acy igu es when e alua ed in a common and con olled amewo k (Cuad os and Rigau, 2006). Howe e , no all hese la ge-scale esou ces en- code seman ic ela ions be ween synse s. In some cases, only ela ions be ween synse s and wo ds ha e been acqui ed. This is he case o he Topic Signa u es acqui ed om he web (Agi e and de Lacalle, 2004). This is one o he la ges seman- ic esou ces e e buil wi h a ound one hund ed million ela ions be ween synse s and seman ically ela ed wo ds 2. A knowledge ne o KnowNe (KN), is an ex en- sible, la ge and accu a e knowledge base, which has been de i ed by seman ically disambigua ing small po ions o he Topic Signa u es acqui ed om he web. Basically, he me hod uses a o- bus and accu a e knowledge-based Wo d Sense Disambigua ion algo i hm o assign he mos ap- p op ia e senses o he opic wo ds associa ed o a pa icula synse . The esul ing knowledge-base which connec s la ge se s o opically- ela ed con- cep s is a majo s ep owa ds he au onomous ac- quisi ion o knowledge om aw ex . Table 1 compa es he di e en olumes o se- man ic ela ions be ween synse pai s o a ail- able knowledge bases and he newly c ea ed KnowNe s3. Va ying om i e o wen y he numbe o p o- cessed wo ds om each Topic Signa u e, we c e- a ed au oma ically ou di e en KnowNe e - sions wi h millions o new seman ic ela ions be- ween synse s. In ac , KnowNe is se e al imes la ge han Wo dNe , and when e alua ed empi - ically in a common amewo k, he knowledge i con ains ou pe o ms any o he seman ic esou ce. A e his in oduc ion, sec ion 2 desc ibes he Topic Signa u es acqui ed om he web. Sec ion 2A ailable a h p://ixa.si.ehu.es/Ixa/ esou ces/senseco pus 3These KnowNe e sions a e a ailable a h p://adimen.si.ehu.es Sou ce # ela ions P ince on WN3.0 235,402 Selec ional P e e ences om SemCo 203,546 eX ended WN 550,922 Co-occu ing ela ions om SemCo 932,008 New KnowNe -5 231,163 New KnowNe -10 689,610 New KnowNe -15 1,378,286 New KnowNe -20 2,358,927 Table 1: Numbe o synse ela ions 3 p esen s he app oach we ollowed o building highly dense and accu a e knowledge bases om he Topic Signa u es. In sec ion 4, we p esen he e alua ion amewo k used in his s udy. Sec ion 5 desc ibes he esul s when e alua ing di e en e - sions o KnowNe and inally, sec ion 6 p esen s some concluding ema ks and u u e wo k. 2 Topic Signa u es Topic Signa u es (TS) a e wo d ec o s ela ed o a pa icula opic (Lin and Ho y, 2000). Topic Sig- na u es a e buil by e ie ing con ex wo ds o a a ge opic om a la ge co po a. This s udy con- side s wo d senses as opics. Basically, he acqui- si ion o TS consis s o : •acqui ing he bes possible co pus examples o a pa icula wo d sense (usually cha ac e - izing each wo d sense as a que y and pe o m- ing a sea ch on he co pus o hose examples ha bes ma ch he que ies) •building he TS by selec ing he con ex wo ds ha bes ep esen he wo d sense om he selec ed co po a. The Topic Signa u es acqui ed om he web (he eina e TSWEB) cons i u es one o he la ges seman ic esou ce a ailable wi h a ound 100 mil- lion ela ions (be ween synse s and wo ds) (Agi e and de Lacalle, 2004). Inspi ed by he wo k o (Leacock e al., 1998), TSWEB was cons uc ed using monosemous ela i es om WN (synonyms, hype nyms, di ec and indi ec hyponyms, and sib- lings), que ying Google and e ie ing up o one housand snippe s pe que y ( ha is, a wo d sense), ex ac ing he salien wo ds wi h dis inc i e e- quency using TFIDF. Thus, TSWEB consis o la ge o de ed lis s o wo ds wi h weigh s associ- a ed o he polysemous nouns o WN1.6. The numbe o cons uc ed opic signa u es is 35,250 wi h an a e age size pe signa u e o 6,877 wo ds. ammany#n 0.0319 ede alis #n 0.0315 whig#n 0.0300 missiona y#j 0.0229 Democ a ic#n 0.0218 nazi#j 0.0202 epublican#n 0.0189 cons i u ional#n 0.0186 conse a i e#j 0.0148 socialis #n 0.0140 Table 2: TS o pa y#n#1 ( i s 10 ou o 12,890 o al wo ds) When e alua ing TSWEB, we used a maximum he i s 700 wo ds while o building KnowNe we used a maximum he i s 20 wo ds. Fo example, able 2 p esen s he i s wo ds (lemmas and pa -o -speech) and weigh s o he Topic Signa u e acqui ed o pa y#n#14. 3 Building highly connec ed and dense knowledge bases We acqui ed by ully au oma ic means highly connec ed and dense knowledge bases by disam- bigua ing small po ions o he Topic Signa u es ob ained om he web, inc easing he o al num- be o seman ic ela ions om less han one mil- lion ( he cu en numbe o a ailable ela ions) o millions o new and accu a e seman ic ela ions be ween synse s. We applied a knowledge–based all–wo ds Wo d Sense Disambigua ion algo i hm o he Topic Signa u es o de i ing a sense ec o om each wo d ec o . 3.1 SSI-Dijks a We ha e implemen ed a e sion o he S uc- u al Seman ic In e connec ions algo i hm (SSI), a knowledge-based i e a i e app oach o Wo d Sense Disambigua ion (Na igli and Vela di, 2005). The SSI algo i hm is e y simple and consis s o an ini- ializa ion s ep and a se o i e a i e s eps (see al- go i hm 1). Gi en W, an o de ed lis o wo ds o be dis- ambigua ed, he SSI algo i hm pe o ms as ol- lows. Du ing he ini ializa ion s ep, all monose- mous wo ds a e included in o he se I o al eady in e p e ed wo ds, and he polysemous wo ds a e included in P (all o hem pending o be disam- bigua ed). A each s ep, he se I is used o disam- bigua e one wo d o P, selec ing he wo d sense which is close o he se I o al eady disam- 4This o ma s ands o wo d#pos#sense. bigua ed wo ds. Once a sense is selec ed, he wo d sense is emo ed om P and included in o I. The algo i hm inishes when no mo e pending wo ds emain in P. Algo i hm 1 SSI-Dijks a Algo i hm SSI (T: lis o e ms) o each { ∈T}do I[ ] = ∅ i is monosemous hen I[ ] := he only sense o else P:= P∪ { } end i end o epea P0:= P o each { ∈P}do Bes Sense := ∅ MaxV alue := 0 o each {sense s o }do W[s] := 0 N[s] := 0 o each {sense s0∈I}do w:= Dijsk aSho es Pa h(s, s0) i w > 0 hen W[s] := W[s] + (1/w) N[s] := N[s]+1 end i end o i N[s]>0 hen NewV alue := W[s]/N[s] i NewV alue > MaxV alue hen MaxV alue := NewV alue Bes Sense := s end i end i end o i MaxV alue > 0 hen I[ ] := Bes Sense P:= P { } end i end o un il P6=P0 e u n (I, P); Ini ially, he lis I o in e p e ed wo ds should in- clude he senses o he monosemous wo ds in W, o a ixed se o wo d senses5. Howe e , when dis- 5I no monosemous wo ds a e ound o i no ini ial senses a e p o ided, he algo i hm could make an ini ial guess based on he mos p obable sense o he less ambiguous wo d o W. ambigua ing a TS o a wo d sense s( o ins ance pa y#n#1), he lis I al eady includes s. In o de o measu e he p oximi y o one synse o he es o synse s o I, we use pa o he knowledge al eady a ailable o build a e y la ge connec ed g aph wi h 99,635 nodes (synse s) and 636,077 edges. This g aph includes he se o di ec ela ions be ween synse s ga he ed om Wo dNe and eX ended Wo dNe . On ha g aph, we used a e y e icien g aph lib a y, Boos - G aph6 o compu e he Dijks a algo i hm. The Dijks a algo i hm is a g eedy algo i hm o com- pu ing he sho es pa h dis ance be ween one node an he es o nodes o a g aph. In ha way, we can compu e e y e icien ly he sho es dis ance be- ween any wo gi en nodes o a g aph. We call his e sion o he SSI algo i hm, SSI-Dijks a. SSI-Dijks a has e y in e es ing p ope ies. Fo ins ance, i always p o ides he minimum dis ance be ween wo synse s. Tha is, he algo i hm always p o ides an answe being he minimum dis ance close o a . In con as , he o iginal SSI algo i hm no always p o ides a pa h dis ance because i de- pends on a p ede ined g amma o seman ic ela- ions. In ac , he SSI-Dijks a algo i hm compa es he dis ances be ween he synse s o a wo d and all he synse s al eady in e p e ed in I. A each s ep, he SSI-Dijks a algo i hm selec s he synse which is close o I ( he se o al eady in e p e ed wo ds). Fu he mo e, his app oach is comple ely lan- guage independen . The same g aph can be used o any language ha ing wo ds connec ed o Wo d- Ne . 3.2 Building KnowNe We de eloped KnowNe (KN), a la ge-scale and ex ensible knowledge base, by applying SSI- Dijks a o each opic signa u e om TSWEB. We ha e gene a ed ou di e en e sions o KnowNe applying SSI-Dijks a o only he i s 5, 10, 15 and 20 wo ds o each TS. SSI-Dijks a used only he knowledge p esen in Wo dNe and eX ended Wo dNe which consis o a e y la ge connec ed g aph wi h 99,635 nodes (synse s) and 636,077 edges (seman ic ela ions). We gene a ed each KnowNe by applying he SSI-Dijks a algo i hm o he whole TSWEB (p o- cessing he i s wo ds o each o he 35,250 opic signa u es). Fo each TS, we ob ained he di ec ela ions om he opic (a wo d sense) 6h p://www.boos .o g KB WN+XWN # ela ions #synse s KN-5 3,1% 231,163 39,864 KN-10 5,0% 689,610 45,817 KN-15 6,9% 1,378,286 48,521 KN-20 8,5% 2,358,927 50,789 Table 3: Size and pe cen age o o e lapping ela- ions be ween KnowNe e sions and WN+XWN o he disambigua ed wo d senses o he TS ( o ins ance, pa y#n#1–> ede alis #n#1), bu also he indi ec ela ions be ween disambigua ed wo ds om he TS ( o ins ance, ede alis #n#1– > epublican#n#1). Finally, we emo ed symme - ic and epea ed ela ions. Table 3 shows he o e laping pe cen age be- ween each KnowNe and he knowledge con- ained in o Wo dNe and eX ended Wo dNe , and he o al numbe o ela ions and synse s o each esou ce. Fo ins ance, only 8,5% o he o al di- ec ela ions included in o WN+XWN a e also p esen in KnowNe -20. This means ha he es o ela ions om KnowNe -20 a e new. As ex- pec ed, each KnowNe is e y la ge, anging om hund eds o housands o millions o new seman ic ela ions be ween synse s among inc easing se s o synse s. 4 E alua ion amewo k In o de o empi ically es ablish he ela i e qual- i y o hese new seman ic esou ces, we used he e alua ion amewo k o ask 16 o SemE al-2007: E alua ion o wide co e age knowledge esou ces (Cuad os and Rigau, 2007). In his amewo k all knowledge esou ces a e e alua ed on a common WSD ask. In pa icu- la , we used he noun-se s o he English Lexi- cal Sample ask o Sense al-3 and SemE al-2007 exe cises which consis s o 20 and 35 nouns e- spec i ely. All pe o mances a e e alua ed on he es da a using he ine-g ained sco ing sys em p o- ided by he o ganize s. Fu he mo e, ying o be as neu al as possible wi h espec o he esou ces s udied, we applied sys ema ically he same disambigua ion me hod o all o hem. Recall ha ou main goal is o es- ablish a ai compa ison o he knowledge e- sou ces a he han p o iding he bes disambigua- ion echnique o a pa icula knowledge base. All knowledge bases a e e alua ed as opic signa u es. Tha is, wo d ec o s wi h weigh s associa ed o a pa icula synse which a e ob ained by collec ing hose wo d senses appea ing in he synse s di ec ly ela ed o he opics. This simple ep esen a ion ies o be as neu al as possible wi h espec o he esou ces used. A common WSD me hod has been applied o all knowledge esou ces. A simple wo d o e lapping coun ing is pe o med be ween he opic signa u e ep esen ing a wo d sense and he es example7. The synse ha ing highe o e lapping wo d coun s is selec ed. In ac , his is a e y simple WSD me hod which only conside s he opical in o ma- ion a ound he wo d o be disambigua ed. Finally, we should ema k ha he esul s a e no skewed ( o ins ance, o esol ing ies) by he mos e- quen sense in WN o any o he s a is ically p e- dic ed knowledge. 4.1 Baselines We ha e designed a numbe o baselines in o de o es ablish a comple e e alua ion amewo k o compa ing he pe o mance o each seman ic e- sou ce on he English WSD asks. RANDOM: Fo each a ge wo d, his me hod selec s a andom sense. This baseline can be con- side ed as a lowe -bound. SEMCOR-MFS: This baseline selec s he mos equen sense o he a ge wo d in SemCo . WN-MFS: This baseline is ob ained by se- lec ing he mos equen sense ( he i s sense in WN1.6) o he a ge wo d. Wo dNe wo d- senses we e anked using SemCo and o he sense- anno a ed co po a. Thus, WN-MFS and SemCo - MFS a e simila , bu no equal. TRAIN-MFS: This baseline selec s he mos equen sense in he aining co pus o he a ge wo d. TRAIN: This baseline uses he aining co pus o di ec ly build a Topic Signa u e using TFIDF measu e o each wo d sense and selec ing a max- imum he i s 450 wo ds. No e ha in WSD e al- ua ion amewo ks, his is a e y basic baseline. Howe e , in ou e alua ion amewo k, his ”WSD baseline” could be conside ed as an uppe -bound. We do no expec o ob ain be e opic signa u es o a pa icula sense han om i s own anno a ed co pus. 4.2 O he La ge-scale Knowledge Resou ces In o de o measu e he ela i e quali y o he new esou ces, we include in he e alua ion a wide 7We also conside hose mul iwo d e ms appea ing in WN. ange o la ge-scale knowledge esou ces con- nec ed o Wo dNe . WN (Fellbaum, 1998): This esou ce uses he di e en di ec ela ions encoded in WN1.6 and WN2.0. We also es ed WN2using ela ions a dis- ance 1 and 2, WN3using ela ions a dis ances 1 o 3 and WN4using ela ions a dis ances 1 o 4. XWN (Mihalcea and Moldo an, 2001): This esou ce uses he di ec ela ions encoded in eX- ended WN. spBNC (McCa hy, 2001): This esou ce con- ains 707,618 selec ional p e e ences acqui ed o subjec s and objec s om BNC. spSemCo (Agi e and Ma inez, 2002): This esou ce con ains he selec ional p e e ences ac- qui ed o subjec s and objec s om SemCo . MCR (A se ias e al., 2004): This esou ce in- eg a es he di ec ela ions o WN, XWN and spSemCo . TSSEM (Cuad os e al., 2007): These Topic Signa u es ha e been cons uc ed using Sem- Co .Fo each wo d-sense appea ing in SemCo , we ga he all sen ences o ha wo d sense, building a TS using TFIDF o all wo d-senses co-occu ing in hose sen ences. 4.3 In eg a ed Knowledge Resou ces We also e alua ed he pe o mance o he in eg a- ion ( emo ing duplica ed ela ions) o some o hese esou ces. WN+XWN: This esou ce in eg a es he di- ec ela ions o WN and XWN. We also es ed (WN+XWN)2(using ei he WN o XWN ela- ions a dis ances 1 and 2). MCR (A se ias e al., 2004): This esou ce in- eg a es he di ec ela ions o WN, XWN and spSemCo . WN+XWN+KN-20: This esou ce in eg a es he di ec ela ions o WN, XWN and KnowNe - 20. 5 KnowNe E alua ion We e alua ed KnowNe using he same amewo k explained in sec ion 4. Tha is, he noun pa o he es se om he English Sense al-3 and SemE al- 2007 English lexical sample asks. 5.1 Sense al-3 e alua ion Table 4 p esen s o de ed by F1 measu e, he pe - o mance in e ms o p ecision (P), ecall (R) and KB P R F1 A . Size TRAIN 65.1 65.1 65.1 450 TRAIN-MFS 54.5 54.5 54.5 WN-MFS 53.0 53.0 53.0 TSSEM 52.5 52.4 52.4 103 SEMCOR-MFS 49.0 49.1 49.0 MCR245.1 45.1 45.1 26,429 WN+XWN+KN-20 44.8 44.8 44.8 671 MCR 45.3 43.7 44.5 129 KnowNe -20 44.1 44.1 44.1 610 KnowNe -15 43.9 43.9 43.9 339 spSemCo 43.1 38.7 40.8 56 KnowNe -10 40.1 40.0 40.0 154 (WN+XWN)238.5 38.0 38.3 5,730 WN+XWN 40.0 34.2 36.8 74 TSWEB 36.1 35.9 36.0 1,721 XWN 38.8 32.5 35.4 69 KnowNe -5 35.0 35.0 35.0 44 WN335.0 34.7 34.8 503 WN433.2 33.1 33.2 2,346 WN233.1 27.5 30.0 105 spBNC 36.3 25.4 29.9 128 WN 44.9 18.4 26.1 14 RANDOM 19.1 19.1 19.1 Table 4: P, R and F1 ine-g ained esul s o he esou ces e alua ed a Sense al-3, English Lexical Sample Task. F1 measu e (F1, ha monic mean o ecall and p e- cision) o each knowledge esou ce on Sense al-3 and he a e age size o he TS pe wo d-sense. The di e en KnowNe e sions appea ma ked in bold and he baselines appea in i alics. As expec ed, RANDOM ob ains he poo es e- sul . The mos equen senses ob ained om Sem- Co (SEMCOR-MFS) and WN (WN-MFS) a e bo h below he mos equen sense o he aining co pus (TRAIN-MFS). Howe e , all o hem a e a below o he Topic Signa u es acqui ed using he aining co pus (TRAIN). The bes esul s a e ob ained by TSSEM (wi h F1 o 52.4). The lowes esul is ob ained by he knowledge di ec ly ga he ed om WN mainly be- cause o i s poo co e age (R o 18.4 and F1 o 26.1). In e es ingly, he knowledge in eg a ed in he MCR al hough pa ly de i ed by au oma ic means pe o ms much be e in e ms o p ecision, ecall and F1 measu es han using hem sepa a ely (F1 wi h 18.4 poin s highe han WN, 9.1 han XWN and 3.7 han spSemCo ). Despi e i s small size, he esou ces de i ed om SemCo ob ain be e esul s han i s coun- e pa s using much la ge co po a (TSSEM s. TSWEB and spSemCo s. spBNC). Rega ding he baselines, all knowledge e- sou ces su pass RANDOM, bu none achie es nei- he WN-MFS, TRAIN-MFS no TRAIN. Only TSSEM ob ains be e esul s han SEMCOR-MFS and is e y close o he mos equen sense o WN (WN-MFS) and he aining (TRAIN-MFS). Rega ding he expansions and combina ions, he pe o mance o WN is imp o ed using wo ds a dis ances up o 2, and up o 3, bu i dec eases using dis ances up o 4. In e es ingly, none o hese WN expansions achie e he esul s o XWN. Finally, (WN+XWN)2pe o ms be e han WN+XWN and MCR2sligh ly be e han MCR8. The di e en e sions o KnowNe consis en ly ob ain be e pe o mances as hey inc ease he window size o p ocessed wo ds o TSWEB. As expec ed, KnowNe -5 ob ain he lowe esul s. Howe e , i pe o ms be e han WN (and all i s ex ensions) and spBNC. In e es ingly, om KnowNe -10, all KnowNe e sions su pass he knowledge esou ces used o hei cons uc ion (WN, XWN, TSWEB and WN+XWN). In ac , KnowNe -10 also ou pe o ms (WN+XWN)2wi h much mo e ela ions pe sense. Also in e es ing is ha KnowNe -10 and KnowNe -20 ob ain be - e pe o mance han spSemCo which was de i ed om anno a ed co po a. Howe e , KnowNe -20 only pe o ms sligh ly be e han KnowNe -15 while almos doubling he numbe o ela ions. These ini ial esul s seem o be e y p omis- ing. I we do no conside he esou ces de i ed om manually sense anno a ed da a (spSemCo , MCR, TSSEM, e c.), KnowNe -10 pe o ms be - e ha any knowledge esou ce de i ed by man- ual o au oma ic means. In ac , KnowNe -15 and KnowNe -20 ou pe o ms spSemCo which was de i ed om manually anno a ed co po a. This is a e y in e es ing esul since hese KnowNe e - sions ha e been de i ed only wi h he knowledge coming om WN and he web ( ha is, TSWEB), and WN and XWN as a knowledge sou ce o SSI- Dijks a9. Rega ding he in eg a ion o esou ces, WN+XWN+KN-20 pe o ms be e han MCR and simila ly o MCR2(ha ing less han 50 imes i s size). Also in e es ing is ha WN+XWN+KN- 20 ha e be e pe o mance han hei indi idual esou ces, indica ing a complemen a y knowledge. In ac , WN+XWN+KN-20 pe o ms much be e han he esou ces om which i de i es (WN, XWN and TSWEB). 8No u he dis ances ha e been es ed 9eX ended Wo dNe only has 17,185 manually labeled senses. KB P R F1 A . Size TRAIN 87.6 87.6 87.6 450 TRAIN-MFS 81.2 79.6 80.4 WN-MFS 66.2 59.9 62.9 WN+XWN+KN-20 53.0 53.0 53.0 627 (WN+XWN)254.9 51.1 52.9 5,153 TSWEB 54.8 47.8 51.0 700 KnowNe -20 49.5 46.1 47.7 561 KnowNe -15 47.0 43.5 45.2 308 XWN 50.1 39.8 44.4 96 KnowNe -10 44.0 39.8 41.8 139 WN+XWN 45.4 36.8 40.7 101 SEMCOR-MFS 42.4 38.4 40.3 MCR 40.2 35.5 37.7 149 TSSEM 35.1 32.7 33.9 428 KnowNe -5 35.5 26.5 30.3 41 MCR232.4 29.5 30.9 24,896 WN329.3 26.3 27.7 584 RANDOM 27.4 27.4 27.4 WN225.9 27.4 26.6 72 spSemCo 31.4 23.0 26.5 51.0 WN426.1 23.9 24.9 2,710 WN 36.8 16.1 22.4 13 spBNC 24.4 18.1 20.8 290 Table 5: P, R and F1 ine-g ained esul s o he e- sou ces e alua ed a SemE al-2007, English Lexi- cal Sample Task. 5.2 SemE al-2007 e alua ion Table 5 p esen s o de ed by F1 measu e, he pe - o mance in e ms o p ecision (P), ecall (R) and F1 measu e (F1) o each knowledge esou ce on SemE al-2007 and i s a e age size o he TS pe wo d-sense10. Again, he di e en KnowNe e - sions appea ma ked in bold and he baselines ap- pea in i alics. As in he p e ious e alua ion, RANDOM ob- ains he poo es esul . The mos equen senses ob ained om SemCo (SEMCOR-MFS) and WN (WN-MFS) a e bo h a below he mos equen sense o he aining co pus (TRAIN-MFS), and all o hem a e below he Topic Signa u es acqui ed using he aining co pus (TRAIN). In e es ingly, on SemE al-2007, all he knowl- edge esou ces beha e di e en ly. Now, he bes indi idual esul s a e ob ained by TSWEB, while in his case TSSEM ob ains e y modes esul s. The lowes esul is ob ained by he knowledge en- coded in spBNC. Rega ding he baselines, spBNC, WN (and also WN2and WN4) and spSemCo do no su pass RANDOM, and none achie es nei he WN-MFS, TRAIN-MFS no TRAIN. Now, WN+XWN, XWN, TSWEB and (WN+XWN)2ob ain be e 10The a e age size is di e en wi h espec Sense al-3 be- cause he wo ds selec ed o his ask a e di e en esul s han SEMCOR-MFS bu a below he mos equen sense o WN (WN-MFS) and he aining (TRAIN-MFS). Rega ding o he expansions and combina ions, he pe o mance o WN is imp o ed using wo ds a dis ances up o 2, and up o 3, bu i dec eases using dis ances up o 4. Again, none o hese WN expansions achie e he esul s o XWN. Finally, (WN+XWN)2pe o ms be e han WN+XWN and MCR2sligh ly be e han MCR11. On SemE al-2007, he di e en e sions o KnowNe consis en ly ob ain be e pe o mances as hey incease he window size o p ocessed wo ds o TSWEB. As expec ed, KnowNe -5 ob- ain he lowe esul s. Howe e , i pe o ms be e han spBNC, WN (and all i s ex ensions), spSem- Co and MCR2. This ime, all KnowNe e - sions pe o m wo se han TSWEB. Howe e , as in he p e ious e alua ion, KnowNe -10 ou pe o ms WN+XWN, and his ime, also TSSEM and he MCR, wi h much mo e ela ions pe sense. Also in e es ing is ha om KnowNe -10, all KnowNe e sions pe o m be e han he esou ces de i ed om manually sense anno a ed co po a (spSem- Co , MCR, TSSEM, e c.). Rega ding he in eg a ion o esou ces, WN+XWN+KN-20 pe o ms be e han any knowledge esou ce de i ed by manual o au o- ma ic means. Again, i is in e es ing o no e ha WN+XWN+KN-20 ha e be e pe o mance han hei indi idual esou ces, indica ing a comple- men a y knowledge. In ac , WN+XWN+KN-20 pe o ms much be e han he esou ces om which i de i es (WN, XWN and TSWEB). 5.3 Discussion When compa ing he anking o he di e en knowledge esou ces, he di e en e sions o KnowNe seem o be mo e obus and s able ac oss co po a changes. Fo ins ance, in bo h e alua ion amewo ks (Sense al-3 and SemE al- 2007), KnowNe -20 anks 5 h and 4 h, espec- i ely ((WN+XWN)2 anks 8 h and 2nd, TSSEM anks 1s and 10 h, MCR anks 4 h and 9 h, TSWEB anks 11 h and 3 d, e c.). In ac , WN+XWN+KN-20 anks 3 d and 1s , espec- i ely. 11No u he dis ances ha e been es ed 6 Conclusions and u u e esea ch I is ou belie , ha accu a e seman ic p ocessing (such as WSD) would ely no only on sophis i- ca ed algo i hms bu on knowledge in ensi e ap- p oaches. The esul s p esen ed in his pape sug- ges s ha much mo e esea ch on acqui ing and using la ge-scale seman ic esou ces should be ad- d essed. The knowledge acquisi ion bo leneck p oblem is pa icula ly acu e o open domain (and also domain speci ic) seman ic p ocessing. The ini- ial esul s ob ained o he di e en e sions o KnowNe seem o be a majo s ep owa ds he au- onomous acquisi ion o knowledge om aw co - po a, since hey a e se e al imes la ge han he a ailable knowledge esou ces which encode e- la ions be ween synse s, and he knowledge hey con ain ou pe o m any o he esou ce when is em- pi ically e alua ed in a common amewo k. I emains o u u e esea ch he e alua ion o hese KnowNe e sions in combina ion wi h o he la ge-scale seman ic esou ces o in a c oss-lingual se ing. Acknowledgmen s We wan o hank Ai o So oa o his echnical suppo and he anonymous e iewe s o hei commen s. This wo k has been suppo ed by KNOW (TIN2006-15049-C03-01) and KYOTO (ICT-2007-211423). Re e ences Agi e, E. and O. Lopez de Lacalle. 2004. Publicly a ailable opic signa u es o all wo dne nominal senses. In P oceedings o LREC, Lisbon, Po ugal. Agi e, E. and D. Ma inez. 2001. Lea ning class- o-class selec ional p e e ences. In P oceedings o CoNLL, Toulouse, F ance. Agi e, E. and D. Ma inez. 2002. In eg a ing selec- ional p e e ences in wo dne . In P oceedings o GWC, Myso e, India. ´ Al ez, J., J. A se ias, J. Ca e a, S. Climen , A. Oli e , and G. Rigau. 2008. Consis en anno a ion o eu- owo dne wi h he op concep on ology. In P o- ceedings o Fou h In e na ional Wo dNe Con e - ence (GWC’08). A se ias, J., L. Villa ejo, G. Rigau, E. Agi e, J. Ca - oll, B. Magnini, and Piek Vossen. 2004. The mean- ing mul ilingual cen al eposi o y. In P oceedings o GWC, B no, Czech Republic. Cuad os, M. and G. Rigau. 2006. Quali y assessmen o la ge scale knowledge esou ces. In P oceedings o he EMNLP. Cuad os, M. and G. Rigau. 2007. Seme al-2007 ask 16: E alua ion o wide co e age knowledge e- sou ces. In P oceedings o he Fou h In e na ional Wo kshop on Seman ic E alua ions (SemE al-2007). Cuad os, M., G. Rigau, and M. Cas illo. 2007. E al- ua ing la ge-scale knowledge esou ces ac oss lan- guages. In P oceedings o RANLP. Fellbaum, C., edi o . 1998. Wo dNe . An Elec onic Lexical Da abase. The MIT P ess. Leacock, C., M. Chodo ow, and G. Mille . 1998. Using Co pus S a is ics and Wo dNe Rela ions o Sense Iden i ica ion. Compu a ional Linguis ics, 24(1):147–166. Lin, C. and E. Ho y. 2000. The au oma ed acquisi- ion o opic signa u es o ex summa iza ion. In P oceedings o COLING. S asbou g, F ance. Magnini, B. and G. Ca agli` a. 2000. In eg a ing subjec ield codes in o wo dne . In P oceedings o LREC, A hens. G eece. McCa hy, D. 2001. Lexical Acquisi ion a he Syn ax- Seman ics In e ace: Dia hesis A e na ions, Sub- ca ego iza ion F ames and Selec ional P e e ences. Ph.D. hesis, Uni e si y o Sussex. Mihalcea, R. and D. Moldo an. 2001. ex ended wo d- ne : P og ess epo . In P oceedings o NAACL Wo kshop on Wo dNe and O he Lexical Resou ces, Pi sbu gh, PA. Na igli, R. and P. Vela di. 2005. S uc u al seman- ic in e connec ions: a knowledge-based app oach o wo d sense disambigua ion. IEEE T ansac ions on Pa e n Analysis and Machine In elligence (PAMI), 27(7):1063–1074. Niles, I. and A. Pease. 2001. Towa ds a s anda d up- pe on ology. In P oceedings o he 2nd In e na- ional Con e ence on Fo mal On ology in In o ma- ion Sys ems (FOIS-2001), pages 17–19. Ch is Wel y and Ba y Smi h, eds. Suchanek, Fabian M., Gje gji Kasneci, and Ge ha d Weikum. 2007. Yago: A Co e o Seman ic Knowl- edge. In 16 h in e na ional Wo ld Wide Web con- e ence (WWW 2007), New Yo k, NY, USA. ACM P ess. Vossen, P., edi o . 1998. Eu oWo dNe : A Mul ilingual Da abase wi h Lexical Seman ic Ne wo ks . Kluwe Academic Publishe s .