Tí ol: Causal Rela ion Ex ac ion
Volum: I
Alumne: Edua do Blanco Villa
Di ec o /Ponen : Dan Moldo an / Nu ia Cas ell A iño
Depa amen : LSI
Da a: 7 no emb e 2007
DADES DEL PROJECTE
Tí ol del P ojec e: Causal Rela ion Ex ac ion
Nom de l'es udian : Edua do Blanco Villa
Ti ulació: Enginye ia en In o mà ica
C èdi s: 37.5
Di ec o /Ponen : Dan Moldo an / Nu ia Cas ell A iño
Depa amen : LSI
MEMBRES DEL TRIBUNAL (nom i signa u a)
P esiden :
Vocal:
Sec e a i:
QUALIFICACIÓ
Quali icació numè ica:
Quali icació desc ip i a:
Da a:
Causal Rela ion Ex ac ion
Edua do Blanco
No embe 7, 2007
1

2
P e acio
El p esen e documen o desc ibe el abajo hecho du an e la ealizaci´on del P oyec o
Final de Ca e a (PFC), ´ul imo equisi o pa a ob ene los ´ı ulos de Enginye ia en
In o m`a ica yM`as e Eu opeu en Llengua ge i Pa la po la Uni e si a Poli `ecnica
de Ca alunya (UPC).
El abajo ha sido ealizado en el Human Language Technology Resea ch Ins i u e
(HLTRI), pe enecien e a The Uni e si y o Texas a Dallas (UTD). El di ec o es
Dan Moldo an, Co-Di ec o del HLTRI. Nu ia Cas ell es la p o eso a de la UPC
que me puso en con ac o con Dan Moldo an y sigui´o mis pasos en el HLTRI. La
es ancia en el HLTRI se ex endi´o desde el 1 de eb e o al 31 de agos o de 2007,
comp endiendo un o al de sie e meses.
El PFC puede se cali icado como un abajo de in es igaci´on. Se han es udiado
las elaciones sem´an icas de causalidad y se p opone un m´e odo pa a su de ecci´on
y ex acci´on. No se p esen a un sis ema come cial o un p o o ipo lis o pa a su
u ilizaci´on di ec a.
Las a eas desa olladas han sido las ´ıpicas de es e ipo de abajos: lec u a
de li e a u a elacionada con el campo y an´alisis del es ado del a e (2 meses),
de inici´on de los obje i os y expe imen os a ealiza (1 mes), ealizaci´on de los
expe imen os y ex acci´on de esul ados (3 meses), y po ´ul imo, esc i u a de la
memo ia explicando el abajo ealizado (1 mes).
3
P e aci
Aques documen desc iu el eball e du an la eali zaci´o del P ojec e Final de
Ca e a (PFC), ´ul im equisi pe ob eni els ´ı ols d’Enginye ia en In o m`a ica i
M`as e Eu opeu en Llengua ge i Pa la pe la Uni e si a Poli `ecnica de Ca alunya
(UPC).
La eina ha sigu eali zada en el Human Language Technology Resea ch Ins i u e
(HLTRI), pe anyen a The Uni e si y o Texas a Dallas (UTD). El di ec o ´es
Dan Moldo an, Co-Di ec o del HLTRI. Nu ia Cas ell ´es la p o esso a de la UPC
que em a posa en con ac e amb Dan Moldo an i qui ha segui les me es passes
en el HLTRI. L’es ada al HLTRI a comen¸ca l’1 de eb e de 2007 i a acaba el
31 d’agos del ma eix any.
El PFC es po conside a un eball d’in es igaci´o. S’han es udia les elacions
sem`an iques de causali a y es p oposa un m`e ode pe la se a de ecci´o i ex acci´o.
No es p esen a un sis ema come cial ni un p o o ipus p epa a pe se u ili za
di ec amen .
Les asques desen olupades s´on les ´ıpiques d’un eball d’aques es ca ac e ´ıs iques:
lec u a de la li e a u a elacionada amb el camp y an`alisi de l’es a de l’a (2
mesos), de ini i´o dels objec ius i expe imen s a eali za (1 mes), eali zaci´o dels ex-
pe imen s i ex acci´o de esul a s (3 mesos), i pe ´ul im, esc ip u a de la mem`o ia
explican la eina e a (1 mes).
CONTENTS 4
Con en s
1 In oduc ion 6
2 Seman ic Rela ions 8
2.1 Examples o se s o seman ic ela ions . . . . . . . . . . . . . . . . 8
2.2 The exp ession o seman ic ela ions . . . . . . . . . . . . . . . . . 9
2.3 The di icul y o deciding he seman ic ela ion . . . . . . . . . . . 11
3 Causa ions 13
3.1 InPhilosophy ............................. 13
3.2 InPsychology............................. 14
3.3 In Theo e ical Linguis ics . . . . . . . . . . . . . . . . . . . . . . 15
3.3.1 In Typological Linguis ics . . . . . . . . . . . . . . . . . . 17
3.3.2 In he Cogni i e App oach . . . . . . . . . . . . . . . . . . 17
3.3.3 Wi hin Discou se Theo y . . . . . . . . . . . . . . . . . . . 18
3.4 Causa ion and o he seman ic ela ions . . . . . . . . . . . . . . . 18
3.5 Encoding o Causa ion . . . . . . . . . . . . . . . . . . . . . . . . 19
4 P e ious wo k 21
4.1 On Seman ic ela ions . . . . . . . . . . . . . . . . . . . . . . . . 21
4.2 Focused on causal ela ions . . . . . . . . . . . . . . . . . . . . . . 22
5 The me hod 25
5.1 Syn ac ic pa e ns ha encode causa ion . . . . . . . . . . . . . . 25
5.2 Pa e n ma ching . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
5.3 Machine Lea ning app oach . . . . . . . . . . . . . . . . . . . . . 30
5.3.1 Da a p epa a ion . . . . . . . . . . . . . . . . . . . . . . . 31
5.3.2 Fea u e selec ion . . . . . . . . . . . . . . . . . . . . . . . 31
5.3.3 Fea u e ex ac ion . . . . . . . . . . . . . . . . . . . . . . 35
5.3.4 Machine Lea ning algo i hm . . . . . . . . . . . . . . . . . 37
5.3.5 E alua ion........................... 39
5.3.6 E o analysis . . . . . . . . . . . . . . . . . . . . . . . . . 40
6 Limi a ions, Scope and Ex ensions 42
7 Conclusions and u he wo k 44
Re e ences 46
A Lis o POS ags 50
B Decision ees lea ned 51
B.1 Decision T ee no. 1: . . . . . . . . . . . . . . . . . . . . . . . . . . 51
B.2 Decision T ee no. 2: . . . . . . . . . . . . . . . . . . . . . . . . . . 51
CONTENTS 5
B.3 Decision T ee no. 3: . . . . . . . . . . . . . . . . . . . . . . . . . . 52
B.4 Decision T ee no. 4: . . . . . . . . . . . . . . . . . . . . . . . . . . 53
B.5 Decision T ee no. 5: . . . . . . . . . . . . . . . . . . . . . . . . . . 53
B.6 Decision T ee no. 6: . . . . . . . . . . . . . . . . . . . . . . . . . . 54
B.7 Decision T ee no. 7: . . . . . . . . . . . . . . . . . . . . . . . . . . 54
B.8 Decision T ee no. 8: . . . . . . . . . . . . . . . . . . . . . . . . . . 55
B.9 Decision T ee no. 9: . . . . . . . . . . . . . . . . . . . . . . . . . . 59
B.10 Decision T ee no. 10: . . . . . . . . . . . . . . . . . . . . . . . . . 59
2 SEMANTIC RELATIONS 12
o he o m [Noun Noun], 0.68 when classi ying complex nominals o he o m
[Adjec i e Noun], and 0.65 when classi ying geni i es. [Chang and Choi, 2006]
epo an ag eemen o 0.73 when classi ying manually a se o examples be ween
only wo classes: encoding causa ion and no encoding causa ion.
The low coe icien s gi e an idea o he di icul y o he ask: e en humans
anno a o s o en disag ee when classi ying manually.

3 CAUSATIONS 13
3 Causa ions
B oadly speaking, causa ion can be de ined as a ela ion be ween wo e en s:
cause and e ec . Cause is he p oduce o he e ec , and e ec he esul o he
cause. This de ini ion is ob iously ambiguous, and i doesn’ d aw a line be ween
wha is a causa ion and wha is no .
Causal ela ions ha e been s udied in se e al ields. Di e en heo ies o cau-
sa ion coexis in Philosophy, Psychology and Linguis ics.
3.1 In Philosophy
In his p esen a ion we ollow he ideas o [Whi e, 1990], which o e s summa y
ske ches o he mos impo an poin s o each heo y.
In Regula i y Theo ies [Hume, 1739], causa ion is seen as a cons an conjunc-
ion be ween e en s, associa ed wi h p io i y in ime and con igui y in ime and,
whe e ele an , space. The conjunc ion be ween e en s is hough o be impe -
ec , inde e mina e, asymme ic and spu ious. The e o e, causa ion is de ined as a
p obable conjunc ion o wo e en s and h ea ed using P obabili y Theo ies. The
basic o m o he heo y says ha one e en causes ano he i i is ollowed by i
and is such ha e en s o he i s kind a e egula ly ollowed by e en s o he
second kind.
Fo o he s, causa ion is a condi ion o he occu ence o an e en [Mill, 1843],
[Sosa, 1975], [Mackie, 1980] [Suppes, 1970]:
•A Su icien Condi ion
CAUSE(e en 1, e en 2), i he exis ence o e en 1implies he exis ence o
e en 2.
•A Necessa y and Su icien Condi ion
CAUSE(e en 1, e en 2), i he exis ence o e en 1implies he exis ence o
e en 2and he exis ence o e en 2implies he exis ence o e en 1.
•Insu icien bu Necessa y pa o an Unnecessa y bu Su icien
(INUS) Condi ion
CAUSE(e en 1, e en 2) i e en 1doesn’ imply he exis ence o e en 2, bu
is pa o a mo e complica ed condi ion ha imply e en 2.
A good example o unde s and INUS condi ions can be ound in [Hi ch-
cock, 2007]: “. . . an INUS condi ion o some e ec is an insu icien bu
non- edundan pa o an unnecessa y bu su icien condi ion. Suppose, o
example, ha a li ma ch causes a o es i e. The ligh ing o he ma ch,
by i sel , is no su icien ; many ma ches a e li wi hou ensuing o es i es.
The li ma ch is, howe e , a pa o some cons ella ion o condi ions ha
3 CAUSATIONS 14
a e join ly su icien o he i e. Mo eo e , gi en ha his se o condi-
ions occu ed, a he han some o he se su icien o i e, he ligh ing o
he ma ch was necessa y: i es do no occu in such ci cums ances when li
ma ches a e no p esen .”
The Coun e ac ual App oach [Lewis, 1973a,Lewis, 1973b] sees causa ion as
wha would ha e happened i some hing we e he case han in ac is no he
case. In o he wo ds, e en 1causes e en 2jus in case i is ue ha i e en 1had
no occu ed, hen e en 2would no ha e occu ed. Se e al esea che s c i icize
his app oach and p opose di e en solu ions o he p oblems o he heo y. Fo
example, [Mu ali, 1997] p oposes a solu ion o he p eemp ion p oblem, he
p oblem o agile causes and inde e minis ic causa ion.
3.2 In Psychology
Psychology is conce ned wi h how people unde s and and pe cei e causa ion, as
well as how hey make causal in e ences and a ibu ions. Wha a e he conse-
quences o hinking ha wo e en s a e causally ela ed is also s udied in Psy-
chology.
Di e en hypo hesis ha e been made abou he so o hings people may belie e
can be causes [Whi e, 1990]:
•E en s o happenings.
•S anding condi ions o s a es o a ai s, such as causal powe s o ma e ial
pa icula s.
•In e ac ions be ween occu ences and s able p ope ies o hings.
•Condi ions, such as necessa y and su icien condi ions o INUS condi ions.
Psychologis s ha e pe o med se e al expe imen s in o de o explain empi ically
he pe cep ion o causa ions by humans and he implica ions o conside ing wo
e en s casually ela ed. [Glymou , 2003] o e s a o malism based on Bayes ne s
which allows o ein e p e expe imen s on human judgmen , o e s a p ecise in e -
p e a ion o mechanisms and allows gene aliza ions o exis ing heo ies o causal
lea ning.
[Eagleman and Holcombe, 2002] p esen an expe imen ha ela es causali y and
he pe cep ion o ime. They no e ha i someone holds ou his hand and snaps
his inge s, he will no no ice any di e ence in he ime ha he in ends o mo e
his inge s and he ime he hea s he snap. Howe e , humans a e able o pe cei e
a ound 25 miliseconds di e ences in iming and he audi o y signals co espond-
ing o he snap we e being p ocessed by he ne ous sys em o mo e han 100
miliseconds.
3 CAUSATIONS 15
In a e y simple expe imen , subjec s olun a ily p essed a key ha caused a
one o ollow. When asked o judge he ime o he keyp ess and he ime o he
one, subjec s pe cei ed he keyp ess o occu la e and he one ea lie han i
hese e en s had occu ed alone. The au ho s conclude ha when wo e en s a e
casually ela ed, he pe cei ed ime o hese wo e en s shi s owa ds each o he .
3.3 In Theo e ical Linguis ics
Causa ion has been in es iga ed by linguis ics unde a a ie y o opics. In his
sec ion we will ocus on he wo k done on Typological Linguis ics, he Cogni i e
App oach and Discou se Theo y.
[S e anowi sch, 2001] o e s h ee p inciples o causa ion cons ual. These a e
concep ual s a egies used o cons ue causal links be ween e en s and pa ici-
pan s; hey a emp o iden i y wha in he human language hough leads o he
pe cep ion o causal ela ions:
1. The Tempo al Succession P inciple
Gi en wo e en s, Aand B, a concep ualize Cpe cei es Aas he cause o
Bi and only i
(a) A( egula ly) p ecedes B, and
(b) Aand Ba e empo ally con iguous, and/o
(c) Aand Ba e spa ially con iguous.
2. The Coun e ac uali y P inciple
Gi en wo e en s, Aand B, a concep ualize Cpe cei es Aas he cause o
Bi and only i
(a) Aoccu s and Boccu s, and
(b) (Cbelie es ha ) Bwould no ha e occu ed i Ahad no occu ed.
3. The T ansmission o Ene gy
Gi en wo e en s, Aand B, a concep ualize Cpe cei es Aas he cause o
Bi and only i
(a) Cknows ha Acan ansmi ene gy, and
(b) Cbelie es ha A ansmi ed ene gy o some pa o he eal wo ld
and he eby gene a ed B.
Two e en s a e conside ed o be causally ela ed i one o mo e o he h ee p in-
ciples apply.
[S e anowi sch, 2001] also o e s a se o a ibu es o causa ions e en s. These
a ibu es help speci ying a causa ion and classi y hem:
3 CAUSATIONS 16
•The cause is he p oduce o he cause. We can cha ac e ize i using he
ollowing se o p ope ies:
–Anima e
∗in en ional o non in en ional
∗ oli ional o non oli ional
∗ he deg ee o he cause ’s con ol
∗whe he o no he cause is in a posi ion o au ho i y o e he
causee
–Inanima e
∗objec o e en
No e ha in en ionali y en ails oli ionali y and con ol.
•The causee is he p oduce o he e ec . We can cha ac e ize i using he
ollowing se o p ope ies:
–Anima e
∗ he deg ee o con ol ha he causee has o e he esul .
∗whe he o no he causee esis he cause ; i he causee esis
he e migh be a ious mo i a ions o doing so.
–Inanima e
∗objec o e en
•The causing e en is he e en pe o med by he cause . We can cha ac-
e ize i using he ollowing p ope ies:
–Di ec i in ol es con ac be ween he cause and causee
–Indi ec i i doesn’ in ol e con ac .
•The esul ing e en is he e en p e o med by he causee. We can dis in-
guish be ween:
–immedia e o nonimmedia e, depending on he iming o he causing
and esul ing e en s. I hey occu simul aneously wi h o he esul ing
e en occu s igh a e he causing e en , i is immedia e; i a delay
exis s be ween bo h e en s i is nonimmedia e.
–s a ic o dynamic, depending on he na u e o he e en .
3 CAUSATIONS 17
3.3.1 In Typological Linguis ics
[Shiba ani, 1976] p oposed se e al seman ic pa ame e s o causa i e cons uc-
ions, including:
•Coe ci e o noncoe ci e, depending on he deg ee o o ce ha he cause
imposes on he causee. The cause may o ce,pe suade o gen ly sugges he
causee.
•Pe missi e o nonpe missi e. I pe missi e, we can subclassi y hem in
ou ypes:
–The cause o bea s (o omi s) p e en ion (o in e en ion).
–The cause ac i ely gi es pe mission o he causee o do some hing.
–The cause a emp s bu ails o p e en some hing om happening.
–The cause gi es up and does no in e ene wi h he caused e en .
•Di ec o indi ec , depending on he p e ious planing o he cause abou
b inging a change. I di ec , we can dis inguish be ween di ec i e and ma-
nipula i e. I he causee unc ions as a oli ional en i y, i is di ec i e, i he
oli ion o he causee is absen hen is manipula i e.
Acco ding o he au ho , his seman ic classi ica ion has been p o en ele an
o many languages, and hus seems o be uni e sally applicable.
3.3.2 In he Cogni i e App oach
Wi hin he cogni i e app oach one o he mos impo an heo ies is Fo ce Dy-
namics [Talmy, 2000]. Fo ce Dynamics is a seman ic ca ego y co e ing how
en i ies in e ac wi h espec o o ce. I s a ed ou as a gene aliza ion o e
he adi ional no ion o causa i e, analyzing causa ion in o ine p imi i es and
b inging he no ions o le ing,hinde ing, and helping in o he discussion. The
heo y de ines concep s such as exe ion o o ce, esis ance o such exe ion and
he o e coming o such esis ance,blockage o a o ce and he emo al o such
blockage, e c.
The basic concep ual p imi i es a e [Talmy, 2000]:
•Two en i ies exe a o ce each on he o he . One is o ego unded o singled
ou o ocal a en ion ( he agonis ); he o he is conside ed o he e ec i
has on he agonis ( he an agonis ).
•an en i y is aken o exe a o ce by i ue o an in insic endency owa ds
mani es ing i , owa ds mo ion (ac ion) o owa ds es (inac ion).
•opposed o ces ha e di e en ela i e s eng h, he en i y ha is able o
mani es i s endency a he expense o i s opponen is he s onge .

3 CAUSATIONS 18
•acco ding o hei ela i e s eng hs, he opposing o ces yield a esul an ,
assessed only o he agonis , as ac ion o inac ion.
A simple example may help unde s anding he basis o he heo y. “The
woman ell because he loo wan slippe y.” would be conside ed as a causa ion
by mos o he people. In he example, he agonis is he woman and he an ag-
onis he loo . Bo h en i ies exe a o ce ( he woman (suposedly) didn’ wan
o all; he loo , by being slippe y, makes people all); he esul is he woman
alling, so he an agonis wins and he esul is an ac ion.
The comple e heo y is much deepe and is conside ed one o he majo con-
ibu ions in he s udy o g amma and seman ics.
3.3.3 Wi hin Discou se Theo y
Discou se Theo y ies o de ine he seman ic ela ions ha hold be ween di e en
discou se uni s. Thus, casusa ion is h ea ed as ano he seman ic ela ion. Sec ion
2.1 desc ibes di e en se s o seman ic ela ions p oposed so a and sec ion 3.4
s a es he di e ences be ween a causa ion and i s closes seman ic ela ions.
3.4 Causa ion and o he seman ic ela ions
As we ha e al eady discussed, i ’s no always easy o decide wha seman ic e-
la ion holds be ween wo ex spans. The limi s be ween causa ions and o he
seman ic ela ions a e explained in his sec ion [Talmy, 2000], [Lako , 1987].
The closes seman ic ela ions o causa ion a e condi ion,consequence,
eason and in luence.
condi ion is a causa ion whose cause is hypo he ical (e.g. “I he we e hand-
some, he would be ma ied.”). consequence is a causa ion whose e ec is in-
di ec o unin ended (e.g. “His esigna ion caused eg e among all classes.”);
eason is a causa ion o decision, belie , eeling o ac ing (e.g. “I wen because
I hough i would be in e es ing.”). In his wo k we conside all o hem as cau-
sa ion.
The dis inc ion be ween in luence and causa ion is a ma e o deg ee. The e
is a con inuum be ween o ally independen e en s and o ally casually ela ed
e en s. An in luence holds be ween e en 1and e en 2i e en 1a ec s he
manne o in ensi y o e en 2, bu does no a ec he occu ence o e en 2. I
i a ec s he occu ence, hen i is a causa ion. Fo example, “Ta ge ing skin
cance ela i es imp o es sc eening.” encodes an in luence.
pu pose can be in e p e ed as an in ended causa ion, i.e., e en 1is he pu pose
o e en 2i e en 1implies e en 2and he cause is willing he e en 2 o exis . We
3 CAUSATIONS 19
don’ conside pu pose in his wo k.
As we ha e al eady men ioned, seman ic ela ions may o e lap, i.e., be ween
wo ex spans mo e han one seman ic ela ion may hold. A clea o e lap exis s
be ween causa ion and empo al ela ions. By de ini ion, he cause should
always occu be o e he e ec , so i e en 1causes e en 2, hen e en 1should occu
be o e han e en 2.
We should make clea he dis inc ion be ween wo e en s causally ela ed and
wo e en s co ela ed. The age o a human being and he size o his ocabula y
ipically a e co ela ed: he olde , he mo e ocabula y. Howe e , we all know
ha ge ing olde does no cause o ha e a bigge ocabula y.
3.5 Encoding o Causa ion
The basic a gumen s o a causa ion a e he cause and he e ec . F om he poin
o iew o de ec ing causa ions, he ollowing dis inc ions may be use ul:
•Ma ked o unma ked A causa ion is ma ked i he e is a speci ic linguis ic
uni ha signal he ela ion. Fo example:
–[ma ked]: I bough his book because I ead a good e iew.
–[unma ked]: Be ca e ul. I ’s uns able.
Unma ked causa ions a e mo e di icul o de ec and usually i is no easy
o decide whe he hey encode a causa ion o no .
•Ambigui y I he causa ion is ma ked, he ma k can signal always a causa-
ion o i can signal some imes a causa ion. The o me kind is unambiguous,
he la e ambiguous. Fo example:
–[unambiguous]: He died because o he ex en o his inju ies.
–[ambiguous signaling causa ion]: The s o m p oduced se e al small
o nadoes and gus nadoes.
–[ambiguous no signaling causa ion]: The ac o y p oduces wo kinds
o cable.
•Explici o implici A causa ion is explici i bo h a gumen s a e p esen .
I any o i s a gumen s a e missing, hen i is a implici causa ion. Fo
example:
–[explici ]: She was h own ou o he swanky Ho el Excelsio a e she
had un naked h ough i s ma ble halls.
3 CAUSATIONS 20
–[implici ]: John killed Bob.
Al hough i migh no be ob ious, he second example encodes a cau-
sa ion. The e ec a gumen , Bob’s dea h, is no explici .
In his wo k, we ocus in ma ked and explici causa ions.
4 PREVIOUS WORK 21
4 P e ious wo k
In his sec ion we p esen he wo k al eady done in Compu a ional Linguis ics on
he de ec ion and ex ac ion o seman ic ela ions and pa icula ly on causa ions.
4.1 On Seman ic ela ions
One o he i s p oposed sys ems o pe o m seman ic analysis was Tanka, [Ba ke
and Szpakowicz, 1995]. The sys em pe o ms seman ic analysis in h ee le els:
Clause-Le el Rela ionships Analysis (seman ic ela ionships be ween ac s, e en s
o s a es ep esen ed syn ac ically by synde ically connec ed ini e clauses), Case
Analysis (be ween he main e b and i s syn ac ic a gumen s) and Noun-Modi ie
Rela ionships Analysis (be ween he head noun o a noun ph ase and i s modi ie s).
The Clause-Le el Rela ionships hey conside ha e been de ailed in sec ion
2.1. The sys em is based on a lexicon manually buil and a se o ules ha allows
o decide he igh ela ionships o a pa icula pai o ex spans. The au ho s
e alua e he sys em wi h a legal ex ; he as majo i y o ela ionships sugges ed
a e co ec . The e alua ion is pe o med wi h 100 sen ences andomly selec ed,
94 sugges ions a e co ec and 2 w ong; he sys em canno make a decision o 4
sen ences and he e o e he sugges ion consis s o a se o possible ela ionships.
Seman ic ela ions in Noun Ph ases ha e been s udied in [Moldo an e al., 2004].
Using he se o 35 seman ic ela ions desc ibed in able 1 o sec ion 2, hey com-
pa e he pe o mance o h ee lea ning models: Nai e Bayes, Decision T ees and
a new model, Seman ic Sca e ing. The bes esul s a e ob ained wi h Seman ic
Sca e ing, he F-measu e a ies om 0.33 o 0.75 depending on he syn ac ic
pa e n. As ea u es, hey only used he seman ic classes o he head and modi ie
noun.
In [Gi ju e al., 2004] he au ho s s udy he seman ic ela ions in Nominalized
Noun Ph ases. They use a se o lexical, syn ac ic and seman ic ea u es. The
bes esul s a e ob ained wi h Suppo Vec o Machines, he F-measu e a ies
om 0.61 o 0.71 depending on he syn ac ic pa e n.
O he app oaches wo k wi h P obabili y Theo y and he lexico-syn ac ic pa -
e ns ha exp ess causa ions. These app oaches almos don’ need supe ision,
bu equi e a la ge co pus.
In [Tu ney, 2006] he au ho s de ine he pe inence o a pa e n o a wo d pai .
When classi ying noun-modi ie pai s, hey epo a F-measu e o 0.50. They only
conside i e seman ic ela ions: causali y,pa icipan ,quali y,spa ial
and empo ali y. Thei sys em is unsupe ised and he only equi emen s
a e a la ge co pus (a ound 5 ×1010 wo ds) and a se o wo ds pai s wi h hei
co esponding labels (a leas 600 pai s in hei expe imen s).
[Pan el and Pennacchio i, 2006] also wo k wi h p obabili ies and p opose wo
new measu es: pa e n eliabili y and ins ance eliabili y. The key idea is ha in
5 THE METHOD 28
Pa e n
Ma ching
Syn ac ic pa e ns
ha encode causa ion
Co pus (SemCo 2.1) POS agging
and
Chunke
Shallow
pa sed sen ences
Ma ches
No ma ches
Figu e 2: Flowcha o he pa e n ma ching.
and e bs he Wo dNe 2.1 sense numbe . Howe e , all he e b o ms ha e he
POS ag VB, which migh no be enough i we wan o ell he di e ence be ween
di e en e b enses. In o de o ge he igh POS ag o all he e b o ms, we
use a POS agge [Schmid, 1994].
While clus e ing he di e en examples in he syn ac ic pa e n (1), we disco e ed
ha he mos common ela o s exp essing causa ion a e as,a e ,because and
since. The e o e, we decided o ocus only in he causa ions ha :
•a e ins ances o pa e n (1), and
•a e signaled by he ela o as, since, a e o because.
We ound 1068 sen ences ha sa is y bo h condi ions. The dis ibu ion o he
ela o s is shown in able 5.
Table 5: Dis ibu ion o he ela o s.
Rela o Ca dinali y
because 381
as 330
a e 228
since 129
To al 1068
The ollowing is a eal example con ained in he co pus SemCo 2.1. The o ma
is he ollowing: i s column POS ag2, second lemma, hi d Wo dNe 2.1 sense
numbe and ou h wo d o m:
2A lis o ags and he pa s o speech co esponding o hem can be ound in appendix A

5 THE METHOD 29
POS ag lemma wnsn wo d o m
NN o dina y 1 O dina y
NNP pe son 1 Williams
VB say 1 said
PRP he -1 he
, , -1 ,
RB oo 2 oo
, , -1 ,
VB was -1 was
VB subjec 1 subjec ed
VB o -1 o
JJ anonymous 1 anonymous
NN call 1 calls
RB soon 1 soon
IN a e -1 a e
PRP he -1 he
VB schedule 1 scheduled
DT he -1 he
NN elec ion 1 elec ion
. . -1 .
A e he POS agging o a ach he igh POS ag o e bs and he chunking,
he esul is he ollowing:
POS ag lemma wnsn wo d o m chunke
NN o dina y 1 O dina y B-NP
NNP pe son 1 Williams I-NP
VBD say 1 said B-VP
PRP he -1 he B-NP
, , -1 , O
RB oo 2 oo B-ADVP
, , -1 , O
VBD be -1 was B-VP
VBN subjec 1 subjec ed I-VP
IN o -1 o B-PP
JJ anonymous 1 anonymous B-NP
NN call 1 calls I-NP
RB soon 1 soon B-ADVP
IN a e -1 a e B-SBAR
PRP he -1 he B-NP
VBN schedule 1 scheduled B-VP
DT he -1 he B-NP
NN elec ion 1 elec ion I-NP
. . -1 . O
No e ha he example shown ma ches he pa e n we conside : O dina y
Williams said he, oo, [was subjec ed]V P o anonymous calls soon [a e ] el [he
[scheduled]V PC he elec ion]C. Using he o mula (2), he a gumen o de is i s
e ec and hen cause, a gumen sO de =EC ∗middle =EC, so he e ec is
encoded in V P and he cause in V PC.
No e ha he ou ela o s chosen no always signal a causa ion:
5 THE METHOD 30
•Rachel s ayed on a e he doc o had gone.
•He has li ed in many houses since he mo ed o he S a es.
•She was ying o con ain he bi e ness o he oice as she enuncia ed he
wo ds oo dis inc ly.
5.3 Machine Lea ning app oach
The es o he asks in o de o c ea e a model o he de ec ion o causa ions a e
ep esen ed in igu e 3.
Ma ches Manual Clasi ica ion (causa ion / no causa ion)
+
ea u e selec ion
Fea u e ex ac ion
ML algo i hm
E alua ion
T aining da a
E alua ion da a
Model
Classi ied da a
Pe o mance
Figu e 3: Flowcha o he lea ning p ocess and e alua ion.
So a we ha e a se o sen ences ha a e likely o exp ess causa ion, since
hei syn ac ic s uc u e is likely o encode one. Ou goal is o classi y hem in
sen ences exp essing causa ion and sen ences no exp essing causa ion. In o de
o do so, we decided o use Machine Lea ning echniques. The p oblem i s easily
in he pa adigm o classi ica ion.
F om now on, we will call ins ance he ma ches ob ained a e he syn ac ic pa -
e n ma ching. Thus, an ins ance should be classi ied as cause i i encodes a
causa ion, o ¬cause, i i doesn’ .
Le us de ine Xas he se o all he ins ances and he se C={cause, ¬cause}.
The unc ion de ined in (3) associa es each ins ance wi h i s igh classi ica ion.
Ob iously is unknown a p io i.
:X−→ C(3)
5 THE METHOD 31
The challenge o he Machine Lea ning pa adigm is o lea n he unc ion
based on some examples. In o de o do so, he lea ning algo i hm akes as inpu
o each example a ea u e ec o and i s igh classi ica ion. The algo i hm is
able o c ea e a model ha is an app oxima ion o he unc ion .
Once he model is lea ned, we need o es ima e i s pe o mance. In o de o
do so, a new se o examples is needed. The pe o mance is measu ed conside ing
he di e ences be ween he classi ica ion acco ding o he model ob ained and
he igh classi ica ion. Typically he pe o mance is no pe ec ; i a ies a lo
depending on he ask.
Fo mally, o each ins ance xi∈X, we de ine he ec o Fi= (xi1, xi2. . . xim),
co esponding o he ea u es o he ins ance xi.xij deno es he ea u e j o he
ins ance i. We deno e he se o examples used o aining T and he se used
o es ing T e.T and T e a e o he o m {F1c1, F2c2. . . Fmcm};ci∈C.
Since we ha e a ini e numbe o ins ances, we need o di ide hem in he T
and T e se s a he e y beginning.
5.3.1 Da a p epa a ion
The da a p epa a ion basically consis ed on classi ying each ins ance as causa ion
o no causa ion. Ou o he 1068 ins ances, 517 we e classi ied as cause and 551
as ¬cause3.
This ask is e y slow and comple ely manual. A manual anno a o has o exam-
ine each ins ance and decide i i encodes a causa ion o no . Fu he mo e, he
ask is no as easy as i may seem. Some imes i is no clea whe he a sen ence
encodes a causa ion o no ; ambigui y occu s mo e o en han expec ed.
Conside he example “A e ou yea s o ha d wo k, she g adua ed”. One could
a gue ha he cause o he g adua ion is he ac ha she wo ked ha d o ou
yea s. Howe e , aking in o accoun only he in o ma ion con ained in he sen-
ence we canno conclude ha a causa ion holds.
5.3.2 Fea u e selec ion
A he same ime han he manual classi ica ion, a ea u e selec ion was pe o med.
I is no ob ious wha kind o ea u es a e he good ones o ou p oblem. Typical
ea u es o Machine Lea ning app oaches in he Na u al Language P ocessing
ield a e a mix u e o lexical and syn ac ic; la ely some esea che s ha e achie ed
g ea esul s conside ing seman ic ea u es oo. Following his app oach, we look
o clues in he h ee le el o analysis ha de e mine good ea u es.
3This means he baseline o he classi ica ion ask is 0.516, since 51.6% o he ins ances
belong o ¬cause
5 THE METHOD 32
I may be a gued ha he e o needed o come up wi h a good se o ea u es
is oo much. I is de ini ely a lo o wo k o do, specially conside ing ha i is
manual. Howe e , he ca e ul examina ion o he examples allows us o eally
unde s and he na u e o causa ions.
The se o ea u es ha we e de ec ed as po en ially good ea u es, hei a ionale,
alues, de ec ion and examples a e de ailed below:
1. Rela o
•Ra ionale: A ela o can signal always, ne e o some imes a causa-
ion. Fu he mo e, di e en combina ion o ela o s and o he ea u es
can signal causa ion always, ne e o some imes.
•Values: because, as, a e , since.
•De ec ion: a he same ime han he pa e n ma ching
•Examples:
–Rela o s unambiguously causal: because
∗[cause]Pa - ime a me s gene ally mus pay highe p ices o
supplies han ull- ime a me s because hey buy in smalle
quan i ies.
∗[cause]Leade ship is lacking in ou socie y because i has no
legi ima e place o de elop.
–Rela o s ambiguous: a e ,since,as
∗[¬cause]F ank and he had me abou wo yea s a e she had
a i ed.
∗[cause]Ma y s ood o se e al momen s wi h his mou h hang-
ing open oolishly a e i had happened.
∗[¬cause]They had been a lessons in he schoolhouse since
hey e u ned om Ha pe s Fe y.
∗[cause]The child en a e sa is ied wi h smalle amoun s o ood
since all o i is high in quali y.
2. Rela o Modi ie s
•Ra ionale: causa ions can ha dly be signaled by a ela o modi ied
by an ad e b o p eposi ion.
•Values: POS ags.
•De ec ion: a he same ime han he pa e n ma ching.
•Examples:
–ad e b + a e almos always signals a empo al ela ion, no a
causa ion:
5 THE METHOD 33
∗Tom B annon had caugh up wi h he ou i sho ly a e
he Magui es joined i .
∗This was long a e Mo se had le he house.
–as + p eposi ion can ha dly signal a causa ion:
∗. . . he el he was no ing i , as i i we e some hing he migh
hink abou when . . .
∗Alex nodded o he maid as hough no hing unusual we e
aking place and en e ed he doc o ’s oom.
3. Seman ic Class Cause Ve b
•Ra ionale: only ce ain e bs can exp ess a cause.
•Values: Wo dNe 2.1 seman ic class4.
•De ec ion: a he same ime han he pa e n ma ching.
•Examples:
–I he ela o is a e and he cause e b seman ic class is be- -3
56, hen i is a empo al ela ion, no a causa ion:
∗We hea d him be o e he e e showed, and we hea d him yelling
a e he was ou o sigh .
4. Cause Ve b is Po en ially Causal
•Ra ionale: i a e b sense’s gloss con ains he wo ds cause o o
change, o is subsumed by a e b sense ha con ains he wo ds cause
o o change, hen is mo e likely o exp ess a cause 7.
•Values: yes, no.
•De ec ion: examine gloss cause e bs and i s subsume s
•Examples:
– ing- -1 is subsumed by sound- -2, which gloss is “cause o sound”.
5. Seman ic Class E ec Ve b
•Ra ionale: only ce ain e bs can exp ess an e ec .
•Values: Wo dNe 2.1 seman ic class.
•De ec ion: a he same ime han he pa e n ma ching.
4By Seman ic Class we mean he mos common subsume o he e b in Wo dNe 2.1. Fo
example, he seman ic class o umina e- -1 is ea - -2
5be- -3 should be eaded as he hi d meaning o he e b be
6The gloss o be- -3 is “occupy a ce ain posi ion o a ea; be somewhe e; ‘Whe e is my
umb ella?’ ‘The oolshed is in he back’; ‘Wha is behind his beha io ?’ ”
7Acco ding wi h he de ini ion, we ound 7,370 ou o 24,890 e b senses po encially causal

5 THE METHOD 34
•Examples:
– eac - -1’s seman ic class is ac - -1,allow- -10’s seman ic class is
ac - -1 and ma ginalize- -1 ’s seman ic class is ac - -1.
–I he ela o is a e and he e ec e b seman ic class is exp ess-
-28, hen is no a causa ion:
∗“My name’s Gisele”, he blonde said a e she o de ed a Sco ch.
6. E ec Ve b is Po en ially Causal
•Ra ionale: i a e b sense’s gloss con ains he wo ds cause o o
change, o is subsumed by a e b sense ha con ains he wo ds cause
o o change, hen is mo e likely o exp ess an e ec .
•Values: yes, no.
•De ec ion: examine gloss cause e bs and i s subsume s
•Examples:
–walk- -3 is subsumed by a el- -1, which gloss is “change loca-
ion’.
7. Ve b Tense Cause and E ec Ve b
•Ra ionale: depending on he ela o , some e b enses a e no likely
o exp ess causa ion
•Values: p esen , pas , u u e, pe ec i e, p og essi e, condi ional, obli-
ga ion, possibili y, . . .
•De ec ion: examine he POS ags o he cause and e ec e bs
•Examples:
–I he ela o is as o a e , he cause e b is no a copula (seman ic
class di e en han be- -1 9), and he cause e b ense is p esen ,
hen is no a causa ion:
∗[¬cause]Hen ie a was disco e ing in he p ocess o w i ing,
as he bo n w i e does, no me ely . . .
∗[¬cause]To play he gui a as he aspi es will de ou his . . .
∗[¬cause]The Thaye Schools o e s a yea o pos g adua e
s udy in somewha he same way, a e a boy wins a B.S.
in enginee ing.
∗[cause]. . . you mus o gi e me as I am so o ge ul.
8The gloss o exp ess- -2 is “a icula e; ei he e bally o wi h a c y, shou , o noise; ‘She
exp essed he ange ’; ‘He u e ed a cu se’ ”
9The gloss o be- -1 is “ha e he quali y o being; (copula, used wi h an adjec i e o a
p edica e noun); ‘John is ich’; ‘This is no a good answe ’ ”
5 THE METHOD 35
–I he ela o is as and he e ec e b is condi ional, hen is no a
causa ion:
∗She wouldn’ go o New Yo k as Maude sugges ed . . .
–I he e ec e b is p og essi e, hen is no a causa ion:
∗The bu den o his sec e was p essing down on him, as i
was on Lieu enan Becks om and his six enlis ed men.
∗. . . said Juani a, holding he ace e y s ill, ying o con-
ain he bi e ness o he oice as enuncia ed he wo ds oo
dis inc ly.
–i he ela o is as and he e ec e b exp ess obliga ion, hen is a
causa ion:
∗You mus do ha as I helped you.
–i he e ec e b es passi e, hen i is mo e likely o exp ess a
causa ion:
∗. . . and hen Richa d was shocked as, all a once, lames sho
ou om he sha p ea u es o . . .
8. Cause and E ec Ve b Same Tense
•Ra ionale: i bo h he cause and e ec e b a e pas simple, i can
ha dly be a causa ion, mos o he ime i is jus a empo al ela ion.
•Values: yes, no.
•De ec ion: cause and e ec e b same ense
•Examples:
–... ailed o lou ish in New England as i did in o he pa s o
he coun y. Fle ched nodded as he lis ened o he ins uc ions
and said he would . . .
5.3.3 Fea u e ex ac ion
The ea u e ex ac ion is done ex ac ing he in o ma ion de ailed below om he
ela o , he wo ds su ounding i , and bo h VPs. No e ha wi h he ou ela o s
chosen, VP exp ess allways he e ec and he VP con ained in C (VPC) he cause.
The inal se o ea u es a e he ollowing:
• ela o = {since, because, as, a e }
• ela o Le Modi ica ion = {POS ags}
• ela o Righ Modi ica ion = {POS ags}
•Seman icClassVCause = {Wo dNe 2.1 sense numbe }
5 THE METHOD 36
• e bCauseIsPo en iallyCausal = {yes, no}
•Seman icClassVE ec = {Wo dNe 2.1 sense numbe }
• e bE ec IsPo en iallyCausal = {yes, no}
•Fo bo h Ve b Ph ases, he ollowing ea u es a e ex ac ed:
–P esen = {yes, no}
–Pas = {yes, no}
–Modal = {condi ional, obliga ion, possibili y, u u e, no}
–Pe ec i e = {yes, no}
–P og essi e = {yes, no}
–Passi e = {yes, no}
We exempli y he ea u e ex ac ion wi h he sen ence p esen ed in sec ion
5.2, O dina y Williams said he, oo, [was subjec ed]V P o anonymous calls soon
[a e ] el [he [scheduled]V PC he elec ion]C:
ela o a e cP esen no eP esen no
ela o Le Modi ica ion RB cPas yes ePas yes
ela o Righ Modi ica ion PRP cModal no eModal no
seman icClassVCause 1 cPe ec i e no ePe ec i e no
e bCauseIsPo en iallyCausal yes cP og essi e no eP og essi e no
seman icClassVE ec 1 cPassi e no ePassi e yes
e bE ec IsPo en iallyCausal no
In he example, ‘cP esen ’ means “cause e b ense is p esen ”, ‘eP esen ’
means “e ec e b ense is p esen ”, and so on.
The ex ac ion o he non ob ious ea u es is depic ed below:
•We can easily ge he main e b o a VP: i always co esponds o he las
elemen o he VP [Qui k e al., 1985]. Since we ha e al eady iden i ied he
VP encoding cause and e ec du ing he pa e n ma ching, we can easily ge
hei seman ic classes looking o he in o ma ion con ained in he co pus.
•Once we know he seman ic class, we only need o examine he gloss o know
i any o he e bs is po en ially causal.
•The boolean ea u es ha iden i y he ense o he VP a e de ined as ollows:
–P esen : yes i he POS ags VBP o VPZ a e p esen in he VP; no
o he wise.
–Pas : yes i he POS ag VBD is p esen in he VP; no o he wise.
5 THE METHOD 37
–Modal:
∗condi ional i he lemma would is p esen in he VP,
∗obliga ion i he lemmas mus ,ough o should a e p esen in he
VP,
∗possibili y i he lemmas can,could,may o migh a e p esen in
he VP,
∗ u u e i he lemmas will o shall a e p esen in he VP,
∗no o he wise.
–Pe ec i e: yes i he cons uc ion ha e + VBN (lemma ha e ollowed
by pa iciple) is p esen ; no o he wise.
–P og essi e: yes i he POS ag VBG is p esen in he VP; no o he wise.
–Passi e: yes i he cons uc ion be + VBN (lemma be ollowed by
pa iciple) is p esen ; no o he wise.
No e ha di e en combina ion o hese boolean ea u es can cap u e enses
like ‘pas and passi e’, e.g. was subjec ed, o ‘obliga ion, p esen and pe ec i e’,
e.g. should ha e gone.
5.3.4 Machine Lea ning algo i hm
We ied di e en Machine Lea ning algo i hms using 10- old c oss- alida ion.
The bes esul s ob ained a e shown in able 6. They we e ob ained using Bag-
ging wi h C4.5 ees.
Table 6: Resul s ob ained using Bagging wi h C4.5 ees du ing aining.
Class P ecision Recall F-Measu e
cause 0.969 0.839 0.899
¬cause 0.865 0.975 0.917
The measu es o e alua ing he pe o mance o he Machine Lea ning al-
go i hm a e he mos commonly used. P ecision measu es he po ion o he
assigned ca ego ies ha we e co ec ; Recall he po ion o he co ec ca ego ies
ha we e assigned; F-measu e is he weigh ed ha monic mean o P ecision and
Recall. The h ee measu es ange om 0 o 1. Fo mally, we can de ine he h ee
measu es as ollows:
Co ec = X Co ec = Y
Assigned = X a b
Assigned = Y c d
7 CONCLUSIONS AND FURTHER WORK 44
7 Conclusions and u he wo k
We ha e p oposed a sys em which yields a high P ecision and Recall o he
de ec ion o causa ions encoded by one o he mos common syn ac ic pa e ns
exp essing a causa ion: [VP el C], [ el C, VP]. The sys em is ela i ely simple
and is able o de ec causa ions encoded in an open domain ex .
So a esea ch in causal ela ion de ec ion has ocused in causa ion exp essed
wi h noun ph ases, e.g. “The [inciden ]NP1p o oked [widesp ead p o es ]NP2’. We
ocused on he de ec ion o causa ion be ween a e b and a subo dina e clause.
[Ba ke and Szpakowicz, 1995] also wo k wi h clauses, bu hei app oach is e y
di e en .
The sys em p oposed uses shallow syn ac ic pa sing. S a e o he a chunke s
yield pe o mance o 95 %, so we can assume we de ec mos o he pa e n in-
s ances p esen in he inpu . Howe e , chunke s ake as inpu POS agged ex ,
and POS agge s also make mis akes.
A key elemen o eally see he po en ial o he sys em would be o in eg a e
i wi h o he sys ems ha ex ac seman ic ela ions. By doing so we could
expe imen wi h in e ence ules ha combine causa ion and o he seman ic e-
la ions. Fo example, i e en 1causes e en 2and e en 3is subsumed by e en 1,
we can conclude han e en 3causes e en 2; i e en 1causes e en 2and e en 2
en ails e en 3, hen e en 1causes e en 3. Ano he possible in e ence ule would
exp ess he ansi i y p ope y o causa ions. An example may cla i y he las
wo examples:
•Being i ed makes me sno e. We all know ha in o de o sno e you need
o be asleep.
causa ion(being i ed, sno e)∧en ail(sno ing, sleeping)⇒causa ion(being
i ed, sleep)
•He died o cance . Cance is caused by smoking.
causa ion(cance , dea h)∧causa ion(smoking, cance )⇒causa ion(smoking,
dea h)
No e ha he in e ence ule allows us o ex ac mo e causal knowledge, bu
he causa ions ex ac ed a e indi ec . The disco e ing o new ules and he ali-
da ion o he new ules seem o be a g ea challenge.
Ano he possible ex ension would be o deal wi h causal chains and in ica e
causal ela ions. A causal chain can be de ined as a sequence o e en s ha lead
up o some inal e ec . Examples (1) and (2) exempli y a causal chain and a
in ica e causal ela ion espec i ely.

7 CONCLUSIONS AND FURTHER WORK 45
1. A wo ks become a only when hey anscend he simple ac s o hei
exis ence, and hey can do ha only when hey blend wi h he sensibili y
o he iewe .
( hey (a wo ks) blend wi h he sensibili y o he iewe )⇒( hey (a wo ks)
anscend he simple ac s o hei exis ence)⇒(a wo ks become a )10.
2. I is lined p ima ily by indus ial de elopmen s and conc e e-block walls
because he cons an a ic and emissions do no make i an a ac i e
neighbo hood.
((cons an a ic and emissions)⇒(no an a ac i e neighbo hood)) ⇒
(lined up by indus ial de elopmen s and conc e e-block walls).
10‘x ⇒y’ should be eaded as “x causes y”
REFERENCES 46
Re e ences
[Bake e al., 1998] Bake , C. F., Fillmo e, C. J., and Lowe, J. B. (1998). The
Be keley F ameNe p ojec . In P oceedings o he Thi y-Six h Annual Mee ing
o he Associa ion o Compu a ional Linguis ics and Se en een h In e na ional
Con e ence on Compu a ional Linguis ics, pages 86–90, San F ancisco, CA,
USA. Mo gan Kau mann Publishe s.
[Ba ke and Szpakowicz, 1995] Ba ke , K. and Szpakowicz, S. (1995). In e ac i e
seman ic analysis o clause-le el ela ionships. In P oceedings o he Second Con-
e ence o he Paci ic Associa ion o Compu a ional Linguis ics (PACLING–
95), pages 22–30, B isbane, Aus alia.
[Blahe a and Cha niak, 2000] Blahe a, D. and Cha niak, E. (2000). Asigning
unc ion ags o pa sed ex . In P oceedings o he 1s Annual Mee ing o
he No h Ame ican Chap e o Associa ion o Compu a ional Linguis ics
(NAACL), Sea le, WA, USA.
[Chang and Choi, 2006] Chang, D.-S. and Choi, K.-S. (2006). Inc emen al cue
ph ase lea ning and boo s apping me hod o causali y ex ac ion using cue
ph ase and wo d pai p obabili ies. In o ma ion P ocessing and Managemen :
an In e na ional Jou nal, 42(3):662–678.
[Eagleman and Holcombe, 2002] Eagleman, D. and Holcombe, A. (2002). Causal-
i y and he pe cep ion o ime. T ends in Cogni i e Sciences, 6(8):323–325.
[Ga cia, 1997] Ga cia, D. (1997). Coa is, an nlp sys em o loca e exp essions o
ac ions connec ed by causali y links. In EKAW ’97: P oceedings o he 10 h Eu-
opean Wo kshop on Knowledge Acquisi ion, Modeling and Managemen , pages
347–352, London, UK. Sp inge -Ve lag.
[Gildea and Ju a sky, 2002] Gildea, D. and Ju a sky, D. (2002). Au oma ic label-
ing o seman ic oles. Compu a ional Linguis ics, 28(3):245–288.
[Gi ju, 2003] Gi ju, R. (2003). Au oma ic de ec ion o causal ela ions o ques-
ion answe ing. In P oceedings o he 41s ACL, Wo kshop on Mul ilingual
Summa iza ion and Ques ion Answe ing.
[Gi ju e al., 2003] Gi ju, R., Badulescu, A., and Moldo an, D. (2003). Lea n-
ing seman ic cons ain s o he au oma ic disco e y o pa -whole ela ions.
In NAACL ’03: P oceedings o he 2003 Con e ence o he No h Ame ican
Chap e o he Associa ion o Compu a ional Linguis ics on Human Language
Technology, pages 1–8, Mo is own, NJ, USA.
[Gi ju e al., 2006] Gi ju, R., Badulescu, A., and Moldo an, D. (2006). Au oma ic
disco e y o pa -whole ela ions. Compu a ional Linguis ics, 32(1):83–135.
REFERENCES 47
[Gi ju e al., 2004] Gi ju, R., Giuglea, A.-M., Ol eanu, M., Fo u, O., Bolohan,
O., and Moldo an, D. (2004). Suppo ec o machines applied o he classi i-
ca ion o seman ic ela ions in nominalized noun ph ases. In P oceedings o he
HLT-NAACL Wo kshop on Compu a ional Lexical Seman ics.
[Gi ju and Moldo an, 2002] Gi ju, R. and Moldo an, D. (2002). Mining answe s
o causa ion ques ions. In P oceedings o he Ame ican Associa ion o A i i-
cial In elligence (AAAI) - Sp ing Symposium.
[Glymou , 2003] Glymou , C. (2003). Lea ning, p edic ion and causal bayes ne s.
T ends in Cogni i e Sciences, (7):43–48.
[Hi chcock, 2007] Hi chcock, C. (Fall 2007). P obabilis ic causa ion. In Zal a,
E. N., edi o , The S an o d Encyclopedia o Philosophy. The Me aphysics Re-
sea ch Lab. Cen e o he S udy o Language and In o ma ion. S an o d Uni-
e si y.
[Hobbs, 2005] Hobbs, J. R. (2005). Towa d a use ul concep o causali y o lexical
seman ics. Jou nal o Seman ics, 22(2):181–209.
[Hume, 1739] Hume, D. (1978, i s published anonymously in 1739). A ea ise
o human na u e. Ox o d Uni e si y P ess.
[Joskowiscz e al., 1989] Joskowiscz, L., Ksiezyk, T., and G ishman, R. (1989).
Deep domain models o discou se analysis. In The annual AI sys ems in go -
e nmen con e ence, pages 195–200.
[Kaplan and Be y-Rogghe, 1991] Kaplan, R. M. and Be y-Rogghe, G. (1991).
Knowledge-based acquisi ion o causal ela ionships in ex . Knowl. Acquis.,
3(3):317–337.
[Khoo e al., 2000] Khoo, C. S. G., Chan, S., and Niu, Y. (2000). Ex ac ing
causal knowledge om a medical da abase using g aphical pa e ns. In ACL
’00: P oceedings o he 38 h Annual Mee ing on Associa ion o Compu a ional
Linguis ics, pages 336–343, Mo is own, NJ, USA. Associa ion o Compu a-
ional Linguis ics.
[Kingsbu y e al., 2002] Kingsbu y, P., Palme , M., and Ma cus, M. (2002).
Adding seman ic anno a ion o he penn eebank. In P oceedings o he Human
Language Technology Con e ence (HLT’02).
[Lako , 1987] Lako , G. (1987). Women, Fi e, and Dange ous Things. Uni e si y
o Chicago P ess.
[Lewis, 1973a] Lewis, D. (1973a). Causa ion. Jou nal o Philosophy, 70:556–567.
[Lewis, 1973b] Lewis, D. (1973b). Coun e ac uals. Ox o d: Blackwell.
REFERENCES 48
[Mackie, 1980] Mackie, J. L. (1980). The cemen o he Uni e se. A S udy o
Causa ion. Ox o d Uni e si y P ess.
[Ma cu and Echihabi, 2002] Ma cu, D. and Echihabi, A. (2002). An unsupe ised
app oach o ecognizing discou se ela ions. P oceedings o he 40 h Annual
Mee ing o he Associa ion o Compu a ional Linguis ics (ACL’02), pages 368–
375.
[Mill, 1843] Mill, J. (1967, o iginal wo k published 1843). A sys em o logic a i-
ocina i e and induc i e. Longman.
[Moldo an e al., 2004] Moldo an, D., Badulescu, A., Ta u, M., An ohe, D., and
Gi ju, R. (2004). Models o he seman ic classi ica ion o noun ph ases. P o-
ceedings o he Compu a ional Lexical Seman ics Wo kshop a HLT-NAACL.
[Mu ali, 1997] Mu ali, R. (1997). A coun e ac ual analysis o causa ion. Mind,
(106):263–277.
[Pan el and Pennacchio i, 2006] Pan el, P. and Pennacchio i, M. (2006).
Esp esso: le e aging gene ic pa e ns o au oma ically ha es ing seman ic e-
la ions. In ACL ’06: P oceedings o he 21s In e na ional Con e ence on Com-
pu a ional Linguis ics and he 44 h annual mee ing o he ACL, pages 113–120,
Mo is own, NJ, USA.
[Quinlan, 1993] Quinlan, J. (1993). C4.5: P og ams o Machine Lea ning. Mo -
gan Kau mann.
[Quinlan, 1996] Quinlan, J. (1996). Bagging, boos ing and c4.5. In AAAI/IAAI,
Vol. 1, pages 725–730.
[Qui k e al., 1985] Qui k, R., G eenbaum, S., Leech, G., and S a ick, J. (1985).
A Comp ehensi e G amma o he English Language. Longman.
[Schmid, 1994] Schmid, H. (1994). P obabilis ic pa -o -speech agging using deci-
sion ees. In In e na ional Con e ence on New Me hods in Language P ocessing,
Manches e , UK.
[Shiba ani, 1976] Shiba ani, M. (1976). The G amma o Causa i e Cons uc-
ions: A Conspec us. Academic P ess.
[Sosa, 1975] Sosa, E. (1975). Causa ions and condi ionals. Ox o d Uni e si y
P ess.
[S e anowi sch, 2001] S e anowi sch, A. (2001). Cons uc ion Causa ion: A
Couns uc ional G amma App oach o Analy ic Causa i es. PhD hesis, Rice
Uni e si y.
REFERENCES 49
[Suppes, 1970] Suppes, P. (1970). A p obabilis ic heo y o causali y. No h-
Holland Publishing Co.
[Talmy, 2000] Talmy, L. (2000). Towa d a Cogni i e Seman ics. MIT P ess.
[Ta u, 2005] Ta u, M. (2005). Au oma ic disco e y o in en ions in ex and i s
applica ion o ques ion answe ing. In P oceedings o he ACL S uden Resea ch
Wo kshop, Ann A bon, MI, USA.
[Tu ney, 2006] Tu ney, P. D. (2006). Exp essing implici seman ic ela ions wi h-
ou supe ision. In ACL ’06: P oceedings o he 21s In e na ional Con e ence
on Compu a ional Linguis ics and he 44 h annual mee ing o he ACL, pages
313–320, Mo is own, NJ, USA.
[Whi e, 1990] Whi e, P. (1990). Ideas abou causa ion in philosophy and psychol-
ogy. Psychological Bulle in, 108(1):3–18.
[Wi en and F ank, 2005] Wi en, I. and F ank, E. (2005). Da a Mining: P ac i-
cal machine lea ning ools and echniques. Mo gan Kau mann.

A LIST OF POS TAGS 50
A Lis o POS ags
Tag Pa o speech
CC Coo dina ing conjunc ion
CD Ca dinal numbe
DT De e mine
EX Exis en ial he e
FW Fo eign wo d
IN P eposi ion o subo dina ing conjunc ion
JJ Adjec i e
JJR Adjec i e, compa a i e
JJS Adjec i e, supe la i e
LS Lis i em ma ke
MD Modal
NN Noun, singula o mass
NNS Noun, plu al
NNP P ope noun, singula
NNPS P ope noun, plu al
PDT P ede e mine
POS Possessi e ending
PRP Pe sonal p onoun
PRP$ Possessi e p onoun
RB Ad e b
RBR Ad e b, compa a i e
RBS Ad e b, supe la i e
RP Pa icle
SYM Symbol
TO o
UH In e jec ion
VB Ve b, base o m
VBD Ve b, pas ense
VBG Ve b, ge und o p esen pa iciple
VBN Ve b, pas pa iciple
VBP Ve b, non-3 d pe son singula p esen
VBZ Ve b, 3 d pe son singula p esen
WDT Wh-de e mine
WP Wh-p onoun
WP$ Possessi e wh-p onoun
WRB Wh-ad e b
B DECISION TREES LEARNED 51
B Decision ees lea ned
B.1 Decision T ee no. 1:
ela o = a e : n
ela o = as: n
ela o = since
| ela o Le Modi ica ion = yJJ: y
| ela o Le Modi ica ion = yIN: y
| ela o Le Modi ica ion = yJJR: y
| ela o Le Modi ica ion = yVBZ: y
| ela o Le Modi ica ion = yDT: y
| ela o Le Modi ica ion = yTO: y
| ela o Le Modi ica ion = yVBG: y
| ela o Le Modi ica ion = yRB: n
| ela o Le Modi ica ion = yVBP: n
| ela o Le Modi ica ion = yVBD: y
| ela o Le Modi ica ion = yPRP: y
| ela o Le Modi ica ion = yCD: y
| ela o Le Modi ica ion = yVBN
| | ePe ec i e = yes: n
| | ePe ec i e = no
| | | lexicalClue = yes: n
| | | lexicalClue = no: y
| ela o Le Modi ica ion = yCC: n
| ela o Le Modi ica ion = yVB: y
| ela o Le Modi ica ion = yNNP: y
| ela o Le Modi ica ion = yNNS: n
| ela o Le Modi ica ion = yNN
| | ela o Righ Modi ica ion = yPP: n
| | ela o Righ Modi ica ion = yJJ: y
| | ela o Righ Modi ica ion = y*: n
| | ela o Righ Modi ica ion = yIN: y
| | ela o Righ Modi ica ion = yVBZ: n
| | ela o Righ Modi ica ion = yDT
| | | e bCausePo Causal = yes: y
| | | e bCausePo Causal = no: n
| | ela o Righ Modi ica ion = yTO: n
| | ela o Righ Modi ica ion = yVBG: n
| | ela o Righ Modi ica ion = yRB: y
| | ela o Righ Modi ica ion = yWDT: n
| | ela o Righ Modi ica ion = yVBP: n
| | ela o Righ Modi ica ion = yVBD: n
| | ela o Righ Modi ica ion = yPRP: n
| | ela o Righ Modi ica ion = yVBN: n
| | ela o Righ Modi ica ion = yCD: n
| | ela o Righ Modi ica ion = yWRB: n
| | ela o Righ Modi ica ion = yVB: n
| | ela o Righ Modi ica ion = yNNP: n
| | ela o Righ Modi ica ion = yNNS: n
| | ela o Righ Modi ica ion = yEX: n
| | ela o Righ Modi ica ion = yMD: n
| | ela o Righ Modi ica ion = yPDT: n
| | ela o Righ Modi ica ion = yNN: n
| | ela o Righ Modi ica ion = yWP: n
| | ela o Righ Modi ica ion = yPUNCT: n
| ela o Le Modi ica ion = yPUNCT: y
ela o = because: y
Numbe o Lea es : 49
Size o he ee : 55
B.2 Decision T ee no. 2:
ela o = a e : n
B DECISION TREES LEARNED 52
ela o = as: n
ela o = since
| ePe ec i e = yes: n
| ePe ec i e = no
| | ela o Le Modi ica ion = yJJ: y
| | ela o Le Modi ica ion = yIN: y
| | ela o Le Modi ica ion = yJJR: y
| | ela o Le Modi ica ion = yVBZ: y
| | ela o Le Modi ica ion = yDT: y
| | ela o Le Modi ica ion = yTO: y
| | ela o Le Modi ica ion = yVBG: y
| | ela o Le Modi ica ion = yRB: y
| | ela o Le Modi ica ion = yVBP: n
| | ela o Le Modi ica ion = yVBD: y
| | ela o Le Modi ica ion = yPRP: y
| | ela o Le Modi ica ion = yCD: y
| | ela o Le Modi ica ion = yVBN: y
| | ela o Le Modi ica ion = yCC: n
| | ela o Le Modi ica ion = yVB: y
| | ela o Le Modi ica ion = yNNP: y
| | ela o Le Modi ica ion = yNNS: n
| | ela o Le Modi ica ion = yNN
| | | cPas = yes: n
| | | cPas = no: y
| | ela o Le Modi ica ion = yPUNCT: y
ela o = because: y
Numbe o Lea es : 24
Size o he ee : 28
B.3 Decision T ee no. 3:
ela o = a e : n
ela o = as: n
ela o = since
| ePe ec i e = yes: n
| ePe ec i e = no
| | ela o Le Modi ica ion = yJJ: y
| | ela o Le Modi ica ion = yIN: n
| | ela o Le Modi ica ion = yJJR: y
| | ela o Le Modi ica ion = yVBZ: y
| | ela o Le Modi ica ion = yDT: y
| | ela o Le Modi ica ion = yTO: y
| | ela o Le Modi ica ion = yVBG: y
| | ela o Le Modi ica ion = yRB: n
| | ela o Le Modi ica ion = yVBP: n
| | ela o Le Modi ica ion = yVBD: y
| | ela o Le Modi ica ion = yPRP: y
| | ela o Le Modi ica ion = yCD: y
| | ela o Le Modi ica ion = yVBN
| | | ela o Righ Modi ica ion = yPP: y
| | | ela o Righ Modi ica ion = yJJ: y
| | | ela o Righ Modi ica ion = y*: y
| | | ela o Righ Modi ica ion = yIN: y
| | | ela o Righ Modi ica ion = yVBZ: y
| | | ela o Righ Modi ica ion = yDT: y
| | | ela o Righ Modi ica ion = yTO: y
| | | ela o Righ Modi ica ion = yVBG: y
| | | ela o Righ Modi ica ion = yRB: y
| | | ela o Righ Modi ica ion = yWDT: y
| | | ela o Righ Modi ica ion = yVBP: y
| | | ela o Righ Modi ica ion = yVBD: y
| | | ela o Righ Modi ica ion = yPRP: y
| | | ela o Righ Modi ica ion = yVBN: y
| | | ela o Righ Modi ica ion = yCD: y
B DECISION TREES LEARNED 53
| | | ela o Righ Modi ica ion = yWRB: y
| | | ela o Righ Modi ica ion = yVB: n
| | | ela o Righ Modi ica ion = yNNP: y
| | | ela o Righ Modi ica ion = yNNS: y
| | | ela o Righ Modi ica ion = yEX: y
| | | ela o Righ Modi ica ion = yMD: y
| | | ela o Righ Modi ica ion = yPDT: y
| | | ela o Righ Modi ica ion = yNN: y
| | | ela o Righ Modi ica ion = yWP: y
| | | ela o Righ Modi ica ion = yPUNCT: y
| | ela o Le Modi ica ion = yCC: n
| | ela o Le Modi ica ion = yVB: y
| | ela o Le Modi ica ion = yNNP: y
| | ela o Le Modi ica ion = yNNS: n
| | ela o Le Modi ica ion = yNN
| | | cPas = yes: n
| | | cPas = no: y
| | ela o Le Modi ica ion = yPUNCT: y
ela o = because: y
Numbe o Lea es : 48
Size o he ee : 53
B.4 Decision T ee no. 4:
ela o = a e : n
ela o = as: n
ela o = since
| ePe ec i e = yes: n
| ePe ec i e = no
| | ela o Le Modi ica ion = yJJ
| | | lexicalClue = yes: n
| | | lexicalClue = no: y
| | ela o Le Modi ica ion = yIN: n
| | ela o Le Modi ica ion = yJJR: y
| | ela o Le Modi ica ion = yVBZ: y
| | ela o Le Modi ica ion = yDT: y
| | ela o Le Modi ica ion = yTO: y
| | ela o Le Modi ica ion = yVBG: y
| | ela o Le Modi ica ion = yRB
| | | cPas = yes: n
| | | cPas = no: y
| | ela o Le Modi ica ion = yVBP: n
| | ela o Le Modi ica ion = yVBD: y
| | ela o Le Modi ica ion = yPRP: y
| | ela o Le Modi ica ion = yCD: y
| | ela o Le Modi ica ion = yVBN
| | | lexicalClue = yes: n
| | | lexicalClue = no: y
| | ela o Le Modi ica ion = yCC: n
| | ela o Le Modi ica ion = yVB: y
| | ela o Le Modi ica ion = yNNP: y
| | ela o Le Modi ica ion = yNNS: n
| | ela o Le Modi ica ion = yNN
| | | cPas = yes: n
| | | cPas = no: y
| | ela o Le Modi ica ion = yPUNCT: y
ela o = because: y
Numbe o Lea es : 27
Size o he ee : 34
B.5 Decision T ee no. 5:
ela o = a e : n
B DECISION TREES LEARNED 60
| | ela o Le Modi ica ion = yCC: n
| | ela o Le Modi ica ion = yVB: y
| | ela o Le Modi ica ion = yNNP: y
| | ela o Le Modi ica ion = yNNS: n
| | ela o Le Modi ica ion = yNN
| | | cPas = yes
| | | | ela o Righ Modi ica ion = yPP: n
| | | | ela o Righ Modi ica ion = yJJ: n
| | | | ela o Righ Modi ica ion = y*: n
| | | | ela o Righ Modi ica ion = yIN: n
| | | | ela o Righ Modi ica ion = yVBZ: n
| | | | ela o Righ Modi ica ion = yDT: y
| | | | ela o Righ Modi ica ion = yTO: n
| | | | ela o Righ Modi ica ion = yVBG: n
| | | | ela o Righ Modi ica ion = yRB: y
| | | | ela o Righ Modi ica ion = yWDT: n
| | | | ela o Righ Modi ica ion = yVBP: n
| | | | ela o Righ Modi ica ion = yVBD: n
| | | | ela o Righ Modi ica ion = yPRP: n
| | | | ela o Righ Modi ica ion = yVBN: n
| | | | ela o Righ Modi ica ion = yCD: n
| | | | ela o Righ Modi ica ion = yWRB: n
| | | | ela o Righ Modi ica ion = yVB: n
| | | | ela o Righ Modi ica ion = yNNP: n
| | | | ela o Righ Modi ica ion = yNNS: n
| | | | ela o Righ Modi ica ion = yEX: n
| | | | ela o Righ Modi ica ion = yMD: n
| | | | ela o Righ Modi ica ion = yPDT: n
| | | | ela o Righ Modi ica ion = yNN: n
| | | | ela o Righ Modi ica ion = yWP: n
| | | | ela o Righ Modi ica ion = yPUNCT: n
| | | cPas = no: y
| | ela o Le Modi ica ion = yPUNCT: y
ela o = because: y
Numbe o Lea es : 48
Size o he ee : 53