An efficient data structure for decision rules discovery

An Eff1c1ent Data 5tructure f0r Dec1510n Ru1e5 D15c0very* Ra01 61r~1de2 C0mputer 5c1ence Dpmt. Un1ver51ty 0f 5ev111e Avda. Re1na Mercede5 51n 41012 5ev111e 5pa1n 91ra1de2•151.u5.e5 Je505 5. A9u11ar-Ru12 C0mputer 5c1ence Dpmt. Un1ver51ty 0f 5ev111e Avda. Re1na Mercede5 51n 41012 5ev111e 5pa1n a9u11ar•151.u5.e5 J056 C. R14ue1me C0mputer 5c1ence Dpmt. Un1ver51ty 0f 5ev111e Avda. Re1na Mercede5 51n 41012 5ev111e 5pa1n r14ue1me•151, u5.e5 A857RAC7 7he 1ncrea51n9 am0unt 0f 1nf0rmat10n ava11a61e 15 enc0ura9- 1n9 the 5earch f0r eff1c1ent techn14ue5 t0 1mpr0ve the data m1n1n9 meth0d5, e5pec1a11y th05e wh1ch c0n5ume 9reat c0m- putat10na1 re50urce5. We pre5ent a n0ve1 5tructure , ca11ed EE5, wh1ch he1p5 the data m1n1n9 a190r1thm5 wh1ch 9enerate dec1510n ru1e5 t0 reduce the af0rement10ned c05t. 61ven that dec1510n ru1e5 e5ta6115h c0nd1t10n5 f0r data6a5e attr16ute5, EE5 5t0re5 the 1nf0rmat10n 1n 5uch a way that the 5earch can 6e carr1ed 0ut 6y attr16ute5 1n5tead 0f 6y examp1e5. EE5 c0u1d 6e u5efu1 f0r any meth0d wh1ch 9enerate5 dec1- 510n ru1e5. M0re0ver, 1t 15 0f part1cu1ar 1ntere5t when the 5earch f0r the 501ut10n 1nv01ve5 a 9reat many hyp0thet1ca1 501ut10n5. 7hu5, th15 5tructure 15 de519ned f0r 5peed1n9 up the ru1e-eva1uat10n pr0ce55 1n meth0d5 6a5ed 0n Ev01ut10n- ary A190r1thm5. 7he trad1t10na1 5tructure, 6a5ed 0n vect0r5 0f examp1e5 (1n wh1ch the data6a5e 15 5t0red) 15 eva1uated and c0mpared w1th EE5, 1nc1ud1n9 the c05t5 f0r a 5trat1f1ed 5et 0f ca5e5. F1na11y, the exper1menta1 re5u1t5 dem0n5trate the 4ua11ty 0f 0ur pr0p05a1, reduc1n9 the c0mputat10na1 c05t 6y appr0x1mate1y 50%. Keyw0rd5 Data M1n1n9, Dec1510n Ru1e5, Prepr0ce551n9 1. 1N7R0DUC710N W1th1n the c0ntext 0f 5uperv15ed 1earn1n9 there ex15t 1n the 11terature a 1ar9e num6er 0f meth0d5 and a190r1thm5 that extract 1nherent kn0w1ed9e fr0m a 1a6e11ed data6a5e and 6u11d a m0de1 wh1ch repre5ent5 the extracted 1nf0rma- t10n. A 9reat num6er 0f the5e meth0d5 (CN2 [3], R15E [5], 0C1 [14], 6A81L [4], 61L[12], etc) u5e pr06a61115t1c a190- r1thm5 t0 100k f0r 501ut10n5 1n 5pace, w1th a h19h c0mputa- t10na1 c05t, pr1nc1pa11y due t0 the repet1t1ve eva1uat10n 0f the *7h15 re5earch wa5 5upp0rted 6y the 5pan15h Re5earch A9ency C1CY7 under 9rant 71C2001-1143-C03-02. Perm15510n t0 make d191ta1 0r hard c0p1e5 0f a11 0r part 0f th15 w0rk f0r per50na1 0r c1a55r00m u5e 15 9ranted w1th0ut fee pr0v1ded that c0p1e5 are n0t made 0r d15tr16uted f0r pr0f1t 0r c0mmerc1a1 advanta9e and that c0p1e5 6ear th15 n0t1ce and the fu11 c1tat10n 0n the f1r5t pa9e. 70 c0py 0therw15e, t0 repu6115h, t0 p05t 0n 5erver5 0r t0 red15tr16ute t0 115t5, re4u1re5 pr10r 5pec1f1c perm15510n and/0r a fee. 5AC 2003 Me160urne, F10r1da, U5A cand1date 501ut10n5. 7h15 w0rk f0cu5 0n th05e 5y5tem5 that def1ne a pr06a61115t1c heur15t1c f0r the 9enerat10n 0f dec1510n ru1e5, e5pec1a11y 0n th05e meth0d5 that u5e an Ev01ut10n- ary A190r1thm (hencef0rth EA) 1n 0rder t0 carry 0ut 5uch a1m. 7hu5, we pr0p05e a data 5tructure wh1ch reduce5 the c0mputat10na1 c05t 0f eva1uat10n 1n the 1earn1n9 pr0ce55. 7he af0rement10ned meth0d5 u5ua11y eva1uate the ru1e5 d1- rect1y fr0m the data6a5e. 7hat 15 t0 exp10re 5uch data6a5e 5e4uent1a11y, tak1n9 each 0f the examp1e5 and te5t1n9 the 4ua11ty 0f the ru1e thr0u9h the c0rrect c1a551f1cat10n 0f th05e examp1e5. We can 5ee, theref0re, that the 1earn1n9 pr0ce55 0f the5e 5y5tem515 very c05t1y 1n term5 0f t1me and 5pace. 50me auth0r5 [17, 19] have c0ncentrated the1r eff0rt5 0n 1mpr0v1n9 the 1earn1n9 pr0ce55 6y 5peed1n9 up the a190r1thm, 1n 0rder t0 reduce 1t5 c0mputat10na1 c05t. 0ther5 have appr0ached the pr061em fr0m the per5pect1ve 0f 5ca1a6111ty [20]. H0w- ever, the appr0pr1ate 0r9an15at10n 0f the 1nf0rmat10n c0u1d a150 c0ntr16ute t0 the reduct10n 0f c0mput1n9 t1me. 7h15 a5- pect, n0 1e55 1mp0rtant than the prev10u5 0ne, ha5 perhap5 6een m0re ne91ected. 7here are 1n the 11terature numer0u5 pr0p05a15 0n data 5tructure5 and 0r9an15at10n 0f the 1nf0rmat10n wh1ch e55en- t1a11y 5peed up the 5earch f0r 1nf0rmat10n 1n mu1t1d1men- 510na1 5pace5, 5uch a5 th05e kn0wn a5 Mu1t1d1men510na1 Acce55 Meth0d5 (MAM)[7]: P01nt Acce55 Meth0d5 (6r1d F11e [15], KD8-tree [18], L5D-tree [9], 8V-tree [6], etc) and 5pa- t1a1 Acce55 Meth0d5 (K-D-7ree [2], R-tree [8], P-tree [11], 5DK-tree [16], etc). H0wever, 91ven the pecu11ar1ty 0f the pr061em we are fac1n9, MAM d0 n0t, 1n them5e1ve5, pr0v1de a 501ut10n t0 5uch pr061em, 51nce they 1ndex a data5et w1th the 90a1 t0 5peed up the 4uer1e5 0n 5uch data. H0wever, we want t0 reduce the c05t 0f eva1uat10n 0f dec1510n ru1e5. 70 d0 1t, we d15tr16ute5 the examp1e5 acc0rd1n9 t0 the va1ue5 that they take f0r each attr16ute. 7hu5, we kn0w exact1y what examp1e5 fu1f111 the5e ru1e5. F0r 1n5tance, AD-7ree [13] 15 the MAM that 91ve5 the neare5t 501ut10n t0 the pr061em we want t0 501ve. Neverthe1e55, 1f we w0u1d want t0 eva1uate a dec1510n ru1e5 u51n9 th15 5tructure, we w0u1d have t0 6u11d the n0 5par5e AD-7ree, that 15 t0 5t0re a11 p055161e 4uer1e5 1n add1t10n t0 the 1ndexe5 0f the examp1e5 that each 4uery 1nc1ude5. 7h15 mean5 very h19h c0mputat10na1 c05t, 51nce t00 much redundant 1nf0rmat10n 15 5t0red. 1n th15 w0rk we are pre5ent1n9 a data 5tructure ca11ed EE5 (Eff1c1ent Eva1uat10n 5tructure) wh1ch 15 de519ned 5pec1f1- ca11y f0r acce1erat1n9 the eva1uat10n pr0ce55 0f ru1e5 dur1n9 the app11cat10n 0f data m1n1n9 a190r1thm5 6a5ed 0n EA5. 7he EE5 5tructure 0r9an15e5 the 1nf0rmat10n fr0m a data6a5e 1n 5uch a way that 1t 15 n0t nece55ary t0 pr0ce55 a11 the ex- • 2003 ACM 1-58113-624-2/03/03...$5.00. 475 Ru1e: 1f A 11n [3.5, 4.9] end A. 21n {V1, V2, V3, V5, V7 } and ... and A m 1n [7.0,12.7] then C1a55 8 A~ A. 2 ... Am C1a.~ C0vered c1a55~ed E,~,,~5 E~4 ~115 v5 Vr ...••" ~°~97.9 1 c 5 --.-•-* .0 y"= Y••• F19ure 1: Eva1uat10n u51n9 a vect0r 0f examp1e5. amp1e5 1n 0rder t0 eva1uate dec1510n ru1e5 9enerated 6y a 5uperv15ed 1earn1n9 5y5tem. An EA c0d1f1e5 dec1510n ru1e5 a5 1nd1v1dua15 1n the 9enet1c p0pu1at10n. 0nce the p0pu1at10n 15 c0n5tructed the 1nd1- v1dua15 are eva1uated and depend1n9 0n the 4ua11ty 0f each 0ne 0f them, the cr0550ver and mutat10n 0perat0r5 are app11ed, thu5 c0n5truct1n9 the next 9enerat10n. 1t 15 51mp1e t0 5ee that th05e 5y5tem5 that app1y EA5 need t0 carry 0ut c0n5tant eva1uat10n5 thr0u9h0ut the 1earn1n9 pr0ce55, 51nce they have t0 eva1uate each 0ne 0f the 1nd1v1dua15 0f each 0ne 0f the p0pu1at10n5 that they 9enerate. F0r examp1e, an EA that carr1e5 0ut 300 9enerat10n5, each c0nta1n1n9 100 1nd1- v1dua15, need5 t0 carry 0ut at 1ea5t 30,000 eva1uat10n5. 1f the data6a5e c0nta1n5 1000 examp1e5 the pr0ce55 w111 6e carr1ed 0ut 30 m11110n t1me5. 1n the part1cu1ar ca5e 0f EA5 app11ed t0 the 9enerat10n 0f ru1e5, the data6a5e 15 5t0red 1n a vect0r 0f examp1e5 (5ee F19ure 1), wh1ch 15 u5ed f0r eva1uat1n9 0f every 1nd1v1dua1. Eva1uat10n 6y mean5 0f a 11near 5earch pr0ce55e5 each and every 0ne 0f the examp1e5 1n the data6a5e 1ndependent1y 0f the c0nd1t10n5 e5ta6115hed 6y the ru1e. A5 can 6e 5een 1n F19ure 1, n0t a11 the c0vered examp1e5 are c0rrect1y c1a551f1ed (0n1y Examp1e5 4 and N). 7heref0re the c0mputat10na1 c05t 0f a 51n91e eva1uat10n 15 0(NM), where N 15 the num6er 0f examp1e5 and M 15 the num6er 0f attr16ute5 1n the data6a5e. 7he ru1e-1earn1n9 meth0d5 that EA5 u5e 1nve5t appr0x1mate1y 85% 0f the1r t1me 1n eva1uat1n9 the 1nd1v1dua15 (the mean 0f the execut10n5 0f a 10-f01d cr055-va11dat10n w1th 20 UC1 Rep051t0ry data6a5e5 [1]). 7h15 15 the rea50n why we pr0p05e the EE5 5tructure 1n 0rder t0 0r9an15e the 1nf0rmat10n 1n 5uch a way that 0n1y the nece55ary examp1e5 fr0m the data6a5e w111 6e dea1t w1th, and n0t a11 data. 2. DE5CR1P710N EE5 d15tr16ute5 the 1nf0rmat10n fr0m the data6a5e 1n 5uch a way 1t 15 p055161e t0 carry 0ut a 5earch 1n the 5pace 6y attr16ute 1n5tead 0f 6y examp1e. 7he data 5tructure mu5t 6e capa61e 0f 5t0r1n9 th15 1nf0rmat10n 1ndependent 0f the type 0f attr16ute (c0nt1nu0u5 0r d15crete). 1n the ca5e 0f c0nt1nu0u5 attr16ute5 1t 15 c0nven1ent t0 app1y a meth0d t0 tran5f0rm them 1nt0 d15crete wh1ch reduce5 the card1na11ty 0f the 5et 0f va1ue5 that th15 type 0f attr16ute5 can eventua11y take. 1n certa1n ca5e5 the meth0d5 0f ru1e-9enerat10n app1y a meth0d 0f d15cret12at10n a5 pre-pr0ce551n9 0f the data 1n 0rder t0 ca1cu1ate 5uch 1nterva15. EE5 mu5t 6e c0n5tructed u51n9 the 5ame 5et5 0f 1nterva15 06ta1ned 6y the d15cret12a- t10n meth0d app11ed 6y the ru1e-9enerat10n meth0d. 7h15 d0e5 n0t 1mp1y any k1nd 0f 11m1tat10n a5 re9ard5 the data 5tructure, 51nce the 1atter 15 t0ta11y f1ex161e a5 0pp05ed t0 the d15cret12at10n u5ed 6y the c0nt1nu0u5 attr16ute5. 7he 0n1y re5tr1ct10n demanded 0f the d15cret12at10n meth0d 15 that the 9enerated 1nterva15 6e d15j01nt, 91ven that 1f th15 were n0t the ca5e there c0u1d ex15t 1n the data6a5e a va1ue perta1n1n9 t0 var10u5 1nterva15. 1n any ca5e we w111 6e a61e t0 ma1nta1n 1nf1n1te rank, that 15 t0 5ay, we w111 n0t need t0 app1y any d15cret12at10n meth0d. 7he EE5 5tructure w0u1d 5t111 6e va11d: 1n th15 ca5e the num6er 0f n0de5 w0u1d 1n- crea5e and the 512e 0f the 115t5 a550c1ated w1th the n0de5 w0u1d 6e reduced even t0 1en9th e4ua1 t0 0ne. 1n 9enera1, f0r every attr16ute A1 1n the data6a5e we w111 den0te the f1n1te 5et 0f va1ue5 that A1 can take 6y f~1. 1n the ca5e 0f A1 6e1n9 a d15crete attr16ute f~1 w111 c0nta1n va1ue5 wh1ch we w111 repre5ent a5 ~j- (1 < j < [f~1[). 0n the 0ther hand, 1f we axe dea11n9 w1th a c0nt1nu0u5 attr16ute, ~1 w111 c0nta1n 1nterva15 wh1ch we w111 ca11 11j (1 < j < 1f~1), the 10wer and upper 60und5 0f wh1ch we w111 den0te 6y 11j and u1j re5pect1ve1y. 1n th15 manner EE5 w111 5t0re the 1nf0rmat10n 0f the c0nt1nu0u5 attr16ute5 1n a 51m11ar way t0 that wh1ch 1t d0e5 f0r the d15crete attr16ute5: the 10wer and upper 11m1t5 0f each 1nterva1 w111 6e 5aved 1n p1ace 0f the un14ue va1ue5 that are 5t0red f0r the d15crete attr16ute5. EE5 arran9e5 the 1nf0rmat10n fr0m the data6a5e 1n a vect0r 0f 61nary and 6a1anced 5earch-tree5 1n 5uch a way that the 1 th e1ement 0f the vect0r w111 c0nta1n 1nf0rmat10n a60ut the 1 ~h attr16ute (A1) 1n the data6a5e. 5pec1f1ca11y, the d1f- ferent va1ue5 0r 1nterva15 that A1 can take are 5t0red 1n the tree, wh1ch we w111 den0te 6y 71. 1n add1t10n t0 ~j 0r 11j , each n0de N1j 0f the tree 71 c0nta1n5 a 115t (L1j) 0f num6er5 wh1ch 1nd1cate the p051t10n5 0f examp1e5 1n the data6a5e. 1f A1 15 d15crete, the 1ndexe5 c0nta1ned 1n the 115t L1j w111 c0r- re5p0nd t0 th05e examp1e5, the 1 th attr16ute5 0f wh1ch take the jth va1ue (V~j) w1th1n the f~1 5et 0f p055161e va1ue5 0f 5uch attr16ute. 1f A~ 15 c0nt1nu0u5, the 1ndexe5 c0nta1ned 1n L1j w111 c0rre5p0nd t0 th05e examp1e5, the va1ue 0f wh1ch f0r the 1 th attr16ute 15 1nc1uded 1n the jth 1nterva1 (11j) w1th1n the f~1 5et 0f p055161e 1nterva15 0f 5uch attr16ute. F19ure 2 5h0w5 a examp1e 0f the data 5tructure f0r a data6a5e w1th 15 examp1e5 and 2 attr16ute5, where the 15t attr16ute 15 c0nt1n- u0u5 wh11e the 2 n4 15 d15crete. 1t 15 1mp0rtant t0 n0te that 1n the ca5e 0f c0nt1nu0u5 attr16ute5 the tree 15 50rted (1n0rder) 6y (d15j01nt) 1nterva15 and 1n the ca5e 0f d15crete attr16ute5 1t 15 arran9ed a1pha6et1ca11y 6y the d15crete va1ue. 1n th15 manner, any 5earch w1th1n the tree ha5 a 109ar1thm1c c05t. 2.1 C0n5truct10n 0fthe EE5 Let u5 5tart fr0m the 1nf0rmat10n c0nta1ned 1n a 1a6e11ed data6a5e w1th N examp1e5 1ndexed fr0m 0ne t0 N, each 0f them w1th M attr16ute5 0f any type (5ee F19ure 2). 1n the5e M attr16ute5 we are n0t 1nc1ud1n9 the c1a55. Prev10u51y a d15- cret12at10n meth0d 51m11ar t0 the meth0d 1R de5cr16ed 6y H01te 1n [10] ha5 6een app11ed, a1th0u9h any 0ther 5uperv15ed d15cret12at10n meth0d c0u1d have 6een u5ed. 7h15 meth0d 06ta1n5 a 5et 0f d15j01nt 1nterva15 f0r each 0ne 0f the c0n- t1nu0u5 attr16ute5 1n the data6a5e. F0r each attr16ute (A1) a 6a1anced 5earch-tree 15 created and 1n5erted 1n the 5truc- ture 1n the c0rre5p0nd1n9 p051t10n (1). 7he n0de5 0f the5e tree5 c0nta1n the va1ue5 0r 1nterva15 acc0rd1n9 re5pect1ve1y t0 whether the attr16ute 15 d15crete 0r c0nt1nu0u5. 0nce a11 the tree5 have 6een created and 1n5erted, the 115t5 f0r each n0de 476 A,:512e 2.0 1.0 1.2 1.4 1.0 1.4 1.7 1.8 1.2 2.1 2.2 1.0 1.2 2.0 1.6 A2:C010ur 61ue 9reen 9reen 61ue red red 61ue 9reen red 61ue 9reen 61ue 61ue 9reen 61ue C1a55 A A A A A A 8 8 8 A A A 8 A 8 A1(~Unu0u ~ A~(d1aaete) 7ree7, 7re; 7, F19ure 2: Examp1e. are c0mp1eted. A 11near 5earch 0f the data6a5e 15 carr1ed 0ut, pa551n9 thr0u9h a11 the examp1e5. F0r each attr16ute A1 0f a 5pec1f1c examp1e, the n0de c0rre5p0nd1n9 t0 the va1ue 0f 5uch attr16ute w111 6e 5earched f0r 1n the tree 71. 0nce the n0de ha5 6een 10cated, the 1ndex 0f the examp1e 6e1n9 pr0ce55ed 15 1•n5erted 1nt0 the 115t 0f 5uch n0de and the next attr16ute 0f th15 examp1e 15 pr0ce55ed. When a11 the attr16ute5 0f an examp1e have 6een treated we 5ay that the examp1e ha5 6een 1n5erted 1nt0 the 5tructure, and we 90 0n t0 pr0ce55 the next examp1e. At the p01nt at wh1ch a11 the examp1e5 have 6een 1n5erted, the 5tructure c0nta1n5 the 5ame 1nf0rmat10n a5 the data6a5e, except1n9 the c1a55 0f each examp1e. H0w- ever, th15 1nf0rmat10n 15 d1rect1y acce55161e dur1n9 the u5e 0f the 5tructure, 51nce the 1atter 5t0re5 the 1ndexe5 0f examp1e5. 7he c0mputat10na1 c05t 0f the c0n5truct10n pr0ce55 0f the data 5tructure 15 0(NM1092[-~[), where --f~ 15 the mean num6er 0f tree-n0de5. 2.2 U5e 0fthe EE5 7he fundamenta1 pr0perty that the data 5tructure 0f- fer5 15 the p05516111ty 0f acce551n9 the 1nf0rmat10n fr0m the data6a5e thr0u9h attr16ute5 1n5tead 0f thr0u9h examp1e5. 7he ma1n a1m that we are 100k1n9 f0r 1n the u5e 0f the data 5tructure 15 n0t hav1n9 t0 pr0ce55 th05e examp1e5, the va1ue5 0f wh1ch are n0t c0vered 6y the ru1e wh1ch 15 6e1n9 eva1u- ated. 1fwe take a n0de N1j 0f each 71, the 1nter5ect10n 0fthe 115t5 L1j w111 6e the 5et 0f the 1ndexe5 0f examp1e5 wh1ch fu1- f111 that each 0ne 0f 1t5 attr16ute5 A1 take va1ue5 wh1ch are c0vered 6y each c0rre5p0nd1n9 n0de N1j. 7he advanta9e that the EE5 0ffer5 11e5 1n the fact that the 1nter5ect10n5 are carr1ed 0ut 1n an 1ncrementa1 manner - that 15 t0 5ay, f1r5t1y the 1nter5ect10n 0f the 115t f0r the attr16ute A1 and the 115t f0r the attr16ute A2 15 carr1ed 0ut. 1f 5uch 1nter5ect10n 15 n0t empty, the 115t f0r the attr16ute A3 15 5earched f0r and a new 1nter5ect10n 6etween th15 and the re5u1t 0f the prev10u5 1nter5ect10n 15 created. 7h15 pr0ce55 15 repeated unt11 a11 the attr16ute5 are c0mp1eted, 0r unt11 0ne 0f the 1nter5ect10n5 rema1n5 empty. 1f the pr0ce55 c0nc1ude5, the re5u1t1n9 115t w111 c0nta1n the 1ndexe5 0f the examp1e5 wh1ch fu1f111 the va1ue5 0r 1nterva15 0f a11 the 5e1ected n0de5 N1j. 1f the5e n0de5 N1j are 5earched f0r acc0rd1n9 t0 the c0nd1t10n5 e5ta6115hed 6y a dec1510n ru1e, we w111 6e eva1uat1n9 5uch ru1e. 7he p5eud0- c0de 0f the eva1uat10n a190r1thm f0r dec1510n ru1e5 u51n9 the EE5 15 5h0wn 1n F19ure 3. 7he n0tat10n u5ed 1n the a190r1thm 1n F19ure 3 15 the 5ame Funct10n EVALUA7E 1nput5: R: Dec1510n Ru1e; E: Data 5tructure 0utput: L: L15t 0f 1ndexe5 (examp1e5 c0vered 6y R) 6e91n 1:=1 L1 := L15tUn10n(R, E, 1) wh11e 1 < Num6er0f Attr16ute5(E) A L1 ~ 0 1:----1+1 L1 :---- L~-1 N L15tUn10n(R, E, 1) end wh11e L := L1 end EVALUA7E Funct10n L15tUn10n 1nput5: R: Dec1510n Ru1e; E: Data 5tructure k: 1nte9er (attr16ute 10cat10n) 0utput: Lu: L15t 0f 1ndexe5(¢xamp1e, c0vered 6y R k ) 6e91n Lu := 0; 7~ := E[k]; 1f Ak 15 C0nt1nu0u5 f0r every 1kj E 7k 1 1j, j C Rk Lu := LuULkj end f0r e15e /* Ak 15 D15crete */ f0r every V~j E 7~ 1 V~j C R~ Lu := LuULkj end f0r end 1f end L15tUn10n F19ure 3: A190r1thm f0r the u5e 0f EE5. a5 that u5ed 1n the 9enera1 5tructure 0f F19ure 2, except1n9 the f0110w1n9 5ym6015: E repre5ent5 the data 5tructure and R 15 the ru1e t0 6e eva1uated, wh11e R1 15 the c0nd1t10n that R e5ta6115he5 f0r the attr16ute A1. 7he 5ym601 L1 (w1th a 51n91e 5u61ndex) repre5ent5 the 115t 0f accumu1ated 1nter- 5ect10n5 unt11 the 1 th 1terat10n, that 15 t0 5ay, unt11 the 1 th attr16ute. 7he a190r1thm 15 d1v1ded 1n tw0 funct10n5. 7he f1r5t, EVALUA7E, 15 the ma1n funct10n and ha5 a5 1t5 1nput parameter5 the dec1510n ru1e R and the data 5tructure E. 7he ru1e R ha5 the 5tructure that 15 5h0wn 1n F19ure 1, the c0nd1- t10n that R e5ta6115he5 f0r the k th attr16ute 1n the data6a5e 6e1n9 repre5ented 6y Rk. 7he funct10n EVALUA7E 0ffer5 a 51n91e 0utput parameter (L), wh1ch 15 the 115t 0f 1ndexe5 0f examp1e5 re5u1t1n9 fr0m the eva1uat10n 0f the ru1e R 0n the data 5tructure E. 1n th15 way, the funct10n EVALUA7E 5earche5 the ru1e R and the 5tructure E 51mu1tane0u51y, ca1- cu1at1n9 the 1nter5ect10n5 0f the 115t5 and return1n9 the f1na1 115t (L). 7he ma1n funct10n u5e5 an aux111ary funct10n named L15tUn10n, that ha5 the ru1e R, the 5tructure E and a 1nte- 477 9er wh1ch 1nd1cate5 the attr16ute wh1ch 1t ha5 t0 dea1 w1th. 7he 0n1y 0utput parameter 0f th15 funct10n 15 the 115t 0f 1n- dexe5 (Lu) c0rre5p0nd1n9 t0 the un10n 0f the 115t5 Lkj f0r the attr16ute A~ (w1th 1 < j < 1~k1). 7h15 un10n 0f 115t515 nec- e55ary 0w1n9 t0 the fact that the c0nd1t10n 0f the ru1e P~ f0r the attr16ute Ak can 1nc1ude var10u5 n0de50f the tree 7~. 1n the ca5e 0f c0nt1nu0u5 attr16ute5 the f1r5t n0de 1nc1uded 1n the c0nd1t10n that the ru1e e5ta6115he5155earched f0r 1n the tree. 0nce 10cated, a 1n0rder-wa1k 0f the tree 5tart1n9 fr0m 5uch n0de and 5t0pp1n9 when 1t f1nd5 the f1r5t n0de wh1ch 15 n0t c0vered 6y the c0nd1t10n 15 carr1ed 0ut. 1n the ca5e 0f d15crete, 1t 15 nece55ary t05earch f0r a11 the va1ue5 that the ru1e e5ta6115he5 f0r 5uch attr16ute 1n the c0rre5p0nd1n9 tree. When dea11n9 w1th 60th ca5e50f 6a1anced 5earch tree5, a11 the 5earche5 carr1ed 0ut 1n them have a 109ar1thm1c c05t. 8D 7a61e 1" C0mpar150n 0f avera9e re5u1t5 A.7. Vect0r 1mpr0v.(%) A.7. E55 1.464 4.370 66.5 5.572 13.421 58.5 2.173 4.870 55.4 3.460 6.042 42.7 1.634 2.884 43.3 0.720 1.499 52.0 3.018 6.228 51.5 0.517 1.536 66.3 20.689 34.905 40.7 222.556 842.784 73.6 4.283 11.309 62.1 1.661 2.808 40.8 7.623 11.031 30.9 8.937 18.785 52.4 2.084 4.120 49.4 6upa 11ver d150rder 6rea5t cancer (W15c.) car5 c1eve1and 91a55 haye5-r0th heart de5ea5e 1r15 1ed7 1etter p1ma 1nd1an 50y6ean- t1c-tac-t0e veh1c1e w1ne 3. EXPER1MEN75 7he exper1ment5 carr1ed 0ut t0 te5t the eff1c1ency 0f the 5tructure c0n515t 0f eva1uat1n9 var10u5 5et5 0f dec1510n ru1e5 f0r 15 d1fferent data6a5e5 fr0m the UC1 Rep051t0ry, a5 0p- p05ed t0 the 11near 5earch ha61tua11y u5ed f0r the eva1ua- t10n 0f dec1510n ru1e5. F0r each data6a5e 9r0up5 0f ru1e5 are 9enerated 1n an a1eat0ry manner, thr0u9h u5e 0f a meth0d wh1ch a55ure5 the un1f0rm d15tr16ut10n 0f the5e ru1e5. 5u6- 5e4uent1y, 5uch ru1e5 are eva1uated u51n9 the 11near meth0d and the data 5tructure EE5 and the re5u1t506ta1ned are c0mpared. 7he 11near meth0d 0f eva1uat10n u5ed 1n the te5t515 the m05t eff1c1ent 0ne p055161e. F0r each ru1e th15 meth0d 5earche5 the data6a5e, wh1ch ha5 prev10u51y 6een 5t0red 1n an vect0r 0f examp1e5, pr0ce551n9 each and every 0ne 0f the examp1e5. L1kew15e, f0r each examp1e, the ver1f1cat10n that 1t5 attr16ute5 fu1f111 the c0nd1t10n50f the ru1e 15 a150 carr1ed 0ut 1n a 11near manner. H0wever, 1t 15 n0t a1way5 nece55ary t0 pr0ce55 a11 the attr16ute50f every examp1e. 1f, dur1n9 the pr0ce551n90f an examp1e, 0ne 0f 1t5 va1ue5 d0e5 n0t fu1f111 the c0nd1t10n that the ru1e e5ta6115he5 f0r the c0rre5p0nd1n9 attr16ute, that examp1e 15 n010n9er pr0ce55ed a51t w111 n0 10n9er 6e a61e t0 fu1f111 the ru1e, 1ndependent 0f the va1ue5 that the re5t 0f the attr16ute5 take, and the next examp1e w1116e pr0ce55ed. 61ven that c0u1d th1nk that the maj0r advanta9e 0f the 5tructure 11e51n the eva1uat10n 0f ru1e5 wh1ch d0 n0t c0ver examp1e5, 1n 0rder t0 en5ure that the te5t-meth0d 15 fa1r tw0 type50f ru1e5 have 6een 9enerated and eva1uated: va11d and 1nva11d (5ee the re5u1t5 vary1n9 the percenta9e 0f va11d ru1e51n F19ure 4). Let u5 ca11 a ru1e va11d 1f 1t c0ver5 at 1ea5t 0ne examp1e 1n the data6a5e. 1n c0mpar150n, a ru1e wh1ch d0e5 n0t c0ver any 0f the examp1e5 1n the data6a5e 5ha116e ca11ed 1nva11d. Acc0rd1n9 t0 the5e def1n1t10n5 a va11d ru1e w111 en5ure that the 5tructure 15 c0m- p1ete1y 5earched, wh11e 1n the ca5e 0f an 1nva11d ru1e the eva1uat10n pr0ce550f the data 5tructure w1116e ha1ted 6e- f0re the 5earch 0f the 1atter ha56een c0mp1eted. A1th0u9h, a pr10r1, the eva1uat10n 0f 1nva11d ru1e55eem5 t06e p01nt1e55, th1515 n0t 50, 51nce the 1earn1n9 meth0d56a5ed 0n EA 9en- erate 1ntermed1ate ru1e5 wh1ch 1n many ca5e5 d0 n0t c0ver any examp1e5. 1t 15 w0rth n0t1n9 t0 take th151nt0 acc0unt, 5ee1n9 that EE515 f0cu50n th15 k1nd 0f meth0d5. 7a61e 15h0w5 the f0110w1n9: f0r each data6a5e (f1r5t c01- umn) the avera9e t1me u5ed 6y the EE55tructure (5ec0nd c01umn) and that u5ed 6y the 11near meth0d (th1rd c01umn), a5 we11 a5 the 1mpr0vement 06ta1ned 6y EE5 a5 c0mpared w1th eva1uat10n u51n9 the vect0r (f1na1 c01umn). F0r each data6a5e the 1mpr0vement 1591ven 6y E4uat10n 1, wh1ch rePre5ent55tructure w1ththe re5pectPercenta9et0 the°f t1met1me :::dd6Yth71vne9c/tthe:E5e avera9e-ca5e. 1mpr0vement = 100 x 71me(Vect0r) - 71me(EE5) (1) 71me(Vect0r) A5 can 6e 065erved 1n 7a61e 1, f0r a11 the data6a5e5 the avera9e t1me taken 6y the EE515 n0t1cea61y 1e55 than that taken 6y the vect0r. 7h15 en5ure5 that the 1mpr0vement w111 a1way56e p051t1ve, that 15 t05ay, that the EE5 a1way5 1mpr0ve50n the 11near 5tructure. 1f we ca1cu1ate the mean 1mpr0vement f0r the 15 data6a5e5, we 06ta1n the re5u1t that the avera9e 1mpr0vement 1552.4%, wh1ch 15 t05ay that the 5tructure take5 pract1ca11y ha1f a5 much t1me 1n eva1uat1n9 the ru1e5. 50me 0f exper1menta1 re5u1t5 are 9raph1ca11y 5h0wn 1n F19- ure 4. 7h15 F19ure c0nta1n529raph5 (1r15 and P1rna D1a- 6ete5) wh1ch repre5ent the eva1uat10n t1me 1n 5ec0nd5 a9a1n5t the percenta9e 0f va11d ru1e51n the 5et50f ru1e5 eva1uated. 1n turn, each 9raph c0nta1n5 tw0 curve5: the 9rey 11ne 5h0w5 the temp0ra1 re5u1t506ta1ned f0r the eva1uat10n u51n9 the vect0r 0f examp1e5, wh11e the 61ack 11ne refer5 t0 the re5u1t5 f0r the eva1uat10n u51n9 the EE55tructure. 80th repre5en- tat10n55h0w the temp0ra1 var1at10n a5 the percenta9e 0f va11d ru1e5151ncrea5ed. A59raph55h0w, f0r the5e data6a5e5 u5ed the 6ehav10ur 0f the EE55tructure 15 very fav0ura61e 1n c0mpar150n w1th the vect0r 0f examp1e5. 7he f1r5t re5u1t that 5tand50ut 15 that f0r a11 the data6a5e5 u5ed, the eva1u- at10n t1me taken 6y the EE55tructure 151nfer10r t0 the t1me 5pent 6y the 11near meth0d. 7he re5u1t55h0w that EE515 h19h1y eff1c1ent 1ndependent1y 0f the type 0f ru1e (va11d 0r 1n- va11d), wh1ch 91ve an 1dea 0f the r06u5tne550f the 5tructure. 4. C0NCLU510N5 1n th15 w0rk we are pre5ent1n9 the data 5tructure ca11ed EE5, the 90a10f wh1ch 15 t00r9an15e the 1nf0rmat10n fr0m a data6a5e 1n 5uch a manner that the eff1c1ency 0f the meth- 0d50f 9enerat1n9 dec1510n ru1e56a5ed 0n Ev01ut10nary A1- 90r1thm5151mpr0ved. E55 a110w5 u5 t0 pr0ce550n1y th05e examp1e5, the va1ue50f wh1ch are c0vered 6y 5uch ru1e w111 6e pr0ce55ed, and n0t the t0ta11ty 0f the data6a5e. 1n th15 478 2 ......................... ~,~,~- ~--, ....... 1.8 .... • ..... ~- - ~ - -~- - 4 - - 1.6 1--• -• ............... -- 121.4 - 1 ~=~:. 1 ;-- ~-t~,t,..-,~- :..,_.,. _,.. 1- °:= ° --9-- ••1-- 1 =•--: 111f1 -1- 1111 ......... Va11d Ru1e= (%) (a) 1r15 Va11d Ru1e5 (%) (6) P1ma D1a6ete5 F19ure 4:Re5u1t5 f0r 1r15 and P1ma D1a6ete5. way, the eva1uat10n pr0ce55 0f examp1e5 15 1ncrementa1, that 15 t0 5ay, 1t 5tart5 fr0m a num6er 0f c0vered examp1e5 that 15 reduced a5 we ana1y5e m0re attr16ute5 0f the 5tructure. When the num6er 0f c0vered examp1e5 15 reduced t0 0, the ru1e cea5e5 t0 6e 0f 1ntere5t, and theref0re 15 n0 10n9er eva1u- ated. 7hu5 EE5 ca1cu1ate5 what examp1e5 are w1th1n a ru1e, 1f any; and n0t check 1f each examp1e 5at15f1e5 a11 the c0nd1- t10n5 0f a ru1e, a5 the vect0r 0f examp1e5 w0u1d d0. 50 th15 5tructure 15 n0t a meth0d f0r 1ndex1n9 examp1e5. F1na11y, the exper1menta1 re5u1t5 5h0w the 4ua11ty 0f 0ur pr0p05e. 5. ADD1710NAL AU7H0R5 Dan1e1 Mate05, C0mputer 5c1ence Department, Un1ver51ty 0f 5ev111e, mate05•151.u5.e5 6. REFERENCE5 [1] J. 5. A9u11ar. D15c0ver1n9 H1erarch1ca1 Dec1510n Ru1e5 w1th Ev01ut10nary A190r1thm5 1n 5uperv15ed Learn1n9. PhD the515, Un1ver51ty de 5ev111e, 2001. [2] J. L. 8ent1ey and J. H. Fr1edman. Data 5tructure5 f0r ran9e 5earch1n9. A CM C0mput1n9 5urvey5, 11(4):397-409, 1979. [3] P. C1ark and R. 805we11. Ru1e 1nduct10n w1th cn2: 50me recent5 1mpr0vement5. Mach1ne Learn1n9: Pr0ceed1n95 0f EW5L-91, pp. 51-163, 1991. [4] K. A. De30n9, W. M. 5pear5, and D. F. 60rd0n. U51n9 9enet1c a190r1thm5 f0r c0ncept 1earn1n9. Mach1ne Learn1n9, 1(13):161-188, 1993. [5] P. D0m1n905. Ru1e 1nduct10n and 1n5tance-6a5ed 1earn1n9: A un1f1ed appr0ach. 1n Pr0ceed1n95 0f 479 1nternat10na1 J01nt C0nference 0n Art1f1c1a1 1nte1119ence, 1995. [6] M. Free5t0n. A 9enera1 501ut10n 0f the n-d1men510na1 6-tree pr061em. 1n M. J. Carey and D. A. 5chne1der, ed1t0r5, Pr0ceed1n95 0f the 1995 ACM 516M0D 1nternat10na1 C0nference 0n Mana9ement 0f Data, 5an J05e, Ca11f0rn1a, May 22-25, 1995, pa9e5 80-91. ACM Pre55, 1995. [7] V. 6aede and 0. 611nther. Mu1t1d1men510na1 acce55 meth0d5. ACM C0mput1n9 5urvey5, 30(2):170-231, 1998. [8] A. 6uttman. R-7ree5 : A dynam1c 1ndex 5tructure f0r 5pat1a1 5earch1n9. 1n Pr0e. 0f the ACM 516M0D 1nt1. C0nf. 0n Mana9ement 0f Data, pa9e5 pp. 47-57, 1984. [9] A. Henr1ch, H.-W. 51x, and P. W1dmayer. 7he L5D tree: 5pat1a1 acce55 t0 mu1t1d1men510na1 p01nt and n0np01nt 06ject5. 1n P. M. 6. Aper5 and 6. W1ederh01d, ed1t0r5, Pr0ceed1n95 0f the F1fteenth 1nternat10na1 C0nference 0n Very Lar9e Data 8a5e5, Au9u5t 22-25, 1989, Am5terdam, 7he Nether1and5, pa9e5 45-53. M0r9an Kaufmann, 1989. [10] R. C. H01te. Very 51mp1e c1a551f1cat10n ru1e5 perf0rm we11 0n m05t c0mm0n1y u5ed data5et5. Mach1ne 1earn1n9, 11:63-91, 1993. [11] H. V. Ja9ad15h. 5pat1a1 5earch w1th p01yhedra. 1n Pr0ceed1n95 0f the 51xth 1nternat10na1 C0nference 0n Data En91neer1n9, Fe6ruary 5-9, 1990, L05 An9e1e5, Ca11f0rn1a, U5A, pa9e5 311-319. 1EEE C0mputer 50c1ety, 1990. [12] C. 2. Jan1k0w. A kn0w1ed9e-1nten51ve 9enet1c a190r1thm f0r 5uperv15ed 1earn1n9. Mach1ne Learn1n9, 1(13):169-228, 1993. [13] Andrew W. M00re and Mary 5. Lee. Cached 5uff1c1ent 5tat15t1c5 f0r Eff1c1ent Mach1ne Learn1n9 w1th Lar9e Data5et5. 1n J0urna1 0f Art1f1c1a1 1nte1119ence Re5earch, v01. 8, pa9e5 67-91, 1998. [14] 5. K. Murthy, 5. Ka51f, and 5. 5a126er9. A 5y5tem f0r 1nduct10n 0f 06114ue dec1510n tree5. J0urna1 0] Art1f1c1a1 1nte1119ence Re5earch, 1994. [15] J. N1ever9e1t, H. H1nter6er9er, and K. C. 5evc1k. 7he 9r1d f11e: An adapta61e 5ymmetr1c mu1t1key f11e 5tructure. A CM 7ran5act10n5 0n Data6a5e 5y5tem5, ACM CR 8411-0931, 9(1), 1984. [16] 8. C. 001. 5pat1a1 KD-7ree: A data 5tructure f0r 9e09raph1c data6a5e. 1n 8W7, pa9e5 247-258, 1987. [17] R. Qu1n1an. 5ee5.0 (5ttp://www.ru1e4ue5t.c0m), 1998-2001. [18] J. 7. R061n50n. 7he K-D-8-7ree: A 5earch 5tructure f0r 1ar9e mu1t1d1men510na1 dynam1c 1ndexe5. 1n Y. E. L1en, ed1t0r, Pr0ceed1n95 0f the 1981 ACM 516M0D 1nternat10na1 C0nference 0n Mana9ement 0f Data, Ann Ar60r, M1ch19an, Apr11 29 - May 1, 1981, pa9e5 10-18. ACM Pre55, 1981. [19] 5. Ru991er1. Eff1c1ent C4.5. 7echn1ca1 Rep0rt 7R-00-01, 2, 2000. [20] K. 5h1m. 516KDD Exp10rat10n5. Decem6er 2000. V01ume 2, 155ue 2.