1
Scien i ic RepoR s | 6:30828 | DOI: 10.1038/s ep30828
www.na u e.com/scien i ic epo s
A Compa ison o Rule-based
Analysis wi h Reg ession Me hods
in Unde s anding he Risk Fac o s
o S udy Wi hd awal in a Pedia ic
S udy
Mona Haghighi1, Suzanne Benne Johnson2, Xiaoning Qian3, K is ian F. Lynch4,
Kend a Vehik4, Shuai Huang5 & The TEDDY S udy G oup†
Reg ession models a e ex ensi ely used in many epidemiological s udies o unde s and he linkage
be ween speci ic ou comes o in e es and hei isk ac o s. Howe e , eg ession models in gene al
examine he a e age e ec s o he isk ac o s and igno e subg oups wi h di e en isk p o iles. As
a esul , in e en ions a e o en gea ed owa ds he a e age membe o he popula ion, wi hou
conside a ion o he special heal h needs o di e en subg oups wi hin he popula ion. This pape
demons a es he alue o using ule-based analysis me hods ha can iden i y subg oups wi h
he e ogeneous isk p o iles in a popula ion wi hou imposing assump ions on he subg oups o me hod.
The ules de ine he isk pa e n o subse s o indi iduals by no only conside ing he in e ac ions
be ween he isk ac o s bu also hei anges. We compa ed he ule-based analysis esul s wi h he
esul s om a logis ic eg ession model in The En i onmen al De e minan s o Diabe es in he Young
(TEDDY) s udy. Bo h me hods de ec ed a simila sui e o isk ac o s, bu he ule-based analysis was
supe io a de ec ing mul iple in e ac ions be ween he isk ac o s ha cha ac e ize he subg oups.
A u he in es iga ion o he pa icula cha ac e is ics o each subg oup may de ec he special heal h
needs o he subg oup and lead o ailo ed in e en ions.
Unde s anding he ac o s associa ed wi h he isk o indi iduals wi hd awing om a s udy is an impo an i s
s ep owa ds iden i ying he e en ual heal h needs o di e en indi iduals wi hin a popula ion1. This lays he
ounda ion o de elop and deli e app op ia e esou ces o he igh a ge s, called “ ailo ed heal h in e en ions”.
E idence sugges s ha indi iduals p e e ailo ed ca e o a s anda dized ca e ha is designa ed o he a e age
popula ion2–5. The e o e, heal h p o essionals need o iden i y he subg oups o indi iduals cha ac e ized by di -
e en pa e ns o isk ac o s. Howe e , a he han iden i ying subg oups, adi ional in e en ion s udies o en
ocus on iden i ica ion o isk ac o s ha a e associa ed wi h he ou come o in e es o he popula ion as a
whole1,6,7. One commonly adop ed app oach is o use logis ic eg ession o iden i y ac o s associa ed wi h s udy
wi hd awal8–10. Howe e , his app oach only models he a e age e ec s o he isk ac o s. Consequen ly, i is
likely ha he in e en ions de eloped om eg ession models will be gea ed owa d he a e age membe o he
popula ion, wi h less conside a ion o he special needs o di e en subg oups11.
The aim o he p esen s udy is o illus a e he use o he ule-based analysis12–14 as an explo a o y echnique in
an epidemiologic con ex . The ule-based analysis12–14 is pa icula ly use ul o iden i ying he subg oups embed-
ded in a da ase —whose membe s sha e simila isk pa e ns— ha in luence he ou come o in e es . A ule
1Depa men o Indus ial and Managemen Sys ems Enginee ing, Uni e si y o Sou h Flo ida, Tampa, Flo ida, USA.
2Depa men o Beha io al Sciences and Social Medicine, College o Medicine, Flo ida S a e Uni e si y, Tallahassee,
Flo ida, USA. 3Depa men o Elec ical and Compu e Enginee ing, Texas A&M Uni e si y, College S a ion, Texas,
USA. 4Heal h In o ma ics Ins i u e, Uni e si y o Sou h Flo ida, Tampa, Flo ida, USA. 5Depa men o Indus ial &
Sys ems Enginee ing, Uni e si y o Washing on, Sea le, Washing on, USA. †A comp ehensi e lis o conso ium
membe s appea s a he end o he pape . Co espondence and eques s o ma e ials should be add essed o S.H.
(email: [email p o ec ed])
ecei ed: 07 Ap il 2016
Accep ed: 11 July 2016
Published: 26 Augus 2016
OPEN
www.na u e.com/scien i ic epo s/
2
Scien i ic RepoR s | 6:30828 | DOI: 10.1038/s ep30828
desc ibes he ange o alues on one o mo e isk ac o s ha a e associa ed wi h ei he an inc ease o dec ease
in isk o wi hd awal in a subse o indi iduals. Thus, ules p o ide a na u al seman ics o de ine he isk pa e n
o subse s o indi iduals while each ule may indica e a speci ic unme heal h need o wa ning signal o s udy
wi hd awal. By iden i ying he unknown ules om obse a ional s udies, a comp ehensi e se o isk-p edic i e
ules can be conside ed as a se o senso s, p o iding us pe sonalized isk es ima ion by looking in o he isk
pa e ns endo sed by each indi idual.
Speci ically, we used a ecen ly de eloped ule-disco e y algo i hm o he ule-based analysis, he RuleFi
me hod14, which is one example om a huge a ay o ule-based me hods ha a e p omising o epidemiologic
esea ch. The RuleFi me hod has an ad an age o e logis ic eg ession because i elies on a nonpa ame ic
model wi h ewe modeling assump ions, andom o es 13, which is capable o iden i ying he isk p edic i e
ules. The e is no need o explici ly include co a ia e in e ac ions o ans o ma ions in o he model because o
he ecu si e spli ing s uc u e used in gene a ing he andom o es . Also, he ule-based analysis pe mi s an
indi idual’s isk o be p edic ed on he basis o only one, o a mos a ew, isk ac o s, whe eas sco es de i ed om
eg ession models equi e ha all co a ia es be a ailable.
We demons a e he ule-based analysis using da a om a la ge mul ina ional epidemiological na u al his o y
s udy o ype 1 diabe es melli us (T1DM), he En i onmen al De e minan s o Diabe es in he Young (TEDDY)
s udy15. Speci ically, we use he ule-based analysis o p edic ing s udy wi hd awal du ing he i s yea o he
TEDDY s udy, by e ec i ely in eg a ing he psychosocial, demog aphic, and beha io al isk ac o s collec ed a
s udy incep ion. We compa e he ule-based analysis wi h a p e ious analysis ha was conduc ed on he same
da a10. The p e ious analysis used adi ional logis ic eg ession me hods o iden i y ac o s collec ed a s udy
incep ion ha we e s ongly associa ed wi h s udy wi hd awal du ing he i s yea o TEDDY10. Howe e , he
way hese ac o s in e ac wi h each o he and he way hese in e ac ions migh de ine subg oups in he s udy
popula ion wi h di e en isk le els emain unknown. The e o e, we es ed he hypo hesis ha he ule-based
analysis can iden i y he isk-p edic i e ules use ul o s a i ying he s udy popula ion in o di e en subg oups
wi h di e en isk le els o s udy wi hd awal in he i s yea o TEDDY. The p e ious analysis10 p o ided us an
oppo uni y o c i ically e alua ing he po en ial added alue o a ule-based analysis o e ha p o ided by adi-
ional logis ic eg ession me hods. Also, we conside ed how he ule-based me hod could lead o mo e in o med
in e en ion s a egies o p io i iza ion o he in e en ion alloca ion o he s udy pa icipan s. By conduc ing his
compa ison, we also hoped o iden i y some p ac ical guidelines o when we should use ule-based me hods and
when eg essions model would be mo e p e e able, en iching he analy ic oolbox o oday’s epidemiologis s o
add ess he eme ging da a challenges.
Ma e ials and Me hods
The TEDDY s udy. TEDDY is a na u al his o y s udy ha seeks o iden i y he en i onmen al igge s o
au oimmuni y and T1DM onse in gene ically a - isk child en iden i ied a h ee cen e s in he Uni ed S a es
(Colo ado, Washing on, and Geo gia/Flo ida) and h ee cen e s in Eu ope (Finland, Ge many, and Sweden).
In an s om he gene al popula ion wi h no immedia e amily his o y o T1DM, as well as in an s who ha e a
i s deg ee ela i e wi h T1DM, a e sc eened o gene ic isk a bi h using human leukocy e an igen geno yping.
Pa en s wi h in an s a inc eased gene ic isk o T1DM a e in i ed o pa icipa e in TEDDY. Pa en s a e ully
in o med o he child’s inc eased gene ic isk and he p o ocol equi emen s o he TEDDY s udy, including he
equi emen ha eligible in an s mus join TEDDY be o e he in an is 4.5 mon hs o age. The TEDDY p o ocol is
demanding wi h s udy isi s o blood d aws and o he da a and sample collec ion scheduled e e y h ee mon hs
du ing he i s ou yea s o he child’s li e and biannually he ea e . Pa en s a e also asked o keep de ailed
eco ds o he child’s die , illnesses, li e s esses and o he en i onmen al exposu es. TEDDY ob ains w i en con-
sen om he pa en s sho ly a e child’s bi h o ob aining gene ic and o he samples om he in an and also
pa en s. De ailed s udy design and me hods ha e been p e iously published15. The s udy me hods ha e been ca -
ied ou in acco dance wi h he app o ed guidelines by local Ins i u ional Re iew o E hics Boa ds and moni o ed
by an Ex e nal E alua ion Commi ee o med by he Na ional Ins i u es o Heal h. The expe imen al p o ocols o
he s udy we e app o ed by he Na ional Ins i u e o Heal h.
S udy sample. This analysis ocused on wo g oups o amilies om he gene al popula ion used in he
p e ious logis ic eg ession s udy10: 2,994 amilies who had been ac i e in TEDDY o ≥ 1 yea and 763 amilies
who wi hd ew om TEDDY du ing he i s yea . Bo h he p io and cu en analyses we e limi ed o gene al
popula ion amilies because s udy wi hd awal among he i s deg ee ela i es popula ion was a e.
S udy a iables. S udy a iables we e selec ed om da a collec ed on he sc eening o m a he ime o he
child’s bi h and om in e iew and ques ionnai e da a collec ed a he baby’s i s TEDDY isi . These a iables
included: demog aphic cha ac e is ics—TEDDY coun y (Finland, Ge many, Sweden, Uni ed S a es); mo he ’s
age (in yea s); child’s gende ; ma e nal heal h du ing p egnancy‒ numbe o illnesses, ges a ional diabe es o ype
2 diabe es (yes/no); mo he ’s li es yle beha io s du ing p egnancy—smoked a any ime du ing p egnancy (yes/
no), alcohol consump ion (no alcohol, 1–2 imes pe mon h, ≥ 3 imes pe mon h du ing each imes e ), employ-
men s a us (wo ked du ing all 3 imes e s/did no wo k a all o educed wo k hou s); baby’s heal h s a us‒ bi h
complica ions (yes/no), heal h p oblems since bi h (yes/no), hospi aliza ions a e bi h (yes/no); numbe o
s ess ul li e e en s du ing and a e p egnancy; mo he ’s emo ional s a us including wo y and sadness du ing
p egnancy ( a ed on 5 poin scales), anxie y abou he child’s isk o de eloping diabe es measu ed by a six-i em
scale adap ed om he S a e componen o he S a e-T ai Anxie y In en o y2–4; he accu acy o he mo he ’s pe -
cep ion o he child’s isk o de eloping diabe es (accu a e: indica ing he child’s T1DM isk was highe o much
highe han o he child en’s T1DM isk; inaccu a e: indica ing he child’s T1DM isk was he same, somewha
www.na u e.com/scien i ic epo s/
3
Scien i ic RepoR s | 6:30828 | DOI: 10.1038/s ep30828
lowe o much lowe han o he child en’s T1DM isk); and whe he he child’s a he comple ed he ini ial s udy
ques ionnai e (yes/no).
P e ious logis ic eg ession esul s. Mul iple logis ic eg ession models we e used o iden i y signi ican
p edic o s o ea ly wi hd awal om TEDDY. Va iables we e en e ed in blocks in he ollowing o de : demo-
g aphic a iables (coun y o esidence, child’s gende , mo he ’s age); p egnancy/bi h a iables (ma e nal diabe-
es, illness in mo he o child, bi h complica ions, ma e nal smoking; ma e nal d inking; ma e nal employmen
ou side he home, ma e nal wo y o sadness du ing p egnancy, numbe o s ess ul li e e en s occu ing du ing
p egnancy o a e he child’s bi h); a he ’s pa icipa ion in TEDDY de ined by a he ’s comple ion o a b ie
ques ionnai e; and mo he ’s eac ions o he baby’s inc eased T1DM isk (anxie y and accu acy o mo he ’s pe -
cep ion o he child’s T1DM isk). Nine pe cen o he s udy sample (N = 326) had missing da a on one o mo e
a iables. As expec ed, hose subjec s who had di icul y in complying wi h all da a collec ion (35%) we e mo e
likely o wi hd aw han hose wi h high da a collec ion compliance (19%). While i is unknown wha is he unde -
lying mechanism ha could explain his associa ion, we suspec ha his could indica e ha he pe cen age o
missing da a is a good indica o ha sugges s a need o TEDDY s udy o be e communica e wi h pa icipan
amilies and emo e any possible di icul ies o hem o pa icipa e in he s udy. The analysis was i s comple ed
o hose wi h no missing da a and hen e un o he ull sample using mul iple impu a ion me hods o gene a e
app op ia e pa ame e es ima es o missing da a using he P oc MI and P oc MIANALYZE p ocedu es a ailable
om SAS 9.15. Table1 p o ides he esul s o he inal logis ic eg ession model o he sample o 3,431 TEDDY
pa icipan s wi h no missing da a. The model was highly signi ican (Chi-Squa e = 264.87 (12), p < 0001) and
accu a ely placed 81.6% o he sample in o hei espec i e g oup (Ac i es e sus Wi hd awals). The da a in
Table1 also p o ides he inal logis ic eg ession model o he o al sample, wi h mul iple impu a ion me hods
used o eplace missing da a. Because he ea ly wi hd awal a e was highe among pa icipan s wi h missing da a,
we added a a iable o he impu ed model, > 1 missing da a poin (yes/no). The p esence o > 1 missing da a
poin s p edic ed ea ly d op-ou o e and abo e all o he a iables in he model. The desc ip i e in o ma ion o
each o he signi ican p edic o s is p o ided in Table2.
S a is ical me hods. Basic idea o he RuleFi me hod. We use RuleFi 14 o disco e he hidden ules ha
may be p edic i e o he isk o ea ly wi hd awal in subse s o TEDDY indi iduals. A ule consis s o se e al in e -
ac ing isk ac o s and hei anges. We a e in e es ed in he ules by which he subjec s can be s a i ied by dis-
inc isk le els. Fo example, a ule consis ing o S a e Anxie y In en o y Sco e > 45 and Dad Pa icipa ion = NO
P edic o a iable
Sample wi h No Missing Da a (N= 3431)
Sample wi h missing da a
impu ed (N = 3757)
Es ima e SE P- alue OR
95% Con idence
In e al βSE P- alue
In e cep 1.126 0.424 0.008 0.982 0.400 0.014
Coun y
Uni ed S a es e e
Finland − 0.420 0.130 0.001 0.657 0.509 0.848 − 0.431 0.123 0.0004
Ge many 0.278 0.222 0.211 1.321 0.854 2.042 0.154 0.218 0.481
Sweden − 0.342 0.110 0.002 0.711 0.572 0.882 − 0.346 0.104 0.002
Child sex emale No e
Yes 0.160 0.092 0.081 2.316 1.840 2.915 0.217 0.086 0.012
Ma e nal age (yea s) − 0.058 0.009 < 0.0001 0.944 0.927 0.961 − 0.053 0.009 < 0.0001
Ma e nal Li es yle Beha io s du ing P egnancy
Smoked No e e
Yes 0.841 0.117 < 0.0001 2.318 1.841 2.918 0.803 0.117 < 0.0001
Alcohol consump ion
in las imes e
None e
1–2 imes/mon h − 0.343 0.148 0.020 0.709 0.531 0.948 − 0.280 0.140 0.045
> 2 imes/mon h − 0.424 0.319 0.183 0.654 0.350 1.222 − 0.401 0.299 0.180
Wo ked all imes e s No e e
Yes − 0.396 0.095 < 0.0001 0.673 0.559 0.811 − 0.364 0.090 < 0.0001
Dad pa icipa ion No e e
Yes − 0.569 0.162 0.0005 0.566 0.412 0.778 − 0.608 0.146 < 0.0001
Risk pe cep ion Unde es ima e e e
Accu a e − 1.257 0.375 0.0008 0.284 0.137 0.593 − 1.032 0.354 0.004
S a e Anxie y In en o y sco e 0.001 0.006 0.835 1.001 0.989 1.014 0.001 0.006 0.825
S a e Anxie y In en o y sco e x isk
pe cep ion 0.023 0.009 0.011 1.023 1.005 1.041 0.018 0.009 0.039
> 1 missing da a poin s 1.321 0.464 0.007
Table 1. P e ious logis ic eg ession esul s o he sample wi h no missing da a and he o al sample wi h
missing da a impu ed: Va iables associa ed wi h s udy wi hd awal in he i s yea o TEDDY. (Rep in ed
om Johnson, S. B. e al.10 wi h pe mission om John Wiley and Sons Inc).
www.na u e.com/scien i ic epo s/
4
Scien i ic RepoR s | 6:30828 | DOI: 10.1038/s ep30828
would be use ul i he subjec s who can be cha ac e ized by his ule ha e a highe isk o ea ly wi hd awal.
RuleFi is a compu a ional algo i hm ha can scale up o high-dimensional applica ions (e.g., wi h a la ge num-
be o a iables) o ule disco e y, which is capable o exhaus i ely sea ching o po en ial ules on a la ge num-
be o candida e isk ac o s. I has wo phases, he “ ule gene a ion phase” and “ ule p uning phase”.
Rule gene a ion. A his s age, andom o es 13 is used o exhaus i ely sea ch o candida e ules o e he po en-
ial isk ac o s. Random o es is a high-dimensional ule disco e y app oach ha ex ends adi ional decision
ee models12. Speci ically, a andom o es es ima es a numbe o ees, wi h each ee being es ima ed on a ela-
i ely homogenous subpopula ion gene a ed by boo s apping he o iginal da ase . Since each ee employs a se
o ules o cha ac e ize a subpopula ion, he andom o es is ac ually a comp ehensi e collec ion o ules ha a e
able o cha ac e ize he whole da ase .
Cha ac e is ic
Ac i es
(n = 2994)
Wi hd awals
(n = 763)
To al Sample
(n = 3757)
Coun y N (%) N (%) N
Finland 747(84%) 140(16%) 887
Ge many 106(75%) 36(25%) 142
Sweden 1052(82%) 231(18%) 1283
Uni ed S a es 1089(75%) 356(25%) 1445
Child sex N (%) N (%) N
Male 1538 (81%) 352 (19%) 1890
Female 1456 (78%) 411 (22%) 1867
Ma e nal age (yea s) M (SD) M (SD) M (SD)
30.8 (5.0) 28.5 (5.7) 30.4(5.2)
Ma e nal Li es yle Beha io s Du ing P egnancy
Smoking N (%) N (%) N
Smoked 296(63%) 171(37%) 467
Did no smoke 2602(84%) 510(16%) 3112
Da a missing 96(54%) 82(46%) 178
Alcohol consump ion a 3 d imes e N (%) N (%) N
Alcohol 1-2 imes pe mon h 474(87%) 72(13%) 546
Alcohol ≥ 3 ime pe mon h 105(89%) 13(11%) 118
No alcohol 2359(79%) 609(21%) 2968
Da a missing 56(45%) 69(55%) 125
Employmen s a us N (%) N (%) N
Wo ked all 3 imes e s 1418(85%) 251(15%) 1669
Reduced wo k, qui , o did no wo k a all 1426(77%) 417(23%) 1843
Da a missing 150(61%) 95(39%) 245
Dad Pa icipa ion in TEDDY N (%) N (%) N
Pa icipa ed 2813(82%) 624(18%) 3437
Did No Pa icipa e 181(57%) 139(43%) 320
Ma e nal Reac ions o Child’s Inc eased TIDM Risk
Risk pe cep ion N (%) N (%) N
Accu a e 1809(84%) 355(16%) 2164
Unde es ima e 1132(77%) 343(23%) 1475
Da a missing 53(45%) 65(55%) 118
S a e Anxie y In en o y sco e M (SD) M (SD) M (SD)
To al Sample 38.7(9.7) 40.8(10.6) 39.1(9.9)
Risk Pe cep ion: Accu a e 38.8(10.2) 41.7(10.4) 39.3(9.6)
Risk Pe cep ion: Unde es ima e 38.4(10.2) 39.9(10.8) 38.8(10.4)
N (%) N (%) N
Da a missing 46 (42%) 63 (58%) 109
Missing Da a N (%) N (%) N
≤ 1missing da a poin s 2944 (81%) 695 (19%) 3639
> 1 missing da a poin s 50 (42%) 68 (58%) 118
Table 2. Cha ac e is ics o TEDDY Ac i es and Wi hd awals. (Rep in ed om Johnson, S. B. e al.10 wi h
pe mission om John Wiley and Sons Inc).
www.na u e.com/scien i ic epo s/
5
Scien i ic RepoR s | 6:30828 | DOI: 10.1038/s ep30828
Rule p uning. As a heu is ic and exhaus i e sea ch app oach, he andom o es may p oduce a la ge numbe
o ules ha can be edundan o i ele an o p edic ing ea ly wi hd awal due o o e i ing. To add ess his,
he spa se eg ession model16,17 can be applied o selec a minimum se o isk-p edic i e ules, by using all
he po en ial ules as p edic o s and he wi hd awal s a us as he ou come. The spa se eg ession model is a
high-dimensional a iable selec ion model ha can be applied on a la ge numbe o a iables, and has been
widely used in bioin o ma ics and sys ems biology18,19.
In wha ollows, we illus a e he de ails o how he RuleFi me hod uses he h ee models, he decision ee,
andom o es , and spa se linea eg ession models, in he ule gene a ion s age and he ule p uning s age:
S age 1 o RuleFi - Rule gene a ion. Rule gene a ion is compu a ionally challenging, since he numbe o po en-
ial ules g ows exponen ially in ela ionship o he numbe o isk ac o s. Gi en such a la ge numbe o po en ial
ules, an in elligen ule gene a o is needed o na ow down he sea ch by e ec i ely de ec ing high-quali y
isk-p edic i e ules. Decision ee lea ning me hod p o ides such an in elligen ule gene a o . A decision ee
is a echnique o segmen ing he popula ion in o di e en subg oups using a se o ules. Fo example, we use
he decision ee model o analyzing he TEDDY da ase o di ide he popula ion in o homogeneous subg oups
based on he pe cen age o s udy wi hd awals in each subg oup. The decision ee model is a nonpa ame ic
me hod ha au oma ically explo es he gi en isk ac o s and hei in e ac ions o a ee ha has high accu acy
in p edic ing s udy wi hd awal. In ou analysis, as shown in Fig.1, h ee subg oups wi h dis inc isk le els a e
iden i ied and can be cha ac e ized by ules de ined by ma e nal age, smoking s a us, numbe o missing da a,
and a geog aphical indica o o Finland. Fo example, he le mos node cha ac e izes a subg oup o subjec s, in
which all o hem ha e Ma e nal age < 27.5 and Finland = NO. The isk o s udy wi hd aw in his subg oup is 0.38.
This analysis demons a es ha he decision ee model is a powe ul ool o de ec ing he subg oups ha can be
cha ac e ized by ules. No e ha , he cu -o alue o each ac o used in Fig.1 is au oma ically de e mined by he
Recu si e Pa i ioning Algo i hm (RPA).
One limi a ion o he decision ee is ha only exclusi e ules can be iden i ied. Fo ins ance, he decision
ee in Fig.1 implies ha each pa icipan can only be cha ac e ized by one single ule, which doesn’ conside
he possibili y ha a pa icipan may ha e mul iple isk pa e ns cha ac e ized by di e en ac o s o di e en
in e ac ions be ween ac o s. As a emedy, andom o es 13 is a high-dimensional ule disco e y app oach ha
ex ends adi ional decision ee models. I es ima es a numbe o ees: in each i e a ion, we es ima e a decision
ee on a boo s apped sample o he aining se , and his p ocess i e a es un il he p e-speci ied numbe o ees
is achie ed13.
To unde s and he andom o es , i is wo h men ioning ha he essence o his i e a i e p ocedu e is o
gene a e a la ge numbe o subs an ially di e en ees, since he mo e simila he ees a e, he less ad an age
es ima ing mul iple ees has. In o de o achie e his goal, andomiza ion me hods a e used, which is he eason
o he name “ andom o es ”. Speci ically, in es ima ing each ee, he boo s ap echnique is used o gene a -
ing a di e en aining sample by andomly eweigh ing he o iginal da ase . Subsequen ly, in he es ima ion o
each ee, a subse o isk ac o s is andomly selec ed o es ima ing he ee. The e o e, as each ee is buil o a
sub-popula ion using a subse o isk ac o s, he he e ogenei y o he pa icipan s is well add essed in he andom
o es model, inc easing he likelihood o de ec ing meaning ul isk-p edic i e ules o di e en subg oups13. As
each ee can be decomposed o a numbe o ules, e.g., in Fig.1, we could ex ac a leas i e ules while each ule
co esponds o a lea node in he ee, wi h andom o es we could collec many ules.
Figu e 1. A decision ee lea ned om he TEDDY da a.

www.na u e.com/scien i ic epo s/
6
Scien i ic RepoR s | 6:30828 | DOI: 10.1038/s ep30828
S age 2 o RuleFi - Rule p uning. Rule p uning is essen ially a p ocedu e o selec ing a subse o ules ou o a
pool o q candida e ules, deno ed as R = [R1, R2, … , Rq], which a e p edic i e o he ou pu a iable Y. This p ob-
lem is pa icula ly challenging in high-dimensional se ings whe e we ha e a la ge numbe o gene a ed ules and
q is la ge. One solu ion o selec he mos c i ical ules is o adop he Leas Absolu e Sh inkage Selec ion Ope a o
(LASSO)16, which is a spa se linea eg ession model ha is capable o iden i ying a subse o ele an a iables
ou o a huge lis o candida e a iables. Speci ically, he o mula ion o LASSO is
βλβ−+.
β
minY R(1)
2
2
1
He e, he squa e e o e m,
β−YR
2
2
is used o measu e he model i . The L1-no m penal y e m ||β||1 (16),
de ined as he sum o he absolu e alues o all elemen s in β, is used o measu e he complexi y o he eg ession
model. The use -speci ied penal y pa ame e , λ, aims o achie e an op imal balance be ween he model i ness and
model complexi y – la ge λ will esul in spa se es ima e o β. I has been shown ha LASSO is consis en on
a iable selec ion bo h om heo e ical esea ch16 and empi ical s udies17–20. E icien algo i hms ha e been
de eloped o sol e he op imiza ion p oblem, such as he shoo ing algo i hm16, p oximal g adien algo i hms17,
e c. Th ough LASSO, we expec ha he ules wi h c i ical isk ac o pa e ns will be iden i ied wi h con olled
edundancy. In ou s udy, since he ou pu a iable Y, i.e., he wi hd awal s a us, is a bina y a iable, he spa se
logis ic eg ession17 is a be e choice han linea eg ession, which can be eadily implemen ed in he R package
o RuleFi 14.
In summa y, RuleFi is compu a ionally e icien since e icien algo i hms ha e been de eloped o bo h
Random Fo es and spa se linea eg ession models. RuleFi has an au oma ed c oss- alida ion p ocedu e o
uning i s pa ame e s, such as he numbe o ees, he size o he ees and he penal y pa ame e λ in LASSO,
which can be used o ob ain a se o high-quali y ules. Mo e de ails abou RuleFi can be ound in14. Figu e2 also
p o ides a schema ic desc ip ion o he Rule i algo i hm.
Resul s
Iden i ied isk-p edic i e ules. Table3 p o ides he isk-p edic i e ules iden i ied by he RuleFi algo-
i hm o he Ac i e and Wi hd awn amilies used in he p e ious logis ic eg ession analysis10. The isk ac o s
iden i ied in he isk-p edic i e ules a e he same as hose iden i ied in he p e ious logis ic eg ession analysis:
demog aphic ac o s including ma e nal age and coun y, ma e nal li es yle ac o s du ing p egnancy including
as smoking, d inking, and wo king ou side he home, psychosocial ac o s including he mo he ’s pe cep ion o
he child’s isk and he anxie y abou he child’s isk, dad pa icipa ion, and he numbe o missing da a poin s.
In addi ion, he in e ac ion be ween he s a e anxie y in en o y sco e wi h he isk pe cep ion accu acy ound
in he p e ious s udy, which u he alida ed in he ule-based analysis (see Table1 and Table3). Howe e , he
ule-based analysis was mo e powe ul a de ec ing he in e ac ions be ween he isk ac o s. In addi ion, he ule-
based app oach iden i ied he numbe o nega i e li e e en s as a isk ac o , a a iable ha was no signi ican
in he p io logis ic eg ession analysis. And he ule-based app oach ound no signi ican ole o child gende ,
which had a weak e ec in he p io analysis (see Table3). No e ha he ules shown in Table3 we e iden i ied by
LASSO om mo e han 2000 candida e ules gene a ed by andom o es .
In es iga ion o he isk le els o endo sing he isk pa e ns. We nex in es iga ed he isk le el
o endo sing each o he ules by compu ing he s udy wi hd awal a e o each subg oup ha endo sed a ule.
Figu e3 illus a es he wi hd awal a es o each o he eigh iden i ied ules as well as he o e all wi hd awal a e
Figu e 2. Flow diag am o he RuleFi algo i hm.
www.na u e.com/scien i ic epo s/
7
Scien i ic RepoR s | 6:30828 | DOI: 10.1038/s ep30828
o he whole s udy popula ion. The numbe o subjec s in each subg oup is also shown in he igu e. I is clea ha
endo sing any o he i s ou ules will boos he isk o ea ly wi hd awal d ama ically, while endo sing any o
he la e ou ules will help dec ease he isk signi ican ly. App oxima ely 10 pe cen o he s udy popula ion did
no all in o any subg oup and hei wi hd awal a es we e ela i ely high. I could be possible ha he e a e o he
impo an subg oups ha we e no de ec able wi h he a ailable measu es. I could also be possible ha o his
small g oup he RuleFi is no powe ul enough o de ec any signi ican ules, indica ing he need o mo e pow-
e ul ule me hods. Mo eo e , i is possible ha because he d opou mechanism could be e y complica ed and
in ol es many aspec s such as socioeconomic and psychological ac o s, he exis ence o his subg oup indica es
a ce ain le el o unp edic abili y o some cases.
In es iga ion o he edundancy o he ules. One impo an echnical issue in he ule-based analysis
is he con ol o edundancy o he ules. Two ules a e edundan i a pa icipan endo ses one ule, his pa ic-
ipan will endo se he o he ule. Ob iously, i is less desi able o ha e wo ules ha la gely o e lap wi h each
o he . We in es iga ed he edundancy o he 8 ules and p esen ed he esul s in Fig.4. Figu e4 can be ead in his
way: he pie g aph on ow i (co esponds o ule i) and column j (co esponds o ule j) eco ds he p opo ion o
he pa icipan s endo sing ule i who also endo se ule j. I can be seen ha , he o e all edundancy o he ules is
sligh , al hough he e a e some co ela ions be ween some ules, such as ule 1 and ule 4, ule 5 and ule 7. The
eason o a co ela ion be ween wo ules may be ha bo h ules sha e some common isk ac o s, e.g., bo h ule
1 and ule 4 in ol e ma e nal age < 27.5 in hei de ini ions.
Discussion
In his a icle, he ule-base analysis14 has been p oposed o en ich he oolbox o epidemiological in e en ion
s udies ha ha e been elying on eg ession models. We used da a om he TEDDY s udy and demons a ed ha
he ule-based analysis can e ec i ely iden i y isk-p edic i e ules om he psychosocial, demog aphic, and
beha io al isk ac o s. The 8 iden i ied ules a e ound p edic i e o ea ly wi hd awal du ing he i s yea o he
TEDDY s udy. The 8 ules in ol e di e en se s o isk ac o s, highligh ing he di e en na u e o he wi hd awal
Rule 1 ( isk inc easing ule) Rule 2 ( isk inc easing ule)
Ma e nal age < 27.5 Finland = NO Smoke du ing p egnancy = YES Accu a e isk
pe cep ion = NO S a e anxie y in en o y sco e > 45
Rule 3 ( isk inc easing ule) Rule 4 ( isk inc easing ule)
S a e anxie y in en o y sco e > 45 Dad
pa icipa ion = NO
Ma e nal age < 27.5 Accu a e isk pe cep ion = NO
Alcohol consump ion in las imes e < 2 imes pe
mon h
Rule 5 ( isk dec easing ule) Rule 6 ( isk dec easing ule)
Wo ked all imes e s = YES Smoke du ing
p egnancy = NO
Finland = NO Alcohol consump ion in las imes e > 0
Numbe o nega i e e en s < 2
Rule 7 ( isk dec easing ule) Rule 8 ( isk dec easing ule)
Smoke du ing p egnancy = NO S a e anxie y in en o y
sco e < 45 Numbe o missing da a poin s < = 1
Ma e nal age > 27.5 Smoke du ing p egnancy = NO
Numbe o missing da a poin s < = 1
Table 3. The 8 ules iden i ied by he RuleFi me hod.
Figu e 3. P opo ion o ea ly wi hd awal o he eigh ules and he o e all popula ion.
www.na u e.com/scien i ic epo s/
8
Scien i ic RepoR s | 6:30828 | DOI: 10.1038/s ep30828
isk o each o hese subg oups. No e ha hese 8 ules a e no exclusi e, gi ing he lexibili y ha an indi idual
can show mul iple isk pa e ns simul aneously.
We also compa ed he ule-based analysis wi h he p e ious analysis ha was conduc ed on he same da a10.
We ound ha bo h me hods de ec ed almos he same sui e o isk ac o s, p o iding alida ion o ou ule-base
analysis. No e ha he p e ious analysis only iden i ied he a e age e ec s o hese isk ac o s ac oss he whole
popula ion, wi hou conside ing how hese isk ac o s in e ac wi h each o he in de e mining he isk o ea ly
wi hd awal. Al hough i iden i ied he in e ac ion be ween he mo he ’s s a e anxie y in en o y sco e and he
isk pe cep ion accu acy, many o he in e ac ions emained unde ec ed. The ule-based analysis was supe io a
de ec ing mul iple in e ac ions be ween he isk ac o s, in addi ion o he in e ac ion be ween he mo he ’s s a e
anxie y in en o y sco e and he isk pe cep ion accu acy.
As each ule cha ac e izes a dis inc isk pa e n ha consis s o di e en isk ac o s, a u he in es iga ion
o he pa icula cha ac e is ics o each ule may help iden i y he special heal h needs o he subg oup whose
membe s endo se his ule, leading o ailo ed in e en ions. Fo example, as e ealed in ule 3, o mo he s who
a e highly anxious abou hei child’s T1D isk wi h a s a e anxie y in en o y sco e > 45, he lack o pa icipa ion
o he a he inc eases he isk o s udy wi hd awal. In an e o o ailo an in e en ion o his speci ic subg oup,
a s udy nu se migh be assigned o he amily ha ing his isk pa e n o enhance he psychological suppo o he
mo he and encou age he pa icipa ion o he a he . On he o he hand, he ules a e also help ul o de elop-
ing gene al-pu pose in e en ions. Fo ins ance, as smoking du ing p egnancy was impo an in mul iple ules,
in es iga ions may be conduc ed o unde s and why his beha io is ela ed o he isk o s udy wi hd awal. I
smoking du ing p egnancy was ound o be an indica o o less heal h-conscious a i udes, a ailo ed in e en ion
migh be de eloped o mo he s who smoked du ing p egnancy o inc ease hei heal h consciousness in an e o
o educe he isk o s udy wi hd awal. As ailo ed in e en ions a e de eloped and deployed, i is also impo an
o e alua e he e icacy o hese in e en ions o he subg oups sepa a ely, in o de o iden i y he bes in e en-
ion s a egy o each subg oup.
The ule-based analysis also iden i ied he nega i e li e e en s as a isk ac o o he ea ly wi hd awal, which
was no de ec ed by he logis ic eg ession model used in he p e ious s udy10. P e ious s udies ha e linked nega-
i e li e e en s wi h immune sys em unc ioning21,22 and he onse o T1DM23,24. While he mechanism unde lying
he linkage be ween he nega i e li e e en s and s udy wi hd awal emains unknown, i is easonable o expec
ha mo he s expe iencing nume ous nega i e li e s esses may no ha e he pe sonal esou ces o emain in he
s udy. Ce ainly ailo ing an in e en ion o his subg oup o indi iduals seems wa an ed.
The ule-based me hod has a numbe o ad an ages when handling complex da ase s. I can be used wi h a
mix o nominal, o dinal, coun o con inuous a iables and i can combine a mix u e o a iables—demog aphic,
biological, psychological—wi hou in e p e a ion di icul y. Also, as ules a e scale independen , da a do no need
Figu e 4. In es iga ion o he edundancy o he 8 ules. The pie g aph on ow (co esponds o ule) and
column (co esponds o ule j) eco ds he p opo ion o he pa icipan s endo sing ule i who also endo se ule.
www.na u e.com/scien i ic epo s/
9
Scien i ic RepoR s | 6:30828 | DOI: 10.1038/s ep30828
o be s anda dized. Finally, he ules will pe mi some indi iduals o be classi ied on he basis o only one, o a
mos a ew, isk ac o s, whe eas isk sco es de i ed om eg ession models equi e ha all he isk ac o s a e
a ailable.
The e a e limi a ions o he ule-based app oach o epidemiologic s udies. Fi s , i is no sui able o s udying
he o e all impac o a single independen a iable on he ou come a iable. This is because a single independ-
en a iable may play a ole in mul iple ules, which esul s in di icul y o in es iga e i s o e all e ec on he
whole popula ion. Also, domain insigh is e y impo an in he iden i ica ion o he ules using RuleFi . Due
o he au oma ic na u e o he ule-based app oach, i is emp ing o simply en e all he possible candida e a -
iables in o he p og am wi hou jus i ica ion o which independen a iables should be conside ed. I has been
ecommended in he li e a u e25 ha he p io knowledge ega ding he ela ionship be ween he independen
and dependen a iables should be inco po a ed wi h he ule-based models. One o he easons he ule-based
app oach yielded ema kably simila indings o he logis ic eg ession app oach in e ms o iden i ying isk
ac o s pe se, is ha conside able hough was pu in o a iable selec ion and measu emen . Rule-based models
should no be used o blind explo a ion o la ge da a se s and should bene i om domain expe s’ supe ision.
We ag ee ha a gene al guidance o use machine lea ning models o analyzing complex da ase such as TEDDY
da a is ha he new ool should be app op ia e o he esea ch ques ion. This is ac ually one main mo i a ion o
ou s udy. As wi h mos obse a ional s udies, a signi ican amoun o a ia ion exis s among TEDDY subjec s.
Con en ional models such as he logis ic eg ession model canno su icien ly cha ac e ize hese a ia ions, since
logis ic eg ession model essen ially aims o cha ac e ize he a e age e ec s o he isk ac o s on a homogeneous
popula ion. I is easonable o suspec ha TEDDY popula ion consis s o a mix o he e ogeneous subpopula ions
while in e ac ions be ween a iables a e essen ial o de ine and unde s and hese subpopula ions. Thus, he main
mo i a ion o his s udy is o demons a e he u ili y o he ule-based app oach o analyzing complex da ase s
such as TEDDY da a. Mo eo e , TEDDY da a exhibi s some o he signi ican challenges o con en ional models
ha he ule-based me hod can easily handle as we ha e a icula ed abo e.
Th ough his s udy, one o ou co-au ho s (who has been a pedia ic psychia is o many yea s and led he
p e ious s udy on he same da ase using logis ic eg ession model10) ound ha he ule-based me hod could
be a aluable new ool o augmen con en ional hypo hesis-d i en esea ch, pa icula ly when heo y-d i en
esea che s ha e limi ed insigh o de ailed knowledge abou he da ase o be analyzed (e.g., his is e y likely as
con empo a y epidemiologis s need o analyze da ase s wi h a di e se se o a iables ha include adi ional epi-
demiological a iables as well as gene ic a iables (such as SNP a ian s), i us exposu es, omics a iables, e c.).
Toge he wi h he ac ha he cu en s udy has demons a ed ha he ule-based app oach could iden i y isk
ac o s ha a e consis en wi h he p e ious hypo hesis-d i en esea ch, ou s udy implies ha , o hose complex
da ase s, he ule-based me hod could be used o ini ia e he analysis p ocess o iden i y unknown bu in o m-
a i e pa e ns om he da ase ha may help heo y-d i en esea che s o gene a e new hypo hesis and be e
o mula e hei s udies. To acili a e his ole, we d aw he ollowing p ac ical guidance o how o in eg a e he
ule-based analysis me hods in o he exis ing epidemiological oolbox. I he e is a s ong p emise ha mul iple
subg oups may exis in he da ase , he ule-based me hod could be a e y use ul app oach. On he o he hand,
subg oups may a y om da ase o da ase , and he ules (and he isk ac o s in ol ed in hese subg oups) iden-
i ied by he ule-based me hod may a y om da ase o da ase as well. I is impo an o unde s and ha he
ule-based me hod is a cus omized me hod ha is ailo ed o analyzing an indi idual da ase , so whe he o no
he esul s iden i ied om one da ase could be gene alized o ano he da ase depends on he subg oup s uc u e
o he new da ase . While lexibili y o an analy ic me hod usually comes wi h isk o o e i ing, a cus omized
me hod also needs cus omized expe ise o solid domain knowledge o he da ase . Finally, ule-based me hods
can be conside ed as oppo unis ic me hods ha aim o disco e posi i e pa e ns, bu he esul s iden i ied by
ule-based me hods a e no necessa y exclusi e. Fo example, i is possible ha he e a e mo e ules besides he
eigh ules iden i ied om TEDDY coho by he RuleFi .
In summa y, we belie e ha he ule-based app oach will be use ul in many epidemiologic s udies, pa icu-
la ly wi h he e ogeneous popula ions consis ing o subg oups o indi iduals. The dis inc isk ac o s ha de ine
each subg oup could also e lec a di e en mechanism o wi hd awing om he s udy, leading o de elopmen
o di e en in e en ion s a egies. Besides he u ili y in designing ailo ed in e en ion, i can also help wi h he
p io i iza ion o he in e en ion a ge s, e.g., we could choose elimina e a pa icula ly high- isk subg oup a he
beginning o a clinical s udy. No e ha he RuleFi algo i hm in oduced he e is one example om a huge a ay
o he ule-based me hods ha a e p omising o epidemiologic esea ch in gene al. How o p ope ly adop hem
o add essing he inc easing analy ic challenges in epidemiologic s udies will be an impo an u u e esea ch
opic. Also, we will in es iga e how o build p edic i e models based on he disco e ed ules, and u he alida e
i s p edic i e pe o mance on ano he alida ion da ase ha is being collec ed a TEDDY s udy.
Re e ences
1. G eenland, S., Gago-Dominguez, M. & Cas elao, J. E. The alue o isk- ac o (“black-box”) epidemiology. Epidemiology 15, 529–535
(2004).
2. E ans, L. K. Knowing he pa ien : he ou e o indi idualized ca e. J. Ge on ol. Nu s. 22, 15–19; quiz 52 (1996).
3. Radwin, L. & Als e , K. Indi idualized nu sing ca e: an empi ically gene a ed de ini ion. In . Nu s. Re . 49, 54–63 (2002).
4. Ryan, P. & Lau e , D. R. The e icacy o ailo ed in e en ions. Jou nal o Nu sing Schola ship 34, 331–337 (2002).
5. Whi emo e, R. Consequences o No ” Knowing he Pa ien ”. Clinical Nu se Specialis 14, 75–81 (2000).
6. Susse , M. Does isk ac o epidemiology pu epidemiology a isk? Pee ing in o he u u e. J. Epidemiol. Communi y Heal h 52,
608–611 (1998).
7. Sk abanek, P. Risk ac o epidemiology: Science o non-science. Social A ai s Uni , Heal h, Li es yle and En i onmen : Coun e ing
he Panic, London: Social A ai s Uni (1991).
8. Seigel, D. G. & G eenhouse, S. W. Mul iple ela i e isk unc ions in case-con ol s udies. Am. J. Epidemiol. 97, 324–331 (1973).