h p://www.aimsp ess.com/jou nal/mbe
MBE, 19(7): 6435–6454.
DOI: 10.3934/mbe.2022302
Recei ed: 03 Ma ch 2022
Re ised: 21 Ma ch 2022
Accep ed: 28 Ma ch 2022
Published: 24 Ap il 2022
Resea ch a icle
A me hod o de e mining g oups in nonpa ame ic eg ession cu es:
Applica ion o p e on al co ex neu al ac i i y analysis
Ja ie Roca-Pa di˜
nas1, Celes ino O d´
o˜
nez2,*and Lu´
ıs Mei a Machado3
1Depa men o S a is ics and Ope a ional Resea ch, Vigo Uni e si y, Vigo 36310, Spain
2Depa men o Mining Exploi a ion and P ospec ing, Geoma ics and Compu e G aphics Lab,
O iedo Uni e si y, Mie es 33600, Spain
3Cen e o Ma hema ics, Minho Uni e si y, B aga 4704-553, Po ugal
*Co espondence: Email: o donezceles ino@unio i.es; Tel: +34985458027.
Abs ac : Gene alized addi i e models p o ide a lexible and easily-in e p e able me hod o unco -
e ing a nonlinea ela ionship be ween esponse and co a ia es. In many si ua ions, he e ec o a
con inuous co a ia e on he esponse a ies ac oss g oups de ined by he le els o a ca ego ical a i-
able. When con on ed wi h a conside able numbe o g oups de ined by he le els o he ca ego ical
a iable and a ac o -by-cu e in e ac ion is de ec ed in he model, i hen becomes impo an o com-
pa e hese eg ession cu es. When he null hypo hesis o equali y o cu es is ejec ed, leading o he
clea conclusion ha a leas one cu e is di e en , we may assume ha indi iduals can be g ouped in o
a numbe o classes whose membe s all sha e he same eg ession unc ion. We p opose a me hod ha
allows de e mining such g oups wi h an au oma ic selec ion o hei numbe by means o boo s ap-
ping. The alidi y and beha io o he p oposed me hod we e e alua ed h ough simula ion s udies.
The applicabili y o he p oposed me hod is illus a ed using eal da a om an expe imen al s udy in
neu ology.
Keywo ds: clus e ing o eg ession cu es; ac o -by-cu e in e ac ion; gene alized addi i e model;
mul iple eg ession cu es; nonlinea eg ession; numbe o g oups
1. In oduc ion
One o he main goals o s a is ical modeling is o unde s and he e ec o some explana o y a i-
ables, Xi,i=1, ..., p, on a dependen a iable, Y, also known as he esponse. This ype o dependence
is o en modeled using a gene alized linea eg ession model (GLM) [1] ha imposes a linea ela ion-
ship be ween explana o y and dependen a iables wi hou speci ying in ad ance he unc ion ha links
6436
hem. Gene alized addi i e models (GAMs) [2, 3] o e an ex ension o he GLM h ough he inco -
po a ion o nonlinea o ms o he explana o y a iables. GAM has been widely used as an e ec i e
echnique o conduc ing nonlinea eg ession analysis in a ious ields such as economics, inance, he
en i onmen , medicine, and biology. In a wide ange o hese applica ions, p oblems can be obse ed
which i migh be use ul o compa e wo o mo e eg ession cu es. This o en occu s when i is nec-
essa y o check i he e ec o a con inuous co a ia e on he esponse a ies ac oss g oups de ined by
le els o a ca ego ical a iable. Tes ing he hypo hesis o equali y o he eg ession unc ions is a opic
o s a is ical in e ence ha has been widely esea ched in he li e a u e. One impo an e iew on his
opic may be ound in [4] (see hei Sec ion 7). Examples o me hods o clus e ing eg ession cu es
using L2dis ance and o he me ics a e in [5–7]. Fo nonpa ame ic models, mos app oaches ely
on smoo hing echniques, such as splines o Nada aya-Wa son [8], in he cons uc ion o he nonpa a-
me ic es ima o s o eg ession unc ions. The gene al idea is o use hese nonpa ame ic es ima o s
di ec ly, ei he by con as ing all indi idual es ima o s wi h he pooled es ima o , o by pe o ming
pai wise compa isons. O he au ho s conside ed he use o empi ical p ocesses o a oid he necessi y
o selec ing a smoo hing pa ame e equi ed in he cons uc ion o he nonpa ame ic es ima o s o
eg ession unc ions [9–14]. The idea behind hese es s is o use he dis ibu ion o he esiduals and
o compa e hem ia Kolmogo o -Smi no o C am´
e - on Mises ype s a is ics. ANOVA- ype es
s a is ics ha e also been used o compa e eg ession unc ions (see o example [15] and [16]). Re -
e ence [17] illus a es how he SiZe explo a o y ool is capable o compa ing mul iple cu es based
on he esiduals. In [18] a ke nel-based nonpa ame ic app oach is conside ed while [19] p oposed a
es ing p ocedu e o single index models.
A second, hough ela ed ques ion is how o de e mine g oups among a se ies o eg ession cu es
when he null hypo hesis o equali y o he eg ession unc ions is ejec ed. Though he a o emen ioned
me hods can be used o compa e eg ession cu es, o he bes o ou knowledge, hose me hods canno
be used o de e mine g oups among a se ies o eg ession cu es, o example, among g oups de ined
by he le els o a ca ego ical a iable. Some o hem a e no ecommended when con on ed wi h a
conside able numbe o cu es. I he null hypo hesis o equali y o cu es is ejec ed, hen his leads
o he clea conclusion ha a leas one eg ession cu e is di e en . Howe e , hese me hods canno be
used o asce ain whe he indi iduals can be g ouped in o a educed numbe o classes whose membe s
all sha e he same eg ession unc ion o i all he eg ession cu es a e di e en om each o he . One
na¨
ı e app oach would be o pe o m pai wise compa isons. Howe e , his app oach would lead o a
la ge numbe o compa isons (e.g., 7 g oups would lead o 21 pai wise compa isons). This could be
done bu wi hou he possibili y o de e mining g oups wi h simila eg ession cu es.
A s a is ical p ocedu e o es ima e he unknown g oup s uc u e was p oposed in [20] and la e
in [21]. The i s pape o hese au ho s was abou inancial ime se ies whe eas he second deals
wi h en i onmen al s a is ics. Mo e ecen ly, he compa ison o su i al cu es was add essed [22].
Simila ly, in his pape , we p opose an app oach ha allows de e mining g oups ha sha e he same
eg ession unc ion wi h an au oma ic selec ion o hei numbe . The p oposed me hod can be used,
o ins ance, o es ablish g oups wi h simila mono onic ends in he eg ession cu es (e.g., o e he
le els o ca ego ical a iables). The p oposed me hodology will be shown in he amewo k o he
gene alized addi i e model wi h a bina y ou come and a logi link unc ion. The gene aliza ion o he
p oposed me hods o o he link unc ions is possible.
The emainde o he pape is o ganized as ollows: The ollowing sec ion p o ides he no a ion
Ma hema ical Biosciences and Enginee ing Volume 19, Issue 7, 6435–6454.
6437
and he me hodological backg ound. Then, he pe o mance o he p oposed me hods is in es iga ed
h ough simula ions, and hei usage is illus a ed h ough he analysis o a eal da a se om neu onal
ac i i y in he p e on al co ex o monkeys du ing a disc imina ion ask. Finally, he las sec ion
con ains a discussion and he main conclusions o he wo k.
2. Ma e ials and me hods
2.1. Ma hema ical model
Le Ydeno e he bina y (0/1) a ge esponse and X=(X1,...,Xp) a se o pcon inuous co a ia es.
In his eg ession con ex , he logis ic gene alized addi i e model exp esses he condi ional p obabili y
P(X)=P(Y=1|X) as
log P(X)
1−P(X)=α+
p
X
j=1
j(Xj) (2.1)
whe e αis a cons an and each pa ial unc ion j ep esen s he e ec o he co a ia e Xj. These
unknown smoo h pa ial unc ions a e modeled om he da a wi hou speci ying in ad ance any pa a-
me ic s uc u e. The e o e, he GAM in (2.1) combines lexibili y wi h in e p e abili y, in which each
o he addi i e componen s desc ibes he in luence o each co a ia e sepa a ely.
In p ac ice, he e ec o a con inuous co a ia e Xjcan a y ac oss he le els 1,...,Ko a ca ego ical
ac o F. Fo simplici y o no a ion, he e we conside ha only he e ec o he las co a ia e, Xp,
depends on he le els o F, so he pu e GAM in (2.1) can inco po a e his ype o ac o -by-cu e
ela ionship as ollows:
logi (F,X)=log P(Y=1|F,X)
1−P(Y=1|F,X)=α+
p−1
X
j=1
j(Xj)+

















g1(Xp) i F=1
g2(Xp) i F=2
.
.
..
.
..
.
.
gK(Xp) i F=K
(2.2)
whe e g1,...,gK ep esen he speci ic e ec o Xp o each o he possible Kle els es ablished by he
ac o F.
2.2. Es ima ing pa i ions
The s udy o he pa ial unc ions g1,...,gKcan be use ul in he compa ison o wo o mo e g oups,
which is an impo an p oblem associa ed wi h s a is ical in e ence. In e es cen e s on he null hypo h-
esis H0:g1=. . . =gK, namely, ha he e ec o Xpdoes no depend on he le els o he ac o F.
When he equali y o he Kcu es is ejec ed, leading o he clea conclusion ha a leas one cu e
is di e en , i can be in e es ing o asce ain whe he g oups can be pe o med o , by con as , ha all
hese cu es a e di e en om each o he . Mo e clea ly, i H0is ejec ed, i could be in e es ing o de-
e mine: a) i any pai o cu es a e di e en om each o he , ha is gi,gj, o all i,j∈ {1,...,K},
Ma hema ical Biosciences and Enginee ing Volume 19, Issue 7, 6435–6454.
6438
o b) i he indices in {1,...,K}can be g ouped in a pa i ion o J(J<K) g oups I=(I1,...,IJ) wi h
J<K, so ha gi=gj o each i,j∈Ik, o some k. No e ha Imus be a pa i ion o {1,...,K}and,
he e o e, mus sa is y I1∪. . . ∪IJ={1,...,K}and Ii∩Ij=∅ o all i,j∈I.
In some si ua ions, he possible pa i ion I=(I1,...,IJ), can be es ablished in ad ance, and he
in e es is o alida e such pa i ion by es ing he null model
logi (F,X)=α+
p−1
X
j=1
j(Xj)+
















g1(Xp) i F∈I1
g2(Xp) i F∈I2
.
.
..
.
..
.
.
gK(Xp) i F∈IJ
(2.3)
e sus he gene al model gi en in (2.2). Howe e , in gene al, he pa i ions a e unknown and will ha e
o be es ima ed om he da a.
Be o e explaining he p ocedu e o de e mining he g oups, le us in oduce some no a ion. Gi en
a pa i ion I=(I1, . . . IJ), we de ine a unc ion GI:{1,...,K} → {1,...,J}so GI(k)=ji k∈Ij.
Now, gi en a sample {Fi,Xi,Yi}n
i=1and using he es ima es ˆgk(k=1,...,K) o he model in (2.2), he
es ima ed pa i ion ˆ
I=(ˆ
I1, . . . ˆ
IJ) can be ob ained by minimizing he ollowing C am´
e - on Mises ype
dis ance
min
I=(I1,...,IJ)
K
X
k=1Zxˆgk(x)−ˆ
CGI(k)(x)2dx (2.4)
Al e na i ely, he es ima ed pa i ion can be ob ained using he Kolmogo o -Smi no ype dis ance
min
I=(I1,...,IJ)
K
X
k=1Zxˆgk(x)−ˆ
CGI(k)(x)dx (2.5)
In bo h cases he es ima ed cen oids o j=1,...,Ja e de ined as
ˆ
Cj(x)=PK
k=1ˆgk(x)·I{GI(k)=j}
PK
k=1I{GI(k)=j}(2.6)
The abo e minimiza ion p oblems can ha e a high compu a ional cos o la ge alues o K, because
hey equi e he e alua ion o all he di e en combina ions o he Kcu es in o Jg oups. To sol e he
quad a ic minimiza ion p oblem in (2.4) and L1-no m minimiza ion p oblem in (2.5) we p opose he
use o he k-means o k-medians algo i hms. In bo h cases, he pa ial unc ions ha e o be es ima ed
in a common g id o Xp,x•
1< . . . < x•
N, o size N, leading o a ma ix












ˆg1(x•
1) . . . ˆgK(x•
1)
.
.
..
.
.
ˆg1(x•
N) . . . ˆgK(x•
N)











N×K
This ma ix will be he inpu o bo h heu is ic me hods, k-means and k-medians, and om hese, he
es ima ed pa i ion ˆ
I=(ˆ
I1,...,ˆ
IJ) is ob ained.
Ma hema ical Biosciences and Enginee ing Volume 19, Issue 7, 6435–6454.
6439
2.3. De e mining he numbe o g oups
The p ocedu e o ob ain he es ima ion pa i ion ˆ
I=(ˆ
I1,...,ˆ
IJ) o a gi en J(J<K) has been
explained in he p e ious sec ion. This sec ion ocuses on de e mining he numbe o g oups o he
eg ession unc ions. Wi h his goal in mind, one possible app oach consis s in i ing he model in
(2.3) o each alue o j=1,...,J, and selec ing he numbe o g oups K ha minimizes he Bayesian
In o ma ion C i e ion (BIC). BIC is one o he mos widely used ools in s a is ical model selec ion. I s
popula i y is de i ed om i s compu a ional simplici y and e ec i e pe o mance in many modeling
app oaches. Ou simula ions on cu e clus e ing demons a e ha he BIC c i e ion can be conside ed
a alid op ion, pe o ming qui e sa is ac o ily choosing he co ec numbe o clus e s in simula ion
scena ios wi h a la ge numbe o cu es and complex da a se s. Ano he possibili y would be o es ,
o a gi en J, he null hypo hesis H0(J) ha a leas one pa i ion Io leng h Jexis s, so ha he model
in (2.3) is ul illed, agains he al e na i e ha o any J-pa i ion a leas a g oup Ijexis s in which
gm,gn o some m,n∈Ij. In o de o es H0(J), o a gi en sample {Fi,Xi,Yi}n
i=1, we conside he
ollowing wo s a is ics
DCM =
K
X
k=1
n
X
i=1ˆgk(Xip)−ˆg◦
Gˆ
I(k)(Xip)2(2.7)
DKS =
K
X
k=1
n
X
i=1ˆgk(Xip)−ˆg◦
Gˆ
I(k)(Xip)(2.8)
whe e he es ima ed pa i ion ˆ
I=(ˆ
I1,...,ˆ
IJ) is ob ained by sol ing he minimiza ion p oblems in (2.4)
o (2.5), and ˆg◦
j unc ions a e ob ained om model (2.3) using he same pa i ion. Al e na i ely, we
can use he de iance as an app op ia e measu e o disc epancy be ween obse ed and i ed alues and
conside he ollowing likelihood a io es
DLR =Pn
i=1De i(Yi,ˆ
Pi)−Pn
i=1De i(Yi,ˆ
P◦
i)
Pn
i=1De i(Yi,ˆ
Pi)(2.9)
which compa es he de iance o he null model (2.3) wi h he de iance o he gene al model in (2.2),
whe e ˆ
P◦
iand ˆ
Pia e he es ima es alues o he ue p obabili ies Pi=P(Yi=1|Fi,Xi) unde he
null and gene al model, espec i ely. The indi idual de iance De i(Yi,ˆ
Pi) is de ined as De i(Yi,ˆ
Pi)=
−2Yilog ˆ
Pi+(1 −Yi) log(1 −ˆ
Pi).
No e ha i H0(J) is e i ied, he alue o he es s a is ic Dshould be close o ze o. The decision
ule based on each o he h ee s a is ics, D, consis s in ejec ing he null hypo hesis i Dis g ea e han
(1 −α)-pe cen ile ob ained unde he null hypo hesis. Ne e heless, he heo y o de e mining such
pe cen iles is no closed. To app oxima e he dis ibu ions o he es s a is ics, esampling me hods
such as he boo s ap me hod can be applied [23–26]. The bina y boo s ap used in ou pape is a
pa icula case o he boo s ap echniques sugges ed by [27] and [28] o in e ence in nonpa ame ic
models wi h esponse belonging o he bina y amily. The bina y boo s ap in ol es he ollowing
s eps:
S ep 1. Using he o iginal sample da a {Fi,Xi,Yi}n
i=1, compu e he es s a is ics Das explained abo e.
Then, ob ain o i=1,...,n he es ima ed ˆ
P◦
io he ue p obabili ies Pi=P(Yi=1|Fi,Xi) ob ained
om he null model in (2.3).
Ma hema ical Biosciences and Enginee ing Volume 19, Issue 7, 6435–6454.

6440
S ep 2. Fo b=1,...,B, gene a e he boo s ap sample {Fi,Xi,Y•
i}n
i=1, wi h Y•
i∈Be noulli(ˆ
P◦
i), and
compu e he boo s ap s a is ics D•,b. No e ha he es ima ion o he pa i ion is equi ed in his S ep.
Finally, he decision ule, based on each es s a is ic D, consis s in ejec ing he null hypo hesis i
D>ˆ
D1−α, whe e ˆ
D1−αis he empi ical (1−α)-pe cen ile o he alues D•1,...,D•Bob ained p e iously.
The boo s ap-based es in oduced he e can be e y use ul o au oma ically de e mine he numbe
o g oups J. Speci ically, he ule p oposed is as ollows: he p ocedu e begins by es ing H0(1). I his
hypo hesis is ue, we decide ha J=1 (all eg ession cu es a e equal). When he p e ious hypo hesis
is ejec ed, i will be necessa y o es H0(2). I his new hypo hesis is no ejec ed, we decide ha J=2.
When H0(2) is ejec ed, i will be equi ed o es H0(3) and so on un il a ce ain H0(J) is no ejec ed.
Finally, no e ha we a e awa e ha his app oach could deal wi h he p oblem o mul iple hypo hesis
es ing whe e a se o Jp- alues co esponding o he Jnull hypo heses, H0(1),H0(2),...,H0(J), a e
gi en. E en hough se e al me hods ha e been p oposed o deal wi h his p oblem (see e.g. [29] o an
in oduc ion o his a ea), he e is s ill a challenge because he e is no in o ma ion abou he minimum
numbe o es s needed o apply hese echniques. Acco dingly, and conside ing ha ou ule inishes
wi h a low numbe o hem, we ha e no conside ed his p oblem.
2.4. Simula ion s udy
This sec ion epo s he esul s o a simula ion s udy conduc ed o e alua e he p ac ical pe o mance
o he p oposed me hodology. The esponse Ywas gene a ed unde model (2.3) wi h
log P(Y=1|F,X)
1−P(Y=1|F,X)=





















g1(X)=1.5Xi F∈I1={1,2,3}
g2(X)=2X2−3 i F∈I2={4,5}
g3(X)=2 sin(2X)−2 i F∈I3={6,7}
g4(X)=2 cos(2X) i F∈I4={8,9}
g5(X)=2 cos(2X)+aexp(0.5X) i F∈I5={10,11,12,13}
g6(X)=4−2Xi F∈I6={14,15}
(2.10)
The ca ego ical co a ia e Xwas d awn om an uni o m dis ibu ion U[−2,2], and i s ac o s, F, we e
gene a ed by aking a andom alue in {1,...,15}wi h associa ed p obabili ies p=(p1,...,p15)=
n/Pnk,k=1,...,15, whe e:
n=(n1,...,n15)=(1.0,2.0,1.0,1.5,1.0,2.0,1.5,2.0,2.0,1.0,1.0,1.0,1.5,1.5,1.0)
In his way, and as can be seen in Table 1, we ha e conside ed unequal sample sizes o each (k=
1,...,15) cu e.
We explo e he alidi y o he p oposed es s assuming ha we aim o es i he K=15 eg ession
cu es can be g ouped in i e (J=5) g oups. To s udy he size and powe o he p oposed es s,
di e en alues we e conside ed o a, anging om 0 o 2. I should be no ed ha g5(X)=g4(X)+
aexp(0.5X), so a alue o a=0 co esponds o a null hypo hesis o i e g oups, while a alue o a>0
co esponds o a null hypo hesis o six g oups.
2.5. Applica ion o neu al ac i i y in he p e on al co ex du ing a disc imina ion ask
The da a analyzed in his sec ion come om labo a o y expe imen s o he ex a-cellula single uni
ac i i y in he p e on al co ex o a monkey. These expe imen s we e conduc ed in he Labo a o y o
Ma hema ical Biosciences and Enginee ing Volume 19, Issue 7, 6435–6454.
6441
Table 1. Sample size nk o each eg ession cu e k(k=1,...,15) o di e en o al sample
sizes n=Pnk.
n n1n2n3n4n5n6n7n8n9n10 n11 n12 n13 n14 n15
300 14 28 14 21 14 28 21 28 28 14 14 14 21 21 14
500 23 47 23 35 23 47 35 47 47 23 23 23 35 35 23
1000 47 95 47 71 47 95 71 95 95 47 47 47 71 71 47
2000 95 190 95 142 95 190 142 190 190 95 95 95 142 142 95
4000 190 380 190 285 190 380 285 380 380 190 190 190 285 285 190
Neu ophysiology o he Uni e si y o Vigo (Spain). The monkey was ained o disc imina e be ween
di e en s imuli. The s imuli consis ed o s a iona y b igh line segmen s p esen ed on a moni o sc een
in on o he monkey ha changed hei o ien a ion o e ime.
A ial was ini ia ed when he monkey p essed a le e key wi h i s igh hand and hen wo s imuli
( e e ence and es ), each o 500 ms du a ion, we e p esen ed in sequence wi h a ixed in e -s imulus
in e al (ISI: 1000 ms). A he end o he second s imulus, he subjec eleased he key in a 1200 ms
ime window and p essed one o he wo swi ches (le o igh ), indica ing whe he he o ien a ion o
he second s imulus was clockwise ( igh ) o coun e -clockwise (le ) o he e e ence s imulus. While
he monkey wo ked on he ask, i s ex acellula uni ac i i y was eco ded. The monkey was ewa ded
o co ec disc imina ion.
To ga he enough da a and o accoun o he cell esponse a iabili y, he neu on was eco ded
o e a numbe o T=80 ials. Fo each o he ials, we conside he angle ( ial speci ic), TA,
co esponding o he es s imulus. The e e ence angle is 90◦. The ollowing TAs we e conside ed in
he expe imen :
•T A ∈ {78◦,102◦}: es s imuli mo e sepa a ed om he e e ence, and he e o e e y easy o
disc imina e
•T A ∈ {81◦, 99◦}: es s imuli is easy o disc imina e
•T A ∈ {84◦, 96◦}: es s imuli is di icul o disc imina e
•T A ∈ {87◦,93◦}: es s imuli closes o he e e ence, he e o e mo e di icul o disc imina e
In his expe imen , he ou come o in e es is he neu onal ac i i y in he in e al ∈[ min, max]=
[−500,3000] in ms. A each ins an and ial j=1,...,T, his ou come may hen be ep esen ed by
a empo al bina y sequence, Yj
, whe e Yj
=1 i he e is a spike in [ , +1) ms and 0 o he wise. The
explana o y a iables a e he ime when a spike was de ec ed and he in e al o ime ∆ since he las
spike. The las one is an adjus men a iable ha akes in o accoun ha he neu on migh i e mo e
easily i i has been i ed ecen ly. Acco dingly, o each ial, he da a se consis s o he ollowing
in o ma ion
{T A,( ,∆ ),Y } max
= min
We use gene alized addi i e modeling o assess whe he he associa ion be ween neu al ac i i y and
decision-making depends on he di icul y o he disc imina ion ask ela ed o he angle o he es
s imulus. We conside he ollowing GAM:
Ma hema ical Biosciences and Enginee ing Volume 19, Issue 7, 6435–6454.
6442
log P(T A, ,∆ )
1−P(T A, ,∆ )!=α+ (∆ )+





































g1( ) i T A =78◦
g2( ) i T A =102◦
g3( ) i T A =81◦
g4( ) i T A =99◦
g5( ) i T A =84◦
g6( ) i T A =96◦
g7( ) i T A =87◦
g8( ) i T A =93◦
(2.11)
whe e P(T A, ,∆ )=P(Y =1|F, ,∆ ), αis a ixed pa ame e , and gk o k=1,...,8 a e he speci ic
ime unc ions associa ed o each o he es angles conside ed in he s udy.
3. Resul s
3.1. Resul s on simula ion da a se
Type I e o a es and powe alues o he p oposed es s as a unc ion o aa e shown in Figu e 1.
They we e calcula ed om 1000 simula ion uns using B=400 boo s ap samples a each epe i ion.
We compa e he esul s o he h ee es s a is ics unde sample sizes n=1000 and 2000, a he
signi icance le els o 0.05 and 0.10. As can be seen in Figu e 1, he h ee cu es show he expec ed
beha io pa e n, wi h an inc ease in he powe as ainc eases, and an imp o emen in i as he sample
size g ows.
n=1000
a
Powe (%)
DCM
DKS
DLR
0.00 0.50 1.00 1.50 2.00
0 20 40 60 80 100
n=1000
a
Powe (%)
0.00 0.50 1.00 1.50 2.00
0 20 40 60 80 100
n=2000
a
Powe (%)
0.00 0.50 1.00 1.50 2.00
0 20 40 60 80 100
n=2000
a
Powe (%)
0.00 0.50 1.00 1.50 2.00
0 20 40 60 80 100
Figu e 1. Rejec ion p obabili ies (powe o he hypo hesis es ) o he p oposed es s a is ics
as a unc ion o a, o sample size n=1000 and n=2000 (uppe and lowe panels, espec-
i ely) a a 0.05 and 0.10 (le and igh panels, espec i ely) signi ican le els ( ed line).
Ma hema ical Biosciences and Enginee ing Volume 19, Issue 7, 6435–6454.
6443
The es s a is ic based on he likelihood a io es (labeled as DLR) is ha wi h he highes alues o
powe o n=1000, bu he di e ences be ween he h ee es s a is ics a e minimal o n=2000. The
C am´
e - on Mises– ype s a is ic es (labeled as DCM) e ealed a poo e beha io wi h lowe ejec ion
p obabili ies o a sample size o 1000. Values o ype I e o , o he h ee es s a is ics and o
di e en signi icance le els and di e en sample sizes (n=1000,2000, and 3000), a e epo ed in
Table 2. The h ee es s a is ics wo k sa is ac o ily acco ding o he ype I e o , coming qui e close o
he nominal le el ega dless o he sample size.
Table 2. Es ima ed ype I e o (in %) and nominal le el pe cen ages (1, 5, 10, 15, and 20)
o di e en sample sizes.
le el DCM DKS DLR
n=1000
1 1.4 1.3 1.0
5 5.5 5.3 5.6
10 9.7 9.8 11.5
15 14.5 15.3 16.5
20 18.7 20.0 21.1
n=2000
1 0.9 0.7 0.8
5 4.2 3.8 5.7
10 8.4 8.0 9.4
15 12.8 12.4 15.6
20 16.5 18.0 21.8
n=3000
1 0.9 0.7 1.2
5 4.2 4.0 6.5
10 8.8 7.6 11.1
15 13.0 12.1 16.2
20 16.2 17.0 22.3
In Table 3, we epo esul s ha can be used o e alua e he accu acy o he boo s ap-based al-
go i hm in oduced in Sec ion 2.3. Again, di e en alues o awe e conside ed, anging om 0 o 2.
Recall ha he alue a=0 co esponds o he null hypo hesis, which assumes ha he i een eg ession
unc ions can be classi ied in o i e g oups, while, when a,0, he numbe o g oups is six. No e ha
o selec he co ec numbe Jo g oups o eg ession unc ions, he boo s ap-based algo i hm mus
i s ejec he i s null hypo hesis, H0(1), hen ejec he second hypo hesis, H0(2), and so on un il i
accep s H0(5) i a=0, o un il H0(6) when a>0. Resul s shown in Table 3 o he h ee es s a is ics,
using a nominal le el o 5%, display he numbe o imes ha he p ocedu e selec s he numbe o
g oups J. Resul s in bold deno e he co ec classi ica ions acco ding o model (2.3), e ealing he high
accu acy o he p oposed boo s ap-based algo i hm o a=0, showing he co ec numbe o g oups
in pe cen ages qui e close o he nominal le el. As he alue o ainc eases, so does he pe cen age o
cases in which he p oposed me hod sugges s six g oups. When compa ing he h ee es s a is ics, i
can be obse ed ha all ha e a simila pe o mance wi h a small ad an age o he es s a is ic based
on he likelihood a io es (labeled as DLR).
Ma hema ical Biosciences and Enginee ing Volume 19, Issue 7, 6435–6454.
6450
han 90◦(clockwise o a ion) we e p e iously epo ed in a s udy [26].
Time(ms)
Fi ing a e (spikes/s)
−500 0 500 1000 1500 2000 2500 3000
0 5 10 15 20 25 30
Time (ms)
Fi ing a e (spikes/s)
Tes Re e ence ISI
−500 0 500 1000 1500 2000 2500 3000
10 15 20 25 30 35
Figu e 5. Rep esen a ion o neu al ac i i y o e ime. Obse ed spikes pool coun s o spikes
wi hin 10 ms in e als (le ) and he co esponding smoo hing e sion ( igh plo ).
Time (ms)
Fi ing a e (spikes/s)
1500 2000 2500 3000
10 20 30 40 50 60
78º
81º
84º
87º
93º
96º
99º
102º
Figu e 6. Fi ed eg ession cu es o he eigh di e en expe imen al condi ions.
Resul s o he boo s ap me hod using he h ee es s a is ics de ined in Sec ion 3 a e shown in
Table 7. As can be seen, he h ee es s a is ics indica e ha he e a e wo g oups. DKD, and sligh ly
less so DCM, a e close o ipping he balance in a o o h ee g oups.
We ha e also applied he BIC c i e ion o de e mine he numbe o g oups. The esul s o his clus e
solu ion also lead o he same conclusion, namely, J=2 g oups. These indings a e in ag eemen wi h
hose epo ed in ou simula ions ha indica e ha BIC is mo e conse a i e, bu capable o de ec ing
Ma hema ical Biosciences and Enginee ing Volume 19, Issue 7, 6435–6454.

6451
impo an di e ences.
Table 7. P obabili y alues o es ing he null hypo hesis H0(J). Resul s based on he
boo s ap me hod wi h di e en es s a is ics.
JDCM DKS DLR
1<0.001 <0.001 <0.001
2<0.06 <0.05 <0.29
3<0.19 <0.13 <0.53
4<0.09 <0.76 <0.48
Clus e s o a ixed numbe o g oups be ween J=2 and J=5 a e shown in Figu e 7 o acili a e he
comp ehension o he p oblem, al hough he esul s ob ained indica e ha he e a e only wo g oups.
Fi ing a e (spikes/s)
12345
78º
81º
84º
87º
93º
96º
99º
102º
2 g oups
Fi ing a e (spikes/s)
78º
81º
84º
87º
93º
96º
99º
102º
3 g oups
Time (ms)
Fi ing a e (spikes/s)
12345
1500 2000 2500 3000
78º
81º
84º
87º
93º
96º
99º
102º
4 g oups
Time (ms)
Fi ing a e (spikes/s)
1500 2000 2500 3000
78º
81º
84º
87º
93º
96º
99º
102º
5 g oups
Figu e 7. Es ima ed eg ession cu es acco ding o he g oups o which hey belong. The
cu es assigned in o wo (J =2), un il i e g oups (J =5) a e ep esen ed in each o he ou
panels. The s a is ics used o es ima e he numbe o g oups ound only wo.
4. Conclusions
In his wo k we p opose a boo s ap-based me hod o de e mine he numbe o g oups in gene alized
addi i e models when he e ec o a con inuous co a ia e on he esponse a ies ac oss g oups de ined
by le els o a ca ego ical a iable. The simula ion analysis con i ms he capaci y o he h ee p oposed
s a is ical es s o ep oduce he heo e ical alues o ype I and ype II e o s, e en in he p esence o a
Ma hema ical Biosciences and Enginee ing Volume 19, Issue 7, 6435–6454.
6452
la ge numbe o cu es. Howe e , he powe u ned ou o be highe o he es based on he likelihood
es a io. As expec ed, he pe cen age o misclassi ica ions dec eases wi h he sample size, n, being
almos null o n=4000.
The simula ed da a we e also used o compa e ou me hod wi h BIC, a well-known app oach ha has
been la gely used p e iously in clus e ing analysis o es ima e he numbe o clus e s in an a i icial
da a se . The numbe o de ec ed g oups was he same o bo h me hods, bu he simula ion s udy
showed ha BIC has a lowe s a is ical powe han he boo s ap me hod, al hough hey a e close o
la ge sample sizes. Acco dingly, BIC could be used o ob ain an ini ial es ima ion o he numbe o
clus e s ha could be uned using ou me hod.
The applica ion o he p oposed me hod o an expe imen o de e mine he ac i i y o a neu on o
a monkey subjec o isual s imuli led o he same conclusions when we compa ed ou app oach wi h
BIC. In bo h cases he neu on only eac s o wo o he eigh s imuli, co esponding o he mos e iden
di e ences wi h espec o a e e ence s a e. Rega ding he h ee s a is ics es ed, hey p o ide he same
esul , al hough he e idence o wo clus e s is s onge o he s a is ic based on he likelihood a io
es .
Finally, i can be said ha al hough he p oposed me hod was designed o de ec g oups o eg es-
sion cu es in gene alized addi i e models wi h a bina y esponse, i can be ex ended wi hou much
di icul y o de e mine g oups in models wi h ano he kind o esponse.
Acknowledgmen s
This wo k was pa ially suppo ed by p ojec 2017/00001/006/001/097: Ayudas pa a el man-
enimien o de ac i idades de in es igaci´on de ins i u os uni e si a ios de in es igaci´on y g upos de
in es igaci´on de la Uni e sidad de O iedo pa a el eje cicio 2021.
Lu´
ıs Mei a-Machado acknowledges inancial suppo om Po uguese Funds h ough FCT -
”Fundac¸˜
ao pa a a Ciˆ
encia e a Tecnologia”, wi hin he p ojec s UIDB/00013/2020, UIDP/00013/2020.
Ja ie Roca-Pa di˜
nas acknowledges inancial suppo om G an PID2020-118101GB-I00, Minis-
e io de Ciencia e Inno aci´
on (MCIN/AEI /10.13039/501100011033).
Con lic o in e es
The au ho s decla e he e is no con lic o in e es .
Re e ences
1. P. McCullagh, J. Nelde , Gene alized Linea Models, 2nd edi ion, Chapman and Hall/CRC, Boca
Ra on, 1989. h ps://doi.o g/10.1201/9780203753736
2. T. J. Has ie, R. J. Tibshi ani, Gene alized Addi i e Models, Chapman & Hall/CRC, New Yo k,
1990.
3. S. Wood, Gene alized Addi i e Models: An In oduc ion wi h R, Chapman & Hall/CRC, 2006.
h ps://doi.o g/10.1201/9781420010404
4. W. Gonz´
alez-Man eiga, R. M. C ujei as, An upda ed e iew o Goodness-o -Fi es s o eg ession
models, Tes ,22 (2013), 361–411. h ps://doi.o g/10.1007/s11749-013-0327-5
Ma hema ical Biosciences and Enginee ing Volume 19, Issue 7, 6435–6454.
6453
5. H. De e, A. Munk, Tes ing he e ocedas ici y in nonpa ame ic eg ession, J. R. S a . Soc. B,60
(1998), 693–708. h ps://doi.o g/10.1111/1467-9868.00149
6. H. De e, N. Neumeye , Nonpa ame ic analysis o co a iance, Ann. S a .,29 (2001), 1361–1400.
h ps://doi.o g/10.1214/aos/1013203458
7. L. Ga c´
ıa-Escude o, A. Go daliza, A p oposal o obus cu e clus e ing, J. Classi .,22 (2005),
185–201. h ps://doi.o g/10.1007/s00357-005-0013-8
8. E. A. Nada aya, On es ima ing eg ession, Theo y P obab. I s Appl.,9(1964), 141–142.
h ps://doi.o g/10.1137/1109020
9. M. A. Delgado, Tes ing he equali y o nonpa ame ic eg ession cu es, S a . P obab. Le .,17
(1993), 199–204. h ps://doi.o g/10.1016/0167-7152(93)90167-H
10. K. B. Kulaseke a, Compa ison o eg ession cu es using quasi- esiduals, J. Am. S a . Assoc.,90
(1995), 1085–1093. h ps://doi.o g/10.1080/01621459.1995.10476611
11. K. B. Kulaseke a, J. Wang, Smoo hing pa ame e selec ion o powe op imal-
i y in es ing o eg ession cu es, J. Am. S a . Assoc.,92 (1997), 500–511.
h ps://doi.o g/10.1080/01621459.1997.10474003
12. K. B. Kulaseke a, J. Wang, Bandwid h selec ion o powe op imali y in a es o equali y
o eg ession cu es, S a . P obab. Le .,37 (1998), 287–293. h ps://doi.o g/10.1016/S0167-
7152(97)84155-7
13. N. Neumeye , H. De e, Nonpa ame ic compa ison o eg ession cu es: An empi ical p ocess
app oach, Ann. S a .,31 (2003), 31880–31920.
14. J. C. Pa do-Fe n´
andez, I. Keilegom, W. Gonz´
alez-Man eiga, Tes ing o he equali y o k eg es-
sion cu es, S a . Sin.,17 (2007), 1115–1137.
15. S. G. Young, A. W. Bowman, Non-pa ame ic analysis o co a iance, Biome ics,51 (1995), 920–
931. h ps://doi.o g/10.2307/2532993
16. J. C. Pa do-Fe n´
andez, M. D. Jim´
enez-Game o, A. Ghouch, A non-pa ame ic ANOVA- ype
es o eg ession cu es based on cha ac e is ic unc ions. Scand. J. S a .,42 (2015), 197–213.
h ps://doi.o g/10.1111/sjos.12102
17. C. Pa k, K. Kang, Size analysis o he compa ison o eg ession cu es, Compu . S a . Da a.
Anal.,52 (2008), 3954–3970. h ps://doi.o g/10.1016/j.csda.2008.01.006
18. C. Pa k, J. Hannig, K. Kang, Nonpa ame ic compa ison o mul iple eg es-
sion cu es in scale-space, J. Compu . G aphical S a .,23 (2014), 657–677.
h ps://doi.o g/10.1080/10618600.2013.822816
19. W. Lin, K. B. Kulaseke a, Tes ing he equali y o linea single-index models, J. Mul i a . Anal.,
101 (2010), 1156–1167.
20. M. Vog , O. Lin on, Classi ica ion o non-pa ame ic eg ession unc ions in longi udinal da a
models, J. R. S a . Soc. Se . B S a . Me hodol.,79 (2017), 5–27. h ps://doi.o g/10.1111/ ssb.12155
21. M. Vog , O. Lin on, Mul iscale clus e ing o nonpa ame ic eg ession cu es, J. Econome ics,
216 (2020), 305–325.
Ma hema ical Biosciences and Enginee ing Volume 19, Issue 7, 6435–6454.
6454
22. N. M. Villanue a, M. Ses elo, L. Mei a-Machado, A me hod o de e mining g oups in mul iple
su i al cu es, S a . Med.,38 (2019), 866–877. h ps://doi.o g/10.1002/sim.8016
23. P. Hall, J. D. Ha . Boo s ap es o di e ence be ween means in nonpa ame ic eg ession, J. Am.
S a . Assoc.,85 (412), 1039–1049.
24. M. C. Rod ´
ıguez-Campos, W. Gonz´
alez-Man eiga, R. Cao, Tes ing he hypo hesis o a gene alized
linea eg ession model using nonpa ame ic eg ession es ima ion, J. S a . Plan. In e .,67 (1998),
99–122. h ps://doi.o g/10.1016/S0378-3758(97)00098-0
25. J. Roca-Pa di˜
nas, C. Cada so-Su´
a ez, V. N´
ache , C. Acu˜
na, Boo s ap-based me hods
o es ing ac o -by-cu e in e ac ions in Gene alized Addi i e Models: assessing p e-
on al co ex neu al ac i i y ela ed o decision-making, S a . Med.,25(2006), 2483–2501.
h ps://doi.o g/10.1002/sim.2415
26. C. Cada so-Su´
a ez, J. Roca-Pa di˜
nas, G. Molenbe ghs, F. Faes, V. N´
ache , S. Ojeda, e al., Flexible
modelling o neu on i ing a es ac oss di e en expe imen al condi ions. An applica ion o neu al
ac i i y in he p e on al co ex du ing a disc imina ion ask, J. R. S a . Soc. Se . C,55 (2006),
431–447.
27. S. Spe lich, D. Tjøs heim, L.Yang, Nonpa ame ic es ima ion and es ing o in e ac ion in addi i e
models, Econom. Theo y,18 (2002), 197–251. h ps://doi.o g/10.1017/S0266466602182016
28. L. Yang, S. Spe lich, W. H¨
a dle, De i a i e es ima ion and es ing in gene alized addi i e models,
J. S a . Plan. In e .,115 (2003), 521–542. h ps://doi.o g/10.1016/S0378-3758(02)00163-5
29. S. Dudoi , M. J. Van De Laan, Mul iple Tes ing P ocedu es wi h Applica ions o Genomics,
Sp inge , Sp inge Se ies in S a is ics, New Yo k, 2007.
©2022 he Au ho (s), licensee AIMS P ess. This
is an open access a icle dis ibu ed unde he
e ms o he C ea i e Commons A ibu ion License
(h p://c ea i ecommons.o g/licenses/by/4.0)
Ma hema ical Biosciences and Enginee ing Volume 19, Issue 7, 6435–6454.