Conditional dependence via U-Statistics pruning

Accep ed e sion o he a icle published in IEEE Signal P ocessing Le e s, 2025. DOI: 10.1109/LSP.2025.3539587.
A ailable a : h p://ieeexplo e.ieee.o g
Condi ional Dependence ia U-S a is ics P uning
Fe an de Cab e a , Ma c Vil
`
a-Insa , G adua e S uden Membe , IEEE, and Jaume Riba , Senio Membe , IEEE
Abs ac —The p oblem o measu ing condi ional dependence
be ween wo andom phenomena a ises when a hi d one (a
con ounde ) has a po en ial in luence on he amoun o in o ma ion
be ween hem. A ypical issue in his challenging p oblem is he
in e sion o ill-condi ioned au oco ela ion ma ices. This pape
p esen s a no el measu e o condi ional dependence based on
he use o incomple e unbiased s a is ics o deg ee wo, which
allows o e-in e p e independence as unco ela edness on a ini e-
dimensional ea u e space. This o mula ion enables o p une
da a acco ding o obse a ions o he con ounde i sel , hus
a oiding ma ix in e sions al oge he . The p oposed app oach is
a icula ed as an ex ension o he Hilbe -Schmid independence
c i e ion, which becomes exp essible h ough ke nels ha ope a e
on 4- uples o da a.
Index Te ms—Hilbe -Schmid independence c i e ion (HSIC),
ke nel me hods, condi ional dependence, U-S a is ics.
I. INTRODUCTION
C
ONDITIONAL dependence becomes ele an when a
hi d phenomenon migh explain, media e, o con ound
he appa en ela ionship exhibi ed by an o iginal andom pai .
I s measu emen is an impo an ask in causal disco e y and
Bayesian ne wo k lea ning [1]–[3, Ch. 7], which eme ge in
applied ields such as ea h sys em sciences [4] o clinical
da a analysis [5]. Due o i s na u e, empi ically es ima ing
condi ional dependence is a complex p oblem, gi en ha i also
equi es in e ing condi ional ma ginal dis ibu ions. A known
app oach consis s in measu ing he Hilbe -Schmid no m o he
no malized condi ional c oss-co a iance ope a o [6], g ounded
in he heo y o ep oducing ke nel Hilbe spaces (RKHS).
This ke nel-based me hod is capable o measu ing s a is ical
dependence by gauging co ela ion o da a implici ly mapped
on o an in ini e-dimensional space. Howe e , i is also known
o ha e nume ical issues conce ned wi h ma ix in e sions
and egula iza ion due o he usually low- ank s uc u e o he
in ol ed co ela ion ma ices [6]. This also leads o a ai ly
complex me hod ha scales cubically wi h he obse ed da a
size, becoming p ohibi i e o la ge da a se s.
These issues ha e la ely been he subjec o in e es o nume -
ous lines o esea ch. A ke nel-based condi ional independence
es is p oposed in [1] by de i ing he asymp o ic dis ibu ion
o he app op ia e es s a is ic unde he null hypo hesis, which
succeeds in a oiding ma ix in e sions, bu whose complexi y
s ill inc eases cubically wi h he sample size. O he au ho s
ha e p oposed o app oxima e ke nel unc ions by andomly
This wo k was (pa ially) unded by p ojec MAYTE (PID2022-136512OB-
C21) by MICIU/AEI/10.13039/501100011033 and ERDF/EU, g an 2021
SGR 01033, and g an 2022 FI SDUR 00164 by Depa amen de Rece ca i
Uni e si a s de la Gene ali a de Ca alunya.
F. de Cab e a is an independen esea che (e-mail: e andecab -
[email p o ec ed]). M. Vil
`
a-Insa and J. Riba a e wi h he Signal P ocessing and
Communica ions G oup (SPCOM), Uni e si a Poli
`
ecnica de Ca alunya (UPC),
Ba celona, Spain (e-mail: [email p o ec ed]; [email p o ec ed]).
selec ing ea u es [7], by p o iding an al e na i e e iew o
he embedding p ocess on o he RKHS [8], o by le e aging
he in e p e a ion ha dependence p oduces co ela ion o
dis ances [9], i.e. close pai s in one da a se coincide wi h close
pai s in he o he [10]. Ne e heless, none o hese me hods
add esses he a o emen ioned compu a ional issues, since hey
all equi e ma ix in e sions. An ex ension o he Hilbe -
Schmid independence c i e ion (HSIC) [11], a measu e o
uncondi ional independence, is in oduced in [12] wi h he
objec i e o in e ing causali y, a simila endea o o his le e ,
al hough ocused on join independence a he han condi ional
dependence.
This le e p esen s a p ocedu e o measu ing condi ional
dependence by s a is ically condi ioning he obse ed da a o a
po en ial con ounde in a simple way, bypassing ma ix in e -
sion p oblems. The basis o condi ioning is o p une pai wise
di e ences o da a unde he con ol o he con ounde [13],
s emming om dis ance-based me hods [10]. Simul aneously,
he es ima e is cons uc ed as an al e na i e de i a ion o he
ke nel-based HSIC. While hese wo app oaches a e known o
be equi alen [14], we p opose he e an es ima e ha combines
hem, albei o di e en asks. To do so, we i s p o ide a
no el ein e p e a ion o he HSIC in Sec. II by ansla ing he
p oblem o s a is ical dependence in o one o co ela ion a e
mapping da a on o ini e-dimensional spaces based on s ee ing
ec o s [15]. Sec. III b ie ly e iews he heo y o unbiased
s a is ics (U-S a is ics) [16], which is hen employed in Sec. IV
o show ha condi ional dependence can be ob ained by p uning
U-S a is ics. The ob ained measu e, named condi ional HSIC
(C-HSIC), is based on a classical signal p ocessing s uc u e
bu linked o ke nel me hods, which emb aces he HSIC as a
pa icula case when he U-S a is ic is comple e.
II. MARGINAL DEPENDENCE AS CORRELATION
Le
d:R→CM
be a mapping based on windowed s ee ing
ec o s. The m h elemen o dcan be exp essed as
[d(·)]m≜1
4
√MGm
√Mejπm
√M
·,(1)
whe e
1m∈ {−M/2, . . . , M/2−1}
and
G : R→R
is an
e en and absolu ely in eg able unc ion wi h uni
L2
-no m and
maximum a
G(0)
. Gi en wo andom a iables
x
and
y
, we
de ine a pai o ans o med andom ec o s
2u≜d(x)
and
≜d(y)
. Thei c oss-co a iance ma ix
Cu, ∈CM×M
is
de ined as
Cu, ≜Eu H−E[u] E[ ]H
, whe e
E[·]
deno es
he expec a ion ope a o . Then, he ollowing implica ion
holds [11]:
lim
M→∞∥Cu, ∥2
F= 0 ⇐⇒ x,ya e independen ,(2)
1Mis assumed e en o ma hema ical con enience.
2
Wi hou loss o gene ali y, and o he sake o simplici y, we le
u
and
ha e he same dimensionali y.
©2025 IEEE. Pe sonal use o his ma e ial is pe mi ed. Pe mission om IEEE mus be ob ained o all o he uses, in any cu en o u u e
media, including ep in ing/ epublishing his ma e ial o ad e ising o p omo ional pu poses, c ea ing new collec i e wo ks, o esale o
edis ibu ion o se e s o lis s, o euse o any copy igh ed componen o his wo k in o he wo ks.
whe e
∥·∥F
deno es he F obenius no m. Te ms
E[d(x)]
and
E[d(y)]
become dense samplings o he weigh ed (by
G(·)
)
ma ginal cha ac e is ic unc ion o
x
and
y
, espec i ely, in he
limi o
M→ ∞
. Simila ly,
E[d(x)dH(y)]
becomes a dense
sampling o he weigh ed join cha ac e is ic unc ion (JCF).
The e o e,
(2)
is e ec i ely checking he sepa abili y o he JCF
a e e y sample poin (i.e. he ac o iza ion o
E[d(x)dH(y)]
as a p oduc o expec a ions), which becomes equi alen o
an unco ela edness p ope y [15], [17], akin o he dis ance
co a iance in [10].
Conside
L
i.i.d. samples o
x
and
y
,
{x(l), y(l)}l=1,...,L
,
om which we ob ain wo ans o med complex da a ma ices
U∈CM×Land V∈CM×L, de ined as ollows:
U≜[u1,...,uL]=[d(x(1)),...,d(x(L))] (3a)
V≜[ 1,..., L] = [d(y(1)),...,d(y(L))].(3b)
The sample c oss-co a iance be ween he ans o med ec o s
is gi en by
b
Cu, ≜1
L−1
L
X
l=1 ul−1
L
L
X
i=1
ui! l−1
L
L
X
j=1
j!H
=1
L−1UPVH,(4)
whe e
P≜I−1
L11T∈RL×L
is he da a cen e ing ma ix,
a well-known p ojec ion ma ix in he signal p ocessing and
ke nel me hods li e a u e [18, Appx. B.7]. Using
(2)
, a ma ginal
dependence measu e is gi en by
∥b
Cu, ∥2
F= (b
CH
u, b
Cu, ) = 1
(L−1)2VPHUHUPVH
=1
(L−1)2 (PUHUPVHV),(5)
whe e he ci cula i y o he ace ope a o has been used. To
see he link o
(5)
wi h he HSIC, le us examine he limi o
M→ ∞
o he ke nel ma ices
K≜limM→∞ UHU∈RL×L
and
Q≜limM→∞ VHV∈RL×L
. The elemen s o
K
(and
analogously hose o Q) a e he ollowing:
[K]l,l′= lim
M→∞
dH(x(l))d(x(l′))
= lim
M→∞
M
2−1
X
m=−M
2
1
√MG2m
√Me−j2π(x(l)−x(l′)) m
√M(6)
=Z∞
−∞
G2( )e−j2π(x(l)−x(l′)) d ≜κx(l)−x(l′),
whe e
κ(·)
is he ke nel unc ion ha esul s om he Fou ie
ans o m
3
o
G2( )
, and he in eg al is he limi o he Da boux
sum in he second line. As a esul , he en ies o
K
and
Q
a e jus he e alua ion o
κ(·)
a he di e ence be ween wo
da a samples. Then, gi en
(5)
and aking he limi
M→ ∞
,
we ob ain he HSIC [11, Sec. 3.1]:
HSIC(x;y)≜1
(L−1)2 (PKPQ) = lim
M→∞∥b
Cu, ∥2
F,(7)
wi h x= [x(1), . . . , x(L)]Tand y= [y(1), . . . , y(L)]T.
3
F om he p ope ies imposed on
G2( )
,
κ(·)
becomes na u ally an
au oco ela ion ansla ion-in a ian ke nel, such ha
κ(0) = 1 ≥ |κ(s)|
and
κ(∞) = κ(−∞) = 0
. The eade is e e ed o [19, Sec. 1.4] whe e
hese ideas eme ge in he ligh o Bochne ’s heo em.
In summa y, he al e na i e o mula ion exposed abo e shows
ha he HSIC esul s om measu ing co ela ion in a ini e-
dimensional space based on s ee ing ec o s, whe e he ke nel
o mula ion a ises in a second s age once he dimensionali y
g ows wi hou bound. The in e play be ween ke nels and c oss-
co a iance ma ices o mapped da a will be used la e on o
condi ion he ke nel ma ices by p uning da a. Be o ehand, we
will e o mula e he es ima ion o c oss-co a iance ma ices
le e aging incomple e U-s a is ics, which will se e as he basis
o da a p uning.
III. SAMPLE COVARIANCE MATRIX AS AN INCOMPLETE
U-STATISTIC
Conside a lis con aining all he unique
Kmax ≜L(L−1)/2
pai s ha can be cons uc ed om
L
di e en samples o a
andom a iable. This lis is o de ed a bi a ily such ha a
single index
k
iden i ies bo h elemen s o a pai . Le
1(k)
and
2(k)
be wo unc ions ha e u n he co esponding indices
o each e m o he
k
h pai . Gi en
K≤Kmax
indices, we
cons uc he pai wise di e ences
˚
uk≜1
√2u 1(k)−u 2(k),˚
k≜1
√2 1(k)− 2(k),(8)
o
k∈ {1, . . . , K}
, co esponding o he samples o wo
i ual sou ces
˚
u
and
˚
. I is wo h no ing ha hese a e ze o-
mean a iables o any pai wi h
1(k)= 2(k)
, since
u
and
a e i.i.d.. Thei sample c oss-co a iance ma ix
b
C˚
u,
˚
∈CM×M
is hen he ollowing:
b
C˚
u,
˚
=1
K˚
U˚
VH,(9)
whe e
˚
U≜[˚
u1,...,˚
uK]∈CM×K
and
˚
V≜[˚
1,...,˚
K]∈
CM×K
. No e ha , hanks o he cons an ac o in
(8)
,
˚
u
and
˚
ha e he same a e age co a iance ma ix as
u
and
,
espec i ely. In con as o
(4)
,
P
is missing in
(9)
as a esul
o cons uc ing ze o-mean i ual da a in
(8)
, which will lead
o a cleane implemen a ion and ewe ma ix ope a ions.
Equa ion
(9)
is in ac an ins ance o a U-S a is ic. Fo
K=
Kmax
,i.e. when all da a pai s a e aken, we ob ain he equali y
b
C˚
u,
˚
=b
Cu,
om U-S a is ics heo y [16]. In con as , o
K < Kmax
,i.e. when
(9)
is an incomple e U-S a is ic, al hough
b
C˚
u,
˚
emains unbiased, i s es ima ion a iance inc eases due
o he p uning o da a. Ne e heless,
b
C˚
u,
˚
is s ill a consis en
es ima e o Cu, p o ided ha K→ ∞ as L→ ∞.
Rema k 1 (Robus ness o p uning): A no able p ope y o
incomple e U-s a is ics is ha hei obus ness agains da a
p uning inc eases he la ge
L
is [13]. To b ie ly illus a e
his p ope y, conside aking only he
K=⌊L/2⌋
da a
pai s wi h no indices in common. The numbe o emaining,
unused, da a pai s is equal o
L(L−1)/2− ⌊L/2⌋
, which
inc eases wi h
O(L2)
. To only use he i.i.d., unique, e ms
is e ec i ely equi alen o compu ing
b
Cu,
wi h hal o he
a ailable samples [20]. I is hen sa e o assume ha hei
con ibu ion o he o e all accu acy o he sample co a iance is
highe han hose wi h epea ed indices. The e o e, he la ge
L
is, he highe he amoun o pai s ha can be p uned o
some speci ied deg ada ion in he es ima ion accu acy o he
esul ing sample co a iance. The implica ion is ha
K
in he
incomple e U-S a is ic can be designed o g ow wi h
O(L)
ins ead o
O(L2)
, which p o ides conside able lexibili y o
p uning, eases he o e all compu a ional complexi y, and will
be used o choosing he numbe o pai s in he nex sec ion.
Once we ha e de e mined he incomple e U-s a is ic o mu-
la ion o he c oss-co a iance ma ix, we a e now in a posi ion
o es ablish he unde lying ule o
1(·)
and
2(·)
by looking
a sample pai wise di e ences o he con ounde . A e wa ds,
we will be employing he p uned c oss-co a iance ma ix o
e u n o ke nel me hods in he same way as po ayed in
(7)
.
IV. CONDITIONAL DEPENDENCE VIA U-STATISTICS
The condi ional c oss-co a iance ma ix be ween
u
and
is
de ined as:
Cu, |z
≜ZR
Cu, |z=zdFz(z) = ZR
C˚
u,
˚
|z=zdFz(z),(10)
whe e
C˚
u,
˚
|z=z=Cu, |z=z
(gi en
(8)
) is he c oss-co a iance
ma ix condi ioned o a speci ic alue
z
o con ounde
z
, and
Fz(z)
is i s cumula i e dis ibu ion. Wi h he goal o de i ing
an es ima o o
(10)
, we de ine he i ual andom a iable
˚
z≜z1−z2
, whe e
z1
and
z2
a e mu ually independen and
dis ibu ed as
z
. In eg a ing o e all alues o
z
is equi alen
o doing so o e he e en s in which
z1
and
z2
ake he same
alue, i.e.
˚
z= 0
. The e o e, he expec a ion in
(10)
can be
al e na i ely exp essed as
Cu, |z=ZZR2
C˚
u,
˚
|˚
z=0 dFz(z1) dFz(z2)
=C˚
u,
˚
|˚
z=0 ZZR2
dFz(z1) dFz(z2) = C˚
u,
˚
|˚
z=0,(11)
whe e
RRR2dFz(z1) dFz(z2)=1
, and
C˚
u,
˚
|˚
z=0
does no
depend on he speci ic alues o
z1
and
z2
bu a he on hem
being equal. In consequence, condi ioning wi h espec o
z
is
equal o condi ioning wi h espec o
˚
z= 0
. The e o e,
(11)
sugges s o le he p uning o he incomple e U-S a is ics in
(9)
be handled by he po en ial con ounde . Howe e , since
˚
z= 0
is an e en o ze o p obabili y o con inuous andom a iables,
da a con ol should be based on me ely small alues o
|˚z|
[13].
Wi h he in en ion o choosing da a pai s o p une acco ding
o |˚z|, we de ine he samples o ˚
zas ollows:
˚zl,l′≜z(l)−z(l′), l =l′,(12)
whe e
z(l)
and
z(l′)
a e i.i.d. samples d awn om
z
. Then,
we le he so ing o
|˚zl,l′|
(in ascending o de ) be he one
ha de e mines he o de ing o he index pai s p o ided by
1(·)
and
2(·)
in Sec. III. Mo eo e , in iew o Rema k 1, he
amoun o pai s Kis se o g ow as O(L)wi h
K=Lα
2,(13)
being
1≤α≪(L−1)
a uning hype -pa ame e . While
α= 1
ensu es ha only e y small alues o
|˚zl,l′|
a e conside ed,
he U-S a is ic becomes comple e and he e is no condi ioning
a all o
α=L−1
, hus yielding he HSIC as a pa icula
case as in
(7)
. Then a na u al ade-o eme ges: low alues o
α
a e desi able o p o ide s ong condi ioning o da a, bu may
also lead o excessi e p uning wi h low s a is ical accu acy
in
(9)
. Rema kably, he selec ion o
α
becomes a mino issue
p o ided ha
L
is su icien ly la ge, as shown in [13] unde he
co ela ion measu e amewo k be ween a pai o ec o s. This
will be u he discussed in Sec. V wi h a nume ical example.
A. Condi ional HSIC
Now ha we ha e de e mined he so ing and p uning o da a
pai s acco ding o he con ounde
z
, le us w i e a condi ional
dependence measu e as he F obenius no m o (9):
b
CH
˚
u,
˚
|zb
C˚
u,
˚
|z=1
K2 ˚
UH˚
U˚
VH˚
V,(14)
whe e he ci cula i y o he ace has been used. To link he
p e ious exp ession wi h ke nel-based me hods, le us ew i e
he ze o-mean i ual da a ma ix ˚
Uas ollows:
˚
U=1
√2(U1−U2),Ua≜[u a(1),...,u a(K)],(15)
o
a={1,2}
. The same is done o
˚
V
. Acco dingly,
(14)
is
hen ew i en as
b
CH
˚
u,
˚
|zb
C˚
u,
˚
|z=(16)
1
4K2 (U1−U2)H(U1−U2)(V1−V2)H(V1−V2).
Taking he limi o
M→ ∞
, ke nel ma ices a e ob ained
om he p oduc s among Uand Vin (16):
Ka,a′≜lim
M→∞UH
aUa′,Qa,a′≜lim
M→∞VH
aVa′,(17)
whose elemen s a e
[Ka,a′]k,k′=κx( a(k)) −x( a′(k′)),(18a)
[Qa,a′]k,k′=κy( a(k)) −y( a′(k′)),(18b)
whe e
κ(·)
is he ke nel unc ion as in
(6)
. Finally, he esul ing
C-HSIC can be exp essed as ollows:
C-HSICα(x;y)≜1
4K2 ˘
K˘
Q(19)
wi h
˘
K≜K1,1+K2,2−K1,2−KH
1,2
and
˘
Q≜Q1,1+
Q2,2−Q1,2−QH
1,2
. No e ha , as a esul o he U-S a is ics
implemen a ion, each en y o he new ke nel-based ma ices
in ol es ou da a samples o he same sou ce (
4
- uples), in
con as o only he pai s ypically in ol ed in classical ke nel
me hods. This ac , along wi h he lack o
P
, a e he main
dis inc i e ea u es o he C-HSIC
(19)
s. HSIC
(7)
. In con as
o o he condi ional dependence measu es,
(19)
does no equi e
any ma ix in e sion. Since
K
is se o g ow linea ly wi h
L
in (13), he compu a ional complexi y is O(L2).
V. NUMERICAL ILLUSTRATIONS
To es he p oposed me hod, we aim a gene a ing unco e-
la ed da a wi h a con olled amoun o condi ional dependence.
Two scena ios a e s udied,
M+
and
M−
, de ined as ollows:
M+:




x=√γap +
y=√γaq +w
z=a
,M−:




x=√γbp +
y=√γcq +w
z=b−c
.(20)
The in e nal i.i.d. andom a iables a e dis ibu ed as
a,b,c∼
U(0,√3)
(uni o m),
,w∼ N(0,1)
(no mal), and
p,q∼
Be n1/2{−1,1}
(equip obable Be noulli). In
M+
, he pai
{x,y}
a e dependen , i.e. hei mu ual in o ma ion is g ea e
han ze o, bu hey a e condi ionally independen , since knowing
-10 0 10 20 30
0
0.01
0.02
0.03
0.04
0.05
0.06
100 200 300 400 500 600
0
1
2
3
4
5
6
710-3
(a) Model M+.
-10 0 10 20 30
0
0.005
0.01
0.015
0.02
0.025
0.03
100 200 300 400 500 600
0
0.005
0.01
0.015
0.02
0.025
0.03
(b) Model M−.
Fig. 1: Dependence measu es as a unc ion o
γ
(wi h
L=
100
) and
L
(wi h
γ= 10
dB). C-HSIC measu es condi ional
dependence while HSIC measu es uncondi ional dependence.
Ma ke s deno e he empi ical a e age and bands indica e he
s anda d de ia ion.
z
implies ha
x
and
y
become solely d i en by independen
phenomena (
and
w
). By con as ,
x
and
y
a e ma ginally
independen in
M−
, bu become condi ionally dependen , since
knowledge o
z
co ela es he possible join alues o
b
and
c
. Pa ame e
γ
is he signal- o-noise a io associa ed wi h he
measu emen s and con ols he o al amoun o uncondi ional
dependence in
M+
and condi ional dependence in
M−
. In
bo h models,
x
,
y
and
z
a e mu ually unco ela ed due o
he mul iplica i e e ec o mu ually independen a iables
p
and
q
, so co ela ion measu es a e unable o disco e
any da a associa ion. Simila ideas on modeling condi ional
dependencies can be ound in [21], which a e inspi ed by
co-in o ma ion [22], [23].
The uni e sal Gaussian ke nel is used as he ke nel unc ion
o choice, which yields
κ(s) = exp(−(s
ˆσL−1/5)2)
being
ˆσ
he
sample s anda d de ia ion. We use his exp ession because o
i s associa ion wi h ke nel densi y es ima ion, known o be
ela ed o ke nel me hods when es ima ing ce ain dependence
measu es [24, Ch. 2]. This connec ion also jus i ies he adop ion
o he powe law O(L−1/5)[25, Ch. 3], [15, Appx. D].
Fig. 1 shows measu es o dependence o bo h models wi h
wo
α
alues om
(13)
: a mode a e
α= 4
and a six een old
inc ease
α= 64
. This yields
4%
and
64.5%
, espec i ely, o he
o al da a pai s o
L= 100
, o
0.7%
and
10.7%
o
L= 600
(and e en less o highe
L
). Fo
M+
(Fig. 1-(a)), C-HSIC
co ec ly depic s condi ional independence o an inc easing
alue o
γ
, while HSIC con i ms ha ma ginal dependence
is high as
γ
inc eases. Con e sely, Fig. 1-(b) exhibi s he
capabili y o C-HSIC o disco e condi ional dependence o
mode a e and high
γ
alues in
M−
, while ma ginal dependence
is con i med o be small by HSIC a any alue o γ.
To ge u he insigh s on he p oposed ideas, Fig. 1-(a.1)
and Fig. 1-(b.1) also show he pe o mance o C-HSIC when
p uning da a pai s is pe o med andomly in he U-s a is ic,
i.e. he pai gi en by
1(·)
and
2(·)
is no con olled by he
con ounde . As a esul o no being p ope ly condi ioned, C-
HSIC p oduces a measu e o ma ginal dependence simila o
HSIC, bu wi h inc eased a iance due o he p uning i sel . I
is also wo h no ing ha andom and o de ly p uning pe o m
simila ly o low alues o
γ
, co esponding o he egime
whe e he con ounde has no in luence on he pai
{x,y}
, and
s a o de ia e as
γ
inc eases. Addi ionally, i can be seen
ha he expec ed alue de e io a es o
α= 64
, inc easing in
M+
and dec easing in
M−
. This is a consequence o a mild
p uning o
L= 100
(whe e
α= 64
is in en ionally chosen o
illus a e his e ec ), wo sening he condi ioning p ocess and
s a ing o beha e as an uncondi ioned measu e.
Finally, Fig. 1-(a.2) and Fig. 1-(b.2) illus a e he beha io
o C-HSIC o mul iple alues o
L
a
γ= 10
dB. Model
M+
, which is condi ionally independen , shows ha C-HSIC
co ec ly app oaches ze o as
L
inc eases. Con e sely, i ends
o some nonze o alue in
M−
. As men ioned abo e,
α= 64
is using oo many da a pai s o small L alues and p o ides
a deg aded empi ical a e age. In bo h models,
α= 4
shows
an inc eased a iance wi h espec o
α= 64
due o a mo e
ho ough p uning. Howe e , his is educed o su icien ly la ge
da a sizes due o Rema k 1. Simila ly, he gap be ween expec ed
alues o
α= 4
and
α= 64
is educed as
L
inc eases,
albei he o me has he ad an age o a lowe compu a ional
complexi y. These esul s show ha
α
is a nonc i ical hype -
pa ame e , p o ided i is easonably small.
VI. CONCLUSIONS
This wo k has p oposed a p oo o concep o a modi ica ion
on he classical HSIC unde he condi ional dependence
amewo k, named C-HSIC. I s in e p e a ion as a co ela ion
measu e on a ini e bu high-dimensional space allows an
insigh ul connec ion wi h ke nels by le ing he dimension g ow
wi hou bound. Mo eo e , i opens he possibili y o le e aging
U-S a is ics o his ask. Thanks o his o mula ion, we can
p o ide a no el app oach o pe o ming s a is ical condi ioning
by p uning da a pai s based on he pai wise di e ences o a
po en ial con ounde . Fu he mo e, he p oposed measu e o
condi ional dependence does no equi e ma ix in e sions,
which has he ad an age o educed compu a ional complexi y
and he a oidance o add essing ill-condi ioned ma ices.
Nume ical illus a ions ha e shown ha C-HSIC is capable
o measu ing bo h condi ional dependence and independence
in wo di e en scena ios. Fu he esea ch should s udy he
po en ial o he p oposed me hod wi h iche da a se s, as
well as p o ide a ho ough compa ison wi h o he me hods o
measu ing condi ional dependence.
REFERENCES
[1]
K. Zhang, J. Pe e s, D. Janzing, and B. Sch
¨
olkop , Ke nel-based
Condi ional Independence Tes and Applica ion in Causal Disco e y.
P oceedings o he 27 h Annual Con e ence on Unce ain y in A i icial
In elligence (UAI), 2011, pp. 804–813.
[2]
J. Zhang, “Causal easoning wi h ances al g aphs,” Jou nal o Machine
Lea ning Resea ch, ol. 9, no. 7, 2008.
[3]
J. Pe e s, D. Janzing, and B. Sch
¨
olkop , Elemen s o causal in e ence:
ounda ions and lea ning algo i hms. Camb idge, MA, USA: The MIT
P ess, 2017.
[4]
J. Runge, S. Ba hiany, E. Boll , G. Camps-Valls, D. Coumou, E. Deyle,
C. Glymou , M. K e schme , M. D. Mahecha, J. Mu
˜
noz-Ma
´
ıe al.,
“In e ing causa ion om ime se ies in ea h sys em sciences,” Na u e
communica ions, ol. 10, no. 1, 2019.
[5]
X. Shen, S. Ma, P. Vemu i, and G. Simon, “Challenges and oppo uni ies
wi h causal disco e y algo i hms: applica ion o alzheime ’s pa hophysi-
ology,” Scien i ic epo s, ol. 10, no. 1, 2020.
[6]
K. Fukumizu, A. G e on, X. Sun, and B. Sch
¨
olkop , “Ke nel measu es o
condi ional dependence,” in Ad ances in Neu al In o ma ion P ocessing
Sys ems, ol. 20, 2007.
[7]
E. V. S obl, K. Zhang, and S. Visweswa an, “App oxima e ke nel-based
condi ional independence es s o as non-pa ame ic causal disco e y,”
Jou nal o Causal In e ence, ol. 7, no. 1, p. 20180017, 2019.
[8]
J. Pa k and K. Muande , “A measu e- heo e ic app oach o ke nel condi-
ional mean embeddings,” Ad ances in neu al in o ma ion p ocessing
sys ems, ol. 33, pp. 21 247–21 259, 2020.
[9]
X. Wang, W. Pan, W. Hu, Y. Tian, and H. Zhang, “Condi ional dis ance
co ela ion,” Jou nal o he Ame ican S a is ical Associa ion, ol. 110,
no. 512, pp. 1726–1734, 2015.
[10]
G. J. Sz
´
ekely, M. L. Rizzo, and N. K. Baki o , “Measu ing and es ing
dependence by co ela ion o dis ances,” The Annals o S a is ics, ol. 35,
no. 6, pp. 2769 – 2794, 2007.
[11]
A. G e on, O. Bousque , A. Smola, and B. Sch
¨
olkop , “Measu ing s a is-
ical dependence wi h Hilbe -Schmid no ms,” In e na ional con e ence
on algo i hmic lea ning heo y, pp. 63–77, 2005.
[12]
N. P is e , P. B
¨
uhlmann, B. Sch
¨
olkop , and J. Pe e s, “Ke nel-based es s
o join independence,” Jou nal o he Royal S a is ical Socie y Se ies
B: S a is ical Me hodology, ol. 80, no. 1, pp. 5–31, 2018.
[13]
M. Vil
`
a and J. Riba, “A es o condi ional co ela ion be ween andom
ec o s based on weigh ed U-s a is ics,” in 2022 IEEE In e na ional
Con e ence on Acous ics, Speech and Signal P ocessing (ICASSP), 2022,
pp. 5792–5796.
[14]
T. Sheng and B. K. S ipe umbudu , “On dis ance and ke nel measu es o
condi ional dependence,” Jou nal o Machine Lea ning Resea ch, ol. 24,
no. 7, pp. 1–16, 2023.
[15]
F. de Cab e a and J. Riba, “Regula ized es ima ion o in o ma ion ia
canonical co ela ion analysis on a ini e-dimensional ea u e space,”
IEEE T ansac ions on In o ma ion Theo y, ol. 69, no. 8, pp. 5135–5150,
2023.
[16]
A. J. Lee, U-S a is ics: Theo y and P ac ice. Boca Ra on, FL, USA:
Rou ledge, 1990.
[17]
F. de Cab e a and J. Riba, “A no el o mula ion o independence de ec ion
based on he sample cha ac e is ic unc ion,” in 26 h Eu opean Signal
P ocessing Con e ence (EUSIPCO), 2018, pp. 2608–2612.
[18]
D. Ram
´
ı ez, I. San ama
´
ıa, and L. Scha , Cohe ence: In Signal
P ocessing and Machine Lea ning. Cham, Swi ze land: Sp inge Na u e,
2023.
[19]
W. Rudin, Fou ie Analysis on G oups. Hoboken, NJ, USA: John Wiley
& Sons, L d, 1990.
[20]
X. Chen and K. Ka o, “Randomized incomple e
U
-s a is ics in high
dimensions,” The Annals o S a is ics, ol. 47, no. 6, pp. 3127 – 3156,
2019.
[21]
B. P
´
oczos and J. Schneide , “Nonpa ame ic es ima ion o condi ional
in o ma ion and di e gences,” in P oceedings o 15 h In e na ional
Con e ence on A i icial In elligence and S a is ics (AISTATS ’12), Ap il
2012, pp. 914 – 923.
[22]
A. J. Bell, “The co-in o ma ion la ice,” in P oceedings o he i h
in e na ional wo kshop on independen componen analysis and blind
signal sepa a ion: ICA, ol. 2003, 2003.
[23]
A. Ghassami and N. Kiya ash, “In e ac ion in o ma ion o causal
in e ence: The case o di ec ed iangle,” in 2017 IEEE In e na ional
Symposium on In o ma ion Theo y (ISIT), 2017, pp. 1326–1330.
[24]
J. C. P
´
ıncipe, In o ma ion Theo e ic Lea ning: R
´
enyi’s En opy and
Ke nel Pe spec i es. New Yo k, NY, USA: Sp inge , 2010.
[25]
B. Sil e man, Densi y es ima ion o s a is ics and da a analysis. London,
UK: Chapman and Hall, 1986.