This is a sel -a chi ed e sion o an o iginal a icle. This e sion
may di e om he o iginal in pagina ion and ypog aphic de ails.
Au ho (s):
Ti le:
Yea :
Ve sion:
Copy igh :
Righ s:
Righ s u l:
Please ci e he o iginal e sion:
CC BY 4.0
h ps://c ea i ecommons.o g/licenses/by/4.0/
Modelling mul i a ia e spa io- empo al da a wi h iden i iable a ia ional au oencode s
© 2024 The Au ho s. Published by Else ie L d.
Published e sion
Sipilä, Mika; Cappello, Claudia; De Iaco, Sand a; No dhausen, Klaus; Taskinen,
Sa a
Sipilä, M., Cappello, C., De Iaco, S., No dhausen, K., & Taskinen, S. (2025). Modelling
mul i a ia e spa io- empo al da a wi h iden i iable a ia ional au oencode s. Neu al Ne wo ks,
181, A icle 106774. h ps://doi.o g/10.1016/j.neune .2024.106774
2025
Con en s lis s a ailable a ScienceDi ec
Neu al Ne wo ks
jou nal homepage: www.else ie .com/loca e/neune
Full Leng h A icle
Modelling mul i a ia e spa io- empo al da a wi h iden i iable a ia ional
au oencode s
Mika Sipilä a,∗, Claudia Cappello b, Sand a De Iacob, Klaus No dhausen a, Sa a Taskinen a
aDepa men o Ma hema ics and S a is ics, Uni e si y o Jy äskylä, Finland
bDSE - Sec ion o Ma hema ics and S a is ics, Uni e si y o Salen o, I aly
ARTICLE INFO
Keywo ds:
Blind sou ce sepa a ion
Dimension es ima ion
K iging
Me eo ological da a
Shapley alues
ABSTRACT
Modelling mul i a ia e spa io- empo al da a wi h complex dependency s uc u es is a challenging ask bu
can be simpli ied by assuming ha he o iginal a iables a e gene a ed om independen la en componen s.
I hese componen s a e ound, hey can be modelled uni a ia ely. Blind sou ce sepa a ion aims o eco e
he la en componen s by es ima ing he unknown linea o nonlinea unmixing ans o ma ion based on he
obse ed da a only. In his pape , we ex end ecen ly in oduced iden i iable a ia ional au oencode o he
nonlinea nons a iona y spa io- empo al blind sou ce sepa a ion se ing and demons a e i s pe o mance using
comp ehensi e simula ion s udies. Addi ionally, we in oduce wo al e na i e me hods o he la en dimension
es ima ion, which is a c ucial ask in o de o ob ain he co ec la en ep esen a ion. Finally, we illus a e
he p oposed me hods using a me eo ological applica ion, whe e we es ima e he la en dimension and he
la en componen s, in e p e he componen s, and show how nons a iona i y can be accoun ed and p edic ion
accu acy can be imp o ed by using he p oposed nonlinea blind sou ce sepa a ion me hod as a p ep ocessing
me hod.
1. In oduc ion
Many eal wo ld phenomena, such as wea he , epidemiological
pa e ns and ecosys em dynamics, a e mul i a ia e spa io- empo al,
meaning ha mul i a ia e obse a ions 𝒙(𝒔, 𝑡) ∶= 𝒙∈R𝑆a e obse ed
in a spa ial loca ion 𝒔∈⊂R𝐷a ime 𝑡∈⊂R, whe e is called
a spa ial domain, is called a empo al domain and 𝐷is a spa ial
dimension. Wi hou loss o gene ali y, we assume om now on ha
𝐷= 2. A mul i a ia e obse a ion 𝒙con ains measu emen s o mul iple,
usually dependen , andom a iables desc ibing he phenomenon o
in e es . When modelling such mul i a ia e spa io- empo al da a, one
has o accoun no only he dependence be ween he a iables, bu
also he dependences in space and in ime. The dependence s uc-
u e is o en desc ibed h ough spa io- empo al co a iance unc ion
𝑪(𝒙(𝒔, 𝑡),𝒙(𝒔′, 𝑡′)), whe e (𝒔, 𝑡)and (𝒔′, 𝑡′)a e wo spa io- empo al lo-
ca ions. The co a iance 𝑪is a 𝑆×𝑆ma ix alued unc ional wi h
elemen s 𝐶𝑖𝑗 ,𝑖, 𝑗 = 1,…, 𝑆, de ined as 𝐶𝑖𝑗 =𝐶(𝑥𝑖(𝒔, 𝑡), 𝑥𝑗(𝒔′, 𝑡′)) =
𝐸(𝑥𝑖(𝒔, 𝑡)𝑥𝑗(𝒔′, 𝑡′))−𝐸(𝑥𝑖(𝒔, 𝑡))𝐸(𝑥𝑗(𝒔′, 𝑡′)). Modelling he co a iance unc-
ion 𝑪is usually a highly demanding ask, and o en, in o de o make
he modelling easible, some se e ely es ic ing assump ions, such as
s a iona i y o sepa abili y, a e made. When he spa io- empo al ield
is assumed o be s a iona y, he co a iance unc ion can be simpli ied
∗Co esponding au ho .
E-mail add ess: [email p o ec ed] (M. Sipilä).
o
𝑪(𝒙(𝒔, 𝑡),𝒙(𝒔′, 𝑡′)) = 𝑪(‖𝒔−𝒔′‖,|𝑡−𝑡′|),(1)
meaning ha he alue o he unc ion depends only on he dis ance
be ween he spa ial loca ions and he dis ance be ween empo al loca-
ions. I (1) does no hold, he da a a e nons a iona y, meaning ha
he co a iance unc ion may di e when spa ial o empo al loca ion is
al e ed. When sepa abili y is assumed, he spa io- empo al co a iance
unc ion can be w i en as a p oduc o spa ial and empo al co a iance
unc ions as
𝑪(𝒙(𝒔, 𝑡),𝒙(𝒔′, 𝑡′)) = 𝑪(𝒔,𝒔′)𝑪(𝑡, 𝑡′),
meaning ha he spa ial and empo al co a iance models can be i ed
independen ly and ha he spa io- empo al in e ac ion is no con-
side ed. The assump ions o s a iona i y o sepa abili y o en lead
o un ealis ically simple models ha hence p oduce nonop imal e-
sul s unde nons a iona y o nonsepa able da a. Accoun ing complex
nonsepa able and nons a iona y co ela ion s uc u es is complica ed
al eady in he uni a ia e case, o which an o e iew can be ound
in Chen, Gen on, and Sun (2021). Fo mul i a ia e da a, he ask is
e en mo e demanding and compu a ionally challenging as he c oss-
dependencies be ween he a iables ha e o be aken in o accoun .
h ps://doi.o g/10.1016/j.neune .2024.106774
Recei ed 6 June 2024; Recei ed in e ised o m 11 Sep embe 2024; Accep ed 29 Sep embe 2024
Neu al Ne wo ks 181 (2025) 106774
A ailable online 9 Oc obe 2024
0893-6080/© 2024 The Au ho s. Published by Else ie L d. This is an open access a icle unde he CC BY license (
h p://c ea i ecommons.o g/licenses/by/4.0/ ).
M. Sipilä e al.
Fo mo e de ails o complexi y o nons a iona y co a iance unc ions
o mul i a ia e spa io- empo al da a, see Po cu, Fu e , and Nychka
(2021), Sal ana and Gen on (2020).
Ano he app oach o simpli y he modelling is o assume ha he
obse a ions a e composed o 𝑃la en , mu ually independen com-
ponen s (ICs) 𝒛(𝒔, 𝑡) ∶= 𝒛∈R𝑃 h ough some mixing en i onmen .
The main mo i a ion o assuming he ICs is, ha i he la en com-
ponen s 𝒛a e eco e ed, hey can be modelled uni a ia ely making
o example nons a iona i y much easie o accoun o . Being able
o model componen s uni a ia ely is especially desi able in spa io-
empo al se ings, whe e mul i a ia e modelling is highly demanding
and compu a ionally challenging as discussed p e iously. Addi ionally,
he ICs may e eal some meaning ul pa e ns and s uc u es in he
obse ed da a ha can lead o new insigh s o he phenomenon o
in e es . A linea blind sou ce sepa a ion (BSS) (Comon & Ju en,2010)
is a popula app oach o eco e he la en componen s 𝒛. In linea
BSS, i is assumed ha he mixing en i onmen is linea and usually
also ha 𝑆=𝑃meaning ha a 𝑃- a ia e obse able andom ec o
𝒙= (𝑥1,…, 𝑥𝑃)⊤is gene a ed as
𝒙=𝑨𝒛,(2)
whe e 𝑨is an in e ible 𝑃×𝑃mixing ma ix and 𝒛= (𝑧1,…, 𝑧𝑃)⊤a e
he 𝑃- a ia e la en componen s. The objec i e is o eco e 𝑨and 𝒛
using only 𝒙and a ying assump ions on 𝒛depending on he me hod
used. Fo example, spa ial BSS (SBSS) (Bachoc, Gen on, No dhausen,
Ruiz-Gazen, & Vi a,2020;No dhausen, Oja, Filzmose , & Reimann,
2015), which is a me hod o mul i a ia e s a iona y spa ial da a,
assumes spa ially s a iona y 𝒛, and a nons a iona y ex ension o SBSS,
spa ial nons a iona y sou ce sepa a ion (SNSS) (Muehlmann, Bachoc, &
No dhausen,2022), assumes 𝒛 o ha e nons a iona y spa ial co a iance
unc ion. SBSS and SNSS eco e he la en componen s by join ly
diagonalizing wo o mo e momen -based ma ices. Recen ly, SBSS
was also ex ended o s a iona y spa io- empo al da a yielding spa io-
empo al BSS (STBSS) (Muehlmann, De Iaco, & No dhausen,2023).
A d awback o STBSS and linea BSS me hods in gene al is ha hey
assume linea mixing (2) which may be oo es ic i e assump ion
o many eal li e applica ions. Simila ly, he assump ion ha he e
a e as many la en ‘‘signal’’ componen s as obse ed a iables is in
many applica ions undesi able and i is o en hoped ha he e a e
signi ican ly ewe signals. This assump ion is o en needed simply due
o he lack o ools o es ima ing he co ec numbe o signals. Finally,
STBSS is de eloped only o s a iona y da a, and o ou knowledge,
he e a e no spa io- empo al al e na i es a ailable o nons a iona y
da a cases.
Recen ad ancemen s in unsupe ised deep lea ning, such as a ia-
ional au oencode s (VAEs) (Kingma & Welling,2013) and gene a i e
ad e sa ial ne wo ks (GANs) (Good ellow e al.,2020), ha e inc eased
in e es o de eloping nonlinea BSS me hods, whe e he mixing unc-
ion is no es ic ed o be linea , bu can be any injec i e unc ion
𝒇∶R𝑃→R𝑆, which gene a es he obse ed da a 𝒙as
𝒙=𝒇(𝒛).(3)
The objec i e is hen o iden i y an unmixing ans o ma ion 𝒒∶R𝑆→
R𝑃, which e u ns he la en componen s 𝒛as
𝒛=𝒒(𝒙)
based on he obse a ions 𝒙only. Wi hou any addi ional assump ions
on he mixing ans o ma ions 𝒇o on he la en componen s 𝒛, he
model is uniden i iable as he e exis s in ini e nonlinea ans o ma-
ions o gene a e mu ually independen componen s om he obse -
a ions (Hy ä inen & Pajunen,1999). Fo his eason bo h VAEs and
GANs, in gene al, su e om he uniden i iabili y issue. Howe e , in
many ecen s udies (Häl ä & Hy ä inen,2020;Hy ä inen & Mo ioka,
2016,2017;Hy ä inen, Sasaki, & Tu ne ,2019;Khemakhem, Kingma,
Mon i, & Hy ä inen,2020) he iden i iabili y ha e been achie ed by
in oducing some cons ain s on he dis ibu ion o he la en com-
ponen s 𝒛. The main assump ion leading o iden i iabili y is ha he
componen s 𝑧1,…, 𝑧𝑃a e s a is ically dependen on a 𝑚-dimensional
auxilia y a iable 𝒖, and ha he componen s a e condi ionally inde-
penden yielding he join dis ibu ion 𝑝(𝒛|𝒖) = ∏𝑃
𝑖=1 𝑝(𝑧𝑖|𝒖). In p e ious
s udies, he main ocus has been in ime se ies da a o which se e al
algo i hms and examples o auxilia y a iables exis in he li e a u e.
In case o s a iona y ime se ies da a, pe mu a ion con as i e lea ning
(PCL) (Hy ä inen & Mo ioka,2017) can be used, o which 𝒖is usually
gi en by one o mo e p e ious obse a ions in ime. Fo nons a iona y
ime se ies da a, he a ailable me hods a e ime con as i e lea ning
(TCL) (Hy ä inen & Mo ioka,2016), hidden Ma ko nonlinea ICA
(HM-NICA) (Häl ä & Hy ä inen,2020) and empo al iden i iable VAE
(iVAE) (Khemakhem e al.,2020), all o which use he ime segmen o
he obse a ion as 𝒖. Gene alized con as i e lea ning (Hy ä inen e al.,
2019) and nonlinea ICA wi h swi ching linea dynamical sys ems (𝛥-
SNICA) (Häl ä, Le Co , Lehé icy, So, Zhu, Gassia , & Hy ä inen,2021)
can accoun bo h s a iona y and nons a iona y ime se ies. In HM-NICA
and 𝛥-SNICA, he auxilia y a iables 𝒖a e no explici ly p o ided by he
use , bu hey a e ins ead assumed o be hidden s a es ha a e modelled
simul aneously by he algo i hms. In Sipilä, No dhausen and Taskinen
(2024), iVAE was s udied u he and ex ended o nons a iona y spa ial
se ing, whe e spa ial segmen a ion was used as 𝒖.Häl ä e al. (2021)
also in oduced a s uc u ed nonlinea ICA amewo k which could
be used o spa ial p ocess, bu did no p o ide any algo i hm o
he me hod. In addi ion o hese mo e gene al iden i iable nonlinea
BSS me hods, many o he deep lea ning based BSS me hods (Ansa i,
Ala any, Alnajja , Kha e , Mahmoud, Al-Jumeily, & Hussain,2023)
ha e been in oduced o mainly acous ic signal speci ic se ings, whe e
only se ial dependence is p esen . Howe e , he spa io- empo al da a as
discussed in his pape is special in sense ha in he empo al domain
he e is na u al di ec ion o dependence (pas - u u e) while in he
spa ial domain such di ec ion is missing and he dependence is usually
conside ed as a unc ion o he dis ance be ween wo poin s. Hence,
none o he p e ious me hods a e di ec ly applicable o op imal o
such spa io- empo al da a. No e ha egula ly spaced spa io- empo al
da a is o en ep esen ed as enso da a, and BSS me hods de eloped
o such speci ic cases, like hose in Vi a and No dhausen (2017), a e
gene ally no applicable o b oade spa io- empo al se ings.
In pa icula , we a e in e es ed in iVAE, which u ilizes he auxil-
ia y a iable o make VAE iden i iable. iVAE is capable o es ima ing
nonlinea injec i e mixing unc ion, meaning ha i allows he la en
dimension 𝑃 o be less o equal o he obse ed dimension 𝑆. Howe e ,
he la en dimension 𝑃has o be es ima ed be o ehand, and cu en ly
he nonlinea BSS amewo k lacks me hods o he la en dimension
es ima ion.
In his pape , iVAE is ex ended o nons a iona y spa io- empo al
se ing by in oducing h ee no el app oaches o cons uc he auxilia y
a iables. The p oposed me hods add ess wo key limi a ions o p e i-
ous STBSS app oaches: hey accommoda e nonlinea mixing unc ions
and allow o mo e obse ed a iables han la en componen s. Mo e-
o e , he de eloped me hods a e sui able o nons a iona y da a, unlike
ea lie STBSS me hods, which ely on he assump ion o s a iona i y.
The h ee de eloped me hods, coo dina e based, segmen a ion based
and adial basis unc ion based iVAE algo i hms, a e s udied using
comp ehensi e simula ion s udies o ind how a ious ypes o nons a-
iona i y a ec he pe o mance o he me hods. The bes pe o ming
me hod, adial basis unc ion based iVAE, is illus a ed in eal li e
me eo ological applica ion whe e he eco e ed la en componen s a e
in e p e ed, and a new p ocedu e o accoun o nons a iona i y in
modelling and p edic ing mul i a ia e da a is demons a ed. Mo eo e ,
nonlinea BSS amewo k cu en ly lacks me hods o es ima ing he
la en dimension 𝑃, which is a c ucial ask in o de o eco e he ue
la en componen s and o ob ain as low dimensional ep esen a ion o
he da a as possible wi hou losing much in o ma ion. The e o e, wo
al e na i e p ocedu es o la en dimension es ima ion a e in oduced.
To conclude, he main con ibu ions o his pape a e:
Neu al Ne wo ks 181 (2025) 106774
2
M. Sipilä e al.
Fig. 1. Schema ic ep esen a ions o VAE (le ) and iVAE ( igh ) models. Fo VAE, he lowe bound o he da a log likelihood (ELBO) is o med o 𝒙,𝒙′,𝒛′,𝝁𝒛|𝒙and 𝝈𝒛|𝒙. In
iVAE, ELBO has in addi ion 𝝁𝒛|𝒖and 𝝈𝒛|𝒖which a e p o ided by he auxilia y unc ion. The la en componen s a e ob ained as 𝝁𝒛|𝒙,𝒖.
1. Ex ending iVAE o he nons a iona y spa io- empo al se ing
by p oposing h ee no el app oaches o cons uc ing auxilia y
a iables.
2. In oducing wo al e na i e p ocedu es o la en dimension es-
ima ion.
3. De eloping a new iVAE-based me hod o add essing nons a-
iona i y in he modelling and p edic ion o spa io- empo al
da a.
The es o he pape is o ganized as ollows. In Sec ion 2we e iew
basic heo y behind VAE and iVAE, and discuss he iden i iabili y, a e
which he spa io- empo al iVAE ex ensions a e in oduced in Sec ion 3.
In Sec ion 4, he in oduced me hods a e compa ed using simula ion
s udies, and wo al e na i e la en dimension es ima ion me hods a e
s udied. Finally, Sec ion 5shows a eal da a example and Sec ion 6
concludes he pape .
2. Va ia ional au oencode s and iden i iabili y
Le 𝒙∈R𝑆be an obse able andom ec o and 𝒛∈R𝑃,𝑃≤𝑆, be
a la en andom ec o , i.e., a sou ce ec o . Va ia ional au oencode s
(VAE) (Kingma & Welling,2013) assume ha he obse ed da a a e
gene a ed om a deep la en a iable model wi h he s uc u e
𝑝∗(𝒙,𝒛) = 𝑝∗(𝒙|𝒛)𝑝∗(𝒛),
whe e 𝑝∗is a ue, unknown gene a i e dis ibu ion, 𝒛∼𝑝∗(𝒛)and
𝒙∼𝑝∗(𝒙|𝒛). The dis ibu ion o he obse ed da a is hen ob ained as
𝑝∗(𝒙) = ∫𝑝∗(𝒙,𝒛)𝑑𝒛.
VAE consis s o an encode 𝒈(𝒙)and a decode 𝒉(𝒛), which a e pa-
ame e ized by deep neu al ne wo ks wi h pa ame e s 𝜽= (𝜽⊤
𝒈,𝜽⊤
𝒉)⊤.
The encode maps he obse ed da a o mean ec o 𝝁𝒛|𝒙∈R𝑃and
a iance ec o 𝝈𝒛|𝒙∈R𝑃, which a e used o sample a new la en
ep esen a ion 𝒛′by applying he epa ame iza ion ick (Kingma &
Welling,2013). The decode ans o ms he la en ep esen a ion 𝒛′
back o he obse able da a 𝒙′. The VAE amewo k allows e ec i e
op imiza ion o he pa ame e s 𝜽so ha a e op imiza ion we ha e
ha
𝑝𝜽(𝒙) ≈ 𝑝∗(𝒙).
The VAE amewo k lea ns he ull gene a i e model 𝑝𝜽(𝒙,𝒛) =
𝑝𝜽(𝒙|𝒛)𝑝𝜽(𝒛)and a a ia ional app oxima ion 𝑞𝜽(𝒛|𝒙)o he pos e io
dis ibu ion 𝑝𝜽(𝒛|𝒙)by maximizing he lowe bound o he da a log-
likelihood, o e idence lowe bound (ELBO), de ined as
(𝜽|𝒙)≥𝐸𝑞𝜽(𝒛|𝒙)(log 𝑝𝜽(𝒙|𝒛) + log 𝑝𝜽(𝒛) − log 𝑞𝜽(𝒛|𝒙))
wi h espec o he pa ame e ec o 𝜽. The p oblem howe e is ha
he model is no iden i iable, meaning ha e en hough we ha e a good
es ima e o he ma ginal dis ibu ion 𝑝∗(𝒙), he e is no gua an ee ha
𝑝𝜽(𝒙,𝒛) ≈ 𝑝∗(𝒙,𝒛). Mo e o mally, he model is iden i iable i o all
(𝒙,𝒛)i holds ha
∀(𝜽,𝜽′) ∶ 𝑝𝜃(𝒙) = 𝑝𝜃′(𝒙)⟹𝑝𝜽(𝒙,𝒛) = 𝑝𝜽′(𝒙,𝒛).
This means ha i we ind a pa ame e ec o 𝜽 o which 𝑝𝜃(𝒙) = 𝑝∗(𝒙),
we also ha e ha 𝑝𝜽(𝒙,𝒛) = 𝑝∗(𝒙,𝒛). This leads o he ac ha we
ha e ound he co ec sou ce densi y dis ibu ion 𝑝𝜽(𝒛) = 𝑝∗(𝒛)and
co ec condi ional dis ibu ions 𝑝𝜽(𝒙|𝒛) = 𝑝∗(𝒙|𝒛)and 𝑝𝜽(𝒛|𝒙) = 𝑝∗(𝒛|𝒙).
The whole VAE model is illus a ed in Fig. 1.
In he nonlinea BSS amewo k, he iden i iabili y has been ecen ly
achie ed by assuming ha he la en sou ces 𝒛ha e a condi ional
dis ibu ion 𝑝(𝒛|𝒖), whe e 𝒖∈R𝑚is an auxilia y a iable. The auxilia y
a iable can o example be p e ious obse a ions in ime (Hy ä inen
& Mo ioka,2017) o cu en ime index (Häl ä & Hy ä inen,2020;
Hy ä inen & Mo ioka,2016;Hy ä inen e al.,2019). Simila ly, by
assuming ha he ue la en gene a ing model has he o m
𝑝∗(𝒙,𝒛|𝒖) = 𝑝∗(𝒙|𝒛)𝑝∗(𝒛|𝒖),(4)
he iden i iabili y can be achie ed in he VAE amewo k, yielding iden-
i iable VAE (iVAE) (Khemakhem e al.,2020). In iVAE, he dis ibu ion
𝑝∗(𝒙|𝒛)is de ined as
𝑝∗(𝒙|𝒛) = 𝑝∗
𝝐(𝒙−𝒇(𝒛)),
which means ha 𝒙can be decomposed in o 𝒙=𝒇(𝒛) + 𝝐, whe e
𝝐is an independen noise ec o wi h densi y 𝑝𝝐. Assuming he non-
noisy nonlinea BSS model (3), he dis ibu ion 𝑝𝝐can be modelled
wi h Gaussian dis ibu ion wi h in ini esimal a iance. The unc ion
𝒇∶R𝑃→R𝑆is an injec i e, bu possibly nonlinea unc ion. The
condi ional dis ibu ion o la en sou ces 𝒛is assumed o be a pa o
he exponen ial amily, ha is,
𝑝𝑻,𝝀(𝒛|𝒖) =
𝑃
∏
𝑖=1
𝑄𝑖(𝑧𝑖)
𝑍𝑖(𝒖)exp [𝑘
∑
𝑗=1
𝑇𝑖,𝑗 (𝑧𝑖)𝜆𝑖,𝑗 (𝒖)],(5)
whe e 𝑄𝑖(𝑧𝑖)is a base measu e, 𝑍𝑖(𝒖)is a no malizing cons an , 𝑻𝑖(𝑧𝑖) =
(𝑇𝑖,1(𝑧𝑖),…, 𝑇𝑖,𝑘(𝑧𝑖))⊤con ains su icien s a is ics, and 𝝀𝑖(𝒖)=(𝜆𝑖,1(𝒖),…,
𝜆𝑖,𝑘(𝒖))⊤con ains he pa ame e s depending on 𝒖. The dimension 𝑘o
each su icien s a is ic 𝑻𝑖(𝑧𝑖)and 𝝀𝑖(𝒖)is assumed o be ixed. The
la en componen s 𝒛a e iden i iable up o pe mu a ion and signed
scaling unde some gene ally mild condi ions on he mixing unc ion
𝒇, he su icien s a is ics 𝑻and he auxilia y a iable 𝒖. In his s udy,
we cons uc iVAE assuming Gaussian la en componen s. Then, o
Neu al Ne wo ks 181 (2025) 106774
3
M. Sipilä e al.
iden i iabili y, he a iances o he la en componen s a e equi ed
o a y enough based on he auxilia y a iable 𝒖and he mixing
unc ion 𝒇is equi ed o ha e con inuous pa ial de i a i es. The exac
iden i iabili y condi ions can be ound in Khemakhem e al. (2020).
The iVAE model is simila o he egula VAE model wi h he
excep ion ha iVAE has an addi ional auxilia y unc ion 𝒘(𝒖)and i s
pa ame e s 𝜽𝒘 o be es ima ed, and he encode 𝒈(𝒙,𝒖) akes bo h,
obse a ions 𝒙and he auxilia y a iables 𝒖as an inpu . The auxilia y
unc ion 𝒘maps 𝒖in o 𝝁𝒛|𝒖and 𝝈𝒛|𝒖, which a e used o calcula e he
loss. Fo iVAE model, ELBO is ob ained as
(𝜽|𝒙,𝒖)≥𝐸𝑞𝜽(𝒛|𝒙,𝒖)(log 𝑝𝜽𝒉(𝒙|𝒛) + log 𝑝𝜽𝒘(𝒛|𝒖) − log 𝑞𝜽𝒈(𝒛|𝒙,𝒖)),
whe e log 𝑝𝜽𝒉(𝒙|𝒛)con ols he econs uc ion accu acy and log
𝑝𝜽𝒘(𝒛|𝒖) − log 𝑞𝜽𝒈(𝒛|𝒙,𝒖)is a Kullback–Leible (KL) di e gence be ween
𝑝𝜽𝒘(𝒛|𝒖)and 𝑞𝜽𝒈keeping he dis ibu ions 𝑝𝜽𝒘and 𝑞𝜽𝒈as simila
as possible. ELBO is maximized o ob ain he es ima ed pa ame e s
𝜽= (𝜽⊤
𝒈,𝜽⊤
𝒉,𝜽⊤
𝒘)⊤. The dis ibu ions 𝑝𝜽𝒉,𝑝𝜽𝒘and 𝑞𝜽𝒈a e ypically
Gaussian dis ibu ions, whe e he unc ions 𝒉,𝒘and 𝒈gi e he mean
and a iance pa ame e s o he dis ibu ions. The dis ibu ions can
also be o he han Gaussian as long as he esampling can be done
using he epa ame iza ion ick o allow he backp opaga ion go
h ough he esampling node. Then, he unc ions 𝒉,𝒘and 𝒈do no
gi e mean and a iance, bu he pa ame e s acco ding he chosen
dis ibu ions. As we assume Gaussian la en da a in his pape , we
ha e 𝑝𝜽𝒘=𝑁(𝒛|𝝁𝒛|𝒖,diag(𝝈𝒛|𝒖)),𝑞𝜽𝒈=𝑁(𝒛|𝝁𝒛|𝒙,𝒖,diag(𝝈𝒛|𝒙,𝒖)) and
𝑝𝜽𝒉=𝑁(𝒙|𝒙′, 𝛽𝑰), whe e 𝛽 > 0is a small cons an as 𝑝𝜽𝒉(𝒙|𝒛)es ima es
he ue dis ibu ion 𝑝∗(𝒙|𝒛)wi h in ini esimal a iance. By inc easing
𝛽, he weigh o he econs uc ion accu acy in he ELBO dec eases.
Based on ou empi ical in es iga ions, we use 𝛽= 0.02 which p o ides
a good balance be ween he econs uc ion e o and KL di e gence
in ELBO, and leads o good pe o mance. Fig. 1 has ep esen a ions o
bo h VAE and iVAE models and illus a es he di e ence be ween he
models.
3. iVAE o STBSS
To pe o m nonlinea spa io- empo al blind sou ce sepa a ion using
iVAE, he auxilia y a iables 𝒖mus be selec ed app op ia ely. The
main assump ion o iden i iabili y in spa io- empo al se ing is ha
he a iances o he la en componen s a e a ying in space and/o
in ime. This assump ion is me by assuming ha he la en com-
ponen s a e second-o de nonhomogeneous, meaning ha he second
momen o he ma ginal dis ibu ion 𝑝(𝑧𝑖)is no in a ian wi h espec
o he loca ion shi in space and/o in ime. In addi ion, he la en
componen s a e allowed o be i s -o de nonhomogeneous, meaning
ha he componen s can ha e noncons an spa io- empo al end. The
auxilia y a iables a e cons uc ed in a way ha he auxilia y unc-
ion 𝒘is capable o lea n and es ima e he mean and he a iance
ec o s o he loca ion o he co esponding mul i a ia e obse a ion.
We p opose h ee spa io- empo al iVAE me hods; a nai e coo dina e
based me hod, a segmen a ion based me hod ex ended om spa ial
iVAE (Sipilä e al.,2024) and a adial basis unc ion based me hod
u ilizing ideas o Nag, Sun, and Reich (2023). Each o he h ee me hods
cons uc he auxilia y da a di e en ly based on he spa io- empo al
loca ion o he obse a ion. No ice ha al hough in many op ions
below he auxilia y a iables a e cons uc ed sepa a ely o spa ial
coo dina es and empo al coo dina es, he auxilia y unc ions can s ill
lea n complex spa io- empo al in e ac ions in he mean and in he a i-
ance as he auxilia y unc ions a e modelled by deep neu al ne wo ks.
Fu he mo e, e en hough he me hods o cons uc ing he auxilia y
da a a e de ined he e o spa ial dimension 𝐷= 2, he same ideas apply
also o highe 𝐷.
The app oaches p esen ed he e a e well scalable in e ms o com-
pu a ion ime. The compu a ion ime g ows sublinea ly wi h espec o
he sample size 𝑛(as ewe aining epochs a e ypically needed wi h
la ge da ase s) and linea ly wi h espec o he dimensions o obse ed
da a, la en da a and auxilia y da a. Howe e , i he dimension o he
auxilia y da a and sample sizes a e la ge, he memo y usage may g ow
unless he auxilia y da a a e o med ba ch-wise. An in-dep h analysis
o compu a ional complexi y is p o ided in Appendix A.
3.1. Coo dina e based algo i hm
In coo dina e based iVAE, he p ep osessed coo dina es a e used
di ec ly as auxilia y a iable. The p ep osessed coo dina es a e ob-
ained by applying min–max no maliza ion o each dimension. The
p ep osessed coo dina es a e hen
𝑠1=𝑠1−𝑠min
1
𝑠max
1−𝑠min
1
, 𝑠2=𝑠2−𝑠min
2
𝑠max
2−𝑠min
2
and
𝑡=𝑡−𝑡min
𝑡max −𝑡min ,
whe e 𝑠min
1, 𝑠min
2and 𝑡min a e he minimum coo dina es o he loca ions
o he obse a ions, and 𝑠max
1, 𝑠max
2and 𝑡max a e he maximum coo -
dina es o he loca ions o he obse a ions. The algo i hm wi h he
p ep osessed coo dina es, 𝒖(𝒔, 𝑡)=(𝑠1, 𝑠2,
𝑡)⊤, as auxilia y a iable is
deno ed by iVAEc.
3.2. Segmen a ion based algo i hm
In segmen a ion based iVAE, a spa io- empo al segmen a ion is used
as an auxilia y a iable. The spa io- empo al segmen a ion means ha
he domain ×is di ided in o 𝑚nonin e sec ing segmen s 𝑖∈×
so ha 𝑖∩𝑗= ∅ o all 𝑖≠𝑗,𝑖, 𝑗 = 1,…, 𝑚, and ∪𝑚
𝑖=1𝑖=×. By
using an indica o unc ion 1, he auxilia y a iable o he obse a ion
𝒙(𝒔, 𝑡)can be w i en as 𝒖(𝒔, 𝑡) = (1((𝒔, 𝑡) ∈ 1),…,1((𝒔, 𝑡) ∈ 𝑚)))⊤,
whe e 1((𝒔, 𝑡) ∈ 𝑖) = 1, i he loca ion (𝒔, 𝑡)is wi hin he segmen
𝑖, and o he wise 1((𝒔, 𝑡) ∈ 𝑖) = 0. This esul s in o 𝑚-dimensional
s anda d basis ec o , whe e he alue 1 gi es he spa io- empo al
segmen in which he loca ion o he obse a ion belongs.
I he spa io- empo al domain is la ge and small segmen s a e used,
he dimension 𝑚o he auxilia y a iable becomes e y la ge. To
lowe he dimension, he spa ial and empo al segmen a ions can be
conside ed sepa a ely. This means ha he auxilia y da a is composed
o 𝑚𝑆spa ial segmen s 𝑖∈and 𝑚𝑇 empo al segmen s 𝑖∈so
ha 𝑖∩𝑗= ∅ o all 𝑖≠𝑗,𝑖, 𝑗 = 1,…, 𝑚𝑆,∪𝑚𝑆
𝑖=1𝑖=,𝑖∩𝑗= ∅ o
all 𝑖≠𝑗,𝑖, 𝑗 = 1,…, 𝑚𝑇, and ∪𝑚𝑇
𝑖=1𝑖=. Then, he auxilia y a iable
o he obse a ion 𝒙(𝒔, 𝑡)is 𝒖(𝒔, 𝑡) = (1(𝒔∈1),…,1(𝒔∈𝑚𝑆),1(𝑡∈
1),…,1(𝑡∈𝑚𝑇)))⊤. The auxilia y a iable is (𝑚𝑆+𝑚𝑇)-dimensional
and has wo nonze o en ies o each obse a ion. The dimension can
be educed e en u he by conside ing also he 𝑥-axis and 𝑦-axis o
he spa ial domain sepa a ely. Segmen a ion based auxilia y a iables
a e illus a ed in Fig. 2, in which spa ial and empo al segmen a ions
a e conside ed sepa a ely. We deno e he algo i hm wi h all dimensions
segmen ed sepa a ely as iVAEs1, wi h space and ime segmen ed sep-
a a ely as iVAEs2, and wi h spa io- empo al segmen a ion as iVAEs3,
espec i ely.
3.3. Radial basis unc ion based algo i hm
In adial basis unc ion based iVAE, he auxilia y a iable is de ined
using adial basis unc ions (see e.g. Has ie, Tibshi ani, F iedman, and
F iedman (2009)). The idea is ha wi h la ge numbe o app op ia e a-
dial basis unc ions, he model inco po a es much mo e spa io- empo al
in o ma ion han by using he coo dina es only. Simila ideas ha e been
used ecen ly in Chen, Li, Reich, and Sun (2024), Nag e al. (2023) o
pe o m deep lea ning based spa ial and spa io- empo al p edic ing by
using he spa ial and spa io- empo al loca ions ans o med in o adial
basis unc ions as inpu o deep neu al ne wo ks. Following Nag e al.
(2023), we ans o m spa ial and empo al loca ions sepa a ely in o
adial basis unc ions. Le {𝒐
𝑖},𝑖= 1,…, 𝐾, whe e 𝒐
𝑖∈, be a se
o spa ial node poin s, and le {𝑜
𝑖},𝑖= 1,…, 𝐾, whe e 𝑜
𝑖∈, be a
Neu al Ne wo ks 181 (2025) 106774
4
M. Sipilä e al.
Fig. 2. Illus a ions o auxilia y a iables o segmen a ion based iVAE (iVAEs2) (a) and adial basis unc ion based iVAE (b). The op igu e o (a) illus a es spa ial segmen a ion,
whe e each segmen has size 20 ×20 p oducing 25 spa ial segmen s, and he bo om igu e illus a es empo al segmen a ion, whe e each segmen has 5 ime poin s, p oducing
20 empo al segmen s. In (b), he black lines in he op igu e a e he no malized 𝑥and 𝑦 alues a 1∕(𝐻+ 2) = 1∕4 and 1∕(𝐻+ 2) + 1∕𝐻= 3∕4 o med by esolu ion le el 𝐻= 2,
and he ed poin s ep esen he p oduced spa ial node poin s. The blue poin s ep esen empo al node poin s o esolu ion le el 𝐺= 5. Spa ial and empo al Gaussian adial
basis unc ions a e illus a ed in (c) and (d), espec i ely. The adial basis unc ions a e unc ions o dis ance be ween node poin and a spa ial o empo al loca ion.
se o empo al node poin s. The pa ame e 𝜁is a scale pa ame e . The
spa ial and empo al adial basis unc ions a e gi en as
𝑣(𝒔;𝜁, 𝒐
𝑖) = 𝑣(‖𝒔−𝒐
𝑖‖∕𝜁)and 𝑣(𝑡;𝜁, 𝑜
𝑖) = 𝑣(|𝑡−𝑜
𝑖|∕𝜁),
whe e 𝑣is a ke nel unc ion such as he Gaussian ke nel 𝑣𝐺(𝑑) = 𝑒−𝑑2,
o one o he Wendland ke nels (Wendland,1995) such as
𝑣𝑊(𝑑) = {(1 − 𝑑)6(35𝑑2+ 18𝑑+ 3)∕3, 𝑑 ∈ [0,1]
0,o he wise.
Following Nychka, Bandyopadhyay, Hamme ling, Lindg en, and Sain
(2015), we use a mul i- esolu ion app oach o o m he spa ial and
empo al adial basis unc ions. Each esolu ion le el is composed o i s
own numbe o e enly spaced node poin s and own scaling pa ame e .
A low le el esolu ion wi h small numbe o node poin s and la ge alue
o he scaling pa ame e aims o cap u e la ge-scale spa ial o empo al
dependencies, while a high le el esolu ion wi h many node poin s and
small scaling pa ame e aims o ind ine de ails o he dependence
s uc u e.
To o m he adial basis unc ions, we i s p ep ocess he spa ial and
empo al loca ions o ange [0,1] using min–max no maliza ion. A 𝐻-
le el spa ial esolu ion is o med o e enly spaced g id o node poin s
{𝒐
𝑖}wi h spacing 1∕𝐻and an o se 1∕(𝐻+ 2) be o e he i s node
poin , meaning ha 𝐻-le el esolu ion has he node poin s {(𝑖, 𝑗) ∶ 𝑖, 𝑗 ∈
{1
𝐻+2 ,1
𝐻+2 +1
𝐻,…,1 − 1
𝐻+2 }}. Fo example 2-le el spa ial esolu ion
is hen composed o he node poin s {(𝑖, 𝑗) ∶ 𝑖, 𝑗 ∈ {0.25,0.75}}.
Simila ly, a 𝐺-le el empo al esolu ion is o med o e enly spaced one
dimensional node poin s {𝑜
𝑖}wi h spacing 1∕𝐺and an o se 1∕(𝐺+2),
meaning ha 𝐺-le el empo al esolu ion is composed o he node
poin s {1
𝐺+2 ,1
𝐺+2 +1
𝐺,…,1 − 1
𝐺+2 }. As he scaling pa ame e s 𝜁𝐻and
𝜁𝐺, o spa ial and empo al adial basis unc ions, we use 𝜁𝐻=1
2.5𝐻
ollowing Nychka e al. (2015) and 𝜁𝐺=|𝑜
1−𝑜
2|
√2 ollowing Nag e al.
(2023). Spa ial and empo al node poin s and adial basis unc ions a e
illus a ed in Fig. 2 o 𝐻= 2 spa ial esolu ion, p oducing 4 spa ial
adial basis unc ions, and 𝐺= 5, p oducing 5 empo al adial basis
unc ions. In p ac ice, mul iple spa ial and empo al esolu ion le els,
such as 𝐻= (𝐻1, 𝐻2) = (2,9), and 𝐺= (𝐺1, 𝐺2, 𝐺3) = (9,17,37),
should be used o cap u e bo h la ge scale and ine dependencies. An
ad an age o using adial basis unc ions as auxilia y a iables ins ead
o spa io- empo al segmen s is ha by using adial basis unc ions,
iVAE’s auxilia y unc ion p o ides a smoo h spa io- empo al end and
a iance unc ions, which can be used la e o u he analysis such as
o p edic ion pu poses. The adial basis unc ion based iVAE is deno ed
as iVAE in he es o he pape .
4. Simula ion s udies
The aim o his sec ion is o demons a e and compa e he pe o -
mances o iVAE me hods using simula ion s udies and o disco e how
a ious ypes o nons a iona i y in a iance a ec he pe o mance.
The sec ion begins wi h a sho e iew o some common p ocedu es
o gene a ing spa io- empo al da a and ways o in oduce nons a ion-
a i y in i . The emainde o he sec ion con ains a la ge simula ion
s udy showing he unmixing pe o mances o he iVAE me hods unde
di e en ypes o nons a iona i y scena ios, and hen in oduces wo
me hods o es ima e he numbe o la en signals. Finally, he la en
dimension es ima ion me hods a e illus a ed in a small simula ion
s udy. All simula ions can be ep oduced using R 4.3.0 (R Co e Team,
2023) oge he wi h R packages as ICA (Ma chini, Hea on, & Ripley,
2021), SpaceTimeBSS (Muehlmann, Piccolo o, Cappello, De Iaco, &
No dhausen,2022) and Nonlinea BSS. Nonlinea BSS package con ains
R implemen a ions o all p oposed spa io- empo al iVAE a ian s, and
is a ailable in h ps://gi hub.com/mikasip/Nonlinea BSS. The simu-
la ions we e execu ed on he CSC Puh i clus e , a high-pe o mance
compu ing en i onmen .
Neu al Ne wo ks 181 (2025) 106774
5
M. Sipilä e al.
4.1. Nons a iona y spa io- empo al da a gene a ion
Spa io- empo al da a a e ypically composed o 𝑛𝑠spa ial loca ions
and 𝑛𝑡 empo al poin s, making he o al numbe o obse a ions 𝑛=
𝑛𝑠𝑛𝑡usually e y high. The obse a ions a e o en collec ed egula ly,
o example daily o hou ly, by some moni o ing s a ions in di e en
loca ions. This makes he obse ed da a quickly e y dense in ime bu
mo e spa se in space. To s udy he p ope ies o he models unde he
na u e o eal li e spa io- empo al da a, gene a ing la ge da ase s wi h
a ious spa io- empo al co a iance models is equi ed. In he ollowing
simula ions, we exploi a compu a ionally e icien ec o au o eg es-
si e p ocess, see o example (Papalexiou & Se inaldi,2020;Sig is ,
Künsch, & S ahel,2012;Xu & Ga doni,2018;Yan, Huang, & Gen on,
2021), and a simpli ied e sion o imp o ed la en space app oach
(ILSA) (Xu & Ga doni,2018) o gene a e nons a iona y spa io- empo al
da a.
Assume spa ial ield a ime 𝑡= 1,…, 𝑛𝑡 o be 𝜹(𝑡) = (𝛿(𝒔1, 𝑡),…,
𝛿(𝒔𝑛𝑠, 𝑡))⊤, whe e 𝒔1,…𝒔𝑛𝑠a e he spa ial loca ions in he spa io- empo al
ield. The ec o au o eg essi e p ocess can be w i en as
𝜹(𝑡) =
𝑅
∑
𝑟=1
𝜌𝑟𝑲𝑟𝜹(𝑡−𝑟) + 𝝐𝜹(𝑡),(6)
whe e 𝑟= 1,…, 𝑅 is an au o eg essi e o de , 𝜌𝑟is 𝑟 h baseline au o e-
g essi e coe icien , 𝑲𝑟is a 𝑛𝑠×𝑛𝑠spa ial ke nel ma ix de e mining
he change o empo al co ela ion wi h spa ial loca ions, and 𝜖𝜹(𝑡)is
a𝑛𝑠-dimensional noise ec o wi h co a iance 𝐶(𝜖𝜹(𝒔, 𝑡), 𝜖𝜹(𝒔′, 𝑡)) wi h
𝒔,𝒔′∈ {𝒔1,…,𝒔𝑛𝑠}.
Wi h a simpli ied e sion o ILSA, one can gene a e nons a iona y
spa io- empo al da a by using ec o au o eg essi e p ocess (6) as o -
mula ed nex . Le
𝒔(𝒔) = [𝑠1,…, 𝑠𝑑]be a 𝑑-dimensional ans o ma ion
o he o iginal coo dina e 𝒔, whe e he ans o med coo dina es 𝑠𝑖,
𝑖= 1,…, 𝑑, a e called eg esso s o la en coo dina es. Le 𝑑𝒔𝑖𝒔𝑗=
[‖𝑠1
1−𝑠2
1‖,‖𝑠1
2−𝑠2
2‖]⊤,𝑑
𝒔𝑖
𝒔𝑗= [‖𝑠1
1−𝑠2
1‖,…,‖𝑠1
𝑑−𝑠2
𝑑‖]⊤and 𝑉 o be any
s a iona y co a iance unc ion. Simpli ied ILSA has he o mula ions
𝐾𝑟{𝑖,𝑗}=||||
𝜽𝒔,𝑟 𝜽
𝒔,𝑟||||
−1
2exp (−[𝒅𝒔𝑖𝒔𝑗𝒅
𝒔𝑖
𝒔𝑗][𝜽𝒔,𝑟 𝜽
𝒔,𝑟][𝒅𝒔𝑖𝒔𝑗
𝒅
𝒔𝑖
𝒔𝑗]),
𝐶(𝜖𝜹(𝒔, 𝑡), 𝜖𝜹(𝒔′, 𝑡)) = 𝜎[
𝒔(𝒔),𝒔, 𝑡]𝜎[
𝒔(𝒔′),𝒔′, 𝑡]𝑉(𝑄𝑡)(7)
whe e 𝑄𝑡=([𝒅𝒔𝑖𝒔𝑗𝒅
𝒔𝑖
𝒔𝑗][𝜽𝒔𝜽
𝒔][𝒅𝒔𝑖𝒔𝑗
𝒅
𝒔𝑖
𝒔𝑗])1
2
and 𝜽𝒔,𝑟 =diag(𝜃𝑠1,𝑟, 𝜃𝑠2,𝑟),𝜽
𝒔,𝑟 =diag(𝜃𝑠1,𝑟,…, 𝜃𝑠𝑑,𝑟),𝜽𝒔=diag(𝜃𝑠1, 𝜃𝑠2)
and 𝜽
𝒔=diag(𝜃𝑠1,𝑟,…, 𝜃𝑠𝑑,𝑟)a e diagonal ma ices gi ing scaling pa-
ame e s o he spa ial coo dina es and o he la en coo dina es. The
unc ion 𝑄𝑡 ans o ms he o iginal coo dina es based on he scaling
pa ame e s and he la en coo dina es. By using his app oach, one can
easily in oduce complex, nons a iona y and nonsepa able co a iance
s uc u es h ough la en coo dina es
𝒔, ime a ying spa ial ke nel
ma ices 𝑲𝑟and nons a iona y a iance unc ion 𝜎. In he ollowing
simula ions, we a e mainly in e es ed in ha ing nons a iona i y in
he a iance as ha is equi ed o he iden i iabili y o he la en
componen s.
4.2. Fini e sample e iciencies
In his sec ion, ou di e en iVAE con igu a ions – egula VAE,
symme ic Fas ICA (FICA) (Hy ä inen,1999) wi h hype bolic an-
gen nonlinea i y, and STBSS – a e compa ed using simula ed spa io-
empo al da a. Al hough FICA is no designed o spa io- empo al da a
o nonlinea mixing, i is included as a linea baseline o da a wi h non-
s a iona y a iances. STBSS, de eloped o s a iona y spa io- empo al
da a and linea mixing, se es as a spa io- empo al baseline.
While he e a e se e al deep lea ning-based app oaches o nonlin-
ea BSS in he li e a u e, see Ansa i e al. (2023) o a ecen e iew,
mos lack iden i iabili y and ocus on acous ic da a, which p ima ily ex-
hibi s se ial dependence, making hem subop imal o spa io- empo al
da a. None heless, we include egula VAE as an uniden i iable deep
lea ning baseline.
The aim o he simula ions is o iden i y how he p oposed iVAE
me hods pe o m as compa ed o o he exis ing me hods unde a ious
ypes o nons a iona y spa io- empo al da a, and how he ype o
nons a iona y a ec s he pe o mance. To iden i y how he educ ion
o ei he empo al o spa ial obse a ions a ec he pe o mance o
he algo i hms, we conside h ee sample sizes composed o 𝑛𝑠spa ial
loca ions and 𝑛𝑡 empo al obse a ions o each spa ial loca ion. The
sample dimensions conside ed a e (𝑛𝑠, 𝑛𝑡) = (150,300),(𝑛𝑠, 𝑛𝑡) = (50,300)
and (𝑛𝑠, 𝑛𝑡) = (150,75) yielding 𝑛= 45000,𝑛= 15000 and 𝑛= 11250
obse a ions, espec i ely. We gene a e he la en da a 𝒛acco ding o
six di e en simula ion se ings. In some se ings he nons a iona i y is
in oduced only in ime, in some se ings only in space, and in some
se ings bo h in space and in ime. In each simula ion se ing, 𝑛𝑠spa ial
loca ions 𝒔𝑖a e sampled uni o mly in spa ial domain = [0,1] × [0,1],
and o each spa ial loca ion 𝒔𝑖,𝑛𝑡obse a ions 𝒙(𝒔𝑖, 𝑡)a e gene a ed.
The ue la en dimension is 𝑃= 5 and he dimension o he obse -
a ions is 𝑆= 8. E e y se ing is epea ed 500 imes o each sample
size and o each algo i hm. Finally, each ial is epea ed using h ee
inc easingly nonlinea mixing unc ions as desc ibed he ea e . The
i s h ee simula ion se ings a e mo e simple ones, ollowed by h ee
mo e complex ones which u ilize he ILSA amewo k. The simula ion
se ings and he mixing p ocedu e a e de ined in he ollowing. A e
in oducing he da a gene a ion o he se ings, he mo i a ion behind
each se ing is ca e ully explained.
Se ing 1. The la en spa io- empo al ield consis s o h ee clus e s
in space and i e segmen s in ime yielding 15 spa io- empo al clus e s,
each o which has hei own unique diagonal co a iance ma ix and
unique mean ec o . Fo 𝑘 h clus e , 𝑘= 1,…,15, he co a iance ma ix
is gi en as 𝑪𝑘=diag(𝜎1,𝑘,…, 𝜎5,𝑘), whe e 𝜎𝑖,𝑘,∼Uni (0.1,5) and unique
mean ec o is gi en as 𝝁𝑘= (𝜇1,𝑘,…, 𝜇5,𝑘)⊤, whe e 𝜇𝑖,𝑘 ∼Uni (−5,5),
𝑖= 1,…,5.
Se ing 2. The la en spa io- empo al ield consis s o 10 segmen s
in ime. The la en componen s a e simula ed by gene a ing i s Gaus-
sian spa ial da a wi h Ma e n co a iance unc ion using unique pa-
ame e s (𝜈𝑖, 𝜙𝑖) o each componen 𝑖= 1,…,5, and hen adding
Gaussian iid da a wi h unique co a iance ma ix and mean ec o o
each ime segmen . The Ma e n pa ame e s a e (𝜈1, 𝜙1) = (0.5,0.30),
(𝜈2, 𝜙2) = (0.1,0.25),(𝜈3, 𝜙3) = (1,0.35),(𝜈4, 𝜙4) = (2,0.20),(𝜈5, 𝜙5) =
(0.25,0.15) and he pa ame e s o he ime segmen 𝑘= 1,…,10
a e 𝝁𝑘= (𝜇1,𝑘,…, 𝜇5,𝑘)⊤, whe e 𝜇𝑖,𝑘 ∼Uni (−0.3,0.3), and 𝜮𝑘=
diag(𝜎1,𝑘,…, 𝜎5,𝑘), whe e 𝜎𝑖,𝑘,∼Uni (0,0.4),𝑖= 1,…,5.
Se ing 3. The la en spa io- empo al ield consis s o i e clus e s
in space and ollows AR1 model. In 𝑘 h clus e , 𝑘= 1,…,5, he la en
componen s 𝑧𝑖,𝑖= 1,…,5, a e gene a ed as
𝑧𝑖(𝒔, 𝑡 + 1) = 𝜌𝑖,𝑘𝑧𝑖(𝒔, 𝑡) + 𝜖𝑖,𝑘,𝑡,
𝜖𝑖,𝑘,𝑡 ∼𝑁(𝜇𝑖,𝑘, 𝜎𝑖,𝑘),
whe e 𝑡= 1,…, 𝑛𝑡− 1 and 𝑧𝑖(𝒔,1) ∼ 𝑁(𝜇𝑖,𝑘, 𝜎𝑖,𝑘). Each clus e has
unique pa ame e s 𝝆𝑘= (𝜌1,𝑘,…𝜌5,𝑘)⊤,𝝁𝑘= (𝜇1,𝑘,…𝜇5,𝑘)⊤and 𝝈𝑘=
(𝜎1,𝑘,…𝜎5,𝑘)⊤gene a ed as 𝜌𝑖,𝑘 ∼Uni (0.05,0.95),𝜇𝑖,𝑘 ∼Uni (−1,1) and
𝜎𝑖,𝑘 ∼Uni (0.1,5).
Se ings 4–6. The la en spa io- empo al ield is gene a ed using
ILSA amewo k. Each se ing has he same highly nons a iona y co-
a iance s uc u e. In addi ion, Se ing 4 has a a iance 𝜎changing in
space, Se ing 5 has a a iance changing in ime, and Se ing 6 has a
a iance changing bo h in space and in ime. The la en coo dina es
𝒔=
(𝑠1, 𝑠2)⊤a e ans o med om he spa ial coo dina es 𝒔= (𝑠1, 𝑠2)⊤by
using a swi l-like coo dina e ans o ma ion acco ding o Papalexiou,
Se inaldi, and Po cu (2021) gi en by
𝑠1= (𝑠1−𝑠∗
1)cos(𝜂exp (−( ℎ∗
𝑏swi l
)2))
− (𝑠2−𝑠∗
2)sin(𝜂exp (−( ℎ∗
𝑏swi l
)2))+𝑠∗
1,
Neu al Ne wo ks 181 (2025) 106774
6
M. Sipilä e al.
Table 1
The ILSA pa ame e s and coo dina e de o ma ion pa ame e s o Se ings 4–6.
𝜽𝒔,𝑙 𝜽
𝒔,𝑙 𝜽𝒔𝜽
𝒔𝜌1𝒔∗𝑏swi l 𝜂 𝜈 𝜙
IC1 (6,4) (7,7) (0.2,0.7) (0.7,0.2) 0.9 (0.5,0.5) 0.7 1.8𝜋0.25 0.5
IC2 (3,6) (4,7) (0.7,0.2) (0.25,0.5) 0.8 (0.7,0.7) 0.4 1.2𝜋0.2 0.9
IC3 (3,3) (6,3) (0.5,0.5) (0.7,0) 0.7 (0.3,0.3) 0.2 2𝜋0.05 1.5
IC4 (7,3) (2,6) (0.2,0.4) (0.3,0.7) 0.6 (0.7,0.3) 10.5𝜋0.1 0.25
IC5 (2,1) (6,2) (0.3,0.3) (0,0.7) 0.5 (0.3,0.7) 0.9 0.9𝜋0.15 1
𝑠2= (𝑠1−𝑠∗
1)sin(𝜂exp (−( ℎ∗
𝑏swi l
)2))
− (𝑠2−𝑠∗
2)cos(𝜂exp (−( ℎ∗
𝑏swi l
)2))+𝑠∗
2,
whe e 𝒔∗= (𝑠∗
1, 𝑠∗
2)is he cen e poin o he de o ma ion, ℎ∗=‖𝒔−𝒔∗‖
is Euclidean dis ance be ween he o iginal loca ion and he cen e
poin , 𝜂is a o a ion angle, and 𝑏swi l is a scaling pa ame e con olling
he magni ude o he swi l. Each la en componen has hei own
se o de o ma ion pa ame e s. The s a iona y co a iance unc ion 𝑉
in (7) is he Ma e n co a iance unc ion wi h pa ame e s (𝜈𝑖, 𝜙𝑖) o
all Se ings 4–6. The de o ma ion pa ame e s, ILSA pa ame e s and
Ma e n pa ame e s mu ual o Se ings 4–6 a e gi en in Table 1. The
au o eg essi e o de is 𝑅= 1 o all se ings.
In Se ing 4, we ha e 𝜎[
𝒔(𝒔),𝒔, 𝑡] = exp(𝜃𝑖
𝜎𝑠(𝑠1−0.5)), whe e 𝜃𝑖
𝜎𝑠is he
scaling pa ame e o a iance in space o 𝑖 h la en componen . This
means ha he a iance o he la en ields a y in space based on he
i s la en coo dina e. The a iance scaling pa ame e s o he la en
componen s 𝑧𝑖,𝑖= 1,…,5, a e 𝜃1
𝜎𝑠= 1,𝜃2
𝜎𝑠= 2,𝜃3
𝜎𝑠= 3,𝜃4
𝜎𝑠= −1 and
𝜃5
𝜎𝑠= −2.
In Se ing 5, he a iances o he la en ields a e changing in ime.
We se 𝜎[
𝒔(𝒔),𝒔, 𝑡] = exp(sin((𝑡+𝜃𝑖
𝜎𝑡1) + 𝜃𝑖
𝜎𝑡2)∕2), whe e 𝜃𝑖
𝜎𝑡1and 𝜃𝑖
𝜎𝑡2
a e a iance coe icien and a iance scaling pa ame e in ime o 𝑖 h
la en componen . The pa ame e s (𝜃𝑖
𝜎𝑡1, 𝜃𝑖
𝜎𝑡2) o he la en componen s
𝑧𝑖,𝑖= 1,…,5, a e (𝜃1
𝜎𝑡1, 𝜃1
𝜎𝑡2) = (50,0.1),(𝜃2
𝜎𝑡1, 𝜃2
𝜎𝑡2) = (0,0.05),(𝜃3
𝜎𝑡1, 𝜃3
𝜎𝑡2) =
(100,0.005),(𝜃4
𝜎𝑡1, 𝜃4
𝜎𝑡2) = (20,0.01) and (𝜃5
𝜎𝑡1, 𝜃5
𝜎𝑡2) = (10,0.03).
In Se ing 6, he a iances o he la en ields a e changing in space
and in ime. We se 𝜎[
𝒔(𝒔),𝒔, 𝑡] = exp(𝜃𝑖
𝜎𝑠(𝑠1−0.5)+sin((𝑡+𝜃𝑖
𝜎𝑡1)+𝜃𝑖
𝜎𝑡2)∕2).
The pa ame e s 𝜃𝑖
𝜎𝑠, 𝜃𝑖
𝜎𝑡1, 𝜃𝑖
𝜎𝑡2 o he la en ields 𝑧𝑖,𝑖= 1,…,5, a e
iden ical as in Se ings 4 and Se ing 5.
Se ing 1 has he simples la en ields by ha ing a diagonal spa io-
empo al co a iance o each la en ield. The a iance and mean a e
changing explici ly be ween he spa io- empo al clus e s as is assumed
o segmen a ion based iVAE. This se ing is a spa io- empo al a ian
o he simula ion se ing used in ime se ies con ex in Hy ä inen and
Mo ioka (2016), Khemakhem e al. (2020), whe e he la en com-
ponen s had mul iple empo al segmen s wi h unique mean and/o
a iance pa ame e s. Se ings 2 and 3 a e s ill ela i ely simple wi h
no spa io- empo al in e ac ion in he la en ields. Se ing 2 is used
o compa e pe o mances in cases whe e la en ields a e s a iona y
in space, bu a iance is changing o e ime. Se ing 3 illus a es a
scena io whe e he la en ields a e s a iona y in ime, bu he a iance
is changing o e he clus e s in space. By ha ing less a iabili y in he
a iance, Se ings 2 and 3 should be less op imal o iVAE. Se ings 4–
6 ha e la en ields wi h a complex spa io- empo al co a iance model
and s ong spa io- empo al in e ac ion. The a iance is no changing
explici ly o e segmen s, bu ins ead h ough a nons a iona y co a i-
ance unc ion. In Se ing 4, he la en ields ha e smoo hly changing
nons a iona y a iance in space, bu he a iance is s a iona y in ime,
and in Se ing 5, he a iance is nons a iona y in ime, bu s a ion-
a y in space. Se ing 6 in oduces nons a iona i y in a iance bo h in
space and in ime. Wi h Se ings 4–6 he aim is hus o ind ou how
he p esence o nons a iona i y in a iance a ec s he pe o mances
o iVAE me hods in se ings wi h mo e ealis ic and mo e complex
spa io- empo al s uc u es.
Nonlinea mixing unc ions. The mixing unc ion 𝑓𝐿is gene a ed
using mul ilaye pe cep on (MLP) ollowing Hy ä inen and Mo ioka
(2016), Hy ä inen e al. (2019), Khemakhem e al. (2020). He e 𝐿
deno es he numbe o mixing laye s used in MLP. To ob ain an
injec i e and di e en iable mixing unc ion, each laye o MLP has
𝑆= 8 hidden uni s wi h he ac i a ion unc ion 𝜔𝑖being ei he linea
o exponen ial linea uni (ELU). The ma ices 𝑩𝑖,𝑖= 1,…, 𝐿, in he
mixing p ocedu e a e no malized o ha e uni leng h ow and column
ec o s o gua an ee ha none o he independen componen s anish
in he mixing p ocess. The mixing unc ion 𝑓𝐿is de ined as
𝒇𝐿(𝒛) = {𝜔𝐿(𝑩𝐿𝒛), 𝐿 = 1,
𝜔𝐿(𝑩𝐿𝒇𝐿−1(𝒛)), 𝐿 ∈ {2,3,…},
whe e 𝑩1is a 8 ×5 ma ix and he o he ma ices, 𝑩𝑖,𝑖≠1, a e 8 ×8
ma ices. In simula ions we use linea ac i a ion 𝜔𝐿(𝑥) = 𝑥 o he las
laye and ELU ac i a ion
𝜔𝑖(𝑥) = {𝑥, 𝑥 ≥0,
exp(𝑥)−1, 𝑥 < 0,
𝑖= 1,…, 𝐿 − 1, o he o he laye s. By his p ocedu e, wi h he
numbe o mixing laye s 𝐿= 1, we ob ain 𝑆= 8 linea mix u es
o he independen componen s. When he numbe o mixing laye s
inc ease, he mix u es become inc easingly nonlinea . In simula ions,
we conside h ee di e en mixing unc ions wi h he numbe o mixing
laye s 𝐿= 1,3,5.
Model speci ica ions. All iVAE models a e se up wi h encode ,
decode and auxilia y unc ion wi h h ee hidden laye s in each. The
hidden laye s consis o 128 neu ons and leaky ec i ied linea uni
(leaky ReLU) ac i a ion (Maas, Hannun, Ng, e al.,2013). iVAEs1,
iVAEs2 and iVAEs3 use 4 ×4 spa ial segmen a ion, esul ing a g id
o 𝑚𝑆= 16 equally sized squa es. The empo al segmen a ion is done
by di iding he empo al domain o equally sized segmen s o leng h 5.
This esul s he numbe o empo al segmen s 𝑚𝑇= 60 when 𝑛𝑡= 300
and 𝑚𝑇= 15 when 𝑛𝑡= 75. Fo iVAE , we use spa ial esolu ion le els
𝐻= (2,9), and empo al esolu ion le els 𝐺= (9,17,37). The iVAE
models a e ained o 60 epochs when (𝑛𝑠, 𝑛𝑡) = (150,300), o 120
epochs when (𝑛𝑠, 𝑛𝑡) = (50,300) and o 150 epochs when (𝑛𝑠, 𝑛𝑡) =
(150,75). The numbe o epochs is inc eased when he sample size is
dec eased, as he numbe o aining s eps in each epoch is lowe o
he smalle sample size. Fo all sample sizes, he numbe o epochs a e
selec ed la ge enough o gua an ee ha he aining con e ges. All iVAE
models use lea ning a e o 0.001 wi h polynomial decay o second-
o de o e 10000 aining s eps, whe e he lea ning a e a e he i s
10000 aining s eps is 0.0001. VAE uses simila pa ame e s as iVAE,
bu i does no use any auxilia y da a o ha e an auxilia y unc ion.
The STBSS model is i ed wi h mul iple di e en ke nel se ings, and
he bes one is selec ed, which is ha ing wo spa ial ing ke nels (0,0.15)
and (0.15,0.3) and ime lag o 1. Fo mo e abou STBSS and i s ke nel
se ings, see Muehlmann e al. (2023).
Pe o mance index. To measu e he pe o mance o he me hods,
he mean co ela ion coe icien (MCC) is used ollowing he p e ious
s udies, e.g.,Häl ä and Hy ä inen (2020), Hy ä inen and Mo ioka
(2017), Hy ä inen e al. (2019), Sipilä e al. (2024). MCC is a unc-
ion o he co ela ion ma ix 𝜴=𝐶𝑜𝑟(𝒛,
𝒛)be ween he ue la en
componen s 𝒛and he es ima ed ones
𝒛. MCC is calcula ed as
MCC(𝜴) = 1
𝑃sup
𝑷∈ (𝑷abs(𝜴)),(8)
whe e is a se o all possible 𝑃×𝑃pe mu a ion ma ices, (⋅)is he
ace o a ma ix and abs(⋅)deno es aking elemen wise absolu e alues
o a ma ix. MCC ge s alues in ange [0,1], whe e he op imal alue 1
is ob ained when he es ima ed sou ces a e co ela ed pe ec ly up o
hei signs wi h he ue sou ces.
Resul s. The simula ion esul s a e p o ided in Fig. 3 o (𝑛𝑠, 𝑛𝑡) =
(150,300) and in Figs. B.8 and B.9 in he Appendix B o (𝑛𝑠, 𝑛𝑡) =
Neu al Ne wo ks 181 (2025) 106774
7
M. Sipilä e al.
Fig. 3. Mean co ela ion coe icien s o 500 ials o Se ings 1–6 o sample size wi h he numbe o spa ial loca ions 𝑛𝑠= 150 and he numbe o empo al obse a ions 𝑛𝑡= 300.
(50,300) and (𝑛𝑠, 𝑛𝑡) = (150,75), espec i ely. Based on he esul s, i
is clea ha only he iVAE me hods a e capable o eco e ing sou ces
h ough nonlinea unmixing en i onmen . The pe o mances o iVAE
me hods a e be e , when he sample size g ows, al hough he di -
e ences a e small in some se ings. The pe o mance o iVAEc is
sligh ly wo se han he pe o mances o he o he iVAE me hods in
e e y se ing. FICA pe o ms well in he linea mixing se ings, when
he la en ields do no con ain end in mean. In nonlinea se ings,
i s pe o mance d ops d ama ically in all simula ion se ings. Simila
beha iou is p esen o STBSS, al hough he pe o mance in ze o
mean se ings does no each FICA. This is no su p ising as STBSS is
de eloped o s a iona y spa io- empo al andom ields. VAE pe o ms
poo ly in almos all se ings, which is expec ed as he model is no
iden i iable.
In Se ings 1–3 all iVAE me hods ou pe o m FICA, STBSS and VAE.
In hese se ings, he bes pe o ming me hod is iVAE , ollowed by
iVAEs1, iVAEs2 and iVAEs3, espec i ely. They all pe o m e y well
unde he linea mixing, bu when he numbe o mixing laye s is
inc eased, iVAE ou pe o ms he h ee o he me hods. iVAEs1 and
iVAEs2 ha e e y simila pe o mance, and hey pe o m sligh ly be e
han iVAEs3. The pe o mance o iVAEc is wo se han pe o mances o
o he iVAE me hods, especially in Se ing 2. FICA pe o ms ela i ely
well in Se ing 3 unde he linea mixing, bu he pe o mance is poo
in o he se ings. STBSS ails o eco e he la en ields in Se ings 1–3
e en unde he linea se ing. VAE ails in Se ings 1 and 2, bu pe o ms
mode a ely in Se ing 3 unde linea mixing.
In Se ings 4–6, he bes me hod unde linea mixing is FICA, bu
i s pe o mance d ops conside ably in nonlinea se ings. In case o
nonlinea mixing, he bes me hods a e iVAEs1 and iVAEs2 and iVAE
ollowed by iVAEs3 and iVAEc, in he o de om bes o wo s . iVAEs1,
iVAEs2 and iVAE pe o m nea ly as well as FICA in linea se ing
and keep up hei good pe o mance also in nonlinea se ings. STBSS
pe o ms a he well unde linea mixing, bu is s ill wo se han FICA
and iVAE a ian s. VAE has sligh ly lowe pe o mance han STBSS
unde linea mixing, bu i also ails unde nonlinea mixing. In Se ing
5, when 𝑛𝑡= 75, he pe o mances o all he me hods d op conside ably.
This is p obably due o he ac ha in Se ing 5, he a iance is
a ying less o e he whole empo al domain when he numbe o
ime poin s is lowe . The bes pe o ming me hods, when 𝑛𝑡= 75, a e
iVAEs1 and iVAEs2 in bo h linea and nonlinea cases. In Se ing 6, he
pe o mance o iVAE me hods d op only sligh ly when he numbe o
mixing laye s is inc eased, especially when he sample size is high. The
bes pe o ming me hods o nonlinea mixing en i onmen a e iVAE ,
iVAEs1 and iVAEs2.
Neu al Ne wo ks 181 (2025) 106774
8
M. Sipilä e al.
Fig. B.9. Mean co ela ion coe icien s o 500 ials o se ings 1–6 o sample size wi h he numbe o spa ial loca ions 𝑛𝑠= 150 and he numbe o empo al obse a ions 𝑛𝑡= 75.
CRediT au ho ship con ibu ion s a emen
Mika Sipilä: W i ing – o iginal d a , Visualiza ion, Valida ion, So -
wa e, Me hodology, In es iga ion, Fo mal analysis, Concep ualiza ion.
Claudia Cappello: W i ing – e iew & edi ing, Valida ion, In es iga-
ion, Fo mal analysis, Da a cu a ion. Sand a De Iaco: W i ing – e iew
& edi ing, Valida ion, Supe ision, In es iga ion. Klaus No dhausen:
W i ing – e iew & edi ing, Supe ision, Resou ces, P ojec admin-
is a ion. Sa a Taskinen: W i ing – e iew & edi ing, Supe ision,
Resou ces, P ojec adminis a ion.
Decla a ion o compe ing in e es
The au ho s decla e ha hey ha e no known compe ing inan-
cial in e es s o pe sonal ela ionships ha could ha e appea ed o
in luence he wo k epo ed in his pape .
Da a a ailabili y
Da a will be made a ailable on eques .
Acknowledgemen s
We acknowledge he suppo om Vilho, Y jö and Kalle Väisälä
ounda ion o MS, he suppo om he Resea ch Council o Fin-
land (453691) o ST, he suppo om he Resea ch Council o Fin-
land (363261) o KN and he suppo om he HiTEc COST Ac ion
(CA21163) o KN and ST.
Appendix A. Compu a ional complexi y analysis
The compu a ional complexi ies o he p oposed algo i hms a e
composed o wo pa s, o ming he auxilia y da a and aining iVAE.
We use Big O no a ion o ep esen he wo s case ime and space
complexi ies, whe e 𝑂(𝑛)deno es linea g ow h in compu a ion ime
o memo y usage wi h espec o he inpu size 𝑛. Fi s , le us add ess
iVAE’s compu a ional complexi y which is e y simila o any eed o -
wa d neu al ne wo k such as egula VAE. Using he Big O no a ion, he
compu a ional ime complexi y o aining he model is 𝑂(𝑛×𝑛𝑤×𝑛𝑒),
whe e 𝑛is he sample size, 𝑛𝑤is he numbe o weigh s in he model and
𝑛𝑒is he numbe o epochs. When he sample size 𝑛g ows, less epochs
a e ypically needed o aining, which makes he model well scaleable
Neu al Ne wo ks 181 (2025) 106774
15
M. Sipilä e al.
Fig. B.10. Mean co ela ion coe icien s o di e en adial basis unc ion pa ame e se ings o iVAE . The boxplo s p esen 500 ials o Se ing 6 wi h he numbe o spa ial
loca ions 𝑛𝑠= 150 and he numbe o empo al obse a ions 𝑛𝑡= 300.
in e ms o sample size. The memo y usage, i.e. space complexi y, is
𝑂(𝑛𝑤) o s o ing he weigh s o he model. The numbe o weigh s
𝑛𝑤can be b oken down o numbe o weigh s 𝑛𝑤1in encode –decode
pa and he numbe o weigh s 𝑛𝑤2in auxilia y unc ion. 𝑛𝑤1and 𝑛𝑤2
a e hea ily dependen on he numbe and size o he hidden laye s.
Typically ai ly small neu al ne wo ks (e.g. 3 laye s wi h 128 uni s)
in encode , decode and auxilia y unc ions a e su icien . In addi ion,
𝑛𝑤1depends linea ly on he dimension o he inpu da a and la en
dimension whe eas 𝑛𝑤2depends linea ly on he dimension o auxilia y
da a. Also, i he inpu dimension is e y la ge (e.g. mo e han 100),
a la ge encode –decode ne wo k migh be needed. Hence, he ime
complexi y g ows mo e when inpu dimension, la en dimension o
auxilia y da a dimension g ow.
In iVAEc, he ime and space complexi ies a e he lowes as he
coo dina es a e only scaled o o m he auxilia y a iables, meaning
ha using Big O no a ion, bo h ime and space complexi ies a e 𝑂(𝑛) o
o ming and s o ing he auxilia y da a. The auxilia y da a is only wo
dimensional, which makes ime complexi y sligh ly lowe compa ed o
o he algo i hms. In iVAEs1-iVAEs3, he ime complexi y o o ming
he auxilia y da a is 𝑂(𝑚1×𝑚2×𝑚×𝑛), whe e 𝑚1, 𝑚2and 𝑚a e
he numbe o segmen s along each dimension. Howe e , o iVAEs1,
whe e all dimensions a e conside ed join ly, and hence he dimension
o auxilia y da a can be e y la ge, he space complexi y is 𝑂(𝑚1×𝑚2×
𝑚×𝑛). Fo iVAEs2, he space complexi y is 𝑂((𝑚1×𝑚2+𝑚) × 𝑛)
and o iVAEs3, i is 𝑂((𝑚1+𝑚2+𝑚) × 𝑛). In e ms o compu a ion
ime, iVAEs3 is he mos e icien o segmen a ion based algo i hms as
he auxilia y dimension is he lowes . iVAEs2 is also e icien i he
numbe o spa ial segmen s is no e y high. In iVAE , he ime and
space complexi ies o o ming and s o ing auxilia y da a a e 𝑂(𝐾+
𝐾) × 𝑛, whe e 𝐾and 𝐾a e numbe s o spa ial and empo al node
poin s, espec i ely. In all abo e iVAE a ian s, he space complexi y
can be educed u he by cons uc ing auxilia y a iables ba ch-wise
du ing aining p ocess. In conclusion, he algo i hms a e well scalable
in e ms o sample size 𝑛and ela i ely well scalable in e ms o
dimensions o inpu da a, la en da a and auxilia y da a (linea ime
complexi y). Howe e , i he auxilia y da a a e no o med ba ch-wise,
memo y consump ion may g ow la ge i dimension o auxilia y a iable
is e y la ge. Since he algo i hm is essen ially composed o h ee eed
o wa d neu al ne wo ks, encode , decode and auxilia y unc ion, s an-
da d pa alleliza ion me hods such as da a pa allelism, which dis ibu es
he da a ba ch-wise ac oss mul iple compu a ion uni s, can be applied
o u he educe he o e all compu a ion ime.
Appendix B. Addi ional simula ion esul s
See Figs. B.8–B.10
Re e ences
Ansa i, S., Ala any, A. S., Alnajja , K. A., Kha e , T., Mahmoud, S., Al-Jumeily, D., e
al. (2023). A su ey o a i icial in elligence app oaches in blind sou ce sepa a ion.
Neu ocompu ing,561, A icle 126895. h p://dx.doi.o g/10.1016/j.neucom.2023.
126895.
Bachoc, F., Gen on, M. G., No dhausen, K., Ruiz-Gazen, A., & Vi a, J. (2020).
Spa ial blind sou ce sepa a ion. Biome ika,107, 627–646. h p://dx.doi.o g/10.
1093/biome /asz079.
Cappello, C., De Iaco, S., & Palma, M. (2022). Compu a ional ad ances o
spa io- empo al mul i a ia e en i onmen al models. Compu a ional S a is ics,37(2),
651–670.
Cappello, C., De Iaco, S., & Posa, D. (2020). Co a es : an R package o selec ing a
class o space- ime co a iance unc ions. Jou nal o S a is ical So wa e,94, 1–42.
Chen, W., Gen on, M. G., & Sun, Y. (2021). Space- ime co a iance s uc u es and
models. Annual Re iew o S a is ics and I s Applica ion,8, 191–215.
Chen, W., Li, Y., Reich, B. J., & Sun, Y. (2024). Deepk iging: Spa ially dependen deep
neu al ne wo ks o spa ial p edic ion. S a is ica Sinica,34, 291–311.
Neu al Ne wo ks 181 (2025) 106774
16
M. Sipilä e al.
Comon, P., & Ju en, C. (2010). Handbook o blind sou ce sepa a ion: Independen
componen analysis and applica ions. Academic P ess, h p://dx.doi.o g/10.1016/
C2009-0-19334-0.
De Iaco, S., Mye s, D., Palma, M., & Posa, D. (2013). Using simul aneous diagonaliza ion
o iden i y a space– ime linea co egionaliza ion model. Ma hema ical Geosciences,
45, 69–86.
De Iaco, S., Mye s, D. E., & Posa, D. (2001). Space– ime analysis using a gene al
p oduc –sum model. S a is ics & P obabili y Le e s,52(1), 21–28.
De Iaco, S., Mye s, D. E., & Posa, D. (2002). Nonsepa able space– ime co a iance
models: Some pa ame ic amilies. Ma hema ical Geology,34, 23–42.
De Iaco, S., Mye s, D., & Posa, D. (2003). The linea co egionaliza ion model and he
p oduc –sum space– ime a iog am. Ma hema ical Geology,35, 25–38.
De Iaco, S., Palma, M., & Posa, D. (2005). Modeling and p edic ion o mul i a ia e
space– ime andom ields. Compu a ional S a is ics & Da a Analysis,48(3), 525–547.
De Iaco, S., Palma, M., & Posa, D. (2019). Choosing sui able linea co egionaliza-
ion models o spa io- empo al da a. S ochas ic En i onmen al Resea ch and Risk
Assessmen ,33, 1419–1434.
De Iaco, S., & Posa, D. (2012). P edic ing spa io- empo al andom ields: some
compu a ional aspec s. Compu a ional Geosciences,41, 12–24.
De Iaco, S., & Posa, D. (2013). Posi i e and nega i e non-sepa abili y o space– ime
co a iance models. Jou nal o S a is ical Planning and In e ence,143(2), 378–391.
De Iaco, S., Posa, D., Cappello, C., & Maggio, S. (2019). Iso opy, symme y, sepa abili y
and s ic posi i e de ini eness o co a iance unc ions: a c i ical e iew. Spa ial
S a is ics,29, 89–108.
Feng, L., Nowak, G., O’Neill, T., & Welsh, A. (2014). CUTOFF: A spa io- empo al
impu a ion me hod. Jou nal o Hyd ology,519, 3591–3605.
Good ellow, I., Pouge -Abadie, J., Mi za, M., Xu, B., Wa de-Fa ley, D., Ozai , S., e
al. (2020). Gene a i e ad e sa ial ne wo ks. Communica ions o he ACM,63(11),
139–144.
Häl ä, H., & Hy ä inen, A. (2020). Hidden Ma ko nonlinea ICA: Unsupe ised
lea ning om nons a iona y ime se ies. In Con e ence on Unce ain y in A i icial
In elligence (pp. 939–948). PMLR.
Häl ä, H., Le Co , S., Lehé icy, L., So, J., Zhu, Y., Gassia , E., e al. (2021).
Disen angling iden i iable ea u es om noisy da a wi h s uc u ed nonlinea ICA.
Ad ances in Neu al In o ma ion P ocessing Sys ems,34, 1624–1633.
Hans, W. (2003). Mul i a ia e Geos a is ics: An In oduc ion wi h Applica ions. Sp inge
Science & Business Media.
Has ie, T., Tibshi ani, R., F iedman, J. H., & F iedman, J. H. (2009). ol. 2,The Elemen s
o S a is ical Lea ning: Da a Mining, In e ence, and P edic ion. Sp inge .
Hy ä inen, A. (1999). Fas and obus ixed-poin algo i hms o independen
componen analysis. IEEE T ansac ions on Neu al Ne wo ks,10(3), 626–634.
Hy ä inen, A., & Mo ioka, H. (2016). Unsupe ised ea u e ex ac ion by ime-
con as i e lea ning and nonlinea ICA. Ad ances in Neu al In o ma ion P ocessing
Sys ems,29.
Hy ä inen, A., & Mo ioka, H. (2017). Nonlinea ICA o empo ally dependen s a iona y
sou ces. In A i icial in elligence and s a is ics (pp. 460–469). PMLR.
Hy ä inen, A., & Pajunen, P. (1999). Nonlinea independen componen analysis:
Exis ence and uniqueness esul s. Neu al Ne wo ks,12(3), 429–439.
Hy ä inen, A., Sasaki, H., & Tu ne , R. (2019). Nonlinea ICA using auxilia y a iables
and gene alized con as i e lea ning. In The 22nd in e na ional con e ence on a i icial
in elligence and s a is ics (pp. 859–868). PMLR.
Khemakhem, I., Kingma, D., Mon i, R., & Hy ä inen, A. (2020). Va ia ional au oen-
code s and nonlinea ICA: A uni ying amewo k. In In e na ional Con e ence on
A i icial In elligence and S a is ics (pp. 2207–2217). PMLR.
Kingma, D. P., & Welling, M. (2013). Au o-encoding a ia ional Bayes. a Xi p ep in
a Xi :1312.6114.
Ky iakidis, P. C., & Jou nel, A. G. (1999). Geos a is ical space– ime models: A e iew.
Ma hema ical Geology,31, 651–684.
Lundbe g, S. M., & Lee, S.-I. (2017). A uni ied app oach o in e p e ing model
p edic ions. Ad ances in Neu al In o ma ion P ocessing Sys ems,30.
Luo, W., & Li, B. (2016). Combining eigen alues and a ia ion o eigen ec o s o o de
de e mina ion. Biome ika,103, 875–887.
Luo, W., & Li, B. (2021). On o de de e mina ion by p edic o augmen a ion. Biome ika,
108, 557–574.
Maas, A. L., Hannun, A. Y., Ng, A. Y., e al. (2013). Rec i ie nonlinea i ies imp o e
neu al ne wo k acous ic models. 30, In P oc. icml (1), (p. 3). A lan a, GA.
Ma chini, J. L., Hea on, C., & Ripley, B. D. (2021). Fas ICA: Fas ICA algo i hms o
pe o m ICA and p ojec ion pu sui . URL h ps://CRAN.R-p ojec .o g/package=
as ICA.
Ma cílio, W. E., & Ele , D. M. (2020). F om explana ions o ea u e selec ion: assessing
SHAP alues as ea u e selec ion mechanism. In 2020 33 d SIBGRAPI Con e ence
on G aphics, Pa e ns and Images (SIBGRAPI) (pp. 340–347). Ieee.
Muehlmann, C., Bachoc, F., & No dhausen, K. (2022). Blind sou ce sepa a ion o non-
s a iona y andom ields. Spa ial S a is ics,47, A icle 100574. h p://dx.doi.o g/
10.1016/j.spas a.2021.100574.
Muehlmann, C., Bachoc, F., No dhausen, K., & Yi, M. (2024). Tes o he la en
dimension o a spa ial blind sou ce sepa a ion model. S a is ica Sinica,34, 837–865.
h p://dx.doi.o g/10.5705/ss.202021.0326.
Muehlmann, C., De Iaco, S., & No dhausen, K. (2023). Blind eco e y o sou ces o
mul i a ia e space- ime andom ields. S ochas ic En i onmen al Resea ch and Risk
Assessmen ,37, 1593–1613. h p://dx.doi.o g/10.1007/s00477-022-02348-2.
Muehlmann, C., Piccolo o, N., Cappello, C., De Iaco, S., & No dhausen, K. (2022).
SpaceTimeBSS: Blind sou ce sepa a ion o mul i a ia e spa io- empo al da a. URL
h ps://CRAN.R-p ojec .o g/package=SpaceTimeBSS.
Nag, P., Sun, Y., & Reich, B. J. (2023). Spa io- empo al DeepK iging o in e pola ion
and p obabilis ic o ecas ing. Spa ial S a is ics,57, A icle 100773. h p://dx.doi.
o g/10.1016/j.spas a.2023.100773.
No dhausen, K., Oja, H., Filzmose , P., & Reimann, C. (2015). Blind sou ce sepa a ion
o spa ial composi ional da a. Ma hema ical Geosciences,47(7), 753–770. h p:
//dx.doi.o g/10.1007/s11004-014-9559-5.
No dhausen, K., Taskinen, S., & Vi a, J. (2022). Signal dimension es ima ion in
BSS models wi h se ial dependence. In 2022 In e na ional Con e ence on Elec ical,
Compu e , Communica ions and Mecha onics Enginee ing (ICECCME) (pp. 1–7).
Nychka, D., Bandyopadhyay, S., Hamme ling, D., Lindg en, F., & Sain, S. (2015). A
mul i esolu ion Gaussian p ocess model o he analysis o la ge spa ial da ase s.
Jou nal o Compu a ional and G aphical S a is ics,24(2), 579–599.
Papalexiou, S. M., & Se inaldi, F. (2020). Random ields simpli ied: P ese ing ma ginal
dis ibu ions, co ela ions, and in e mi ency, wi h applica ions om ain all o
humidi y. Wa e Resou ces Resea ch,56(2), A icle e2019WR026331.
Papalexiou, S. M., Se inaldi, F., & Po cu, E. (2021). Ad ancing space- ime simula ion
o andom ields: F om s o ms o cyclones and beyond. Wa e Resou ces Resea ch,
57(8), A icle e2020WR029466.
Po cu, E., Fu e , R., & Nychka, D. (2021). 30 yea s o space– ime co a iance unc ions.
WIREs Compu a ional S a is ics,13(2), A icle e1512. h p://dx.doi.o g/10.1002/
wics.1512.
R Co e Team (2023). R: A Language and En i onmen o S a is ical Compu ing. Vienna,
Aus ia: R Founda ion o S a is ical Compu ing, URL h ps://www.R-p ojec .o g/.
Radojičić, U., & No dhausen, K. (2024). O de de e mina ion in second-o de sou ce
sepa a ion models using da a augmen a ion. In J. Ansa i, S. Fuchs, W. T u schnig,
M. A. Lubiano, M. A. Gil, P. G zego zewski, & O. H yniewicz (Eds.), Combining,
Modelling and Analyzing Imp ecision, Randomness and Dependence (pp. 371–379).
Cham: Sp inge .
Sal ana, M. L. O., & Gen on, M. G. (2020). Nons a iona y c oss-co a iance unc ions
o mul i a ia e spa io- empo al andom ields. Spa ial S a is ics,37, A icle 100411.
Sig is , F., Künsch, H. R., & S ahel, W. A. (2012). A dynamic nons a iona y spa io-
empo al model o sho e m p edic ion o p ecipi a ion. The Annals o Applied
S a is ics,6(4), 1452–1477. h p://dx.doi.o g/10.1214/12-AOAS564.
Sipilä, M., Muehlmann, C., No dhausen, K., & Taskinen, S. (2024). Robus second-o de
s a iona y spa ial blind sou ce sepa a ion using gene alized sign ma ices. Spa ial
S a is ics,59, A icle 100803. h p://dx.doi.o g/10.1016/j.spas a.2023.100803.
Sipilä, M., No dhausen, K., & Taskinen, S. (2024). Nonlinea blind sou ce sepa a ion
exploi ing spa ial nons a iona i y. In o ma ion Sciences,665, A icle 120365. h p:
//dx.doi.o g/10.1016/j.ins.2024.120365.
Vi a, J., & No dhausen, K. (2017). Blind sou ce sepa a ion o enso - alued ime se ies.
Signal P ocessing,141, 204–216. h p://dx.doi.o g/10.1016/j.sigp o.2017.06.008.
Vi a, J., & No dhausen, K. (2021). De e mining he Signal Dimension in Second O de
Sou ce Sepa a ion. S a is ica Sinica,31, 135–156.
Wendland, H. (1995). Piecewise polynomial, posi i e de ini e and compac ly suppo ed
adial unc ions o minimal deg ee. Ad ances in Compu a ional Ma hema ics,4,
389–396.
Xu, H., & Ga doni, P. (2018). Imp o ed la en space app oach o modelling
non-s a iona y spa ial– empo al andom ields. Spa ial S a is ics,23, 160–181.
Xu, R., Vaida, F., & Ha ing on, D. P. (2009). Using p o ile likelihood o semipa ame ic
model selec ion wi h applica ion o p opo ional haza ds mixed models. S a is ica
Sinica,19(2), 819.
Yan, Y., Huang, H.-C., & Gen on, M. G. (2021). Vec o au o eg essi e models wi h
spa ially s uc u ed coe icien s o ime se ies on a spa ial g id. Jou nal o
Ag icul u al, Biological and En i onmen al S a is ics,26(3), 387–408.
Yi, M., & No dhausen, K. (2023). Elasso o es ima ing he signal dimension in ICA.
In 2023 31s Eu opean Signal P ocessing Con e ence (EUSIPCO) (pp. 2023–2027).
h p://dx.doi.o g/10.23919/EUSIPCO58844.2023.10289956.
Neu al Ne wo ks 181 (2025) 106774
17
