Trainable and explainable simplicial map neural networks

T ainable and explainable simplicial map neu al ne wo ks Edua do Paluzo-Hidalgo a,∗, Rocio Gonzalez-Diazb, Miguel A. Gu ié ez-Na anjo c aDepa men o Quan i a i e Me hods, Uni e sidad Loyola Andalucía, Campus Se illa, Dos He manas, Se ille, Spain bDepa amen o de Ma emá ica Aplicada I, Uni e sidad de Se illa, Se ille, Spain cDepa amen o de Ciencias de la Compu ación e In eligencia A ificial, Uni e sidad de Se illa, Se ille, Spain A R T I C L E I N F O A B S T R A C T Keywo ds: T aining neu al ne wo k Simplicial maps Explainable a ificial in elligence Simplicial map neu al ne wo ks (SMNNs) a e opology-based neu al ne wo ks wi h in e es ing p ope ies such as uni e sal app oxima ion abili y and obus ness o ad e sa ial examples unde app op ia e condi ions. Howe e , SMNNs p esen some bo lenecks o hei possible applica ion in high-dimensional da ase s. Fi s , SMNNs ha e p ecompu ed fixed weigh and no SMNN aining p ocess has been defined so a , so hey lack gene aliza ion abili y. Second, SMNNs equi e he cons uc ion o a con ex poly ope su ounding he inpu da ase . In his pape , we o e come hese issues by p oposing an SMNN aining p ocedu e based on a suppo subse o he gi en da ase and eplacing he cons uc ion o he con ex poly ope by a me hod based on p ojec ions o a hype sphe e. In addi ion, he explainabili y capaci y o SMNNs and effec i e implemen a ion a e also newly in oduced in his pape . 1. In oduc ion In ecen yea s, A ificial In elligence (AI) me hods in gene al and Machine Lea ning me hods in pa icula ha e eached success in eal-li e p oblems ha we e unexpec ed only a ew yea s ago. Many diffe en a eas ha e con ibu ed o his de elopmen . Among hem, we can ci e he esea ch on new heo e ical algo i hms, he inc easing compu a ional powe o he la es gene a ion ha dwa e, and he apid access o a huge amoun o da a. Such a combina ion o ac o s leads o he de elopmen o inc easingly complex sel - egula ed AI me hods. Many AI models cu en ly used a e based on backp opaga ion algo i hms, which ain and egula e hemsel es o achie e a goal, such as classifica ion, ecommenda ion, o p edic ion. These sel - egula ing models achie e some kind o knowledge as hey success ully e alua e es da a independen o he da a used o ain hem. None heless, such knowledge is usually exp essed in a non-human- eadable way. To fill he gap be ween he ecen de elopmen o AI models and hei social use, many esea che s ha e ocused on he de elop- men o Explainable A ificial In elligence (XAI), which consis s o a se o echniques o p o ide clea , unde s andable, anspa en , in elligible, us wo hy, and in e p e able explana ions o he decisions, p edic ions, and easoning p ocesses made by he AI models, a he han jus p esen ing hei ou pu , especially in domains whe e AI decisions can ha e significan consequences on human li e. A global axonomy o in e p e able AI wi h he aim o uni ying e minology o achie e cla i y and efficiency in he defini ion o * Co esponding au ho . E-mail add esses: [email p o ec ed] (E. Paluzo-Hidalgo), [email p o ec ed] (R. Gonzalez-Diaz), [email p o ec ed] (M.A. Gu ié ez-Na anjo). URLs: h ps://pe sonal.us.es/ ogodi (R. Gonzalez-Diaz), h p://www.cs.us.es/~na anjo/ (M.A. Gu ié ez-Na anjo). h ps://doi.o g/10.1016/j.ins.2024.120474 Recei ed 26 Oc obe 2023; Recei ed in e ised o m 22 Janua y 2024; Accep ed 11 Ma ch 2024 A ailable online 18 Ma ch 2024 0020-0255/© 2024 The Au ho (s). (h p://c ea i ecommons.o g/licenses/by/4.0/). In o ma ion Sciences 667 (2024) 120474 2 E. Paluzo-Hidalgo, R. Gonzalez-Diaz and M.A. Gu ié ez-Na anjo egula ions o he de elopmen o e hical and eliable AI can be ound in [1]. Mo eo e , a nice in oduc ion and gene al ision can be ound in [2]. Ano he cla i ying pape wi h defini ions, concep s, and applica ions o XAI in [3]. The so-called Simplicial Map Neu al Ne wo ks (SMNNs) we e in oduced in [4]as a cons uc i e app oach o he p oblem o app oxima ing a con inuous unc ion on a compac se in a iangula ed space. Since he o iginal aim o he defini ion o SMNNs was ocused on building a cons uc i e app oach, he compu a ion o hei weigh s was no based on an op imiza ion p ocess, as usual in neu al ne wo ks, bu on a de e minis ic calculus. The a chi ec u e o SMNNs and he compu a ion o he se o weigh s a e based on a combina o ial opology ool called simplicial maps. Mo eo e , SMNNs can be used o classifica ion and can be cons uc ed o be obus o ad e sa ial examples [5]. Besides, hei a chi ec u e can be educed while main aining accu acy [6], being in a ian o ans o ma ion i he ans o ma ion p ese es he ba ycen ic coo dina es (scale, o a ion, symme ies, e c.). As defined in [4,6,5], SMNNs a e buil as a wo-hidden-laye eed- o wa d ne wo k whe e he se o weigh s is p ecompu ed based on he calcula ion o a iangula ed con ex poly ope su ounding he inpu da a. As o he app oxima ions o con inuous unc ions wi h a bi a y p ecision (see, o example, [7]), SMNNs ha e fixed weigh s, which means ha he weigh s depend only on he iangula ion made wi h he poin s o he da ase as he suppo se and no aining p ocess is applied. Summing up, some o he limi a ions o he SMNNs un il now a e ha hey a e cos ly o calcula e since he numbe o neu ons is p opo ional o he numbe o simplices o he iangula ion suppo ed on he inpu da ase , and hey suffe om o e fi ing and he e o e no gene alize well. These aspec s make SMNNs no used in p ac ice so a , al hough he idea o ela ing simplicial maps o neu al ne wo ks is dis up i e and p o ides a new b idge ha can en ich bo h a eas. In his pape , we p opose a me hod o make SMNNs efficien by educing hei size (in e ms o he numbe o neu ons ha depends on he e ices o he iangula ion) and ha success ully makes SMNNs ainable and wi h gene aliza ion abili y. Besides, we also p esen a s udy o he selec ion o he e ices om which we ob ain he iangula ion. Al hough SMNNs conside he e ices o a simplex as pa o he necessa y in o ma ion o he classifica ion ask, he app oach p esen ed in his pape is a om he classic Machine Lea ning ins ance-based me hods. Such me hods ely on a de e minis ic compu a ion based on dis ances, bu , in he app oach p esen ed in his pape , he compu a ion o he weigh s is he esul o an op imiza ion me hod in a p obabili y dis ibu ion space. Finally, om an XAI poin o iew, we will see in his pape ha SMNNs a e explainable models since all decision s eps o compu e he ou pu o SMNNs a e unde s andable and anspa en , and he e o e us wo hy. The pape is o ganized as ollows. Fi s , some concep s o compu a ional opology and he defini ion o SMNNs a e ecalled in Sec ion 2. Nex , in Sec ion 3we de elop se e al echnical de ails needed o he SMNN aining p ocess, which will be in oduced in Sec ion 4. Sec ion 5is de o ed o he explainabili y o he model. Sec ion 6is de o ed o discussion and limi a ions. Finally, he pape ends wi h some expe imen s and conclusions. 2. Backg ound In his sec ion, we assume ha he eade is amilia wi h he basic concep s o compu a ional opology. Fo a comp ehensi e p esen a ion, we e e o [8]. 2.1. Simplicial complexes Conside a fini e se o poin s 𝑉={𝑣1, … , 𝑣𝛽} ⊂ℝ𝑛whose elemen s will be called e ices. A subse 𝜎=⟨𝑣𝑖0,𝑣 𝑖1,…,𝑣 𝑖𝑑⟩ o 𝑉wi h 𝑑+1 e ices (in gene al posi ion) is called a 𝑑-simplex. The con ex hull o he e ices o 𝜎will be deno ed by |𝜎|and co esponds o he se : {𝑥∈ℝ𝑛∶𝑥=∑ 𝑗∈0,𝑑 𝑏𝑗(𝑥)𝑣𝑖𝑗} whe e 𝑎, 𝑏 ={𝑎, 𝑎 +1, … , 𝑏} o 𝑎 <𝑏 ∈ℤ, and 𝑏(𝑥)=(𝑏0(𝑥),𝑏 1(𝑥),…,𝑏 𝑑(𝑥)) a e called he ba ycen ic coo dina es o 𝑥wi h espec o 𝜎, and sa is y ha : ∑ 𝑗∈0,𝑑 𝑏𝑗(𝑥)=1 and 𝑏𝑗(𝑥)≥0∀𝑗∈0,𝑑. The ba ycen ic coo dina es o 𝑥can be in e p e ed as masses placed a he e ices o 𝜎so 𝑥is he cen e o mass. All hese masses a e posi i e i and only i 𝑥is inside 𝜎. Fo example, le us conside he 1-simplex 𝜖=⟨𝑣𝑖0, 𝑣𝑖1⟩which is composed o wo e ices o 𝑉. Then |𝜖|is he se o poin s in ℝ𝑛co esponding o he edge wi h endpoin s 𝑣𝑖0and 𝑣𝑖1, and i , o example, 𝑏(𝑥) =(0.5, 0.5) hen 𝑥is he midpoin o |𝜖|. A simplicial complex 𝐾wi h e ex se 𝑉consis s o a fini e collec ion o simplices sa is ying ha i 𝜎∈𝐾 hen ei he 𝜎=⟨𝑣⟩ o some 𝑣 ∈𝑉o any ace ( ha is, a nonemp y subse ) o 𝜎is a simplex o 𝐾. Fu he mo e, i 𝜎, 𝜇∈𝐾 hen |𝜎| ∩|𝜇| =∅o |𝜎| ∩|𝜇| =|𝛾| o some 𝛾∈𝐾. The se ⋃𝜎∈𝐾|𝜎|will be deno ed by |𝐾|. A maximal simplex o 𝐾is a simplex ha is no he ace o any o he In o ma ion Sciences 667 (2024) 120474 3 E. Paluzo-Hidalgo, R. Gonzalez-Diaz and M.A. Gu ié ez-Na anjo Fig. 1. On he le , wo iangles ha do no in e sec in a common ace (an edge o a e ex). On he igh , he geome ic ep esen a ion |𝐾|o a pu e 2-simplicial complex 𝐾composed o h ee maximal 2-simplices ( he iangles 𝜎1, 𝜎2and 𝜎3). The edge 𝜇2is a common ace o 𝜎2and 𝜎3. The edge 𝜇1is a ace o 𝜎1. simplex o 𝐾. I he maximal simplices o 𝐾a e all 𝑑-simplices hen 𝐾is called a pu e 𝑑-simplicial complex. These concep s a e illus a ed in Fig. 1. The ba ycen ic coo dina es o 𝑥wi h espec o he simplicial complex |𝐾|a e defined as he ba ycen ic coo dina es o 𝑥wi h espec o 𝜎∈𝐾such ha 𝑥 ∈|𝜎|. Le us obse e ha he ba ycen ic coo dina es o 𝑥 ∈|𝐾|a e unique. An example o simplicial complexes is he Delaunay iangula ion Del(𝑉)defined om he Vo onoi diag am o a gi en fini e se o e ices 𝑉. The ollowing esul ex ac ed om [9, page 48] is jus an al e na i e defini ion o Delaunay iangula ions. The emp y ball p ope y [9]: Any subse 𝜎o 𝑉is a simplex o Del(𝑉)i and only i |𝜎|has a ci cumsc ibing open ball emp y o poin s o 𝑉. 2.2. Simplicial maps Le 𝐾be a pu e 𝑛-simplicial complex and 𝐿a pu e 𝑘-simplicial complex wi h e ex se s 𝑉and 𝑊, espec i ely. The map 𝜑(0) ∶𝑉→𝑊is called a e ex map i i sa isfies ha he se ob ained om {𝜑(0)(𝑣𝑖0), … , 𝜑(0)(𝑣𝑖𝑑)}a e emo ing duplica ed e ices is a simplex in 𝐿whene e ⟨𝑣𝑖0, … , 𝑣𝑖𝑑⟩is a simplex in 𝐾. The e ex map 𝜑(0) always induces a con inuous unc ion, called a simplicial map 𝜑 ∶|𝐾| →|𝐿|, which is defined as ollows. Le 𝑏(𝑥) =(𝑏0(𝑥), … , 𝑏𝑛(𝑥)) be he ba ycen ic coo dina es o 𝑥 ∈|𝐾|wi h espec o 𝜎=⟨𝑣𝑖0, … , 𝑣𝑖𝑛⟩ ∈𝐾. Then 𝜑(𝑥)= ∑ 𝑗∈0,𝑛 𝑏𝑗(𝑥)𝜑(0)(𝑣𝑖𝑗). Le us obse e ha 𝜑(𝑥) =𝜑(0)(𝑥)i 𝑥 ∈𝑉. A special kind o simplicial map used o sol e classifica ion asks will be in oduced in he nex subsec ion. 2.3. Simplicial maps o classifica ion asks Nex , we will show how a simplicial map can be used o sol e a classifica ion p oblem (see [5] o de ails). F om now on, we will assume ha he inpu da ase is a fini e se o poin s 𝑉in ℝ𝑛 oge he wi h a se o 𝑘labels Λsuch ha each 𝑣 ∈𝑉is agged wi h a label 𝜆 aken om Λ. Fi s ly, he in ui ion is ha he space su ounding he da ase is labelled as unknown. Fo his, we add a new label o Λ, called unknown label, and a one-ho encoding ep esen a ion 𝑊𝑘+1 ⊂ℝ𝑘+1 o hese 𝑘 +1labels being: 𝑊𝑘+1 ={𝓁𝑗=(0,𝑗 …,0,1,0,𝑘−𝑗 …,0) ∶ 𝑗∈1,𝑘}, whe e he one-ho ec o 𝓁𝑗encodes he 𝑗- h label o Λ o 𝑗∈1, 𝑘and whe e 𝓁0encodes he unknown label. We now conside a con ex poly ope wi h a e ex se 𝑃su ounding he se 𝑉. The poly ope always exis s since 𝑉is fini e. Nex , we define a map 𝜑(0) ∶𝑉∪𝑃→𝑊𝑘+1 mapping each e ex 𝑣 ∈𝑉 agged wi h label 𝜆 o he one-ho ec o in 𝑊𝑘+1 ha encodes he label 𝜆. The e ices o 𝑃a e sen o he e ex 𝓁0. Obse e ha 𝜑(0) is a e ex map. Le 𝐿deno e he simplicial complex wi h e ex se 𝑊𝑘+1 consis ing o only one maximal 𝑘-simplex and le Del(𝑉∪𝑃)deno e he Delaunay iangula ion compu ed o he se o poin s 𝑉∪𝑃. No e ha | Del(𝑉∪𝑃)| =. The simplicial map 𝜑 ∶→|𝐿|is induced by he e ex map 𝜑(0) as explained in Subsec ion 2.2. Rema k 1. The space |𝐿|can be in e p e ed as he disc e e p obabili y dis ibu ion space Ω𝑘+1 wi h 𝑘 +1 a iables. As an example, in Fig. 2, on he le , we can see a da ase wi h ou poin s 𝑉={𝑏, 𝑐, 𝑘, 𝑑}, labelled ed and blue. The g een poin s 𝑃={𝑎, 𝑒, 𝑔, 𝑓}a e he e ices o a con ex poly ope con aining 𝑉and a e sen by 𝜑(0) o he g een e ex 𝓁0on he igh . The simplicial complex 𝐾=Del(𝑉∪𝑃)is d awn on he le and consis s o en maximal 2-simplices. On he igh , he simplicial complex 𝐿consis s o one maximal 2-simplex. The do ed a ows illus a e some examples o 𝜑 ∶→|𝐿|. In o ma ion Sciences 667 (2024) 120474 4 E. Paluzo-Hidalgo, R. Gonzalez-Diaz and M.A. Gu ié ez-Na anjo Fig. 2. Illus a ion o a simplicial map o a classifica ion ask. 2.4. Simplicial map neu al ne wo ks A ificial neu al ne wo ks can be seen as pa ame ized eal- alued mappings be ween mul idimensional spaces. Such mappings a e he composi ion o se e al maps (usually many o hem) ha can be s uc u ed in laye s. In [5], he simplicial map 𝜑defined abo e was ep esen ed as a wo-hidden-laye eed- o wa d neu al ne wo k 𝜑. This kind o a ificial neu al ne wo k is called simplicial map neu al ne wo k (SMNN). In he o iginal defini ion [5], he fi s hidden laye o an SMNN compu es he ba ycen ic coo dina es o Del(𝑉∪𝑃). Howe e , we will see he e ha i we p ecompu e he ba ycen ic coo dina es, we can simpli y he a chi ec u e o 𝜑as ollows. As be o e, conside an inpu da ase consis ing o a fini e se 𝑉⊂ℝ𝑛endowed wi h a se o 𝑘labels and a con ex poly ope  wi h a se o e ices 𝑃su ounding 𝑉. Le Del(𝑉∪𝑃)be he Delaunay iangula ion wi h e ex se 𝑉∪𝑃={𝜔1,…,𝜔 𝛼}⊆ℝ𝑛. Then, 𝜑(0) ∶𝑉∪𝑃→𝑊𝑘+1 is a e ex map. Le us assume ha , gi en 𝑥 ∈, we p ecompu e he ba ycen ic coo dina es 𝑏(𝑥) = (𝑏0(𝑥), … , 𝑏𝑛(𝑥)) ∈ℝ𝑛+1 o 𝑥wi h espec o he 𝑛-simplex 𝜎=⟨𝜔𝑖0, … , 𝜔𝑖𝑛⟩ ∈Del(𝑉∪𝑃)such ha 𝑥 ∈|𝜎|, and ha we also p ecompu e he ec o 𝜉(𝑥)=(𝜉1(𝑥),…,𝜉 𝛼(𝑥)) ∈ ℝ𝛼 sa is ying ha , o 𝑡 ∈1, 𝛼, 𝜉𝑡(𝑥) =𝑏𝑗(𝑥)i 𝑖𝑗=𝑡 o some 𝑗∈0, 𝑛. Le us ema k ha 𝜉(𝑥)should be a column ec o , bu we will use ow no a ion, o simplici y. The SMNN 𝜑induced by 𝜑 ha p edic s he ℎ-label o 𝑥, o ℎ ∈0, 𝑘, has he ollowing a chi ec u e: •The numbe o neu ons in he inpu laye is 𝛼. •The numbe o neu ons in he ou pu laye is 𝑘 +1. •The se o weigh s is ep esen ed as a (𝑘 +1) ×𝛼ma ix such ha he 𝑗- h column o is 𝜑(0)(𝜔𝑡) o 𝑡 ∈1, 𝛼. Then, 𝜑(𝑥)=⋅𝜉(𝑥). Obse e ha as defined so a , he SMNN weigh s a e p ecompu ed. Fu he mo e, he compu a ion o he ba ycen ic coo dina es o he poin s a ound 𝑉implies he calcula ion o he con ex poly ope su ounding 𝑉. Finally, he compu a ion o he Delaunay iangula ion Del(𝑉∪𝑃)is cos ly i 𝑉∪𝑃has many poin s since i s ime complexi y is 𝑂(𝑛 log 𝑛 +𝑛⌈𝑑 2⌉)(see [9, Chap e 4]). In he nex sec ions, we will p opose some echniques o o e come he SMNN cons uc ion d awbacks while main aining i s ad an ages. We will see ha one way o o e come he compu a ion o he con ex poly ope is o conside a hype sphe e 𝑆𝑛 ins ead. We will also see how o a oid he use o he a ificially c ea ed unknown label. Fu he mo e, o educe he cos o Delaunay compu a ion and add ainabili y o 𝜑 o a oid o e fi ing, a subse 𝑈⊂𝑉 will be conside ed. The se 𝑉will be used o ain and es a map 𝜑(0) 𝑈∶𝑈→ℝ𝑘. Such a map will induce a con inuous unc ion 𝜑𝑈∶𝐵𝑛→|𝐿|which app oxima es 𝜑. 3. The unknown bounda y and he unc ion 𝝋𝑼 In his sec ion, we will see how o compu e a unc ion 𝜑𝑈 ha app oxima es he simplicial map 𝜑and a oids he compu a ion o he con ex poly ope and he a ificial conside a ion o he unknown label, educing, a he same ime, he compu a ion o he Delaunay iangula ion used o cons uc SMNNs. The gene al desc ip ion o he me hodology is desc ibed in Algo i hm 1. Fi s , le us compu e a hype sphe e su ounding 𝑉. One o he simples ways o do ha is o ansla e 𝑉so ha i s cen e o mass is placed a he o igin 𝑜 ∈ℝ𝑛. Then, he hype sphe e In o ma ion Sciences 667 (2024) 120474 5 E. Paluzo-Hidalgo, R. Gonzalez-Diaz and M.A. Gu ié ez-Na anjo Fig. 3. An example o he poin 𝑤𝑥compu ed om 𝑥and he (𝑛−1)-simplex 𝜇=⟨𝑢1,𝑢 2⟩∈Γsuch ha 𝑥∈|𝜎| o 𝜎=⟨𝑤𝑥,𝑢 1,𝑢 2⟩. 𝑆𝑛={𝑤∈ℝ𝑛∶||𝑤||=𝑅} such ha 𝑅 >max{||𝑣|| ∶𝑣 ∈𝑉}sa isfies ha 𝑆𝑛su ounds 𝑉. Second, le us assume ha we ha e selec ed a subse 𝑈= {𝑢1, … , 𝑢𝑚} ⊆𝑉 (we will compa e diffe en s a egies o selec 𝑈in Sec ion 7) and ha we ha e compu ed Del(𝑈). Then, we ha e ha 𝑉⊂𝐵 𝑛={𝑥∈ℝ𝑛∶||𝑥||≤𝑅}and 𝑜∈|Del(𝑈)|. Le us conside he bounda y o Del(𝑈), deno ed as 𝛿Del(𝑈), which consis s o he se o (𝑛 −1)-simplices ha a e aces o exac ly one maximal simplex o Del(𝑈). Now, le us define 𝜉𝑈(𝑥) ∈ℝ𝑚 o any 𝑥 ∈𝐵𝑛as ollows. Gi en 𝑥 ∈𝐵𝑛, o find he 𝑛-simplex 𝜎=⟨𝜔0, … , 𝜔𝑛⟩such ha 𝑥 ∈|𝜎|, we ha e o conside wo cases: 𝑥 ∈| Del(𝑈)|and 𝑥 ∉| Del(𝑈)|. I 𝑥 ∈| Del(𝑈)| hen 𝜎is he 𝑛-simplex in Del(𝑈)such ha 𝑥 ∈|𝜎|. I 𝑥 ∉| Del(𝑈)| hen 𝜎is a new 𝑛-simplex defined by he e ices o an (𝑛 −1)-simplex o 𝛿Del(𝑈)and a new e ex consis ing o he p ojec ion o 𝑥 o 𝑆𝑛. Specifically, i 𝑥 ∉| Del(𝑈)| hen 𝜎is compu ed in he ollowing way: 1. Conside he se Γ={𝜇∈𝛿Del(𝑈)∶(𝑁⋅𝑢𝑖0+𝑐)(𝑁⋅𝑥+𝑐)<0} whe e 𝑁is he ec o no mal o he hype plane con aining 𝜇=⟨𝑢𝑖1, … , 𝑢𝑖𝑛⟩, 𝑐=𝑁⋅𝑢𝑖1, and 𝑢𝑖0∈𝑈such ha ⟨𝑢𝑖0, 𝑢𝑖1, … , 𝑢𝑖𝑛⟩ ∈ Del(𝑈). 2. Compu e he poin 𝑤𝑥=𝑅𝑥 ||𝑥||∈𝑆𝑛. 3. Find 𝜎=⟨𝑤𝑥, 𝑢𝑖1, … , 𝑢𝑖𝑛⟩such ha 𝜇=⟨𝑢𝑖1,…,𝑢 𝑖𝑛⟩∈Γand 𝑥∈|𝜎|. Obse e ha , by cons uc ion, 𝜇always exis s since | Del(𝑈)|is a con ex poly ope. Now, le 𝑏(𝑥) =(𝑏0(𝑥), … , 𝑏𝑛(𝑥)) ∈ℝ𝑛+1 be he ba ycen ic coo dina es o 𝑥wi h espec o 𝜎. Then, 𝜉𝑈(𝑥) =(𝜉1(𝑥), … , 𝜉𝑚(𝑥)) is he poin in ℝ𝑚sa is ying ha , o 𝑡 ∈1, 𝑚, 𝜉𝑡(𝑥)={𝑏𝑗(𝑥)i 𝑢𝑡=𝜔𝑗 o some 𝑗∈0,𝑛, 0o he wise. Obse e ha 𝜉𝑈(𝑥)always exis s and is unique. An example o poin s 𝑥and 𝑤𝑥and simplex 𝜇is shown in Fig. 3and Example 1. Le us obse e ha , hanks o he new defini ion o 𝜉𝑈(𝑥) o 𝑥 ∈𝐵𝑛, i we ha e a map 𝜑(0) 𝑈∶𝑈→ℝ𝑘 hen i induces a con inuous unc ion 𝜑𝑈∶𝐵𝑛→|𝐿|defined o any 𝑥 ∈𝐵𝑛as: 𝜑𝑈(𝑥)=so max (∑𝑡∈1,𝑚𝜉𝑡(𝑥)𝜑(0) 𝑈(𝑢𝑡)) =so max(𝑈⋅𝜉𝑈(𝑥)) whe e o 𝑧 =(𝑧1, … , 𝑧𝑘) ∈ℝ𝑘, so max(𝑧)=(𝑒𝑧1 ∑ℎ∈1,𝑘𝑒𝑧ℎ ,…,𝑒𝑧𝑘 ∑ℎ∈1,𝑘𝑒𝑧ℎ). In o ma ion Sciences 667 (2024) 120474 6 E. Paluzo-Hidalgo, R. Gonzalez-Diaz and M.A. Gu ié ez-Na anjo Fig. 4. The ela i e posi ions o he e ices 𝑣𝑖 o 𝑖∈1,5and he poin s 𝑥1and 𝑥2o Example 1. Le us obse e ha , o ob ain a ca ego ical dis ibu ion om 𝜑𝑈(𝑥) ∈ℝ𝑘, we could jus di ide each o i s coo dina es by he o al sum. Howe e , so max is in oduced he e o ob ain a simplified o mula o he g adien descen algo i hm as shown in Theo em 1. Example 1. Le us conside 𝑉={𝑣1=(1 2,1 2),𝑣 2=(1 2,1),𝑣 3=(1,1 2),𝑣 4=(1,1)} wi h label 𝜆1=0 o 𝑣𝑖wi h 𝑖 =1, 2and 𝜆2=1 o 𝑣𝑖wi h 𝑖 =3, 4. Fi s ly, we ansla e 𝑉so ha he cen e o mass o 𝑉is he o igin 𝑜 ∈ℝ2. The ansla ed da ase  𝑉is {𝑣1=(−1 4,−1 4),𝑣2=(−1 4,1 4),𝑣3=(1 4,−1 4),𝑣4=(1 4,1 4)}. Le us conside 𝑥1=(3 4, 3 5)and 𝑥2=(3 4, 5 4). Hence, he ansla ed inpu da a is 𝑥1=(0, −3 20 )and 𝑥2=(0, 1 2). To simpli y he explana ion o he me hod, conside 𝑈= 𝑉, ha is, 𝑢𝑖=𝑣𝑖 o 𝑖 ∈1, 4. Then, he maximal simplices o Del(𝑈)a e 𝜎1=⟨𝑣1, 𝑣2, 𝑣3⟩and 𝜎1=⟨𝑣2, 𝑣3, 𝑣4⟩ •The ma ix 𝑈is: (1100 0011 ) • Since he ba ycen ic coo dina es o 𝑥1wi h espec o |𝜎1|a e (0.5, 0.3, 0.2) hen 𝑥1is in |𝜎1| ⊂| Del(𝑈)|and 𝜉𝑈(𝑥1) = (0.3, 0.2, 0, 0.5). Then 𝜑𝑈(𝑥1)=so max(𝑈⋅𝜉𝑈(𝑥1)) = (0.5,0.5). •On he o he hand, he poin 𝑥2is ou side | Del(𝑈)|. Assuming ha , o example, we ha e fixed 𝑅 =1, we add a new simplex 𝜎3={𝑤𝑥, 𝑣1, 𝑣2}whe e 𝑤𝑥=𝑣5=(0, 1) which is he p ojec ion o 𝑥2 o he hype sphe e o adius 𝑅cen e ed in he o igin. See Fig. 4. Then, he ba ycen ic coo dina es o 𝑥2wi h espec o 𝜎3a e (0.33, 0.33, 0.33) and hen 𝜉𝑈(𝑥2) =(0, 0.33, 0.33, 0), concluding ha 𝜑𝑈(𝑥2)=so max(𝑈⋅𝜉𝑈(𝑥2)) = (0.5,0.5). The pseudocode o compu ing 𝜑𝑈(𝑥)is p o ided in Algo i hm 1. The ollowing p ope y holds. Lemma 1 (Con inui y). Le 𝑥 ∈𝐵𝑛. Then, lim 𝑦→𝑥𝜉𝑈(𝑦)=𝜉𝑈(𝑥). P oo . I 𝑥 ∈| Del(𝑈)| hen he esul holds due o he con inui y o he ba ycen ic coo dina es ans o ma ion. I 𝑥 ∉| Del(𝑈)|, since he o igin 𝑜 ∈| Del(𝑈)|, hen ||𝑥|| ≠0. The e o e, o 𝑦close o 𝑥, ||𝑦|| ≠0and 𝑤𝑦=𝑅𝑦 ||𝑦||∈ℝ𝑛. Besides, lim 𝑦→𝑥𝑤𝑦=𝑤𝑥 and he e o e In o ma ion Sciences 667 (2024) 120474 7 E. Paluzo-Hidalgo, R. Gonzalez-Diaz and M.A. Gu ié ez-Na anjo Algo i hm 1 Pseudocode o compu e 𝜑𝑈(𝑥) o 𝑥 ∈𝐵𝑛and a subse 𝑈o an inpu da ase 𝑉su ounded by a hype sphe e o adius 𝑅. Inpu : 𝑈⊂𝑉⊂ℝ𝑛labelled using a se o labels Λ ={𝜆1, … , 𝜆𝑘}, a adius 𝑅, and a poin 𝑥 ∈𝐵𝑛. Ou pu : The alue o 𝜑𝑈(𝑥) compu e Del(𝑈) 𝑊𝑘∶= {𝓁𝑗∶= (0, 𝑗−1 …, 0, 1, 0, 𝑘−𝑗 …, 0) o 𝑗∈1, 𝑘} ini an emp y ma ix 𝑈 ini an emp y ec o 𝑥𝑖(𝑥) o 𝑢 ∈𝑈do i 𝜆𝑗is he label o 𝑢 hen add 𝓁𝑗as a column o 𝑈 end i end o o 𝜎maximal simplex o Del(𝑈)do compu e 𝑏(𝑥) i 𝑏𝑗≥0 o all 𝑗∈0, 𝑛 hen s op: compu e 𝜉𝑈(𝑥) end i end o i 𝜉𝑈(𝑥)is emp y hen compu e Γ, 𝑤𝑥, 𝜇and 𝜎 compu e 𝑏(𝑥)wi h espec o 𝜎and 𝜉𝑈(𝑥) end i 𝜑𝑈(𝑥) ∶= so max(𝑈⋅𝜉𝑈(𝑥)) lim 𝑦→𝑥𝜉𝑈(𝑦)=𝜉𝑈(𝑥), concluding he p oo . □ Lemma 2 (Consis ence). Le 𝜑(0) be he map defined in Subsec ion 2.3. I 𝑈=𝑉and 𝜑(0) 𝑈(𝑢) =𝜑(0)(𝑢) o all 𝑢 ∈𝑈 hen a g max 𝜑𝑈(𝑥) = a g max 𝜑(𝑥); o all 𝑥∈Del(𝑉). P oo . Le us obse e ha i 𝑈=𝑉and 𝜑(0) 𝑈(𝑢) =𝜑(0)(𝑢) o all 𝑢 ∈𝑈 hen, o any 𝑥 ∈Del(𝑉), we ha e ha : 𝜑(𝑥)= ∑ 𝑡∈1,𝑚 𝜉𝑡(𝑥)𝜑(0)(𝑢𝑡)= ∑ 𝑡∈1,𝑚 𝜉𝑡(𝑥)𝜑(0) 𝑈(𝑢𝑡). Then, 𝜑𝑈(𝑥) =so max(𝜑(𝑥))and a g max 𝜑𝑈(𝑥) = a gmax𝜑(𝑥).□ One o he keys o ou s udy is he iden ifica ion o he poin s o ℝ𝑛alloca ed inside a gi en simplex, wi h he se o all p obabili y dis ibu ions wi h 𝑛 +1suppo alues. In his way, he ba ycen ic coo dina es o a poin can be seen as a p obabili y dis ibu ion. F om his poin o iew, gi en 𝑥 ∈𝐵𝑛, hen 𝜑(𝑥)and 𝜑𝑈(𝑥)a e bo h in he se |𝐿|o p obabili y dis ibu ions wi h 𝑘suppo poin s. This is why he ca ego ical c oss-en opy loss unc ion will be used o compa e he simila i y be ween 𝜑and 𝜑𝑈. Specifically, o 𝑣 ∈𝑉, is defined as: (𝜑𝑈,𝜑,𝑣)=− ∑ ℎ∈1,𝑘 𝑦ℎlog(𝑠ℎ), whe e 𝜑(0)(𝑣) =(𝑦1, … , 𝑦𝑘)and 𝜑𝑈(𝑣) =(𝑠1, … , 𝑠𝑘). The ollowing lemma es ablishes a specific se 𝑈⊂𝑉 and a unc ion 𝜑𝑈such ha (𝜑𝑈, 𝜑, 𝑣) =0 o all 𝑣 ∈𝑉. Lemma 3 (-op imum simplicial map). Le 𝑈be a subse o 𝑉sa is ying, o all 𝑢 ∈𝑈, ha : 1. 𝜑(0) 𝑈(𝑢) =𝜑(0)(𝑢), and 2. i 𝑣 ∈𝑉such ha 𝜑(0)(𝑣) ≠𝜑(0)(𝑢)and ⟨𝑣, 𝑢⟩ ∈Del(𝑉) hen 𝑣 ∈𝑈. Then, a g max 𝜑𝑈(𝑥) = a g max 𝜑(𝑥); o all 𝑥∈Del(𝑈). P oo . As p o ed in [6], unde he assump ions s a ed in his lemma, we ha e ha , o all 𝑥 ∈<| Del(𝑈)|: 𝜑(𝑥)= ∑ 𝑡∈1,𝑚 𝜉𝑡(𝑥)𝜑(0) 𝑈(𝑢𝑡) and hen a g max 𝜑𝑈(𝑥) = a g max 𝜑(𝑥).□ In o ma ion Sciences 667 (2024) 120474 8 E. Paluzo-Hidalgo, R. Gonzalez-Diaz and M.A. Gu ié ez-Na anjo Un o una ely, o compu e he subse 𝑈sa is ying he condi ions s a ed in Lemma 3, we need o compu e he en i e iangula ion Del(𝑉)which is compu a ionally expensi e, as we ha e al eady men ioned abo e. 4. T aining SMNNs The no el idea o his pape is o lea n he unc ion 𝜑(0) 𝑈 o any gi en 𝑈⊂𝑉, using he g adien descen algo i hm, in o de o minimize he loss unc ion (𝜑𝑈, 𝜑, 𝑣) o any 𝑣 ∈𝑉. The ollowing esul p o ides an exp ession o he g adien o in e ms o he unc ions 𝜑𝑈and 𝜑, and he se 𝑉. Theo em 1. Le 𝑈={𝑢1, … , 𝑢𝑚}be a subse wi h 𝑚elemen s aken om a fini e se o poin s 𝑉∈ℝ𝑛 agged wi h labels aken om a se o 𝑘labels. Le 𝜑𝑈∶𝐵𝑛→|𝐿|and 𝜑(0) ∶𝑉→𝑊𝑘. Le us conside ha {𝜑(0) 𝑈(𝑢𝑡)=(𝑝𝑡 1,…,𝑝 𝑡 𝑘)∶ 𝑡∈1,𝑚} is a se o a iables. Then, o 𝑣 ∈𝑉, 𝜕(𝜑𝑈,𝜑,𝑣) 𝜕𝑝𝑡 𝑗 =(𝑠𝑗−𝑦𝑗)𝜉𝑡(𝑣) whe e 𝑗∈1, 𝑘, 𝑡 ∈1, 𝑚, 𝜑(0)(𝑣) =(𝑦1, … , 𝑦𝑘)and 𝜑𝑈(𝑣) =(𝑠1, … , 𝑠𝑘). P oo . We ha e: 𝜕(𝜑𝑈,𝜑,𝑣) 𝜕𝑝𝑡 𝑗 =− 𝜕(∑ℎ∈1,𝑘𝑦ℎlog(𝑠ℎ)) 𝜕𝑝𝑡 𝑗 =− ∑ ℎ∈1,𝑘 𝑦ℎ 𝜕log(𝑠ℎ) 𝜕𝑝𝑖 𝑗 =− ∑ ℎ∈1,𝑘 𝑦ℎ 𝜕log(𝑠ℎ) 𝜕𝑧𝑗 𝜕𝑧𝑗 𝜕𝑝𝑖 𝑗 . Since 𝑠ℎ=𝑒𝑧ℎ ∑𝑡∈1,𝑘𝑒𝑧𝑡 hen 𝜕log(𝑠ℎ) 𝜕𝑧𝑗 = 𝜕log(𝑒𝑧ℎ ∑𝑡∈1,𝑘𝑒𝑧𝑡) 𝜕𝑧𝑗 =𝜕log(𝑒𝑧ℎ) 𝜕𝑧𝑗 − 𝜕log(∑𝑡∈1,𝑘𝑒𝑧𝑡) 𝜕𝑧𝑗 =𝜕𝑧ℎ 𝜕𝑧𝑗 −1 ∑𝑡∈1,𝑘𝑒𝑧𝑡∑ 𝑡∈1,𝑘 𝜕𝑒𝑧𝑡 𝜕𝑧𝑗 =𝛿ℎ𝑗 −𝑒𝑧𝑗 ∑𝑡∈1,𝑘𝑒𝑧𝑡=𝛿ℎ𝑗 −𝑠𝑗. Besides, since 𝑧𝑗=∑ℎ∈1,𝑚𝜉ℎ(𝑣)𝑝ℎ 𝑗 hen 𝜕𝑧𝑟 𝜕𝑝𝑡 𝑗 =∑ ℎ∈1,𝑚 𝜉ℎ(𝑣)𝜕𝑝ℎ 𝑟 𝜕𝑝𝑡 𝑗 =𝜉𝑡(𝑣). Finally, 𝜕(𝜓,𝜑,𝑣) 𝜕𝑝𝑡 𝑗 =− ∑ ℎ∈1,𝑘 𝑦ℎ(𝛿ℎ𝑗 −𝑠𝑗)𝜉𝑡(𝑣) =−𝜉𝑡(𝑣)(∑ ℎ∈1,𝑘 𝑦ℎ𝛿ℎ𝑗 −𝑠𝑗∑ ℎ∈1,𝑘 𝑦ℎ) =(𝑠𝑗−𝑦𝑗)𝜉𝑡(𝑣).□ Le us now see how we add ainabili y o he SMNN 𝜑𝑈induced by 𝜑𝑈. Le 𝑉be he aining se and le 𝑈be a suppo se lying in he same space as 𝑉. Fi s , assuming ha 𝑈={𝑢1, … , 𝑢𝑚}has 𝑚elemen s, hen 𝜑𝑈is a mul iclass pe cep on wi h an inpu laye wi h 𝑚neu ons ha p edic s he ℎ- h label o ℎ ∈1, 𝑘using he o mula: 𝜑𝑈(𝑥)=so max( ⋅𝜉𝑈(𝑥)) In o ma ion Sciences 667 (2024) 120474 9 E. Paluzo-Hidalgo, R. Gonzalez-Diaz and M.A. Gu ié ez-Na anjo Fig. 5. Two-dimensional syn he ic da ase wi h poin s di ided in o wo classes: blue and yellow. T iangle-shaped poin s belong o he es se and squa e-shaped poin s belong o he suppo se 𝑈. The diamond-shaped poin is he e ex on he hype sphe e ( he blue ci cum e ence) used o classi y he iangle-shaped poin 𝑣 (su ounded by a small ed ci cum e ence) ou side he iangula ion. whe e  =(𝑝𝑡 𝑗)𝑗∈1,𝑘,𝑡∈1.𝑚is a ma ix o weigh s and 𝜉𝑈(𝑥) ∈ℝ𝑚is ob ained om he ba ycen ic coo dina es o 𝑥 ∈𝐵𝑛as in Sec ion 3. Le us obse e ha so max ( ⋅𝜉𝑈(𝑥))∈|𝐿|. The idea is o modi y he ini ial alues o 𝜑(0) 𝑈(𝑢𝑡)=(𝑝𝑡 1,…,𝑝 𝑡 𝑘) o 𝑢𝑡∈𝑈and 𝑡∈1,𝑚, in o de o ob ain new alues o 𝜑𝑈(𝑣) o 𝑣 ∈𝑉in a way ha he e o (𝜑𝑈,𝜑, 𝑣)dec eases. We will do i by a oiding ecompu ing Del(𝑈)o he ba ycen ic coo dina es (𝑏0(𝑣), … , 𝑏𝑛(𝑣)) o each 𝑣 ∈𝑉du ing he aining p ocess. In his way, gi en 𝑣 ∈𝑉, i 𝑣 ∈| Del(𝑈)|, we compu e he maximal simplex 𝜎=⟨𝑢𝑖0, … , 𝑢𝑖𝑛⟩ ∈Del(𝑈)such ha 𝑣 ∈|𝜎|and 𝑖ℎ∈1, 𝑚 o ℎ ∈0, 𝑛. I 𝑣 ∉| Del(𝑈)|, we compu e 𝑤 ∈𝑆𝑛and he simplex 𝜎=⟨𝑤, 𝑢𝑖1, … , 𝑢𝑖𝑛⟩such ha 𝑣 ∈|𝜎|and 𝑖ℎ∈1, 𝑚 o ℎ ∈1, 𝑛. Then we compu e he ba ycen ic coo dina es 𝑏(𝑣)o 𝑣wi h espec o 𝜎and he poin 𝜉𝑈(𝑥) =(𝜉1(𝑥), … , 𝜉𝑚(𝑥)) ∈ℝ𝑚 as in Sec ion 3. Using he g adien descen algo i hm, we upda e he a iables 𝑝𝑡 𝑗 o 𝑗∈1, 𝑘and 𝑡 ∈1, 𝑚as ollows: 𝑝𝑡 𝑗∶= 𝑝𝑡 𝑗−𝜂 𝜕(𝜑𝑈,𝜑, 𝑣) 𝜕𝑝𝑡 𝑗 =𝑝𝑡 𝑗−𝜂(𝑠𝑗−𝑦𝑗)𝜉𝑡(𝑣). An illus a i e pic u e o he ole o each poin in a simple wo-dimensional bina y classifica ion p oblem is p o ided in Fig. 5. The pseudocode o he me hod o ain SMNNs using S ochas ic G adien Descen is p o ided in Algo i hm 2. Algo i hm 2 Pseudocode o he p oposed me hod o ain SMNNs using SGD. Inpu : Da ase 𝑉⊂ℝ𝑛su ounded by a hype sphe e o adius 𝑅and a model 𝜑𝑈. Pa ame e : 𝜂>0 Ou pu : The ained model 𝜑𝑈 ini  , he ma ix o weigh s o 𝜑𝑈 o 𝑣 ∈𝑉do compu e 𝜉𝑈(𝑣)as in Sec ion 3 o each column 𝑝o do 𝑝 ∶= 𝑝 −𝜂𝜕(𝜑𝑈,𝜑,𝑣) 𝜕𝑝 end o end o 𝜑𝑈(𝑥) ∶= so max(  ⋅𝜉𝑈(𝑥)) 5. Explainabili y In his sec ion, we p o ide insigh in o he explainabili y capabili y o SMNNs. In he li e a u e, many diffe en app oaches can be ound o wha is an explana ion o he p edic ion o an AI model. In ou case, explainabili y will be p o ided based on simila i ies and dissimila i ies o he poin 𝑥 o be explained wi h he poin s co esponding o he e ices o he simplex 𝜎con aining i . Based on