Trainable and explainable simplicial map neural networks

Trainable and explainable simplicial map neural networks Eduardo Paluzo-Hidalgo a,∗, Rocio Gonzalez-Diazb, Miguel A. Gutiérrez-Naranjo c aDepartment of Quantitative Methods, Universidad Loyola Andalucía, Campus Sevilla, Dos Hermanas, Seville, Spain bDepartamento de Matemática Aplicada I, Universidad de Sevilla, Seville, Spain cDepartamento de Ciencias de la Computación e Inteligencia Artificial, Universidad de Sevilla, Seville, Spain A R T I C L E I N F O A B S T R A C T Keywords: Training neural network Simplicial maps Explainable artificial intelligence Simplicial map neural networks (SMNNs) are topology-based neural networks with interesting properties such as universal approximation ability and robustness to adversarial examples under appropriate conditions. However, SMNNs present some bottlenecks for their possible application in high-dimensional datasets. First, SMNNs have precomputed fixed weight and no SMNN training process has been defined so far, so they lack generalization ability. Second, SMNNs require the construction of a convex polytope surrounding the input dataset. In this paper, we overcome these issues by proposing an SMNN training procedure based on a support subset of the given dataset and replacing the construction of the convex polytope by a method based on projections to a hypersphere. In addition, the explainability capacity of SMNNs and effective implementation are also newly introduced in this paper. 1. Introduction In recent years, Artificial Intelligence (AI) methods in general and Machine Learning methods in particular have reached success in real-life problems that were unexpected only a few years ago. Many different areas have contributed to this development. Among them, we can cite the research on new theoretical algorithms, the increasing computational power of the latest generation hardware, and the rapid access to a huge amount of data. Such a combination of factors leads to the development of increasingly complex self-regulated AI methods. Many AI models currently used are based on backpropagation algorithms, which train and regulate themselves to achieve a goal, such as classification, recommendation, or prediction. These self-regulating models achieve some kind of knowledge as they successfully evaluate test data independent of the data used to train them. Nonetheless, such knowledge is usually expressed in a non-human-readable way. To fill the gap between the recent development of AI models and their social use, many researchers have focused on the development of Explainable Artificial Intelligence (XAI), which consists of a set of techniques to provide clear, understandable, transparent, intelligible, trustworthy, and interpretable explanations of the decisions, predictions, and reasoning processes made by the AI models, rather than just presenting their output, especially in domains where AI decisions can have significant consequences on human life. A global taxonomy of interpretable AI with the aim of unifying terminology to achieve clarity and efficiency in the definition of * Corresponding author. E-mail addresses: [email protected] (E. Paluzo-Hidalgo), [email protected] (R. Gonzalez-Diaz), [email protected] (M.A. Gutiérrez-Naranjo). URLs: https://personal.us.es/rogodi (R. Gonzalez-Diaz), http://www.cs.us.es/~naranjo/ (M.A. Gutiérrez-Naranjo). https://doi.org/10.1016/j.ins.2024.120474 Received 26 October 2023; Received in revised form 22 January 2024; Accepted 11 March 2024 Available online 18 March 2024 0020-0255/© 2024 The Author(s). (http://creativecommons.org/licenses/by/4.0/). Information Sciences 667 (2024) 120474 2 E. Paluzo-Hidalgo, R. Gonzalez-Diaz and M.A. Gutiérrez-Naranjo regulations for the development of ethical and reliable AI can be found in [1]. Moreover, a nice introduction and general vision can be found in [2]. Another clarifying paper with definitions, concepts, and applications of XAI in [3]. The so-called Simplicial Map Neural Networks (SMNNs) were introduced in [4]as a constructive approach to the problem of approximating a continuous function on a compact set in a triangulated space. Since the original aim of the definition of SMNNs was focused on building a constructive approach, the computation of their weights was not based on an optimization process, as usual in neural networks, but on a deterministic calculus. The architecture of SMNNs and the computation of the set of weights are based on a combinatorial topology tool called simplicial maps. Moreover, SMNNs can be used for classification and can be constructed to be robust to adversarial examples [5]. Besides, their architecture can be reduced while maintaining accuracy [6], being invariant to transformation if the transformation preserves the barycentric coordinates (scale, rotation, symmetries, etc.). As defined in [4,6,5], SMNNs are built as a two-hidden-layer feed-forward network where the set of weights is precomputed based on the calculation of a triangulated convex polytope surrounding the input data. As other approximations to continuous functions with arbitrary precision (see, for example, [7]), SMNNs have fixed weights, which means that the weights depend only on the triangulation made with the points of the dataset as the support set and no training process is applied. Summing up, some of the limitations of the SMNNs until now are that they are costly to calculate since the number of neurons is proportional to the number of simplices of the triangulation supported on the input dataset, and they suffer from overfitting and therefore not generalize well. These aspects make SMNNs not used in practice so far, although the idea of relating simplicial maps to neural networks is disruptive and provides a new bridge that can enrich both areas. In this paper, we propose a method to make SMNNs efficient by reducing their size (in terms of the number of neurons that depends on the vertices of the triangulation) and that successfully makes SMNNs trainable and with generalization ability. Besides, we also present a study of the selection of the vertices from which we obtain the triangulation. Although SMNNs consider the vertices of a simplex as part of the necessary information for the classification task, the approach presented in this paper is far from the classic Machine Learning instance-based methods. Such methods rely on a deterministic computation based on distances, but, in the approach presented in this paper, the computation of the weights is the result of an optimization method in a probability distribution space. Finally, from an XAI point of view, we will see in this paper that SMNNs are explainable models since all decision steps to compute the output of SMNNs are understandable and transparent, and therefore trustworthy. The paper is organized as follows. First, some concepts of computational topology and the definition of SMNNs are recalled in Section 2. Next, in Section 3we develop several technical details needed for the SMNN training process, which will be introduced in Section 4. Section 5is devoted to the explainability of the model. Section 6is devoted to discussion and limitations. Finally, the paper ends with some experiments and conclusions. 2. Background In this section, we assume that the reader is familiar with the basic concepts of computational topology. For a comprehensive presentation, we refer to [8]. 2.1. Simplicial complexes Consider a finite set of points 𝑉={𝑣1, … , 𝑣𝛽} ⊂ℝ𝑛whose elements will be called vertices. A subset 𝜎=⟨𝑣𝑖0,𝑣 𝑖1,…,𝑣 𝑖𝑑⟩ of 𝑉with 𝑑+1vertices (in general position) is called a 𝑑-simplex. The convex hull of the vertices of 𝜎will be denoted by |𝜎|and corresponds to the set: {𝑥∈ℝ𝑛∶𝑥=∑ 𝑗∈0,𝑑 𝑏𝑗(𝑥)𝑣𝑖𝑗} where 𝑎, 𝑏 ={𝑎, 𝑎 +1, … , 𝑏}for 𝑎 <𝑏 ∈ℤ, and 𝑏(𝑥)=(𝑏0(𝑥),𝑏 1(𝑥),…,𝑏 𝑑(𝑥)) are called the barycentric coordinates of 𝑥with respect to 𝜎, and satisfy that: ∑ 𝑗∈0,𝑑 𝑏𝑗(𝑥)=1 and 𝑏𝑗(𝑥)≥0∀𝑗∈0,𝑑. The barycentric coordinates of 𝑥can be interpreted as masses placed at the vertices of 𝜎so 𝑥is the center of mass. All these masses are positive if and only if 𝑥is inside 𝜎. For example, let us consider the 1-simplex 𝜖=⟨𝑣𝑖0, 𝑣𝑖1⟩which is composed of two vertices of 𝑉. Then |𝜖|is the set of points in ℝ𝑛corresponding to the edge with endpoints 𝑣𝑖0and 𝑣𝑖1, and if, for example, 𝑏(𝑥) =(0.5, 0.5) then 𝑥is the midpoint of |𝜖|. A simplicial complex 𝐾with vertex set 𝑉consists of a finite collection of simplices satisfying that if 𝜎∈𝐾then either 𝜎=⟨𝑣⟩for some 𝑣 ∈𝑉or any face (that is, a nonempty subset) of 𝜎is a simplex of 𝐾. Furthermore, if 𝜎, 𝜇∈𝐾then |𝜎| ∩|𝜇| =∅or |𝜎| ∩|𝜇| =|𝛾| for some 𝛾∈𝐾. The set ⋃𝜎∈𝐾|𝜎|will be denoted by |𝐾|. A maximal simplex of 𝐾is a simplex that is not the face of any other Information Sciences 667 (2024) 120474 3 E. Paluzo-Hidalgo, R. Gonzalez-Diaz and M.A. Gutiérrez-Naranjo Fig. 1. On the left, two triangles that do not intersect in a common face (an edge or a vertex). On the right, the geometric representation |𝐾|of a pure 2-simplicial complex 𝐾composed of three maximal 2-simplices (the triangles 𝜎1, 𝜎2and 𝜎3). The edge 𝜇2is a common face of 𝜎2and 𝜎3. The edge 𝜇1is a face of 𝜎1. simplex of 𝐾. If the maximal simplices of 𝐾are all 𝑑-simplices then 𝐾is called a pure 𝑑-simplicial complex. These concepts are illustrated in Fig. 1. The barycentric coordinates of 𝑥with respect to the simplicial complex |𝐾|are defined as the barycentric coordinates of 𝑥with respect to 𝜎∈𝐾such that 𝑥 ∈|𝜎|. Let us observe that the barycentric coordinates of 𝑥 ∈|𝐾|are unique. An example of simplicial complexes is the Delaunay triangulation Del(𝑉)defined from the Voronoi diagram of a given finite set of vertices 𝑉. The following result extracted from [9, page 48] is just an alternative definition of Delaunay triangulations. The empty ball property [9]: Any subset 𝜎of 𝑉is a simplex of Del(𝑉)if and only if |𝜎|has a circumscribing open ball empty of points of 𝑉. 2.2. Simplicial maps Let 𝐾be a pure 𝑛-simplicial complex and 𝐿a pure 𝑘-simplicial complex with vertex sets 𝑉and 𝑊, respectively. The map 𝜑(0) ∶𝑉→𝑊is called a vertex map if it satisfies that the set obtained from {𝜑(0)(𝑣𝑖0), … , 𝜑(0)(𝑣𝑖𝑑)}after removing duplicated vertices is a simplex in 𝐿whenever ⟨𝑣𝑖0, … , 𝑣𝑖𝑑⟩is a simplex in 𝐾. The vertex map 𝜑(0) always induces a continuous function, called a simplicial map 𝜑 ∶|𝐾| →|𝐿|, which is defined as follows. Let 𝑏(𝑥) =(𝑏0(𝑥), … , 𝑏𝑛(𝑥)) be the barycentric coordinates of 𝑥 ∈|𝐾|with respect to 𝜎=⟨𝑣𝑖0, … , 𝑣𝑖𝑛⟩ ∈𝐾. Then 𝜑(𝑥)= ∑ 𝑗∈0,𝑛 𝑏𝑗(𝑥)𝜑(0)(𝑣𝑖𝑗). Let us observe that 𝜑(𝑥) =𝜑(0)(𝑥)if 𝑥 ∈𝑉. A special kind of simplicial map used to solve classification tasks will be introduced in the next subsection. 2.3. Simplicial maps for classification tasks Next, we will show how a simplicial map can be used to solve a classification problem (see [5]for details). From now on, we will assume that the input dataset is a finite set of points 𝑉in ℝ𝑛together with a set of 𝑘labels Λsuch that each 𝑣 ∈𝑉is tagged with a label 𝜆taken from Λ. Firstly, the intuition is that the space surrounding the dataset is labelled as unknown. For this, we add a new label to Λ, called unknown label, and a one-hot encoding representation 𝑊𝑘+1 ⊂ℝ𝑘+1 of these 𝑘 +1labels being: 𝑊𝑘+1 ={𝓁𝑗=(0,𝑗 …,0,1,0,𝑘−𝑗 …,0) ∶ 𝑗∈1,𝑘}, where the one-hot vector 𝓁𝑗encodes the 𝑗-th label of Λfor 𝑗∈1, 𝑘and where 𝓁0encodes the unknown label. We now consider a convex polytope with a vertex set 𝑃surrounding the set 𝑉. The polytope always exists since 𝑉is finite. Next, we define a map 𝜑(0) ∶𝑉∪𝑃→𝑊𝑘+1 mapping each vertex 𝑣 ∈𝑉tagged with label 𝜆to the one-hot vector in 𝑊𝑘+1 that encodes the label 𝜆. The vertices of 𝑃are sent to the vertex 𝓁0. Observe that 𝜑(0) is a vertex map. Let 𝐿denote the simplicial complex with vertex set 𝑊𝑘+1 consisting of only one maximal 𝑘-simplex and let Del(𝑉∪𝑃)denote the Delaunay triangulation computed for the set of points 𝑉∪𝑃. Note that | Del(𝑉∪𝑃)| =. The simplicial map 𝜑 ∶→|𝐿|is induced by the vertex map 𝜑(0) as explained in Subsection 2.2. Remark 1. The space |𝐿|can be interpreted as the discrete probability distribution space Ω𝑘+1 with 𝑘 +1variables. As an example, in Fig. 2, on the left, we can see a dataset with four points 𝑉={𝑏, 𝑐, 𝑘, 𝑑}, labelled red and blue. The green points 𝑃={𝑎, 𝑒, 𝑔, 𝑓}are the vertices of a convex polytope containing 𝑉and are sent by 𝜑(0) to the green vertex 𝓁0on the right. The simplicial complex 𝐾=Del(𝑉∪𝑃)is drawn on the left and consists of ten maximal 2-simplices. On the right, the simplicial complex 𝐿consists of one maximal 2-simplex. The dotted arrows illustrate some examples of 𝜑 ∶→|𝐿|. Information Sciences 667 (2024) 120474 4 E. Paluzo-Hidalgo, R. Gonzalez-Diaz and M.A. Gutiérrez-Naranjo Fig. 2. Illustration of a simplicial map for a classification task. 2.4. Simplicial map neural networks Artificial neural networks can be seen as parametrized real-valued mappings between multidimensional spaces. Such mappings are the composition of several maps (usually many of them) that can be structured in layers. In [5], the simplicial map 𝜑defined above was represented as a two-hidden-layer feed-forward neural network 𝜑. This kind of artificial neural network is called simplicial map neural network (SMNN). In the original definition [5], the first hidden layer of an SMNN computes the barycentric coordinates of Del(𝑉∪𝑃). However, we will see here that if we precompute the barycentric coordinates, we can simplify the architecture of 𝜑as follows. As before, consider an input dataset consisting of a finite set 𝑉⊂ℝ𝑛endowed with a set of 𝑘labels and a convex polytope  with a set of vertices 𝑃surrounding 𝑉. Let Del(𝑉∪𝑃)be the Delaunay triangulation with vertex set 𝑉∪𝑃={𝜔1,…,𝜔 𝛼}⊆ℝ𝑛. Then, 𝜑(0) ∶𝑉∪𝑃→𝑊𝑘+1 is a vertex map. Let us assume that, given 𝑥 ∈, we precompute the barycentric coordinates 𝑏(𝑥) = (𝑏0(𝑥), … , 𝑏𝑛(𝑥)) ∈ℝ𝑛+1 of 𝑥with respect to the 𝑛-simplex 𝜎=⟨𝜔𝑖0, … , 𝜔𝑖𝑛⟩ ∈Del(𝑉∪𝑃)such that 𝑥 ∈|𝜎|, and that we also precompute the vector 𝜉(𝑥)=(𝜉1(𝑥),…,𝜉 𝛼(𝑥)) ∈ ℝ𝛼 satisfying that, for 𝑡 ∈1, 𝛼, 𝜉𝑡(𝑥) =𝑏𝑗(𝑥)if 𝑖𝑗=𝑡for some 𝑗∈0, 𝑛. Let us remark that 𝜉(𝑥)should be a column vector, but we will use row notation, for simplicity. The SMNN 𝜑induced by 𝜑that predicts the ℎ-label of 𝑥, for ℎ ∈0, 𝑘, has the following architecture: •The number of neurons in the input layer is 𝛼. •The number of neurons in the output layer is 𝑘 +1. •The set of weights is represented as a (𝑘 +1) ×𝛼matrix such that the 𝑗-th column of is 𝜑(0)(𝜔𝑡)for 𝑡 ∈1, 𝛼. Then, 𝜑(𝑥)=⋅𝜉(𝑥). Observe that as defined so far, the SMNN weights are precomputed. Furthermore, the computation of the barycentric coordinates of the points around 𝑉implies the calculation of the convex polytope surrounding 𝑉. Finally, the computation of the Delaunay triangulation Del(𝑉∪𝑃)is costly if 𝑉∪𝑃has many points since its time complexity is 𝑂(𝑛 log 𝑛 +𝑛⌈𝑑 2⌉)(see [9, Chapter 4]). In the next sections, we will propose some techniques to overcome the SMNN construction drawbacks while maintaining its advantages. We will see that one way to overcome the computation of the convex polytope is to consider a hypersphere 𝑆𝑛 instead. We will also see how to avoid the use of the artificially created unknown label. Furthermore, to reduce the cost of Delaunay computation and add trainability to 𝜑to avoid overfitting, a subset 𝑈⊂𝑉 will be considered. The set 𝑉will be used to train and test a map 𝜑(0) 𝑈∶𝑈→ℝ𝑘. Such a map will induce a continuous function 𝜑𝑈∶𝐵𝑛→|𝐿|which approximates 𝜑. 3. The unknown boundary and the function 𝝋𝑼 In this section, we will see how to compute a function 𝜑𝑈that approximates the simplicial map 𝜑and avoids the computation of the convex polytope and the artificial consideration of the unknown label, reducing, at the same time, the computation of the Delaunay triangulation used to construct SMNNs. The general description of the methodology is described in Algorithm 1. First, let us compute a hypersphere surrounding 𝑉. One of the simplest ways to do that is to translate 𝑉so that its center of mass is placed at the origin 𝑜 ∈ℝ𝑛. Then, the hypersphere Information Sciences 667 (2024) 120474 5 E. Paluzo-Hidalgo, R. Gonzalez-Diaz and M.A. Gutiérrez-Naranjo Fig. 3. An example of the point 𝑤𝑥computed from 𝑥and the (𝑛−1)-simplex 𝜇=⟨𝑢1,𝑢 2⟩∈Γsuch that 𝑥∈|𝜎|for 𝜎=⟨𝑤𝑥,𝑢 1,𝑢 2⟩. 𝑆𝑛={𝑤∈ℝ𝑛∶||𝑤||=𝑅} such that 𝑅 >max{||𝑣|| ∶𝑣 ∈𝑉}satisfies that 𝑆𝑛surrounds 𝑉. Second, let us assume that we have selected a subset 𝑈= {𝑢1, … , 𝑢𝑚} ⊆𝑉 (we will compare different strategies to select 𝑈in Section 7) and that we have computed Del(𝑈). Then, we have that 𝑉⊂𝐵 𝑛={𝑥∈ℝ𝑛∶||𝑥||≤𝑅}and 𝑜∈|Del(𝑈)|. Let us consider the boundary of Del(𝑈), denoted as 𝛿Del(𝑈), which consists of the set of (𝑛 −1)-simplices that are faces of exactly one maximal simplex of Del(𝑈). Now, let us define 𝜉𝑈(𝑥) ∈ℝ𝑚for any 𝑥 ∈𝐵𝑛as follows. Given 𝑥 ∈𝐵𝑛, to find the 𝑛-simplex 𝜎=⟨𝜔0, … , 𝜔𝑛⟩such that 𝑥 ∈|𝜎|, we have to consider two cases: 𝑥 ∈| Del(𝑈)|and 𝑥 ∉| Del(𝑈)|. If 𝑥 ∈| Del(𝑈)|then 𝜎is the 𝑛-simplex in Del(𝑈)such that 𝑥 ∈|𝜎|. If 𝑥 ∉| Del(𝑈)|then 𝜎is a new 𝑛-simplex defined by the vertices of an (𝑛 −1)-simplex of 𝛿Del(𝑈)and a new vertex consisting of the projection of 𝑥to 𝑆𝑛. Specifically, if 𝑥 ∉| Del(𝑈)|then 𝜎is computed in the following way: 1. Consider the set Γ={𝜇∈𝛿Del(𝑈)∶(𝑁⋅𝑢𝑖0+𝑐)(𝑁⋅𝑥+𝑐)<0} where 𝑁is the vector normal to the hyperplane containing 𝜇=⟨𝑢𝑖1, … , 𝑢𝑖𝑛⟩, 𝑐=𝑁⋅𝑢𝑖1, and 𝑢𝑖0∈𝑈such that ⟨𝑢𝑖0, 𝑢𝑖1, … , 𝑢𝑖𝑛⟩ ∈ Del(𝑈). 2. Compute the point 𝑤𝑥=𝑅𝑥 ||𝑥||∈𝑆𝑛. 3. Find 𝜎=⟨𝑤𝑥, 𝑢𝑖1, … , 𝑢𝑖𝑛⟩such that 𝜇=⟨𝑢𝑖1,…,𝑢 𝑖𝑛⟩∈Γand 𝑥∈|𝜎|. Observe that, by construction, 𝜇always exists since | Del(𝑈)|is a convex polytope. Now, let 𝑏(𝑥) =(𝑏0(𝑥), … , 𝑏𝑛(𝑥)) ∈ℝ𝑛+1 be the barycentric coordinates of 𝑥with respect to 𝜎. Then, 𝜉𝑈(𝑥) =(𝜉1(𝑥), … , 𝜉𝑚(𝑥)) is the point in ℝ𝑚satisfying that, for 𝑡 ∈1, 𝑚, 𝜉𝑡(𝑥)={𝑏𝑗(𝑥)if 𝑢𝑡=𝜔𝑗for some 𝑗∈0,𝑛, 0otherwise. Observe that 𝜉𝑈(𝑥)always exists and is unique. An example of points 𝑥and 𝑤𝑥and simplex 𝜇is shown in Fig. 3and Example 1. Let us observe that, thanks to the new definition of 𝜉𝑈(𝑥)for 𝑥 ∈𝐵𝑛, if we have a map 𝜑(0) 𝑈∶𝑈→ℝ𝑘then it induces a continuous function 𝜑𝑈∶𝐵𝑛→|𝐿|defined for any 𝑥 ∈𝐵𝑛as: 𝜑𝑈(𝑥)=softmax (∑𝑡∈1,𝑚𝜉𝑡(𝑥)𝜑(0) 𝑈(𝑢𝑡)) =softmax(𝑈⋅𝜉𝑈(𝑥)) where for 𝑧 =(𝑧1, … , 𝑧𝑘) ∈ℝ𝑘, softmax(𝑧)=(𝑒𝑧1 ∑ℎ∈1,𝑘𝑒𝑧ℎ ,…,𝑒𝑧𝑘 ∑ℎ∈1,𝑘𝑒𝑧ℎ). Information Sciences 667 (2024) 120474 6 E. Paluzo-Hidalgo, R. Gonzalez-Diaz and M.A. Gutiérrez-Naranjo Fig. 4. The relative positions of the vertices 𝑣𝑖for 𝑖∈1,5and the points 𝑥1and 𝑥2of Example 1. Let us observe that, to obtain a categorical distribution from 𝜑𝑈(𝑥) ∈ℝ𝑘, we could just divide each of its coordinates by the total sum. However, softmax is introduced here to obtain a simplified formula for the gradient descent algorithm as shown in Theorem 1. Example 1. Let us consider 𝑉={𝑣1=(1 2,1 2),𝑣 2=(1 2,1),𝑣 3=(1,1 2),𝑣 4=(1,1)} with label 𝜆1=0for 𝑣𝑖with 𝑖 =1, 2and 𝜆2=1for 𝑣𝑖with 𝑖 =3, 4. Firstly, we translate 𝑉so that the center of mass of 𝑉is the origin 𝑜 ∈ℝ2. The translated dataset  𝑉is {𝑣1=(−1 4,−1 4),𝑣2=(−1 4,1 4),𝑣3=(1 4,−1 4),𝑣4=(1 4,1 4)}. Let us consider 𝑥1=(3 4, 3 5)and 𝑥2=(3 4, 5 4). Hence, the translated input data is 𝑥1=(0, −3 20 )and 𝑥2=(0, 1 2). To simplify the explanation of the method, consider 𝑈= 𝑉, that is, 𝑢𝑖=𝑣𝑖for 𝑖 ∈1, 4. Then, the maximal simplices of Del(𝑈)are 𝜎1=⟨𝑣1, 𝑣2, 𝑣3⟩and 𝜎1=⟨𝑣2, 𝑣3, 𝑣4⟩ •The matrix 𝑈is: (1100 0011 ) • Since the barycentric coordinates of 𝑥1with respect to |𝜎1|are (0.5, 0.3, 0.2) then 𝑥1is in |𝜎1| ⊂| Del(𝑈)|and 𝜉𝑈(𝑥1) = (0.3, 0.2, 0, 0.5). Then 𝜑𝑈(𝑥1)=softmax(𝑈⋅𝜉𝑈(𝑥1)) = (0.5,0.5). •On the other hand, the point 𝑥2is outside | Del(𝑈)|. Assuming that, for example, we have fixed 𝑅 =1, we add a new simplex 𝜎3={𝑤𝑥, 𝑣1, 𝑣2}where 𝑤𝑥=𝑣5=(0, 1) which is the projection of 𝑥2to the hypersphere of radius 𝑅centered in the origin. See Fig. 4. Then, the barycentric coordinates of 𝑥2with respect to 𝜎3are (0.33, 0.33, 0.33) and then 𝜉𝑈(𝑥2) =(0, 0.33, 0.33, 0), concluding that 𝜑𝑈(𝑥2)=softmax(𝑈⋅𝜉𝑈(𝑥2)) = (0.5,0.5). The pseudocode for computing 𝜑𝑈(𝑥)is provided in Algorithm 1. The following property holds. Lemma 1 (Continuity). Let 𝑥 ∈𝐵𝑛. Then, lim 𝑦→𝑥𝜉𝑈(𝑦)=𝜉𝑈(𝑥). Proof. If 𝑥 ∈| Del(𝑈)|then the result holds due to the continuity of the barycentric coordinates transformation. If 𝑥 ∉| Del(𝑈)|, since the origin 𝑜 ∈| Del(𝑈)|, then ||𝑥|| ≠0. Therefore, for 𝑦close to 𝑥, ||𝑦|| ≠0and 𝑤𝑦=𝑅𝑦 ||𝑦||∈ℝ𝑛. Besides, lim 𝑦→𝑥𝑤𝑦=𝑤𝑥 and therefore Information Sciences 667 (2024) 120474 7 E. Paluzo-Hidalgo, R. Gonzalez-Diaz and M.A. Gutiérrez-Naranjo Algorithm 1 Pseudocode to compute 𝜑𝑈(𝑥)for 𝑥 ∈𝐵𝑛and a subset 𝑈of an input dataset 𝑉surrounded by a hypersphere of radius 𝑅. Input: 𝑈⊂𝑉⊂ℝ𝑛labelled using a set of labels Λ ={𝜆1, … , 𝜆𝑘}, a radius 𝑅, and a point 𝑥 ∈𝐵𝑛. Output: The value of 𝜑𝑈(𝑥) compute Del(𝑈) 𝑊𝑘∶= {𝓁𝑗∶= (0, 𝑗−1 …, 0, 1, 0, 𝑘−𝑗 …, 0) for 𝑗∈1, 𝑘} init an empty matrix 𝑈 init an empty vector 𝑥𝑖(𝑥) for 𝑢 ∈𝑈do if 𝜆𝑗is the label of 𝑢then add 𝓁𝑗as a column of 𝑈 end if end for for 𝜎maximal simplex of Del(𝑈)do compute 𝑏(𝑥) if 𝑏𝑗≥0for all 𝑗∈0, 𝑛then stop: compute 𝜉𝑈(𝑥) end if end for if 𝜉𝑈(𝑥)is empty then compute Γ, 𝑤𝑥, 𝜇and 𝜎 compute 𝑏(𝑥)with respect to 𝜎and 𝜉𝑈(𝑥) end if 𝜑𝑈(𝑥) ∶= softmax(𝑈⋅𝜉𝑈(𝑥)) lim 𝑦→𝑥𝜉𝑈(𝑦)=𝜉𝑈(𝑥), concluding the proof. □ Lemma 2 (Consistence). Let 𝜑(0) be the map defined in Subsection 2.3. If 𝑈=𝑉and 𝜑(0) 𝑈(𝑢) =𝜑(0)(𝑢)for all 𝑢 ∈𝑈then arg max 𝜑𝑈(𝑥) = arg max 𝜑(𝑥); for all 𝑥∈Del(𝑉). Proof. Let us observe that if 𝑈=𝑉and 𝜑(0) 𝑈(𝑢) =𝜑(0)(𝑢)for all 𝑢 ∈𝑈then, for any 𝑥 ∈Del(𝑉), we have that: 𝜑(𝑥)= ∑ 𝑡∈1,𝑚 𝜉𝑡(𝑥)𝜑(0)(𝑢𝑡)= ∑ 𝑡∈1,𝑚 𝜉𝑡(𝑥)𝜑(0) 𝑈(𝑢𝑡). Then, 𝜑𝑈(𝑥) =softmax(𝜑(𝑥))and arg max 𝜑𝑈(𝑥) = argmax𝜑(𝑥).□ One of the keys to our study is the identification of the points of ℝ𝑛allocated inside a given simplex, with the set of all probability distributions with 𝑛 +1support values. In this way, the barycentric coordinates of a point can be seen as a probability distribution. From this point of view, given 𝑥 ∈𝐵𝑛, then 𝜑(𝑥)and 𝜑𝑈(𝑥)are both in the set |𝐿|of probability distributions with 𝑘support points. This is why the categorical cross-entropy loss function will be used to compare the similarity between 𝜑and 𝜑𝑈. Specifically, for 𝑣 ∈𝑉, is defined as: (𝜑𝑈,𝜑,𝑣)=− ∑ ℎ∈1,𝑘 𝑦ℎlog(𝑠ℎ), where 𝜑(0)(𝑣) =(𝑦1, … , 𝑦𝑘)and 𝜑𝑈(𝑣) =(𝑠1, … , 𝑠𝑘). The following lemma establishes a specific set 𝑈⊂𝑉 and a function 𝜑𝑈such that (𝜑𝑈, 𝜑, 𝑣) =0for all 𝑣 ∈𝑉. Lemma 3 (-optimum simplicial map). Let 𝑈be a subset of 𝑉satisfying, for all 𝑢 ∈𝑈, that: 1. 𝜑(0) 𝑈(𝑢) =𝜑(0)(𝑢), and 2. if 𝑣 ∈𝑉such that 𝜑(0)(𝑣) ≠𝜑(0)(𝑢)and ⟨𝑣, 𝑢⟩ ∈Del(𝑉)then 𝑣 ∈𝑈. Then, arg max 𝜑𝑈(𝑥) = arg max 𝜑(𝑥); for all 𝑥∈Del(𝑈). Proof. As proved in [6], under the assumptions stated in this lemma, we have that, for all 𝑥 ∈<| Del(𝑈)|: 𝜑(𝑥)= ∑ 𝑡∈1,𝑚 𝜉𝑡(𝑥)𝜑(0) 𝑈(𝑢𝑡) and then arg max 𝜑𝑈(𝑥) = arg max 𝜑(𝑥).□ Information Sciences 667 (2024) 120474 8 E. Paluzo-Hidalgo, R. Gonzalez-Diaz and M.A. Gutiérrez-Naranjo Unfortunately, to compute the subset 𝑈satisfying the conditions stated in Lemma 3, we need to compute the entire triangulation Del(𝑉)which is computationally expensive, as we have already mentioned above. 4. Training SMNNs The novel idea of this paper is to learn the function 𝜑(0) 𝑈for any given 𝑈⊂𝑉, using the gradient descent algorithm, in order to minimize the loss function (𝜑𝑈, 𝜑, 𝑣)for any 𝑣 ∈𝑉. The following result provides an expression of the gradient of in terms of the functions 𝜑𝑈and 𝜑, and the set 𝑉. Theorem 1. Let 𝑈={𝑢1, … , 𝑢𝑚}be a subset with 𝑚elements taken from a finite set of points 𝑉∈ℝ𝑛tagged with labels taken from a set of 𝑘labels. Let 𝜑𝑈∶𝐵𝑛→|𝐿|and 𝜑(0) ∶𝑉→𝑊𝑘. Let us consider that {𝜑(0) 𝑈(𝑢𝑡)=(𝑝𝑡 1,…,𝑝 𝑡 𝑘)∶ 𝑡∈1,𝑚} is a set of variables. Then, for 𝑣 ∈𝑉, 𝜕(𝜑𝑈,𝜑,𝑣) 𝜕𝑝𝑡 𝑗 =(𝑠𝑗−𝑦𝑗)𝜉𝑡(𝑣) where 𝑗∈1, 𝑘, 𝑡 ∈1, 𝑚, 𝜑(0)(𝑣) =(𝑦1, … , 𝑦𝑘)and 𝜑𝑈(𝑣) =(𝑠1, … , 𝑠𝑘). Proof. We have: 𝜕(𝜑𝑈,𝜑,𝑣) 𝜕𝑝𝑡 𝑗 =− 𝜕(∑ℎ∈1,𝑘𝑦ℎlog(𝑠ℎ)) 𝜕𝑝𝑡 𝑗 =− ∑ ℎ∈1,𝑘 𝑦ℎ 𝜕log(𝑠ℎ) 𝜕𝑝𝑖 𝑗 =− ∑ ℎ∈1,𝑘 𝑦ℎ 𝜕log(𝑠ℎ) 𝜕𝑧𝑗 𝜕𝑧𝑗 𝜕𝑝𝑖 𝑗 . Since 𝑠ℎ=𝑒𝑧ℎ ∑𝑡∈1,𝑘𝑒𝑧𝑡then 𝜕log(𝑠ℎ) 𝜕𝑧𝑗 = 𝜕log(𝑒𝑧ℎ ∑𝑡∈1,𝑘𝑒𝑧𝑡) 𝜕𝑧𝑗 =𝜕log(𝑒𝑧ℎ) 𝜕𝑧𝑗 − 𝜕log(∑𝑡∈1,𝑘𝑒𝑧𝑡) 𝜕𝑧𝑗 =𝜕𝑧ℎ 𝜕𝑧𝑗 −1 ∑𝑡∈1,𝑘𝑒𝑧𝑡∑ 𝑡∈1,𝑘 𝜕𝑒𝑧𝑡 𝜕𝑧𝑗 =𝛿ℎ𝑗 −𝑒𝑧𝑗 ∑𝑡∈1,𝑘𝑒𝑧𝑡=𝛿ℎ𝑗 −𝑠𝑗. Besides, since 𝑧𝑗=∑ℎ∈1,𝑚𝜉ℎ(𝑣)𝑝ℎ 𝑗then 𝜕𝑧𝑟 𝜕𝑝𝑡 𝑗 =∑ ℎ∈1,𝑚 𝜉ℎ(𝑣)𝜕𝑝ℎ 𝑟 𝜕𝑝𝑡 𝑗 =𝜉𝑡(𝑣). Finally, 𝜕(𝜓,𝜑,𝑣) 𝜕𝑝𝑡 𝑗 =− ∑ ℎ∈1,𝑘 𝑦ℎ(𝛿ℎ𝑗 −𝑠𝑗)𝜉𝑡(𝑣) =−𝜉𝑡(𝑣)(∑ ℎ∈1,𝑘 𝑦ℎ𝛿ℎ𝑗 −𝑠𝑗∑ ℎ∈1,𝑘 𝑦ℎ) =(𝑠𝑗−𝑦𝑗)𝜉𝑡(𝑣).□ Let us now see how we add trainability to the SMNN 𝜑𝑈induced by 𝜑𝑈. Let 𝑉be the training set and let 𝑈be a support set lying in the same space as 𝑉. First, assuming that 𝑈={𝑢1, … , 𝑢𝑚}has 𝑚elements, then 𝜑𝑈is a multiclass perceptron with an input layer with 𝑚neurons that predicts the ℎ-th label for ℎ ∈1, 𝑘using the formula: 𝜑𝑈(𝑥)=softmax( ⋅𝜉𝑈(𝑥)) Information Sciences 667 (2024) 120474 9 E. Paluzo-Hidalgo, R. Gonzalez-Diaz and M.A. Gutiérrez-Naranjo Fig. 5. Two-dimensional synthetic dataset with points divided into two classes: blue and yellow. Triangle-shaped points belong to the test set and square-shaped points belong to the support set 𝑈. The diamond-shaped point is the vertex on the hypersphere (the blue circumference) used to classify the triangle-shaped point 𝑣 (surrounded by a small red circumference) outside the triangulation. where  =(𝑝𝑡 𝑗)𝑗∈1,𝑘,𝑡∈1.𝑚is a matrix of weights and 𝜉𝑈(𝑥) ∈ℝ𝑚is obtained from the barycentric coordinates of 𝑥 ∈𝐵𝑛as in Section 3. Let us observe that softmax ( ⋅𝜉𝑈(𝑥))∈|𝐿|. The idea is to modify the initial values of 𝜑(0) 𝑈(𝑢𝑡)=(𝑝𝑡 1,…,𝑝 𝑡 𝑘)for 𝑢𝑡∈𝑈and 𝑡∈1,𝑚, in order to obtain new values for 𝜑𝑈(𝑣)for 𝑣 ∈𝑉in a way that the error (𝜑𝑈,𝜑, 𝑣)decreases. We will do it by avoiding recomputing Del(𝑈)or the barycentric coordinates (𝑏0(𝑣), … , 𝑏𝑛(𝑣)) for each 𝑣 ∈𝑉during the training process. In this way, given 𝑣 ∈𝑉, if 𝑣 ∈| Del(𝑈)|, we compute the maximal simplex 𝜎=⟨𝑢𝑖0, … , 𝑢𝑖𝑛⟩ ∈Del(𝑈)such that 𝑣 ∈|𝜎|and 𝑖ℎ∈1, 𝑚for ℎ ∈0, 𝑛. If 𝑣 ∉| Del(𝑈)|, we compute 𝑤 ∈𝑆𝑛and the simplex 𝜎=⟨𝑤, 𝑢𝑖1, … , 𝑢𝑖𝑛⟩such that 𝑣 ∈|𝜎|and 𝑖ℎ∈1, 𝑚 for ℎ ∈1, 𝑛. Then we compute the barycentric coordinates 𝑏(𝑣)of 𝑣with respect to 𝜎and the point 𝜉𝑈(𝑥) =(𝜉1(𝑥), … , 𝜉𝑚(𝑥)) ∈ℝ𝑚 as in Section 3. Using the gradient descent algorithm, we update the variables 𝑝𝑡 𝑗for 𝑗∈1, 𝑘and 𝑡 ∈1, 𝑚as follows: 𝑝𝑡 𝑗∶= 𝑝𝑡 𝑗−𝜂 𝜕(𝜑𝑈,𝜑, 𝑣) 𝜕𝑝𝑡 𝑗 =𝑝𝑡 𝑗−𝜂(𝑠𝑗−𝑦𝑗)𝜉𝑡(𝑣). An illustrative picture of the role of each point in a simple two-dimensional binary classification problem is provided in Fig. 5. The pseudocode of the method to train SMNNs using Stochastic Gradient Descent is provided in Algorithm 2. Algorithm 2 Pseudocode of the proposed method to train SMNNs using SGD. Input: Dataset 𝑉⊂ℝ𝑛surrounded by a hypersphere of radius 𝑅and a model 𝜑𝑈. Parameter: 𝜂>0 Output: The trained model 𝜑𝑈 init  , the matrix of weights of 𝜑𝑈 for 𝑣 ∈𝑉do compute 𝜉𝑈(𝑣)as in Section 3 for each column 𝑝of do 𝑝 ∶= 𝑝 −𝜂𝜕(𝜑𝑈,𝜑,𝑣) 𝜕𝑝 end for end for 𝜑𝑈(𝑥) ∶= softmax(  ⋅𝜉𝑈(𝑥)) 5. Explainability In this section, we provide insight into the explainability capability of SMNNs. In the literature, many different approaches can be found to what is an explanation of the prediction of an AI model. In our case, explainability will be provided based on similarities and dissimilarities of the point 𝑥to be explained with the points corresponding to the vertices of the simplex 𝜎containing it. Based on