Unveiling Agents' Confidence in Opinion Dynamics Models via Graph Neural Networks

IEEE TRANSACTIONS ON COMPUTATIONAL SOCIAL SYSTEMS, VOL. 12, NO. 2, APRIL 2025 725 Unveiling Agents’ Confidence in Opinion Dynamics Models via Graph Neural Networks Víctor A. Vargas-Pérez , Jesús Giráldez-Cru , Pablo Mesejo , and Oscar Cordón ,Fellow, IEEE Abstract—Opinion Dynamics models in social networks are a valuable tool to study how opinions evolve within a population. However, these models often rely on agent-level parameters that are difficult to measure in a real population. This is the case of the confidence threshold in opinion dynamics models based on bounded confidence, where agents are only influenced by other agents having a similar opinion (given by this confidence threshold). Consequently, a common practice is to apply a universal threshold to the entire population and calibrate its value to match observed real-world data, despite being an unrealistic assumption. In this work, we propose an alternative approach using graph neural networks to infer agent-level confidence thresholds in the opinion dynamics of the Hegselmann-Krause model of bounded confidence. This eliminates the need for additional simulations when faced with new case studies. To this end, we construct a comprehensive synthetic training dataset that includes different network topologies and configurations of thresholds and opinions. Through multiple training runs utilizing different architectures, we identify GraphSAGE as the most effective solution, achieving a coefficient of determination R2 above 0.7 in test datasets derived from real-world topologies. Remarkably, this performance holds even when the test topologies differ in size from those considered during training. Index Terms—Agent-based modeling, bounded confidence, graph neural networks, Hegselmann–Krause model, opinion dynamics. I. INTRODUCTION AGENT-BASED models (ABM) [1],[2] are a leading technology to study complex systems. ABMs rely on the definition of agents’behavioral microrules [3], and the complex behavior of the system is inferred from the aggregation of Received 27 May 2024; revised 10 October 2024 and 21 November 2024; accepted 26 November 2024. Date of publication 11 December 2024; date of current version 3 April 2025. This work was supported in part by MCIN/AEI/10.13039/501100011033 and ERDF “A way of making Europe” under Grant CONFIA PID2021-122916NB-I00, in part by the FPU Program under Grant FPU20/02441, and in part by Grant RYC2022-036395-I funded by MICIU/AEI/10.13039/501100011033 and ESF+. Funding for open access charge: Universidad de Granada/CBUA. (Corresponding author: Víctor A. Vargas-Pérez.) Víctor A. Vargas-Pérez, Pablo Mesejo, and Oscar Cordón are with the Department of Computer Science and Artificial Intelligence (DECSAI), University of Granada (UGR), 18071 Granada, Spain, and also with Andalusian Research Institute in Data Science and Computational Intelligence (DaSCI), 18071 Granada, Spain (e-mail: [email protected];[email protected].es; [email protected].es). Jesús Giráldez-Cru is with the University of Seville (US), 41044 Seville, Spain, and also with Andalusian Research Institute in Data Science and Computational Intelligence (DaSCI), 18071 Granada, Spain (e-mail: jgiraldez@ us.es). Digital Object Identifier 10.1109/TCSS.2024.3508452 those agents’ behaviors in a bottom–up manner [4].Inmany contexts, this methodology is more accurate than deriving a global definition of the system by top–down approaches. ABMs have been extensively used in many contexts, including modeling emergency evacuations [5], the analysis of political rallies [6], and the study of artificial societies [7], among others. A straightforward application of ABM is opinion dynamics (OD) [8],[9],[10],[11]. OD models aim to study the evolution of opinions in a population [12]. In this scenario, agents’ opinions evolve as a consequence of interactions between them [13], as well as other endogenous and exogenous factors, such as stubbornness [14] or mass communication [15], for instance. OD models are commonly used to study whether the opinion fusion rule (i.e., the agents’ micro-rule that defines how opinions are updated after agents’ interactions) is able to reach a consensus, a polarization, or a fragmentation of opinions [16]. Consensus is the state where all opinions have the same value, while polarization and fragmentation are cases where several clusters of opinions co-exist: two clusters in the former case, and more than two clusters in the latter [13]. A very well-studied OD model is Bounded Confidence (BC) [17],[18],[19]. The main principle of the BC model is that agents are only influenced by other agents having a similar opinion to them, where a confidence threshold determines the area of social influence for each agent. The rationale behind this principle is social confirmation bias [20], i.e., the interactions that influence individuals’ own opinions are mostly those occurring with other individuals similar to them. In this work, we focus on the Hegselmann–Krause (HK) model [18], where an agent is randomly selected at each time step, and updates its opinion by averaging the agents’ opinions in its confidence area (i.e., the neighbors of its social network whose opinions are within the confidence threshold). It is well known that, for a given threshold, a low value produces opinion consensus, whereas a high value results in a fragmentation of opinions [21],[22]. However, it remains unclear how to determine the confidence threshold for a given population, and this is the problem addressed in this work. The task of determining confidence thresholds that yield specific final opinions can be framed as a model parameter calibration problem. Heuristic search methods such as evolutionary algorithms (EAs) can be used to address this problem [23],[24], but require separate calibration for each new case study. In this work, we aim to develop a method that generalizes the knowledge from training. This will allow us to effectively determine, in negligible time, the confidence thresholds for © 2024 The Authors. This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ 726 IEEE TRANSACTIONS ON COMPUTATIONAL SOCIAL SYSTEMS, VOL. 12, NO. 2, APRIL 2025 case studies that differ from those observed during the training phase, even with varying population sizes. It is well established the interplay between machine learning (ML) techniques and ABMs, and how the former can help to improve the performance of the latter [25],[26], including learning micro-agent level situational awareness, micro-agent level behavior intervention, macro-ABM level emulation, and sequential decision-making, among other tasks. Among the diversity of existing ML techniques, deep learning (DL) algorithms stand out nowadays because of the massive amount of available data [27]. In this context, graph neural networks (GNN) are DL models especially designed to deal with non-Euclidean data that can be represented in the form of graphs [28]. They are able to perform most ML tasks, including classification and regression tasks at the level of nodes, edges, and the whole graph [29]. Notice that ABM in general, and OD models in particular, are good examples of such graph-structured data, and the aforementioned open question of determining the confidence threshold of a given population can be seen as a suitable learning task for GNNs. The main contribution of this work is a thorough analysis of predicting the confidence threshold of BC models by means of GNNs. To the best of our knowledge, this is the first work studying OD models by means of GNNs. In particular, we analyze the performance of several GNN layers extensively used in the literature, including graph convolutional network (GCN) [30], GraphSAGE [31], and graphattention network (GAT) [32], in order to predict the confidence thresholds of a population in several synthetic social network topologies. These trained models are further validated in real-world social networks. Our experimental evaluation shows that GraphSAGE and GAT are able to obtain remarkably accurate results on this problem, exhibiting notable generalization capabilities to unseen social networks. Specifically, GraphSAGE is able to achieve consistent results even when the agent population has heterogeneous confidence thresholds. This means that the challenging question of determining the confidence threshold of BC OD models can be effectively addressed by GNNs. The rest of this work is organized as follows. Section II presents the preliminary background on OD models and GNNs. The methodology of our proposal is presented in Section III, while Section IV is devoted to the empirical validation of it. Finally, we conclude and discuss potential direction of future research in Section V. II. BACKGROUND This section describes some preliminaries on OD models and GNN frameworks, required to properly understand the developments made in the current contribution. A. Opinion Dynamics Models OD models are usually classified into discrete and continuous models, according to the representation of opinions into discrete (usually binary) and continuous values, respectively [13]. Some examples of discrete OD models are the voter model [9] and the majority rule [10], among others. Unfortunately, discrete models are usually unable to represent the complex nature of opinions in most contexts. In contrast, continuous OD models generally provide a more realistic representation of opinions. Therefore, in the rest of this work we focus on continuous OD. Let oi(t)be the opinion of agent i∈{1,...,N}at time step t. For simplicity, we assume oi(t)∈[0,1], although any other real interval could be used instead. One of first continuous OD model proposed in the literature is the DeGroot [8], where opinions are updated as per the following rule: oi(t+1)= N  j=1 wijoj(t)(1) where wij is the weight agent igives to agent j. In this simple linear model, the sufficient and necessary conditions to reach a consensus are well-known [33]. A first step to introduce nonlinearity in continuous OD model is BC, including the Deffuant–Weisbuch (DW) [17] and the HK [18] models. Although both models rely on only considering opinions within a bounded confidence area (given by a confidence threshold), the DW model only considers random pairwise encounters while the HK model incorporates all the opinions in the neighborhood of an agent. Therefore, the HK model is more suitable for modeling interactions in large groups, such as formal meetings, whereas the DW is better suited for pairwise interactions within a large population [21]. For this reason, we focus on the HK model to study the confidence threshold of a population by means of GNNs. Let us consider a population of Nagents interacting in a social network, which is represented by a graph G(V,A), being Vthe set of nodes (with |V|=N), and Aits N×Nadjacency matrix (i.e., Aij =1 iff. there is an edge between nodes iand jin G,Aij =0 otherwise).1Each agent is represented by a distinct node iin the graph, and interacts with other agents within its neighborhood N(i), i.e., with some agent jin the set N(i)={j∈V|Aij =1}. In the HK model, at each time step t={1,... T}2, an agent i∈{1,...,N}is randomly selected and updates its own opinion oi(t)as oi(t+1)=j∈C(i)oj(t) |C(i)|(2) where C(i)={j∈N(i)∪{i}||oi(t)−oj(t)|<ε i}is the set of node i’s neighbors in its confidence area, with εibeing the confidence threshold of agent i. Notice that this confidence threshold εiis agent-specific, i.e., it may differ from one agent to another. The values of the confidence thresholds are responsible for the distribution of final opinions in an execution of the model, resulting in consensus, polarization, or fragmentation of opinions depending on such a value. Fig. 1depicts an example of this phenomenon, executing the HK model with a uniform confidence threshold (εi=ε) in a fully mixed population of N=1000 agents during T=105time steps, considering three distinct confidence thresholds: (left) ε=0.1, (center) ε=0.2, 1For completeness, we assume Aii =0 for every node i. 2Tis the total number of time steps of a simulation. VARGAS-PÉREZ et al.: UNVEILING AGENTS’ CONFIDENCE IN OPINION DYNAMICS MODELS VIA GNNS727 Fig. 1. Example of the execution of the HK model with several values of the confidence threshold ε. For each execution, the left subplots represent the evolution of opinions during the executions and the right subplots represent the distribution of initial and final opinions. and (right) ε=0.3. It can be observed that the execution with the lowest confidence threshold ε=0.1 results in a profile of final opinions with four clusters of different opinions. With an intermediate value of this confidence threshold, as in the case with ε=0.2, the final opinions are just polarized into two clusters of distinct opinions. Finally, with the highest value of ε, as in the execution with ε=0.3, the population reaches a consensus of opinions. B. Graph Neural Networks GNNs are a family of neural networks (NN) that operate on graph data. The key idea behind them is to generate node embeddings, which are vector representations of the nodes that encode both their individual properties (i.e., node features) and the structural information of the graph [34]. The most standardized framework for designing GNNs is neural message passing, wherein nodes receive information from their neighbors (i.e., messages), aggregate it, and update their internal representations using a NN [35]. This process unfolds over a fixed number of iterations or layers L.Leth(l) i denote the representation of node iat layer l, and xi=h(0) i denote its initial feature vector. The message-passing update at layer l∈{1,... L}can be expressed as shown in (3) h(l+1) i=UPT(l)h(l) i,AGG(l)({m(l) ij ∀j∈N(i)}) m(l) ij =MSG(l)h(l) i,h(l) j(3) where MSG(l),AGG (l), and UPT(l)are differentiable functions that can be implemented with NNs. A foundational example within this framework is GCN [30], which introduced effective strategies for neighborhood aggregation and parameter sharing, forming the basis for many advanced architectures. GCN is defined in (4) as follows: h(l+1) i=σ⎛ ⎝W(l) j∈N(i)∪{i} h(l) j |N(i)||N(j)|⎞ ⎠(4) where σis a nonlinear activation function and Wis a weight matrix.3The aggregation includes symmetric normalization 3For simplicity, the corresponding bias term is omitted in the definition of all GNN layers. and self-loops, stabilizing training and incorporating the center node’s features [34]. However, this formulation processes neighborhood and self-information uniformly, limiting its expressiveness. A more advanced layer is GraphSAGE [31], which allows different ways of aggregating the neighborhood and explicitly defines the update phase. Equation (5) shows this layer with a mean aggregator. By distinguishing between the node and its neighbors, GraphSAGE is capable of learning richer and more flexible representations than GCN, albeit at the cost of requiring a higher number of parameters h(l+1) i=σW(l) 1h(l) i+W(l) 2j∈N(i)h(l) j |N(i)|.(5) In addition to distinguishing between the node and its neighborhood, it may be beneficial to create a distinction among the neighbors themselves, assigning different levels of importance to each of them. This is precisely what GAT [32] accomplishes: it calculates a weighted mean of the neighborhood messages by incorporating an attention mechanism, as depicted in (6)–(8) h(l+1) i=σ⎛ ⎝α(l) ii W(l)h(l) i+ j∈N(i) α(l) ij W(l)h(l) j⎞ ⎠(6) α(l) ij =exp(e(h(l) i,h(l) j)) k∈N(i)∪{i}exp(e(h(l) i,h(l) k)) (7) e(h(l) i,h(l) j)=LeakyReLU a(l)T 1W(l)h(l) i+a(l)T 2W(l)h(l) j (8) where a1and a2are learnable weight vectors, erepresents the importance of node jto node i, and αij is the corresponding attention coefficient after applying softmax normalization over the neighborhood N(i). Although GAT is one of the most popular GNN architectures nowadays, it presents a limitation: its attention mechanism is static, meaning that the resulting ranking of attention coefficients for every neighbor node is unconditioned by the node of origin. This issue was addressed in GATv2 [36], which achieves a dynamic attention mechanism with a small adjustment involving a reordering of internal operations within GAT and an increase in the number of parameters. Specifically, the change entails duplicating parameters in Wby splitting it into two 728 IEEE TRANSACTIONS ON COMPUTATIONAL SOCIAL SYSTEMS, VOL. 12, NO. 2, APRIL 2025 TABLE I NUMBER OF PARAMETERS (INCLUDING BIAS TERMS)AND ASYMPTOTIC TIME COMPLEXITY OF THE DISCUSSED NN LAYERS Layer Type # Parameters Time Complexity MLP Hi·Ho+HoO(N·Hi·Ho) GCN Hi·Ho+HoO(N·Hi·Ho+E·Ho) GraphSAGE 2·Hi·Ho+HoO(N·Hi·Ho+E·Ho) GATv2 2·Hi·Ho+4·HoO(N·Hi·Ho+E·Ho) Note: Let Nbe the number of nodes of the graph, Ebe the number of edges, Hibe the input node embedding size, and Hobe the output node embedding size. different matrices, W1and W2. In turn, a1and a2are condensed into a single smaller weight vector a, which is applied after the LeakyReLU activation function. Thus, (8) is now rewritten as shown in (9) e(h(l) i,h(l) j)=a(l)TLeakyReLU W(l) 1h(l) i+W(l) 2h(l) j. (9) The increasing expressive power of these three layers also implies an increasing number of parameters, as shown in Table I. For illustration, this table also provides the parameter count of a perceptron layer that only considers feature information, not structure. Additionally, we include the theoretical upper bound for the time complexity of a forward pass. Although the asymptotic time complexity is the same for these GNN layers, differences in constant factors and unit operation costs (e.g., square roots in GCN or exponentials in GATv2) can significantly impact their actual runtime. Regardless the layer type considered, GNNs are useful for addressing various ML tasks, such as node-level classification (or equivalently, regression), link prediction, or graph classification [37]. When working at the node level, a distinction must be made between transductive and inductive problems, which are commonly associated with semi-supervised and supervised learning, respectively [34]. In the transductive case, the training and test node sets are connected within the same graph. This implies that test nodes participate in the training process to perform message passing operations, though their labels are neither observed nor used to compute the loss function. Alternatively, the inductive case resembles traditional supervised learning: test nodes are completely separated from the training nodes, residing in disjoint graphs, and thus do not intervene in any way in the training process. As discussed in Section III, our methodology involves solving an inductive node-level regression problem. III. METHODOLOGY Our methodology aims to determine the individual confidence threshold εiof each agent iwithin a population whose interactions are governed by the HK model. To accomplish this task, we propose training a ML model to learn the function that maps the initial and final opinions of each agent, along with the social network connecting them, to the corresponding confidence threshold for each agent. Given the inherent graph structure of this input data, a GNN emerges as the most natural choice for the ML model. A standard approach that closely resembles our objective is ABM calibration. In this methodology, the confidence thresholds are treated as free parameters of the OD model, and the goal is to find a configuration of values that yields specific simulation outputs, namely, the desired set of final opinions. When the dimensionality of the parameters is high, it is appropriate to employ an automated calibration process using heuristic search methods. These methods aim to minimize an objective function that quantifies the distance between the resulting simulation output and the target output [23],[24]. EAs are among the most commonly used optimization techniques for this purpose [38],[39],[40], due to its capability to explore a wide range of parameter values and consider the non-linear interactions between them. However, we must highlight that the objective of this work is not to calibrate the parameters of an ABM to a specific case study. Instead, we aim to develop a method that, after a single training process, is capable of inferring the agent parameters for any new BC case study, without additional ABM simulation runs. To the best of our knowledge, this is the first study that seeks to generalize OD ABM parameter fitting across scenarios that were not observed during the optimization phase and with varying population sizes. The motivation behind this novel approach lies in two key advantages with respect to traditional ABM calibration: 1) A standard calibration approach is specific to each instance of the problem or case study, requiring a comprehensive set of simulations for each new scenario, without direct use of prior knowledge. In contrast, our approach benefits from learning generalization. In a new case study, the trained GNN requires only one inference step to derive the thresholds associated with each agent, without the need for retraining or conducting additional simulation runs of the ABM, which tend to be computationally expensive. 2) It is common in OD to establish a uniform confidence threshold for all agents in the population [15], or to alternatively segment it. This practice aims to prevent a substantial increase in the number of parameters to calibrate, which would be equal to the population size if a fully personalized threshold were adopted. Alternatively, our approach facilitates addressing a more realistic scenario where each individual owns a distinct confidence threshold, without entailing an increase in the number of parameters of the GNN. Indeed, our approach is potentially capable of managing scenarios with varying population sizes, as the same GNN model can be applied to graphs or cases of different sizes. Despite the potential advantages described in our proposal, it is crucial for the GNN training process to be sufficiently robust and generalizable for effective application in real-world scenarios not previously observed. Particularly, to ensure the GNN can generalize across different topologies, the training set must encompass diverse characteristics across different topologies. Fortunately, since the HK model is fully defined (see 2), VARGAS-PÉREZ et al.: UNVEILING AGENTS’ CONFIDENCE IN OPINION DYNAMICS MODELS VIA GNNS729 Fig. 2. Overview of the methodology for training a GNN to predict the confidence threshold of agents in a population governed by the HK OD model. The process involves generating a set of initial topologies A(Step 1), creating a set of Mrandom configurations of initial opinions and thresholds for the agents (Step 2), running the HK model on each graph to obtain the final opinions (Step 3), and constructing the complete dataset G, and splitting it into training (G(Train)), validation (G(Val)), and test (G(Test)) subsets (Step 4). The GNN receives as input the initial and final opinions of each agent (X) along with the social network connecting them (A), and returns as output the predicted confidence threshold of each agent (ˆy). The colors in the nodes from Step 2 indicate the different configurations of initial and final opinions, as well as confidence threshold values. we can generate a synthetic training dataset representing a wide array of problem variants. This includes diverse social network topologies and a multitude of settings of confidence thresholds and initial opinions within the range [0,1]. Moreover, knowledge about the problem allows us to tailor the data generation process to encompass various scenarios while avoiding the over-representation of equivalent scenarios. Specifically, prior research [18] suggests that, despite the theoretical domain of ε∈[0,1], in practical scenarios where ε≥0.25, the population tends to reach a consensus opinion, resulting in similar final outcomes (see Fig. 1). Therefore, it is appropriate to limit the generation of synthetic data with values of εiwithin a narrower domain, such as [0,0.5].4 Fig. 2shows a summary of the steps for training the GNN model, which are defined as follows: 1) Selection and generation of a set of topologies or adjacency matrices Arepresenting different social networks of agents. It is essential to consider a range of topologies based on different network properties such as density, clustering coefficient, and degree distribution, among others, to provide the learning process with enough diversity. 2) Generation of Mdifferent random configurations of initial opinions and thresholds for the agents for each social network topology considered. We denote the resulting set of graphs as G, and for each topology A∈A, we define the subset G(A)={G∈G|G(V,A)}, where 4The upper limit of 0.25 is an indicative empirical value, justifying our choice of a higher upper bound. In fact, these results are made considering a fully connected social network, without considering the impact of the social network topology. |G(A)|=M,∀A∈A. This ensures a comprehensive representation of diverse scenarios across all considered topologies. 3) Running the HK model on each graph G∈G, thereby obtaining the final opinion of each agent.5 This process allows us to construct the dataset associated with each graph, denoted as D(G(V,A)) = ((X,A),y). Here, X∈[0,1]N×2comprises the feature vector of each node i, defined as X={xi=(oi(1), oi(T))|∀i∈{1,...,N}}. Meanwhile, the label vector y∈[0,0.5]Ncontains the confidence threshold of each node, i.e., y={yi=εi|∀i∈{1,...,N}}. The number of time-steps Tof the simulations should be high enough to reach a stationary state, i.e., a state beyond which the agents no longer change their opinions. 4) Training a GNN following a supervised approach to address the regression problem associated with the previous datasets. Formally, the GNN aims to approximate the function f:(X,A)→y. Note that, while the dimensions of X,A, and ymust remain consistent within a specific graph (same number of nodes N), the number of parameters in the GNN model remains constant and independent of the graph size. Thus, we can combine graphs of varying sizes in both training and inference stages. The set of |M|×|A|graphs or case studies is randomly divided into training G(Train), validation 5The HK model has a stochastic component, wherein an agent is randomly chosen at each time-step. Thus, the final opinion oi(T)recorded in the dataset for each agent is actually its average final opinion after 10 Monte Carlo simulation runs. 730 IEEE TRANSACTIONS ON COMPUTATIONAL SOCIAL SYSTEMS, VOL. 12, NO. 2, APRIL 2025 TABLE II CHARACTERISTICS OF THE SYNTHETIC BASE TOPOLOGIES AALONG WITH THE COMMUNITY DETECTION RESULTS Topology Nodes Edges Density Average Degree Diameter Average Clustering Coefficient #Communities Modularity ER(0.1) 1000 49925 9.99e-02 9.99e+01 3 9.95e-02 10 8.04e-02 ER(0.2) 1000 100129 2.00e-01 2.00e+02 2 2.01e-01 8 5.41e-02 ER(0.3) 1000 149559 2.99e-01 2.99e+02 2 2.99e-01 7 3.98e-02 SW(0.3, 3) 1000 1311 2.62e-03 2.62e+00 20 0.00e+00 27 7.36e-01 SW(0.3, 5) 1000 2646 5.30e-03 5.29e+00 9 3.05e-01 26 7.01e-01 SW(0.3, 7) 1000 3882 7.77e-03 7.76e+00 7 3.71e-01 22 7.08e-01 BA(2) 1000 1996 4.00e-03 3.99e+00 7 3.04e-02 19 5.22e-01 BA(4) 1000 3984 7.98e-03 7.97e+00 5 3.53e-02 16 3.11e-01 BA(6) 1000 5964 1.19e-02 1.19e+01 4 4.42e-02 9 2.58e-01 G(Val), and test G(Test)sets. The GNN is trained using the training graphs and validated using the validation ones. Subsequently, the final performance and generalization capacity of the trained GNN are evaluated on the test graphs, which remained completely unobserved during the training process. In addition to decisions regarding synthetic data generation and training process, it is crucial to define the specific architecture of the GNN. This architecture is contingent upon the type of GNN layer chosen, which should be flexible enough to capture the complexity of the problem at hand. Lastly, and not least, it is imperative to verify the underlying hypothesis of our proposal: that determining the confidence thresholds for a population of agents based on their initial and final opinion in an OD diffusion process is nontrivial, and that the structural information of the graph plays a crucial role. These aspects are further explored in the following Section IV. IV. EXPERIMENTS AND ANALYSIS This section presents the experiments conducted to evaluate our methodological proposal. First, we describe the experimental setup and the generation of synthetic datasets in Section IV-A. Next, we detail the considered model architectures, training details, and hyperparameter tuning in Section IV-B.We then proceed to evaluate the best models on the synthetic test dataset GSyn in Section IV-C. In Section IV-D, we perform a final evaluation on test datasets generated from common realworld topologies in the literature, whose union we denote as GReal. Finally, in Section IV-E, we develop an alternative EA method to determine the agents’ thresholds using a more standard heuristic search approach, and compare its results with our GNN-based proposal. A. Experimental Setup and Synthetic Datasets All experiments related to NN training are conducted on a server with 2 ×Intel Xeon CPU E5-2698, 512 GB DDR4, and 8×Tesla V100 32 Gb GPUs. On the other hand, since the experiments related to an EA run (see Section IV-E) do not take advantage of the GPU (they are CPU-oriented processes), we use a server with 2 ×2.2 GHz Intel Xeon Silver 4214 (12 cores), 256 GB DDR4. This architecture allows us to parallelize the simulation runs corresponding to the evaluation of each EA solution, thus reducing computation time. Additionally, to ensure result reproducibility, we have created a public repository containing all the code related to dataset generation and experimentation.6We use the Pytorch Geometric library [41] to implement the GNN models and NetworkX [42] to generate the synthetic topologies. The first step to create the synthetic dataset GSyn is to generate the set of base topologies A. To achieve this, we consider three classic algorithms for generating artificial social networks, each with distinct characteristics. The first is the Ërdos–Rényi (ER) algorithm to create random networks [43], where each pair of nodes is connected with a uniform probability p. As a result, these graphs exhibit a binomial degree distribution. The second algorithm is the Newman–Watts–Strogatz, which generates small-world networks (SW) [44]. This algorithm starts with a regular ring lattice where each node has kneighbors, and then creates shortcuts by randomly adding new connections with a probability p. Thus, the resulting graph has both small paths (similar to a random network) and a high clustering coefficient. Finally, the third algorithm considered is the Barabási–Albert (BA) preferential attachment [45]. This method begins with a fully connected graph and iteratively adds new nodes with m connections to previous ones, such that each node receives a new connection with a probability proportional to its degree. This results in a graph with a power-law degree distribution. In other words, most nodes have a low degree, while a few hubs receive most of the connections. These graphs are also known as scale-free because the distribution is independent of both the number of nodes and the average degree, and many real networks follow this distribution [45]. For each network generation algorithm, we apply three different configurations, resulting in a total of |A|=9 base topologies. In the case of ER, we consider three values of the link probability p. For SW, we set p=0.3 and vary the value of the initial degree k. Lastly, for BA we use different values for the number of links madded per iteration. In all cases, we set N=1000, as we consider this to be a sufficient number of nodes to create a realistic scenario while not dramatically increasing the dataset size. Table II shows the nine configurations and the properties of the resulting networks. This table also includes the results 6https://github.com/vvarper/GNN4BCPrediction VARGAS-PÉREZ et al.: UNVEILING AGENTS’ CONFIDENCE IN OPINION DYNAMICS MODELS VIA GNNS731 of applying the Louvain community detection algorithm [46] to each network, which will be relevant in the next step of the dataset generation. The second step in our methodology is the creation of the M configurations of initial and final opinions, as well as confidence thresholds. To cover a wide variety of scenarios, we consider M=20 configurations. In all of them, the initial opinion of each agent is randomly generated according to a uniform distribution [0,1]. The difference between each of the Mconfigurations lies in the values of the confidence thresholds. To address problem scenarios with incremental difficulty, we consider two cases: 1) Graphs with homogeneous thresholds GHom Syn . In this scenario, we generate Mevenly spaced thresholds in the range [0.1, 0.5] and assign each of them to all nodes of the corresponding graph configuration. 2) Graphs with heterogeneous thresholds by communities GCom Syn . The idea behind this scenario is that an individual has similar characteristics to those close to them, differing from the rest. For every topology, we consider the community partition from Table II and assign a random confidence threshold to each community drawn from a uniform distribution [0,0.5]. The generation of thresholds for each community is repeated Mtimes per topology to obtain the desired number of configurations. The community partitions and their corresponding modularity values are presented in Table II. The final opinions of each graph are obtained by averaging the results of 10 MC simulations of the HK model. Every simulation runs for T=106time-steps, ensuring that a stationary state is reached in all cases. At this point, both GHom Syn and GCom Syn have a total of |A|×M=9×20 =180 graphs each.7 We randomly split each dataset, assigning 64% of the graphs for training (i.e., 115 graphs), 16% for validation (29 graphs), and 20% for testing (36 graphs). B. Models’ Architecture and Hyperparameter Tuning The three GNN architectures previously discussed in Section II-B are considered: GCN, GraphSAGE, and GATv2. Additionally, with the aim of having a base benchmark and verifying that structural information is crucial in our problem, we also consider a multilayer perceptron (MLP) that treats each graph as a dataset in which each node is an independent sample. MLP is a structure-agnostic architecture that aims to predict the confidence thresholds solely based on feature information. We define each model as a concatenation of Llayers selected from the four types defined earlier, such that |h(0) i|= 2, |h(L) i|=1, and |h(j) i|=Hfor j∈{1,...,L−1}and i∈ {1,...,N}. All layers are followed by a ReLU activation function, except the last one, which uses a sigmoid activation function to ensure that the predicted threshold ˆεiis in the range [0,0.5]. Architectures such as convolutional NNs usually 7Note that among these graphs, there are repeated topologies but with different node features. In this work, we distinguish between topology (the structure of the graph) and the graph as a whole, which includes the node information. TABLE III HYPERPARAMETERS TO BE TUNED AND THEIR RESPECTIVE CANDIDATE VALUES Hyperparameter Description Candidate Values L Number of layers of the NN. {4, 5} HNode embedding dimension in hidden layers. {16, 32} Learning Rate Step size on each SGD update. {1.00e-02, 1.00e-03, 1.00e-04} Batch Size Number of graphs used on each SGD update. {2, 4, 8} benefit from lager number of layers, as they facilitate hierarchical feature extraction. However, in the case of GNNs, the number of layers is typically more limited due to the oversmoothing problem [47]. In order to find the best possible model, our experimentation involves hyperparameter tuning for each layer type and dataset. Specifically, we use a grid search, which is one of the most common techniques in the literature for this task [48].Itoffers a comprehensive and straightforward approach to explore the parameter space, ensuring that no region of the predefined spaces is overlooked [49]. We adjust the number of layers L, the dimension of the hidden layers H, the learning rate, and the batch size. Table III shows the candidate values considered for each hyperparameter, resulting in a total of 36 possible settings, and thus, a total of 288 training runs. In all these training processes, we use the Adam optimizer with the mean squared error (MSE) loss function with a maximum of 10,000 epochs. Furthermore, to prevent over-fitting to training data, we apply early stopping if the validation loss does not improve for 1000 consecutive epochs. The results of the hyperparameter tuning are shown in Table IV, where we indicate the best configuration found according to validation loss for each layer type and dataset, along with the corresponding MSE on the training and validation sets. We can observe that the settings found vary in each case, although in the case with homogeneous thresholds, the architecture with L=5 layers and a hidden layer size of H=32 is consistently chosen as the best one. Regarding the quality of the models, we observe that in both problem scenarios, and in both training and validation datasets, the lowest MSE found corresponds to GraphSAGE, followed by GATv2. The worst results are obtained with MLP and GCN, with the ranking varying among them depending on the threshold scenario. Another issue to note is that, in the case with homogeneous thresholds, the MSE of the GraphSAGE and GATv2 models is an order of magnitude lower than that of MLP and GCN, a trend also observed in the community scenario but only with GraphSAGE. Furthermore, although the MSE in training is generally lower than in validation, the difference is not wide, with the magnitude order and the ranking of the models remaining consistent. Therefore, there do not seem to be over-fitting problems in either case. Table IV also includes the number of parameters of each configuration (corresponding to the counts provided in Table I) and the training times. A direct comparison based on the layer type may be misleading, as both values depend on additional factors: while the number of parameters is affected by the 732 IEEE TRANSACTIONS ON COMPUTATIONAL SOCIAL SYSTEMS, VOL. 12, NO. 2, APRIL 2025 TABLE IV BEST HYPERPARAMETER CONFIGURATION FOUND PER LAYER TYPE AND PROBLEM ACCORDING TO THE MSE IN THE VALIDATION SET,INCLUDING THE NUMBER OF PARAMETERS,THE NUMBER OF EPOCHS UNTIL EARLY-STOPPING,AND THE TOTAL TRAINING TIME Dataset Type Layer Learning Rate L H Batch Size # Parameters Epochs Training Time (s) MSE (Train) MSE (Val) Homogeneous Thresholds GHom Syn MLP 1.00e-03 5 32 8 3297 4557 7.28e+02 7.33e-03 6.49e-03 GCN 1.00e-03 5 32 2 3297 10000 8.92e+03 2.78e-03 3.06e-03 GraphSAGE 1.00e-03 5 32 4 6465 6361 2.28e+03 6.32e-04 4.42e-04 GATv2 1.00e-04 5 32 2 6852 10 000 1.29e+04 7.44e-04 8.15e-04 Thresholds by Community GCom Syn MLP 1.00e-02 4 16 4 609 1911 5.17e+02 1.38e-02 1.36e-02 GCN 1.00e-03 4 32 8 2241 7754 1.90e+03 1.53e-02 1.68e-02 GraphSAGE 1.00e-03 5 32 4 6465 2953 1.04e+03 5.22e-03 6.49e-03 GATv2 1.00e-02 4 32 8 4676 3233 1.47e+03 9.55e-03 1.05e-02 Note: The lowest MSE values are highlighted in bold. TABLE V EVALUATION OF THE BEST MODEL FOR EACH LAYER TYPE IN THE SYNTHETIC TEST DATASETS ACCORDING TO MSE, MAE, MAPE, AND R2METRICS Layer Homogeneous Thresholds GHom Syn Thresholds by Community GCom Syn MSE MAE MAPE R2MSE MAE MAPE R2 MLP 6.09e-03 6.17e-02 2.33e-01 5.72e-01 1.60e-02 1.07e-01 1.55e+00 3.08e-01 GCN 3.71e-03 4.62e-02 1.76e-01 7.40e-01 1.97e-02 1.20e-01 2.14e+00 1.50e-01 GraphSAGE 1.69e-03 2.53e-02 9.84e-02 8.81e-01 6.76e-03 6.33e-02 6.07e-01 7.08e-01 GATv2 2.89e-03 3.30e-02 1.44e-01 7.97e-01 1.25e-02 9.08e-02 1.07e+00 4.59e-01 Note: The lowest of MSE, MAE, MAPE and the highest R2values are in bold. Fig. 3. Inference results of best models on the synthetic test dataset with homogeneous thresholds GHom Syn . The x-axis represents the real threshold εiof each node i, while the y-axis represents the predicted threshold ˆεi. number of layers and node embedding dimension, the training time is influenced by the batch size and the number of epochs before early-stopping. Notably, focusing on the most promising layers, the training time for GraphSAGE ranged from 17 to 38 min, whereas GATv2 varied significantly, taking between 24 min (in the community scenario) and 4 h (in the homogeneous scenario). The latter extended training time is due to reaching the maximum number of epochs allowed. C. Evaluation on Synthetic Topologies We continue to delve into the performance of the best models found in the previous section, evaluating them on the synthetic test datasets. The results are shown in Table V, where, in addition to MSE, mean average error (MAE), mean absolute percentage error (MAPE), and the coefficient of determination R2are included. Note that while MSE, MAE, and MAPE are error metrics to minimize, R2is a goodness-of-fit measure where R2=1 indicates a perfect fit and R2=0 indicates that the model is not better than predicting the mean of the target variable. The results remain consistent with those observed previously in Table IV, although in the case of homogeneous thresholds, GraphSAGE and GATv2 show an MSE of one order of magnitude higher than in training and validation datasets. Nevertheless, we should note that R2is above 0.79 in both cases. As could be inferred from the MSE, the GATv2 model exhibits notably lower performance in GCom Syn (R2=4.59e-01), while GraphSAGE maintains remarkable performance (R2= 7.08e-01). The underlying hypothesis of our methodology was that structural information plays a crucial role in this problem, and therefore MLP should serve as a baseline model to measure whether the performance of a GNN model is indeed positive. The results seen so far seem to confirm this, with the only case of discrepancy being GCN, the most basic GNN architecture. To discuss this issue further, we visualize in Fig. 3the predictions of each model against the labels of each node in GHom Syn ,the easiest variant of the problem that also presents fully balanced VARGAS-PÉREZ et al.: UNVEILING AGENTS’ CONFIDENCE IN OPINION DYNAMICS MODELS VIA GNNS733 TABLE VI NEMENYI POSTHOC TEST RESULTS (P-VALUES)ON THE 36 SYNTHETIC TEST GRAPHS Dataset type Layer GCN GraphSAGE GATv2 Homogeneous Thresholds GHom Syn MLP 7.73e-01 1.00e-03 1.00e-03 GCN 1.00e-03 5.60e-03 GraphSAGE 3.55e-01 Thresholds by Community GCom Syn MLP 8.24e-01 1.00e-03 1.00e-03 GCN 1.00e-03 1.00e-03 GraphSAGE 4.08e-03 Note: P-values lower than 1.00e-02 are highlighted in bold. and evenly spaced thresholds values. The visualization for a perfect model should show a diagonal line. MLP is unable to adequately capture the relationship between node features and their thresholds, as it makes noisy predictions around the mean. In this scenario, the visualization for the three GNNs shows a gradual increasing trend in predictions according to the real thresholds, which is particularly noticeable in the GraphSAGE case. Therefore, we can conclude that indeed, the error values for MLP serve as a reference point that must be widely surpassed to claim that a GNN model is effective. Finally, we conduct statistical ranking tests [50] by evaluating each model on the 36 test graphs from each scenario separately. First, the Friedman test is applied to determine if there are significant differences between the models’ performance. The result of this test is χ2 F=4.52e+01 (p-value =8.39e-10) for GHom Syn and χ2 F=9.14e+01 (p-value =1.08e-19) for GCom Syn . Thus, setting a confidence level of 99%, we can affirm that there are significant differences in both scenarios. Subsequently, it makes sense to examine whether there are significant differences between each pair of models with the Nemenyi post-hoc test, whose results are shown in Table VI. We can see that in the scenario with homogeneous thresholds, GraphSAGE and GATv2 are significantly better than MLP and GCN, while in the community scenario, GraphSAGE is also significantly superior to GATv2. However, it cannot be stated that there are significant differences between GCN and MLP in either case. The fact that GCN does not outperform MLP may seem surprising, but there are already previous studies indicating that GCN is a very basic model with limited expressive capacity. In fact, there are cases where it does not provide advantages over structure-agnostic models such as MLP [51]. The intuition behind the insufficient power of GCN can be derived from its mathematical definition. In (4), we observe that GCN uses a singles weight matrix and performs aggregation without distinguishing between the center node and its neighbors. This is a key difference from GraphSAGE (5), which also applies to GATv2: in these layers, different weight matrices are used for the target node and its neighbors, applying different linear transformations to their features. Given the results, this distinction seems to be crucial when addressing this threshold prediction problem. D. Evaluation on Real Topologies We conclude the evaluation of the models with additional test datasets generated from real-world topologies commonly used in the literature [52]. The characteristics of these topologies are shown in Table VII, and it can be seen that they differ from the synthetic topologies (Table II) even in their larger size, ranging from networks of 2000 to 20000 nodes. For each of these topologies, we create a dataset with homogeneous thresholds and another one with thresholds by communities, following the same procedure as in Section IV-A. Again, we consider M=20 configurations per case. The only difference is that this time the topologies are kept in separate datasets and that these are not split into training, validation, and test subsets, but are used directly to evaluate the models previously trained on synthetic topologies. Table VIII shows the results on these real-topology test datasets, which follow a similar trend to those observed in Table V. GraphSAGE and GATv2 remain the models with the best performance, although this time GATv2 outperforms GraphSAGE in the scenarios with homogeneous thresholds. On the other hand, GCN shows even worse results than in the synthetic datasets, being surpassed by MLP in all cases. Once again, we conduct statistical tests to determine if there are significant differences. This time, we combine the datasets of the five real-world topologies into a single test set per case (GHom Real and GCom Real ), resulting in a total of 100 graphs per scenario. The Friedman test result for GHom Real is χ2 F=2.15e+02 (p-value = 2.89e-46), while for GCom Real is χ2 F=2.76e+02 (p-value =1.31e59). The results of the Nemenyi post-hoc test are shown in Table IX, and they indicate that the differences between the models are even more significant than in the synthetic datasets. The main difference is that now GCN is significantly worse than MLP in the scenario with homogeneous thresholds, while the rest of the comparisons remain consistent with the previous datasets (see Table VI). Therefore, we can confidently assert that GraphSAGE and GATv2 are the most suitable models for our task. The significant outperformance of GraphSAGE over GATv2 in one of the scenarios, despite its simpler architecture, may seem surprising. Our intuition is that GraphSAGE already has sufficient expressiveness to capture the relevant information for our problem, while the slightly higher number of parameters in GATv2 does not offer any substantial advantage for this specific task. We must note that the best GATv2 model we achieved for the community scenario has fewer parameters than the best GraphSAGE model found. However, this specific GATv2 configuration, derived from hyperparameter tuning, outperformed other alternative configurations with a higher number of parameters (exceeding those of any GraphSAGE model considered). Therefore, this does not invalidate our current discussion. Even if extended training allowed some GATv2 configurations to improve their performance, they would still not surpass GraphSAGE as a better alternative, since the latter already provides strong performance with lower computational cost (see Table IV). The performance of GraphSAGE is always notably superior to that of MLP, consistently achieving a R2value above 0.7. This outcome remains true even when acting on real-world topologies not seen during training time and with networks of