Evolution of 'pay-it-forward' in the presence of the temptation to free-ride

Uchida, Satoshi; Sasaki, Tatsuya; Yamamoto, Hitoshi; Okada, Isamu Article Evolution of 'pay-it-forward' in the presence of the temptation to free-ride Games Provided in Cooperation with: MDPI – Multidisciplinary Digital Publishing Institute, Basel Suggested Citation: Uchida, Satoshi; Sasaki, Tatsuya; Yamamoto, Hitoshi; Okada, Isamu (2024) : Evolution of 'pay-it-forward' in the presence of the temptation to free-ride, Games, ISSN 2073-4336, MDPI, Basel, Vol. 15, Iss. 3, pp. 1-15, https://doi.org/10.3390/g15030016 This Version is available at: https://hdl.handle.net/10419/330085 Standard-Nutzungsbedingungen: Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen Zwecken und zum Privatgebrauch gespeichert und kopiert werden. Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich machen, vertreiben oder anderweitig nutzen. Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen (insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten, gelten abweichend von diesen Nutzungsbedingungen die in der dort genannten Lizenz gewährten Nutzungsrechte. Terms of use: Documents in EconStor may be saved and copied for your personal and scholarly purposes. You are not to copy documents for public or commercial purposes, to exhibit the documents publicly, to make them publicly available on the internet, or to distribute or otherwise use the documents in public. If the documents have been made available under an Open Content Licence (especially Creative Commons Licences), you may exercise further usage rights as specified in the indicated licence. https://creativecommons.org/licenses/by/4.0/ Citation: Uchida, S.; Sasaki, T.; Yamamoto, H.; Okada, I. Evolution of “Pay-It-Forward” in the Presence of the Temptation to Free-Ride. Games 2024,15, 16. https://doi.org/ 10.3390/g15030016 Academic Editors: Ulrich Berger and Alberto Antonioni Received: 17 February 2024 Revised: 16 April 2024 Accepted: 19 April 2024 Published: 25 April 2024 Copyright: © 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/). games Article Evolution of “Pay-It-Forward” in the Presence of the Temptation to Free-Ride Satoshi Uchida 1,2,*, Tatsuya Sasaki 3, Hitoshi Yamamoto 4and Isamu Okada 5 1Research Center for Ethi-Culture Studies, RINRI Institute, Tokyo 102-8561, Japan 2High-Tech Research Center, Kokushikan University, Tokyo 154-8515, Japan 3Department of Community Development, Koriyama Women’s College, Fukushima 963-8503, Japan; [email protected] 4Faculty of Business Administration, Rissho University, Tokyo 141-8602, Japan; [email protected] 5Department of Business Administration, Soka University, Tokyo 192-8577, Japan; [email protected] *Correspondence: [email protected].jp Abstract: “Paying it forward” is a behavior in which people help someone else because they were helped in the past. Although experimental evidence exists that indicates that real human beings often “pay-it-forward” even in the face of free-rider risks, the theoretical basis for the evolution of this behavior remains unclear. In this paper, we propose a game-theoretical model that explains how payit-forward behavior can evolve despite the temptation to free-ride. By assuming that human beings exhibit cognitive distortions, as predicted by prospect theory, and that free-riding is punished with a tiny probability, we demonstrate that pay-it-forward, alongside unconditional altruistic behavior, can evolve and effectively deter free-riding behavior. Keywords: pay-it-forward; cooperation; social dilemma; punishment; evolutionary game theory; prospect theory; expected utility theory 1. Introduction People often “pay-it-forward” by passing on benefits they have received from someone in the past to others in the future. Experimental and observational studies have provided evidence that people indeed engage in decisions reflective of pay-it-forward behavior [ 1 , 2 ]. For instance, in [ 1 ], game experiments conducted in a laboratory setting revealed that the altruistic behavior of one game player spread to other players within the social network. This propagation of altruistic behaviors from one player to another encourages cooperation among players. The presence of pay-it-forward behaviors in the real world presents a significant challenge to game theory, especially within the context of evolutionary game theory. Essentially, when a player chooses to pass on benefits received from someone in the past to others, they do not gain any immediate personal profits. Consequently, individuals who opt not to pay-it-forward (i.e., those who engage in “free-riding”) often find themselves at a greater advantage compared to those who participate in pay-it-forward activities (“pay-it-forwarders”). Without introducing an additional mechanism, natural selection or social learning processes are expected to favor non-participants [ 3 ]. Thus, identifying a mechanism that discourages free-riding behavior has become a crucial theoretical challenge. Unconditional altruistic behavior, in which individuals help others without expecting anything in return, often leads to what is known as the social dilemma, or free-rider problem, as documented in the literature [ 4 – 10 ]. Pay-it-forward practices, however, represent a form of conditional altruistic behavior where individuals help others only if they themselves have received help in the past. This fact highlights that conditional altruistic behavior shares the same underlying structure as the social dilemma, characterized by the following: (1) individuals in a society have the choice to either engage in pay-it-forward Games 2024,15, 16. https://doi.org/10.3390/g15030016 https://www.mdpi.com/journal/games Games 2024,15, 16 2 of 15 practices or to free-ride, (2) a society comprised mainly of pay-it-forwarders is generally more beneficial than one filled with free-riders, and (3), on an individual level, free-riders tend to fare better than pay-it-forwarders. This is because pay-it-forwarders distribute the benefits they have received to others, whereas free-riders keep any advantages for themselves. In situations presenting a social dilemma, opting for free-riding becomes more appealing for individuals irrespective of the choices made by others. In this paper, we show that pay-it-forward can be sustainably maintained if individuals adhere to what is known as prospect theory (PT) and if punishment for not helping others is applied, albeit infrequently. This stands in contrast to scenarios where individuals are so rational that they follow the (linear) expected utility theory (EUT), under which pay-itforward practices do not persist. In our previous research, we established that prospect theory facilitates cooperation through peer punishment [ 11 ], whereas EUT does not lead to the same outcome when free-riders are only occasionally punished. We hypothesize that a similar rationale applies to the evolution of cooperation through pay-it-forward mechanisms. Investigating this hypothesis is the main focus of the current paper. Prospect theory is widely recognized as one of the most successful frameworks for describing the cognitive biases inherent in human decision-making [ 12 – 14 ]. Within this theory, distortions in the perception of probabilities and the evaluation of outcomes are represented through a weighting probability function and a value function, respectively. Both functions are characterized by their non-linearity and asymmetry, capturing the essence of human irrationality. This theoretical approach highlights how individuals often deviate from rational decision-making models, particularly in the context of risk and uncertainty, by assigning disproportionate weights to certain outcomes and probabilities. Most studies on the evolution of cooperation in evolutionary game theory have traditionally been conducted under the assumption that individuals adhere to EUT [ 5 ]. EUT posits that individuals are perfectly rational, making their calculations regarding potential benefits, or so-called expected payoffs, with precision. Contrary to this assumption, experimental research on both behavioral economics and experimental economics has consistently demonstrated that real humans do not conform to EUT [ 15 , 16 ]. In reality, humans are incapable of estimating probabilities and values without bias. The capacity to accurately calculate probabilities and values, devoid of any distortions, underpins the concept of linearly calculated expected payoffs [ 17 – 22 ]. However, the payoffs, as perceived by real humans, are subjective, and this subjectivity is a critical factor in the actual decision-making process. This discrepancy between the theoretical assumptions of EUT and the practical observations under PT highlights the complexities and nuances in understanding human behavior in evolutionary game-theoretic contexts. In the subsequent sections of this paper, we demonstrate that, when individuals exhibit the level of irrationality encapsulated by PT, pay-it-forward behaviors emerge, and cooperation persists even within parameter regions where conditional altruistic actions cannot be sustained under the rationality assumptions of EUT. Specifically, the risk associated with free-riding behaviors is mitigated by the cognitive biases and decision-making patterns described by PT. Furthermore, the strategy of pay-it-forward behaviors has the potential to overturn unconditional altruistic behavior, highlighting a critical distinction between paying-it-forward and unconditional cooperation, despite their superficial similarities. Following a brief review of the relevant literature in the next section, we will outline our model before presenting our findings. The paper concludes with a section dedicated to our conclusions, summarizing the implications of our research on the understanding of cooperative behaviors and the influence of cognitive biases on the interactions among individuals. 2. Brief Literature Review There are several recent papers that have combined evolutionary game theory with prospect theory, aiming to derive practical implications for addressing social issues. For instance, the study referenced as [ 23 ] considers the irrationality of players, as described by prospect theory, and constructs an evolutionary game-theoretical model to explore the Games 2024,15, 16 3 of 15 decision-making process of enterprises when choosing between innovation and imitation. A paper by Sun, Gao, and Li published in 2022 [ 24 ] analyzes interactions between enterprises and governments, particularly in the context of establishing carbon emission regulations. Meanwhile, the paper referenced as [ 25 ] focuses on the decision-making processes of polluting enterprises and their interactions with government bodies. A more recent study [ 26 ] investigates the recycling of power batteries within the electric vehicle industry, proposing a game model that involves three distinct types of players (governments, manufacturers, and recycling companies). Each of these papers develops an evolutionary game model that incorporates aspects of prospect theory. The evolution of cooperation has been extensively studied within the field of evolutionary game theory, as highlighted in [ 27 ]. A recent theoretical study [ 28 ] attempts to elucidate the evolution of cooperative (and coordinated) behavior by incorporating the effects of prospect theory into a model. This is achieved through the lens of a coordination game known as the stag-hunt game. In this paper, we aim to show a path to the evolution of cooperation through payit-forward behaviors using a prisoner’s-dilemma-type game called the “donation game”. The mechanism to sustain cooperation via pay-it-forward is a type of so-called indirect reciprocity mechanism. Indirect reciprocity is one of the five typical mechanisms to sustain cooperation [ 29 ]. In indirect reciprocity, beneficial acts are returned not by the recipient, as in the direct reciprocity mechanism, but by third parties. In the literature [ 6 ], indirect reciprocity is classified into the following two categories: downstream reciprocity and upstream reciprocity. Pay-it-forward is categorized as upstream reciprocity, which represents the ”I-help-you-because-someone-helped-me” attitude. Downstream reciprocity represents the “I-help-you-because-you-helped-someone” attitude, which is the mainstream in the study of indirect reciprocity. While the literature on the evolution of cooperation via downstream reciprocity is ample [ 30 – 38 ], evolutionary theory has yet to explain the effectiveness of upstream reciprocity in regard to the evolution of cooperation. A well-cited paper on upstream reciprocity in the context of evolutionary game theory was published in 2007 [ 3 ], but it concludes that upstream reciprocity can evolve as a “by-product of direct reciprocity”, i.e., upstream reciprocity works if it is combined with another effective mechanism, direct reciprocity. In [ 39 ], the evolution of upstream reciprocity was explained in a complex network, and in [ 40 ], it was shown that cooperation can evolve by upstream reciprocity only in small interaction groups. According to a thorough review on the theoretical research of indirect reciprocity published recently [ 41 ], although there is some experimentation on upstream reciprocity, explaining it via theoretical viewpoints is still challenging. 3. Model 3.1. Game To maintain simplicity and facilitate analysis of the model, in line with several studies in the field of the evolution of cooperation [ 11 , 42 – 44 ], we consider a large, well-mixed population of individuals (or players). Two players are chosen at random from the population from time to time. Let us call the chosen players A and B. Players A and B engage in the following “donation game” (or “giving game”) [ 6 ]: each player decides whether to help the opponent by paying a cost c . If Player A (B) chooses to help B (A), the opponent receives a benefit b>c ; otherwise A (or B) does not pay anything and B (A) obtains nothing. Every player in the population is selected as a game player (infinitely) many times and experiences decision-making processes. From here on, we denote the action “cooperate with its opponent (or just cooperate)” by “C” and “defect its opponent (or just defect)” by “D”. Table 1shows the payoffs Player A obtains when playing the donation game with Player B. It consists of four possible outcomes, as both players have two options each. If both players choose C, Player A gains a benefit of b−c . If Player A chooses C while Player B opts for D, Player A incurs a loss of c . Conversely, if Player A selects D and Player B Games 2024,15, 16 4 of 15 chooses C, then Player A receives a benefit of b . If both players select D, Player A receives no benefit (and neither does Player B). Table 1. Payoffs player A obtains in donation game. Player B’s Option Player A’s Option Cooperate (C) Defect (D) Cooperate (C) b −c−c Defect (D) b 0 3.2. Game, Strategies and Payoff Matrix We consider the following three strategies: AC, AD, and S [ 42 ]. Strategy AC prescribes to help others with probability 1 −ε . Due to the shortage of resources, AC-players do not give help with probability ε , even if they would like to. We call ε an implementation error and set its value to 0.05. Players using strategy AD are free-riders and never help others. We call those players AD-players. Strategy S players correspond to pay-it-forward, which prescribes to choosing C (with probability 1 −ε ) if they were helped in the last game. If they were not helped in the last game, S-players choose D in the current game. We also refer to the strategies by numbers: AC = strategy 1, AD = strategy 2, and S = strategy 3. 3.3. Payoff Matrix We examine a type of punishment known as pool punishment [ 45 – 51 ]. In this framework, a punishment institution exists in the population and executes punishment to those who chose D in the donation game. Players in the population pay a cost d , which will be used to maintain the punishment institution. In this paper, we assume that the institution executes punishment with only a small probability to account for situations where the institution functions insufficiently. From the viewpoints of players, after the two selected players have played the donation game, they have a small chance of being punished by the external institution if they chose D in the donation game. The probability of being punished is δ , which is set to be 0.2 in the following numerical simulations. We assume that AD-players, who never give help to others, do not pay d (See Section 3.5 for more details). In this sense, AD-players are not only first-order free-riders but also second-order free-riders. Here, first-order free-riders mean that they do not engage in cooperative activities, while second-order free-riders are defined as those who do not contribute to punishment activities. AD-players are assumed to be both first-order and second-order free-riders at the same time. This is a harsher assumption for the evolution of cooperation than the assumption that AD-players also pay the cost to maintain the punishment institution, as other types of players are at more of a disadvantage if AD-players do not incur the punishment cost. The strength of punishment is denoted by s . Thus, if a player chooses D in the donation game, and if punishment is executed by the institution (which occurs with probability δ ), the player suffers a fine, s . If punishment is not executed, nothing is imposed on the player. As a result, we obtain Table 2as the payoff matrix studied in this paper. This matrix comprises eight elements, as Player A has two options (C or D), and, for each option, there are four possibilities depending on whether the opponent, Player B, chooses C or D and whether the institution executes punishment or not. Table 2. Payoff matrix M(payoffs Player A obtains in total). Outcome Player A’s Option Cooperate Not-Punish (CN) Cooperate Punish (CP) Defect Not-Punish (DN) Defect Punish (DP) Cooperate (C) b−c0−c0 Defect (D) b b −s0−s Games 2024,15, 16 5 of 15 We call this payoff matrix M . In Table 2, for example, “CP” means that Player B helps Player A and the institution executes punishment. In this case, if A does not help B, A obtains benefit b but suffers s at the same time. The total payoff in this case is b−s(=M22) . It is worth noting that if Player A helps Player B, A will never be punished. Consequently, the element of the payoff matrix corresponding to this scenario ( M12 ) is undefined. For the same reason, M14 is also undefined, as Player A’s action of choosing C will never result in punishment. We formally put value “0” for the elements M12 and M14 in the table. However, these values are never used, as these situations never occur. The cost of implementing punishment is not included in the payoff matrix, as it does not result from interactions with other players. The cost for punishment is introduced after estimating the expected payoffs on the basis of interactions with other players (See Section 3.5). 3.4. Strategy Switching Players occasionally switch their strategies by social learning. As a result of the switching process, the time evolution of the proportion of strategy i ( i∈{1, 2, 3} ), denoted by xi, is governed by the so-called replicator equation . xi=xiPi−P, where Pi is the expected payoff of strategy i and P=∑3 i=1xiPi is the expected payoff of the whole population. Since xiis the proportion of strategy i,∑3 i=1xi=1 holds. By numerically integrating the replicator equation mentioned above, we can analyze the stability of each strategy. In the subsequent subsection, we will describe how the expected payoff Piis calculated. 3.5. Probability Distribution on M and Expected Payoff Pi 3.5.1. Linear Expected Utility Theory Expected payoff Pi reflects the long-term payoff of players using strategy i . It can be determined by calculating the probability that each outcome for payoff matrix Mmn occurs. The probabilities are dependent on strategies. By Bi , we denote the probability distribution on Mof players using strategy i. To determine probability distributions, it is crucial to first know the probability of whether players following a certain strategy will offer help to others. For those adhering to the AC strategy, this probability is 1 −ε , while for AD, it is 0. To ascertain the probability that S-players will offer help, we define p as the frequency at which players receive help from a random player within the population. This frequency, p, can also be interpreted as the probability of a player being given help by a random participant in the game. It is important to note that the act of receiving help in the game does not rely on the strategy used by the recipient. Therefore, p is strategy-independent and represents the chance that any given player is helped in a game. Consequently, the probability that S-players will choose action C is represented by (1−ε)p. The frequency p evolves over time as individuals engage in the game. We assume that the dynamics of p occur at a much faster pace than changes in strategy. In essence, players experience various interactions an infinite number of times before altering their strategies. This approach keeps the variables xi constant while allowing p to vary, a premise similarly assumed in Sections 4.2 and 4.3 of [6]. As demonstrated in Appendix A, p reaches an equilibrium defined by the following equation: p=(1−ε)x1+(1−ε)px3. Here, the first term on the right-hand side represents the probability of encountering an AC-player ( x1 ), who then opts for cooperation (C) with probability 1 −ε . The second term accounts for the likelihood of meeting an S-player ( x3 ) who, having previously been given help, chooses C with a probability of (1−ε)p. Games 2024,15, 16 6 of 15 By solving this equilibrium equation, we find pto be: p=(1−ε)x1 1−(1−ε)x3 . Since probability distributions on M depend on strategies, the expected payoff is dependent on strategies. Table 3(a)–(c) show probability distributions Bi with parameters (ε,δ,p). In the tables, ε=1−ε,δ=1−δ,p=1−p. Table 3. Probability distributions for strategy 1 (a), strategy 2 (b), and strategy 3 (c). (a) B1 Outcome Player A’s option Cooperate Not-Punish (CN) Cooperate Punish (CP) Defect Not-Punish (DN) Defect Punish (DP) Cooperate (C) εp0εp0 Defect (D) εδpεδpεδpεδp (b) B2 Outcome Player A’s option Cooperate Not-Punish (CN) Cooperate Punish (CP) Defect Not-Punish (DN) Defect Punish (DP) Cooperate (C) 0 0 0 0 Defect (D) δpδpδpδp (c) B3 Outcome Player A’s option Cooperate Not-Punish (CN) Cooperate Punish (CP) Defect Not-Punish (DN) Defect Punish (DP) Cooperate (C) εp20εpp 0 Defect (D) (1−εp)δp(1−εp)δp(1−εp)δp(1−εp)δp For example, (B1)22 is the product of the following three probabilities: the probability that an AC-player is given help by the opponent player of a game ( p ), the probability that the AC-player chooses D ( ε ), and the probability that this behavior (i.e., choosing D) is punished by the external institution (δ). Note that (Bi)12 =(Bi)14 =0 for any i. With these probability distributions on M , the expected payoff of players using strategy ithrough interactions with other players is given by Qi= 2 ∑ m=1 4 ∑ n=1 (M)mn(Bi)mn. For example, Q1=(b−c)εp−cεp+bεδp+(b−s)εδp−sεδp. Considering that AD-players do not pay any cost to maintain the external punishment institution, we find the final form of the expected payoff of players following strategy i: Pi=Qi−d, for i=1, 3 and Pi=Qi, for i=2. Here, d is the cost to maintain the punishment institution. AD-players are at more of an advantage than AC-players and S-players by the value of d. Games 2024,15, 16 7 of 15 3.5.2. Prospect Theory The way we include prospect theory in our evolutionary game model is essentially the same as in our previous paper [ 11 ]. Thus, we apply a value function and weighting function to each element of the payoff matrix M and to the probability distribution Bi , respectively: (V)mn =v((M)mn), (Wi)mn =w((Bi)mn), where v and w are the value function and the weighting function, respectively. In Appendix B, we describe the appearance of the functions. The subjectively distorted payoff matrix V is independent of strategies. It is wellknown that the nonlinearity of the value function means that the baseline of the payoff matrix, or the reference point, significantly influences the results, differing from EUT. Specifically, within the EUT framework, adding the same constant to all elements of the payoff matrix M does not alter the payoff structure. In other words, the relative superiority or inferiority of strategies remains unchanged, even if each player consistently receives the same payoff. However, under PT, adding the same constant (for instance, by assuming each player obtains a constant payoff in playing a game) can affect the outcomes. We assume the payoff obtained from the outcome (2,3) in M , which is zero, serves as the so-called reference point, with no additional constants added. With these subjective probabilities and payoffs, the expected payoff is calculated as Pi= 2 ∑ m=1 4 ∑ n=1 (V)mn(Wi)mn +v(−d), for i=1, 3 and Pi= 2 ∑ m=1 4 ∑ n=1 (V)mn(Wi)mn, for i=2. Note that we apply only the value function, not the weighting function, to the last term ( −d : the loss of payoff by paying a cost for punishment) for the following reason: it does not represent a loss suffered through interaction in a game and is therefore not a stochastic event (hence, the weighting function plays no role). By applying the value function to d , the perceived impact of the cost for maintaining the punishment institution is regarded as more significant than its actual impact. We also note that the strategy change is described by the ordinary replicator dynamics mentioned above. 4. Results 4.1. Analysis in Case of EUT In the context of expected utility theory (EUT), the replicator dynamics can be thoroughly analyzed due to the simplicity of the expected payoffs. The expected payoffs for AC-players, AD-players, and S-players are, respectively represented as P1=pb −(1−ε)c−εδs−d, P2=pb −δs, P3=pb −(1−ε)pc −[1−(1−ε)p]δs−d. From these equations, it can be demonstrated that AD-players become extinct through the replicator dynamics, while AC-players evolve and are sustained, provided the punishment is effective or substantial enough (that is, if the parameters s or δ surpass certain threshold values). Specifically, P1>P2 suggests that δs>c+d/(1−ε) , while P1>P3 implies δs>c . If these conditions are met, the AC strategy emerges as the unique evolutionarily stable Games 2024,15, 16 8 of 15 strategy (ESS), leading all players to eventually adopt it, resulting in x1= 1. Conversely, when P1<P2, the AD strategy becomes the unique ESS. The S strategy can never be an ESS because a population consisting solely of S-players is vulnerable to invasion by an AD-player. Indeed, at the point where x3= 1, which implies p= 0, we find that P3=−δs−d<−δs=P2 . This indicates the inability of the S strategy to sustain itself as a stable strategy in the population if players follow EUT. In the following numerical investigations, we fix parameter values as b= 5.0, c= 1.0, ε= 0.05, d= 0.05. With these values, the conditions P1>P2 and P1>P3 hold if δs> 20 / 19 ≈ 1.05. Figure 1a–c display trajectories in the state space obtained by numerically integrating the replicator dynamics with δ=0.2 and with s=5.6, 5.1, 4.6, respectively. Games 2024, 15, x FOR PEER REVIEW 8 of 15 implies 𝑝=0, we find that 𝑃3=−𝛿𝑠−𝑑<−𝛿𝑠=𝑃2. This indicates the inability of the S strategy to sustain itself as a stable strategy in the population if players follow EUT. In the following numerical investigations, we fix parameter values as 𝑏 =5.0,𝑐 = 1.0,𝜀 = 0.05,𝑑=0.05. With these values, the conditions 𝑃1>𝑃2 and 𝑃1>𝑃3 hold if 𝛿𝑠>20/19≈1.05. Figure 1a–c display trajectories in the state space obtained by numerically integrating the replicator dynamics with 𝛿=0.2 and with 𝑠=5.6,5.1,4.6, respectively. The replicator dynamics analyzed to produce the figure are given by the following coupled equation system: 𝑑𝑥1 𝑑𝑡 =𝑥1(𝑃1−∑𝑥𝑖𝑃𝑖 3 𝑖=1 ), 𝑑𝑥2 𝑑𝑡 =𝑥2(𝑃2−∑𝑥𝑖𝑃𝑖 3 𝑖=1 ), 𝑑𝑥3 𝑑𝑡 =𝑥3(𝑃3−∑𝑥𝑖𝑃𝑖 3 𝑖=1 ). The initial conditions are set randomly. The state space is represented as the simplex {(𝑥1,𝑥2,𝑥3)|0≤𝑥1≤1,0≤𝑥2≤1,0≤𝑥3≤1,𝑥1+𝑥2+𝑥3=1}, visualized as an equilateral triangle. The vertex AC corresponds to (𝑥1,𝑥2,𝑥3)=(1,0,0), AD to (0,1,0) and S to (0,0,1). The state of the population (𝑥1,𝑥2,𝑥3) evolves within this space according to the replicator equation. The figures also illustrate vector fields generated by the replicator dynamics. If 𝑠 is sufficiently large, for example, 5.6, all trajectories converge to the vertex AC, as predicted by the above analysis (see Figure 1a). Consequently, all players ultimately adopt strategy AC, consistently aiming to help others. (a) (b) AD S AC AD S AC Games 2024, 15, x FOR PEER REVIEW 9 of 15 (c) Figure 1. Trajectories and vector fields yielded by replicator dynamics for case of linear expected utility theory, with 𝑠=5.6 (a), 𝑠=5.1 (b) and 𝑠=4.6 (c). Initial conditions are random. State space is simplex defined by {(𝑥1,𝑥2,𝑥3)|0≤𝑥1≤1,0≤𝑥2≤1,0≤𝑥3≤1,𝑥1+𝑥2+𝑥3=1}, which is represented as an equilateral triangle. Arrows in trajectories show in which direction state (𝑥1,𝑥2,𝑥3) evolves. Other parameters: 𝑏 =5.0,𝑐 =1.0,𝜀 = 0.05,𝑑=0.05,𝛿=0.2 . With lower values of 𝑠, as depicted in Figure 1b,c, all trajectories converge towards the AD strategy. At these parameter values, rational individuals opt for the AD strategy because the punishment executed by the institution is insufficiently severe (i.e., 𝑠 is small) and the probability of being punished is too low (i.e., 𝛿 is small). Consequently, the punitive institution fails to function effectively in this scenario. A notable difference between Figure 1b (𝑠=5.1) and Figure 1c (𝑠=4.6) is observed in the vector directions on the edge between AC and S. In the former scenario, AC is able to invade S, whereas, in the latter, S invades AC. However, it is important to note that S can never achieve evolutionary stability as it is susceptible to invasion by AD. Consequently, we can deduce that the pay-it-forward strategy, S, does not significantly contribute to the evolution of cooperation with the institutional punishment within the EUT framework. 4.2. Numerical Results in Case of PT Figure 2a–c illustrate the trajectories and vector fields when prospect theory is applied, using the same parameter values as in Figure 1. We highlight the following two observations: (1) Figure 2b demonstrates that, even with parameter values for which AD is the unique evolutionarily stable strategy under EUT, cooperation evolves when players adhere to PT; that is, AC is the ESS. (2) In Figure 2c, a new stable fixed point, called T, emerges on the edge between S and AC. This fixed point represents a mixed state mainly comprising S-players, with AC-players also being present. The system exhibits bi-stability, as some trajectories converge towards the vertex AD, depending on their initial conditions. In the case of Figures 1 and 2c in particular, the AD strategy has several advantages. First, the condition 𝑏>𝑠 indicates that the punishment is not severe enough to offset the benefits gained from free-riding. In addition, with 𝛿 being relatively small, choosing the D strategy entails an 80% chance of avoiding punishment. As such, AD players face a low probability (0.2) of being penalized despite consistently choosing D. Moreover, AD players receive an additional payoff from 𝑑, as they incur no costs to support the institution. AD S AC Figure 1. Trajectories and vector fields yielded by replicator dynamics for case of linear expected utility theory, with s= 5.6 (a), s= 5.1 (b) and s= 4.6 (c). Initial conditions are random. State space is simplex defined by {(x1,x2,x3)|0≤x1≤1, 0 ≤x2≤1, 0 ≤x3≤1, x1+x2+x3=1} , which is represented as an equilateral triangle. Arrows in trajectories show in which direction state (x1,x2,x3) evolves. Other parameters: b=5.0, c=1.0, ε=0.05, d=0.05, δ=0.2. Games 2024,15, 16 15 of 15 36. Uchida, S.; Yamamoto, H.; Okada, I.; Sasaki, T. A theoretical approach to norm ecosystems: Two adaptive architectures of indirect reciprocity show different paths to the evolution of cooperation. Front. Phys. 2018,6, 14. [CrossRef] 37. Yamamoto, H.; Okada, I.; Uchida, S.; Sasaki, T. Exploring norms indispensable for both emergence and maintenance of cooperation in indirect reciprocity. Front. Phys. 2022,10, 1019422. [CrossRef] 38. Fujimoto, Y.; Ohtsuki, H. Evolutionary stability of cooperation in indirect reciprocity under noisy and private assessment. Proc. Natl. Acad. Sci. USA 2023,120, e2300544120. [CrossRef] 39. Iwagami, A.; Masuda, N. Upstream reciprocity in heterogeneous networks. J. Theor. Biol. 2010,265, 297–305. [CrossRef] [PubMed] 40. Pfeiffer, T.; Rutte, C.; Killingback, T.; Taborsky, M.; Bonhoeffer, S. Evolution of cooperation by generalized reciprocity. Proc. R. Soc. B2005,272, 1115–1120. [CrossRef] [PubMed] 41. Okada, I. A review of theoretical studies on indirect reciprocity. Games 2020,11, 27. [CrossRef] 42. Ohtsuki, H.; Iwasa, Y. Global analyses of evolutionary dynamics and exhaustive search for social norms that maintain cooperation by reputation. J. Theor. Biol. 2007,244, 518–531. [CrossRef] 43. Uchida, S.; Sigmund, K. The competition of assessment rules for indirect reciprocity. J. Theor. Biol. 2010,263, 13–19. [CrossRef] 44. Murase, Y.; Kim, M.; Baek, S.K. Social norms in indirect reciprocity with ternary reputations. Sci. Rep. 2022,12, 455. [CrossRef] 45. Yamagishi, T. The provision of a sanctioning system as a public good. J. Personal. Soc. Psychol. 1986,51, 110–116. [CrossRef] 46. Boyd, R.; Richerson, P.J. Punishment allows the evolution of cooperation (or anything else) in sizable groups. Ethol. Sociobiol. 1992,13, 171–195. [CrossRef] 47. Milinski, M.; Rockenbach, B. Human behaviour: Punisher pays. Nature 2008,452, 297–298. [CrossRef] 48. Casari, M.; Luini, L. Cooperation under alternative punishment institutions: An experiment. J. Econ. Behav. Organ. 2009,71, 273–282. [CrossRef] 49. Kosfeld, M.; Okada, A.; Riedl, A. Institution formation in public goods games. Am. Econ. Rev. 2009,99, 1335–1355. [CrossRef] 50. Sigmund, K.; de Silva, H.; Traulsen, A.; Hauert, C. Social learning promotes institutions for governing the commons. Nature 2010, 466, 861–863. [CrossRef] 51. Traulsen, A.; Röhl, T.; Milinski, M. An economic experiment reveals that humans prefer pool punishment to maintain the commons. Proc. Biol. Sci. 2012,279, 3716–3721. [CrossRef] Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.