Learning Decision Criteria from Play

Galeazzi, Paolo; Madsen, Mathias W. Article — Published Version Learning Decision Criteria from Play Dynamic Games and Applications Provided in Cooperation with: Springer Nature Suggested Citation: Galeazzi, Paolo; Madsen, Mathias W. (2024) : Learning Decision Criteria from Play, Dynamic Games and Applications, ISSN 2153-0793, Springer US, New York, NY, Vol. 15, Iss. 3, pp. 1037-1069, https://doi.org/10.1007/s13235-024-00595-2 This Version is available at: https://hdl.handle.net/10419/323676 Standard-Nutzungsbedingungen: Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen Zwecken und zum Privatgebrauch gespeichert und kopiert werden. Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich machen, vertreiben oder anderweitig nutzen. Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen (insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten, gelten abweichend von diesen Nutzungsbedingungen die in der dort genannten Lizenz gewährten Nutzungsrechte. Terms of use: Documents in EconStor may be saved and copied for your personal and scholarly purposes. You are not to copy documents for public or commercial purposes, to exhibit the documents publicly, to make them publicly available on the internet, or to distribute or otherwise use the documents in public. If the documents have been made available under an Open Content Licence (especially Creative Commons Licences), you may exercise further usage rights as specified in the indicated licence. http://creativecommons.org/licenses/by/4.0/ Dynamic Games and Applications (2025) 15:1037–1069 https://doi.org/10.1007/s13235-024-00595-2 Learning Decision Criteria from Play Paolo Galeazzi1·Mathias W. Madsen2 Accepted: 5 September 2024 / Published online: 26 September 2024 © The Author(s) 2024 Abstract This paper investigates population games under ambiguity in which players may adopt decision criteria different from one another. After defining equilibria for these situations by extending well-known decision-theoretic criteria to the game-theoretic context, we apply these concepts to examine the case of two-person games played within a population whose relative proportions of decision criteria are unknown to the players. We state necessary and sufficient conditions under which such games prompt the players to reveal their decision criterion through their actions, and we show when the relative proportions may be learned by observing the increasingly informed agents play. Keywords Decision criteria ·Learning ·Population games 1 Introduction In the last decades, the necessity of elevating the analysis of the individuals’ behavior from observed actions to underlying mechanisms has emerged in various ways in different fields, from psychology to ecology to evolutionary game theory [1,9,14,16,17,19–21,23,31]. The general idea, as expressed e.g. by [14], is that Natural environments are so complex, dynamic, and unpredictable that natural selection cannot possibly furnish an animal with an appropriate, specific behavior pattern for every conceivable situation it might encounter. Instead, we should expect animals to have evolved a set of psychological mechanisms which enable them to perform well on average across a range of different circumstances. Here we take this idea seriously and consider a game-theoretic model where a population of agents inhabits an environment consisting of a multitude of different games (which we call multigame), and each individual in the population is endowed with a “psychological mechanism” that produces a specific behavior for any possible game in the environment. The main research question that we want to investigate then concerns the possibility of discerning the different underlying mechanisms by observing the agents’ expressed behaviors only. That BPaolo Galeazzi [email protected] 1University of Bayreuth, Bayreuth, Germany 2Micropsi industries, Berlin, Germany 1038 Dynamic Games and Applications (2025) 15:1037–1069 is: Under which conditions is it possible to distinguish the agents’ underlying mechanisms given their behavior? In this work, we present a case where the agents’ behavior-generating mechanisms are represented by different decision criteria and each of such criteria makes the agent act in a certain way when faced with a specific game in the environment. The decision criteria that we consider in the following are arguably the two main criteria for choice under ambiguity from the decision-theoretic literature, i.e., maxmin expected utility and regret minimization. The following example introduces a simple instance of the population model that we study in this paper. Consider a population living in an environment consisting of the three games below. The first is a Prisoner’s Dilemma (PD), the second is a Stag Hunt (SH), and the third is an anti-coordination game (AG). PD III I2,2 0,3 II 3,0 1,1 SH III I3,3 0,2 II 2,0 2,2 AG III I1,1 2,5 II 5,2 0,0 Individuals from such population randomly meet and play one of these three possible games. Crucially, however, each individual plays the game based on her own behaviorgenerating mechanism, namely, her own decision criterion—which we also refer to as the individual’s type. A maxmin player by definition chooses an action that guarantees the highest minimum payoff. In the SH game, for instance, the minimum payoff that one can get from action Iis 0, while the minimum payoff from action II is 2, and a maxmin player would therefore choose action II in the SH game. A regret-minimizing player instead aims to choose an action that minimizes the regret, defined as the maximum amount possibly given up by playing a certain action. For instance, the maximum regret from action Iin the SH game is 2, which is the payoff given up by playing action Iwhen the opponent plays II.By similar reasoning, the regret from action II in the SH game is 1, which is the payoff given up by choosing II when the opponent chooses I. A regret minimizer would hence play II in the SH game. Similar computations lead to the conclusion that both a maxmin player and a regretminimizing player would choose action II in the PD game too. In an environment consisting uniquely of one or both of these two games the two player types would thus be behaviorally indistinguishable. Looking at the AG game, however, one can compute that a maxmin player would play action Iwhile a regret minimizer would play action II. In the environment including all three games, the different types are distinguishable. In the following, we study the conditions on the games in the environment ensuring that the types in the population are distinguishable. To do that, we first define the concepts of games with ambiguity on the decision criteria and of equilibrium in such games in Sect.2, and then we state the conditions for a 2 ×2 game to be informative, that is, to allow telling different types apart, in Sect. 3. In Sect. 4, we introduce the population multigame, we study the properties of the environment that guarantee that informative 2×2 games can always occur with positive probability, and we generalize the results to the case of n×ngames. Section 5 then considers a specific instance of population multigame and shows how to compute the probability of informative games in that particular case and that the agents can asymptotically learn the precise proportions of types in the population. Section6instead considers a variety of different multigames and computes the probability of strongly informative games in different cases by means of computer simulations. Finally, Sect.7concludes. Before moving to Sect.2, however, in the next subsection we say a few words on the literature related to the present work. Dynamic Games and Applications (2025) 15:1037–1069 1039 1.1 Related Literature Although, as already mentioned above, the necessity of developing models with agents characterized by different behavior-generating mechanisms has been explicitly advocated especially by biologists and ecologists [14,21,23], the game-theoretic literature in economics has almost always focused on models with homogeneous decision criteria—i.e., models where all the agents follow the same decision criterion [3,22,24–28,30,33]. A few exceptions are the following. [2] study necessary and sufficient conditions for the existence of an equilibrium in games under ambiguity where the agents can have very general subjective choice preferences. [10,12,13]and[18] introduce epistemic type spaces that allow the agents to follow different decision criteria and to reason strategically about each other’s criteria, but their results are purely on the epistemic side. On the evolutionary side, the evolution of preferences [9,11] is a branch of evolutionary game theory that studies the evolutionary fitness of different subjective preferences, but the focus there is on the players’ subjective utility functions rather than on the players’ decision criteria. Moreover, here we are primarily interested in the learning and not in the evolution of the players. The idea of investigating multigame models has sometimes appeared in other fields too. [16]and[17] study the evolution of decision criteria in an environment similar to the one we consider here. [4] too consider evolutionary processes driven by a multigame environment, but with the difference that their agents are defined by automata rather than by decision criteria. [38] exploits a multigame consisting of three different games to explain the evolution of fairness, but the types there are decision rules specific to those games and hence simpler than the decision criteria examined here. 2 Games with Criterion Ambiguity In the simple example from the previous section, the players were implicitly assumed to hold uncertainty over the opponents’ actions and to use possibly different decision criteria to cope with such uncertainty and to pick an action for any given game in the environment. In population games, however, it is natural to imagine that the players’ uncertainty comes from the distribution of the different types in the population. In this section, we formally introduce population games with criterion ambiguity and show how the uncertainty on the opponent’s actions is derived from the uncertainty on the type distribution in the population. We interpret these games as modeling a situation in which players are drawn at random from a large and mixed population where maxmin types (M) and regret-minimizing types (R) coexist in unknown proportions. The concept of game with criterion ambiguity and that of equilibrium in such games can then be formalized as follows. Definition 1 Agame with criterion ambiguity consists of the following components: •a set of worlds ; •a set of parameters ; •asetI={1,2,3,...,N}of players; •for each player i, – an action set Ai; – a set of (criterion) types Ti; – a set of signals Si; – a criterion-assignment function τi:→Ti; 1040 Dynamic Games and Applications (2025) 15:1037–1069 – a signal function ςi:→Si; – a utility function ui:A1×···×An×→R; •for each λ∈, a probability distribution Pλover . Apolicy of player iin such a game is a function σi:Ti×Si→Aithat associates each decision criterion and signal with an action.1 Note that if the set is equipped with a probability distribution known to all the players, this definition describes a Bayesian game [29]. However, we are interested in situations where this is not the case and the players hold ambiguity (i.e., non-probabilistic uncertainty) about the parameter λ. Throughout the rest of the paper we denote typical profiles of signals and types by s∈ S:= S1×S2×···×SNand t∈T:= T1×T2×···×TN, respectively. For profiles of types, signals, or policies, we use the notation x−i=(x1,...,xi−1,xi+1,...,xN)for the (N−1)-dimensional vector that results by removing the ith coordinate from the vector x. In games where the uncertainty only concerns the criteria adopted by the others, no private information other than their own decision criterion is revealed to the agents. In the notation above, this can be modeled by assuming that the sets Siare all singletons. To ease notation, we therefore dispense with the specification of the private signals siin the following discussion. For any fixed value of λ, the type of a player is a random variable. Hence, given policies (σi)i∈I, the action ai=σi(ti)too is a random variable, and (a1,a2,...,aN)is a random vector. The utility of each player is therefore also a random variable, which has an expected value for any fixed λ. Since all of these distributions depend on λ, the value of this expected utility is a function of λ. We then let each player resolve the ambiguity about this expected utility by means of the decision criterion given by his or her type. In formulas, the expected utility that follows from type tiusing action ai, given that the other players adopt policies σ−i,is Eλ[ui|ai,ti,σ −i]= ui(ai,σ −i(τ−i(ω)), ω) Pλ(dω|ti). (1) Definition 2 Let an incomplete information game with criterion ambiguity be given as above, with Ti={M,R}for all i∈I. We say that a policy profile (σ∗ i)i∈Iis an equilibrium if, for all i∈I, the policy σ∗ imaximizes the function σi→ inf λ∈Eλ[ui|σi(M), M,σ∗ −i] and minimizes the function σi→ sup λ∈sup ai∈Ai Eλ[ui|ai,R,σ∗ −i]−Eλ[ui|σi(R), R,σ∗ −i]. 2.1 Population Games with Criterion Ambiguity As a main source of ambiguity about the criteria, we consider a large population of agents using different decision criteria that randomly meet to play games. Since it is often unlikely to know the precise distribution of types in a large population, we allow the agents to hold unmeasurable uncertainty about the distribution of criteria in the population they are part 1There is no consensus in game theory about the possibility of using mixed actions (see for instance [32]for a discussion). Here we follow the position of [13]and[12]. Dynamic Games and Applications (2025) 15:1037–1069 1041 of. In particular, the rest of this paper is concerned with a class Gof two-player population games with criterion ambiguity. At first, we focus on 2 ×2 symmetric games, and we later show how the results can be generalized to n×nsymmetric games.2 In the following, we assume that each of the two players i∈{1,2}has his or her own decision criterion ti∈{M,R}revealed, but holds unmeasurable uncertainty about the opponent’s criterion. Specifically, we assume that Pλ(t3−i=R|ti=R)=Pλ(t3−i=R|ti=M)=λ Pλ(t3−i=M|ti=R)=Pλ(t3−i=M|ti=M)=1−λ where λ∈[λ, λ]⊆[0,1]is subject to unmeasurable uncertainty. In the case of 2 ×2 games, the agents are equipped with the binary action sets A1=A2={I,II}, and their utility functions are defined in terms of the symmetric 2 ×2 game matrix III I a,a b,c II c,b d,d for all ω∈,where(a,b,c,d)∈R4. The two players of this game are then agents sampled at random from a large population characterized by the unknown parameter λ.For convenience, we use the vector notation σi=(σi(M), σi(R)) to specify the policy function of each agent i∈{1,2}. Example 3 Suppose two agents are randomly drawn from a population consisting of a proportion λof regret minimizers and 1−λof maximinimizers. The exact value of λis unknown and subject to ambiguity, with λ∈[1/5,1/2]. The two agents thus play the following coordination game under ambiguity about the value of λ. III I1,1 0,0 II 0,0 2,2 What are the equilibria of this game? Each player in this game is a regret type with probability λand a maxmin type with probability 1 −λ, and their policy functions take the form of pairs of pure actions, with one action for each of these two types. By inspecting each of the 16 policy profiles, we can reject the ones in which any of the two players is not using a best reply. Consider first the case in which the row player faces the column policy σ2=(I,I).Given such a homogeneous policy, the action of the column player is deterministic and hence not subject to uncertainty, unmeasurable or otherwise. We therefore find that the unique best reply to this policy is σ1=(I,I). We similarly find that the unique best reply to σ2=(II,II) is σ1=(II,II). Since the exact same argument holds for the column player, it follows that the policies (σ1,σ 2)=((I,I), (I,I)) (σ1,σ 2)=((II,II), (II,II)) are both equilibria of this game, and that the homogeneous policies (I,I)and (II,II)appear in no other equilibria. 2Having symmetric games only allows us to stick with the single-population model, as we need not consider a different population for each role in the game. 1042 Dynamic Games and Applications (2025) 15:1037–1069 Suppose now that the column player uses the policy σ2=(I,II). Then the conditional expected utilities of the row player given λare Eλ[u1|(I,(I,II))]=1−λ Eλ[u1|(II,(I,II))]=2λ For the maxmin type of the row player, action Iis a best reply to σ2=(I,II)since the inequality min λ∈[1/5,1/2]1−λ≥min λ∈[1/5,1/2]2λ reduces to the true statement 1/2≥2/5. For the regret-minimizing type of the row player, on the other hand, action II is a best reply to σ2=(I,II). This follows from the fact that the conditional regrets of action Iand II given λare max a1∈A1 Eλ[u1|(a1,(I,II))]−Eλ[u1|(I,(I,II))]=max{0,3λ−1} max a1∈A1 Eλ[u1|(a1,(I,II))]−Eλ[u1|(I,(I,II))]=max{1−3λ, 0}, and max λ∈[1/5,1/2]{max{0,3λ−1}}≥max λ∈[1/5,1/2]{max{1−3λ, 0}} reduces to the true statement 1/2≥2/5. The policy σ1=(I,II)is thus a best reply to the policy σ2=(I,II). Suppose now that the column player uses σ2=(II,I). We then find that Eλ[u1|(I,(II,I))]=λ Eλ[u1|(II,(II,I))]=2(1−λ) It follows that action II is a best reply for the regret type, since max a1∈A1 Eλ[u1|(a1,(II,I))]−Eλ[u1|(I,(II,I))]=max{0,2−3λ} max a1∈A1 Eλ[u1|(a1,(II,I))]−Eλ[u1|(II,(II,I))]=max{3λ−2,0} and max λ∈[1/5,1/2]{max{0,2−3λ}}≥max λ∈[1/5,1/2]{max{3λ−2,0}} reduces to the true statement 7/5≥0. Action Iis thus not a regret-minimizing reply to the policy σ1=(II,I), and hence σ2=(II,I)cannot be a best reply to σ1=(II,I). Since we have already ruled out the options σ2=(I,I)and σ2=(II,II)above, the only remaining option is σ2=(I,II). However, we have also seen that (σ1,σ 2)=((I,II), (II,I)) is not an equilibrium, and since the game is symmetric, neither is (σ1,σ 2)=((II,I), (I,II)). In sum, we have that the three policy profiles (σ1,σ 2)=((I,I), (I,I)) (σ1,σ 2)=((II,II), (II,II)) (σ1,σ 2)=((I,II), (I,II)) are the only equilibria of the game. Dynamic Games and Applications (2025) 15:1037–1069 1043 We are interested here in symmetric equilibria, i.e., equilibria such that σ1=σ2,asthis is the only type of equilibrium that can be interpreted as a population adaptive strategy. In particular, in the case of 2 ×2 games we are interested in the policy profiles (σ1,σ 2)=((I,II), (I,II)) (σ1,σ 2)=((II,I), (II,I)) since a population playing according to any of these profiles allows the observers to infer the decision criteria of the players from their actions. In games where exactly one of these profiles is the sole symmetric equilibrium, the players necessarily reveal their decision criterion. We then say that such games are strongly informative, and in the next section we provide necessary and sufficient conditions for a game to be strongly informative. 3 Strongly Informative Games Strongly informative games are the key to the learning of the actual proportions of decision criteria in the population, because only by playing strongly informative games the players unambiguously reveal their type. For any interval [λ, λ]with λ < λ, it is solely the positive probability of a strongly informative game to occur that can give the players relevant information about the proportions in the population. In this section, we characterize the region of strongly informative games in R4and we next provide conditions on the distribution of possible games in the class Gthat guarantee the occurrence of strongly informative games. 3.1 Strong Informativity with Full Uncertainty We first formulate the conditions under which a game (a,b,c,d)is strongly informative given that all the players have full unmeasurable uncertainty about the proportions of decision criteria, i.e., for λ∈[0,1]. As a first step, we can immediately reduce the set of strongly informative games by discarding all games that are not anti-coordination games. In the following, informativity in 2 ×2 games will be enough for most of our purposes. However, we can prove the following result for general n×ngames. To that aim, we partition the class of symmetric n×ngames into three sets: •coordination games: ai∈br(ai)for all pure actions ai, •anti-coordination games: ai/∈br(ai)for all pure actions ai, •mixed games: ai∈br(ai)for some ai,anda i/∈br(a i)for some a i, where br(ai)is the set of best replies to action ai. Then the following result holds. Proposition 4 Only anti-coordination games can be strongly informative. Proof See Appendix A.  In the case of 2 ×2 symmetric games, let us denote the two revealing policy functions by σ◦:= (I,II)and σ•:= (II,I). We can then give necessary and sufficient conditions in terms of the utility values (a,b,c,d)for the policy profiles (σ ◦,σ◦)and (σ •,σ•)to be strict equilibria in anti-coordination games, and hence for a game to be strongly informative under full unmeasurable uncertainty. 1044 Dynamic Games and Applications (2025) 15:1037–1069 Theorem 5 Suppose that (a,b,c,d)is an anti-coordination population game and that λ∈ [0,1]. Then the policy profile (σ ◦,σ◦)defines a strict equilibrium if and only if a>d and c −a>b−d. The profile (σ •,σ•)defines a strict equilibrium if and only if a<d and c −a<b−d. Proof See Appendix A.  It is also crucial to note that the profile of policy functions (σ◦,σ◦)and the dual profile (σ•,σ•)cannot be both strict equilibria of the same game. Corollary 6 At most one of the two profiles (σ ◦,σ◦)and (σ •,σ•)can be a strict equilibrium. Proof See Appendix A.  3.2 Strong Informativity with Partial Uncertainty In this section, we find necessary and sufficient conditions for (σ◦,σ◦)and (σ •,σ•)to be strict equilibria in situations where the agents’ uncertainty is not described by the full unit interval [0,1], but by some non-empty subinterval [λ, λ]⊆[0,1]. We derive our results by showing that a game (a,b,c,d)played in this state of partial uncertainty is equivalent to an “inner game” played in a state of full uncertainty. For brevity, we write (a,b,c,d), [λ, λ] for the game with game matrix (a,b,c,d)played in the information state described by the uncertainty interval [λ, λ]. Notice that each game (a,b,c,d)and uncertainty interval [λ, λ], together with a policy profile (σi)i∈{1,2}, define an inner game as follows. Definition 7 For (a,b,c,d), [λ, λ]and policy function σ3−i:{M,R}→{I,II},the corresponding inner game (a,b,c,d)for player iis given by a=Eλ[ui|I,M,σ 3−i]=Eλ[ui|I,R,σ 3−i] b=Eλ[ui|I,M,σ 3−i]=Eλ[ui|I,R,σ 3−i] c=Eλ[ui|II,M,σ 3−i]=Eλ[ui|II,R,σ 3−i] d=Eλ[ui|II,M,σ 3−i]=Eλ[ui|II,R,σ 3−i] Since we are interested in games where players reveal their decision criterion, the relevant inner games are those given by the revealing policy σ◦=(I,II)and by the revealing policy σ•=(II,I).For(a,b,c,d), [λ, λ]and revealing policy σ◦the corresponding inner game (a◦,b◦,c◦,d◦)is thus defined by a◦=a+λ(b−a) b◦=a+λ(b−a) c◦=c+λ(d−c) d◦=c+λ(d−c) Similarly, the inner game (a•,b•,c•,d•)corresponding to σ•=(II,I)is defined by a•=b+λ(a−b) b•=b+λ(a−b) c•=d+λ(c−d) d•=d+λ(c−d) Dynamic Games and Applications (2025) 15:1037–1069 1051 Fig. 3 A map of the four regions of (X,Y)-space on which the conditional probability density function q(x,y) is nonzero. For y<0ory>1, q(x,y)=0 We can find the probability density function of (X,Y)by integrating out Vand Zfrom this density function, using the ranges computed above: q(x,y|A)=T(A) q(x,y,v,z|A)dzdv =v=∞ v=v∗(x,y)z=z∗(x,y,v) z=0 q(x,y,v,z|A)dzdv. The value of the upper bound z∗depends on whether x≤yor x>y, while the lower bound v∗depends on whether x+y≥1orx+y<1. Hence, this double integral can be split up into four subintegrals covering each of these four cases. We compute each of these four double integrals separately, finding q(x,y|A)= ⎧ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎩ 1/(3x3)x>y,x+y≥1 1/(3(1−x)3)x≤y,x+y<1 1/(3y3)x>y,x+y<1 1/(3(1−y)3)x≤y,x+y≥1 0y<0ory>1 Equivalently, this density function can be written as q(x,y|A)=1 31 1 /2+max{|x−1 /2|,|y−1 /2|}3 (8) where (x,y)lies on the horizontal band defined by 0 <y<1. Note that the four different cases are separated by the diagonals of the unit square, as shown in Fig. 3. 5.2 The Probability of Informativity in the Uniform Case Having now found the exact conditional distribution of (X,Y), we can approximate the conditional probability that the informative policy profile (σ◦,σ◦)is the sole symmetric equilibrium of the game. As observed in Sect.3.3, this event coincides with the set ϕ◦=x<0,λ+λ 2<y<λ∪0≤x<1 2,λ+λ 2<y<λ−(λ −λ)x.(9) As previously discussed, a similar set ϕ•exists for the profile (σ•,σ•). We can then compute the conditional probability of ϕ◦given that (a,b,c,d)is an anti-coordination game as two 1052 Dynamic Games and Applications (2025) 15:1037–1069 separate integrals: P(ϕ◦|A)=x=0 x=−∞ y=λ y=λ+λ 2 q(x,y|A)dydx +x=1/2 x=0y=λ−(λ−λ)x y=λ+λ 2 q(x,y|A)dydx. On each of these two regions, qbehaves differently. Since 0 <y<1, when x≤0 we can reduce the density expression q(x,y|A)in Eq. 8to q(x,y|A)=1 31 1−x3 . The first term of P(ϕ◦|A)thus has the value Px<0,λ+λ 2<y<λ A=x=0 x=−∞ y=λ y=λ+λ 2 q(x,y|A)dydx =λ−λ 12 . For 0 ≤x<1/2, we simplify computations by means of the sandwich bounds 1 3≤q(x,y|A)≤1 31 1−x3 . By integrating all three components, we find that λ−λ 24 ≤P0≤x<1 2,λ+λ 2<y<λ−(λ −λ)x A≤λ−λ 12 . Adding up the two terms, we thus have the conditional probability λ−λ 8≤P(ϕ◦|A)≤λ−λ 6.(10) Since P(A)=1/4, this entails that the unconditional probability satisfies λ−λ 32 ≤P(ϕ◦)≤λ−λ 24 .(11) Note that this agrees with the result in Sect.4.1 for the case of λ−λ=1. Finally, the total probability of sampling an informative game is P(ϕ) =P(ϕ◦∪ϕ•)=P(ϕ◦)+P(ϕ•)=2P(ϕ◦)(12) since the sets ϕ◦and ϕ•are symmetric and disjoint. 5.3 The Asymptotic Speed of Learning In the previous subsection, we assumed that the uncertainty interval was fixed and arbitrary and computed some bounds on the probability that a randomly generated game is strongly informative. In this section, we will assume such randomly generated games are played repeatedly between randomly selected pairs of players from a mixed population of maximinimizers and regret minimizers, and that the behavior of the selected players is visible to all members of the population. Dynamic Games and Applications (2025) 15:1037–1069 1053 Under some reasonable assumptions about the learning rule employed by the members of the population, this causes the probability of observing a new strongly informative game to tend to zero. However, as we also will argue, this convergence is slow enough to allow the members of the population to exactly determine the type proportions in the limit. Stipulation of learning rule Every time two randomly selected players are plucked from the population and play a strongly informative game, they both reveal their type. Upon observing a sequence of strongly informative games, one can therefore count the number of times the players revealed themselves as maximinimizers or regret minimizers and use this tally to estimate the probability that a randomly drawn player will be of a certain type. Suppose that kis the number of times the players have revealed themselves to be regret minimizers, and that mis the total number of strongly informative games played. For reasons that we will justify below, we assume that any rational observer of this sequence of events forms the belief that λ, the true proportion of regret minimizers, lies within an uncertainty interval with bounds λm=k m−C √m λm=k m+C √m where C>0 is a fixed but arbitrary constant independent of m. In other words, our assumption is that the width of the uncertainty interval is λm−λm=2C √m(13) after the observation of mstrongly informative games. The waiting-time distribution Between two strongly informative games, a number of noninformative games is played. After the first mstrongly informative games have been played, one needs to play a certain number of games before the next strongly informative game appears. Since the games themselves are generated randomly, this waiting time m≥1is itself a random variable. As we have seen in the previous section, the probability that a specific random game is strongly informative depends on the width of the uncertainty interval, but we have assumed that the uncertainty interval remains unchanged as long as no new information is revealed. The process of waiting for the next strongly informative game can therefore be modeled as the process of flipping a bent coin with a fixed success parameter pmuntil it comes up heads. This means that the random variable mfollows a geometric distribution with parameter pm. This distribution has an expected value of E[m]= 1 pm and a variance of (1−pm)/p2 m≤1/p2 m. Bounds on the waiting time As we have seen above, the probability pmdepends on the width of the uncertainty interval and satisfies λ−λ 16 ≤pm≤λ−λ 12 .(14) With the assumptions above, this is equivalent to C 8√m≤pm≤C 6√m.(15) 1054 Dynamic Games and Applications (2025) 15:1037–1069 As we have seen, the expected waiting time before the (m+1)th strongly informative game arrives is E[m]=1/pm. We thus have 6√m C≤E[m]≤8√m C(16) Note also that Var[m]≤1/pm≤64m/C. Total waiting time Since the waiting time from the mth to the (m+1)th strongly informative game is m, the waiting time until a total of mstrongly informative games have been observed is a random variable T=1+1+···+m By the linearity of expectations, m  i=1 6√i C≤E[T]≤ m  i=1 8√i C which can be weakened to 3 Cm3/2≤E[T]≤8 Cm3/2. Since the waiting times 1, 2,..., mare independent, we also have Var[T]≤ m  i=1 64i C≤64 Cm2. This variance grows quadratically in m,so(T−E[T])2will tend to be larger when mis large. In relative terms, however, we have Var T−E[T] E[T]=EVar[T] E[T]2≤64m2/C (C/(3m3/2))2=64C 9m, which goes to zero as m→∞. Hence T/E[T]will converge to 1 as m→∞by, for instance, Chebyshev’s inequality. Limit-learnability We can now briefly summarize in informal terms what we have seen so far. We have shown that the total number of games required in order to observe mstrongly informative games is on the order of T∼m3/2. Conversely, when one has observed a total of Tgames, one can expect the number of strongly informative games among them to be on the order of m∼T2/3. Since we have assumed that the width of the uncertainty interval is inversely proportional to √m, it will be on the order of λm−λm∼(T2/3)−1/2=T−1/3(17) These results obtain because we have assumed that the agents in the population update their beliefs when new information is revealed, so that the waiting time before the next informative event gradually increases. By contrast, if the agents did not update their beliefs, then the waiting time between strongly informative games would remain constant over time, and mand Twould be of the same order of magnitude. Since T−1/3→0fort→∞, it follows that the players will ultimately learn the proportions of the two agent types in the sense that λm−λm→0 Dynamic Games and Applications (2025) 15:1037–1069 1055 for m→∞. This is a somewhat surprising result since it is also the case that pm→0 for m→∞, so that the strongly informative games become more infrequent as the agents become more informed. Learning rule justification Going back to the stipulated learning rule, we still have to give a justification for the assumption that the width of the uncertainty interval is proportional to 1/√mafter the observation of mstrongly informative games. Our argument for this choice has a positive and a negative aspect. The positive part of the argument shows that it is possible to estimate the parameter of a Bernoulli distribution from mobservations with an expected errorof1/√m, whereas the negative part shows that no substantially lower error is possible. The positive part of our argument consists of the law of large numbers as formulated, for instance, by Chebyshev: Theorem 17 Let X1,X2,X3,...be a series of independent and identically distributed random variables with a shared mean E[Xi]=λand a finite variance, and let ˆ λm=(X1+···+Xm)/m be the empirical average of the first m observations. Then for any α>0there is a constant δ>0such that for all m >0, P(|λ−ˆ λm|<δ/ √m)≥1−α. This theorem ensures that the mean of a distribution is estimated by the empirical mean of a sample with an accuracy proportional to 1/√m. A proof can be found in [15], ch. IX and X. The negative part of our argument relies on much more recent results about the limits on the speed of learning. For the purposes of stating this theorem, let λ1,λ 2∈(0,1)be the parameters of two coin flipping distributions, and let mobservations be drawn from one of these two distributions. A hypothesis test is then a function that maps a data set to one of the two parameter values. We say that the two distributions are (m,α)-distinguishable if there is a hypothesis test that returns the correct parameter value with probability 1 −αfor a data set of size m.Wethenhave: Theorem 18 There are δ>0and α>0such that for all m, if two parameters λ1,λ 2∈(0,1) satisfy the proximity condition |λ1−λ2|<δ/ √m, then the corresponding coin flipping distributions are not (m,α)-distinguishable. This theorem allows us to conclude that the confidence bounds provided by Chebyshev’s inequality are the best we can hope for: the width of the confidence interval cannot shrink at a rate faster than 1/√m. This result can be proven by using the Hellinger distance between two binomial distributions to lower-bound the minimax risk for the hypothesis test, as discussed extensively elsewhere [36,37]. Together, these two results show that a confidence interval of the shape λm=ˆ λm−C √m(18) λm=ˆ λm+C √m(19) will contain the true parameter λwith a probability that neither converges to 0 nor to 1 as m→∞.Bycontrast,if |λm−λm| √m→∞ 1056 Dynamic Games and Applications (2025) 15:1037–1069 for m→∞, the probability of error would tend to 0, and if we had |λm−λm| √m→0 for m→∞, the probability of error would tend to 1. This hence justifies our assumption that any rational agent must use confidence intervals of width proportional to 1/√mwhen estimating the parameter of a Bernoulli distribution from mobservations. 6 Strong Informativity for Beta and Dirichlet Distributions In this section, we expand the analysis from the previous section to games with more than two actions and different utility-matrix distributions. We empirically estimate the probability of encountering a strongly informative game both in the case where each cell in the utility matrix is sampled independently from the beta distribution and in the case where the entire utility matrix is a sample from a Dirichlet distribution (and hence utility values are no longer independent). The beta distribution The beta distribution is a distribution over the unit interval parameterized by two parameters αand β. We focus exclusively on the case where the two parameters are identical, α=β=r. In our first set of experiments, we use such beta distributions to sample each value of the utility matrix independently. The probability density functions of the beta distribution for some of these parameters are shown in Fig. 4. As the figure illustrates, the beta distribution is identical to the uniform distribution when α=β=1, while it comes to resemble a Bernoulli distribution for α, β →0 and an increasingly narrow normal distribution as α, β →∞. The Dirichlet distribution The Dirichlet distribution of order Dis a probability distribution over the (D−1)-dimensional probability simplex (i.e., over (D−1)-vectors vwith 0 ≤vd≤1 and dvd=1). The Dirichlet distribution of order Dis parameterized by a vector of D positive parameters α1, ..., αD. We once again focus on the symmetric parameter vectors, αd=rfor all d. Dirichlet distributions with such parameter vectors are equal to the uniform distribution when r=1. They become increasingly concentrated around the corners of the simplex as r→0and increasingly concentrated around the center point of the simplex for r→∞. Beta-distribution experiments In our first set of experiments, we consider symmetric n×n game matrices whose values are drawn from symmetric beta distributions. We can thus empirically estimate the probability that such randomly generated games are strongly informative. Fig. 4 Examples of beta distributions. Left: α=β=0.25. Center: α=β=1. Right: α=β=50 Dynamic Games and Applications (2025) 15:1037–1069 1057 Fig. 5 Frequencies of strongly informative games for independently beta-distributed utility values in the case of maximum uncertainty [λ, λ]=[0,1]on the left, and in the case of reduced uncertainty [λ, λ]=[0.6,0.8] on the right We set the number of actions equal to n=2,3,5,7,11 and the two (identical) beta parameters equal to r=0.25,0.5,0.75,1,2,3,4,5,6,8,10,20,30,40,50. We additionally try two different uncertainty intervals, [λ, λ]=[0,1]and [λ, λ]=[0.6,0.8]. Figure5tabulates the estimated probability that a random game will be informative for every possible combination of these parameter choices. The estimated probabilities are based on a sample of 105games. A few observations are in order. First, the observed frequency of strongly informative games for the case of maximum uncertainty [λ, λ]=[0,1]with two actions and beta parameters α=β=1 is precisely 0.08296, which is within the theoretical bounds found in Sect.5,0.0625 ≤0.08296 ≤0.0833. This is also true for the case of reduced uncertainty [λ, λ]=[0.6,0.8], where we have 0.0125 ≤0.0159 ≤0.0166. Second, both the case of maximum uncertainty and the case of reduced uncertainty display a similar pattern, with lower frequency of strongly informative games in the bottom left corner, as the number of actions increases and the beta parameters decrease. Overall, 2 ×2 games turn out to be the most informative for all the chosen parameters of the beta distribution. Figure6illustrates the effect of shrinking the uncertainty interval in two cases, one in which the number of actions is held fixed at n=2, and one in which the beta parameters are held fixed at α=β=1. As expected, both tables show that the probability of encountering a strongly informative game tends to 0 as the uncertainty interval shrinks. Perhaps more interestingly, the right-hand inset shows that the probability of encountering a strongly informative game also depends negatively on the number of actions available to the players. Dirichlet-distribution experiments We now turn to the case of Dirichlet-distributed utility values. As mentioned above, we now sample the entire n×nsymmetric game matrix from a Dirichlet distribution of order D=n2with identical parameters α1=···=αD=r.We again choose the following number of actions n=2,3,5,7,11 1058 Dynamic Games and Applications (2025) 15:1037–1069 Fig. 6 Frequencies of strongly informative games during learning. On the left: frequencies for different parameters of the beta distribution during learning in 2 ×2 games. On the right: frequencies for different numbers of actions during learning for beta parameters α=β=1 Fig. 7 Frequencies of strongly informative games for Dirichlet-distributed utility values in the case of maximum uncertainty [λ, λ]=[0,1]on the left, and in the case of reduced uncertainty [λ, λ]=[0.6,0.8]on the right in all combinations with the parameter values r=0.25,0.5,0.75,1,2,3,4,5,6,8,10,20,30,40,50 and in all combinations with two choices of uncertainty interval. The results are shown in Fig.7. There are some noticeable differences between Figs. 5and 7. First, the highest frequencies of strongly informative games are roughly twice as high when utility values are Dirichletdistributed as when utility values are beta-distributed. This holds both in the case of maximum uncertainty (0.15 vs 0.08) and in the case of reduced uncertainty (0.016 vs 0.025). Second, the frequency of strongly informative games for 2 ×2 games is nearly independent of the distribution parameters when the utility values are sampled independently from the beta distribution, whereas it depends heavily on the distribution parameters when the game matrix is sampled in its entirety from a Dirichlet distribution. Figure8instead shows the frequencies of strongly informative games as the uncertainty interval shrinks. Except for the differences just observed, the two graphs of Fig.8look similar to those of Fig.6. Learning dynamics The probability of encountering a strongly informative games decreases as the uncertainty interval shrinks. As we have seen in Sect.5, this has the consequence that Dynamic Games and Applications (2025) 15:1037–1069 1059 Fig. 8 Frequencies of strongly informative games during learning. On the left: frequencies for different parameters of the Dirichlet distribution during learning for two actions. On the right: frequencies for different numbers of actions during learning for Dirichlet parameters (α1, ..., αD)=(1, ..., 1) Fig. 9 Cumulative number of strongly informative games (y-axis) over 10000 games randomly generated by drawing i.i.d. utility values from a symmetric beta distribution (x-axis). The number of games between two subsequent steps in the curves corresponds to the waiting time between a strongly informative game and the next strongly informative game the total number of strongly informative games over time grows slower than linearly when the agents learn as they play. Figures9and 10 illustrate this effect by plotting the cumulative number of strongly informative games under different distributional assumptions. All the curves plotted are concave, illustrating the fact that the waiting time before the next strongly informative game increases as the total number of strongly informative games increases. In particular, the growth rate for the 2 ×2 games with uniformly distributed utility values (α=β=1) is consistent with a growth rate of T2/3strongly informative games after a total of Tgames have been played (Fig. 9, top right inset). 1060 Dynamic Games and Applications (2025) 15:1037–1069 Fig. 10 Cumulative number of strongly informative games (y-axis) over 10000 games randomly generated by drawing i.i.d. matrices of utility values from a Dirichlet distribution (x-axis). The number of games between two subsequent steps in the curves corresponds to the waiting time between a strongly informative game and the next strongly informative game Figure9also shows that random 2×2 games are more likely to be strongly informative than games with n>2 when the utilities are drawn independently from a beta distribution. This contrast is strongest when αand βare closer to 0 and is barely detectable when α=β=10. As Fig.10 shows, random 2 ×2 games are also more likely to be strongly informative in the Dirichlet-distributed case when the parameters α1=···=αDare larger. When the parameters α1=···=αDare close to 0 instead, the informativity with n=2 is lower than the informativity with n>2. 7 Discussion One of the most crucial steps in the development of modern psychology was the realization that an exclusive focus on expressed behavior had started to weigh down the discipline: in order to explain the behaviors, the concept of a private and unobservable mental process would have to be reintroduced. Recent works in biology and ethology have started to suggest that the same might hold for animal behavior in general [14,21,23]. In evolutionary game theory, ideas of this kind have given rise to studies on the evolution of preferences, which elevate the level of explanation from expressed behaviors to subjective utilities (e.g., [1,9, 11]). The present paper too can be read as an attempt to explain observable behaviors in terms of more general processes, in this case different rules for decision making under uncertainty. Rather than considering various decision criteria as competing philosophical theories, they can be interpreted as high-level strategies that may coexist or play off against each other. Dynamic Games and Applications (2025) 15:1037–1069 1067 A.2.3 Proof of Theorem 15 Proof Fix [λ, λ]with λ < λ. By Proposition 14, we can find a non-empty hypercube (L◦,H◦)4where the distribution of (a◦,b◦,c◦,d◦)is absolutely continuous. Hence, by repeating the argument given in Proposition 12 within the smaller hypercube (L◦,H◦)4we get P(L◦<a◦<H◦)>0 P(L◦<d◦<a◦|a◦)>0 P(a◦<c◦<H◦|a◦)>0 P(d◦<b◦<d◦+c◦−a◦|a◦,c◦,d◦)>0 By Theorem 9, this ensures the positive probability of the profile (σ◦,σ◦)being the sole symmetric equilibrium for [λ, λ].Thecaseof(a•,b•,c•,d•)is analogous.  A.2.4 Proof of Theorem 16 Proof Fix [λ, λ]with λ= λ. By Theorem 15, there is positive probability of sampling u1,1,u1,2,u2,1,u2,2that satisfy the anti-coordination inequalities u1,1<u2,1and u2,2< u1,2, and the inequalities u◦ 1,1>u◦ 2,2,u◦ 2,1−u◦ 1,1>u◦ 1,2−u◦ 2,2,and λ<λ ◦<λ, where u◦ 1,1,u◦ 1,2,u◦ 2,1,u◦ 2,2are the equivalent of a◦,b◦,c◦,d◦for u1,1,u1,2,u2,1,u2,2= a,b,c,d. Absolute continuity implies that n×ngames where ui,j<min{u1,j,u2,j}for 2 <i≤nand 1 ≤j≤n have positive probability too. In such games, all actions different from Iand II turn out to be strictly dominated and will not be chosen by either player type in any equilibrium. The only possible equilibria are thus profiles where only actions Iand II are chosen. But then notice that from u◦ 1,1>u◦ 2,2,u◦ 2,1−u◦ 1,1>u◦ 1,2−u◦ 2,2,and λ<λ ◦<λ, it follows that the only possible equilibrium is when type Mplays Iand tyspe Rplays II. Hence, the only equilibrium of the n×ngame is (σ ◦,σ◦), which proves that strongly informative n×ngames have positive probability. The argument for (σ •,σ•)is analogous.  Author Contributions The two authors contributed equally to the development of the results and to the writing of the paper. Funding Open Access funding enabled and organized by Projekt DEAL. Data Availability The Python script that generated the results of Sect. 6is available here. Declarations Conflict of interest There are no conflict of interest to be disclosed. Ethical approval Not applicable. 1068 Dynamic Games and Applications (2025) 15:1037–1069 Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. References 1. Alger I, Weibull JW (2013) Homo moralis: preference evolution under incomplete information and assortative matching. Econometrica 81(6):2269–2302 2. Azrieli Y, Teper R (2011) Uncertainty aversion and equilibrium existence in games with incomplete information. Games Econ Behav 73(2):310–317 3. Battigalli P, Cerreia-Vioglio S, Maccheroni F, Marinacci M (2015) Self-confirming equilibrium and model uncertainty. Am Econ Rev 105(2):646–677 4. Bednar J, Page S (2007) Can game(s) theory explain culture?: The emergence of cultural behavior within multiple games. Ration Soc 19(1):65–97 5. Blau RA (1974) Random-payoff two-person zero-sum games. Oper Res 22(6):1243–1251 6. Cassidy RG, Field CA, Kirby MJL (1972) Solution of a satisficing model for random payoff games. Manag Sci 19(3):266–271 7. Charnes A, Kirby MJL, Raike WM (1968) Zero-zero chance-constrained games. Theory Probab Its Appl 13(4):628–646 8. Cheng J, Leung J, Lisser A (2016) Random-payoff two-person zero-sum game with joint chance constraints. Eur J Oper Res 252(1):213–219 9. Dekel E, Ely JC, Ylankaya O (2007) Evolution of preferences. Rev Econ Stud 74(3):685–704 10. Di Tillio A (2008) Subjective expected utility in games. Theor Econ 3(3):287–323 11. Ely JC, Ylankaya O (2001) Nash equilibrium and the evolution of preferences. J Econ Theory 97:255–272 12. Epstein L (1997) Preference, rationalizability and equilibrium. J Econ Theory 73(1):1–29 13. Epstein L, Wang T (1996) “Beliefs about Beliefs” without probabilities. Econometrica 64(6):1343–73 14. Fawcett T, Hamblin S, Giraldeau L-A (2012) Exposing the behavioral gambit: the evolution of learning and decision rules. Behav Ecol 24:2–11 15. Feller W (1968) An introduction to probability theory and its applications, vol I, 3rd edn. Wiley, New York 16. Galeazzi P, Franke M (2017) Smart representations: rationality and evolution in a richer environment. Philos Sci 84(3):544–573 17. Galeazzi P, Galeazzi A (2021) The ecological rationality of decision criteria. Synthese 198:11241–11264 18. Galeazzi P, Marti J (2023) Choice structures in games. Games Econ Behav 140(C):431–455 19. Gigerenzer G (2008) Why heuristics work. Perspect Psychol Sci 3(1):20–29 20. Gigerenzer G, Goldstein D (1996) Reasoning the fast and frugal way: models of bounded rationality. Psychol Rev 103:650–669 21. Hagen EH, Chater N, Gallistel CR, Houston A, Kacelnik A, Kalenscher T, Nettle D, Oppenheimer D, Stephens DW (2012) 97Decision making: what can evolution do for us? In: Evolution and the mechanisms of decision making. The MIT Press 22. Halpern JY, Pass R (2012) Iterated regret minimization: a new solution concept. Games Econ Behav 74(1):194–207 23. Hammerstein P, Stevens JR (2012) Six reasons for invoking evolution in decision theory. In: Hammerstein P, Stevens JR (eds) Evolution and the mechanisms of decision making. MIT Press, Cambridge 24. Kajii A, Ui T (2005) Incomplete information games with multiple priors. Jpn Econ Rev 56:332–351 25. Klibanoff P (1996) Uncertainty, decision, and normal form games. Manuscript 26. Linhart PB, Radner R (1989) Minimax-regret strategies for bargaining over several variables. J Econ Theory 48(1):152–178 27. Lo KC (1996) Equilibrium in beliefs under uncertainty. J Econ Theory 71(2):443–484 28. Marinacci M (2000) Ambiguous games. Games Econ Behav 31(2):191–219 29. Osborne MJ, Rubinstein A (1994) A course in game theory. MIT Press, Cambridge 30. Renou L, Schlag KH (2010) Minimax regret and strategic uncertainty. J Econ Theory 145(1):264–286 31. Robalino N, Robson A (2016) The evolution of strategic sophistication. Am Econ Rev 106(4):1046–72 Dynamic Games and Applications (2025) 15:1037–1069 1069 32. Rubinstein A (1991) Comments on the interpretation of game theory. Econometrica 59(4):909–24 33. Schlag KH, Zapechelnyuk A (2024) Compromise, donâe™t optimize: generalizing perfect Bayesian equilibrium to allow for ambiguity. J Polit Econ Microecon 2(1):77–128 34. Solan E (2022) A course in stochastic game theory. London mathematical society student texts. Cambridge University Press, Cambridge 35. Song T (1992) On random payoff matrix games. Springer, Boston, pp 291–308 36. Tsybakov AB (2009) Introduction to nonparametric estimation. Springer, New York 37. Yu B (1997) Assouad, Fano, and Le Cam. In: Pollard D, Torgersen E, Yang GL (eds) Festschrift for Lucien Le Cam, chapter 29. Springer, Cham, pp 423–435 38. Zollman KJS (2008) Explaining fairness in complex environments. Polit Philos Econ 7(1):81–97 Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.