Cost of reasoning and strategic Sophistication

Zhao, Wanqun Article Cost of reasoning and strategic Sophistication Games Provided in Cooperation with: MDPI – Multidisciplinary Digital Publishing Institute, Basel Suggested Citation: Zhao, Wanqun (2020) : Cost of reasoning and strategic Sophistication, Games, ISSN 2073-4336, MDPI, Basel, Vol. 11, Iss. 3, pp. 1-27, https://doi.org/10.3390/g11030040 This Version is available at: https://hdl.handle.net/10419/257458 Standard-Nutzungsbedingungen: Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen Zwecken und zum Privatgebrauch gespeichert und kopiert werden. Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich machen, vertreiben oder anderweitig nutzen. Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen (insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten, gelten abweichend von diesen Nutzungsbedingungen die in der dort genannten Lizenz gewährten Nutzungsrechte. Terms of use: Documents in EconStor may be saved and copied for your personal and scholarly purposes. You are not to copy documents for public or commercial purposes, to exhibit the documents publicly, to make them publicly available on the internet, or to distribute or otherwise use the documents in public. If the documents have been made available under an Open Content Licence (especially Creative Commons Licences), you may exercise further usage rights as specified in the indicated licence. https://creativecommons.org/licenses/by/4.0/ games Article Cost of Reasoning and Strategic Sophistication Wanqun Zhao Rady School of Management, University of California, San Diego, CA 92093, USA; [email protected] Received: 30 June 2020; Accepted: 14 September 2020; Published: 22 September 2020   Abstract: I designed an experiment to study the persistence of the prevailing levels of reasoning across games. Instead of directly comparing the k -level(s) of reasoning for each game, I used cognitive load to manipulate the strategic environment by imposing variations on the subject’s cost of reasoning and their firstand second-order beliefs. Subjects have systematic changes in k -level(s) of reasoning across games. That finding suggests that subjects are responsive to changes in the strategic environment. Changes in k -level(s) of reasoning are mostly consistent with the endogenous depth of reasoning model when subjects are more cognitively capable or facing less cognitively capable opponents. Subjects have cognitive bounds, but often choose a lower-type action due to their beliefs about their opponents. Finally, cognitive ability plays a significant role in subjects making strategic adjustments when facing different strategic environments. Keywords: level-kreasoning; guessing game; cognitive load; endogenous depth of reasoning; strategic thinking 1. Introduction The use of the levelk model has prevailed in the literature for characterizing people’s initial responses in laboratory strategic games [ 1 , 2 ]. The model characterizes the player’s systematic deviations from the Nash equilibrium using a bounded rational-type explanation. The level-0 type’s action is assumed to be uniformly distributed over all actions (or in some cases, level-0 type’s action is the most prominent action available), whereas the level-1 type has the best response to the expected action of the level-0 type. The level-2 type has the best response to the expected action of the level-1 type. The iterations follow this pattern, as the levelk type always has the best response to the actions of levelk− 1 type. Such patterns of off-equilibrium play have been evidenced in many laboratory experiments. In Nagel’s p -beauty contest game, Nagel found spikes that correspond to the first and second rounds of iterative best responses [ 1 ]. Stahl and Wilson found similar evidence of level-1 and level-2 types with 10 matrix games [ 2 ]. Camerer et al. developed a cognitive hierarchy model [ 3 ]. Instead of holding a belief that all the other players are type k -1, levelk players in the cognitive hierarchy model assign a probability distribution over all the lower types. Many other studies used the levelk model to explain laboratory data (matrix game [ 4 ]; beauty contest game [ 5 – 8 ]; sequential game [ 9 ]; auction [ 10 , 11 ]; Crawford, Costa-Gomes and Iriberri also provide a comprehensive literature review [12]). However, although the levelk model has proven its usefulness in characterizing initial responses for many laboratory games, its predictive power remains ambiguous because (1) it is often used posteriorly to classify a player’s type given their actions and (2) the model lacks components related to individual characteristics that could help identify different types of players. It is important to understand how certain levels are reached for each individual, as it is a starting point for the discussion of the model’s predictive power. Alaoui and Penta developed a framework called the endogenous depth of reasoning (EDR) model to explain what may happen in a player’s head when they encounter a given strategic situation [ 13 ]. The EDR model captures individual characteristics by introducing cost Games 2020,11, 40; doi:10.3390/g11030040 www.mdpi.com/journal/games Games 2020,11, 40 2 of 27 of reasoning, which is determined both by the strategic environment and by a player’s endogenous cognitive ability. The model includes game-specific characteristics by introducing the benefit of reasoning through payoffs of the games. Lastly, the model allows a clear separation of cognitive bounds and behavioral levels observed in games by introducing higher-order beliefs. Such separation makes room for individual adjustments of k -levels in different strategic environments. As a result, a level-1 action observed from a game does not necessarily classify the player as a level-1 player. Instead, such action can be a product of the player’s cost and benefit analysis and his belief about his opponents. The EDR model provides a plausible starting point to study the persistence of the levelk model. However, as individuals have heterogeneous costs of reasoning and belief systems in all kinds of strategic situations, it is hard to conduct direct comparisons across games to test whether the behavioral k -levels follow the EDR model’s predictions. In this paper, I use Costa-Gomes and Crawford’s two-person guessing games (henceforth CGC06) and cognitive load to create different strategic environments [ 14 ]. By controlling cognitive load, I create a standard for the cost of reasoning for all the subjects. Although individual cognitive ability may still have an effect, by using a within-subject experimental design, the individual effect will no longer impact the comparisons of strategic levels across games for the same subject. The revelation of information about the strategic environment is also carefully manipulated to clearly control the subject’s belief space. The goal was to test whether the EDR model provides directional predictions about the changes on k -levels across games for any given subject. Alaoui and Penta tested the benefit part of their model using the 11–20 money request game with altered bonus rewards [ 13 , 15 ]. To the best of my knowledge, this was the first paper to provide experimental tests of the EDR model by introducing different strategic environments with controlled cost and belief space. With the 18 two-person guessing games in the experiment, the results suggest that the subject’s behavioral levels systematically vary across the games. Subjects are mostly responsive to the changes in the strategic environment. Their directional changes in behavioral levels can be predicted by the EDR model when they are more cognitively capable or their opponent is less cognitively capable. An inherent cognitive bound exists for the subjects in different strategic environments. When comparing a subject’s behavioral levels across all the games while providing the same amount of cognitive resources, their behavioral levels rarely exceed their cognitive bound level for that strategic environment. A few other papers also studied the correlation of individual k -levels with cognitive ability. Allred et al. investigated the effects of cognitive load on strategic sophistication [ 16 ]. In their experiments, they asked the subjects to perform a memorization task of either a threeor nine-digit binary number concurrently with strategic games such as beauty contest, 11–20, and 10 matrix games. They found that subjects with high loads (i.e., nine-digit number) were less capable of computing best responses, especially for the beauty contest game. They were also aware of their strategic disadvantages. The net result of cognitive load depended on the specific strategic context. Burnham et al. used a standard psychometric test to measure the cognitive abilities of their subjects, and correlated the test results with subjects’ performances in a p -beauty contest game [ 17 ]. They found a negative correlation between cognitive test scores and entries in the beauty contest game, indicating that subjects with higher cognitive ability tend to be more strategically sophisticated in such games. Gill and Prowse used a 60-question non-verbal Raven test to assign subjects into highand low-cognitive-ability groups [ 18 ]. They asked the subjects to play a pbeauty contest game for 10 rounds, and found that subjects in the high-cognitive-ability group converged to equilibrium faster. These studies provided some evidence of the correlation of individual k -levels with cognitive ability or carefully controlled cognitive tasks. In my experiment, I used memorization tasks to manipulate the cost of reasoning for the subjects in the context of a two-person guessing game. According to Allred et al., higher cognitive load negatively affects a subject’s ability to calculate the best responses in this type of guessing games [ 16 ]. To attain a higher level of strategic sophistication, players have to exert more effort to combat the effects of cognitive load; therefore, the cost of reasoning increases with cognitive load in this strategic situation. Games 2020,11, 40 3 of 27 Every subject experienced both the low and high cognitive loads at some point during the experiment, so they were fully aware of the additional cost of reasoning that was added by these memorization tasks. As a result, their cost of reasoning and their belief about their opponent’s cost of reasoning can be quantified by the cognitive load. The stability of k -levels is an important aspect in the levelk model literature. Stahl and Wilson used twelve normal-form games to estimate the player’s level [ 19 ]. They found that using a relatively low threshold, 35 out of 48 subjects could be classified as stable across games. Fragiadakis et al. asked the subjects to repeat their decisions in a series of two-person guessing games to subsequently best respond to their past actions [ 20 ]. They found that only 40% of the subjects who were able to replicate the decisions could be classified as a known behavioral type. A few works mentioned the predictive power of strategic sophistication. Arad and Rubinstein used a multidimensional Colonel Blotto game to observe subject’s multidimensional iterative reasoning process [ 21 ]. They found that subjects with a higher level of reasoning in the 11–20 money request game also seem to have more rounds of iterative reasoning in this game. Perhaps the most closely related work to this paper is Georganas, Healy, and Weber’s 2015 paper [ 22 ]. They conducted an experiment to examine the cross-game stability of the k -levels. They used four matrix undercutting games and six two-person guessing games and compared them at the individual level. They found no correlation between the levels of reasoning across games. However, they found some evidence of cross-game stability within the class of undercutting game. I studied a similar question to the cross-game stability of the levelk model. Instead of introducing a second family of games, I used cognitive load to mimic different strategic environments, and restricted the subjects to fixed pairs while playing the games. The belief space was therefore carefully controlled, and the uncertainty from playing against a new random player for each round was completely eliminated. The data suggested that systematic level changes can be predicted by the EDR model under certain conditions. In Section 2, I provide a brief introduction to the EDR model to cover some necessary background and theoretical predictions. In Section 3, the experimental design is introduced in detail. Sections 4and 5 cover the data analysis procedure and the discussion of the results, respectively. Section 6provides the concluding remarks. 2. Theoretical Consideration 2.1. Model I adopted Alaoui and Penta’s EDR model for theoretical predictions [ 13 ]. In this model, players follow an endogenous reasoning process that determines the strategic bound in a particular context. With added structure on beliefs, the model is able to predict a player’s actual level of play in any game that could use a k -level iterative best response reasoning process. The main benefit of using this model is that the structure of the model allowed me to conduct a comparative statics exercise on a player’s reasoning process. One of the main goals of this study was to conduct a comparative static exercise on the cost side. Below, I provide more detailed descriptions of some key features of this model. These features are relevant to the experimental design and predictions for this paper. A player’s cognitive bound is a mapping from the incremental cost of reasoning ( c(k) ) and the incremental value of reasoning (v(k)) at each level to the intersection of the two terms. κ(v,c) = min{k∈N|v(k)≥c(k)and v(k+1)<c(k+1)}(1) A player reaches their cognitive bound at the k th level by having a value of reasoning for that level exceeds cost of reasoning, but their cost–benefit analysis no longer supports the one-higher level (i.e., k+1) of reasoning. Further denote the cognitive bound of player ias ¯ ki, where: ¯ ki=κ(vi,ci). (2) Games 2020,11, 40 4 of 27 According to Alaoui and Penta, the value of reasoning is affected by the payoff of the game [ 13 ]. The cost of reasoning is an endogenous characteristic of an individual, which is largely related to their cognitive or reasoning ability. In this paper, I take their assumption on the value of reasoning and continue to assume that the payoff is the only incentive for players to apply logical reasoning in the games. I provide a further discussion on the cost of reasoning. Beyond an individual’s endogenous ability, the strategic environment (such as cognitive load) provides many challenges for a person in applying strategic reasoning, which alters the cost of reasoning. A player’s belief is represented as a tuple. Since the game in my design is symmetric in payoffs, a player’s belief can be restricted to the beliefs about the cost of reasoning. Therefore, the first element of the tuple, ci , represents player i ’s own cost of reasoning. The second element is player i ’s beliefs of his opponent’s (player j ) cost of reasoning, denoted as ci j . The last element cij i is player i ’s second-order belief, which is their belief about player j ’s belief of themselves. Any higher-order beliefs could be nested to the firstand second-order beliefs; therefore, a player’s belief is represented as: ti= (ci,ci j,cij i). (3) 2.2. Theoretical Predictions I formulated the testable predictions following the EDR construction discussed in Section 2.1. For any game G={Xi , ui}i=1,2 , let ki(xi) be the reflected behavioral level of player i by choosing action xi, where Xiis the set of actions available for player iand uiis the payoff function for player i. 1. Changing the cost of reasoning : For any ci j and cij i , ki(xi) (weakly) decreases with ci . Fixing player i ’s firstand second-order beliefs, their cognitive bound weakly decreases with the cost of reasoning. The observed level of player i from the game will also weakly decrease. In my design, for the first 16 games holding cognitive load and information structure constant for the opponent, players will display lower strategic levels when the memorization task is a string of seven letters. 2. Changing the opponent’s cost of reasoning : For any ci and cij i , ki(xi) (weakly) decreases with ci j . If cij i=ci , then player i ’s cognitive bound is binding if they regard their opponent as more sophisticated. Player i reacts to the change in the cost of reasoning of their opponents. More specifically, if he observes his opponent’s cost of reasoning increasing, he will adjust their strategy in the game to best respond to his opponent. That means they may choose to take an action that corresponds to a lower level of strategic sophistication. However, such adjustments of strategies are binding by the cognitive bound when the player believes their opponent has a lower cost of reasoning compared to their own cost. In the context of my experiment, a player should choose a weakly lower level of strategy if he observes his opponent’s memorization task becoming more difficult (i.e., from a string of three letters to a string of seven letters). 3. Changing the second-order belief : For any ci and ci j , ki(xi) (weakly) decreases with cij i . If ci≥cij i , then player i ’s cognitive bound is binding. By fixing player i ’s own cost of reasoning and his opponent’s cost, through only changing player i0 s second-order belief, player i should adjust their strategic actions. For example, when a player has a low cost of reasoning in the game, if they believe that their opponent has a wrong belief about themselves, namely, they believe that their opponent thinks the cost of reasoning for them is very high, then they can switch to an action that is associated with a lower level of reasoning. However, this adjustment of strategic actions according to the second-order belief is restricted by player i ’s own cognitive bound, meaning that they cannot make any adjustments that requires a higher level of reasoning than their cognitive bound. In the context of my experimental design, players should adjust their actions when the information structure shifts from full revelation of cognitive load to partial revelation. Games 2020,11, 40 5 of 27 4. Cognitive bound : Given ci , for any ci j and cij i , ki(xi) never exceeds ¯ ki . When fixing player i ’s own cost of reasoning, their behavioral level should never exceed their cognitive bound. In the context of this experiment, on an individual level, actions observed in games 17 and 18 should correspond to the highest level of reasoning that one player can achieve under the respective cognitive load. 3. Experimental Design In this section, I present the details of the experimental design. The experiment captured the process of levelk thinking through the two-person guessing game [ 14 ]. I provide a brief introduction to the game first, followed by the treatment design and the experimental timeline. 3.1. The Game The two-person guessing game is an asymmetric, two-player game. Each player has a lower limit, ai> 0, an upper limit, bi> 0, and a target pi∈( 0, 2 ) . Players are required to input a guess that is within their lower and upper limit. However, their actual choice is not restricted by the limit. Denote player i ’s input by xi . If a player guesses a number xi that falls outside the limit interval, then their guess will be adjusted to the closest bound. For example, if xi<ai , then the adjusted guess yi will be yi=ai . If xi>bi , then the adjusted guess yi is yi=bi . However, any guess falling within the limit interval will not be adjusted; i.e., yi=xi. The goal of the game is to make a guess that minimizes the difference between the player’s own guess and the product of their target and his opponent’s guess. Denote the difference by ei=|yi−pi·yj| . The payoff is a quasi-concave function minimized at zero. Player i receives ui=max{ 0, 200 −ei}+max{ 0, 100 −ei 100 } . Since a player’s guesses that have the same adjusted inputs will yield the same outcome for the subject, I use the adjusted guess yi as a proxy of how players perform in the game. In this game, the level-0 player is assumed to play randomly according to a uniform distribution over the action space. Denote the theoretical predicted guess made by a k -level player as xk i . Given the assumption imposed on the level-0 player’s strategy, level-1 players will best respond to the expected value of level-0 player’s guess, i.e., x1 i=pi·E{yj|yj∈[aj , bj]} . The level-2 player’s strategy will then be x2 i=pi· { 1 (x1 j∈[aj , bj]) ·x1 j+ 1 (x1 j<aj)·aj+ 1 (x1 j>bj)·bj} . The reasoning process follows iterative best responses. It converges to the Nash equilibrium after finite rounds of iterations. In this paper, I adopt 14 two-person guessing games used by CGC06 and 4 two-person guessing games used by Georganas et al. [ 14 , 22 ]. The parameters of each game are given in Table 1. All the players survive at least two rounds of iterative best responses before reaching the equilibrium (as stated in Table 1“steps to eqm” column). Since in CGC06, only a few number of subjects reached level 3 in the reasoning process, the choice of parameters in this paper should be sufficient to identify a player’s strategic levels in the game. 3.2. Cognitive Load Before directed to the guessing game, subjects were required to memorize a string of letters and were told that they need to recall the given string after the guessing game. The string was composed of either three or seven random letters, for example, UMH or WIEZOFH. The subjects were given 15 s to memorize the string; then they were automatically directed to the guessing game. I did not pay the subjects specifically for correct recalls. However, their payments on the guessing game were partially dependent on this memorization task. If the recall for the selected payment round was wrong, they were not paid for that round, and left the experiment with only the participation fee. Said payment scheme incentivized the subjects to memorize the cognitive load correctly, and therefore guaranteed the effects of different cognitive load treatments. Games 2020,11, 40 6 of 27 Table 1. The eighteen two-person guessing games. Game P1’s Limits P2’s Limits Treatment P1’s Role P2’s Role Steps Eqm at # & Target & Target to Eqm Boundary 1 [(100,900); 1.5] [(300,500); 0.7] [LL+] role B role A 5+ - 2 [(300,900); 1.3] [(100,500); 0.7] [HL-] role B role A 5+ lower 3 [(300,900); 1.3] [(300,900); 1.3] [HH+] role B role A 3 upper 4 [(300,900); 0.7] [(100,900); 1.3] [LH+] role A role B 5+ lower 5 [(100,500); 1.5] [(100,500); 0.7] [LH-] role B role A 5+ upper 6 [(100,500); 0.7] [(100,900); 0.5] [HL+] role A role B 5 lower 7 [(100,500); 0.7] [(100,500); 1.5] [LH-] role A role B 5+ - 8 [(300,500); 0.7] [(100,900); 1.5] [LL+] role A role B 5+ upper 9 [(100,500); 0.7] [(300,900); 1.3] [HL-] role A role B 5+ - 10 [(300,500); 0.7] [(100,900); 0.5] [HH-] role B role A 3 lower 11 [(100,500); 1.5] [(100,900); 0.5] [LL-] role B role A 5+ - 12 [(300,900); 1.3] [(300,900); 1.3] [HH+] role A role B 3 upper 13 [(100,900); 1.3] [(300,900); 0.7] [LH+] role B role A 5+ - 14 [(100,900); 0.5] [(300,500); 0.7] [HH-] role A role B 4 - 15 [(100,900); 0.5] [(100,500); 0.7] [HL+] role B role A 4 lower 16 [(100,500); 0.5] [(100,500); 1.5] [LL-] role A role B 5+ lower 17 [(100,900); 1.3] [(100,500); 0.5] L - - 5+ - 18 [(100,900); 1.5] [(100,500); 0.7] H - - 5+ - 3.3. Treatments The experiment consisted of two blocks. In the first block, subjects were assigned into pairs. They played 16 two-person guessing games against each other within the fixed pairs. In the second block, they played two guessing games against the computer. There were a total of 18 two-person guessing games for them to complete for this experiment, and no feedback was given throughout the process. 3.3.1. Against Human I adopted a 2 × 2 × 2 design. For ease of explanation, I specify the two players in the guessing game as having role A and role B in this section. However, subjects were not aware of their role during the experiment. Each subject was given the role of A or B for each treatment exactly once. I used a within-subject design. To examine the effects of changing the cost of thinking on a subject’s level of reasoning, I varied the cognitive load for role A, holding role B’s cognitive load constant. As mentioned in the previous section, role A needed to memorize a string of either three or seven random letters when playing the guessing game. To test the effects of changing the opponent’s cost of thinking on a player’s level of strategic sophistication revealed in the game, I also varied role B’s cognitive load by two levels. Changing the cost of thinking of role B essentially tests the effects of changing the first-order belief for role A. Denote the cognitive load of three letters as low load (L) and seven letters as high load (H). Lastly, I varied the disclosure of information on the cognitive load for role B. The exact cognitive load implemented on role A was either fully revealed to role B or partially revealed as a probability distribution. Denote full revelation as [+] and the counterpart as [-]. In the partial revelation treatment, role B was told that role A has a 0.5 probability of memorizing a string of three letters and a 0.5 probability of memorizing a string of seven letters. The full and partial revelations of the cognitive load information on role B were a method of measuring the effects of changing the second-order belief for role A. In the full revelation treatment, both roles A and B were aware that role A’s memorization task is common knowledge. However, in the partial revelation treatment, role A knew their exact memorization task was hidden to role B; therefore, their second-order belief (i.e., their belief about role B’s belief of their own cost of thinking) may not coincide with their actual cost of reasoning. A summary of treatments is provided in Table 2. In later sections, I used role A’s label to identify the Games 2020,11, 40 7 of 27 treatments, as I was essentially examining the treatment effects for role A only. The first letter in the label indicates role A’s cognitive load (either L or H). The second letter indicates role B’s cognitive load (opponent’s cognitive load, either L or H), and the last element of the label indicates full or partial revelation (role A’s second order belief, either [+] or [-]). Role B served as a supporting role to complete the information required for each treatment. The information presented to role B for each treatment is also presented in Table 2. However, when later discussing the experimental results, I only refer to each treatment using role A’s label. Table 1provides a summary of treatments and assignments of roles for each game. Each subject played as either role A or role B exactly once for each treatment. There are in total 16 games. For each treatment, the pair of games are symmetric in game parameters and cognitive load realizations. The games were played in two random orders (the first order was as game numbers listed in Table 1; the second order was: 2, 13, 14, 4, 3, 1, 16, 6, 11, 8, 12, 5, 10, 15, 7, 9, 18, 17. Since for each game, there were two players assigned with different cognitive loads, considering player 2’s order of play, there were essentially four sequences. The number of subjects in each order was roughly balanced. After dropping subjects with missing data, there were 28 subjects playing the first order as player 1, 29 subjects playing the first order as player 2, and 27 subjects playing the second order as player 1 and player 2 respectively.). Before the start of each session, one of the two was randomly selected. Table 2. The eight treatments. Role Label Cost 1st Order Belief 2nd Order Belief 1Role A [LL+] Low Low Low (full revelation) Role B Low Low Low (full revelation) 2Role A [HL+] High Low High (full revelation) Role B Low High Low (full revelation) 3Role A [LH+] Low High Low (full revelation) Role B High Low High (full revelation) 4Role A [HH+] High High High (full revelation) Role B High High High (full revelation) 5Role A [LL-] Low Low 50% Low, 50% High (partial revelation) Role B Low 50% Low, 50% High Low (full revelation) 6Role A [HL-] High Low 50% Low, 50% High (partial revelation) Role B Low 50% Low, 50% High Low (full revelation) 7Role A [LH-] Low High 50% Low, 50% High (partial revelation) Role B High 50% Low, 50% High High (full revelation) 8Role A [HH-] High High 50% Low, 50% High (partial revelation) Role B High 50% Low, 50% High High (full revelation) 3.3.2. Against Computer Subjects played against the computer for the second block of the experiment. The computer always chooses a Nash equilibrium action. The concept of equilibrium was explained to the subjects. For example, subjects were told that “a combination of guesses, one for each person, such that each person’s guess earns them as many points as possible, given the other person’s guess, is called an equilibrium guess.” A similar description of equilibrium guess is found in CGC06. Subjects were also given an example of an equilibrium guess following this description. However, they were not specifically taught how to derive an equilibrium guess. The reason for introducing the equilibrium concept was to encourage the subjects to perform as many rounds of iterative best responses as possible. The two guessing games in this part are labeled 17 and 18 in Table 1. Games 2020,11, 40 8 of 27 3.4. Experimental Timeline A total of 111 subjects were recruited for this experiment. Sessions were conducted at the Incentive Lab at Rady School of Management, University of California—San Diego ( San Diego, CA, USA ). The experiment was programmed and conducted using z-tree [ 23 ]. The complete session lasted for 90 min. Subjects were given a 5 USD show-up fee for attending the experiment and an additional $5 if they passed the understanding test and completed the experiment. They earned an additional $8 on average depending on their decisions for the guessing games. For those who did not pass the understanding test, they spent about 30 min in this experiment and left with the $5 show-up fee. Subjects were given instructions on the two-person guessing game first. After explaining the rules, I introduced four unincentivized practice rounds. During the practice rounds, subjects played against the computer and were told that the computer will always choose the mean of the target interval. After the subjects made a guess, feedback was provided for the subjects to reflect on the game rule and the payoff rule. An understanding test was then administered. The test was composed of six questions, similar to the understanding test in CGC06. Standard questions included calculatiosn of best responses and payoffs. Although subjects in the experiment were not restricted to following a levelk reasoning process, for the purpose of the experiment, I wanted to make sure the subjects were capable of calculating the best responses. A screenshot of the understanding test is shown in Figure 1. Subjects needed to answer four out of six questions correctly to proceed to the main part of the experiment. Figure 1. Screenshot of the understanding test. Before playing the incentivized guessing games, subjects were introduced to the memorization task. They were given two unincentivized practice rounds for the low load and high load treatments. During the practice round, they had the standard 15 s to memorize the string of letters and were asked for immediate recall when the time was up. They, however, did no get to practice the guessing game with the cognitive load implemented. The main experiment consisted of two parts, as discussed in Section 3.3. There were 18 two-person guessing games in total. For the first 16 games, subjects were randomly assigned into pairs and stayed within the same pair for all 16 decisions (one as player 1 and the other as player 2). For each game, subjects were given the same information set that consisted of the types of memorization task (either string of three or seven letters, or a probability distribution) for themselves and their opponents, whether their opponents knew about their exact memorization task, and the targets and limits for both players. An example of the actual decision screen is provided in Figure 2. Subjects were also asked to Games 2020,11, 40 15 of 27 Table 6. The frequency of levels by game. Game # L1 L2 L3 L4 L5 Nash All Guesses 60.26% 16.32% 9.86% 3.90% 3.40% 6.26% 1 72.97% 13.51% 0 2.70% 4.50% 6.31% 2 35.14% 16.22% 19.82% 15.32% 3.60% 9.91% 3 44.14% 22.52% 33.33% 0 0 0 4 63.06% 14.41% 1.80% 7.21% 13.51% 0 5 72.07% 12.61% 0 1.80% 0 13.51% 6 83.78% 4.50% 2.70% 9.01% 0 0 7 53.15% 12.61% 16.22% 5.41% 6.31% 6.31% 8 62.16% 12.61% 1.00% 0 0 24.32% 9 59.46% 6.31% 18.02% 1.80% 7.21% 7.21% 10 54.05% 45.95% 0 0 0 0 11 69.37% 10.81% 7.21% 1.80% 1.80% 9.01% 12 44.14% 23.42% 32.43% 0 0 0 13 70.27% 16.22% 1.00% 1.00% 5.41% 6.31% 14 73.87% 11.71% 14.41% 0 0 0 15 65.77% 15.32% 11.71% 7.21% 0 0 16 64.86% 16.22% 5.41% 2.70% 10.81% 0 17 34.23% 28.83% 3.60% 10.81% 2.70% 19.92% 18 62.16% 9.91% 9.91% 3.60% 5.41% 9.91% The distributions of the levels were fairly similar to the results in CGC06, except that levels 4 and 5 were then included. Level 1 was the most prominent behavioral level. Of 1998 observations, 60.26% were level 1 guesses. In some games, level 1 was even more frequently observed. For example, in game 1, about 70% of the guesses were classified as level 1. A number of observations were levels 2 and 3 and Nash guesses. In my data, the occurrence of level 3 was more frequent in a few games. For example, in game 2 and game 3, more than 20% of observations were assigned to level 3. Although some observations corresponded to exact level 4 or level 5 guesses, the overall frequency of these two higher levels was much lower. In about one-third of the games, no guesses were classified into these two levels. As shown in Table 6, there are a pair of games that have almost identical level distribution, game 3 and game 12. These two games have identical parameters and treatments (as shown in Table 1). Besides these two games, the frequency of levels in other games differed considerably. In some games, behavioral levels congregated toward levels 1 or 2, for example, games 1 and 6. In some games, such as games 2 and 9, behavioral levels spread out across the six categories. The variations in the distribution of levels across games could be due to the differences in the cognitive load tasks. The exact impact of the memorization tasks is discussed in detail in the following subsections. 5.3. Result 1: Increasing Cost of Reasoning As mentioned in Section 2.2, the first testable prediction involved fixing the subject’s firstand second-order beliefs and examining the effect of the changing cost of reasoning on the subject’s behavioral levels. There were essentially two comparisons in this case: a comparison between treatment [LL-] and treatment [HL-], and between treatment [LH-] and [HH-]. Note that in both comparisons, the cost of reasoning for the subject varied from low to high; therefore, it was crucial to have partial revelation of the subject’s (role A) memorization task. In the partial revelation treatment, role B (the opponent) only knew the probability distribution of the subject’s memorization task (0.5, 0.5; L, H); therefore, even with the subject’s own tasks varying between two treatments, the subject’s second-order belief was controlled to be the same. There were 222 pairs of comparison in total. The summary statistics of the comparisons are presented in Table 7. The plotted distribution of behavioral levels is presented in Figure 5. To aid with the interpretation of the results, the behavioral Games 2020,11, 40 16 of 27 levels in the figure are presented in a reverse order (i.e., higher level on the left and lower level on the right). Table 7. The frequency of changing behavioral levels with increasing cost of reasoning. Pair Name (From Game a to Game b) # of Pairs Treatment Decreases Constant Increases G16 to G9 111 LLto HL20.72% 43.24% 36.03% G7 to G14 111 LHto HH39.64% 49.55% 10.81% Combined 222 L?- to H?- 30.18% 46.40% 23.42% 0" 0.2" 0.4" 0.6" 0.8" 1" Eqm" Level"5" Level"4" Level"3" Level"2" Level"1" [LL-]%TO%[HL-]% [LL-]"Cumula8ve" [HL-]"Cumula8ve" (a) 0" 0.2" 0.4" 0.6" 0.8" 1" Eqm" Level"5" Level"4" Level"3" Level"2" Level"1" [LH-]&TO&[HH-]& [LH-]"Cumula9ve" [HH-]"Cumula9ve" (b) Figure 5. Cumulative level distribution for increasing cost of reasoning. When the opponent’s cognitive load was controlled to be high and with partial revelation, subjects weakly decreased their behavioral levels 89.19% of the time (39.64% strict decrease). In Figure 5b, the [LH-] treatment is first-order stochastic dominant over the [HH-] treatment. The Wilcoxon test (Table 8) was significant at the 1% level for the comparison of the distributions of behavioral levels between these two strategic environments. When regressing the behavioral level on the treatment dummy, the result (Table 9) suggested that the coefficient for treatment dummy was 0.77, which was significant at the 1% level. This implied that the estimated behavioral level weakly decreased when the subject’s own cognitive load increased when facing an opponent with high cognitive load. The finding is consistent with the EDR model. The relatively large proportion (49.55%) of constant levels may seem quite surprising at first look. One possible explanation is that these subjects may have had different cognitive bounds in the two treatments. In the [LH-] treatment, subjects may have adjusted their behavioral levels downward from their cognitive bound in that treatment due to some belief they formed when facing opponents with high cognitive loads. In the [HH-] treatment, subjects who had a lower cognitive bound (as they had a high cognitive load) may have displayed a lower behavioral level. When the two behavioral levels from two treatments coincided, I observed no changes in the behavioral levels in the treatment comparison. The result for the comparison between [LL-] and [HL-] is less clear. As shown in Table 7, 63.97% of the comparisons had weakly decreasing behavioral levels (20.72% strict decrease), and a noticeable percentage (36.03%) of the comparisons had increasing levels. The Wilcoxon test statistic rejected the null hypothesis that the two strategic environments have the same distribution of behavioral levels at the 5% level. However, upon further checking using a one-tail Wilcoxon test, the distribution of behavioral levels shifted rightward when cognitive load changed from low to high when facing an opponent with a low cognitive load. When conducting the standard GLS random effect regression, the coefficient on the treatment dummy was positive and significant at the 10% level. Games 2020,11, 40 17 of 27 Table 8. Test results for equality of distribution. Comparison Group (in Treatment) Wilcoxon p-Values (Two-Tailed) Wilcoxon p-Values (One-Tailed) Changing Cost of Reasoning LLto HL0.05 ∗∗ 0.98 LHto HH0.00 ∗∗∗ 0.00 ∗∗∗ Changing Opponent’s Cost of Reasoning LL+ to LH+ 0.01 ∗∗∗ 0.00 ∗∗∗ HL+ to HH+ 0.00 ∗∗∗ 1 Changing Second Order Belief LL+ to LL0.11 0.05 ∗∗ LH+ to LH0.00 ∗∗∗ 0.99 HLto HL+ 0.00 ∗∗∗ 0.00 ∗∗∗ HHto HH+ 0.00 ∗∗∗ 1 Against Computer (Nash) L to H 0.00 ∗∗∗ 0.00 ∗∗∗ ∗indicates <10% significance, ∗∗ indicates <5% significance, and ∗∗∗ indicates <1% significance. Table 9. Regression results for treatment effects. Comparison Group (in Treatment) Relevant Dummy Constant Number of Obs. Changing Cost of Reasoning LLto HL0.34 ∗(0.18) 1.78 ∗∗∗ (0.14) 222 LHto HH- −0.77 ∗∗∗ (0.16) 2.18 ∗∗∗ (0.12) 222 Changing Opponent’s Cost of Reasoning LL+ to LH+ −0.27 ∗(0.14) 2.03 ∗∗∗ (0.11) 444 HL+ to HH+ 0.33 ∗∗∗ (0.10) 1.55 ∗∗∗ (0.07) 444 Changing Second Order Belief LL+ to LL- −0.25 (0.18) 2.04 ∗∗∗ (0.12) 333 LH+ to LH0.41 ∗∗∗ (0.15) 1.77 ∗∗∗ (0.09) 333 HL+ to HL0.57 ∗∗∗ (0.15) 1.55 ∗∗∗ (0.10) 333 HH+ to HH- −0.48 ∗∗∗ (0.09) 1.89 ∗∗∗ (0.06) 333 ∗ indicates < 10% significance, ∗∗ indicates < 5% significance, and ∗∗∗ indicates < 1% significance. Standard errors in parenthesis. In this analysis, I treated equilibrium level as the highest level, since it requires the subjects to perform multiple steps of iterative best responses. However, since many games have equilibrium at the boundary of the limit interval (games 2, 4, 6, 10, 15, and 16 have the equilibrium at the lower limit; games 3, 5, 8, and 12 have the equilibrium at the upper limit), if the subject chooses an equilibrium action by naïvely playing at the boundary, then this behavioral level should not be considered as a higher level than any of the k levels. This was not the case for this comparison pair. Although game 9 ([HL-]) had 7.21% equilibrium guesses, those guesses were not at the boundary. However, upon further checking of games 16 ([LL-]) and 9 ([HL-]), I found that the level-5 type in game 16 had the same strategy as the equilibrium strategy of that game. Therefore, some of the equilibrium strategies in game 16 were pooled into level-5 type, which may be one possible explanation for the significant positive coefficient on the treatment dummy. Another explanation may be that the subjects felt more motivated to reason at higher strategic levels when they saw the opponents had easier strategic environments (memorizing three letters) as opposed to their own difficult strategic environments (memorizing seven letters). As a result, they displayed higher behavioral levels. This explanation suggests that other factors, such as motivation factor, may also play a role in determining a subject’s behavioral levels. Games 2020,11, 40 18 of 27 5.4. Result 2: Increasing Cost of Reasoning for Opponent To examine the effect of changing the first-order belief on a subject’s behavioral level in games, I selected pairs of games with changing cognitive loads for the opponents. For example, a comparison of behavioral levels for games 1 and 4 served the purpose. In game 1 ([LL+]), player 1 has a low cognitive load when facing an opponent with a low cognitive load, and there is full revelation of each other’s strategic environment. In game 4 ([LH + ]), player 1 has a low cognitive load when facing an opponent with high cognitive load, and again, there is full revelation of the treatments. I found 444 pairs of comparison for the cases wherein the subjects had low cognitive loads, and another 444 pairs of comparison for the cases when they had high cognitive loads. The detailed comparison groups and summary statistics are shown in Table 10. The plotted distribution of behavioral levels is presented in Figure 6. Table 10. The frequency of changing behavioral levels with increasing cost of reasoning for opponent. Pair Name (From Game a to Game b) # of Pairs Treatment Decreases Constant Increases (G1, G8) to (G4, G15) 444 LL+ to LH+ 23.87% 55.86% 20.27% (G6, G13) to (G3, G12) 444 HL+ to HH+ 15.54% 41.44% 43.02% Combined 888 ?L+ to ?H+ 19.71% 48.65% 31.64% 0" 0.2" 0.4" 0.6" 0.8" 1" Eqm" Level"5" Level"4" Level"3" Level"2" Level"1" [LL+]%TO%[LH+]% [LL+]"Cumula8ve" [LH+]"Cumula8ve" (a) 0" 0.2" 0.4" 0.6" 0.8" 1" Eqm" Level"5" Level"4" Level"3" Level"2" Level"1" [HL+]&TO&[HH+]& [HL+]"Cumula9ve" [HH+]"Cumula9ve" (b) Figure 6. Level Distribution for increasing cost of reasoning for opponent. The combined results were the opposite of the theory prediction, with a significant 31.64% of cases of increasing behavioral levels. However, upon further checking, the majority of the increasing cases occurred when subjects are having high cognitive load. When subjects had low cognitive load, 79.73% of the time, they weakly decreased their behavioral levels when their opponents’ cognitive loads changed from low to high (23.87% strict decrease). Figure 6a illustrates that [LL+] games had more guesses at higher levels. This result is consistent with the EDR model. When a subject’s cost and second-order belief was controlled across the two strategic environments, he was responsive to the changes in his opponent’s cost of reasoning. However, some of these adjustments in behavioral levels were not strictly decreasing. If the subject believed that the increased opponent’s cost of reasoning was not large enough to decrease the opponent’s behavioral level by one, the subject’s behavioral level remained the same across the two strategic environments. This partially explains the high percentage (55.86% and 41.44%) of constant behavioral levels in Table 10. When the subject had a high cognitive load and his opponent’s cognitive load changed, the result did not comply with the EDR model. A total of 43.02% of the pairs showed increasing behavioral levels across the two strategic environments. The frequency of levels in Table 6reveals that most subjects had level 1 guesses in games 6 (83.78%) and 13 (70.27%). This gave subjects much less room to adjust their behavioral levels downward compared to another strategic situation. Any behavioral level that was beyond level 1 in games 3 and 12 was considered as moving the behavioral level upward. This was one major limitation in observing the effects of changing the first-order belief when the subject had a high cost of reasoning (i.e., high cognitive load). Games 2020,11, 40 19 of 27 The Wilcoxon signed-rank test rejected the null hypothesis that the level distribution was the same for both treatment comparisons ([LL+] to [LH+] and [HL+] to [HH+]). However, the one-tail test suggested that when the subject had low cognitive load, increasing his opponent’s cost of reasoning shifted the former’s level to the left (to lower levels, significant at the 1% level). However, when the subject had high cognitive load, the level distribution shifted to the right. The regression coefficients suggested that increasing the opponent’s cost of reasoning decreased the behavioral level when the subject had a low cognitive load (significant at the 10% level). 5.5. Result 3: Changing the Second-Order Belief In the experiment, I used a (0.5, 0.5) probability distribution on the revelation of cognitive load treatments to control for the subject’s second-order belief. In the full revelation treatment, role B knew the exact memorization task that was received by role A (the subject), either three (low load) or seven letters (high load) with a probability of one. Therefore, role A’s (the subject) second-order belief was either ((1, 0); (L, H)) or ((0, 1); (L, H)). In the partial revelation treatment, role B knew that the probability of three or seven letters for role A was (0.5, 0.5), which made role A (the subject) have a second-order belief of ((0.5, 0.5); (L, H)). If comparing two games with different second-order beliefs for the subject, with everything else controlled as constant, then a second-order belief of low load with probability of one should be considered as more cognitively capable perceived by role B than a second-order belief of((0.5, 0.5); (L, H). The experiment, as shown in Table 11, supported that most subjects had a clear understanding of their opponent’s cognitive load when the load was explicitly elicited, and they almost had uniform beliefs about their opponents’ cognitive loads when they were in the partial revelation treatment as role B. Table 11. Subject’s belief about his opponent’s cognitive load. Belief Elicitation 3 Letters 7 Letters Not Sure Sum Treatments 3 Letters 591 51 24 666 7 Letters 97 546 23 666 (0.5 L, 0.5 H) 143 111 190 444 In the dataset, I found 888 pairs for comparison that allowed me to examine the effect of changing the second-order belief. I separated them into two groups: a comparison between the full revelation of low load to partial revelation, and a comparison between a partial revelation and a full revelation of high load. Both comparisons were performed in the direction of increasing second-order belief ( i.e., cij iincreases ). The detailed comparison pairs and summary statistics are listed in Table 12. The distribution of behavioral levels is plotted in Figure 7. Table 12. The frequency of changing levels with changing second-order belief. Pair Name (From Game a to Game b) # of Pairs Treatment Decreases Constant Increases Second order belief: Low to (0.5 Low, 0.5 High) (G1, G8) to G16 222 LL+ to LL25.68% 51.35% 22.97% (G4, G15) to G7 222 LH+ to LH20.27% 43.69% 36.04% Combined 444 L?+ to L?- 22.97% 47.52% 29.50% Second order belief: (0.5 Low, 0.5 High) to High G9 to (G6, G13) 222 HLto HL+ 35.14% 52.70% 12.16% G14 to (G3, G12) 222 HHto HH+ 14.86% 45.50% 39.64% Combined 444 H?- to H?+ 25.00% 49.10% 25.90% Games 2020,11, 40 20 of 27 0" 0.2" 0.4" 0.6" 0.8" 1" Eqm" Level"5" Level"4" Level"3" Level"2" Level"1" [LL+]%TO%[LL-]% [LL+]"Cumula8ve" [LL-]"Cumula8ve" (a) 0" 0.2" 0.4" 0.6" 0.8" 1" Eqm" Level"5" Level"4" Level"3" Level"2" Level"1" [LH+]&TO&[LH-]& [LH+]"Cumula9ve" [LH-]"Cumula9ve" (b) 0" 0.2" 0.4" 0.6" 0.8" 1" Eqm" Level"5" Level"4" Level"3" Level"2" Level"1" [HL-]&TO&[HL+]& [HL-]"Cumula9ve" [HL+]"Cumula9ve" (c) 0" 0.2" 0.4" 0.6" 0.8" 1" Eqm" Level"5" Level"4" Level"3" Level"2" Level"1" [HH-]%TO%[HH+]% [HH-]"Cumula9ve" [HH+]"Cumula9ve" (d) Figure 7. Level Distribution for changing second-order belief. The effect of changing the second-order belief was generally weak, except for the cases where the subjects had high cognitive loads when facing opponents with low cognitive loads. For the treatment where both players had low cognitive loads, about 77.03% of the pairs had weakly decreasing behavioral levels when second-order belief changed from full to partial revelation. Among these comparisons, only 25.68% had strictly decreasing levels. This finding suggested that the changes in second-order belief may not have been strong enough for the subjects to adjust their behavioral level downward, even though both subjects had a low cognitive load and were relatively competent at contemplating over the strategic environment. To examine the effect of the second-order belief, it was first necessary to determine the effect of changing the first-order belief for the same group of subjects. In Table 10, the subject’s behavioral responses to the changing opponent’s cognitive load were limited when the subject had a high cognitive load. Now, consider the finding in the [LH+] to [LH-] comparison to the [HH-] to [HH+] comparison (Table 12); changing the second-order belief of the subject effectively changed his opponent’s first-order belief. If the subject holds the belief about his opponent (who has a high cognitive load treatment) that the changes in his opponent’s behavioral level are limited, then the subject should not decrease his behavioral level at all. This partially explains the low frequency of strictly decreasing behavioral levels for subjects who faced opponents with high cognitive loads. The comparison between [HL-] and [HL+] is consistent with the EDR model. In Figure 7c, the [HL-] treatment is first-order stochastic dominant over the [HL+] treatment. Of the guesses, 87.84% had weakly decreasing behavioral levels, with 35.14% having strict decreases. Changing from partial revelation to full revelation of high cognitive load, the second-order belief decreased the subject’s cognitive capability perceived by their opponent. Subjects were responsive to this change in the belief system, and adjusted their behavioral levels downward to best respond to their opponents. Testable prediction 3 suggests that if the subject’s behavioral level is binding by their cognitive bound, then they are not able to make further adjustments according to their changing beliefs. The large percentage of constant levels for these comparisons supported this statement. The Wilcoxon test results showed that the level distribution changed for changing second-order belief. When conducting a one-tailed test, the test result suggested that for [LH+ to LH-] and [HHto HH+] treatments, the distribution of levels significantly (at the 1% level) shifted rightward (increasing behavioral levels). This may have occurred due to the subject’s belief that their opponent with high Games 2020,11, 40 21 of 27 cognitive load will engage in higher behavioral level. This result seems to comply with the results in Section 5.4, but the underlying reasons need further investigation. The regression coefficient on the treatment dummy further supported the results. Since the treatment dummy was coded as zero with full revelation and one with partial revelation, the coefficient of 0.57 for [HL-, HL+] comparison suggested that the behavioral level decreased from partial to full revelation. It was significant at the 1% level. Again, the [LH+, LH-] and [HH-, HH+] comparisons were the opposite direction of model predictions, and they were also highly significant. In general, when the subjects faced opponents with high cognitive loads, they were responsive to changing second-order beliefs, but not in the direction that is predicted by the EDR model. However, when they faced more cognitively capable opponents, then they were mostly responsive to this change in the belief system because they thought their opponents were responsive to this information in their strategic environment. This finding is consistent with the EDR model when the opponent has a low cognitive load, which supports the opposite direction when the opponent is in a less cognitively capable situation. 5.6. Result 4: Cognitive Bound In block 2 of the experiment, the subjects played against the computer. They were told that the computer was playing a Nash equilibrium strategy, and the equilibrium concept was explained. However, they were not taught the method to derive the equilibrium. The behavioral levels from the guesses in these two games should be considered as the highest levels they could achieve under each cognitive load treatment. I selected all the games with the same cognitive load treatment, either low cognitive load or high cognitive load, and pooled the results. A pairwise comparison between the pooled data and behavioral level obtained from games 17 and 18 allowed me to examine the existence of cognitive bounds. There were 888 pairs of comparison for each type of cognitive load, and the summary statistics are shown in Table 13. Table 13. The frequency of changing behavioral levels comparing to cognitive bound. Cognitive Bound # of Pairs Decreases Constant Increases Low cognitive bound (G1, G2, G4, G7, G8, G11, G15, G16) to G17 888 18.36% 33.45% 48.20% High cognitive bound (G3, G5, G6, G9, G10, G12, G13, G14) to G18 888 21.96% 48.20% 29.84% Combined 1776 20.16% 40.82% 39.02% The result for low cognitive load treatment was interesting: 48.20% of the guesses from block 1 games had behavioral levels lower than the subject’s respective cognitive bound (level in game 17). Less than 20% of guesses had higher behavioral levels. This suggested that in many block 1 games, subjects purposely adjusted their behavioral levels downward due to different strategic situations, even though they had reached higher levels. For high cognitive load treatment, about 30% of behavioral levels increased from block 1 games to game 18. However, about 50% of the guesses had the same behavioral level across the two situations. Since high cognitive load inherently restricts the subject’s cognitive ability, there may have been less room for downward adjustments for block 1 games. Due to the large percentage of weakly increasing levels from block 1 to block 2 games, I concluded that cognitive bound existed in most cases. In some situations, cognitive bound was strictly higher than the subject’s behavioral levels in games. In some situations, cognitive bound was the same as the behavioral levels. Such cases were largely observed in the high cognitive load treatment. To examine whether high cognitive load had a lower level distribution, I conducted a Wilcoxon signed-rank test on the estimated behavioral levels of games 17 (low load) and 18 (high load). Table 8 shows that the distributions of levels for the two treatments were significantly different at the 1% level. The one-tailed test indicated that the distribution of low load cognitive bound levels was to the right of Games 2020,11, 40 22 of 27 the distribution of high load cognitive bound levels. This finding indicated that subjects had a higher cognitive bound when receiving low cognitive load treatment (memorizing a string of three letters) compared to receiving a high cognitive load treatment (memorizing a string of seven letters). 5.7. Robustness Check During the guessing games, subjects needed to memorize a string of three or seven letters and recall the letters after they finished the guessing game. In this subsection, I present the results of this memorization task. Although the subjects were fully aware that if they failed to recall all the letters correctly, they would earn zero points for that round of the game, there were still some cases of wrong recalls due to reasons such as lack of attention or being too focused on the guessing game. I wanted to control the experimental results for such cases, as the subjects may have engaged in reasoning at higher levels when cognitive load did not fully apply. Table 14 shows the results of the memorization tasks. Most of the memorization tasks were perfectly performed. Not surprisingly, low cognitive load (three-letter memorization task) had more correct recalls, about 7% more than the high cognitive load task. The difference was significant at the 1% level. Table 14. Results of the memorization task. 3 Letters ∗∗∗ (Low Load) 7 Letters (High Load) Total Correct 97.30% 90.89% 94.09% Wrong 2.70% 9.11% 5.91% # of tasks 999 999 1998 ∗indicates <10% significance, ∗∗ indicates <5% significance, and ∗∗∗ indicates <1% significance. To check whether poor performance of the memorization task affected the treatment results, I excluded the data with wrong recalls and performed the analysis again. The comparison pair was dropped from the sample if either game of the pair had incorrect recalls. This was performed to ensure that the cognitive load was fully in effect, so that high cognitive load added difficulties to thinking through the guessing games at higher levels, and the cost of reasoning was higher. Table 15 presents the treatment results after the robustness check. Treatments that involved high cognitive load had more data points dropped. For example, the [HH-] to [HH+] comparison had 444 pairs of comparison in the original sample. After robustness check, about 100 pairs were dropped. However, the results did not change much compared to the results presented in results 1 to 3 (Sections 5.3–5.5). The changes were mostly within 1%. I can therefore safely conclude that the original results were robust. The quality of the memorization task (i.e., whether the letters were correctly recalled) was almost independent of the treatment effects. Even in the cases of wrong recalls, the effect of cognitive load still applied to the subjects. Games 2020,11, 40 23 of 27 Table 15. Summary of the robust results with incorrect recalls dropped. Pair Name (From Game a to Game b) # of Pairs Treatment Decreases Constant Increases Increasing Cost of Reasoning G16 to G9 107 LLto HL20.56% 43.93% 35.51% G7 to G14 77 LHto HH40.26% 48.05% 11.69% Combined 184 L?- to H?- 28.80% 45.65% 25.54% Increasing Cost of Reasoning for Opponent (G1, G8) to (G4, G15) 426 LL+ to LH+ 23.71% 56.10% 20.19% (G6, G13) to (G3, G12) 392 HL+ to HH+ 16.33% 42.09% 41.58% Combined 818 ?L+ to ?H+ 20.17% 49.39% 30.44% Second order belief: Low to (0.5 Low, 0.5 High) (G1, G8) to G16 214 LL+ to LL26.17% 50.93% 22.90% (G4, G15) to G7 211 LH+ to LH21.33% 44.08% 34.60% Combined 425 L?+ to L?- 23.76% 47.53% 28.71% Second order belief: (0.5 Low, 0.5 High) to High G9 to (G6, G13) 202 HLto HL+ 35.64% 52.48% 11.88% G14 to (G3, G12) 152 HHto HH+ 15.13% 45.39% 39.47% Combined 354 H?- to H?+ 26.84% 49.44% 23.73% 5.8. Cognitive Tests In this subsection, I examine the results of the Mensa practice test. The test is composed of 10 questions and has a time limit of 10 min. Some subjects finished earlier, but they could never run overtime. Each correct answer is worth 1 point and all the unattempted questions are marked as 0 points. The score distribution of 104 subjects (seven missing) is presented in Table 16. There are a few very low points (2 or 3), and six subjects had scores of 10. Most subjects earned seven or eight points in this test. Table 16. Summary statistics of cognitive test score and the counts of level changes following theory predictions. (1) (2) (3) (4) (5) (6) Test Score Sum.Strict Sum.Weak Cost.Weak 1st.Weak 2nd.Weak Points possible 10 18 18 2 8 8 Max 10 13 18 2 8 8 Min 2 0 7 0 2 1 Median 7 4 13 2 6 6 Mean 7 4 13 1.5 5.5 6 To examine whether there are heterogeneous treatment effects in this experiment due to exogenous cognitive ability, I first determined a measure of the treatment effect. Out of all the results discussed in results 1 to 3 (Sections 5.3–5.5), there are in total 18 pairs of comparison. For each subject, I recorded one for the pair if the level change followed the theory prediction, and zero otherwise. As listed in Table 16, column Sum.Strict includes all the 18 comparisons, and only strict changes of levels are recognized. For example, if the pair game 16–game 9 had level 2 in both games, it is coded zero under Sum.Strict. However, column Sum.Weak allows weak changes; therefore, the above-mentioned scenario is coded as one under this column. The EDR model mostly discusses weak behavioral level changes because, in some cases, the changes in belief system or costs are not big enough to shift a behavioral level downwards by one level (evidenced by a large percentage of constant levels). Due to this reason, I considered “weak” changes, and decomposed them into columns (4) to (6), which cover the three main results. When limited to strict changes, a number of subjects had zero pairs following theory prediction (10 out of 111 subjects), and most subjects had only three or four pairs that had changes that could be predicted by the EDR model. However, when allowing weak changes, seven subjects had all the comparison pairs that were theory-predicted directional level changes, and most subjects Games 2020,11, 40 24 of 27 had about 13 to 14 comparisons that could be predicted by the EDR model. The last three columns in Table 16 present results for each treatment separately. To test whether cognitive ability had any correlation with the treatment effects, I ran a regression after dropping the subjects with missing test scores. The result is presented in Table 17. I used gender, class standing, and major as control variables. This information was collected at the end of the experiment. It appears that the cognitive test score and the female dummy variable were positively correlated with weak changes (at a 5% significance level), and the treatments changing the opponent’s cost of reasoning (changing first-order belief) and changing second-order belief. The results showed some heterogeneous treatment effects in which the more cognitively capable subjects were more responsive to the treatments as predicted by the EDR model, especially in those requiring adjustments in response to the changing strategic environment of their opponents. When the strategic environment changed, these subjects were more likely to actively adjust their actions to gain possible strategic advantages. Table 17. Regression results for cognitive test scores on correct directional changes of behavioral levels in block 1 games. Sum.Strict Sum.Weak Cost.Weak First Order.Weak Second Order.Weak Test Score 0.03 (0.16) 0.36 ∗∗ (0.18) −0.04 (0.04) 0.23 ∗∗ (0.11) 0.17 ∗ (0.10) Gender (F) −0.73 (0.57) 1.43 ∗∗ (0.60) 0.03 (0.13) 0.83 ∗∗ (0.37) 0.57 ∗ (0.34) Class Standing 0.33 (0.24) −0.18 (0.25) −0.01 (0.05) −0.13 (0.15) −0.04 (0.14) Major −0.02 (0.12) 0.00 (0.13) −0.03 (0.03) −0.03 (0.08) 0.01 (0.07) Constant 3.09 ∗ (1.67) 10.04 ∗∗∗ (1.75) 1.76 ∗∗∗ (0.37) 3.94 ∗∗∗ (1.08) 4.34 ∗∗∗ (1.00) # of Obs. 104 104 104 104 104 “Weak” includes constant levels and decreasing levels, while “strict” only includes strictly decreasing levels. ∗ indicates < 10% significance, ∗∗ indicates < 5% significance, and ∗∗∗ indicates < 1% significance. Standard errors in parenthesis. Since the result above suggested that more cognitively capable subjects’ responses to changing strategic environment were more coherent with the EDR model, I separated the subjects into two groups according to cognitive test scores. Subjects with scores of eight or above were labeled as high cognitive subjects (high), and the remainder were labeled as low cognitive subjects (low). Table 18 presents results 1–3 again, separated by the cognitive test scores. As discussed in results 1 to 3 (Sections 5.3–5.5), I found significant asymmetries arising from the different strategic environments. Separating the subjects into two groups according to cognitive test scores allowed a closer examination of the source of the asymmetry. In Table 18, result 2 and result 3.1 highlight the relatively stable performance for the high cognitive subjects. As discussed in Section 5.4, subjects’ responses to their opponents’ changing cost of reasoning depended on their own cost of reasoning. In general, their adjustments in behavioral levels only followed the EDR model when they had a low cost of reasoning. This observation is untrue for the high cognitive subjects, who showed relatively stable performance regardless of their own strategic environment, with about 20% of the comparisons strictly following the EDR model. I observed a slight increase of 10% for those that did not follow the model; however, in general, the performance did not vary considerably. For the low cognitive subjects, the difference was huge. The 27.54% for comparison pairs that strictly followed the model decreased to 12.29%, and, more strikingly, the percentage of pairs that did not follow the model increased from 19.07% to 51.27%. This huge difference showed that the asymmetry found in the previous results was mostly due to these low cognitive subjects. There was a similar observation for result 3.1, where the high cognitive subjects had relatively stable performance regardless of their opponents’ cognitive loads, whereas the low