Estimating spillovers using panel data, with an application to the classroom
Abstract
EconStor is a publication server for scholarly economic literature, provided as a non-commercial public service by the ZBW.
Full text
Arcidiacono, Peter; Foster, Gigi; Goodpaster, Natalie; Kinsler, Josh Article Estimating spillovers using panel data, with an application to the classroom Quantitative Economics Provided in Cooperation with: The Econometric Society Suggested Citation: Arcidiacono, Peter; Foster, Gigi; Goodpaster, Natalie; Kinsler, Josh (2012) : Estimating spillovers using panel data, with an application to the classroom, Quantitative Economics, ISSN 1759-7331, The Econometric Society, New Haven, CT, Vol. 3, Iss. 3, pp. 421-470, https://doi.org/10.3982/QE145 This Version is available at: https://hdl.handle.net/10419/150340 Standard-Nutzungsbedingungen: Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen Zwecken und zum Privatgebrauch gespeichert und kopiert werden. Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich machen, vertreiben oder anderweitig nutzen. Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen (insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten, gelten abweichend von diesen Nutzungsbedingungen die in der dort genannten Lizenz gewährten Nutzungsrechte. Terms of use: Documents in EconStor may be saved and copied for your personal and scholarly purposes. You are not to copy documents for public or commercial purposes, to exhibit the documents publicly, to make them publicly available on the internet, or to distribute or otherwise use the documents in public. If the documents have been made available under an Open Content Licence (especially Creative Commons Licences), you may exercise further usage rights as specified in the indicated licence. https://creativecommons.org/licenses/by-nc/3.0/
Quantitative Economics 3 (2012), 421–470 1759-7331/20120421 Estimating spillovers using panel data, with an application to the classroom Peter Arcidiacono Duke University Gigi Foster University of New South Wales Natalie Goodpaster Analysis Group, Inc. Josh Kinsler University of Rochester Obtaining consistent estimates of spillovers in an educational context is hampered by at least two issues: selection into peer groups and peer effects emanating from unobservable characteristics. We develop an algorithm for estimating spillovers using panel data that addresses both of these problems. The key innovation is to allow the spillover to operate through the fixed effects of a student’s peers. The only data requirements are multiple outcomes per student and heterogeneity in the peer group over time. We first show that the nonlinear least squares estimate of the spillover parameter is consistent and asymptotically normal for a fixed T. We then provide an iterative estimation algorithm that is easy to implement and converges to the nonlinear least squares solution. Using University of Maryland transcript data, we find statistically significant peer effects on course grades, particularly in courses of a collaborative nature. We compare our method with traditional approaches to the estimation of peer effects, and quantify separately the biases associated with selection and spillovers through peer unobservables. Keywords. Panel data, fixed effects, peer effects, education. JEL classification. C18, C23, I21. Peter Arcidiacono: [email protected] Gigi Foster: [email protected] Natalie Goodpaster: [email protected] Josh Kinsler: [email protected] We thank Joe Altonji, Pat Bayer, Jane Cooley, Paul Frijters, John Haltiwanger, Caroline Hoxby, Tom Nechyba, Bruce Sacerdote, seminar participants at Duke University, University of Georgia, Yale University, and participants at the 2004 Labour Econometrics Workshop held at the University of Auckland and the NBER’s Higher Education Workshop for helpful comments, as well as the University of Maryland administrators who have provided us with the data used in this paper. Shakeeb Khan and Nese Yildiz were particularly helpful. Special thanks go to Chris Giordano and Bill Spann of the Maryland Office of Institutional Research and Planning. We also thank Graham Gower for superior research assistance. Copyright ©2012 Peter Arcidiacono, Gigi Foster, Natalie Goodpaster, and Josh Kinsler. Licensed under the Creative Commons Attribution-NonCommercial License 3.0. Available at http://www.qeconomics.org. DOI: 10.3982/QE145
422 Arcidiacono, Foster, Goodpaster, and Kinsler Quantitative Economics 3 (2012) 1. Introduction The question of how peers affect student achievement underlies many debates in applied economics. Peer effects are relevant to the estimation of the impact of affirmative action, school quality, and public school improvement initiatives such as school vouchers, and are central to more immediate concerns such as how best to group students to maximize learning.1However, despite this wide field of potential relevance, the empirical estimation of spillovers—whether in the education context or elsewhere— is not straightforward. There are at least two barriers that must be overcome when estimating spillovers on student achievement.2The first is the selection problem. When individuals choose their peer groups, high-ability3students may sort themselves into peer groups with other high-ability students. With ability only partially observable, positive estimates of peer effects may result even when no peer effects are present because of a positive correlation between the student’s unobserved ability and the observed ability of his peers. Researchers have undertaken a variety of estimation strategies to try to overcome the selection problem,4but significant empirical problems linger, both because researchers only have access to incomplete measures of ability and because peer effects may operate differently when peers are chosen rather than assigned. A second barrier is that spillovers may work in part through characteristics or actions that are not observed by the econometrician. The importance of peer effects may be significantly understated if the primary channel through which they operate is unobserved. Peer effects through unobservables in education have received little attention outside of Altonji, Huang, and Taber (2004) and Graham (2008).5 We introduce a new algorithm for estimating spillovers using panel data that overcomes both these obstacles. Our key innovation is that the peer effects are captured through a linear combination of individual fixed effects. Utilizing fixed effects to capture the impact of peers is well suited to environments where time-varying peer unobservables do not affect individual choices. This can hold when the outcome of interest is a 1Epple, Romano, and Sieg (2003) showed that prices colleges charge differ by ability, suggesting the importance of peer effects. 2A third barrier is measurement of the peer group. See Chandrasekhar and Lewis (2011) for a discussion of circumventing measurement error in the peer group. 3For ease of exposition, we refer to the bundle of individuals’ performance-relevant characteristics as ability. 4One set of papers uses proxy variables to break the link between unobserved and peer ability (Arcidiacono and Nicholson (2005), Hanushek, Kain, Markman, and Rivkin (2003), and Betts and Morell (1999)). Another set of papers relies on some form of random assignment (Sacerdote (2001),Zimmerman (2003),Winston and Zimmerman (2003),Foster (2006),Lehrer and Ding (2007),Carrell and Hoekstra (2010), Carrell, Fullerton, and West (2009),Carrell, West, and Malmstrom (2008), and Hoxby (2001)). Finally, researchers have tried to circumvent the endogeneity problem with instrumental variables (Evans, Oates, and Schwab (1992)). See Epple and Romano (2011) for a review of the different approaches. 5Graham (2008) required random assignment into classes and because his analysis is cross sectional, must make stronger assumptions on the variance of the idiosyncratic term across classes. Mas and Moretti (2009) estimated spillovers in the workplace through both observables and unobservables. They used a twostage method that yields inconsistent and downward-biased estimates of the spillover parameters when T is fixed. This is likely to be unimportant in their setting because their data contain workers with many observations over time.
Quantitative Economics 3 (2012) Estimating spillovers using panel data 423 choice and individuals only have expectations of peer choices. The assumption on peer unobservables is also valid when individuals only have partial control over their outcomes, as is the case with test scores and grades, and either have expectations of peer choices or peer choices are made based solely on observables. Constructing the spillover as a linear combination of individual fixed effects results in a nonlinear optimization problem. Estimating individual unobserved heterogeneity in non-linear panel data models often results in biased estimates of the key parameters of interest—the incidental parameters problem.6As Ngoes to infinity for a fixed T,the estimation error for the fixed effects often does not vanish as the sample size grows, contaminating the estimates of the parameters of interest.7We show, however, that the nonlinear least squares estimate of the spillover parameter is consistent and asymptotically normal as N→∞with Tfixed, even though the fixed effects themselves are not consistent.8These consistency arguments also extend to the case where peer effects persist over time. While nonlinear least squares yields consistent estimates of the spillover parameters, the dimensionality of the problem renders nonlinear least squares infeasible. We develop an iterative algorithm that, under certain conditions, produces the same estimates as nonlinear least squares. The algorithm toggles between estimating the individual fixed effects and the spillover parameters. Each iteration lowers the sum of squared errors, with a fixed point reached at the nonlinear least squares solution to the full problem. We customize the model for application to peer effects in education, using studentlevel data from the University of Maryland. Six semesters of transcript data are available, covering the semesters from the spring of 1999 to the fall of 2001. We observe grades for every class each student took over the course of this period as long as the student lived on campus during any one of the six semesters. We estimate the model separately for each of three types of courses, finding significant peer effects that vary by course type. A 1 standard deviation increase in peer ability yields average returns similar to those from between a 3 percent and an 11 percent of a standard deviation increase in own ability, depending on the course type and specification. The lowest returns are found in math and science, and the highest returns are found in the social sciences. Our model allows us to quantify selection both within and across course types. Within course types, we compare the amount of selection with respect to observed and unobserved student ability. To arrive at these measures, we decompose each of the estimated student fixed effects, or what we label total ability, into an observed and an unobserved component using typical observed ability measures, such as Scholastic Aptitude 6Neyman and Scott (1948) were the first to document the incidental parameters problem. 7Hahn and Newey (2004) provided two methods of bias correction: a panel jackknife and an analytical correction. Woutersen (2002) and Fernandez-Val (forthcoming) considered estimators from bias-corrected moment conditions. In a similar vein, Arellano and Hahn (2006) and Bester and Hansen (forthcoming) considered bias-correcting the initial objective function. 8Other special cases where the incidental parameters problem does not require a bias correction are Manski (1987),Honore (1992), and Horowitz and Lee (2004).
424 Arcidiacono, Foster, Goodpaster, and Kinsler Quantitative Economics 3 (2012) Test (SAT) scores and high school performance. For all course types, we find greater selection on unobserved ability than observed ability. However, selection is highest when measured using total ability, a reflection of the significant correlation between peer observed and unobserved ability within a section.9Becauseweestimateourmodelsep- arately for each course type, we can compare student ability in their primary course of study to their ability in other fields, thereby quantifying selection across course types. We find strong evidence of both comparative and absolute advantage. On average, students select course types for which they are best suited. However, math and science students show greater aptitude overall in every course type. Finally, we compare our peer effect estimates to those that would be obtained using more conventional methods. In particular, we examine separately the two obstacles present in traditional peer effects estimation: selection into peer groups and the effect of peer unobservables. Controlling for selection only, which is what is accomplished using random assignment, we show that the estimated peer effects are lower than the peer effects obtained using our method. This is because random assignment techniques rely on incomplete measures of peer ability. We then reintroduce selection into the model and show that the bias in the peer effect estimate can be either positive or negative when both issues are present. For humanities courses, the peer effect estimate continues to be biased downward since the peer unobservables problem dominates the selection problem. The opposite is true for math and science courses, where the peer effect estimate using conventional methods is more than four times our original estimate. The differences across course types are due in part to the much higher correlation between individual observed ability and peer unobserved ability in math and science relative to the humanities. The remainder of the paper proceeds as follows. Section 2presents the baseline model, the identification result, and the solution algorithm. Section 3extends the model to incorporate correlated effects, endogenous effects, and heterogeneity in peer spillovers. Monte Carlo evidence on the performance of the algorithm is presented in Section 4. Section 5describes the University of Maryland data and Section 6presents the results. Section 7explores selection within and across course types, and Section 8illus- trates the biases associated with traditional peer effect measures. Section 9concludes. 2. Estimating spillovers with panel data In this section, we present a model and estimation strategy for measuring achievement spillovers using student fixed effects. The model is constructed keeping in mind that our application is measuring peer effects in college, where we are interested in the interactions that occur within discussion sections in large classes. We first consider a case where one’s outcome depends only on one’s own fixed effect and the fixed effects of the other individuals in a predefined peer group. We show that it is possible to obtain consistent estimates of the spillover and that there is a computationally cheap way to obtain the solution. All proofs appear in the Appendix A. 9By construction, observed and unobserved ability are orthogonal in the population.
Quantitative Economics 3 (2012) Estimating spillovers using panel data 425 2.1 Identifying spillovers using panel data Our baseline model has individual i’s outcome at time tin peer group n,Yitn, depending on his own observed and unobserved characteristics, Xit and uit, a linear function of the observed and unobserved characteristics of each of the other students in his peer group, and a transitory error, εitn. Denote as Mtn +1the total number of individuals in peer group nat time t. Each member of peer group nat time tthen has Mtn peers. Denote as Mtn∼ithe set of individuals (numbering Mtn) in peer group nat time twith individual iremoved. Our baseline specification can then be written Yitn =Xitβ1+uitβ2+1 Mtn j∈Mtn∼i (Xjtγ1+ujtγ2)+εitn(1) In addition to the assumption of linearity, the specification in (1) is restrictive along a number of dimensions. There are no endogenous effects, as peer choices do not enter the outcome equation. There are also no correlated effects, as there are no variables to capture the commonality of the environment faced by all members of student i’s time-t peer group. Finally, this specification does not allow for heterogeneity in the susceptibility to peer influence. While each of these restrictions is relaxed in the next section, even in this special case, estimation is problematic when peer groups are chosen. In particular, there may be correlation between uit and the sum of observed peer characteristics, leading to biased estimates of γ1. Also, we are not able to capture the peer influence through unobservables, meaning that γ2is inestimable without further assumptions. While random assignment can remove the correlation between uit and observed peer characteristics, the inability to capture spillovers through unobservables remains.10 We now make an additional assumption: the relevance to outcomes of peer characteristics is proportional to that of own characteristics, meaning that we can write11 γ1=γoβ1 γ2=γoβ2 This implies, for example, that if two dimensions of an individual’s ability are equally important in their effect on Yitn, then those two dimensions of peer ability are also equally important in determining Yitn. This same assumption is used in Altonji, Huang, and Taber (2004). Now define αito =Xitβ1+uitβ2 10Using random assignment to identify the spillover also disregards the possibility that spillovers operate differently in selected versus randomized contexts. 11For the remainder of the paper, we designate population parameters with an osubscript (γo) and designate estimates of the population parameters with a caret ( ˆγ).
426 Arcidiacono, Foster, Goodpaster, and Kinsler Quantitative Economics 3 (2012) We can then rewrite equation (1)as Yitn =αito +γo Mtn j∈Mtn∼i αjto +εitn(2) An individual’s outcome is then a function of the individual’s own ability at tplus the mean ability of the other students in the peer group at t. We are then interested in solutions to the nonlinear least squares problem min αγ N i=1 T t=1Yitn −αit −γ Mtn j∈Mtn∼i αjt2 (3) If individual ability varies over time, as it does in the above specification, γois not identified unless multiple observations per student are available in each time period. However, additional structure can be placed on the evolution of the αo’s to ensure that the spillover parameter is identified even if multiple observations per time period are not available. In the following discussion, we investigate the properties of our estimator of γounder two common assumptions about how ability evolves: A1 Static. Individual ability is assumed fixed over time, αito =αio. An individual’s outcome is then a function of his own fixed effect plus the mean of the fixed effects of the other students in the peer group. A2 Cumulative. Ability accumulates according to the interactions between the individual and his peers. At the initial time period, ability is given by αio. When there is no depreciation, from t=2onward ability accumulates according to αito =αio + t−1 t=1 γo Mtn j∈Mtn∼i αjto(4) Note that with only one observation per student available in each time period, it would be difficult to allow for more flexible forms of time-varying ability. The two approaches to the evolution of ability require estimating the same number of parameters—an αio for each individual plus an estimate of the spillover parameter, γo. Maintaining the assumptions of linearity and proportionality previously discussed, we prove that the least squares solution to equation (3) is a consistent estimator of γo under the following set of assumptions. Theorem 1. Let Ndenote the number of individuals who are observed at least two times and satisfy j∈Mtn∼i(αjto Mtn )=j∈Mtn∼i(αjto Mtn)for some tt.Suppose either A1 or A2 holds. Additionally,suppose the following statements: (i) We have E(εitnεjsk)=0∀j=i t =sn =k. (ii) We have E(εitnαjto)=0∀i j t n. (iii) We have E(α4 ito)<∞∀i t n. (iv) We have E(εitn)=0and E(ε4 itn)<∞∀i t n.
Quantitative Economics 3 (2012) Estimating spillovers using panel data 427 (v) Either E(ε2 itn|nt) =E(ε2 jtn|n t) ∀i j t n or Cov(ε2 itnNi)=0,where Niis the number of observations for individual i.Further,under A2,E(ε2 itn|t) =E(ε2 itn|t). (vi) The parameter γolies in the interior of a compact parameter space Γ,where the largest element of Γis given by γ.Furthermore,γ<M,where Mis the smallest class size. If (i)–(vi) hold,then ˆγis √Nconsistent and an asymptotically normal estimator of γofor fixed T. While most of the above assumptions are standard, a few nonstandard assumptions deserve closer inspection. Assumption (i) requires that the residuals across any two observations be uncorrelated. Any correlation across outcomes for the same individual is captured by the individual fixed effect, while correlation in outcomes across individuals in the same peer group is entirely captured by the peer effect.12 Assumption (v) requires that either the residuals within a peer group have equal variance, implying that only heteroskedasticity across peer groups can be accommodated, or, if heteroskedasticity operates at the individual level, it is uncorrelated with the number of times the individual is observed in the data. This assumption is generally applied in virtually all papers in the peer effects literature, as standard errors are typically clustered at the class level. Assumption (v) does, however, have to be strengthened in the accumulation case, as the variance in the error term cannot be correlated with time. Because the estimates of the individual effects are inconsistent for fixed T,one would expect the estimator of γoto be downward biased as a result of measurement error. Indeed, two-step approaches, such as that taken in Mas and Moretti (2009),inwhich estimates of the individual effects are obtained in a first step and then taken as given in the second-step estimation of the spillover parameter, do suffer from attenuation bias.13 Similarly, an approach that utilizes the average of the peer average grades over time to measure peer ability also is downward biased as a result of measurement error.14 The intuition for why measurement error does not lead to attenuation bias in our case follows directly from the structure of the consistency proof. The proof relies on solving for each of the individual effects as a function of the data and γ, and then substituting these functions for the individual effects in (3). The key is that when solving for αias a function of γ, we account for the direct effect of αion own outcomes and the indirect effect of αi on all the individuals who happen to be paired with i. The strength of the indirect effect is determined by the spillover parameter, and as the spillover parameter moves, so too do the implied estimates of the individual effects. Accounting for the relationship be- 12In the next section, we introduce a procedure for capturing correlated effects that do not work through the peer effect. 13In Mas and Moretti (2009), measurement error is likely less of an issue due to the large number of time periods per individual in their data. However, in other settings, particularly those in education, the number of time periods per individual is likely to be small. 14Using the average of the peer average grades as a proxy for peer ability induces two types of measurement error. The first source of measurement error is related to the fact that grades are a noisy signal of student ability. The second source of measurement error arises from misspecification of the production function, since there are now extra peer terms included in the average grades. When we estimate the model taking this approach, the estimated spillover effect declines by approximately 25 percent.
428 Arcidiacono, Foster, Goodpaster, and Kinsler Quantitative Economics 3 (2012) tween the estimate of the spillover parameter and the estimates of the individual effects removes the measurement error issue. Minimizing the concentrated objective function with respect to γalone yields the desired result. The structure of the proof suggests an obvious estimation strategy: minimize the concentrated least squares problem with respect to γ. However, while concentrating out the α’s is useful for proving consistency, the resulting formulas are quite cumbersome and difficult to calculate. Directly solving (3) is also generally not possible because of the dimensionality of the problem. Instead, we consider an iterative estimation strategy that both circumvents the dimensionality problem and yields the same solution as direct maximization. The next section introduces the computational procedure and discusses how it relates to the broader literature regarding estimation of high-dimensional problems. For the remainder of the paper, we focus on the case where αito does not vary over time, since our empirical application investigates the existence of peer effects in a college setting—a setting where there is heterogeneity in both the number of courses taken and the course topic. The accumulation of skill is likely to be less important here relative to primary or secondary schooling, since college courses generally do not build directly on one another. However, the methods we describe below can easily be adapted to the accumulation case as well. 2.2 Computing spillovers with panel data Before moving directly to the computation of the spillover model outlined in the previous section, we illustrate how our proposed procedure can ameliorate a somewhat simpler computational problem prominently discussed in the literature. An outstanding problem in applied microeconomics is how to estimate models that contain multiple types of fixed effects where each set of fixed effects is of a large dimension.15 We begin with this econometric problem, since the iterative method we employ solves the issue of multiple fixed effects en route to estimating spillovers. We focus on two papers in particular—Rivkin, Hanushek, and Kain (2005) and Abowd, Kramarz, and Margolis (1999)—to illustrate the difficulties in estimating large numbers of fixed effects. Rivkin, Hanushek, and Kain (2005) modeled gains in test scores as a function of the observed characteristics of the students, Xi, and teacher fixed effects, πjo,whereiindexes individuals and jindexes teachers.16 Thechangeintestscoresfromtimet−1to t, given that the individual has teacher jat time t,Y, is then modeled as Y =βoXi+πjo +εit(5) 15Harris and Sass (2006,2011) used our method in estimation of models with multiple classes of highdimensioned fixed effects. Burke and Sass (forthcoming) used our method to estimate peer effects in Florida public schools. 16Our model is presented in levels, since we are working with collegiate data, where defining a baseline of achievement is somewhat difficult. However, the model can be applied exactly as written if gains are the outcome of interest. The key differences are that the outcome is now a gain and that the individual fixed effects reflect heterogeneity in ability to improve. Peer effects in this case would also work through an individual’s ability to improve. To the extent that gains are employed to eliminate time invariant unobserved heterogeneity, our model can handle this directly by estimating fixed effects at multiple levels.
Quantitative Economics 3 (2012) Estimating spillovers using panel data 435 Table 2. Varying class size and heteroskedasticity, γo=015.a Random Assignment Selection Obs. per Peer Group Student Size σεσεσεσε 5U[515]195 115 195 115 ˆγ0158 0150 0160 0145 (0041)(0022)(0059)(0030) R20484 0686 0502 0704 5U[515]N(195009)N(115009)N(195009)N(115009) ˆγ0148 0151 0146 0149 (0037)(0022)(0059)(0026) R20480 0.67 0497 0692 5U[515]N(095 +size 10 009)N(015 +size 10 009)N(095 +size 10 009)N(015 +size 10 009) ˆγ0153 0149 0150 0149 (0049)(0029)(0061)(0027) R20459 0633 0474 0649 aThe R2values reported in this table pertain to the regression of grades onto the constructed fixed effect values. We alter the random error added on to the constructed grade for each student so as to manipulate the amount of variation in performance that is explained by the ability measure. For the bottom two-thirds of the table, the standard deviation of the random error varies across peer groups according to the distributions listed. Parameter values are averages over 100 simulations on a population of 10,000 students. ˆγis more precisely measured when students are randomly assigned to classes. Selection in this case can be thought of as occurring at two levels: the classroom level and the teacher level. Teacher-level sorting refers to the idea that teachers are often assigned students of similar ability over time. These results reflect the fact that sorting at the teacher level confounds the estimate of the correlated effect and reduces the precision of the classroom-level peer effect estimate. In fact, if selection occurred only at the classroom level, the peer effect estimates would be more precise than in the random assignment case (ceteris paribus), since there would be greater variation in peer ability. Second, as the peer group size increases, the precision of ˆγdecreases.26 This is again related to the variation in peer ability across classes. With smaller class sizes, other things equal, there is greater variation in peer ability across classes, which yields more precise estimates of the spillover. Many applications involving peer effects involve possible heteroskedasticity at the class level. Table 2shows the performance of the algorithm in the presence of heteroskedasticity, including the case when heteroskedasticity is a function of the size of the class. The first panel of results illustrates that heterogenous class size does not affect the performance of the peer effects estimator: ˆγremains centered around 0.15 as 26This can be seen in Table 1, since as the number of observations per student increases, we should naturally see an increase in precision. Yet we do not see that increase between the first and second rows, because peer group size increases as well, driving standard errors up. We only see the increase in precision between the second and third rows, when the number of observations per student rises as peer group size is held constant. The negative association of peer group size and precision, as well as all other relationships discussed here, have also been verified in numerous additional Monte Carlo exercises; results are available upon request from the authors.
436 Arcidiacono, Foster, Goodpaster, and Kinsler Quantitative Economics 3 (2012) Table 3. Heterogenous gamma models.a Model Random Assignment Selection Heterogeneity in responsiveness to peers γ1o=015 0151 0146 (0025)(0029) γ2o=010100 0094 (0033)(0032) R20683 0699 Heterogeneity in peer influence γ1o=015 0150 0151 (0029)(0037) γ2o=010102 0098 (0030)(0039) R20687 0684 aThe R2values reported in this table pertain to the regression of grades onto the constructed fixed effect values. Parameter values are averages over 100 simulations on a population of 10,000 students. Each student is observed 5 times with a total group size of 10 students. peer group size varies uniformly between 5 and 15. The second and third panels add heteroskedasticity to the heterogenous class size case. In the second panel, σεis drawn from a Normal distribution with a mean of 1.15 or 1.95 and a standard deviation of 0.3. It is assumed that each peer group draws from the same distribution. In the third panel, the mean of the distribution of σεshifts according to the size of the peer group. The peer effects estimator continues to perform quite well regardless of the type of heteroskedasticity. Across the various distributions of σεand sorting scenarios, we estimate a peer effect centered on the truth. As noted previously, the linear-in-means model may not be the most interesting case from the policy maker’s perspective. We suggested two extensions to the baseline framework that would relax this assumption. Table 3illustrates the performance of the heterogeneous effect models, where the basic structure of the Monte Carlo experiments is kept intact. In each case, we assume students are characterized by one binary variable. The results indicate that the estimation framework previously outlined is amenable to heterogenous peer effects. 5. Data With the model producing consistent estimates of the spillover parameter and performing well in our Monte Carlo simulations, we now turn to the data used in estimation. The administrative data set used in this paper covers all undergraduates observed residing in University of Maryland on-campus housing during any of the following six academic semesters: Spring 1999 (S99), Fall 1999 (F99), Spring 2000 (S00), Fall 2000 (F00), Spring 2001 (S01), or Fall 2001 (F01). The data set includes students living off-campus in a given semester as long as they were observed living on-campus during at least one of the six semesters. Ninety percent of University of Maryland entering freshmen live on campus in their first semester,27 so the data set includes at least 90 percent of the University 27This number is taken from publicly available statistics posted on the university’s web page.
Quantitative Economics 3 (2012) Estimating spillovers using panel data 437 of Maryland undergraduate population who began study sometime in the six-semester period.28 A “section” is a subset of students from an entire course that meets together formally at least once a week. In these smaller groups, greater communication and interaction is expected of students. Our section-level spillover captures how the influence of the ability of other students in the same section impacts on own course grade. To generate the student–section-level sample, we first placed two major restrictions on the data set: students had to have valid A–F grade information for the given section and they could not be the only student observed in the section that semester.29 Students who withdrew from a course, audited, or received a nonletter grade (such as pass) were excluded from the sample due to concerns that they might not have been present during sections and classes.30 Dropping students with no letter grade reduced the sample from 351,940 to 324,181. We then deleted all observations on sections that were not in one of three well defined academic subgroups: (i) humanities (86,844 observations), (ii) social sciences (77,312 observations), and (iii) hard sciences and mathematics (82,675 observations).31 This left a combined sample of 246,831 student–section observations, representing 18,511 individual students. Sample sizes are provided in Table 4. Table 4. Sample sizes.a S99 F99 S00 F00 S01 F01 Total 1. Student–sections 27,900 37,231 37,109 45,991 45,054 53,546 246,831 2. Students 7126 9646 9458 11,760 11,393 13,662 63,045 3. Unique sections 3095 3388 3408 3628 3543 3754 20,816 4. Unique courses 1030 1079 1172 1189 1246 1252 6968 5. Single-section courses 632 682 733 752 778 795 4372 6. Student–sections 23,206 31,627 30,599 38,056 36,260 43,853 203,601 (only multisection courses) aFigures represent the data set after applying the restrictions noted in the text. The unrestricted data set contained 351,940 student–section observations. Rows 3 and 4 show the total number of unique sections and courses, respectively, in which anyone in the sample during the given semester was observed. 28There is a less complete representation for upperclassmen, some of whom entered before our observation period and may not have lived on campus during the period. However, our identification of peer effects comes from large, multisection courses in which freshmen predominate. In tests of whether classes that were underrepresented had lower estimated peer effects than those with a complete representation, we found no meaningful differences. 29Numeric grade equivalents were assigned as follows: A =4,B=3,C=2,D=1, and F =0. 30If two separate grades were recorded for the student for a given section, the highest grade was used. We dropped students who did not receive a final grade in a given course, which assumes that these students did not affect the outcomes of their peers. We could have treated those who attrit as full members of the class provided we observed a grade for these individuals in another course. The other course would then pin down the individual’s fixed effect. 31Excluded courses include those that are generally more vocationally oriented, but very diverse; for example, journalism, nutrition and food science, landscape architecture, and library science. Because our model estimates a homogeneous underlying ability for each course type, we did not include these courses in a separate category due to our concern that the underlying ability necessary to succeed in them is not sufficiently homogeneous across the category.
438 Arcidiacono, Foster, Goodpaster, and Kinsler Quantitative Economics 3 (2012) Finally, while our method does not require the presence of observable characteristics about individuals, the data set to which we apply it does offer an array of observable measures about each student. We examine later in the paper how characteristics such as SAT scores correlate with our estimates of student ability. 6. Estimates of classroom spillovers We now turn to our model specifications and estimates. We first describe and estimate a model that restricts the peer effect such that the spillover depends only on the mean ability in the section. The second specification allows the size of the spillover to depend on one’s own characteristics. For example, those who have high SAT verbal scores may receive higher benefits from their peers than those who have low SAT verbal scores. With the results of the two models in hand, we then show how predictable ability is given observable measures such as SAT scores, high school grade point average, and demographics. 6.1 Homogeneous gamma model With n,c,andtindexing sections, courses, and semesters, we have the same specification as in equation (11) except that now, with the number of individuals in each section varying, we restrict the spillover to depend on the mean fixed effect of the other individuals in the same section of a course: Yitnc =αio +γo Mtn j∈Mtn∼i αjo +δtco +εitnc(13) Because grades are assigned at the course level, there is a relationship between students who share a course but are not in the same section that cannot be captured by the section peer effect, γo. We might expect, for example, that if the course is graded on a curve and the entire class is extremely able, a mediocre student’s grade may suffer. By including fixed effects at the course level, the δtco’s, we can make the outcome measure comparable across classes. Consistent with the data section, we split courses into three types: humanities, social sciences, and math and science. A student’s performance in each type of course will differ according to the particular student’s strengths and weaknesses. Therefore, instead of encapsulating all the attributes of a student into one ability measure, we allow students to have separate ability measures for each course type in which they are enrolled. As noted above, all courses used in our analysis are classified as belonging to one of the following course types: humanities, social sciences, or math and science. We estimate an independent ability measure for each type of course for each student, conditional on the student’s enrollment in at least one class within that course type.32 Another supporting rationale for the empirical division into course types is that the amount of interaction, and therefore the size of the peer effect, may differ by course type. The algorithm is then 32Information as to which courses were assigned to which course types is available from the authors upon request.
Quantitative Economics 3 (2012) Estimating spillovers using panel data 439 run separately for each type of course, yielding three sets of peer and class effects estimates, as well as separate student ability measures for each course type taken.33 Table 5shows the results from estimating equation (13) for each of the three types of courses. Standard errors are calculated using a wild bootstrap procedure.34 The results indicate positive and significant section peer effects for all course types. The magnitudes of the section-level peer effects suggest that peer effects are most important in the social sciences and least important in math and science. This pattern may reflect the amount of collaborative work required in each course type as well as the differing amounts of discussion that occur in the sections. To understand the importance of peer ability relative to own ability, we need to take into account the differences in variation of peer and own ability. There is likely to be less variation in peer ability than in own ability, as peer ability averages over a cross section of students, leading to some heterogeneity canceling out. The second and third columns of Table 6show the standard deviation of mean peer ability and the standard deviation of individual ability, respectively. The fourth column then shows the fraction of a standard deviation of own ability that is equivalent, in terms of its effect on grades, to a 1 standard deviation increase in peer ability. This is calculated by dividing the standard deviation Table 5. Peer effects results by course type: homogeneous gamma model.a Humanities Soc. Sci. Math/Sci. Section peer ability 01613 01960 00483 (0016)(0019)(0013) N86,844 77,312 82,675 R206373 06321 06861 aThe dependent variable is the grade in the class. Class fixed effects are estimated in all specifications. Standard errors are obtained using a wild bootstrap. Table 6. Standard deviations of estimated ability and marginal effects: homogeneous gamma model.a Course Type Section SD Population SD Marginal Effect Ratio Humanities 02823 06804 00669 Social science 03103 07125 00853 Math and science 05952 09498 00302 aSection SD is the standard deviation of average peer ability across the sample of student–section observations of the given course type. Population SD is the standard deviation of ability across the sample of student–section observations in courses of the given course type. These calculations are both based on the fixed effects estimated by the homogeneous gamma model. Marginal Effect Ratio shows the ratio of the effect on grades from a 1 standard deviation increase in peer ability to the effect on grades from a 1 standard deviation increase in own ability. 33Note that while classes with only one section do not help in the estimation of the spillover parameter directly, these classes are still useful in pinning down the student fixed effects. 34The wild bootstrap is advantageous in this setting since it allows for heteroskedasticity of an unknown form and does not require any resampling. For additional details on the wild bootstrap, see Davidson and MacKinnon (2006).
440 Arcidiacono, Foster, Goodpaster, and Kinsler Quantitative Economics 3 (2012) of mean peer ability by the standard deviation of individual ability and multiplying this number by the estimated γ. The gap evident in the raw marginal effects between math and science and the other course types is somewhat mitigated because there is relatively more heterogeneity in peer ability in math and science courses than in humanities or social science courses. A 1 standard deviation increase in peer ability is shown to be equivalent to a maximum of 9 percent of the effect of a 1 standard deviation increase in individual ability in the social sciences, and to a minimum of 3 percent of the effect of a 1 standard deviation increase in individual ability in math and science courses. 6.2 Heterogeneous gamma model Table 7shows the results of a peer effect model that allows for heterogeneity in the response to peers.35 Response to peer ability is allowed to vary according to an individual’s gender, race, and SAT scores. The qualitative results for humanities and social sciences are the same. Relative to white males, Asians see less of a return to peer ability, while females and other nonwhite students see higher returns. Both SAT math and SAT verbal scores are associated with higher returns to peer ability. While Asians in math and science again see lower returns to peer ability, females and other nonwhites also see lower returns relative to their white male counterparts. The interaction of the peer effect with SAT verbal score is once again positive, but the sign on the SAT math interaction is now negative. The differences in the SAT interactions across fields suggest that two competing forces may be at play. First, those who have higher test scores may have skills that make Table 7. Peer effects results by course type: heterogeneous gamma model.a Section Peer Ability Humanities Soc. Sci. Math/Sci. Overall 02058 02227 00940 (0020)(0025)(0018) Female 00970 00584 −00517 (0019)(0021)(0018) Asian −00098 −00347 −00346 (0035)(0030)(0023) Other nonwhite 00375 00252 −00420 (0028)(0031)(0025) SAT math 00410 00507 −00560 (0012)(0016)(0011) SAT verbal 00222 00147 00635 (0011)(0014)(0009) N86,844 77,312 82,675 R206376 06323 06864 aThe dependent variable is the grade in the class. Class fixed effects are estimated in all specifications. Standard errors are obtained using a wild bootstrap. 35We also allowed for peer effects to be stronger for those of similar races and genders, with little change in the results. Standard errors were obtained using the wild bootstrap.
Quantitative Economics 3 (2012) Estimating spillovers using panel data 441 Table 8. Standard deviations of estimated ability and marginal effects: heterogeneous gamma model.a Course Type Section SD Population SD Avg. Marginal Effect Marginal Effect Ratio Humanities 02383 06368 02595 00970 (00622) Social science 02849 06747 02480 01048 (00554) Math and science 06028 09660 00555 00347 (00636) aSection SD is the standard deviation of average peer ability across the sample of student–section observations of the given course type. Population SD is the standard deviation of ability across the sample of student–section observations in courses of the given course type. These calculations are both based on the fixed effects estimated by the heterogeneous gamma model. The fourth column shows the marginal effect on grades from a 1-point increase in peer ability. Marginal Effect Ratio shows the ratio of the effect on grades from a 1 standard deviation increase in peer ability to the effect on grades from a 1 standard deviation increase in own ability. The numbers in parentheses are standard deviations of the marginal effects of a 1-point increase in peer ability that are estimated to occur in the sample. them better able to benefit from their peers. Working against this, however, is that there is more scope for students to benefit the lower they are in the ability distribution. SAT verbal and math scores may not be highly correlated with the ability to perform well in the humanities and social sciences, implying that the first effect dominates in these course types. However, the SAT math score may be highly correlated with the ability to perform well in math and science classes, leading to the second effect dominating. Averaging across all individuals within a course type, the relative magnitude of the peer effects is unchanged from the homogeneous gamma model. Peer ability is most important in social science courses and least important in math and science courses. However, the overall magnitude of the peer effects is significantly higher, increasing by an average of over 40 percent across course type. Relative to a 1 standard deviation increase in own ability, the effects of a 1 standard deviation increase in peer ability are also higher in the heterogeneous gamma model as shown in the final column of Table 8.The ratio of the effects of a 1 standard deviation increase in mean peer ability to a 1 standard deviation increase in own ability range from a low of 3.5 percent for math and science to a high of 10.5 percent in the social sciences. 6.3 Analysis of ability Next, we explore the extent to which the fixed effects from our iterative algorithm are predictable using the observed proxies for ability that are consistently used in related literature, and the extent to which the fixed effects estimated using the two methods differ. To facilitate this comparison, we use SAT scores, high school performance, and a host of other observable student attributes as regressors to construct a conglomerate observable measure of ability. This approach is analogous to the creation of an academic index (as employed by Sacerdote (2001)). For each course type, we regress our estimated student fixed effects on an array of previous performance measures and demographics. These results are presented Table 9.
442 Arcidiacono, Foster, Goodpaster, and Kinsler Quantitative Economics 3 (2012) Table 9. Regression of fixed effects on observed ability.a Homogeneous Gamma Model Heterogenous Gamma Model Hum. Soc. Sci. Math/Sci. Hum. Soc. Sci. Math/Sci. SAT math 000 011 036 −020 −004 051 (001)(001)(001)(001)(001)(001) SAT verbal 008 011 000 −004 006 −019 (001)(001)(001)(001)(001)(001) HS GPA 049 049 073 049 049 072 (001)(002)(002)(001)(002)(002) Honors 014 015 017 014 015 017 (002)(002)(002)(002)(002)(002) Sports −007 −014 001 −008 −014 000 (002)(003)(003)(002)( 003)(003) In state 005 008 012 005 008 012 (003)(003)(003)(003)(003)(004) N17,332 15,264 16,077 17,332 15,264 16,077 R2022 024 034 013 017 036 Race/gender dummies Y Y Y Y Y Y aThe dependent variable in the second through fourth columns is the student-level fixed effects estimated in the homogeneous gamma model; the dependent variable in the last three columns is the student-level fixed effects estimated in the heterogeneous gamma model. All regressions also include a female dummy variable and dummy variables for Black, Hispanic, Asian, and American Indian. Standard errors are robust to heteroskedasticity. The second through fourth of Table 9show results when we use the fixed effects estimated in our homogeneous gamma model as the dependent variable. The statistical significance and magnitudes of the coefficients on SAT math and SAT verbal scores vary across the four different course types in predictable ways. For example, SAT math scores are insignificant when explaining ability in humanities courses, but are a better proxy for math and science ability. The opposite is true for SAT verbal scores, with higher SAT verbal scores associated with higher ability in the humanities, but uncorrelated with ability in math and science. The last three columns in Table 9show the corresponding results for the heterogeneous gamma model. Recalling the results found in Table 7regarding the positive association of SAT verbal scores with stronger peer effects across all course types, it is unsurprising that the coefficient on own SAT verbal score is smaller and even negative for some course types when predicting own ability.36 Combining these findings with the results presented here suggests that SAT verbal scores have very little to do with ability in the absolute, but rather reflect how capable an individual is at extracting rents from others. The R2for these regressions ranges from 0.13 to 0.36, depending on the course type and whether we use the homogeneous or heterogeneous gamma model to generate the individual fixed effects. That these observable characteristics only explain a small portion of our estimated ability measures suggests the possibility of large biases associated 36Previous works such as Arcidiacono (2004),Arcidiacono and Vigdor (forthcoming), and Arcidiacono, Cooley, and Hussey (2008) have all found no returns to verbal test scores in the labor market.
Quantitative Economics 3 (2012) Estimating spillovers using panel data 443 with the unobserved ability problem when following a selection-on-observables, or random assignment, approach. At the same time, because our estimated fixed effects are noisy estimates of unobserved ability, the R2of these regressions is biased downward. As a result, we likely overstate the role of unobserved ability. 7. Quantifying selection In this section we quantify how much selection is taking place within course types with respect to both observed and unobserved ability.37 For ease of exposition, we refer to the predicted values of the regressions in Table 9as observed ability and to the residuals of those regressions as unobserved ability. Thus, by construction, observed and unobserved ability are uncorrelated at the individual level. However, unobserved individual ability and observed peer ability will be correlated if students sort by total ability. By decomposing ability into its observed and unobserved components, we can calculate the correlation between unobserved individual ability and observed peer ability, which is the crux of the selection problem. We also examine selection across course types. Because we estimate separate abilities for each course type, we are able to determine whether students choose to take more courses in areas where they are comparatively more able. 7.1 Selection within course types Table 10 provides information by course type on the selection evident with respect to both observed and unobserved ability. We use the underlying ability as estimated by the homogeneous gamma model and the heterogeneous gamma model, as well as the observed and unobserved portions of this ability. The first row for every course type shows the section-size-weighted average of the sectionwide standard deviation of the variable in question, across all sections in the particular course type; the second row for every course type shows the simple standard deviation of the variable in question across the sample of students taking courses of the given course type. The third row provides the ratio of the two. The smaller are the numbers in the third row, the tighter is the distribution of the variable within sections relative to the unsorted distribution and, therefore, the more selection is evident with respect to that variable. For all three course types, there is more selection on unobserved ability than on observed ability. In the social sciences and, in particular, for the humanities, there is more selectionontheestimatedα’s as a whole than on either observed ability or unobserved ability separately, with the highest levels of selection found in math and science. These patterns are driven by the correlation between peer observed and unobserved ability. For the homogeneous gamma specification, the correlation coefficients between peer observed and unobserved ability are 0.03, 0.07, and 0.35 in the humanities, social sciences, and math and science, respectively. The selection on the estimated α’s in math and science is particularly strong relative to selection on either observed or unobserved 37See Altonji, Elder, and Taber (2005) for more discussion of selection on observed and unobserved factors.
444 Arcidiacono, Foster, Goodpaster, and Kinsler Quantitative Economics 3 (2012) Table 10. Selection based on observables and estimated ability.a Homogeneous Gamma Heterogenous Gamma Course Type αˆαα uαˆαα u Humanities Avg. section-level SD 06154 03372 05385 05849 02487 05415 Population SD 08046 03773 07106 07619 02711 07121 Ratio 07649 08937 07578 07677 09174 07604 Social science Avg. section-level SD 06365 03750 05667 06056 02994 05688 Population SD 08618 04228 07510 08226 03343 07517 Ratio 07386 08869 07546 07362 08956 07567 Math and science Avg. section-level SD 07475 05036 06666 07636 05323 06649 Population SD 10567 06165 08583 10719 06432 08575 Ratio 07074 08169 07767 07124 08276 07754 aEach set of rows corresponds to sections in one course type; each set of columns corresponds to one version of the model (homogeneous gamma versus heterogeneous gamma). Variables under analysis appear in the heading row: αis ability as estimated by our model, ˆαis observed ability, and αuis unobserved ability. Avg. section-level SD is the average (across all sections, and weighted by section size) of the standard deviation of the variable within a section. Population SD is the standard deviation of the variable in the population of students taking courses of the given course type. Ratio is the ratio of the first of these to the second, and shows the degree of selection into sections with respect to each variable displayed. ability, which is consistent with a high correlation between peer observed and unobserved ability. With the observed and unobserved ability measures, it is also to possible to estimate the correlation between unobserved individual ability and observed peer ability, which feeds directly into the bias associated with the selection problem. The correlation coefficients for unobserved individual ability and observed peer ability are 0.03, 0.06, and 0.20 for humanities, social sciences, and math and science, respectively. The high correlation coefficient for math and science suggests that the upward bias associated with peer effect estimation using a selection-on-observables approach might be quite large. 7.2 Selection across course types Because the vast majority of students are observed in courses of multiple types during their tenure at Maryland, we obtain multiple estimates of ability for most students. Calculating the correlations between estimated ability levels illuminates the extent to which good performance in each of the three course types is driven by similar student attributes as performance in the other course types and, therefore, provides an empirical index of the academic similarity of course types.38 Panel A of Table 11 shows the correlation coefficients among estimated ability levels across the three course types from the homogeneous gamma model. These correlations are created using estimated abilities from students observed in all course types.39 The 38For ease of exposition, we focus on the homogeneous gamma model for the rest of the paper. 39Correlations among estimated abilities were also calculated for all students who were observed in each pair of course types. Similar coefficients resulted.
Quantitative Economics 3 (2012) Estimating spillovers using panel data 451 and α3n=y3n+γ2y2n−γy11n−γ3y12n 1−γ4 We then show the form of the minimization problem when the α’s are concentrated out. Lemma 2. Concentrating the α’s out of the original least squares problem results in an optimization problem over γthat takes the form min γ 1 N N n=1 (y11n−y12n+γ(y3n−y2n))2 2(1+γ2) Our nonlinear least squares problem now has only one parameter, γ.Wearenowina position to investigate the properties of our estimator of γo. For ease of notation, define q(wγ) as q(wγ) =(y11 −y12 +γ(y3−y2))2 2(1+γ2) where w≡y.WeletWdenote the subset of R4representing the possible values of w.Our key result is then Lemma 3, which establishes identification. Lemma 3. We have Eq(wγo)<E q(wγ)∀γ∈Γγ=γo Theorem 12.2 of Wooldridge (2002) establishes that sufficient conditions for consistency are identification and uniform convergence. Having already established identification, Lemma 4shows uniform convergence. Lemma 4. We have max γ∈Γ 1 N N n=1 q(wnγ)−Eq(wγ) p →0 Consistency then follows from Theorem 12.2 of Wooldridge: ˆγp →γo. Finally, we establish asymptotic normality of ˆγ. Denote s(wγo)and H(wγo)as the first and second derivatives of q(wγ) evaluated at γo. Then Lemma 5completes the proof. Lemma 5. We have √N(ˆγ−γo)d →N0A−1 oBoA−1 o
452 Arcidiacono, Foster, Goodpaster, and Kinsler Quantitative Economics 3 (2012) where Ao≡EH(wγo) and Bo≡Es(wγo)2=Vars(wγo) Proof of Lemma 1. Our objective is to show that the system of equations obtained by differentiating equation (14)withrespecttoαcan be expressed as a series of equations in terms of γand y, and that these expressions are as given in Lemma 1. Again, conditional on γ, the estimates of individual effects in one block will not affect the estimates of the individual effects in another block. Thus, we can work with the system of firstorder conditions within one block and then generalize the results to the full system of equations. The first-order condition for α1n(student in each block who is observed in both time periods) is given by 0=−2 N(y11n−α1n−γα2n)+(y12n−α1n−γα3n)+γ 3 i=2 (yin −αin −γα1n) while the first-order condition for α2nand α3nare, respectively, given by 0=−2 N(y2n−α2n−γα1n)+γ(y11n−α1n−γα2n) and 0=−2 N(y3n−α3n−γα1n)+γ(y12n−α1n−γα3n) Within each block, this yields a relatively simple system of three equations and three unknown abilities. The first-order conditions for α2nand α3ncan be rearranged such that α2n=y2n+γy11n−2γα1n 1+γ2 and α3n=y3n+γy12n−2γα1n 1+γ2 Notice that the equation for α2ndepends only on the own outcome, the outcome of individual 1 when grouped with individual 2, and the ability of individual 1. A similar result occurs for α3n. Thus, the only thing linking individuals 2 and 3 within a block is the ability of individual 1. Rearranging the first-order condition for α1nsuch that the α1nare grouped on the left-hand side of the equation results in α1n2+2γ2=y11n+y12n+γ(y2n+y3n)−2γ(α2n+α3n)
Quantitative Economics 3 (2012) Estimating spillovers using panel data 453 and substituting for α2nand α3nusing the previously derived formulas yields α1n2+2γ2=y11n+y12n+γ(y2n+y3n) −2γ 1+γ2y2n+y3n+γ(y11n+y12n)−4γα1n Moving all the α1nterms to the left side and finding common denominators on both sides of the equation results in α1n((2+2γ2)(1+γ2)−8γ2) 1+γ2 =(1+γ2)(y11n+y12n+γ(y2n+y3n)) −2γ(y2n+y3n+γ(y11n+y12n)) 1+γ2 Canceling out the denominators and simplifying both sides of the equation yields α1n21−γ22=1−γ2(y11n+y12n)−γ1−γ2(y2n+y3n) Dividing both sides of the equation by 2(1−γ2)2yields the desired result that α1n=y11n+y12n−γ(y2n+y3n) 2(1−γ2) The solution for α1ncan now be substituted back into the first-order conditions for α2nand α3nto yield solutions strictly as functions of γand y. Substituting α1ninto the equation for α2nand finding a common denominator yields α2n=2(1−γ2)(y2n+γy11n)−2γ(y11n+y12n−γ(y2n+y3n)) 2(1−γ2)(1+γ2) Factoring out the 2 in the numerator and expanding the resulting expression yields α2n=(1−γ2+γ2)y2n+(γ(1−γ2)−γ)y11n−γy12n+γ2y3n (1−γ2)(1+γ2) Some simple manipulation leads to the final result that α2n=y2n+γ2y3n−γy12n−γ3y11n 1−γ4 Obtaining the solution for α3nproceeds in exactly the same way, and yields a formula that mirrors the solution for α2nwith the appropriate indices changed to reflect when individual 3 is grouped with individual 1. The result is α3n=y3n+γ2y2n−γy11n−γ3y12n 1−γ4 Proof of Lemma 2. Lemma 1provides a solution for αstrictly as a function of yand γ. We can substitute this solution back into the original optimization problem to derive the result in Lemma 2.
454 Arcidiacono, Foster, Goodpaster, and Kinsler Quantitative Economics 3 (2012) Consider minimizing the sum of squared residuals within a particular block n.There are four residuals within each block: two for the student observed twice and one each for the corresponding peer. We begin by simplifying the residual for the first observation of the student observed twice, which is given by the expression e11n=y11n−α1n−γα2n Substituting for α1nand α2nin e11nwith the results from Lemma 1results in e11n=y11n−y11n+y12n−γ(y2n+y3n) 2(1−γ2)−γ(y2n+γ2y3n−γy12n−γ3y11n) 1−γ4 Finding a common denominator and combining like terms in the numerator yields e11n=21−γ4−1+γ2+2γ4y11n−1+γ2−2γ2y12n +γ1+γ2−2γy2n+γ1+γ2−2γ3y3n21−γ4 Simplifying the numerators on each of the yterms and factoring the denominator yields e11n=(1−γ2)y11n−(1−γ2)y12n−γ(1−γ2)y2n+γ(1−γ2)y3n 2(1−γ2)(1+γ2) Finally, we can cancel all the (1−γ2)terms to arrive at e11n=y11n−y12n+γ(y3n−y2n) 2(1+γ2) The expression for e12nas a function of γand ycan be similarly derived by substituting in α1nand α3n. However, the expressions for e12nand α3nare mirror images of the expressions for e11nand α2n.Thus,e12nwill take the exact same form as e11nexcept the subscripts denoting the period or classmate are swapped. The expression is e12n=y12n−y11n+γ(y2n−y3n) 2(1+γ2) The residuals for the one observation individuals in each block, e2nand e3n,aregiven by e2n=y2n−α2n−γα1n and e3n=y3n−α3n−γα1n To write these strictly as functions of γand y, we again use the results of Lemma 1. Substituting for α1nand α2nin e2nyields e2n=y2n−y2n+γ2y3n−γy12n−γ3y11n 1−γ4−γ(y11n+y12n−γ(y2n+y3n)) 2(1−γ2)
Quantitative Economics 3 (2012) Estimating spillovers using panel data 455 Finding a common denominator and simplifying the resulting expressions yields e2n=γ(y12n−y11n+γ(y2n−y3n)) 2(1+γ2) The expression for e3nis similar to that of e2n, except the subscripts differ to reflect the time period in which individual 3 is grouped with 1. Thus the solution for e3nwill mirror the solution for e2n, except that the appropriate subscripts are swapped across terms. The final expression for e3nis e3n=γ(y11n−y12n+γ(y3n−y2n)) 2(1+γ2) The original optimization problem written as a function of the residuals in each block ntakes the form min αγ 1 N N n=1e2 11n+e2 12n+e2 2n+e2 3n Now we can substitute in for each residual using the formulas previously derived. However, a cursory glance at the formulas for e11n,e12n,e2n,ande3nreveals that e11n=−e12n=−γe2n=γe3n Using these relationships, we can rewrite the least squares problem as min αγ 1 N N n=12+2γ2e2 11n Substituting in with our solution for e11nyields min γ 1 N N n=12+2γ2(y11n−y12n+γ(y3n−y2n))2 (2(1+γ2))2 Canceling terms results in the optimization problem min γ 1 N N n=1 (y11n−y12n+γ(y3n−y2n))2 2(1+γ2) Proof of Lemma 3. The population objective function as a function of γis given by Eq(wγ)=E(y11 −y12 +γ(y3−y2))2 2(1+γ2) Substituting for ywith the data generating process yields Eq(wγ)=Eα1o+γoα2o+ε11 −(α1o+γoα3o+ε12) +γα3o+γoα1o+ε3−(α2o+γoα1o+ε2)22(1+γ2)
456 Arcidiacono, Foster, Goodpaster, and Kinsler Quantitative Economics 3 (2012) Canceling the appropriate terms and combining like terms in the numerator leaves Eq(wγ)=E((γo−γ)(α2o−α3o)+(ε11 −ε12)+γ(ε3−ε2))2 2(1+γ2) Opening up the square term leaves Eq(wγ)=E1 2(1+γ2)(γo−γ)2(α2o−α3o)2+(ε11 −ε12)2+γ2(ε3−ε2)2 +2(γo−γ)(α2o−α3o)(ε11 −ε12) +2γ(γo−γ)(α2o−α3o)(ε3−ε2) +2γ(ε11 −ε12)(ε3−ε2) By Theorem 1(i) and (ii), the final three terms in the numerator all have expectation 0. Similarly, any covariance terms associated with the first three terms in the numerator will have expectation 0. The final simplified expression is given by Eq(wγ)=(γo−γ)2E[(α2o−α3o)2]+E[ε2 11]+E[ε2 12]+γ2(E[ε2 3]+E[ε2 2]) 2(1+γ2) which we can rewrite in the manner Eq(wγ)=(γo−γ)2E[(α2o−α3o)2] 2(1+γ2)+E[ε2 11]+E[ε2 12]+γ2(E[ε2 3]+E[ε2 2]) 2(1+γ2) Note that by Theorem 1(v) E[ε2 11]=E[ε2 2]and E[ε2 12]=E[ε2 3], implying that we can rewrite the above equation as Eq(wγ)=(γo−γ)2E[(α2o−α3o)2] 2(1+γ2)+Eε2 11+Eε2 12/2 The first term in the above expression is strictly greater than 0 for all γ= γoand the second term does not depend upon γ.Asaresult,E[q(wγo)]<E[q(wγ)]for all γ∈Γ when γ=γo. Proof of Lemma 4. Uniform convergence, requires that max γ∈Γ 1 N N n=1 q(wnγ)−Eq(wγ) p →0 Theorem 12.1 in Wooldridge states four conditions that the data and qmust satisfy so that the above condition holds. 1. The parameter Γis compact. This condition is satisfied by Theorem 1(vi). 2. For each γ∈Γ,q(·γ) is Borel measurable on W. Since q(·γ) is a continuous function of w, it is also Borel measurable.
Quantitative Economics 3 (2012) Estimating spillovers using panel data 457 3. For each w∈W,q(w·)is continuous on Γ. Our concentrated objective function is continuous in γ. 4. For all γ∈Γ,|q(wγ)|≤b(w),wherebis a nonnegative function on Wsuch that E[b(w)]<∞. Note that q(wγ) is always positive, so we can ignore the absolute value. We derive a bounding function b(w) in the manner q(wγ) =(y11 −y12 +γ(y3−y2))2 2(1+γ2) =(y11 −y12)2+γ2(y3−y2)2+2γ(y3−y2)(y11 −y12) 2(1+γ2) ≤2(y11 −y12)2 2(1+γ2)+2γ2(y3−y2)2 2(1+γ2) ≤(y11 −y12)2+(y3−y2)2 where the third line follows from the triangle inequality. Our bounding function is then b(w) =(y11 −y12)2+(y3−y2)2 wherewehaveshownthatb(w) ≥q(wγ) for all y. We now show that E[b(w)]<∞, completing the proof. Note that E[b(w)]is given by Eb(w)=E(y11 −y12)2+(y3−y2)2 Using the triangle inequality, we can rewrite the above expression as Eb(w)≤E2y2 11 +2y2 12 +2y2 3+2y2 2 ≤2Ey2 11+Ey2 12+Ey2 3+Ey2 2 Next we substitute in for yusing the data generating process. Consider E[y2 11],whichis given by Ey2 11=E(α1o+γoα2o+ε11)2 Applying the triangle inequality again yields Ey2 11≤3Eα2 1o+γ2 oEα2 2o+Eε2 11 Theorem 1(iii) and (iv) ensure that all of the terms on the right-hand side of the inequality in the above equation are finite. Thus, E[y2 11]is finite. By a similar argument, it can be shown that all the terms in E[b(w)]are finite. Proof of Lemma 5. Theorem 12.3 in Wooldridge (2002) states six conditions that must hold for ˆγto be distributed asymptotically normal. Many of these conditions involve the first and second derivatives of q(wγ). We begin our proof of asymptotic normality by deriving the first and second derivatives of the objective function.
458 Arcidiacono, Foster, Goodpaster, and Kinsler Quantitative Economics 3 (2012) The first derivative of the objective function, or the score, is given by s(wγ) =1 4(1+γ2)221+γ22(y3−y2)y11 −y12 +γ(y3−y2) −4γy11 −y12 +γ(y3−y2)2 Expanding the square and grouping on the γterms yields s(wγ) =1 (1+γ2)21−γ2(y11 −y12)(y3−y2)+γ−(y11 −y12)2+(y3−y2)2 The Hessian of the objective function is simply the derivative of the score, ∂s(yγ) ∂γ , and is written H(wγ) =1 (1+γ2)4 ×1+γ22(y3−y2)2−(y11 −y12)2−2γ(y3−y2)(y11 −y12) −4γ1+γ2 ×γ(y3−y2)2−(y11 −y12)2+1−γ2(y3−y2)(y11 −y12) Factoring out a (1+γ2)and combining like terms greatly simplifies the above expression, leaving H(wγ) =1 (1+γ2)31−3γ2(y3−y2)2−(y11 −y12)2 −2γ3−γ2(y3−y2)(y11 −y12) We now show that the six conditions of Theorem 12.3 in Wooldridge (2002)aresatisfied. We will refer to the above formulations of the score and Hessian throughout. 1. The parameter γomust be in the interior of Γ. This condition is satisfied by Theorem 1(vi). 2. The function s(w·)is continuously differentiable on the interior of Γfor all w∈ W. Since H(wγ) is continuous in γ,s(w·)is continuously differentiable. 3. Each element of H(wγ) is bounded in absolute value by a function b(w),where E[b(w)]<∞. We derive a bounding function b(w) in the manner H(wγ) =(1−3γ2)((y3−y2)2−(y11 −y12)2)−2γ(3−γ2)(y3−y2)(y11 −y12) (1+γ2)3 H(wγ)≤1−3γ2(y3−y2)2−(y11 −y12)2 −2γ3−γ2(y3−y2)(y11 −y12) H(wγ)≤1−3γ2(y3−y2)2−(y11 −y12)2 +2γ3−γ2(y3−y2)(y11 −y12)
Quantitative Economics 3 (2012) Estimating spillovers using panel data 459 H(wγ)≤1+3γ2(y3−y2)2+(y11 −y12)2 +3+γ22γ(y3−y2)(y11 −y12) H(wγ)≤1+3γ2(y3−y2)2+(y11 −y12)2 +3+γ2γ(y3−y2)2+(y11 −y12)2 H(wγ)≤1+3γ2(y3−y2)2+(y11 −y12)2 +3+γ2|γ|(y3−y2)2+(y11 −y12)2 where the second to last line utilizes the fact that (y3−y2)2+(y11 −y12)2>2(y3−y2)(y11 − y12)as ((y3−y2)−((y11 −y12))2>0.Letγand γdenote the largest and smallest elements of the set Γ.Theγthat maximizes the right-hand side is given by γ∗=max{γ−γ}<∞. Our bounding function is then b(w) =1+3γ∗2(y3−y2)2+(y11 −y12)2 +γ∗3+γ∗2(y3−y2)2+(y11 −y12)2 =1+γ∗γ∗2+3γ∗+3(y3−y2)2+(y11 −y12)2 wherewehaveshownthatb(w) ≥H(wγ) for all w. Notice that the absolute value of γ is no longer necessary, since by definition γ∗is always positive. We now show that E[b(w)]<∞, completing the proof: Eb(w)=1+γ∗γ∗2+3γ∗+3E(y3−y2)2+(y11 −y12)2 When deriving the bounding function for q(wγ), we showed that E[(y3−y2)2+(y11 − y12)2]<∞. Since γ∗is also finite, E[b(w)]<∞. 4. The equality Ao≡E[H(wγo)]is positive definite. We first note that we can interchange the expectations and the partial derivatives: E[H(wγ)]=∂2E[q(wγ)]/∂γ2. From Lemma 3, we know that we can write Eq(wγ)=(γ −γo)2E[(α2o−α3o)2] 2(1+γ2)+Eε2 11+Eε2 12/2 Note that γaffects two terms: (γ −γo)2and the denominator. However, because we are going to evaluate the expected Hessian at γo, we only need the second derivative of the first term, (γ −γo)2. All of the other partial derivatives will either be multiplied by (γ − γo)2or (γ −γo), both of which are zero when γ=γo. The second derivative of (γ −γo)2 with respect to γis positive. This second derivative is then multiplied by the expectation of a squared object in the numerator and divided by the sum of squared objects in the denominator. Thus, the expectation of the Hessian evaluated at γois strictly positive. 5. We have E[s(wγo)]=0. Note that E[s(wγ)]=∂E[q(wγ)]/∂γ. Differentiating E[q(wγ)]with respect to γleaves terms that are multiplied by (γ −γo)or by (γ −γo)2, implying that if we evaluate the derivative at γ=γo, then the expected score is zero.
460 Arcidiacono, Foster, Goodpaster, and Kinsler Quantitative Economics 3 (2012) 6. Each element of s(wγo)has finite second moment. Given that the score has only one element, this condition boils down to E[s(wγo)2]<∞. To show this, we square the score function, repeatedly apply the triangle equality, and evaluate the expected value at the true γ: Es(wγo)2=E1 (1+γ2 o)41−γ2 o(y11 −y12)(y3−y2) +γo−(y11 −y12)2+(y3−y2)22 Repeatedly applying the triangle inequality yields Es(wγo)2≤E1 (1+γ2 o)421−γ2 o2(y11 −y12)2(y3−y2)2 +2γ2 o−(y11 −y12)2+(y3−y2)22 ≤E4 (1+γ2 o)421−γ2 o2y2 11 +y2 12y2 3+y2 2 +γ2 o(y11 −y12)4+(y3−y2)4 ≤E4 (1+γ2 o)421−γ2 o2y4 11 +y4 12 +y4 3+y4 2 +4γ2 oy2 11 +y2 122+y2 3+y2 22 ≤E8 (1+γ2 o)41−γ2 o2y4 11 +y4 12 +y4 3+y4 2 +4γ2 oy4 11 +y4 12 +y4 3+y4 2 ≤E8 (1+γ2 o)2y4 11 +y4 12 +y4 3+y4 2 ≤8 (1+γ2 o)2Ey4 11 +y4 12 +y4 3+y4 2 Now we substitute for ywith the data generating process. Consider E[y4 11],whichisgiven by Ey4 11=E(α1o+γoα2o+ε11)4 Repeatedly applying the triangle inequality yields Ey4 11≤9Eα2 1o+γ2 oα2 2o+ε2 112 ≤27Eα4 1o+γ4 oEα4 2o+Eε4 11
Quantitative Economics 3 (2012) Estimating spillovers using panel data 467 References Abowd, J. M., R. H. Creecy, and F. Kramarz (2002), “Computing person and firm effects using linked longitudinal employer–employee data.” Technical report, Center for Economic Studies, U.S. Census Bureau. [429] Abowd, J. M. and F. Kramarz (1999), “The analysis of labor markets using matched employer–employee data.” In Handbook of Labor Economics, Vol. 3B (O. C. Ashenfelter and D. Card, eds.), chapter 40, 2629–2710, Elsevier Science B.V., Amsterdam. [429] Abowd, J. M., F. Kramarz, and D. N. Margolis (1999), “High wage workers and high wage firms.” Econometrica, 67 (2), 251–333. [428,429] Altonji, J. G., T. E. Elder, and C. R. Taber (2005), “Selection on observed and unobserved variables: Assessing the effectiveness of catholic schools.” Journal of Political Economy, 113 (1), 151–184. [443] Altonji, J. G., C.-I. Huang and C. R. Taber (2004), “Estimating the cream skimming effect of private school vouchers on public school students.” Working paper 16579, NBER. [422,425] Arcidiacono, P. (2004), “Ability sorting and the returns to college major.” Journal of Econometrics, 121 (1–2), 343–375. [442] Arcidiacono, P., J. Cooley, and A. Hussey (2008), “The economic returns to an MBA.” International Economic Review, 49 (3), 873–899. [442] Arcidiacono, P. and S. Nicholson (2005), “Peer effects in medical school.” Journal of Public Economics, 89 (2–3), 327–350. [422] Arcidiacono, P. and J. Vigdor (forthcoming), “Does the river spillover? Estimating the economic returns to attending a racially diverse college.” Economic Inquiry.[442] Arellano, M. and J. Hahn (2006), “A likelihood-based approximate solution to the incidental parameter problem in dynamic nonlinear models with multiple effects.” Working paper wp2006_0613, CEMFI. [423] Bester, C. A. and C. Hansen (forthcoming), “A penalty function approach to bias reduction in non-linear panel models with fixed effects.” Journal of Business & Economics Statistics.[423] Betts, J. R. and D. Morell (1999), “The determinants of undergraduate grade point average: The relative importance of family background, high school resources, and peer effects.” Journal of Human Resources, 32 (2), 268–293. [422] Bramoulle, Y., H. Djebbari, and B. Fortin (2009), “Identification of peer effects through Social Networks.” Journal of Econometrics, 150 (1), 41–55. [431] Burke, M. and T. Sass (forthcoming), “Classroom peer effects and student achievement.” Journal of Labor Economics.[428] Carrell, S., R. Fullerton, and J. West (2009), “Does your cohort matter? Measuring peer effects in academic achievement.” Journal of Labor Economics, 27, 439–464. [422]
468 Arcidiacono, Foster, Goodpaster, and Kinsler Quantitative Economics 3 (2012) Carrell, S. and M. Hoekstra (2010), “Externalities in the classroom: How domestic violence harms everyone’s kids.” American Economic Journal: Applied Economics,2, 211–228. [422] Carrell, S., J. West, and F. Malmstrom (2008), “Peer effects in academic cheating.” Journal of Human Resources, 43, 173–207. [422] Chandrasekhar, A. G. and R. Lewis (2011), “Econometrics of sampled networks.” Unpublished manuscript, MIT. [422] Cooley, J. (2009a), “Alternative mechanisms of peer spillovers in the classroom: Implications for identification and policy.” Working paper, University of Cambridge. [431, 465] Cooley, J. (2009b), “Desegregation and the achievement gap: Do diverse peers help?” Working paper, University of Cambridge. [465,466] Davidson, R. and J. G. MacKinnon (2006), “Bootstrap methods in econometrics.” In Palgrave Handbook of Econometrics, Vol. 1 (T. Mills and K. Patterson, eds.), 812–838, Palgrave Macmillan, Basingstoke. [439] Durlauf, S. N. and Y. M. Ioannides (2001), “Social interactions.” Annual Review of Economics, 2, 451–478. [431] Epple, D. and R. Romano (2011), “Peer effects in education: A survey of the theory and evidence.” In Handbook of Social Economics (J. Benhabib, M. Jackson, and A. Bisen, eds.), 1053–1164, Elsevier, Amsterdam. [422] Epple, D., R. Romano, and H. Sieg (2003), “Peer effects, financial aid and selection into colleges and universities: An empirical analysis.” Journal of Applied Econometrics, 18, 501–525. [422] Evans, W. N., W. E. Oates, and R. M. Schwab (1992), “Measuring peer group effects: A study of teenage behavior.” Journal of Political Economy, 100 (5), 966–991. [422] Fernandez-Val, I. (forthcoming), “Fixed effects estimation of structural parameters and marginal effects in panel probit models.” Journal of Econometrics.[423] Foster, G. (2006), “It’s not your peers, and it’s not your friends: Some progress toward understanding the peer effect mechanism.” Journal of Public Economics, 90, 1455–1475. [422] Giorgi, G. D., M. Pelizzari, and S. Redaellil (2007), “Be as careful of the books you read as of the company you keep: Evidence on peer effects in educational choices.” Working paper 14948, NBER. [431] Graham, B. (2008), “Identifying social interactions through conditional variance restrictions.” Econometrica, 76, 643–660. [422] Hahn, J. and W. Newey (2004), “Jackknife and analytical bias reduction for nonlinear panel models.” Econometrica, 72, 1295–1319. [423]
Quantitative Economics 3 (2012) Estimating spillovers using panel data 469 Hanushek, E. A., J. F. Kain, J. M. Markman, and S. G. Rivkin (2003), “Does peer ability affect student achievement?” Journal of Applied Econometrics, 18 (5), 527–544. [422] Harris, D. and T. Sass (2006), “Value-added models and the measurement of teacher quality.” Unpublished manuscript, Urban Institute. [428] Harris, D. and T. Sass (2011), “Teacher training, teacher quality and student achievement.” Journal of Public Economics, 95, 798–812. [428] Honore, B. (1992), “Trimmed LAD and least squares estimation of truncated and censoredmodelswithfixedeffects.”Econometrica, 60, 533–565. [423] Horowitz, J. L. and S. Lee (2004), “Semiparametric estimation of a panel data proportional hazard model with fixed effects.” Journal of Econometrics, 119, 155–198. [423] Hoxby, C. (2001), “Peer effects in the classroom: Learning from gender and race variation.” Working paper 7867, NBER. [422] Hoxby, C. and G. Weignarth (2005), “Taking race out of the equation: School reassignment and the structure of peer effects.” Working paper, Harvard University. [432] Lehrer, S. and W. Ding (2007), “Do peers affect student achievement in China’s secondary schools?” Review of Economics and Statistics, 89 (2), 300–312. [422] Lin, X. (2010), “Identifying peer effects in student academic achievement by spatial autoregressive models with group unobservables.” Journal of Labor Economics,28(4), 825–860. [431] Manski, C. F. (1987), “Semiparametric analysis of random effects linear models from binary panel data.” Econometrica, 55, 357–362. [423] Mas, A. and E. Moretti (2009), “Peers at work.” American Economic Review, 99, 112–145. [422,427] Neyman, J. and E. Scott (1948), “Consistent estimates based on partially consistent observations.” Econometrica, 16, 1–32. [423] Paglin, M. and A. Rufolo (1990), “Heterogeneous human capital, occupational choice, and male–female earnings differences.” Journal of Labor Economics, 8 (1), 123–144. [445] Rivkin, S. G., E. A. Hanushek, and J. F. Kain (2005), “Teachers, schools, and academic achievement.” Econometrica, 73 (2), 417–458. [428] Sacerdote, B. I. (2001), “Peer effects with random assignment: Results for dartmouth roommates.” Quarterly Journal of Economics, 116 (2), 681–704. [422,441] Winston, G. C. and D. J. Zimmerman (2003), “Peer effects in higher education.” In College Decisions: How Students Actually Make Them and How They Could (C. Hoxby, ed.), University of Chicago Press, Chicago. [422] Wooldridge, J. M. (2002), Econometric analysis of cross-section and panel data. MIT Press, Cambridge. [451,457,458]
470 Arcidiacono, Foster, Goodpaster, and Kinsler Quantitative Economics 3 (2012) Woutersen, T. (2002), “Robustness against incidental parameters.” Unpublished manuscript, University of Arizona. [423] Zimmerman, D. J. (2003), “Peer effects in higher education: Evidence from a natural experiment.” Review of Economics and Statistics,85(1),9–23.[422] Submitted March, 2011. Final version accepted March, 2012.