ࡱ>  %` bjbj"x"x .L@@JSJSJSJSNTV UUUUUUUU$`hȊ-\UU\\UUlrlrlr\nUUlr\lrlrQ|ɂUU R41KJSa ́&0V߁jqZj$ɂɂj݂U Xvlr{Y,ZUUUbr UUUV\\\\MRR  The Effect of Team Composition on Student Learning in Introductory Economics by Robert L. Moore* Abstract The major advocates of collaborative learning in the college classroom generally suggest that team members should be chosen to complement each other in skills. Similar advice comes from the literature in personnel economics relating to worker productivity in teams Some advocates also suggest that there are advantages in this regard in forming heterogeneous teams in terms of gender, ethnicity and class year, although there are also some drawbacks noted as well from such heterogeneous teams. However, little empirical work has been done on how best to select the most effective student teams (groups) for individual student learning. This paper hopes to begin to fill this gap. It combines information from students admission files with individual student performance data on exams in two sections of an introductory economics course at a selective liberal arts college as well as with information about the team the student was assigned to. The initial empirical results in this paper provide little support for the above suggestions. The results also differ from the results in the only other empirical work on this issue. In particular, heterogeneous teams in terms of gender or ethnicity or class year result in no more individual student learning (and knowledge) transfer than more homogeneous teams, holding other significant variables affecting student learning constant. And there is at least some support for the reverse in terms of class year --- that is, the higher percentage of frosh students on a three or four student team, the more student learning. While it is advantageous to have at least one student on the team with AP Calculus or AP Econ credit, which is consistent with the general advice above, a team with a larger standard deviation of SAT scores (or higher average SAT scores) does not improve individual student learning. There is also some very limited evidence in this study that another characteristic of teams results in positive individual learning, namely if the team had a woman who participated in intercollegiate athletics. Finally, while advocates of collaborative learning sometimes recommend avoiding the isolation of a woman or a student of color on a team, there is no evidence in this study that such isolation disadvantages those students in terms of their exam performance. I. Introduction In recent years several notable economic educators have been advocating the use of various active learning techniques as an alternative to the more traditional chalk and talk teaching pedagogy.1 One such important active learning technique involves collaborative learning in small teams (or groups), and there is a large literature on the effectiveness of this pedagogy.2 The recent TIP program sponsored by the Committee on Economic Education (and the NSF) also encourages its use to participants, as do some notable economic educators.3 The voluminous literature on collaborative learning provides suggestions for how to form these student teams, and there is also some guidance from the emerging field of personnel economics on how to form effective teams to improve productivity. However, while there is a growing body of empirical work on the effectiveness of collaborative learning in economics courses, there is very little empirical work that sheds light on how to choose the most effective teams (groups) for individual student learning.4 The current paper hopes to begin to fill this gap. It combines data on individual student performance on exams in two sections of an introductory economics course at a selective liberal arts college with information from students admission files to investigate empirically the effect of team composition on this individual student learning. The teaching in the course relied heavily on collaborative learning in small groups of three or four students each using randomly selected semi- permanent teams of students, one set of teams for the microeconomics portion of the course, and another set for the macroeconomics portion. The major advocates of collaborative learning as well as the literature from personnel economics generally suggest that team members should be chosen to complement each other in skills. More generally, advocates also suggest that heterogeneous teams in terms of gender, ethnicity and class year are also preferred.5 However, the empirical results in this paper provide little support for these suggestions. The results also differ from the results in the only other empirical work on this issue. In particular, heterogeneous teams in terms of gender or ethnicity or class year result in no more individual student learning (and knowledge transfer) than more homogeneous teams, holding other significant variables affecting student learning constant. And there is at least some support for the reverse --- that is, the higher percentage of freshmen students on a three or four student team, the higher the exam score, holding other variables constant. Further, neither the average SAT scores for the group nor the standard deviation of the SAT scores within the group affect individual student learning, holding other variables constant. However, it does turn out to be advantageous to have at least one student on the team with AP Calculus or AP Econ credit, as well as a female who participates in intercollegiate athletics. Finally, there is no empirical support for the suggestion to avoid isolating a female or a non-Caucasian student on an otherwise homogenous team of males or Caucasians. The remainder of the paper is organized as follows. Section II provides a brief review of the relevant prior work on this issue. Section III then briefly describes how teams and collaborative learning were incorporated into the teaching pedagogy in the introductory economics course. Section IV describes the basic model for estimation as well as the data used. Section V discusses the empirical results, while Section VI concludes. II. Some Relevant Literature Researchers in Personnel Economics, a relatively recent subfield of labor economics, as well as those in organizational economics have emphasized that teams can have positive and negative effects on worker productivity. Teamwork where individuals complement each others skills favors knowledge transfer and also allows for comparative advantage and specialization, thus improving productivity. As such, team diversity in the broadest sense could help via knowledge sharing and coordination, especially if such diversity also entails complementary skills and knowledge.6 But researchers also point to the possibility of free-riding, less peer pressure and various coordination problems that could hinder productivity. Thus, non-skill-related demographic diversity may harm productivity by raising communication costs and making peer pressure less effective.7 Unfortunately, for a variety of reasons beyond the scope of this paper, there is very little empirical work on the effect of team composition on worker productivity in the labor market, not to mention a lack of empirical evidence on the effects of team composition on individual learning and what Hansen, Owan, and Pan refer to as knowledge transfer in the classroom setting.8 Despite the lack of empirical evidence, prominent practitioners of collaborative learning still offer advice on how to form small student teams. For example, Millis and Cottell Jr. summarize these recommendations as follows: Most cooperative-learning theorists and practitioners advocate instructor-selected teams, with a few caveats. Felder and Brent caution against groups where women and minorities are outnumbered. They do not recommend teams composed of only one woman or one minority member..In the team formation process, instructors should distribute students from team to team based on their academic preparation and ability, their gender, their ethnic background, and other characteristics that might prove useful. The idea is to create teams that will build on students variety of strengths..(p. 51) Later they observe that if new teams are to be formed during the semester, say after a midterm, .the faculty member should assign to each group one student who performed well on the exam, one who performed poorly, and two or three who performed close to the mean. Once again, additional heterogeneity can be built into the groups by dividing the students as evenly as possible with respect to gender and ethnicity. Personality factors can be taken into account as well. (p. 56). The current paper is most directly patterned after that of Hansen, Owan and Pan, The Impact of Group Diversity on Performance and Knowledge Spillover An Experiment in a College Classroom. I am not aware of any other work that attempts to empirically estimate the effect of team composition on individual performance. Hansen, Owan and Pan look at student performance (both team performance and individual performance) in an introductory undergraduate management course in the Olin School of Business at Washington University in St. Louis over five different semesters. The course involves large lectures to 100 150 students along with discussion sessions where teaching assistants run classroom games and lead discussions in sections of 12 26 students. The four or five member teams are formed by the teaching assistants. The group work involves three assignments that are weighted one percentage point each of the course grade and that are graded satisfactory/unsatisfactory, along with a group project that counts 25% of the course grade. For their group project, a business book is selected by each group from the list provided by the instructor and the group then writes a paper and gives an oral presentation where they analyze the market environment and discuss other strategic options that are available to the firm. Their data set consists of over 400 students (and about 100 teams) over 5 years under two instructors and numerous Teaching Assistants. While their paper analyzes both the effect of team composition on team performance as well as individual student performance, only the results for individual student performance are relevant here. The authors find that after including individual variables such as SAT scores, age, gender, ethnicity, etc., the age diversity and gender diversity of the team that the student was assigned to both positively affect individual exam scores. They did not find any significant effects of a groups racial diversity on individual student performance.9 III. The Use of Teams in Introductory Economics at a Selective Liberal Arts College In contrast to the above study, this paper looks empirically at the impact of team composition on individual student learning in two sections of an introductory economics course taught at Occidental College, a selective national liberal arts college, with 32 and 31 students respectively, during the Fall Semester of 2007. The introductory economics course in question is similar to the course described by Hansen, Salemi, and Siegfried and emphasizes having students apply a limited number of key concepts to events and circumstances around them. They refer to this as achieving economic literacy, and our first course covers both core microeconomics and macroeconomics Principles.10 It is part of a two semester introductory economics sequence that collectively covers the same material as that in most first year two semester economics sequences of principles of microeconomics and principles of macroeconomics. However, in our case the first semester course emphasizes the key concepts of both micro and macro without as much of the more technical details and graphs. For example, there is an emphasis on welfare economics and the efficiency of competitive markets, as represented in a typical demand and supply diagram using producer and consumer surplus, but no formal theory of the firm with accompanying diagrams. The concepts of elasticity and the inefficiency of monopoly are introduced, but such technical details as to how to calculate the demand elasticity using two points on the demand curve, or how to derive a monopolists MR schedule from the market demand curve are covered in the second semester. In the macroeconomic portion of the first semester course, the basics of the AD/AS diagram and the longer run self-correction mechanism is used for analysis of short run fiscal and monetary policy, but the technical details of the formulas for various fiscal (and other) multipliers as well as the formulas for monetary expansion are again left for the second course. There are seven problem sets, two short writing assignments, three midterms (the first two covering micro topics and the third macro topics) and a comprehensive final exam. The emphasis is on problem solving and the application of the key economic principles to students daily lives and what they might encounter when reading a newspaper. More details on the course coverage including the chapters assigned from Mankiws Principles of Economics text are contained in a footnote.11 The teaching pedagogy in the course again differs from that in this previous study. It perhaps most closely follows the model provided by Bartlett12, which emphasizes 10 15 minute mini-lectures followed by collaborative learning in small groups on worksheet problems and exercises.13 In addition, there is an optional collaborative learning lab (CLL) that takes place outside of regular class hours that most of the teams of students participate in, as described in Moore. In addition to working in teams in most every class and in the CLL, the teams also are responsible for one short oral presentation in each half of the course, but it is not a significant part of the individual student grade. In the collaborative learning lab students take unit tests, which consist of three or four problems, but with their same team members assigned from class. After discussing the answers, an individual team member is then randomly selected to orally explain their teams answers to a trained CLL mentor. Credit is awarded for answering all the questions correctly --a pass-and if students miss a question, they can take a similar unit test until the team passes. Credit is awarded for eventually passing, independent of the number of attempts. While teams were chosen to ensure that members of a team had schedules that would enable them to participate in the CLL program, individuals could also take the unit tests individually, or with less than their entire team present. Examples of CLL unit tests and typical class worksheets/exercises are available from the author.14 IV. Data and Empirical Analysis The basic model and estimating equation for individual performance on the portion of the course covering principles of microeconomics is providing in equation (1): (1) MicroexamScoreij = B Xi + A Zj + nij where --- MicroexamScoreij is the weighted average of the student s scores on the two first midterms covering microeconomic principles and the questions on the final exam covering microeconomic principles, for student i, who was assigned to semi-permanent team j for this portion of the course. --- Xi is a vector of individual characteristics for student i that have shown in past studies to predict performance in principles of economics courses. For this paper I will use the sum of a students scores on the SAT verbal and math tests15, a dummy variable for whether they had received course credit for AP Micro or AP Calculus16, a dummy variable for gender, a dummy variable for ethnicity, a dummy variable for whether they were a freshmen, and a variable that might be viewed as a proxy for effort, namely the number of CLL unit tests they passed in the microeconomics portion of the course. Detailed definitions of each of these variables along with their various descriptive statistics are provided in Table 1 and Table 2 respectively. --- Zj is a vector of group characteristics describing the group that the student was assigned to during the portion of the course where microeconomic principles were covered. For this paper, I will use the groups average for the sum of the members math and verbal SAT scores, the standard deviation of the groups SAT score sum, the percent of the group who were frosh, the percent of the group who were male, the percent of the group who were white17, and a dummy variable indicating whether any member of the group had received credit for AP Micro or AP Calculus. Again, detail definitions and descriptive statistics of each of these group characteristic variables are found in Table 1 and Table 2. (Other variables representing various other group characteristics were also added in order to test specific suggestions made by advocates of collaborative learning. See the discussion that follows.) A similar equation was estimated for the second part of the course, where the principles of macroeconomics was covered. The dependent variable was now the weighted average of the students score on the third midterm (which covered the first part of macroeconomics) and the questions on the final exam that covered the material from the macroeconomics portion of the course. The only changes in the variables that related to student i was that the dummy variable representing whether the student had received AP credit now referred to whether the student had received that AP credit for AP Macro rather than AP Micro (or AP Calculus). A similar slight change was made for the team dummy variable relating to whether any team member had received AP Macro credit or AP Calculus credit. In addition, the number of CLL unit tests passed now referred to those in the macroeconomic portion of the course. The group characteristics for individual i were now based on the group that the student was assigned to for the rest of the semester, covering the macroeconomic content. A variation of each dependent variable in each equation was also used. Instead of using the weighted average of the midterms and final exam scores, only the final exam score on the microeconomic questions on the final exam was used, and similarly only the final exam score on the macroeconomic questions on the final exam was used. The rationale for trying this variation was that students learn about how best to study for the course and the type and level of difficulty of exam questions as the course progresses. Thus, by the time the final exam comes around fewer students are surprised by the type and level of difficulty of the questions, and as such, this might be a better variable to ultimately gauge student learning. Alternatively, one might care most about what the student knew at the end of the course, not along the way. The qualitative nature of the results that follow did NOT vary much based upon which version of the dependent variables was used. However, the variation using the final exam scores as the dependent variables should be more useful going forward, as I intend to gather more data from future Economics 101 students. I intend to use the same final exam for at least several more years. Unlike the midterm exams in the course, the final exam will be kept confidential and not distributed to students. V. Results The results of estimating equation (1) using OLS are contained in Table 3. Note that two versions are reported, one (version A) with the dependent variable consisting of the students score on the microeconomic questions on the final exam only, and version B, consisting of the weighted average of the student microeconomic exam scores (including the final). (Results for the macroeconomic portion of the course are discussed later and will be presented in Table 4.) In both versions, the variables that relate to the individual student are fairly consistent with previous studies that attempt to predict performance in introductory economics courses, and are also fairly consistent with those in Hansen, Owan, and Pan.18 In particular, the sum of the students verbal and math SAT scores positively affects exam performance, as does whether a student has received AP Micro credit/AP Calculus credit, and both variables are significantly differently than zero. Caucasian students also perform better, everything else the same, as do non-freshman students, and both of these variables are again consistently different from zero.19 Contrary to some studies, men do worse than women, but this dummy variable for gender is not significantly different from zero, so no conclusions can be drawn in this regard. Finally, the variable indicating the number of CLL unit tests passed is insignificant; this was somewhat expected, since some of the best students didnt feel it necessary to attempt these unit tests --- the grade boost all but disappears if the student performs well on everything else.20 Of more interest here are the variables that relate to the team that the student was assigned to for the microeconomics portion of the course. Only two team variables are significant at the 10% level or less, namely the percent of the students on the team who were freshman, and the dummy variable of whether any member of the team had received credit for AP Micro or AP Calculus. The results in Table 3 A indicate that the higher the proportion of students who were freshman, the better the performance on the microeconomics questions on the final exam. In particular, a team with all freshman will each score 9 percent higher on the microeconomics portion of the final exam than a team without any freshman. Further, if a team has at least one member with AP credit, each member will score 10% higher, everything else the same. These variables are also significantly different than zero is version B, although the effects are a bit smaller. If one were to use these results to guide the formation of teams, one should try to include at least one student who had received AP micro credit or AP Calculus credit, and try to balance teams in terms of the percent of frosh. In addition, I tried to test the suggestion not to isolate females or non-Caucasians on otherwise all male or all Caucasian teams. The coefficient for this variable wasnt significant in either version. As a result, there was no evidence to confirm that such isolation reduced the performance of these isolated individuals. This author has long felt that intercollegiate athletes, especially women athletes, work particularly well in small group work in the classroom because of their experience on teams in an athletic context. While Hansen, Owan, and Pan did not mention this, I did obtain data from our Athletic Department that enabled me to identify which of the students in the introductory economics course in this study participated in intercollegiate athletics during the same academic year. In order to test this, a dummy variable, TFEMALEATHLETED, was added as an independent variable to equation (1), equal to one if the team the student was assigned to included a female athlete, and zero otherwise. The results are shown in Table 5. The coefficient of this variable was 7.4 (when added to the regression in Table 3), with the t-statistic of 1.47, significant at between the 5 and 10% level. This would imply that having a team member who was a female athlete increased the composite microeconomics score by 7 percentage points. The other coefficients in the equation remained quite similar. (However, this coefficient was not significant when added to any of the other regressions. The results for macroeconomics will be discussed shortly.) Another similar dummy variable was also used to determine if having a male student who participated in intercollegiate athletics on a team positively affected individual student learning. This coefficient was always insignificant, with very small t-statistics. Perhaps the more important conclusion here is that given the limited data available for this study, the rest of the variables describing the team composition dont seem to matter very much in terms of a knowledge transfer to individual exam performance.21 Indeed, the F-statistic testing the null hypothesis that all of the coefficients on the team variables are zero by comparing the sum of squared residuals in the constrained versus the unconstrained equations was .95 for the equation in Table 3 version A, and .84 for the equation in Table 3, version B. The relevant value for the F-statistic with 6, 50 degrees of freedom to reject the null hypothesis is 2.29 (at the 5% level of significance). As such, given the data here we cannot reject the null hypothesis that the team variables dont influence individual exam performance for the microeconomic portion of the course. Another interpretation is that there are not yet enough data to determine whether the team variables matter for individual performance. With additional data using more students over time, more significant results may yet emerge. In this regard, the data set in Hansen, Owan and Pan, was about 7 times larger, with 431 individual students and over 100 teams, versus the 63 individuals and 18 teams here. Contrary to the results here, they found that in Management 101, students in teams with a greater diversity of age (a proxy for class year) and greater diversity in gender performed better in Management 101, everything else the same. They attributed the positive effect of this age diversity to a greater variation in knowledge. While older students dont necessarily have more knowledge of introductory economics, the effect of more knowledge may be picked up in Table 3 with the dummy variable for AP Micro/Calculus. They have a variety of explanations for why mixed gender groups performed better than male dominant groups, including the possibility that males are lazier and that females are better in cooperating, organizing, and coordinating activities, an effect that may be picked in the present study with the female athletes. They also speculated that males may be more motivated to work hard in the presence of female students.22 As in this study, whether a student was assigned a group that was mixed in terms of ethnicity made no significant difference in individual performance. The results using the macroeconomics portion of the course are shown in Table 4. There are again two versions, depending on whether the dependent variable is the student performance on the macroeconomic questions on the final exam (Version A), or whether it is the weighted average of the performance on midterm three and the performance on the macroeconomic question on the final exam (version B). All of the team variables now relate to the characteristics that the student was assigned to for the macroeconomic portion of the course. Further, for the individual variables, the dummy variable for AP credit now relates to whether the student received credit for AP Macro instead of AP Micro, and the number of CLL unit tests passed relates to those for the macroeconomic portion of the course. The most important results in Table 4 indicate that none of the variables describing the macroeconomic team the student was assigned significantly affected individual student performance on macroeconomic exams, under either version of the dependent variable. The F-statistic testing the null hypothesis that all the coefficients for the team variables are zero was .6 for the equation in Table 4 version A, and .60 the equation in Table 4, version B. As for the individual variables, for both versions the coefficient on the dummy variable whether the student received AP credit is larger and even more significant than in the microeconomic estimations in Table 3, while the coefficient on the sum of the SAT scores is slightly smaller and no longer significant at the 10% level, let alone the 5% level as it was in Table 3. As in Table 3, Caucasian students perform better, everything else the same, and the result is significant at the 1% level. However, frosh no longer perform significantly worse, as they did for the microeconomic portion of the course, perhaps because they have become better adjusted to the expectations in the college and the course. Also, the coefficient on the number of CLL unit tests passed is now positive, and significantly different than zero. This could reflect the fact that fewer CLL tests were passed for the macroeconomics portion of the course and there was a higher standard deviation. As for the microeconomic equations, there is no evidence that being an isolated female or non-Caucasian negatively influenced such isolated individuals, although these results are not shown.23 VI. Conclusions This paper has tried to investigate empirically whether the advice from advocates of collaborative learning on how to form small teams of students actually translates into more individual learning. These initial results from the use of randomly selected teams in the introductory course at Occidental College that made extensive use of semi-permanent teams provide very little confirmation at this stage in the investigation. Having a team member who has received AP Economics credit or AP Calculus credit does significantly improve individual exam performance in the microeconomic portion of the course, as predicted from the literature of personnel economics. But there is no empirical verification for the other suggestions, namely to choose teams that are heterogeneous with respect to gender, ethnicity, and class year. As I continue to gather data from future Economics 101 students at Occidental College, it may still turn out that such verification will eventually emerge. Table 1 Definitions of Key Variables Dependent Variable: MICROFGRADE = the score on the final exam questions that related to the material from the microeconomic portion of the course. (There six questions worth ten points each, and one portion of another question worth 3 points. The raw score was converted to the percent correct.) MICROCOPG = the weighted average of the scores on the first two midterm exams and the portion of the final exam questions that related to the microeconomic material. The first midterm counted 8% of the course grade, the second midterm counted 16% since it was cumulative and covered material for the entire first half of the course, and the final exam counted 30%. Thus, the score of the first midterm received a weight of 8/54, with the second midterm receiving a weight of 16/54, and the final exam score receiving a weight of 30/54. Each raw exam score was converted to the percent correct before computing the weighted average.) MACROFGRADE = the score on the final exam questions that related to the material from the macroeconomic portion of the course. (There were four questions worth 10 points each, and a portion of another question worth 7 points.) The raw score was again converted to the percent correct. MACROCOPG = the weighted average of the scores on the third midterm and the portion of the final exam questions that related to the macroeconomic material. The third midterm counted 8% because it was based only on material since the second midterm, while the final exam counted 30%. All raw exam scores were converted to the percent correct before the weighted average was computed.) Independent Variables Relating to the Individual Student: SATSUM = the sum of the math and verbal portions of the SAT. APD = dummy variable equal to 1 if a student received credit for either AP Micro or any AP Calculus; 0 if no AP credit in either. APDMACRO = dummy variable equal to 1 if a student received credit for either AP Macro or any AP Calculus; 0 if no AP credit in either. SEXD = dummy variable equal to 1 if student male; 0 if female. WHITED = dummy variable equal to 1 if student Caucasian; 0 if African-American, Hispanic, Asian/Pacific, or Mixed. CLL = the number of CLL units (0, 1, 2, or 3) the student passed during the microeconomics portion of the course. CLLMACRO = the number of CLL units (0, 1, 2, or 3) the student passed during the macroeconomics portion of the course. Independent Variables Relating to the Team the Student Was Assigned for Micro Portion: TSATAVE = the average of the sum of the verbal and math SAT scores for the members of the team the student was assigned to during the microeconomics portion of the course. TMACROSATSUM = the average of the sum of the verbal and math SAT scores for the members of the team the student was assigned to during the macroeconomics portion of the course. TSATSD = the standard deviation of the SAT score sum for the members of the team the student was assigned to during microeconomic portion of the course. TMACROSATSD = the standard deviation of the SAT score sum for the members of the team the student was assigned to during macroeconomic portion of the course. TMALE (and TMACROMALE) = the proportion of the team members who were male on the team the student was assigned to for the microeconomic (macroeconomic) portion of the course; value varies from 0 to 1. TFROSH (and TMACROFROSH) = the proportion of the team members who were freshman on the team the student was assigned to for the microeconomic (macroeconomic) portion of the course; value varies from 0 to 1. TWHITE (TMACROWHITE) = proportion of team members who are Caucasian on the team the student was assigned to for the microeconomic (macroeconomic) portion of the course; value varies from 0 to 1. HERFETHN (and HERFETHNMACRO) = a substitute variable to the variable immediate above to indicate the extent of ethnic diversity of the team the student was assigned to for the microeconomic (macroeconomic) portion of the course. It is calculated as the Herfindahl Index, in this case equal to the sum of the square of the portions of each ethnicity, and varies from 0 to 10,000. For example, if there were two Caucasians and two Asian students, the value would be (50)2 + (50)2 = 5,000. If all members were Asian, the value would be (100)2 = 10,000. If each member of a three member team was a different ethnicity, then the value would be (33)2 + (33)2 + (33)2 = 3267, etc. TAPD (and TMACROAPD) = a dummy variable equal to 1 if any member of the team the student was assigned to for the microeconomic (macroeconomic) portion of the course had received credit for AP Micro (Macro) or any AP Calculus. TFEMALEATHLETED = a dummy variable equal to 1 if any member of the team was a female competed in intercollegiate athletics during the year. TABLE 2 Descriptive Statistics of Key Variables Individual Variables Mean Stand. Deviation Maximum Minimum Microfgrade 60.6 14.4 89 7 Microcopg 62.8 11.4 89 42 Macrofgrade 58.6 22.4 100 21 Macrocopg 61.0 19.6 94 27 SATSUM 1268.9 120.4 1520 870 APD .29 .46 1 0 APDMACRO .27 .45 1 0 SEXD (male = 1) .78 .42 1 0 FROSHD (freshman = 1) .62 .49 1 0 WHITED (Caucasian = 1) .50 ..50 1 0 CLL 2.5 .88 3 0 CLLMACRO 1.9 1.08 3 0 Team Variables TSATAVE 1269.5 71.9 1378 1135 TMACROSATAVE 1272.9 54.9 1358 1170 TSATSD 85.1 42.7 170 15 TMACROSATSD 97.0 48.7 209 30 TMALE .78 .27 1.0 .25 TMACROMALE .78 .20 1.0 .5 TFROSH .62 .47 1 0 TMACROFROSH .62 .23 1 .25 TWHITE .51 .31 1 0 TMACROWHITE .51 .16 1 .25 HERFETHN 5716 2282 10,000 3227 HERFETHNMACRO 4545 1004 6,250 3750 TAPD .76 .43 1 0 TMACROAPD .68 .47 1 0 TFEMALEATHLETED .24 .43 0 1 Table 3 Estimation of Equation (1) --- Microeconomic Portion Version A: Dependent Variable = Performance on Micro Portion of Final Exam Independent Variable Coefficient t-Statistic (Individual Variables) SATSUM .040 (2.18)** APD 6.15 (1.46)* SEXD -5.23 (-.97) FROSHD -8.83 (-1.97)** WHITED 9.04 (-1.97)** CLL -.59 (-.31) (Variables for Team Individual Assigned) TSATAVE -.026 (-.71) TSATSD .02 (.40) TMALE -4.64 (-.55) TFROSH 9.43 (1.32)* TWHITE 4.08 (.52) TAPD 10.33 (2.12)** Constant 35.79 (1.11) R-squared = .38 F statistic = 2.59 Observations = 63 Table 3 Version B: Dependent Variable = Weighted Ave. of Micro Exams Independent Variable Coefficient t-Statistic (Individual Variables) SATSUM .014 (1.81)** APD 10.01 (3.22)*** SEXD -5.28 (-1.32)* FROSHD -5.72 (-1.72)** WHITED 8.09 (2.38*** CLL -.41 (-.29) (Variables for Team Individual Assigned) TSATAVE -.008 (--.27) TSATSD .001 (.03) TMALE -1.53 (-.25) TFROSH 6.64 (1.25)* TWHITE -2.39 (-.41) TAPD 6.32 (1.76)** Constant 35.79 (1.11) R-squared = .46 F statistic = 3.56 Observations = 63 Table 4 Estimation of Equation (1) Macro Portion Version A: Dependent Variable = Performance on Macro Portion of Final Exam Independent Variable Coefficient t-Statistic (Individual Variables) SATSUM .027 (.99) APDMACRO 26.8 (3.67)*** SEXD 3.8 (.51) FROSHD --4.50 (-.71) WHITED 14.29 (2.37)*** CLLMACRO -.41 (-.29) (Variables for Team Individual Assigned) TMACROSATAVE -.06 (-.92) TMACROSATSD .04 (.73) TMACROMALE 7.28 (.41) TMACROFROSH -2.18 (-.15) MACROWHITE -15.78 (-.74) TMACROAPD -2.71 (.38) Constant 87.5 (1.13) R-squared = .365 F statistic = 2.40 Observations = 63 Table 4 Version B: Dependent Variable = Weighted Average of Macro Exams Independent Variable Coefficient t-Statistic (Individual Variables) SATSUM .025 (1.09) APDMACRO 21.9 (3.55)*** SEXD 3.63 (.57) FROSHD --5.50 (-1.02) WHITED 12.36 (2.42)*** CLLMACRO 3.06 (1.36)* (Variables for Team Individual Assigned) TMACROSATAVE -.06 (-1.0) TMACROSATSD .04 (.79) TMACROMALE 7.40 (.49) TMACROFROSH -5.81 (-.15) MACROWHITE -5.78 (-.48) TMACROAPD -7.91 (-.44) Constant 83.9 (1.28) R-squared = .40 F statistic = 2.82 Observations = 63 Table 5 -- Estimation of Equation 1 with Female Athletes Dependent Variable = Weighted Ave. of Micro Exams Independent Variable Coefficient t-Statistic (Individual Variables) SATSUM .02 (1.72)** APD 10.90 (3.47)*** SEXD -5.68 (-1.43)* FROSHD -5.28 (-1.60)** WHITED 8.20 (2.44)*** CLL -.3941 (-.28) (Variables for Team Individual Assigned) TSATAVE -.009 (-.33) TSATSD -.013 (..32) TMALE 8.34 (-.91) TFROSH 6.2 (1.18) TWHITE -3.49 (-.61) TAPD 3.72 (.94) TFEMALEATHLETED 7.39 (1.46)* Constant 35.79 (1.11) R-squared = .48 F statistic = 3.53 Observations = 63 Footnotes *Elbridge Amos Stuart Professor of Economics, Occidental College, and Director, Center for Teaching Excellence, Occidental College. The author would like to thank his colleagues in the Economics Department at Occidental College for comments on an earlier draft. He alone is responsible for any errors. 1. See the numerous chapters in Becker and Watts, ed., Teaching Economics to Undergraduates: Alternatives to Chalk and Talk, 1998, Elgar Publishing Limited, and Becker, Watts, and Becker, Teaching Economics: More Alternatives to Chalk and Talk. 2006. 2. See Mills, Barbara J. and Philip G. Cottell, Jr., Cooperative Learning for Higher Education Faculty, American Council on Education, 1998, and Johnson, Johnson, and Smith, Active Learning: Cooperation in the College Classroom, Interaction Book Company, 1991. 3. See Bartlett, Robin, The Evolution of Cooperative Learning and Economics Instruction, Chapter 3 in Teaching Economics: More Alternatives to Chalk and Talk., op. cit. Also, for more on the TIP program, see the Report of the Committee on Economic Education, May 2008 Papers and Proceedings of the , pp. 609-10. 4. One important exception is the recent working paper by Hansen, Owan, and Pan, The Impact of Group Diversity on Performance and Knowledge Turnover An Experiment in a College Classroom. NBER Working paper 12251. Their results will be discussed later in this paper. 5. See Mills and Cottell, Jr., op. cit., pp. 50 53. 6. See Hansen, Owan, and Pan, op. cit., pp. 1-2. For the theory of team formation and worker productivity from the field of Personnel Economics, see Lazear, Personnel Economics for Managers, chapter 12. 7. See Hansen, Owan, and Pan, op. cit., p. 2. 8. This borrows from the discussion in Hansen, Owan, and Pan, op. cit., op. 2 3. 9. Hansen, Owan, and Pan, op. cit., section V, B. Using interaction terms, they also found that it was the older students, not the younger students, who benefited from age heterogeneity. They speculate that this may be due to the leadership role taken on by older students, which helps their learning. They also find that the negative effects of male dominate groups decreases as students get older, and that there is more knowledge transfer from stronger students to weaker students. 10. Hansen, Salemi, and Siegfried, Use It or Lose It: Teaching Literacy in the Economics Principles Cousre, Review, May, 2002, p. 463. 11. The chapters assigned from Mankiws Principles of Economics, 4th edition, include 1 11,18, 19, 20, 23, 24, 25, 28, 30, 33, and 34, and portions of several other macro chapters. Nevertheless, the student is not responsible for all of the material in these chapters. Again, the emphasis is on using the supply and demand diagram for welfare analysis, and using the basic AD/AS model to understand fiscal and monetary policy. Several chapters from Baumol and Blinder, 10th edition, are also assigned to supplement the macroeconomic coverage, namely chapters # 27, and 28. 12. Bartlett, Robin, The Evolution of Cooperative Learning and Economics Instruction, in Teaching Economics: More Alternatives to Talk and Chalk, op. cit, chapter # 3. 13. To encourage the teams to stay on task, about once a week students would turn in their answers to a particular in-class exercise/problem, along with a bet of up to ten points on whether they got the answer correct. If the students answer was correct, their bet was added to the ten points they received for showing up that day. If their answer was incorrect, their bet was subtracted from the ten points. Thus, the maximum points on a U-bet exercise was 20 points. Students were also told that if they got the maximum possible number of U-Bet points, up to 4% of their course grade would be recorded as an A, with the remaining 96% based on the grade they would have received without U-Bet. In reality, a student would have to be right on the border between a grade for this to actually change their course grade. Nevertheless, students worked more diligently on the U-Bet problems and were more interested in the discussion of the answer than to other in-class problems on the worksheets. 14. See Moore, Teaching Introductory Economics with a Collaborative Learning Lab Component, Journal of Economic Education, Fall 1998. 15. Five students out of the 63 actually took the ACT instead of the SAT. Their scores were converting to the SAT equivalent of the verbal and math sum using the adjustment provided by ETS ACT-SAT equivalency table based on the results of over 100,000 students who took both. A similar adjustment was used by Hansen, Owan, and Pan. 16. It turned out that only 3 students received course credit for AP Micro, while 15 received course credit for AP Calculus. 17. Another variable representing the extent of ethnic diversity of the group was also used, which was quite similar to the Herfindahl index, but the results did not change; See the later discussion of the results in Table 3. 18. There were a few differences in specification. For example, Hansen, Owan, and Pan used a dummy variable for whether a student was from the School of Engineering, while I used the dummy variable of whether they had received AP Microeconomics credit/AP Calculus. Also, instead of the dummy variable for Frosh vs. Non-Frosh, Hansen, Owan and Pan used a students age. (I had a higher proportion of frosh in my course.) Finally, I used the CLL variable for use as a proxy for student effort. No such variable was available for Hansen, Owan, and Pan. 19. When individual dummy variables for each ethnicity was substituted for the dummy variable for Caucasian vs. non-Caucasian, each of the coefficients for these dummy variables was negative (vs. Caucasians), and the coefficients for Asian/pacific Islanders and Hispanics were significantly different than zero. 20. The CLL resulted in up to 9% of your grade being an A, with the remaining 91% being determined by the grade you received without the CLL unit tests. Thus, a student who was averaging an A or A- without the CLL would not have received any grade boost, while a student who was averaging a C gets a substantial grade boost, enough to increase their course grade by 1/3 of a grade, to a C+. 21. Slightly alternative specifications didnt affect these conclusions. For example, entering the percent white on the team and the percent white squared didnt change the insignificance of this variable. In addition, I substituted a variable to measure ethnic diversity on the team similar to a Herfindahl Index, namely the sum of the squared percent of each ethnicity, which varied from 2500 to 10,000. It, too, was insignificantly different than zero and hardly changed the other coefficients. See Tables 1 and 2 for more on the definition and descriptive statistics for this variable.) 22. See Hansen, Owan, and Pan, op. cit., pp. 17 20. 23. The other coefficients were not significantly changed by the addition of this variable, and the t-statistics were never significant at even the 10% level, and ranged from .02 and .96 for the microeconomic equations and .57 and .24 for the macroeconomic equations. (The sign of the coefficient was negative for the microeconomic equations, as predicted, yet positive for the macroeconomic equations, but since the coefficient is never significantly different than zero, no special significance should be attached to the sign of the coefficient. ) References Bartlett, Robin, The Evolution of Cooperative Learning and Economics Instruction, in Teaching Economics: More Alternatives to Chalk and Talk, 2006. Becker, William, and Michael Watts, ed., Teaching Economics: Alternatives to Chalk and Talk, Elgar Publishing Limited, 1998. Becker, William, et. al., ed., Teaching Economics to Undergraduates: More Alternatives to Chalk, Elgar Publishing Limited, 2006 Hansen, W. Lee, et. al, Use It or Lose It. Teaching Literacy in the Economics Principles Course, Review, May 2002. Hansen, Zeynep, et. al., The Impact of Group Diversity on Performance and Knowledge Spillover An Experiment in a College Classroom, NBER Working Paper 12251, 2006. Johnson, David W., et. al., Active Learning: Cooperative Learning: Increasing College Faculty Instructional Productivity, Interaction Book Company, 1991. Lazear, Edward P., Personnel Economics for Managers, Wiley, 1998. Millis, Barbara J. and Philip G. Cottell, Jr., Cooperative Learning for Higher Education Faculty, American Council on Education, 1998. Moore, Robert L., Teaching Introductory Economics with a Collaborative Learning Component, Journal of Economic Education, Fall 1998. ./78QRSghpqrs   b c d m ! # , 7 8 = W \ ' ( * . ǿ{hzh}o6hxhSh}o6 h}o6hxhSh&66hxhShP6hxhSh_6 hv6 h&66hxhShxhS6 h6hxhShGs6 hW6hZhK$hZ5CJaJhK$h%5CJaJhK$huW`5CJaJh%hzhGs0/RSgh* ZU\gdxhSgduW`gdxhSgdzgd&6$a$gdZ$a$gduW`. 6 H V X t * .57C]kn=W] ,⽵ţ⛓͛͛hzh6hzh_6hzh-6 h[ 6 h,]f6hzh6hzhZ6hzhz6hzhP6 h"p6hzh;6hzhGs6hzhv6hzh}o6 hh?6:,:?DKNjYZrsux~%&')TUuvhV'hV'H*hV'h4 h)H*h)ht\H*h&hW^ht\h)h)H*h)hkhc+hNthWhNtCJaJhWhK$CJaJhWCJaJhWhzhz6hzhGs6hzh_64BC[\j  67M[\r%08cd57Sest%&37DhhUhq6h7Ghq6h+F:H*hq6hq6H*h8<h2Sh%h4hz?h&h+F:hW^L\ \dtZ f }+=DCgdxhSDMUVXYZ_  *+ef 4:|} *+[$%'<=CD^h.$h2Sh/=hnhh6h7Gh7GH*h7Gh_h&huW`h+F:hUhz?L>?BCDNOTU  "%&<Ubcmst      # . / 4 V X Z b ɲ𮪮h$Rh,:qhqhWhqCJaJhWCJaJhWh}nCJaJhK$h .hhz?h2Sh&hNth.$h8<h/=hc+ACO c  . / /!}!!'"}""(#x##1$$$!%x%% &[(gd}ngdxhSgdxhSb d l o .!/!|!}!!!!!!!!&"'"t"u"v"w"x"|"}"""""##'#(#>#?#w#x########0$1$4$G$$$$$$$%%% %!%&%w%x%%%%%%%h.$hah h|H*hNlh,:qH*hNlhNlH*h .h .H*he$>h .hzAPh,:qh$Rh|K%% &((D(G(S([((((((@*B*G*\*j******* ++<+G+H+++++++,,:,;,G,H,j,,,,,,,,,,,-N-O-{---------H.N.O.m.........hZh2nh%hc+h.$hhqhahoh hh hhzAPhzAPh$RL[(((B***H+++H,,,O---O../]////D000<111G2gdxhS.// / /\/]///////////// 00C0D0t0~0000000000;1<1B1C1D1O11111111'2E2F2G2H2J2K2L2N222222%353O3P3W3ɾɶɲ䦪he$>hh(@h`h LhWCJaJhWhK<CJaJhWhz?CJaJhohahaH*hh.$hbIhzhah2nhc+hZAG2H2222P333T444O555Q667Z778c889]99:g::;gdxhSgduW`gdxhSW3^3a3b3s33333333333444455 55'5,5M5N5O5P5Y5Z5\5]5^555555555555G6P6Q66666666677Y7Z77777778888b8c8888899 9 9h7ahc+h[h#hohoH*hK<h3hoh Lh2nh%bhh(@h.$O 9X9\9]9999 ::!:8:E:[:_:f:g:::::::::::;;;;0;F;m;n;u;v;;;<<<-<0<G<L<P<Q<i<k<l<r<<<<<<<<<<<<<<===== =!=\=heh;h;H*h;hxhShhhh(@h7ah7aH*h.$h.$6hzBh.$hK<hf'h}nh7ah[h#G;n;;<m<<!=z==0>>>????G@@@SAAA\BBBBB4CCCgdt 5gdxhSgdxhS\=^=y=z=====>&>->.>/>0>1>;>>>?>>>>>>>>>>>>>>>>:?;?>??????????????%.01ҰѰ۰ܰ԰@@@@@@@@ԯ@@@@鴡A&|c+}$>8.$;;*ezK<OAAAAQB[B\BBBBBBBBBBB C3C4CRCCCCCCCCCDDDDDDDDD.DJDƻzrmrhdhOb[ ht 5H* hf'H*hf'hf'H*hf'hf'OJQJhf'ht 5ht 5H*hWhe$>H*hhht 5hWhUCJaJhWhzBCJaJhWhAACJaJhWh[CJaJhWht 5CJaJhWCJaJh8h#H*h8h8H*h#hK<h8'CD&DDtEF9FFFHGGGGHHH0III5JJJ:KKKHLLLMjMgdxhS$a$gd8JDPDDDDrEtEEEEEEEFFFFF7F8F=F>?ԹɹFFFF.02ҳijGGGGGGGGGұHHHHHHHH.04ﱹ>*/K.<hE.<hE.<H*h3hhH*h3h6hh6H* hh h6hOb[h6hhhOb[hhH*?4I5I7I8IIIII%J.J4J5JJJJJJJJJ9K:KKKKKKKKKKKKLLMMiMjMMMMMNNXNYNxNyNNNNNNN(O)OMONO~OOOOO)P*P{P|Pϻ价价价价hLOhG5hOb[6h>aChOb[h~h>*hdachKh/w hE.<H*hhH*h%&hG5hhhH*h3h3H*h3CjMMNyNN)OOO*P|PP QcQQReRRStSS&TTT*UUU*V[V\VgdxhSgdxhS|PPPPPPPQ QQ5QGQaQbQcQiQQQQQRRdReRnRRRRRSSASBSsStSSSSTT%T&TaTTTTTTTTTTTTTTTTTTTTT)U*UKUiU~UUUZV[V\V^V`VŽhWCJaJhWhCJaJh4lh E`hhLOhfhDLh?h>aChG5hOb[hOb[6hOb[J\VhViVV!WxWW+XJXXYVYYZ^ZZ[m[[#\|\\$]|]],^^^gdJ.gdxhSgd3gdxhS`VhViVnVoVVVVVVVVVVVW W!WNWOWuWvWwWxWWWWWW*X+XIXJXOX_X`XXXYYNYOYPYQYUYVYYYYYYZ]Z^ZZZZZZZZZZZZZZ[[Z[g[k[l[ռĸh$ hlg|h]uh6:H hE.<H*h>aCh>aCH*h@2#h;hhDh>*hIh>aChKhWhCJaJHl[m[q[r[v[[["\#\l\p\{\|\\\\\\\\\#]$]*]/]_]h]{]|]]]]]]]+^,^Q^``` ``````CaDauaaaaa b bAbBbWbXbbbbbbbccc4cEcucvccccc dh]uhu#hJ.h+hhL<h` hE.<H*hE.<hE.<H*hLOh6:Hhlg|h@2#h$ M^0___4```DaaaXbbbcvcc!d~ddetee&fzff*g|gg-hgdxhSgdJ. d!d}d~dddddddddeeXeheieseteeeeeeeeeefff%f&fCfyfzfffff)g*gLgggggg,h-hGhHhfhghkhhhhhhhhhhhhh$iDiEiNi[i\i_ieih%&h$hJ.hlg|hL<hnE=hu h uh u6hyC h uhWL9h`+Nh`h/ 0hu#h]uJ-hhhipii(jzjj.kkk*h hE.<H*h)Sh)SH*h,hnE=h$h`+NNq4rrr*ha?{[{{{{{{C|D|||||S}T}}}~ ~]~^~~~ ghgd6:HgdxhSgdxhS|||||||||'}/}1}Z}z}}}}}}~~ ~W~\~^~m~~~~~~~~ :='23;<=>?+ƾha?, hzq%6hzq% hg5ha?,hzq%5hxhShd96hxhShzq%6 hp6hWhd96hWh6 hW6hWhK$h=hhph5t@2τQɆK҇oƉs%\%bgdxhS$a$gdZgdK$τ؄لDžPQfgdžɆ̆͆KS҇ևJKmnoƉ͉>As%+,Zb̋\ h2k5hVhxhShg6 h @qh @qh2k h @q5h @qh @qh @q5hghg5hg hghghxhSh*66hxhSh6hxhSha?,6hVhzq%5ha?,ha?,hzq%5 hg5hzq%3\ars$%,>EHÍ 8ƎӎՎ؎ڎwאؐABJLRSTabfgvw)1꾺澲ꙕhfD"hfD"hfD"5 h#BH* h#Bh#B h#B6h#Bh#BH*h%&h#Bh2kh2k5 h,h, h,5h,hh5ha?,h2khhV h2k5hVhV5:bEӒԒՒ֒R!@ p͖* gd_gd> $a$gdZ $a$gd> $a$gdZ $a$gd>*gdxhS1ETӒ֒QRtx ,;>?@ )* !$kњҚӚ5Nŵh/'h) h_h>h>h>6 h>h> h>h_h>hOAh_h= h}[)hZ h>6h>hZ6h2k5CJaJhZh2k5CJaJhZ5CJaJhshshs5hfD"4ט=fњҚӚ5k? gd_NO1> ]^UV[MNSӨ"l*+ z hL h{fLhsh{fL h{fL5h'$h{fL5h{fL5CJaJ h65h E`5CJaJhWh E`5 hL h E`h E` h E`5h'$h E`5h'$h E`5CJaJh'$h> h_h>h_h|hlg|h;0h)h*J0^R;٠K3~Ѣ/D&d P gd E`gd E` $a$gd E`gd_DVWXYZH4ѥDަ+wɧ'<NOP&d P gd E`gd E`PQRSӨ0x UP֫W  &d P gd E` $a$gd E`gd E` !"lɭY;/q6 $a$gd E`&d P gd E`gd E`+qԲ$s [ʹgJն &d P gd{fLgd{fL $a$gd6gd'$ GHIQUn{˸ $4BHOWʿ|t|okch .h)6h . hmv6hV'hV'6hV'h)6hmvhmv6hYuh)6hYuhV'6hYuhmv6hmvh) h)h)hvhCJaJhhCJaJh5CJaJh65CJaJh6h65CJaJh .5CJaJh>j5CJaJ h{fL5&IOZͼڽ gVD}Z[FyMgd$ gd^gd/wgd'$gdvgd $a$gd6߹:RXZbxǺٺۺ n̼ͼռ̽ٽڽ 9`egojmSسثطзз؛ؗؓh$:hah hNlh .h .6hxh7GhLhq6h . h .6h .hV'6hGhmv6hGh|hv hD hD hD 6hD hD hD 6hmvhV'h)8SUVZ]_&0=>ѼѼѴՇ{/.<hYhhY(hehe6heh;h;6 hN.6h7ph7p6h7ph;hG h7ah7aH* h7a6h7ah7a6h7ah6hohoho6h3 hG hv hv5 h65 hsW5.FOPjkxyMVWZ[x"/: ,l0HU)+jЯhD h|hPXB6 h|hPXB ht[6ht[ht[6ht[ hPXB6ht[hPXB6hPXBh`h`5CJaJh`hU0h h)ShWhGh7Uh$ h^hl/hE.<h>aC8M!"#$%&'()*+,-./:MUagdt[gdPXB$a$gd`gd`gd$ jkx ;a h`h`ht[ht[6 ht[ht[ h|hxhxhx6hxhPXBht[hPXB6 hPXB6agdt[,1h/ =!"#$% @@@ NormalCJ_HaJmH sH tH Z@Z xhS Heading 1$<@&5CJ KH OJQJ\^JaJ \@\ xhS Heading 2$<@& 56CJOJQJ\]^JaJDAD Default Paragraph FontRiR  Table Normal4 l4a (k(No List2B@2 xhS Body Textx(O( xhSByline4/@4 ZListh^h`L/RSgh* Z  U  \ \ dtZ f }+=DCO c.//}'}(x1!x [ B"""H###H$$$O%%%O&&']''''D(((<)))G*H****P+++T,,,O---Q../Z//0c001]112g223n334m44!5z550666?777G888S999\:::::4;;;;; <]<<<<===:>>>>???+@}@@1AAA4BBBFCiCC DiDDEtEE"FvFFGrGGH]HHIeIIJrJJ#KtKK"L|LLLL M MjMMNsNNNOOOORPPQ[QQRlRRSlSSTyTT+UxUU,VVV8WWWEXXXUYcYYZqZZ![u[[\p\\]q]]^s^^&_Y__`k``auaa0bbb;ccc"d}dd5eee6fff@gggOhhh/iii6jjj@kkklgllmrmmm,nnn?oooLpppNqqqUr}r~rrrrr>s@sssssStTtttuuSuTuuu v vgvhvvvvvvvwSzr{|.}l}}u~~'iȁaȃZvwxybĊ$Np͍9Î#z= tuvؑFdI”LDޖ+|<֘!t™ҙC;כ#t4Νlʞߞ+vӟbC7y<£ãģţUlIޥ%NҦT٧&6K]^_`abΨ+wǩb'p V*xZp} w^Osp;}00000000000000000 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 000:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:000q*0q*0q*0q*0q*0q*0q*0q*0q*0q*0q*0q*0q*0q*0q*0q*0q*0q*0q*0q*0q*0q*0q*0q*0q*0q*0q*0q*0q*0q*0q*0q*0q*0q*0q*0q*0q*0q*0q*0q*0q*0q*0q*0q*0q*0q*0q*0q*000:00:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:00:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:0:000000003L03L03L03L03L03L03L03L03L003L03L03L03L0003L03L03L003L03L03L03L03L03L03L03L03L03L03L03L03L03L03L03L03L03L000000003L000000000003L03L003L03L03L03L03L03L03L003L03L03L03L03L03L03L03L03L03L03L03L03L03L003L03L03L03L03L03L03L03L03L03L03L03L03L03L03L03L03L03L03L03L03L03L003L03L03L03L00000000q0q000q0000q0q0q0q0q0q0q0q0q0q0q000000000u0u0u0u0u0u0u0u0u0u0u0u0u0u0u0u0u0u0u0u0u0u0u0u0u0u0u0u00u0u0u0u0u0u0u0u0u0u0u0u0u0u0u0u0u0u0u0u0u0u0u0u0u0u0u0u0u0u0u0u0u0u0u0u0u0u0u0u0u0u0u0u0u0u0u0u0u000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000u@0u@0u0u0u0u0u0u0u@0u0u0u0u0u0u0u0u@0u0u0u@0u@0u0u0u0u0u0u0u0u0u0u0u0u0u0u0u0u0u0u0u0u0u00000000000000000000000000000000000*/ B"K00K00<@0K000c1J. ,Db %.W3 9\=AJD4I|P`Vl[ deinw|\1N SFjmpqrtuwxz|}\C[(G2;CjM\V^-hq{bDP Mansvy{~o8@0(  B S  ?%9 L: lx ; y <  != !!> !? D!@  A  B 6C t D <{ E Đ F z G ܨ8H !I !J ܦ8K D!L !M $ N |!O | P  Q  R ̬ S !T "U T"V t W  X  Y  !Z te4[ e4\ e4] | H$S$a$a$****a+a+l+P6P6sss%v%v0v,,7       !"#$R$]$j$j$****k+s+s+U6U6sss/v7v7v6>>  !"#$=$*urn:schemas-microsoft-com:office:smarttags PlaceType=%*urn:schemas-microsoft-com:office:smarttags PlaceName9"*urn:schemas-microsoft-com:office:smarttagsplace8#*urn:schemas-microsoft-com:office:smarttagsCity %$#""%%$%"$#"%"$%"$%$%"$"%$"#"#"$"%#"""##G,M,'4/4c8f8;;;;;;;;<<==>>??'(*+OOܰ[cc܆Ċϊ$-ݔٖޖ+.<A֘ܘ!%;?כݛ#&49Νԝӟٟ#bfCK7AyâlrIOޥNZҦܦT^+1wzǩ˩bh'.pv« V\%óGM ciȷhnhl;?BF nq%/jn;ASUsv  Z ^    \ d  \ b dp$t|% fh '}+4=DCN cl/9}'1}(3x~13xQ T ""H#P###H$$$$$$O%S%%%%%O&U&&&' ']'f'''D(J(((<)B)))))K*Q**+P+V++++,,,,-O-`----.Q.S.../ /Z/\///00c0f0001 11122g2k2223%3n3q3334!444z5506;66666?7B77777G8M88888S9Y9999:\:^:::4;:;;;;; <<]<e<<<<=>=====;>C>>>>>????+@3@}@@@@1A3AAAAA4B?BBBBBCC DDiDkDDDEEEE"F'FvFFFFG)GrGvGH H]HbHHHIIeIkIJJrJ|JJJ#K*KtK}KKK#L)L|LLLLjMmMMMN%NsNvNNNOOSOOOOPRP]PPPQQ[QdQQQRRlRrRRRS$SSST*TyT}TTT+U5UxUUUUUU,V3VVVVV8W@٥ۥ:Csu©ĩ^`?rzֳ۳zTr 33333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333fh L*N*::::::MMs^s^RoSo-qCqMqNqXqYqhqhqqqqqqqrr}r~rrrrrvvyyaabb[0}%%&z<#U~6 Y zZFtazƾ^`OJPJQJ^J. ^`hH. pLp^p`LhH. @ @ ^@ `hH. ^`hH. L^`LhH. ^`hH. ^`hH. PLP^P`LhH.808^8`0o(. ^`hH. pLp^p`LhH. @ @ ^@ `hH. ^`hH. L^`LhH. ^`hH. ^`hH. PLP^P`LhH.808^8`0o(. ^`hH. pLp^p`LhH. @ @ ^@ `hH. ^`hH. L^`LhH. ^`hH. ^`hH. PLP^P`LhH.808^8`0o(. ^`hH. pLp^p`LhH. @ @ ^@ `hH. ^`hH. L^`LhH. ^`hH. ^`hH. PLP^P`LhH.808^8`0o(. ^`hH. pLp^p`LhH. @ @ ^@ `hH. ^`hH. L^`LhH. ^`hH. ^`hH. PLP^P`LhH.808^8`0o(. ^`hH. pLp^p`LhH. @ @ ^@ `hH. ^`hH. L^`LhH. ^`hH. ^`hH. PLP^P`LhH.%az#U6 YZFt[0N         2:        T!ni        訪D                \6(        xqW^Wv^ e8Y(|L u G yC p>'$*JYq3gvGsAAamJ.*K$kf~.#[ = $ B fD"@2#u#MT%zq%%& '/'V'f'2(}[)c+a?, .N.?h?h(@A#BPXBzB>aC*F6:HIbI LL{fL`+NzAP Q)S2SxhSmTsW-MZ[Ob[)\ ^ E`uW`7adac,]fj>j4l|l2n}n"p7p,:q @q[\rlsTIt u]umv/wXx"| c|lg|$4~-/%hU0`7Uo,$)vS5Ug`t[9Z7Gb&Nt:/2kD_,3mc_LOl/4t<OADL}z?6ZtYu6+$RmrnOy>*ڵ%h=K8<~_Gs!;q6)Wa}ot\)Vf'G5zPx&N6%3WNlr.Z;%Y$:.$ _D  /=Vi|;0WU|5t@\"cc(;<@@@D@@UnknownGz Times New Roman5Symbol3& z ArialSMS Reference 1Symbol"qh1ʆ:ˆ"f G ojz ojz!24d%%2QKX)?uW`22The Effect of Team Composition on Student Learningmooremoore$      Oh+'0$ 4@ ` l x 4The Effect of Team Composition on Student Learningmoore Normal.dotmoore12Microsoft Office Word@j( @dJ@Y:@$1K o՜.+,D՜.+,` hp|  nazj%' 3The Effect of Team Composition on Student Learning Title<8 #_NewReviewCycle_EmailSubject _AuthorEmail_AuthorEmailDisplayNamePFYI -- Draft of My Working Paper for Econ. Dept. Research Seminar on Oct. 20moore@oxy.eduRobert Moore  !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~ Root Entry Fpm41K Data 1TableWordDocument.LSummaryInformation(DocumentSummaryInformation8CompObjq  FMicrosoft Office Word Document MSWordDocWord.Document.89q