ࡱ> ] bjbjcTcT 4>>D ---AAA8yM,ARy (&(&(& R R R R R R R$VNY/R]-(&(&(&(&(&/RRGGG(&-- RG(& RGGVN@QO!A9N QR0RNxYBYQOQOY-eO(&(&G(&(&(&(&(&/R/RfE\(&(&(&R(&(&(&(&Y(&(&(&(&(&(&(&(&(& : module four, part THREE: sample selection in Economic Education research using STATA Part Three of Module Four provides a cookbook-type demonstration of the steps required to use STATA in situations involving estimation problems associated with sample selection. Users of this model need to have completed Module One, Parts One and Three, but not necessarily Modules Two and Three. From Module One users are assumed to know how to get data into STATA, recode and create variables within STATA, and run and interpret regression results. Module Four, Parts Two and Four demonstrate in LIMDEP (NLOGIT) and SAS what is done here in STATA. THE CASE, DATA, AND ROUTINE FOR EARLY HECKMAN ADJUSTMENT The change score or difference in difference model is used extensively in education research. Yet, before Becker and Walstad (1990), little if any attention was given to the consequence of missing student records that result from: 1) "data cleaning" done by those collecting the data, 2) student unwillingness to provide data, or 3) students self-selecting into or out of the study. The implications of these types of sample selection are shown in the work of Becker and Powers (2001) where the relationship between class size and student learning was explored using the third edition of the Test of Understanding in College Economics (TUCE), which was produced by Saunders (1994) for the National Council on Economic Education (NCEE), since renamed the Council for Economic Education. Module One, Part Three showed how to get the Becker and Powers data set beck8WO.csv into STATA. As a brief review this was done with the insheet command: . insheet a1 a2 x3 c al am an ca cb cc ch ci cj ck cl cm cn co cs ct cu /// > cv cw db dd di dj dk dl dm dn dq dr ds dy dz ea eb ee ef /// > ei ej ep eq er et ey ez ff fn fx fy fz ge gh gm gn gq gr hb /// > hc hd he hf using "F:\BECK8WO2.csv", comma (64 vars, 2849 obs) where A1: term, where 1= fall, 2 = spring A2: school code, where 100/199 = doctorate, 200/299 = comprehensive, 300/399 = lib arts, 400/499 = 2 year hb: initial class size (number taking preTUCE) hc: final class size (number taking postTUCE) dm: experience, as measured by number of years teaching dj: teachers highest degree, where Bachelors=1, Masters=2, PhD=3 cc: postTUCE score (0 to 30) an: preTUCE score (0 to 30) ge: Student evaluation measured interest gh: Student evaluation measured textbook quality gm: Student evaluation measured regular instructors English ability gq: Student evaluation measured overall teaching effectiveness ci: Instructor sex (Male = 1, Female = 2) ck: English is native language of instructor (Yes = 1, No = 0) cs: PostTUCE score counts toward course grade (Yes = 1, No = 0) ff: GPA*100 fn: Student had high school economics (Yes = 1, No = 0) ey: Students sex (Male = 1, Female = 2) fx: Student working in a job (Yes = 1, No = 0) Separate dummy variables need to be created for each type of school (A2), which is done with the following code: recode a2 (100/199=1) (200/299=2) (300/399=3) (400/499=4) generate doc=(a2==1) if a2!=. generate comp=(a2==2) if a2!=. generate lib=(a2==3) if a2!=. generate twoyr=(a2==4) if a2!=. To create a dummy variable for whether the instructor had a PhD we use generate phd=(dj==3) if dj!=. To create a dummy variable for whether the student took the postTUCE we use generate final=(cc>0) if cc!=. To create a dummy variable for whether a student did (noeval = 0) or did not (noeval = 1) complete a student evaluation of the instructor we use generate noeval=(ge + gh + gm + gq == -36) Noeval reflects whether the student was around toward the end of the term, attending classes, and sufficiently motivated to complete an evaluation of the instructor. In the Saunders data set evaluation questions with no answer where coded -9; thus, these four questions summing to -36 indicates that no questions were answered. And the change score is created with generate change=cc-an Finally, there was a correction for the term in which student record 2216 was incorrectly recorded: recode hb (90=89) All of these recoding and create commands are entered into the STATA command file as follows: recode a2 (100/199=1) (200/299=2) (300/399=3) (400/499=4) gen doc=(a2==1) if a2!=. gen comp=(a2==2) if a2!=. gen lib=(a2==3) if a2!=. gen twoyr=(a2==4) if a2!=. gen phd=(dj==3) if dj!=. gen final=(cc>0) if cc!=. gen noeval=(ge+gh+gm+gq==-36) gen change=cc-an recode hb (90=89) To remove records with missing data the following is entered: drop if an==-9 drop if hb==-9 drop if ci==-9 drop if ck==-9 drop if cs==0 drop if cs==-9 drop if a2==-9 drop if phd==-9 The use of these data entry and management commands will appear in the STATA output file for the equations to be estimated in the next section. THE PROPENSITY TO TAKE THE POSTTEST AND THE CHANGE SCORE EQUATION To address attrition-type sample selection problems in change score studies, Becker and Powers first add observations that were dropped during the early stage of assembling data for TUCE III. Becker and Powers do not have any data on students before they enrolled in the course and thus cannot address selection into the course, but to examine the effects of attrition (course withdrawal) they introduce three measures of class size (beginning, ending, and average) and argue that initial or beginning class size is the critical measure for assessing learning over the entire length of the course. To show the effects of initial class size on attrition (as discussed in Module Four, Part One) they employ what is now the simplest and most restrictive of sample correction methods, which can be traced to James Heckman (1979), recipient of the 2000 Nobel Prize in Economics. From Module Four, Part One, we have the data generating process for the difference between post and preTUCE scores for the ith student ( EMBED Equation.DSMT4 ):  EMBED Equation.DSMT4  (1) where the data set of explanatory variables is matrix X, where Xi is the row of xji values for the relevant variables believed to explain the ith students pretest and posttest scores, the EMBED Equation.3 s are the associated slope coefficients in the vector  EMBED Equation.3 , and  EMBED Equation.3  is the individual random shock (caused, for example, by unobservable attributes, events or environmental factors) that affect the ith students test scores. Sample selection associated with students unwillingness to take the postteest (dropping the course) results in population error term and regressor correlation that biases and makes coefficient estimators in this change score model inconsistent. The data generating process for the  EMBED Equation.3  students propensity to take the posttest is:  EMBED Equation.3  (2) where  EMBED Equation.3 , if  EMBED Equation.3 , and student  EMBED Equation.3  has a posttest score, and  EMBED Equation.3 , if  EMBED Equation.3 , and student  EMBED Equation.3  does not have a posttest score.  EMBED Equation.3 is the vector of all students propensities to take a posttest.  EMBED Equation.3  is the matrix of explanatory variables that are believed to drive these propensities.  EMBED Equation.3  is the vector of slope coefficients corresponding to these observable variables.  EMBED Equation.3 is the vector of unobservable random shocks that affect each students propensity. The effect of attrition between the pretest and posttest, as reflected in the absence of a posttest score for the ith student  EMBED Equation.3  and a Heckman adjustment for the resulting bias caused by excluding those students from the change-score regression requires estimation of equation (2) and the calculation of an inverse Mills ratio for each student who has a pretest. This inverse Mills ratio is then added to the change-score regression (1) as another explanatory variable. In essence, this inverse Mills ratio adjusts the error term for the missing students. For the Heckman adjustment for sample selection each disturbance in vector  EMBED Equation.3 , equation (1), is assumed to be distributed bivariate normal with the corresponding disturbance term in the  EMBED Equation.3  vector of the selection equation (2). Thus, for the  EMBED Equation.3  student we have:  EMBED Equation.3  bivariate normal EMBED Equation.3  (3) and for all perturbations in the two-equation system we have:  EMBED Equation.3  (4) That is, the disturbances have zero means, unit variance, and no covariance among students, but there is covariance between selection in getting a posttest score and the measurement of the change score. The regression for this censored sample of  EMBED Equation.3  students who took the posttest is now:  EMBED Equation.DSMT4  , for  EMBED Equation.3  (5) which suggests the Heckman adjusted regression to be estimated:  EMBED Equation.DSMT4  (6) where  EMBED Equation.3  is the inverse Mills ratio (or hazard) such that  EMBED Equation.3 , and  EMBED Equation.3  and  EMBED Equation.3  are the normal density and distribution functions.  EMBED Equation.3  is the standardized mean of the disturbance term  EMBED Equation.3 , for the  EMBED Equation.3 student who took the posttest; it is close to zero only for those well above the  EMBED Equation.3  threshold. The values of  EMBED Equation.3  are generated from the estimated probit selection equation (2) for all students. STATAs built-in heckman command estimates both the selection and outcome equation using either the full-information maximum likelihood or Heckmans original two-step estimator (which uses the Mills ratio as a regressor). The default heckman command implements the maximum likelihood estimation, including  EMBED Equation.3 and  EMBED Equation.3 , and is written: heckman change hb doc comp lib ci ck phd noeval, /// select (final = an hb doc comp lib ci ck phd noeval) vce(opg) while the Mills ratio two-step process can be implemented by specifying the option twostep after the command. The option vce(opg) specifies the outer-product of the gradient method to estimate standard errors, as opposed to STATAs default Hessian method. As described in Module One, Part Three, entering all of these commands into the command window in STATA and pressing enter (or alternatively, highlighting the commands in a do file and pressing ctrl-d) yields the following output file: . insheet /// > A1 A2 X3 C AL AM AN CA CB CC CH CI CJ CK CL CM CN CO CS CT /// > CU CV CW DB DD DI DJ DK DL DM DN DQ DR DS DY DZ EA EB EE EF /// > EI EJ EP EQ ER ET EY EZ FF FN FX FY FZ GE GH GM GN GQ GR HB /// > HC HD HE HF /// > using "C:\BECK8WO.csv", comma (64 vars, 2837 obs) . recode a2 (100/199=1) (200/299=2) (300/399=3) (400/499=4) (a2: 2837 changes made) . gen doc=(a2==1) if a2!=. . gen comp=(a2==2) if a2!=. . gen lib=(a2==3) if a2!=. . gen twoyr=(a2==4) if a2!=. . gen phd=(dj==3) if dj!=. . gen final=(cc>0) if cc!=. . gen noeval=(ge+gh+gm+gq==-36) . gen change=cc-an . recode hb (90=89) (hb: 96 changes made) . drop if an==-9 | hb==-9 | ci==-9 | ck==-9 | cs==0 | cs==-9 | a2==-9 | phd==-9 (250 observations deleted) . heckman change hb doc comp lib ci ck phd noeval, select (final = an hb doc comp lib ci ck phd noeval) vce(opg) Iteration 0: log likelihood = -6826.563 Iteration 1: log likelihood = -6826.4685 Iteration 2: log likelihood = -6826.4674 Iteration 3: log likelihood = -6826.4674 Heckman selection model Number of obs = 2587 (regression model with sample selection) Censored obs = 510 Uncensored obs = 2077 Wald chi2(8) = 211.39 Log likelihood = -6826.467 Prob > chi2 = 0.0000 ------------------------------------------------------------------------------ | OPG | Coef. Std. Err. z P>|z| [95% Conf. Interval] -------------+---------------------------------------------------------------- change | hb | -.0097802 .0055923 -1.75 0.080 -.0207408 .0011805 doc | 1.997291 .5534814 3.61 0.000 .912487 3.082094 comp | -.361983 .4332653 -0.84 0.403 -1.211167 .4872015 lib | 2.23154 .505341 4.42 0.000 1.24109 3.22199 ci | .3940114 .2533859 1.55 0.120 -.1026158 .8906386 ck | -2.743372 .3803107 -7.21 0.000 -3.488767 -1.997976 phd | .6420888 .2896418 2.22 0.027 .0744013 1.209776 noeval | -.6320101 1.269022 -0.50 0.618 -3.119248 1.855227 _cons | 6.817536 .7238893 9.42 0.000 5.398739 8.236332 -------------+---------------------------------------------------------------- final | an | .0227793 .009396 2.42 0.015 .0043634 .0411953 hb | -.0048868 .0020624 -2.37 0.018 -.008929 -.0008447 doc | .9715436 .150756 6.44 0.000 .6760672 1.26702 comp | .4043055 .1443272 2.80 0.005 .1214295 .6871815 lib | .5150521 .1908644 2.70 0.007 .1409648 .8891394 ci | .1992685 .0905382 2.20 0.028 .0218169 .37672 ck | .0859013 .1190223 0.72 0.470 -.1473781 .3191808 phd | -.1320764 .0978678 -1.35 0.177 -.3238939 .059741 noeval | -1.929021 .0713764 -27.03 0.000 -2.068916 -1.789126 _cons | .9901789 .240203 4.12 0.000 .5193897 1.460968 -------------+---------------------------------------------------------------- /athrho | .0370755 .3578813 0.10 0.917 -.6643589 .73851 /lnsigma | 1.471813 .0160937 91.45 0.000 1.44027 1.503356 -------------+---------------------------------------------------------------- rho | .0370585 .3573898 -.581257 .6282441 sigma | 4.357128 .0701223 4.221836 4.496756 lambda | .1614688 1.55763 -2.89143 3.214368 ------------------------------------------------------------------------------ LR test of indep. eqns. (rho = 0): chi2(1) = 0.03 Prob > chi2 = 0.8612 ------------------------------------------------------------------------------ The above output provides maximum likelihood estimation of both the probit equation and the change score equation with separate estimation of EMBED Equation.3 and  EMBED Equation.3 . The bottom panel provides the probit coefficients for the propensity equation, where it is shown that initial class size is negatively and significantly related to the propensity to take the posttest with a one-tail p value of 0.009. The tob panel gives the change score results, where initial class size is negatively and significantly related to the change score with a one-tail p value of 0.04. Again, it takes approximately 100 students to move the change score in the opposite direction by a point. Alternatively, the following command estimates the Heckman model using the Mills ratio as a regressor: . heckman change hb doc comp lib ci ck phd noeval, select (final = an hb doc comp lib ci ck phd noeval) twostep Heckman selection model -- two-step estimates Number of obs = 2587 (regression model with sample selection) Censored obs = 510 Uncensored obs = 2077 Wald chi2(16) = 931.46 Prob > chi2 = 0.0000 ------------------------------------------------------------------------------ | Coef. Std. Err. z P>|z| [95% Conf. Interval] -------------+---------------------------------------------------------------- change | hb | -.0102219 .0056305 -1.82 0.069 -.0212575 .0008137 doc | 2.079684 .5764526 3.61 0.000 .9498578 3.20951 comp | -.329457 .4426883 -0.74 0.457 -1.19711 .5381962 lib | 2.274478 .5373268 4.23 0.000 1.221337 3.327619 ci | .4082326 .2592943 1.57 0.115 -.0999749 .9164401 ck | -2.730737 .377552 -7.23 0.000 -3.470725 -1.990749 phd | .6334483 .2910392 2.18 0.030 .063022 1.203875 noeval | -.8843357 1.272225 -0.70 0.487 -3.377851 1.60918 _cons | 6.741226 .7510686 8.98 0.000 5.269159 8.213293 -------------+---------------------------------------------------------------- final | an | .022039 .0094752 2.33 0.020 .003468 .04061 hb | -.0048826 .0019241 -2.54 0.011 -.0086537 -.0011114 doc | .9757148 .1463617 6.67 0.000 .6888511 1.262578 comp | .4064945 .1392651 2.92 0.004 .13354 .679449 lib | .5214436 .1766459 2.95 0.003 .175224 .8676632 ci | .1987315 .0916865 2.17 0.030 .0190293 .3784337 ck | .08779 .1342874 0.65 0.513 -.1754085 .3509885 phd | -.133505 .1030316 -1.30 0.195 -.3354433 .0684333 noeval | -1.930522 .0723911 -26.67 0.000 -2.072406 -1.788638 _cons | .9953498 .2432624 4.09 0.000 .5185642 1.472135 -------------+---------------------------------------------------------------- mills | lambda | .4856741 1.596833 0.30 0.761 -2.644061 3.61541 -------------+---------------------------------------------------------------- rho | 0.11132 sigma | 4.3630276 lambda | .48567415 1.596833 ------------------------------------------------------------------------------ The estimated probit model (in the bottom portion of the above output) is Estimated propensity to take the posttest = 0.995 + 0.022(preTUCE score) " 0 .005(initial class size) + 0.976(Doctoral Institution) + 0.406 (Comprehensive Institution) + 0.521(Liberal Arts Institution) + 0.199 (Male instructor) + 0.0878(English Instructor Native Language) " 0.134(Instructor has PhD ) " 1.930(No Evaluation of Instructor) The beginning or initial class size is negatively and highly significantly related to the propensity to take the posttest, with a one-tail p value of 0.011. The corresponding change-score equation employing the inverse Mills ratio is in the upper portion of the above output: Predicted Change = 6.741 " 0.010(initial class size) + 2.080(Doctoral Institution) " 0.329 (Comprehensive Institution) + 2.274 Liberal Arts Institution) + .408(Male instructor) " 2.731(English Instructor Native Language) + 0.633(Instructor has PhD) " 0.88434(No Evaluation of Instructor) + 0 .486 EMBED Equation.DSMT4  The change score is negatively and significantly related to the class size, with a one-tail p value of 0.0345, but it takes an additional 100 students to lower the change score by a point. An Application of Propensity Score Matching Unfortunately, we are not aware of a study in economic education for which propensity score matching has been used. Thus, we looked outside economic education and elected to redo the example reported in Becker and Ichino (2002). This application and data are derived from Dehejia and Wahba (1999), whose study, in turn was based on LaLonde (1986). The data set consists of observed samples of treatments and controls from the National Supported Work demonstration. Some of the institutional features of the data set are given by Becker and Ichino. The data were downloaded from the website  HYPERLINK "http://www.nber.org/~rdehejia/nswdata.html" http://www.nber.org/~rdehejia/nswdata.html. The data set used here is in the original text form, contained in the data file matchingdata.txt. They have been assembled from the several parts in the NBER archive. Becker and Ichino report that they were unable to replicate Dehejia and Wahbas results, though they did obtain similar results. (They indicate that they did not have the original authors specifications of the number of blocks used in the partitioning of the range of propensity scores, significance levels, or exact procedures for testing the balancing property.) In turn, we could not precisely replicate Becker and Ichinos results we can identify the reason, as discussed below. Likewise, however, we obtain similar results. There are 2,675 observations in the data set, 2,490 controls (with t = 0) and 185 treated observations (with t = 1). The variables in the raw data set are t = treatment dummy variable age = age in years educ = education in years black = dummy variable for black hisp = dummy variable for Hispanic marr = dummy variable for married nodegree = dummy for no degree (not used) re74 = real earnings in 1974 re75 = real earnings in 1975 re78 = real earnings in 1978 the outcome variable We will analyze these data following Becker and Ichinos line of analysis. We assume that you have completed Module One, Part Three, and thus are familiar with placing commands in the command window or in a do file. In what follows, we will simply show the commands you need to enter into STATA to produce the results that we will discuss. First, note that STATA does not have a default command available for propensity score matching. Becker and Ichino, however, have created the user-written routine pscore that implements the propensity score matching analysis underlying Becker and Ichino (2002). As described in the endnotes of Module Two, Part Three, users can install the pscore routine by typing findit pscore into the command window, where a list of information and links to download this routine appears. Click on one of the download links and STATA automatically downloads and installs the routine for use. Users can then access the documentation for this routine by typing help pscore. Installing the pscore routine also downloads and installs several other routines useful for analyzing treatment effects (i.e., the routines attk, attnd and attr, discussed later in this Module). To begin the analysis, the data are imported by using the command (where the data file is on the C drive but your data could be placed wherever): insheet /// t age educ black hisp marr nodegree re74 re75 re78 /// using "C:\matchingdata.txt" Transformed variables added to the equation are age2 = age squared educ2 = educ squared re742 = re74 squared re752 = re75 squared blacku74 = black times 1(re74 = 0) In order to improve the readability of some of the reported results, we have divided the income variables by 10,000. (This is also an important adjustment that accommodates a numerical problem with the original data set. This is discussed below.) The outcome variable is re78. The data are set up and described first. The transformations used to create the transformed variables are gen age2=age^2 gen educ2=educ^2 replace re74=re74/10000 replace re75=re75/10000 replace re78=re78/10000 gen re742=re74^2 gen re752=re75^2 gen blacku74=black*(re74==0) global X age age2 educ educ2 marr black hisp re74 re75 re742 re752 blacku74 The data are described with the following statistics: . sum Variable | Obs Mean Std. Dev. Min Max -------------+-------------------------------------------------------- t | 2675 .0691589 .2537716 0 1 age | 2675 34.22579 10.49984 17 55 educ | 2675 11.99439 3.053556 0 17 black | 2675 .2915888 .4545789 0 1 hisp | 2675 .0343925 .1822693 0 1 -------------+-------------------------------------------------------- marr | 2675 .8194393 .3847257 0 1 nodegree | 2675 .3330841 .4714045 0 1 re74 | 2675 1.823 1.372225 0 13.71487 re75 | 2675 1.785089 1.387778 0 15.66532 re78 | 2675 2.050238 1.563252 0 12.11736 -------------+-------------------------------------------------------- age2 | 2675 1281.61 766.8415 289 3025 educ2 | 2675 153.1862 70.62231 0 289 re742 | 2675 5.205628 8.465888 0 188.0976 re752 | 2675 5.111751 8.908081 0 245.4024 blacku74 | 2675 .0549533 .2279316 0 1 We next fit the logit model for the propensity scores. An immediate problem arises with the data set as used by Becker and Ichino. The income data are in raw dollar terms the mean of re74, for example is $18,230.00. The square of it, which is on the order of 300,000,000, as well as the square of re75 which is similar, is included in the logit equation with a dummy variable for Hispanic which is zero for 96.5% of the observations and the blacku74 dummy variable which is zero for 94.5% of the observations. Because of the extreme difference in magnitudes, estimation of the logit model in this form is next to impossible. But rescaling the data by dividing the income variables by 10,000 addresses the instability problem. These transformations are shown in the replace commands above. This has no impact on the results produced with the data, other than stabilizing the estimation of the logit equation. The following command estimates the logit model from which the propensity scores are obtained and tests the balancing hypothesis. The logit model from which the propensity scores are obtained is fit using: . global X age age2 educ educ2 marr black hisp re74 re75 re742 re752 blacku74 . pscore t $X, logit pscore(_pscore) blockid(_block) comsup where the logit option specifies that propensity scores should be estimated using the logit model, the blockid and pscore options define two new variables created by STATA representing each observations propensity score and block id, and the comsup option restricts the analysis to observations in the common support. (Note: Becker and Ichinos coefficients on re74 and re75 are multiplied by 10,000, and coefficients on re742 and re752 are multiplied by 100,000,000. Otherwise, the output presented here matches that of Becker and Ichino) . pscore t $X, logit pscore(_pscore) blockid(_block) comsup ** Algorithm to estimate the propensity score ** The treatment is t t | Freq. Percent Cum. ------------+----------------------------------- 0 | 2,490 93.08 93.08 1 | 185 6.92 100.00 ------------+----------------------------------- Total | 2,675 100.00 Estimation of the propensity score Iteration 0: log likelihood = -672.64954 Iteration 1: log likelihood = -506.34385 Iteration 2: log likelihood = -385.59357 Iteration 3: log likelihood = -253.47057 Iteration 4: log likelihood = -239.00944 Iteration 5: log likelihood = -216.46206 Iteration 6: log likelihood = -209.42835 Iteration 7: log likelihood = -205.15188 Iteration 8: log likelihood = -204.97706 Iteration 9: log likelihood = -204.97537 Iteration 10: log likelihood = -204.97536 Iteration 11: log likelihood = -204.97536 Logistic regression Number of obs = 2675 LR chi2(12) = 935.35 Prob > chi2 = 0.0000 Log likelihood = -204.97536 Pseudo R2 = 0.6953 ------------------------------------------------------------------------------ t | Coef. Std. Err. z P>|z| [95% Conf. Interval] -------------+---------------------------------------------------------------- age | .3316903 .1203299 2.76 0.006 .0958482 .5675325 age2 | -.0063668 .0018554 -3.43 0.001 -.0100033 -.0027303 educ | .849268 .3477058 2.44 0.015 .1677771 1.530759 educ2 | -.0506202 .0172493 -2.93 0.003 -.0844282 -.0168122 marr | -1.885542 .2993309 -6.30 0.000 -2.472219 -1.298864 black | 1.135972 .3517854 3.23 0.001 .4464852 1.825459 hisp | 1.96902 .5668594 3.47 0.001 .857996 3.080044 re74 | -1.058961 .3525178 -3.00 0.003 -1.749883 -.3680387 re75 | -2.168541 .4142324 -5.24 0.000 -2.980422 -1.35666 re742 | .2389164 .0642927 3.72 0.000 .112905 .3649278 re752 | .0135926 .0665375 0.20 0.838 -.1168185 .1440038 blacku74 | 2.14413 .4268152 5.02 0.000 1.307588 2.980673 _cons | -7.474743 2.443511 -3.06 0.002 -12.26394 -2.68555 ------------------------------------------------------------------------------ Note: 22 failures and 0 successes completely determined Note: the common support option has been selected The region of common support is [.00061066, .97525407] Description of the estimated propensity score in region of common support Estimated propensity score ------------------------------------------------------------- Percentiles Smallest 1% .0006426 .0006107 5% .0008025 .0006149 10% .0010932 .0006159 Obs 1342 25% .0023546 .000618 Sum of Wgt. 1342 50% .0106667 Mean .1377463 Largest Std. Dev. .2746627 75% .0757115 .974804 90% .6250822 .9749805 Variance .0754396 95% .949302 .9752243 Skewness 2.185181 99% .970598 .9752541 Kurtosis 6.360726 The next set of results summarizes the tests of the balancing hypothesis. By specifying the detail option in the above pscore command, the routine will also report the separate results of the F tests within the partitions as well as the details of the full partition itself. The balancing hypothesis is rejected when the p value is less than 0.01 within the cell. Becker and Ichino do not report the results of this search for their data, but do report that they ultimately found seven blocks. They do not report the means by which the test of equality is carried out within the blocks or the critical value used. **** Step 1: Identification of the optimal number of blocks Use option detail if you want more detailed output **** The final number of blocks is 7 This number of blocks ensures that the mean propensity score is not different for treated and controls in each blocks *** Step 2: Test of balancing property of the propensity score Use option detail if you want more detailed output *** Variable black is not balanced in block 1 The balancing property is not satisfied Try a different specification of the propensity score Inferior | of block | t of pscore | 0 1 | Total -----------+----------------------+---------- 0 | 924 7 | 931 .05 | 102 4 | 106 .1 | 56 7 | 63 .2 | 41 28 | 69 .4 | 14 21 | 35 .6 | 13 20 | 33 .8 | 7 98 | 105 -----------+----------------------+---------- Total | 1,157 185 | 1,342 Note: the common support option has been selected *** End of the algorithm to estimate the pscore *** The final portion of the pscore output presents the blocks used for the balancing hypothesis. Again, specifying the detail option will report the results of the balancing property test for each of the independent variables, which are excluded here for brevity. This part of the analysis also recommends that the analyst reexamine the specification of the propensity score model. Because this is not a numerical problem, the analysis continues with estimation of the average treatment effect on the treated. The first example below shows estimation using the kernel estimator to define the counterpart observation from the controls and using only the subsample in the common support. This stage consists of nboot + 1 iterations. In order to be able to replicate the results, we set the seed of the random number generator before computing the results: set seed 1234579 attk re78 t $X, pscore(_pscore) bootstrap comsup reps(25) Recall, we divided the income values by 10,000. The value of .153795 reported below thus corresponds to $1,537.95. Becker and Ichino report a value (see their section 6.4) of $1,537.94. Using the bootstrap replications, we have estimated the asymptotic standard error to be $856.28. A 95% confidence interval for the treatment effect is computed using $1537.95 1.96(856.27) = (-$229.32,$3,305.22). . attk re78 t $X, pscore(_pscore) bootstrap comsup reps(25) The program is searching for matches of each treated unit. This operation may take a while. ATT estimation with the Kernel Matching method --------------------------------------------------------- n. treat. n. contr. ATT Std. Err. t --------------------------------------------------------- 185 1157 0.154 . . --------------------------------------------------------- Note: Analytical standard errors cannot be computed. Use the bootstrap option to get bootstrapped standard errors. Bootstrapping of standard errors command: attk re78 t age age2 educ educ2 marr black hisp re74 re75 re742 re752 blacku74 , pscore(_pscore) comsup bwidth(.06) statistic: attk = r(attk) note: label truncated to 80 characters Bootstrap statistics Number of obs = 2675 Replications = 25 ------------------------------------------------------------------------------ Variable | Reps Observed Bias Std. Err. [95% Conf. Interval] -------------+---------------------------------------------------------------- attk | 25 .1537945 -.0050767 .0856277 -.0229324 .3305215 (N) | -.0308111 .279381 (P) | -.0308111 .2729317 (BC) ------------------------------------------------------------------------------ Note: N = normal P = percentile BC = bias-corrected ATT estimation with the Kernel Matching method Bootstrapped standard errors --------------------------------------------------------- n. treat. n. contr. ATT Std. Err. t --------------------------------------------------------- 185 1157 0.154 0.086 1.796 --------------------------------------------------------- Note that the estimated asymptotic standard error is somewhat different. As we noted earlier, because of differences in random number generators, the bootstrap replications will differ across programs. It will generally not be possible to exactly replicate results generated with different computer programs. With a specific computer program, replication is obtained by setting the seed of the random number generator. (The specific seed chosen is immaterial, so long as the same seed is used each time.) The next set of estimates is based on all of the program defaults. The single nearest neighbor is used for the counterpart observation; 25 bootstrap replications are used to compute the standard deviation, and the full range of propensity scores (rather than the common support) is used. Intermediate output is also suppressed. Once again, we set the seed for the random number generator before estimation. In this case, the pscore calculation is not used, and we have instead estimated the nearest neighbor matching and the logit propensity scores in the same command sequence by specifying the logit option rather than the pscore option. Skipping the pscore routine essentially amounts to ignoring any test of the balancing hypothesis. For the purposes of this Module, this is a relatively innocuous simplification, but in practice, the pscore routine should always be used prior to estimating the treatment effects. . attnd re78 t $X, logit bootstrap reps(25) The program is searching the nearest neighbor of each treated unit. This operation may take a while. ATT estimation with Nearest Neighbor Matching method (random draw version) Analytical standard errors --------------------------------------------------------- n. treat. n. contr. ATT Std. Err. t --------------------------------------------------------- 185 57 0.167 0.211 0.789 --------------------------------------------------------- Note: the numbers of treated and controls refer to actual nearest neighbour matches Bootstrapping of standard errors command: attnd re78 t age age2 educ educ2 marr black hisp re74 re75 re742 re752 blacku74 , pscore() logit statistic: attnd = r(attnd) note: label truncated to 80 characters Bootstrap statistics Number of obs = 2675 Replications = 25 ------------------------------------------------------------------------------ Variable | Reps Observed Bias Std. Err. [95% Conf. Interval] -------------+---------------------------------------------------------------- attnd | 25 .1667644 .012762 .1160762 -.0728051 .4063339 (N) | -.1111108 .3704965 (P) | -.1111108 .2918935 (BC) ------------------------------------------------------------------------------ Note: N = normal P = percentile BC = bias-corrected ATT estimation with Nearest Neighbor Matching method (random draw version) Bootstrapped standard errors --------------------------------------------------------- n. treat. n. contr. ATT Std. Err. t --------------------------------------------------------- 185 57 0.167 0.116 1.437 --------------------------------------------------------- Note: the numbers of treated and controls refer to actual nearest neighbour matches Using the full sample in this fashion produces an estimate of $1,667.64 for the treatment effect with an estimated standard error of $1,160.76. In comparison, using the 1,342 observations in their estimated common support, and the same 185 treated observations, Becker and Ichino reported estimates of $1,667.64 and $2,113.59 for the effect and the standard error, respectively and use 57 of the 1,342 controls as nearest neighbors. The next set of results uses the radius form of matching and again restricts attention to the estimates in the common support. . attr re78 t $X, logit bootstrap comsup radius(0.0001) reps(25) The program is searching for matches of treated units within radius. This operation may take a while. ATT estimation with the Radius Matching method Analytical standard errors --------------------------------------------------------- n. treat. n. contr. ATT Std. Err. t --------------------------------------------------------- 23 66 -0.555 0.239 -2.322 --------------------------------------------------------- Note: the numbers of treated and controls refer to actual matches within radius Bootstrapping of standard errors command: attr re78 t age age2 educ educ2 marr black hisp re74 re75 re742 re752 blacku74 , pscore() logit comsu > p radius(.0001) statistic: attr = r(attr) note: label truncated to 80 characters Bootstrap statistics Number of obs = 2675 Replications = 25 ------------------------------------------------------------------------------ Variable | Reps Observed Bias Std. Err. [95% Conf. Interval] -------------+---------------------------------------------------------------- attr | 25 -.554614 -.0043318 .5369267 -1.662776 .5535483 (N) | -1.64371 .967416 (P) | -1.357991 .967416 (BC) ------------------------------------------------------------------------------ Note: N = normal P = percentile BC = bias-corrected ATT estimation with the Radius Matching method Bootstrapped standard errors --------------------------------------------------------- n. treat. n. contr. ATT Std. Err. t --------------------------------------------------------- 23 66 -0.555 0.537 -1.033 --------------------------------------------------------- Note: the numbers of treated and controls refer to actual matches within radius The estimated treatment effects are now very different. We see that only 23 of the 185 treated observations had a neighbor within a range (radius in the terminology of Becker and Ichino) of 0.0001. Consistent with Becker and Ichinos results, the treatment effect is estimated to be -$5,546.14 with a standard error of $5,369.27. Becker and Ichino state that that these nonsensical results illustrate both the differences in caliper vesus radius matching as well as the sensitivity of the estimator to the choice of radius. In order to implement a true caliper matching process, the user-written psmatch2 routine should be used. After installing the psmatch2 routine, caliper matching with logit propensity scores and common support can be implemented with the following command: . psmatch2 t $X, common logit caliper(0.0001) outcome(re78) Logistic regression Number of obs = 2675 LR chi2(12) = 935.35 Prob > chi2 = 0.0000 Log likelihood = -204.97536 Pseudo R2 = 0.6953 ------------------------------------------------------------------------------ t | Coef. Std. Err. z P>|z| [95% Conf. Interval] -------------+---------------------------------------------------------------- age | .3316903 .1203299 2.76 0.006 .0958482 .5675325 age2 | -.0063668 .0018554 -3.43 0.001 -.0100033 -.0027303 educ | .849268 .3477058 2.44 0.015 .1677771 1.530759 educ2 | -.0506202 .0172493 -2.93 0.003 -.0844282 -.0168122 marr | -1.885542 .2993309 -6.30 0.000 -2.472219 -1.298864 black | 1.135972 .3517854 3.23 0.001 .4464852 1.825459 hisp | 1.96902 .5668594 3.47 0.001 .857996 3.080044 re74 | -1.058961 .3525178 -3.00 0.003 -1.749883 -.3680387 re75 | -2.168541 .4142324 -5.24 0.000 -2.980422 -1.35666 re742 | .2389164 .0642927 3.72 0.000 .112905 .3649278 re752 | .0135926 .0665375 0.20 0.838 -.1168185 .1440038 blacku74 | 2.14413 .4268152 5.02 0.000 1.307588 2.980673 _cons | -7.474743 2.443511 -3.06 0.002 -12.26394 -2.68555 ------------------------------------------------------------------------------ Note: 22 failures and 0 successes completely determined. There are observations with identical propensity score values. The sort order of the data could affect your results. Make sure that the sort order is random before calling psmatch2. -------------------------------------------------------------------------------------- Variable Sample | Treated Controls Difference S.E. T-stat ----------------------------+--------------------------------------------------------- re78 Unmatched | .634914353 2.1553921 -1.52047775 .115461434 -13.17 ATT | .672171543 .443317968 .228853575 .438166333 0.52 ----------------------------+--------------------------------------------------------- Note: S.E. for ATT does not take into account that the propensity score is estimated. psmatch2: | psmatch2: Common Treatment | support assignment | Off suppo On suppor | Total -----------+----------------------+---------- Untreated | 0 2,490 | 2,490 Treated | 162 23 | 185 -----------+----------------------+---------- Total | 162 2,513 | 2,675 The difference column in the ATT row of the above results presents the estimated treatment effect. Using a true caliper matching process, the estimates of $2,228.85 and $4,381.66 for the effect and the standard error, respectively, are much more comparable to the results previously obtained. Concluding comments Results obtained from the two equation system advanced by Heckman over 30 years ago are sensitive to the correctness of the equations and their identification. On the other hand, methods such as the propensity score matching depend on the validity of the logit or probit functions estimated along with the methods of getting smoothness in the kernel density estimator. Someone using Heckmans original selection adjustment method can easily have their results replicated in LIMDEP, STATA and SAS. Such is not the case with propensity score matching. Propensity score matching results are highly sensitive to the computer program employed while Heckmans original sample selection adjustment method can be relied on to give comparable results across programs. References Becker, William and William Walstad. Data Loss From Pretest to Posttest As a Sample Selection Problem, Review of Economics and Statistics, Vol. 72, February 1990: 184-188, Becker, William and John Powers. Student Performance, Attrition, and Class Size Given Missing Student Data, Economics of Education Review, Vol. 20, August 2001: 377-388. Becker, S. and A. Ichino, Estimation of Average Treatment Effects Based on Propensity Scores, The Stata Journal, Vol. 2, November 2002: 358-377. Deheija, R. and S. Wahba Causal Effects in Nonexperimental Studies: Reevaluation of the Evaluation of Training Programs, Journal of the American Statistical Association, Vol. 94, 1999: 1052-1062. Heckman, James. Sample Bias as a Specific Error. Econometrica, Vol. 47, 1979: 153-162. Huynh, Kim, David Jacho-Chavez, and James K. Self.The Efficacy of Collaborative Learning Recitation Sessions on Student Outcomes? Review, (Forthcoming May 2010). LaLonde, R., Evaluating the Econometric Evaluations of Training Programs with Experimental Data, Review, Vol. 76, 4, 1986, 604-620. Saunders, Phillip. The TUCE III Data Set: Background information and file codes (documentation, summary tables, and five 3.5-inch double-sided, high density disks in ASCII format). New York: National Council on Economic Education, 1994. EndnoteS     Ian McCarthy, 3-30-2010  PAGE \* MERGEFORMAT 1  Huynh, Jacho-Chavez, and Self (2010) have a data set that enables them to account for selection into, out of and between collaborative learning sections of a large principles course in their change-score modeling.  Users can also estimate the logit model with STATAs default logit command. The predicted probabilities from the logit estimation are equivalent to the propensity scores automatically provided with the pscore command. Since STATA does not offer any default matching routine to use following the default logit command, we adopt the use of the pscore routine (the download of which includes several matching routines to calculate treatment effects). The pscore routine also tests the balancing hypothesis and provides other relevant information for propensity score matching which is not provided by the default logit command.  The Kernel density estimator is a nonparametric estimator. Unlike a parametric estimator (which is an KQVWX]bqr 9 U k n q r ̼߫}oaSESaah CJOJQJ^JaJh CJOJQJ^JaJh6CJOJQJ^JaJhUiCJOJQJ^JaJhs\hp&CJOJQJaJhetTCJOJQJ^JaJ hs\hI9zCJOJQJ^JaJ hs\hp&CJOJQJ^JaJh_GS5;CJOJQJ\aJ%h hp&5;CJOJQJ\aJ"h hp&5;CJOJQJaJh_GS5;CJOJQJaJ,WX nU1 @ ^@ `gdis^gdisgdisd7$8$H$^gdV?BCDEYZ[\efghopʷh CJOJQJaJh CJOJPJQJ^JaJhVOPRпq`O`> hhhzCJOJQJ^JaJ hhh CJOJQJ^JaJ hhhdCJOJQJ^JaJhhCJOJQJ\^JaJh CJOJQJ^JaJ hhhhCJOJQJ^JaJ#hhhhCJOJQJ\^JaJhdCJOJQJ^JaJ hbKhdCJOJQJ^JaJ h^h^CJOJQJ^JaJh<8CJOJQJ^JaJ h^h<8CJOJQJ^JaJ=>PQR7Rk# p@ P pd7$8$H$^pgdUcd7$8$H$gd:Rd7$8$H$gddd7$8$H$gdhgdbK޾ޭrddSE9hetTCJOJQJaJhV/jsN h^EhjCJEHOJQJUVaJ h^EhjCJEHOJQJaJ)jh^EhjCJEHOJQJUaJ)j] h^EhjCJEHOJQJUaJ/jsN h^EhjCJEHOJQJUVaJ h^EhjCJEHOJQJaJh^EhjCJOJQJaJhjCJOJQJaJ)jh^EhjCJEHOJQJUaJ)j_h^EhjCJEHOJQJUaJ$$$$$A%B%C%U%V%W%X%\%_%i%%%%%&&ƺƺթ|պpdƺUEh^Eh^6CJOJQJaJh^Eh^CJOJQJaJhbKCJOJQJaJh CJOJQJaJ)j&$h^EhjCJEHOJQJUaJ/jsN h^EhjCJEHOJQJUVaJ h^EhjCJEHOJQJaJhjCJOJQJaJh^EhjCJOJQJaJ)jh^EhjCJEHOJQJUaJ)jA"h^EhjCJEHOJQJUaJ&!&"&*&+&=&>&?&@&O&&&&&&'D'F'c''''' (Ϻ|pdXpIpXdpXp=Xh-eCJOJQJaJh^EhjCJOJQJaJhRCJOJQJaJh CJOJQJaJhjCJOJQJaJ)j &h^Eh^CJEHOJQJUaJ/jsN h^Eh^CJEHOJQJUVaJ h^Eh^CJEHOJQJaJ)jh^Eh^CJEHOJQJUaJh^Eh^CJOJQJaJh^Eh^CJH*OJQJaJ"h^Eh^6CJH*OJQJaJ (((0(3(>(?(Q(R(S(T((((((((( )) )!)ijijnYijA/jsN h^Eh^CJEHOJQJUVaJ)j *h^Eh^CJEHOJQJUaJ/jsN h^Eh^CJEHOJQJUVaJ)j$(h^Eh^CJEHOJQJUaJ/jsN h^Eh^CJEHOJQJUVaJ h^Eh^CJEHOJQJaJ)jh^Eh^CJEHOJQJUaJhRCJOJQJaJh^Eh^CJOJQJaJh|CJOJQJaJ!)")#)4)5)<)=)>)P)Q)R)S)d)e)w)x)y)ƷƫmXƖ@+)j 0h^Eh^CJEHOJQJUaJ/jsN h^Eh^CJEHOJQJUVaJ)j-h^Eh^CJEHOJQJUaJ/jsN h^Eh^CJEHOJQJUVaJ h^Eh^CJEHOJQJaJ)jh^Eh^CJEHOJQJUaJh^CJOJQJaJh^EhbKCJOJQJaJh^Eh^CJOJQJaJ)jh^Eh^CJEHOJQJUaJ)j+h^Eh^CJEHOJQJUaJy)z))))))))))))****** +겚p_G2p)j6h^Eh^CJEHOJQJUaJ/jsN h^Eh^CJEHOJQJUVaJ h^Eh^CJEHOJQJaJ)jh^Eh^CJEHOJQJUaJ)jN2h^Eh CJEHOJQJUaJ/j0O h^Eh CJEHOJQJUVaJ h^Eh^CJEHOJQJaJhbKCJOJQJaJh^CJOJQJaJh^Eh^CJOJQJaJ)jh^Eh^CJEHOJQJUaJ ++!+%+(+)+?+@+A+B+I+J+\+]+^+_+o+p+++п}lT?}33hRCJOJQJaJ)j4=h^Eh^CJEHOJQJUaJ/jsN h^Eh^CJEHOJQJUVaJ h^Eh^CJEHOJQJaJ)jh^Eh^CJEHOJQJUaJ)j8h^Eh|CJEHOJQJUaJ/jsN h^Eh|CJEHOJQJUVaJ h^Eh|CJEHOJQJaJ)jh^Eh|CJEHOJQJUaJh^Eh^CJOJQJaJh-eCJOJQJaJ++++++++++++++++++++ijzijbM>2hV,?,@,F,G,Y,Z,[,\,a,b,t,пЃn]E0nn])jHh^Eh^CJEHOJQJUaJ/jsN h^Eh^CJEHOJQJUVaJ h^Eh^CJEHOJQJaJ)jh^Eh^CJEHOJQJUaJh^Eh^CJOJQJaJ)jEh^Eh^CJEHOJQJUaJ/jsN h^Eh^CJEHOJQJUVaJ h^Eh^CJEHOJQJaJ)jh^Eh^CJEHOJQJUaJhVX>>>E???2@@@AnAA B BBD2E`gdZ[Q4 J(P x$@)h.38=C0HXMRW\a gHlpqv{d7$8$H$gdp2 dgdhm$BBBBBBBBBBBBDD­}lT?}0h-hZ[QCJOJQJaJ)j+\h^EhZ[QCJEHOJQJUaJ/jsN h^EhZ[QCJEHOJQJUVaJ h^EhZ[QCJEHOJQJaJ)jh^EhZ[QCJEHOJQJUaJh^EhZ[QCJOJQJaJhZ[QCJOJQJaJ)jh^EhZ[QCJEHOJQJUaJ)jHZh^EhZ[QCJEHOJQJUaJ/jsN h^EhZ[QCJEHOJQJUVaJ h^EhZ[QCJEHOJQJaJ D2EEEPPP PVPPPPQ Q Q.QLQPQRQTQVQXQ`QQQQpdXppIh_T2h)gCJOJQJaJh_T2CJOJQJaJhtCJOJQJaJh^X hMRCJOJQJaJh^X h)gCJOJQJaJh_T2hMRCJOJQJaJh1*zh)g6CJOJQJaJhMRh)gCJOJQJaJhRCJOJQJaJhZ[QCJOJQJaJh0rh0rCJOJQJaJ h0rh0rCJOJQJ^JaJh0rCJOJQJaJ2EEEEAFFFF/G0GGGH,H{HHIhIIJUJJJBKQKKK>LL dgd0rgd0rLL+MzMMNgNNNOcO}OOOPPPPPPPQQ^RRxS ^`gd1*z^gd1*zgdp2 dgd0rQQQQQQQR&RZR^RnRRRRRRRRRS&S(S*S,S2S }CJOJQJaJhYf6CJOJQJaJhYfCJOJQJaJhetT6CJOJQJaJhetTCJOJQJaJhUih)[CJOJQJaJhWICJOJQJaJ"eee$f%fTfUfVfifjffffffffff/g0gEg`gjgggggggghQhRhShGiHiȹȂvȹl_h!jR5OJ QJ \^J h!jROJQJ^JhQCJOJQJaJhWIhl~CJOJQJaJ hWIhWIhX# hWIh!jR hWIhX#hWIhX#CJOJQJaJhWIh!jRCJOJQJaJhWIh29CJOJQJaJhYfCJOJQJ^JaJhWIhPCJOJQJaJhWICJOJQJaJ$iffffffgRhShbhshhhhhhhFiGiHi p@ P d7$8$H$gdYf$d7$8$H$a$gdX#`gdWI^gdWIHiIiiiiiiiiiij`jjj5k|kk lQlll&mmmmd7$8$H$]gdYf`gdWId7$8$H$]gdl~HiIiJiri~iiiinnn*o+ooooooooooooopͻsdsdsUsUsUsFsdshcfhcCJOJQJaJhcfh)[CJOJQJaJhcfhX#CJOJQJaJhcfh!jRCJOJQJaJh!jR5OJ QJ \^J #hYfhYfCJOJQJ\^JaJhYf5OJ QJ \^J hX#5OJ QJ \^J #hX#hX#CJOJ QJ \^J aJhcfCJOJQJaJhWICJOJQJaJhWIhX#CJOJQJaJhWIOJ QJ \^J mmBnnnnnnfr6s7sssssu 7$8$H$]gd p@ P d7$8$H$gd+H%`gd`gdcfd7$8$H$]^gd!jRd7$8$H$]gdl~d7$8$H$]gdYfppPpQpppppppppq1q2qDqFqGqHqiqqqqqqqqqqqRrerfrrrrss⬸ӐӄĸĄxl`lhcCJOJQJaJhl~CJOJQJaJhCJOJQJaJh^dCJOJQJaJh^d6CJOJQJaJhcfhM,CJOJQJaJhQCJOJQJaJhM,CJOJQJaJhcfhl~CJOJQJaJhcfhX#CJOJQJaJhcfh!jRCJOJQJaJhcfhACJOJQJaJ%s's3s4s5s6s7s9ssssssss*t1t6t?d7$8$H$]gd=}d7$8$H$]gdad7$8$H$]gd jo`gd;i p@ P d7$8$H$gd\o&$d7$8$H$]a$gd jo`gd ;<LP_`x}ǐ'hn@Az~#ƼƲƨƲƲƨƼڲڄqaqhEoCJOJPJQJ^JaJ$h=}h=}CJOJPJQJ^JaJh\o&CJOJPJQJ^JaJh9OJQJ^JhX \OJQJ^Jh joOJQJ^Jh=}OJQJ^Jh\o&OJQJ^Jh!jROJQJ^Jh;iOJQJ^Jh.OJQJ^Jh\o&CJOJQJ^JaJh=}CJOJQJ^JaJ&?@Aqr ![\ϓ ./ԔK3d7$8$H$]gd=}3і oҗ 8UVʘ?@z{|}~d7$8$H$]gdad7$8$H$]gd=}#BIJKowƚ{|ћқ12Ü'-ES[ҝ؝ Ǟ̧̳͞uhhhhq f6OJPJQJ^Jha6OJPJQJ^Jhq fOJPJQJ^Jh5l{6OJPJQJ^Jh5l{OJPJQJ^JhX \OJPJQJ^Jh9h joOJQJ^JhOJPJQJ^Jh joOJPJQJ^Jh!jROJQJ^Jh joOJQJ^Jh.OJQJ^Jh9OJQJ^J'~|DEF4 J(P x$@)h.38=C0HXMRW\a gHlpqv{d7$8$H$gd=}4 J(P x$@)h.38=C0HXMRW\a gHlpqv{d7$8$H$gd`gd=}`gd9fghӧAm<Bξξξξ|pdpdpdpdpXphOJPJQJ^Jhq fOJPJQJ^JhOJPJQJ^Jh9OJPJQJ^JhEoCJOJPJQJ^JaJ$h=}hEoCJOJPJQJ^JaJ$h=}hq fCJOJPJQJ^JaJh=}CJOJPJQJ^JaJ$h=}h=}CJOJPJQJ^JaJhq fCJOJPJQJ^JaJhq fh=}OJPJQJ^JTȠɠ>x(Lstâ4 J(P x$@)h.38=C0HXMRW\a gHlpqv{d7$8$H$gd=}bI6Jb~Х(b4 J(P x$@)h.38=C0HXMRW\a gHlpqv{d7$8$H$gd=}bצئLfghyd7$8$H$gd`gd9gdq f4 J(P x$@)h.38=C0HXMRW\a gHlpqv{d7$8$H$gd4 J(P x$@)h.38=C0HXMRW\a gHlpqv{d7$8$H$gd=} ҩ۩ܩݩߩ ֱױ19iqŲo\PCPCPohP6OJPJQJ^JhPOJPJQJ^J$hSWhEoCJOJPJQJ^JaJhEoCJOJPJQJ^JaJhSWCJOJPJQJ^JaJ$hSWhSWCJOJPJQJ^JaJhPCJOJPJQJ^JaJ$hvKhvKCJOJPJQJ^JaJ hPhvKCJOJQJ^JaJhSWCJOJQJ^JaJhvKCJOJQJ^JaJhPCJOJQJ^JaJܩݩީߩ%GHIxΪuuuuuuuuuuu4 J(P x$@)h.38=C0HXMRW\a gHlpqv{d7$8$H$gdSW4 J(P x$@)h.38=C0HXMRW\a gHlpqv{d7$8$H$gdvK p@ P d7$8$H$gdP BC}~ 784 J(P x$@)h.38=C0HXMRW\a gHlpqv{d7$8$H$gdSW X@ޮ-|˯߯Dab4 J(P x$@)h.38=C0HXMRW\a gHlpqv{d7$8$H$gdSWְKLֱױS dgdP`gdEogd?4 J(P x$@)h.38=C0HXMRW\a gHlpqv{d7$8$H$gdSW23ѵ op]J7չ$sº`' dgdP']LQ;iƿ#RS~gd? dgdPּR~pz GHtz³zl^^P^hCJOJQJ^JaJhgCJOJQJ^JaJhQCJOJQJ^JaJh5CJOJQJ^JaJh}CJOJQJ^JaJhdCJOJQJ^JaJh9;CJOJQJ^JaJhEo;CJOJQJ^JaJhP;CJOJQJ^JaJhPOJPJQJ^JhPCJOJPJQJ^JaJ$hPhPCJOJPJQJ^JaJ /Tdhmst^_`qsǵ{o_PPP_Ph,h1CJOJQJaJh`hh16CJOJQJaJh`hCJOJQJaJh5h16CJOJQJaJh^Eh1CJOJQJaJh1CJOJQJaJh?;CJOJQJ^JaJ#hmb@h?;CJOJQJ^JaJh;CJOJQJ^JaJhCJOJQJ^JaJh3nCJOJQJ^JaJhgCJOJQJ^JaJTYk=>?@gd9 dgd|`gdgdR$a$gdM%gd1gd?sx{ =?DHNYcgi 7ʾʾʮ٢ٓٓ٢ٓrdrh?CJOJQJ^JaJ hs\h?CJOJQJ^JaJhe&h16CJOJQJaJhoh1CJOJQJaJhe&CJOJQJaJh`hhM%6CJOJQJaJhM%CJOJQJaJhhM%CJOJQJaJh1CJOJQJaJh,h1CJOJQJaJh`hCJOJQJaJ 7OcdjkܽwiiZK9#hmb@h?;CJOJQJ^JaJh;CJOJQJ^JaJh;CJOJQJ^JaJhe&CJOJQJ^JaJh?CJOJQJ^JaJ h2jOh?CJOJQJ^JaJh1hRCJOJQJaJhe&CJOJQJaJh1CJOJQJaJ h1hRCJOJQJ^JaJh?CJOJQJ^JaJ hs\h?CJOJQJ^JaJ#hRh?6CJOJQJ^JaJ!"9:;<?@ABHi ¹үʧtdth+H%h+H%6CJOJQJaJh+H%h+H%CJOJQJaJh+H%jh+H%0JUh9hQOJQJh)&hQOJQJhQOJQJjhQ0JUho~mHnHujhQUhQhUch}afjh}afU#hmb@h;CJOJQJ^JaJh5;CJOJQJ^JaJ#B=ɺ|zk__[I#hmb@h;CJOJQJ^JaJh}afhCJOJQJaJh9hCJOJQJaJU"hL(h6CJOJQJ]aJh6CJOJQJ]aJhL(hCJOJQJaJhjh0JUh+H%hCJOJQJaJhh+H%CJOJQJaJh6CJOJQJaJh+H%h+H%CJOJQJaJhCJOJQJaJgdgdequation), a non-parametric estimator has no fixed structure and is based on a histogram of all the data. Histograms are bar charts, which are not smooth, and whose shape depends on the width of the bin into which the data are divided. In essence, with a fixed bin width, the kernel estimator smoothes out the histogram by centering each of the bins at each data point rather than fixing the end points of the bin. The optimum bin width is a subject of debate and well beyond the technical level of this module. 21h:pf'/ =!"#$% Dd hb  c $A? ?3"`?2c ƅ nJxr:?D`!7 ƅ nJxr:$@h|xRAkQy6 즵P1C- A]`m &d# !o=yX(xٛ`yIQ*\vޛ73 $ɡ4Tixa+ R$jb,VȺC OHjt+ǷJGO@GFAh: ;y?lBaV}paZnݏߗM@FPΟ-a 9s> }XsWٲ˗H_-Z= -uuo'KF:=s3- 'kd -8"B< RnSųy XT֑Wh${0i!t /5 >PDd L  C A?"?2 |J:3ڹR]`! |J:3ڹR`TRxUMhA~3Imښ?XjZCڃ4V!KLZ7MAh M( B{$^ڃCࡢ&l  {{C}av/nJJZ*d9(2rm ]%~f׸G@T3xsw٤-ki(N7.~YwgK11eٮ%b8|^YNoG4"aįe׈{>n;𪷡ǰهcb"Aɻ{"n1%m$ò<<Fpmץ3 +qK N)B\$Vy2sAhC݃{9Y/?O@섢xX e1)Ƌ3x "9P\'m? .!`#<߮T7ogVj;<5zCThYP*,^z3i߯ob@º`HP4q_v0텦v^I*FifӳA޻"xlfr:eZSqyp&= DwЄ =jz "{7g vqGf.gqιk_# JeDd@L  C A?"?2?=68.> 7j `!=68.> 7j:@sx=N a=ߡLVŰ#y/@b1(jwRc@(e3|u9= FI2Hc#Pu*DX2EymX2ҝ$HE{tWo!ma &;gfhvpb"֔Ԧ97%KL}yXHI{+}t\(c#}=H{݅5<@DdhL  C A?"?2pdNZy % mr_L `!DdNZy % mr_@S|xmPJA}3lXb_p?g+D8 VVAʅٙy @b7Bk"R -2KZK^[ͥC|gu "[ZBwZq4U sYMZWDa'lp-hqjޣy/y2'lt3= n.A7uu~C6P'sIy` dfߟ<}+@YNE L4swz/WV? ͤEzkMw3B3f44LdxwAc9Hk<|rfL?MmLvsмuz`aWO!]Az\͡mI'$ n*=ꚎKF!wL-|Ddx|L  C A?"?2  !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~.Root Entry F4GData $bWordDocument4ObjectPoolT0m4_1322056929wF0m0mOle CompObjiObjInfo  %*-058;@EHKPUZ_dgjoty| FMathType 6.0 Equation MathType EFEquation.DSMT49qt%PRD%DSMT6WinAllBasicCodePagesTimes New RomanSymbolCourier NewMT Extra!/ED/APG_APAPAE%B_AC_AE*_HA@AHA*_D_E_E_A  Dy iEquation Native _1321956218 F0m0mOle CompObj i FMathType 6.0 Equation MathType EFEquation.DSMT49qdPRDDSMT6WinAllBasicCodePagesTimes New RomanSymbolCourier NewMT Extra!/ED/APG_APAPAE%B_AC_AE*_HA@AHA*_D_E_E_A  Dy i =ObjInfo Equation Native  _1321956219 F0m0mOle =X i ++e i ==b 1 ++pb j x jij==2k " ++e i FMicrosoft Equation 3.0 DS Equation Equation.39qCompObjfObjInfoEquation Native 6_1321956221F0m0m0  j FMicrosoft Equation 3.0 DS Equation Equation.39q q  FMicrosoft Equation 3.0 DS EqOle CompObjfObjInfoEquation Native )_1321956222"F0m0mOle CompObjfObjInfo!uation Equation.39q  i FMicrosoft Equation 3.0 DS Equation Equation.39q4M i thEquation Native "6_1321956223F0m0mOle #CompObj $fObjInfo!&Equation Native ':_1321956224'$F0m0mOle ( FMicrosoft Equation 3.0 DS Equation Equation.39qOl T i* =H i + i FMicrosoft Equation 3.0 DS EqCompObj#%)fObjInfo&+Equation Native ,k_1321956225)F0m0mOle .CompObj(*/fObjInfo+1Equation Native 2?uation Equation.39q#J T i =1 FMicrosoft Equation 3.0 DS Equation Equation.39q( T i* >_1321956226@.F0m0mOle 3CompObj-/4fObjInfo06Equation Native 7D_13219562273F0m0mOle 9CompObj24:f0 FMicrosoft Equation 3.0 DS Equation Equation.39q ȶ i FMicrosoft Equation 3.0 DS EqObjInfo5<Equation Native =)_13219562281;8F0m0mOle >CompObj79?fObjInfo:AEquation Native B?_1321956229=F0m0muation Equation.39q# T i =0 FMicrosoft Equation 3.0 DS Equation Equation.39q(ķ T i* d"Ole CCompObj<>DfObjInfo?FEquation Native GD0 FMicrosoft Equation 3.0 DS Equation Equation.39q ȶ i FMicrosoft Equation 3.0 DS Eq_13219562306JBF0m0mOle ICompObjACJfObjInfoDLEquation Native M)_1321956231GF0m0mOle NCompObjFHOfuation Equation.39qؑ T * FMicrosoft Equation 3.0 DS Equation Equation.39q tt HObjInfoIQEquation Native R6_1321956232EOLF0m0mOle SCompObjKMTfObjInfoNVEquation Native W)_1321956233QF0m0mOle XCompObjPRYfObjInfoS[Equation Native \) FMicrosoft Equation 3.0 DS Equation Equation.39q \$8  FMicrosoft Equation 3.0 DS Equation Equation.39q_1321956234,VF0m0mOle ]CompObjUW^fObjInfoX`   FMicrosoft Equation 3.0 DS Equation Equation.39q+,ID}I (T i =0)Equation Native a)_1321956235[F0m0mOle bCompObjZ\cfObjInfo]eEquation Native fH_1321956237Yc`F0m0mOle h FMicrosoft Equation 3.0 DS Equation Equation.39q xDA  FMicrosoft Equation 3.0 DS Equation Equation.39qCompObj_aifObjInfobkEquation Native l)_1321956238eF0m0mOle mCompObjdfnfObjInfogpEquation Native q) `G  FMicrosoft Equation 3.0 DS Equation Equation.39q8 i th_1321956239^rjF0m0mOle rCompObjiksfObjInfoluEquation Native v:_1321956240oF0m0mOle wCompObjnpxf FMicrosoft Equation 3.0 DS Equation Equation.39q="L ( i , i )~ FMicrosoft Equation 3.0 DS Equation Equation.39q,     ! "$#%'&(*)+-XY/0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVW[Z\_`abcdefghijklmnopqrstuvwxyz{|}~ObjInfoqzEquation Native {Y_1321956241m|tF0m0mOle }CompObjsu~fObjInfovEquation Native `_1328601344yF0m0m+DIxI (0,0,  ,1,) FMathType 6.0 Equation MathType EFEquation.DSMT49qs(|PRDSMT6WinAllBasicCodePagesOle CompObjxziObjInfo{Equation Native DTimes New RomanSymbolCourier NewMT Extra!/ED/APG_APAPAE%B_AC_AE*_HA@AHA*_D_E_E_A   e`E()==E()==0,E(')==s   e`e  e`2I,E(')==I,andE(ew')==rs   e`e  e`I. FMicrosoft Equation 3.0 DS Equation Equation.39q"M n T=1 FMathType 6.0 Equation MathTy_1321956243~F0m0mOle CompObj}fObjInfoEquation Native >_1321956244hF0m0mOle CompObjipe EFEquation.DSMT49qL%PRD%DSMT6WinAllBasicCodePagesTimes New RomanSymbolCourier NewMT Extra!/ED/APG_APAPAE%B_AC_AE*_HA@AHA*_D_E_E_A  E(Dy i |X i ,T i ==1)==X i ++E(ObjInfoEquation Native 9_1321956245F0m0mOle e i |T i* >>0);i==1,2,...n T==1 FMicrosoft Equation 3.0 DS Equation Equation.39q+4 n T=1 <CompObjfObjInfoEquation Native G_1321956246F0m0mN FMathType 6.0 Equation MathType EFEquation.DSMT49q T%PRD%DSMT6WinAllBasicCodePagesTimes New RomanSymbolCourier NewMT Extra!/ED/APG_APAPAE%B_AC_AE*_HA@AHA*_D_E_EOle CompObjiObjInfoEquation Native &_A  E(Dy i |X i ,T i ==1)==X i ++(rs e )l i ;i==1,2,...n T==1 FMicrosoft Equation 3.0 DS Eq_1321956247F0m0mOle CompObjfObjInfouation Equation.39q̳  i FMicrosoft Equation 3.0 DS Equation Equation.39q+IԃI  i =f(Equation Native 6_1321956248F0m0mOle CompObjfObjInfoEquation Native _1321956249F0m0mOle "T i* )/[1"F("T i* )] FMicrosoft Equation 3.0 DS Equation Equation.39q+pyIăI f(.)CompObjfObjInfoEquation Native 8_1321956250F0m0mOle CompObjfObjInfoEquation Native 8 FMicrosoft Equation 3.0 DS Equation Equation.39q+pyIăI F(.) FMicrosoft Equation 3.0 DS Equation Equation.39q_1321956251F0m0mOle CompObjfObjInfoEquation Native 6_1321956252F0m0mOle CompObjf̳  i FMicrosoft Equation 3.0 DS Equation Equation.39q  i FMicrosoft Equation 3.0 DS EqObjInfoEquation Native 6_1321956253F0m0mOle CompObjfObjInfoEquation Native :_1321956254F0m0muation Equation.39qQ i th FMicrosoft Equation 3.0 DS Equation Equation.39ql T=1Ole CompObjfObjInfoEquation Native 1_1321956255F0m0mOle CompObjfObjInfo FMicrosoft Equation 3.0 DS Equation Equation.39q ?l  FMicrosoft Equation 3.0 DS Equation Equation.39qEquation Native )_1321956256F0m0mOle CompObjfObjInfoEquation Native )_1321956257F0m0mOle  ̳  FMicrosoft Equation 3.0 DS Equation Equation.39qHd  CompObjfObjInfoEquation Native 6_1321956259F0m0mOle CompObjfObjInfoEquation Native ) FMicrosoft Equation 3.0 DS Equation Equation.39q ̳  FMicrosoft Equation 3.0 DS Equation Equation.39q_1321956260F0m0mOle CompObjfObjInfoEquation Native 6_1321956258F0m0mOle CompObjiHd   FMathType 6.0 Equation MathType EFEquation.DSMT49qt%PRD%DSMT6WinAllBasicCodePagesTimes New RomanSymbolCourier NewMT Extra!/ED/APG_APAPAE%B_AC_AE*_HA@AHA*_D_E_EObjInfoEquation Native 1Table^ ZSummaryInformation(     _A  lOh+'0\x   $ 0<DLT,MODULE FOUR, PART TWO: SAMPLE SELECTION Bill BeckerNormal Bill Becker2Microsoft Office Word@F#@B@B[mE+ȝkD*`!mE+ȝkD*z`& +~xRMKQ=j9B⦈IEE{eZ6 $4( -vaBiBMP\}Aͼ{s}*āȣ+13N)s ЫZ s.I%4 @cW- Xd|"$qP (i.YGƍVRŊc׬C۵NA[lɱkx6:K]jǬzJvMѼEo /I9co|7vL$~Fxqj:y ϩ$q~LAn{m/;8"~Capr^2~x_LK'E aӑ>(LmG&p|OztXCDRy*DdlhL  C A?"?2g<;'&ulxf"`!^g<;'&ulx@9|,xQJA}; i!6 -} xFH. E,bH>2fK* fwͬAxezEcF,sc* {u+kk3x3&$ *N[=7׵$E$gqw ^t@)pɺ#[{8:\BjR/q}>3w䢺6#+U?''&;1۸z 1" `q)$ ?! !9%;37oVx[ +TFyQԪ \bwWְ @'_-05vIqOG4g iocٻW>ēy۶ ydEۜBI.x>&<=KrrV)nRC ۫ EӱaJMcw,׬l(~QaDdL   C A ?"? 2E q`HF`l!`! q`HF`l:Zxcdd`` $X bbd12,(ㆫaZA( TA?d=@v+0n RXRY`7S?$L ZZǰ@pT9H P.T mrof%v Hj@vF&&\"CX1 v<^l,3H)f;Dd@,L  C A?"? 2^Fݰ,q:`!2Fݰ,qhxOJQ=3l.BHqB I*$lgmeg_b-$W -3כ8Μ4E"T$D ֚/v?EV@̺d6%Jz"Z`2Lp~B=Rq2ұLswu n0}%8A#νnY|g9IJ.^fcV]]'if{x'm[SwͨX+KDdL  C A?"?2D͓Fh# L!  `!͓Fh# L!:xcdd`` $X bbd12,(ㆫaZQ( TA?dAi@ UXRY7S?$ ZZǰ@pT9 P.T mĕh0xTrIK.1@`` v021)WAYa;v}2 ;DdL  C A?"?2E}n\`t_fxq!"`!}n\`t_fxq:@`s xcdd`` $X bbd12,(ㆫaJ`t`3H1g`Yj@hPs 5< %! `35)d`2w! @Hfnj_jBP~nbS },@@S9\@d % 1 `F&&\ "CX1?> 1<0|m>u->L{E0~V.w|Ja©f6Ƹ`^S^3ژuӦ7lߑx/5Fݑ Q=<ͪ < LDdL  C A?"?2E9@04 ل!!h(`!9@04 ل!: `;!xcdd`` $X bbd12,(ㆫaJ`T`3H1g`Yj@  5< %! `35J YB2sSRs=^~*c*@}Z$Ap]c!``dbR ,.I r1(2t΃!v0es9DdL  C A?"?2C#EAa52@Kԋ5M*`!#EAa52@Kԋ5:`Sw!xcdd`` $X bbd12,(ㆫaJ`l`3H1g`Yj@hPsC0&dT20( {@.dabM-VK-WMc~x V8pPh*6b`cg% v0l#RpeqIj.HA v<^l,3X?H=8yDd@L  C A?"?2[HWήe70,`!/HWήexP=kA}3~O!"!8S3P]qag*?!"ɯ7>}+@YNE L4swz/WV? ͤEzkMw3B3f44LdxwAc9Hk<|rfL?MmLvsмuz`aWO!]Az\͡mI'$ n*=ꚎKF!wL-&DdhL  C A?"?2S>?Rz{j2>b+.`!ZS>?Rz{j2>@ |(x}QMJP4U Q@m`c -lޠGp#tQR][0~3i|a͟C}) Q-sfqRYNݞYѯ즱Ukɥ;7Ɣ@DNB+N W *XN&@ zi$oYxo>'n(Ϛcp2ƬRXet6s7LǹZ{aEv7x;{u'674AHN5jdaDhd|E0IADdhL  C A?"?2&d-u}Q0`!u&d-ud@e |Cx}QJQ=wym. VBTX`FXPp$.X[+HwXYd=3HY.;ggf@:T|eQ$I 4IU,Zr$(Q.bIxLo\>Xǰձ0褌=ϵy\`Jv x tq:Jm2ͷ#as#"8M&m Q`ѧHKÚbÖ,`)Ӣa5'cUhF-V2  ^rX$'#ТZ㺿!HRD*>\raS/"vG{Pzο%y?->WrCI5Y=a𝢯A܃\'u=iscK@mrdU]v6tYqGGDdTL  C A?"?2{=>1b3FJx=`!{{=>1b3FJb  JIxR=KA}; BDia+;k,#(xV C," -zff9 `@XçBEFLOQe&Qyȧip]uS$LN`.׈1r|/w-9(o~-o^P{L$D .Vx2;VD=ġ{M:#M۬8F e!aqB;%.%%g(9aڮ"(|\ 4{\evӀ RgjWY/SmO#m.maBylinr;#𗰢=+f,;DC13T`zDUotBO9i {0G Hl|;Ismmau ҟ |l5l:499rn>S ƲsP'cR<yo~68 lUA;Mi.W +iA~hJ!v9-!:p -}\G]Li D}=hQl]8@@^_@u%C e)FX EnSK9N,79A;K{;R=T5&7d|3tw%C/ͤEE5 2MF;q|0N ˰} fɋ뱴 ;[(X)"4c'Cty8,: PDC o%mY>HqۖyXܺ$# ʊ08^ 0K.|w *r3P/voFLs ["4p}Š![͠=`"Ą+0Ek׊GrjP?f>DdhL  C A?"?2pn!JMHʏL+D`!Dn!JMHʏՒ@|xmPJCA=3& A$N|A~@ H !eV6~B}2&s왇(x[o*cYSiXuëi[Ax{ 7S= }ӫ9nbj/5owjQQx]6'ks󘣪yͮ!0BaF(zu[Iz3e;ST7MDd@L   C A?"?2C?X ĊGQH J`!?X ĊGQH fx}PPQ b% v-؝prc/\] cd @ @Q dY5awD4NGpn좳 2A+Bb10rwߏE4]%>&YyksNsI@|in_Z}N!Oz=5BogO.q盙/wۙ+p6DDdhL ! C A?"? 2pn!JMHʏLL`!Dn!JMHʏՒ@|xmPJCA=3& A$N|A~@ H !eV6~B}2&s왇(x[o*cYSiXuëi[Ax{ 7S= }ӫ9nbj/5owjQQx]6'keAJ3:"DdL & C A$?"?%2C…i@;ZV`!…i@;Z:R]x=O;Aيx@Ea2c#Co!M6 v+Y @ Y}N*cQkK*1{򺖹7TAn:0s1'ϯٰӾ#ciU5(Gt2H 91\0 ub!(3vλ{t6[̠#[K h:T7Dd@hL ' C A%?"?&2cΧJYÒj +?W?X`!7ΧJYÒj +?W@h|xmPJA}3 BD0ZX%,VH Bvi "Y_a}7%Ufg73+(O#LErL4˲p{.9V*VhYCU))=/s*KغjQQ6I^g$ӯ@x]Ɩ0omh^"ݮ'Aw1X`OЅ C>$=v` s8?U㧆ysHADdL ( C A$?"?'2C…i@;ZZ`!…i@;Z:R]x=O;Aيx@Ea2c#Co!M6 v+Y @ Y}N*cQkK*1{򺖹7TAn:0s1'ϯٰӾ#ciU5(Gt2H 91\0 ub!(3vλ{t6[̠#[K h:T7Dd@hL ) C A%?"?(2cΧJYÒj +?W?o\`!7ΧJYÒj +?W@h|xmPJA}3 BD0ZX%,VH Bvi "Y_a}7%Ufg73+(O#LErL4˲p{.9V*VhYCU))=/s*KغjQQ6I^g$ӯ@x]Ɩ0omh^"ݮ'Aw1X`OЅ C>$=v` s8?U㧆ysHADdL * C A&?"?)2Y$e'r^`!Y$e'F`;!xEQkPnn+$& "C">QlA%Ҵ4-[^t""B}ؓJw//pϯs#W`%ԩS Aք1ϵ-/|k@Ӻda̝M9 e>ն5ʊ3ݳƁڱՋR̞n8ao7ʴʴ;;g5ѷ2'­Ѧj7J̓Av(o.7݅XdgƂc7J"!$"Qovdž'Ǻ~2b{9DyK +http://www.nber.org/~rdehejia/nswdata.htmlyK nhttp://www.nber.org/~rdehejia/nswdata.htmlyX;H,]ą'cDocumentSummaryInformation8PMsoDataStore0ނ!JUP54BEUBSSNA==20ނ!Item ՜.+,D՜.+,T hp|  Xa )MODULE FOUR, PART TWO: SAMPLE SELECTION Title(RZ _PID_HLINKS MTWinEqnsA-2~+http://www.nber.org/~rdehejia/nswdata.html   F'Microsoft Office Word 97-2003 Document MSWordDocWord.Document.89qj" 666666666vvvvvvvvv666666>666666666666666666666666666666666666666666666666hH666666666666666666666666666666666666666666666666666666666666666662 0@P`p2( 0@P`p 0@P`p 0@P`p 0@P`p 0@P`p 0@P`p8XV~ OJPJQJ_HmH nH sH tH J`J f'Normal dCJ_HaJmH sH tH DA`D Default Paragraph FontRiR 0 Table Normal4 l4a (k ( 0No List TBT p&0 Body Text dCJ$OJPJQJ^J_H aJ$PP p&0Body Text CharCJ$OJPJQJ^J_H aJ$V@V p& No Spacing dCJOJPJQJ^J_H aJ F+@"F |`0 Endnote Text dCJaJB1B |`0Endnote Text CharCJaJ>*@A> |`Endnote ReferenceH*RZ@RR Vr> X!0HeaderdH$.. X!0 Header Char> @> X!0FooterdH$.. X!0 Footer CharRR X!0 Balloon Text dCJOJ QJ ^J aJNN X!0Balloon Text CharCJOJ QJ ^J aJ@@ MR List Paragraph ^m$6U6 @& Hyperlink >*B*ph>> +H%0 Footnote TextCJaJ<< +H%0Footnote Text Char@&@ +H%0Footnote ReferenceH*PK![Content_Types].xmlj0Eжr(΢Iw},-j4 wP-t#bΙ{UTU^hd}㨫)*1P' ^W0)T9<l#$yi};~@(Hu* Dנz/0ǰ $ X3aZ,D0j~3߶b~i>3\`?/[G\!-Rk.sԻ..a濭?PK!֧6 _rels/.relsj0 }Q%v/C/}(h"O = C?hv=Ʌ%[xp{۵_Pѣ<1H0ORBdJE4b$q_6LR7`0̞O,En7Lib/SeеPK!kytheme/theme/themeManager.xml M @}w7c(EbˮCAǠҟ7՛K Y, e.|,H,lxɴIsQ}#Ր ֵ+!,^$j=GW)E+& 8PK!Ptheme/theme/theme1.xmlYOo6w toc'vuر-MniP@I}úama[إ4:lЯGRX^6؊>$ !)O^rC$y@/yH*񄴽)޵߻UDb`}"qۋJחX^)I`nEp)liV[]1M<OP6r=zgbIguSebORD۫qu gZo~ٺlAplxpT0+[}`jzAV2Fi@qv֬5\|ʜ̭NleXdsjcs7f W+Ն7`g ȘJj|h(KD- dXiJ؇(x$( :;˹! I_TS 1?E??ZBΪmU/?~xY'y5g&΋/ɋ>GMGeD3Vq%'#q$8K)fw9:ĵ x}rxwr:\TZaG*y8IjbRc|XŻǿI u3KGnD1NIBs RuK>V.EL+M2#'fi ~V vl{u8zH *:(W☕ ~JTe\O*tHGHY}KNP*ݾ˦TѼ9/#A7qZ$*c?qUnwN%Oi4 =3ڗP 1Pm \\9Mؓ2aD];Yt\[x]}Wr|]g- eW )6-rCSj id DЇAΜIqbJ#x꺃 6k#ASh&ʌt(Q%p%m&]caSl=X\P1Mh9MVdDAaVB[݈fJíP|8 քAV^f Hn- "d>znNJ ة>b&2vKyϼD:,AGm\nziÙ.uχYC6OMf3or$5NHT[XF64T,ќM0E)`#5XY`פ;%1U٥m;R>QD DcpU'&LE/pm%]8firS4d 7y\`JnίI R3U~7+׸#m qBiDi*L69mY&iHE=(K&N!V.KeLDĕ{D vEꦚdeNƟe(MN9ߜR6&3(a/DUz<{ˊYȳV)9Z[4^n5!J?Q3eBoCM m<.vpIYfZY_p[=al-Y}Nc͙ŋ4vfavl'SA8|*u{-ߟ0%M07%<ҍPK! ѐ'theme/theme/_rels/themeManager.xml.relsM 0wooӺ&݈Э5 6?$Q ,.aic21h:qm@RN;d`o7gK(M&$R(.1r'JЊT8V"AȻHu}|$b{P8g/]QAsم(#L[PK-![Content_Types].xmlPK-!֧6 +_rels/.relsPK-!kytheme/theme/themeManager.xmlPK-!Ptheme/theme/theme1.xmlPK-! ѐ' theme/theme/_rels/themeManager.xml.relsPK] 4gR CCCF b1? o!#l##'$$& (!)y) +++t,--h/u1BDQ,T%XY2\-_aeHipsuԄ#s7gijklnpqrtvwxyz{}~1%#/36=2ELxSz_ifHimuhx!Eb?3~b'hmosu|Kce|j},?AGZ\k~&(k~BUW*=?> Q S ! !"!=!P!R!d!w!y!!!!"""(#?#A#I#\#^#######*$=$?$F$Y$[$a$t$v$$$$$% %%'%)%{%%%%%%N'a'c'h'{'}'::::::L%L'LfOOO::::::::::::::::::::::::::::::::::::::::::X%=?F!(,-o2$>2v6zL?2$ |J:3ڹR2$㩬Å1d%HK2$=68.> 7j2$dNZy % mr_L2$HWήe72$mE+ȝkD*2$g<;'&ulxf2$ELL( 5h=NR Nk2$+@hYYc!2$R-r$T{Xh2$ǔsql#)n2$ q`HF`l!2$Fݰ,q:2$͓Fh# L! 2$}n\`t_fxq!2$!( o0jǯT2$NKCocW2$9@04 ل!!2$#EAa52@Kԋ52$S>?Rz{j2>b2$&d-u}2$Ɖ{Xsd,;{H2$٤С 1PS{e2$\tΎ<G\r2${=>1b3FJ2$Pؾɶ8+!2$n!JMHʏL2$:XD,XUpU2$D q*%2$?X ĊGQH 2$g'(;`tI4J2$w< '2Tԁ52$JxLs }QP2$t0[wP_2$…i@;Z2$ΧJYÒj +?W?2$Y$e'# AA@,0(  B S  ?]dpw   !(*+-.01346:<=?@BCEWYZ\`bBDjqsu& . E L ] _  ? A 2 4 [ ]     X ` '-/146>;!.0'.><?Rlo!$,BD+,9kBZ[h@AN( T V f #!$!4!5!S!T!!!!!!""##(#B#E#m########$$@$B${$$$$$$ % %1%l%%%%%%%%%%)&0&&&' 'd'g''''''''''''''''''''''(Y(`((((()*+ +++++++++++,,!,,,O,Q,[,],,,,,,,,,,,,,,, - -----.-0->-@-D-G-H-N-P-S-T-W-I.L.....|//"0&00022222294;4u5w566^6d6K7Q777y9~999R:X:::::;;'=0=4=;=C=E=S=U=Y=\=]=c=x=z===========+>.>|>>????6@8@rAtABB[BaBCCDDEEEEH#HHHK(LMM(N/N4N9NdNkN4O:OPPPPPPR&R`SdSSSSSSSTTBVHVyVVVV+W1WDWJWKWQWdXjX|XXXXXY Y YYYYYYYYYYYrZvZ ]]]]"]&]]]^^=_A___``bbNcTc(d-deeTfYfffffKgOgVgZgageggggggggggggggghh*h1h6hZ_NSu{pw  WYBDsu? A ] _  ? A k m 2 4 [ ]  7 ? U ] t |  j r &>D!7:RUkn &*59DHRVaept%*DG)+XZ7 > T!]!!!o#t###$$d'h'''''( ((()*******++o+r+++++++++++,,5,8,H,N,[,],t,x,,,x-|-_.i.2/7/90:0000011a1e11122P2R22222333394;44444%5(5u5w55566^6d6;8>8888899::4=;===>>??@#@6@8@@@@@"A%ArAtAAABB[BaBBCGC[C]CCCCCFDJDDDDD5E7EEEEEFFFFlGoGGGGG{HHI IGIKIIIJJ,K4KwK{KKKLSOS`SdS{SSSSSSSSYYYYZ ZS\V\b\e\s\z\\\\\\\\\\\\]]]$^%^i^l^^^^^=_A___ `!<3333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333Kf|j,BG]k)kBX*@> T !#!=!S!d!z!!!""(#B#I#_#####*$@$F$\$a$w$$$$ %%*%{%%%%L(Lޝޝ ץץ?@Kf|j,BG]k)kBX*@> T !#!=!S!d!z!!!""(#B#I#_#####*$@$F$\$a$w$$$$ %%*%{%%%%L(Lޝޝ ץץ{w3,>: X1hfT6/ ^ `OJPJQJ^Jo(" ^ `OJQJ^Jo(o x^x`OJ QJ o( H^H`OJQJo(^`OJQJ^Jo(o ^`OJ QJ o( ^`OJQJo(^`OJQJ^Jo(o X ^X `OJ QJ o(^`OJPJQJ^Jo("^`OJQJ^Jo(o p^p`OJ QJ o( @ ^@ `OJQJo(^`OJQJ^Jo(o ^`OJ QJ o( ^`OJQJo(^`OJQJ^Jo(o P^P`OJ QJ o(^`OJPJQJ^Jo("^`OJQJ^Jo(o p^p`OJ QJ o( @ ^@ `OJQJo(^`OJQJ^Jo(o ^`OJ QJ o( ^`OJQJo(^`OJQJ^Jo(o P^P`OJ QJ o({w3>:hfT&        Β        uI         x HSWy hY/^X ^p&@q^f' yV }}=}mt} ~WID].sJ9abKR:Ri)..}2H cd295|-v^AisR12za_"l~~6-)[Yf0r+;W%btUi -e?gS}R'dc^db'de&|`P IMM,MRRqo~PL)dWX@HXHIJKL@PRTV@@UnknownG*Ax Times New Roman5Symbol3. *Cx ArialeTimes New (W1)Times New Roman7.@ CalibriE=  Lucida Console?= *Cx Courier NewO1 CourierCourier NewKTimesNewRomanPSMTYTimesNewRomanPS-ItalicMTUTimesNewRomanPS-BoldMT5. .[`)Tahoma;WingdingsA BCambria Math"1h9&9&[aX6dc!4dX 2qHX  $Pp&2! xx(MODULE FOUR, PART TWO: SAMPLE SELECTION Bill Becker Bill Becker