“Improving Education through Accountability and Evaluation Lessons from Around the World” Rome, 3 -5 October 2012 The Causal Effect of Class Size on Pupils’ Performance: Evidence from Italian Primary Schools Larysa Minzyuk - Felice Russo Department of Management and Economics - University of Salento (Lecce) larissa. minzuk@unisalento. it felice. russo@unisalento. it
OUTLINES ¨ Motivations and purposes ¨ Identification strategy ¨ Regression Discontinuity design ¨ Data and procedure ¨ Results and conclusive remarks
Motivations and purposes • The relationship between class size and education attainment has been widely explored, but the existing evidence on the class size effect is still contrasting and somewhat inconclusive • In this paper, we estimate the class size effect in Italian public primary schools by using Regression Discontinuity (RD; Thistlewaite-Campbell, 1960, JEP) design, which has recently become a standard evaluation framework for solving causal issues with non-experimental data (not only in education) – in Italy a limited research has been done on this issue so far e. g. data limitations; among the known studies, see Bratti et al. (2007, GE), Brunello and Checchi (2005, EER), Quintano et al. (2009, REST), Russo (2010) • Our main results 1. In Italian data, we do not find a significant evidence which supporting class-size reduction policy 2. There is an evidence of sorting of pupils’ characteristics around cut-offs points (25 pupils): pupils with “unfavorite” socioeconomic background are in smaller classes
Regression Discontinuity Design 1. 2. 3. An RD-based evaluation is appropriate when increases in gradeenrolment (forcing variable) are linked with jumps in class-size (treatment variable) as predicted by: (a) the threshold rule generating a (b) discontinuous relation between the two variables Individuals (schools and families) cannot precisely manipulate the grade-enrolment in order to receive or avoid treatment ( i. e. to affect whether or not the fall on one side of the threshold or the other) Smoothness condition: Other variables are smooth functions of the forcing variable conditional on treatment (i. e. the only reason pupils’ outcomes should jump at the cut-off is due to the discontinuity in the level of treatment) If 1. , 2. , 3. jointly hold, effects of class size on pupils’ test scores can be interpreted as the local average treatment effect of class size
Data and procedure (1) We conduct our study using information from two sources: 1. The first is the INVALSI test results of V grade pupils in primary schools in 2008/09. These results are available for 150, 000 pupils coming from 5, 303 public and private primary schools (‘circoli didattici’) from all Italian regions – we restrict our analysis to the public schools whose testing procedure was assisted by INVALSI supervisors 2. The second source of information is school-level administrative data from the Italian Ministry of Education (MIUR; those data do not contain information about schools in regions with a special statute) – we matched the INVALSI data sample with the dataset on school characteristics and class size coming from MIUR – because of missing data on pupils characteristics in INVALSI data, we have 25, 407 pupils coming from 1, 561 school units
Data and procedure (2) • Predicted class size (Angrist and Lavy, 1999) Φis is the V grade enrolment at school s where the pupil i studied in 2008/09, int ( • ) is the function that takes the greatest integer less than the given argument Average and Predicted Class Size, 2008/09
Compliance of Schools to the Rule, 2008/09
Data and procedure (3) • A standard model of fuzzy RD can be described as follows (van der Klaauw, 2002, p. 1262): Pis is the test score of pupil’s i in school s, CSis is average class size in V grade at school level, Φis is the V grade enrolment at school level, indicates the cut-off values of enrolment (multiples of 25), and α ( • ) and β ( • ) are functions of enrolment • When enrolment is a discrete variable, class-size effect can be estimated only parametrically (Lee and Card, 2008) we decide a linear specification for both control functions α ( • ) and β ( • ), choosing the piecewise linear splines whose kinks correspond to the values of cut-offs (Urquiola and Verhoogen, 2009, AER; Zada et al. , 2009) (2 SLS). For instance (1 st stage; 2 knots): CSis = β + β 1 1[Φis > 25] + β 2 1[Φis > 50] + β 3 Φis + β 4 (Φis - 25)1[Φis > 25] + β 5 (Φis - 50)1[Φis > 50] + μis
First stage and base IV specifications (+/- 3 pupils intervals), 2008 -09 First stage IV stage with selected obs. Italian language Math Italian language -0. 005190 Class size Math -0. 003018 -0. 002389 -0. 000656 1[Φis > 25] yes 1[Φis > 50] 1. 085096*** 1[Φis > 75] 0. 670140** 1[Φis > 100] -0. 695183** Φis 0. 024155*** (Φi - 25)1[Φi > 25] -2. 875964*** yes yes (Φi - 50)1[Φi > 50] -1. 834880*** yes yes (Φi - 75)1[Φi > 75] -0. 852245*** yes yes (Φi - 100)1[Φi > 100] -0. 960405*** yes yes Isei index of father no no no 0. 001056*** 0. 001029*** Isei index of mother no no no 0. 000284*** 0. 000270*** Mother’s education no no no 0. 015243*** 0. 016755*** Pupils with special needs no no no yes 21. 688432*** 0. 710567** 0. 698901* yes Observations 5, 396 5, 396 Adjusted R-squared 0. 2756 0. 0066 0. 0048 0. 0508 0. 0679 Constant Note: In all regressions, standard errors are clustered by enrolment levels, see Lee and Card (2008). *** p<0. 01, ** p<0. 05, * p<0. 1.
Reduced form of selected observables (+/- 3 pupils intervals), 2008/09 ISEI index father ISEI index mother Mother's education Pupils with special needs 1[Φis > 25] yes yes -0. 3069*** 1[Φis > 50] 2. 5542*** -1. 9496* yes 0. 4053*** 1[Φis > 75] yes yes 1[Φis > 100] yes yes -0. 560*** Φis yes yes -0. 00296** (Φi - 25)1[Φi > 25] yes yes 0. 067425*** (Φi - 50)1[Φi > 50] -0. 8283** yes -0. 1689*** (Φi - 75)1[Φi > 75] 2. 031** yes -0. 6218*** -1. 53963*** -0. 2237*** 0. 388616*** 41. 03482*** 29. 28249*** 3. 330471*** 0. 690685*** Observations 5, 396 Adjusted R-squared 0. 0077 0. 0064 0. 0054 0. 0327 (Φi - 100)1[Φi > 100] Constant Note: In all regressions, standard errors are clustered by enrolment levels, see Lee and Card (2008). *** p<0. 01, ** p<0. 05, * p<0. 1.
First stage and base IV specifications (+/- 5 pupils intervals), 2008 -09 First stage Class 1[Φis size > 25] > 50] > 75] > 100] yes yes -0. 935615** IV stage with selected obs. Math Italian language -0. 006296 -0. 004871 -0. 006244 -0. 005164 Φis 0. 035858*** -0. 016370* yes yes (Φi - 25)1[Φi > 25] 0. 035858*** yes yes (Φi - 50)1[Φi > 50] -0. 850894*** yes yes yes (Φi - 100)1[Φi > 100] -0. 850894*** yes 0. 019211* yes Isei index of father Isei index of mother Mother’s education Pupils with special needs Constant Observations Adjusted R-squared no no 20. 758594*** 10, 685 no no 0. 727714*** 10, 685 no no 0. 730816*** 10, 685 0. 000961*** yes 0. 016281*** 0. 000394*** 0. 626042*** 10, 685 0. 001014*** yes 0. 017683*** 0. 000374*** 0. 627706*** 10, 685 0. 3791 0. 003 0. 0008 0. 0527 0. 0732 (Φi - 75)1[Φi > 75] Note: In all regressions, standard errors are clustered by enrolment levels, see Lee and Card (2008). *** p<0. 01, ** p<0. 05, * p<0. 1.
Reduced form of selected observables (+/- 5 pupils intervals), 2008/09 ISEI index father ISEI index mother Mother's education Pupils with special needs 1[Φis > 25] yes yes -0. 187210*** 1[Φis > 50] yes -2. 533225** yes 1[Φis > 75] yes yes 1[Φis > 100] -2. 737642*** -4. 413059*** yes -0. 674291*** Φis 0. 042128*** yes 0. 003121** yes (Φi - 25)1[Φi > 25] -0. 492997** yes yes (Φi - 50)1[Φi > 50] yes 0. 658670* yes (Φi - 75)1[Φi > 75] yes -0. 116766** 0. 386933*** (Φi - 100)1[Φi > 100] -0. 646475*** -1. 600302*** -0. 225947*** yes Constant 40. 229238*** 26. 607819*** 3. 229377*** 0. 636764*** Observations 10, 685 Adjusted R-squared 0. 0067 0. 0043 0. 0051 0. 0131 Note: In all regressions, standard errors are clustered by enrolment levels, see Lee and Card (2008). *** p<0. 01, ** p<0. 05, * p<0. 1.
Treatment effect (+/- 3 and 5 pupils intervals), 2008 -09 1 cut-off (25) 2 cut-off (50) 3 cut-off (75) 4 cut-off (100) -0. 0355* yes 0. 1483** Average class-size (left side of cut-off) yes 23. 078 yes 23. 876 Average class-size (right side of cut-off) yes 20. 568 yes 21. 470 2, 703 1, 544 786 363 Three-pupil interval Dep. Var: Italian yes yes 0. 1137** Average class-size (left side of cut-off) yes yes 23. 876 Average class-size (right side of cut-off) yes yes 21. 470 2, 703 1, 544 786 363 -0. 0133* 0. 0728** yes 0. 2193*** Average class-size (left side of cut-off) 21. 355 22. 983 yes 23. 612 Average class-size (right side of cut-off) 16. 556 20. 028 yes 21. 470 Obs. 5, 414 3, 184 1, 415 671 Five-pupil interval Dep. Var: Italian yes 0. 063** yes 0. 2847*** Average class-size (left side of cut-off) yes 22. 983 yes 23. 612 Average class-size (right side of cut-off) yes 20. 028 yes 21. 470 5, 414 3, 184 1, 415 671 Three-pupil interval Dep. Var: Math Obs. Five-pupil interval Dep. Var: Math Obs. Note: In all regressions, standard errors are clustered by enrolment levels, see Lee and Card (2008). *** p<0. 01, ** p<0. 05, * p<0. 1.
Number of schools in enrolment intervals (+/- 3 pupils), 2008/09 Number of schools in enrolment intervals (+/- 3 pupils, sample compliant), 2008/09
Reduced form estimates of selected observables, sample of compliant schools (+/3 pupils intervals), 2008/09 ISEI index father ISEI index mother Mother's education 1[Φis > 25] -7. 773760*** -10. 318900*** -0. 763182*** 1[Φis > 50] 1. 930775* -4. 608166*** yes 1[Φis > 75] -11. 345361*** yes -0. 186474* 1[Φis > 100] -3. 084358** yes yes yes (Φi - 25)1[Φi > 25] 3. 097997*** 1. 748179*** 0. 255036*** (Φi - 50)1[Φi > 50] -0. 677889* 0. 654220** yes (Φi - 75)1[Φi > 75] 3. 965961*** yes 0. 095906*** (Φi - 100)1[Φi > 100] -0. 052756*** -2. 039584*** -0. 235058*** Constant 40. 956966*** 29. 367030*** 3. 347153*** Observations 4, 111 Adjusted R-squared 0. 0102 0. 0140 0. 0093 Φis Note: In all regressions, standard errors are clustered by enrolment levels, see Lee and Card (2008). *** p<0. 01, ** p<0. 05, * p<0. 1.
Conclusive remarks (1) • In this paper we make an attempt to estimate the class-size effect on the pupils' performance using the data from Italian primary schools. We base our estimation strategy on RD design. To apply RD estimation strategy, practitioners have to test if the assumptions of RD analyses are not infringed, otherwise it would invalid to infer a “treatment” effect of class size on pupils' test results 1. In Italian data, we do not find a significant evidence which would strongly support class-size reduction policy 2. When focusing on small intervals (+/-3 and 5 pupils) around selected enrolment cut-offs, selection problem is not evident, as, on the one hand, we do not observe clear stacking behaviour of schools at the thresholds, and, on the other hand, pupils' characteristics result to be distributed smoothly in the large majority of cut-offs for this subsample
Conclusive remarks (2) 3. In contrast, as we observe in our data, we find that the stacking behavior is more evident in the reduced sample of compliant schools (+/-3 pupils) and there is a clearer evidence of sorting of pupils' characteristics around cut-offs points: in this sample, right sides of cut-off intervals include more pupils with “unfavorite” socio-economic background class size are largely used in primary public schools as a kind of compensatory policy (West-Woessman, 2006) • Urquiola and Verhoogen (2009) have found an evidence that Chilean schools might exercise selection policy on enrolment. The authors suggest caution when using RD, especially in application to private schools that may have a better control over enrolment compared to public schools • Zada et al. (2009) have found an evidence of “selection” policy in public secondary schools in Israel as well. It is worth noting that the authors have not found it in Israelian public primary schools.
Скачать презентацию Improving Education through Accountability and Evaluation Lessons from
6b1a05246887b4ae66a59c37160a4f16.ppt