Скачать презентацию What works and who listens Encouraging the experimental evidence Скачать презентацию What works and who listens Encouraging the experimental evidence

26a55fbff66e7d1bd825ca81ab09f1ed.ppt

  • Количество слайдов: 23

What works…and who listens? Encouraging the experimental evidence base in education and the social What works…and who listens? Encouraging the experimental evidence base in education and the social sciences RCTs in the Social Sciences 9 th Annual Conference ∂ Professor Carole Torgerson School of Education Durham University carole. [email protected] ac. uk September 11 th 2014

∂

∂

First known RCT “Five seniors, each of whom had a good scholarship record, pleasing First known RCT “Five seniors, each of whom had a good scholarship record, pleasing personality, excellent health and fine social environment, were chosen to act as personnel counselors for the members of the freshman class, who at the end of the first eight weeks of school happened to be ∂ delinquent in scholarship in the School of Mechanical Engineering in I 929 -30. The 220 delinquent freshmen were divided into two groups by random sampling. ” [italics added] [Walters, 1931]

 US legislation: Institute of Education Sciences ∂ “Scientifically valid educational evaluation employs experimental US legislation: Institute of Education Sciences ∂ “Scientifically valid educational evaluation employs experimental designs using random assignment, when feasible, and other research methodologies that allow for the strongest possible causal inferences when random assignment is not feasible. ” [page 5] [italics added]

Education Endowment Foundation (EEF) “. . . all EEF-funded projects are independently and rigorously Education Endowment Foundation (EEF) “. . . all EEF-funded projects are independently and rigorously ∂ evaluated. . . The impact of projects on attainment will be evaluated where possible, using randomised controlled trials. ” [italics added]

Every Child Counts The independent evaluation [Torgerson et al, 2013] ∂ Every Child Counts The independent evaluation [Torgerson et al, 2013] ∂

Distribution of schools ∂ Distribution of schools ∂

Early reading policy ∂ Early reading policy ∂

Two alternative designs Systematic review of different methods to teach reading Expert review of Two alternative designs Systematic review of different methods to teach reading Expert review of different methods to teach reading [Rose, 2006] [Torgerson, Brooks and Hall, 2006] – Used systematic review design and methods ∂ – Used expert review design and methods – Included randomised trials undertaken anywhere – Included examples of best practice, two UK-based trials and expert opinion – Examined the quality of the included trials – Did not assess the quality of the included studies

House of Commons Education and Skills Committee, 18 th July 2005 “…in conducting his House of Commons Education and Skills Committee, 18 th July 2005 “…in conducting his review, Jim Rose will have the opportunity to draw on the findings of an independent systematic literature review of phonics use in the teaching and application of reading and spelling which we have commissioned from Professor Greg Brooks and ∂ Carole Torgerson. This delivers on the public commitment we made…in 2003 to publish an analysis of existing research on phonics teaching methodologies. The aim … is to identify what is known from existing literature about how effective different approaches to phonics teaching are in comparison with each other, including the specific area of analytic versus synthetic phonics. ”

Meta-analysis: Forest plot ∂ Meta-analysis: Forest plot ∂

Publication bias: Funnel plot 25 ∂ 0 -3 -2 -1 0 Effect Size 1 Publication bias: Funnel plot 25 ∂ 0 -3 -2 -1 0 Effect Size 1 2 3

Publication bias: Funnel plot ∂ Effect Size Publication bias: Funnel plot ∂ Effect Size

Publication bias: Funnel plot ∂ Missing trials? Effect Size Publication bias: Funnel plot ∂ Missing trials? Effect Size

Synthetic versus analytic phonics ∂ Synthetic versus analytic phonics ∂

Policy decision “I am clear that synthetic phonics should be the first strategy in Policy decision “I am clear that synthetic phonics should be the first strategy in teaching all children to read. ” Ruth Kelly, Secretary of State for Education and Skills ∂ Times Mar. 21 st 2006 “The case for synthetic phonics is overwhelming. ” Sir Jim Rose Times Mar. 21 st 2006

“Synthetic phonics does look promising, ” says Carole Torgerson…one of the report's authors. “We “Synthetic phonics does look promising, ” says Carole Torgerson…one of the report's authors. “We found it had a moderate effect compared with whole-language approaches, but the evidence base for this conclusion was 12 relatively small trials, only one of which was UK-based. This would be an ideal time to do a national evaluation by implementing systematic synthetic ∂ phonics in some schools and not in others and then comparing the two. ” Economist Mar. 26 th 2006

“When the UK government recently introduced the ‘synthetic phonics’ method of teaching young children “When the UK government recently introduced the ‘synthetic phonics’ method of teaching young children to read, they were told by Carole Torgerson…that they could easily bolster the slim evidence base by randomising which schools joined the programme first. ” Financial Times Mar. 18 th 2010 “In 2007 the Government introduced a new reading strategy for ∂ primary schools based on synthetic phonics, which matches sounds to groups of letters. Professor Torgerson urged ministers to start a randomised trial: the introduction of phonics would have been staggered, with schools chosen at random to start it one year or the next. Every child would have received the intervention, but it would have been possible to compare outcomes and establish whether phonics really works. ” Times Sept. 24 th 2011

Evidence of impact? Percentage of pupils reaching level 2 or above Subject 2006 2007 Evidence of impact? Percentage of pupils reaching level 2 or above Subject 2006 2007 2008 Reading 84 84 84 Writing 81 80 80 ∂ 2009 2010 2011 2012 84 85 85 87 81 81 81 83 87 87 88 ‘KS 1 87 results cast doubt on phonics’ 87 http: //www. tes. co. uk/article. aspx? storycode=6021536 Speaking and listening

Study characteristics of educational trials 1990 -2002 ∂ Torgerson et al, British Educational Research Study characteristics of educational trials 1990 -2002 ∂ Torgerson et al, British Educational Research Journal 2006, 31: 6.

Annual conference: Randomised controlled trials in the social sciences ∂ Annual conference: Randomised controlled trials in the social sciences ∂

Thank you for listening! Rose, J. (2006) Independent review of the teaching of early Thank you for listening! Rose, J. (2006) Independent review of the teaching of early reading, London: Department for Education and Skills Torgerson, C. J. , Wiggins, A. , Torgerson, D. J. , Ainsworth, H. , Hewitt, C. (2013) Every Child Counts: Testing policy effectiveness using a randomized controlled trial, designed, conducted and reported to CONSORT standards, Journal of Research in Mathematics Education: 15(2) ∂ Torgerson DJ, Torgerson CJ The design of randomised trials in health, education and the social sciences. Palgrave Macmillan, Basingstoke, 2008 Torgerson, C. J. , Brooks, G. & Hall, J. (2006) Literature review of phonics use. London: Department for Education and Skills Torgerson, C. Systematic Reviews, Continuum, London and New York, 2003 Walters, J. E. (1931) The Journal of Higher Education: 2(8), 1931