Moving from pilot to scale in education What

Moving from pilot to scale in education: What does it take? 2017 ICT 4 D Conference, Hyderabad, India Sarah Pouezevara - @spouez RTI International is a registered trademark and a trade name of Research Triangle Institute. www. rti. org

About this presentation § Using prior analysis of scale up in basic education programs, how do we apply the lessons and frameworks to ICT-based scale up? § What steps or pre-conditions are similar? What needs to be approached differently? § What have been your experiences? ? 2 Question: Do the terms “scale up” and “sustainability” mean no change is possible? RTI International - @spouez

Ingredients for national reading programs § Assuring basic inputs (curriculum, materials) § Improving instruction (pedagogy, training, support) § Monitoring and assessment (monitoring instruction, outcomes and routinized accountability) § Communication (mobilize enthusiasm and create demand) ? 3 Are there common—or added— technical ingredients that need to be aligned for ICT innovations? RTI International - @spouez

Ways to think about scale (and sustainability) 1. Expansion: Originator of an intervention implements it in an increasing number of schools/places 2. Replication: Originator of an intervention partners and networks with other organizations/entities (including Ministry of Education) to implement it in an increasing number of schools/places. 3. Collaboration: Originator of an intervention uses the pilot experience to assist others to implement, including to inform government policy and practice so that elements of the innovation are taken up by the education system and implemented at scale ? 4 How can programs employ a combination of these approaches to get to achieve scale and sustainability? RTI International - @spouez

“Ingredients” for successful scale-up Brookings and MSI Scale Up Frameworks and MSI) Ingredients for Scale Up Brookings: 14 Core Responsiveness to local needs Cost-effectiveness Design Flexibility Elevating teachers Education alliances Champions and leaders (Brookings MSI: Management Framework for Scale Up Create a vision for scale Assess scalability Fill Information gaps Prepare scale up plan Appropriate use of technology Legitimize change Windows of opportunity Build broad constituencies Using data on learning Delivery Realign and mobilize resources Flexible financing Finance Long-term financing “Middle phase” financing Enabling Environment Modify structures/ build capacity Coordinate action Supportive policy environment Culture of R&D Develop Scale Up Plan Track performance and maintain momentum Establish the Pre. Conditions for Scaling Up Implement the Scaling Up Process

Tusome National Tablet Program for Instructional Coaches 2014 Pilot 150 Users and cost effectiveness study 2015 Nat’l scale up 1200+ users 2016 Expansion 2000 users 2017 Enhancements User-centered design Outputs Data at county and national level

Keys to Tusome’s success in scaling up Used a combination of the three approaches (starting small, encouraging others to adopt the model, and working through government). § Uses the pilot experience to inform government policy and practice so that elements of the innovation are taken up by the education system and implemented at scale. § Worked alongside government at all stages. § 7 RTI International - @spouez

Mobi. Literacy Uganda – Urban Planet and RTI 2013 Pilot 150 Users and cost effectiveness study Baseline and endline skills measurement EGRA methodology Plan for scale Telecom partnership Results Effective method but not scaled

Lessons from Mobi. Literacy’s pilot Measured comparative approaches: technology vs. paper vs. oral only. § Scale was planned through partnership with local telecoms, but there is no evidence that this happened. § Did not work alongside government at any stage: example of innovation and scale operating outside of formal system § Lack of ‘champion’ to help create demand? Insufficient investment in partnership strategy? § A = mobile B = paper C = oral 9 RTI International - @spouez

Emerging conclusions § § Innovations must be designed with scale in mind. § The interplay between scale and sustainability is not just in financial terms—it is also about in-country capacities needed to support large scale change. § Long-term sustainability requires demand for the innovation, including mobilizing champions, leaders, and constituencies and alliances to support the innovation. § Most innovations will be altered as they go to scale and they will also alter existing arrangements in the education system; flexibility to respond to changing contexts is a good thing provided the minimum ingredients are still present. § 10 Scale up needs to be based on data about what works (defining the ‘ingredients for success’) Pilots need to be designed with local institutions—including public sector—to build necessary capacity during the process and address alignment/coherence issues (sending the right “signals”) RTI International - @spouez

Some ways that technology can be applied § § Expanding opportunity (e. g. , personalized learning, more feedback, anytime/anywhere learning) § Keep costs low without sacrificing quality (economies of scale, automating processes, generating efficiencies like mobile money) § Accountability and data for monitoring and evaluation § Data gathering for impact evaluation § Mobilizing demand communicating successes, elevating the role of teachers § 11 Filling gaps by unburdening teachers (e. g. , home access to learning) Overcome a context-specific barrier (e. g. , long distances, difficult transportation) RTI International - @spouez

From: Cooley Checklist for evaluating scalability and Kohl (2006) § § § § 12 Credibility - is there sound evidence demonstrating impact, and is that evidence espoused by recognized leaders Observability - teachers, principals, officials, and parents must be able to see tangible results Relevance - does the innovation respond to a recognized need (e. g. , as a response to poor EGRA results) Comparative/Relative Advantage - does it show a cost-effective alternative to existing practice Ease of transfer and adoption - it cannot be too far of a leap for the average teacher Compatibility - it cannot be completely incompatible with existing norms, values, and arrangements (e. g. , it cannot require a complete reimagining and reorganization of the education system) Testability - can potential users try it out safely without risking too much RTI International - @spouez

Discussion “Scalability is demonstrated by (a) the ability to replicate the key elements of a project that were deemed critical to its effectiveness at scale (regional, national) and (b) the project's affordability over the mid and long-term, given a country’s total projected resource envelope. ” - USAID, 2011 Question: Do the terms “scale up” and “sustainability” mean no change is possible? § Question: How to we employ a combination of approaches to planning for scale up? § Question: Are there common—or added—technical ingredients that need to be aligned for ICT innovations? § Question: What else do you consider a critical point on the ‘checklist’ for ICT 4 E innovations? § 13 RTI International - @spouez

References Cooley, L. , and Kohl, R. (2006). Scaling Up- From vision to large-scale change: A management framework for practitioners. Washington, DC: Management Systems International. Perlman Robinson, J. , Winthrop, R. , & Mc. Giveny, E. (2016). Millions learning: Scaling up quality education in developing countries. Washington, DC: Brookings Institute. Piper, B. , Zuilkowski, S. , Kwayumba, N. Strigel, C. (2016). Does technology improve reading outcomes? Comparing the effectiveness and cost-effectiveness of ICT interventions for early grade reading in Kenya. International Journal of Educational Development (49): 204 -214 De. Stefano, J. and Healey, H. (2016). Scale up of early grade reading programs. Prepared for USAID's Education Data for Decision Making Project II (Ed Data II), Task Order 15. Washington, DC: RTI International. US Agency for International Development (USAID). (2011). Education Strategy Implementation Guidance. Washington, DC: USAID, p. 6. 14 RTI International - @spouez

More Information Name Sarah Pouezevara @spouez@rti. org For copies of presentation and other RTIauthored documentation on ICT in education, scale and sustainability, see: http: //shared. rti. org http: //tinyurl. com/m 3 eolte