Six Lessons for Turning Monitoring and Evaluation into

Six Lessons for Turning Monitoring and Evaluation into Delivery Lima July 2012 Ray Shostak, CBE

Remember who makes a difference to the quality of services………. . 2

Setting and Meeting Expectations Engage Empower Co-Design Co-Produce Results

Government Ambition to Delivery Government Priorities CITIZENS

Outcomes Driven Performance ACCOUNTABILITY with consequences To Parliament To President/Prime Minister By Institution By Individual Government Strategy Goals/Ambitions For parliamentary period Set Whole of Government PRIORITIES (for Years 1/2/3) Non Government Policy, Objectives and Targets by and across Ministry Aligned as Government Plan Delivery Partners By Institution By Individual MONITORING, CAPACITY BUILDING and INTERVENTION BUDGETING FOR RESULTS (multiple years) Programme budgets against results Unit costing Monitoring Efficiency Joint procurement ……. Programmes, Objectives, Milestones, Deliverables Institutions and Individuals Outcomes for and with citizens CULTURE Outcomes focus Data rich Can do Real time monitoring • Reform • • SKILLS Problem solving Cost benefit analysis Engagement Evaluation ACTION When off track

Lesson 1: Start from Citizens

Detailed Journey Maps

Lesson 2: Make sure Priorities are Priorities

USA Performance. gov Taxonomy Agency Theme Legend Submitted by agencies – Phase I Agency-Defined Budget Functions Groupings Planned for Phase II Priority Goal Sub-Goal (tag -only) Performance Data Measures Other Information Action Plan Milestones Contributing Programs Other Agency Measures Data. gov or agency site Indicators Explanatory Narratives Financial Resources USASpending Granular Data Budgetary Resources MAX

Fairness and Opportunity for All A better quality of life Stronger communities • Halve the number of children in poverty by 2010 -11, on the way to 6 eradicating child poverty by 2020 • Raise the educational achievement of all children and 2 young people • Narrow the gap in educational achievement between children from low income and 2 disadvantaged backgrounds and their peers • Increase the number of children and young people on the path to success 10 • Address the disadvantage that individuals experience because of their gender, race, disability, age, 4 sexual orientation, religion or belief • Increase the proportion of socially excluded adults in settled accommodation and employment, 8 education or training • Improve the health and 4 wellbeing of children and young people • Improve children and young 6 people’s safety • Tackle poverty and promote greater independence and 3 wellbeing in later life • Promote better health and wellbeing for all 2 • Ensure better care for all 1 • Deliver a successful Olympic Games and Paralympic Games with a sustainable legacy and get more children and young 7 people taking part in high quality PE and sport • Build more cohesive, empowered and active 4 communities • Make communities safer 8 • Deliver a more effective, transparent and responsive 5 Criminal Justice System for victims and the public • Reduce the harm caused by alcohol and drugs 8 • Reduce the risk to the UK and its interests overseas from 15 international terrorism A more secure, fair and environmentally sustainable world • Lead the global effort to avoid 8 dangerous climate change • Secure a healthy natural environment for today and the future • Reduce poverty in poorer countries through quicker progress towards the Millennium Development Goals • Reduce the impact of conflict through enhanced UK and international efforts 3 5 3 Numbers of departments contributing to the PSAs Help people and businesses come through the downturn sooner and stronger, supporting long-term economic growth and prosperity Deliver commitments by the National Economic Council to help people and businesses come through the downturn sooner and stronger, including packages on repossessions, apprenticeships, business access to finance and help for the unemployed. • • • 3 Raise the productivity of the UK economy Improve the skills of the population, on the way to ensuring a world-class skills base by 2020 Ensure controlled, fair migration that protects the public and contributes to economic growth Promote world class science and innovation in the UK Deliver reliable and efficient transport networks that support economic growth Deliver the conditions for business success in the UK Improve the economic performance of all English regions and reduce the gap in economic growth rates between regions Maximise employment opportunity for all Improve long term housing supply and affordability 3 6 4 4 2 3 9 8

Lesson 3: Get Policy Right First Time – Components of Open Policy Making

How……. • “Crowdsourcing” questions to shape the definition of the problem, not just consulting on solutions. • Using ‘Policy Labs’ which draw in expertise • Involving delivery experts early • Creating cross-departmental teams • Using web based tools to widen the policy debate • Making more source data avialable 15

Lesson 4: Understand the system 16

Moving from a ‘delivery chain’… Ministry Regional Government Local Government Service provider Citizen 17

…………’Delivery Systems’ … to a ‘delivery system’ SPORT EDUCATION HEALTH Youth Sport Trust School Sport Partnerships Government Office Health Authority Local Authority School Primary Care Trust School Hospital Family Citizen

Delivery System for PSA Better Health For All 18 – Better Health For All – DH strand Legislation (incs EU legislation); health prevention & promotion advertising Workforce advertising, influencing recruitment & retention; NSFs Influence National Key : Strategic Health Authorities Regional Agencies = Department / Agency = Working jointly and in partnership Performance Management of delivery agreements Delivery Agencies Competition, pricing, vfm joint commissioning Performance Management JSNA DH Primary Care Trusts (inc PBC) LAAs, LSPs Funding, LDPs Acute / MH Trusts and FTs GPs & Practices Specialist Units / Trusts Delivery mechanisms Patients, People who need and use social care, Citizens, Socially Excluded / Disadvantaged Groups, Carers…. Equitable outcomes provides support Choice & Voice influencing provision and commissioning e. g. LINks Public Health Observatories Local Community = national levers and covers HCC, CSCI, NICE, Audit Commission and wider agencies e. g. CQC etc including ‘clinicians’ e. g. GMC, NMC, HPC, etc = joint local working relationship Footnote 1, 07/8 programme to co-locate public health and social care presence in the regions. Footnote 2, consider in light of wider regulatory review. 19

Delivery System for PSA Better Health For All 18 – Better Health For All – DH/CLG strand Legislation (incs EU legislation); health prevention & promotion advertising Workforce advertising, influencing recruitment & retention; NSFs Performance Management, funding Government Offices (incs PH 1) Performance Management of delivery agreements Local Authorities inc OSCs environment, housing, roads, schools, benefits, etc Commissioning incs joint commissioning Influence DCLG Strategic Health Authorities JSNA Performance Management DH Competition, pricing, vfm Primary Care Trusts (inc PBC) LAAs, LSPs Funding, LDPs Acute / MH Trusts and FTs GPs & Practices Specialist Units / Trusts LA Teams and Specialist LA Units Social Care providers Independent providers e. g. ISTCs Patients, People who need and use social care, Citizens, Socially Excluded / Disadvantaged Groups, Carers…. Equitable outcomes provides support Choice & Voice influencing provision and commissioning e. g. LINks Public Health Observatories Voluntary providers Regulators & Inspectorates 2 National Key : Inspect, Reviews & Reports Regional Agencies = Department / Agency = Working jointly and in partnership Delivery Agencies Delivery mechanisms Local Community = national levers and covers HCC, CSCI, NICE, Audit Commission and wider agencies e. g. CQC etc including ‘clinicians’ e. g. GMC, NMC, HPC, etc = joint local working relationship Footnote 1, 07/8 programme to co-locate public health and social care presence in the regions. Footnote 2, consider in light of wider regulatory review. 20

Delivery System for PSA Better Health For All 18 – Better Health For All – Complete system Legislation (incs EU legislation); health prevention & promotion advertising Workforce advertising, influencing recruitment & retention; NSFs Cross-government working DCSF DWP Performance Management, funding HO Government Offices (incs PH 1) influence DCMS Defra OGD influence RDAs, Regional Assemblies, etc DIUS OGD influence Df. T Performance Management of delivery agreements Local Authorities inc OSCs environment, housing, roads, schools, benefits, etc regeneration Business community Regulators & Inspectorates 2 National Key : Commissioning incs joint commissioning Influence DCLG JSNA Strategic Health Authorities influence Performance Management DH Competition, pricing, vfm Primary Care Trusts (inc PBC) LAAs, LSPs Funding, LDPs Acute / MH Trusts and FTs GPs & Practices Specialist Units / Trusts LA Teams and Specialist LA Units Social Care providers Independent providers e. g. ISTCs Patients, People who need and use social care, Citizens, Socially Excluded / Disadvantaged Groups, Carers, etc Equitable outcomes provides support Choice & Voice influencing provision and commissioning e. g. LINks Public Health Observatories Voluntary providers Inspect, Reviews & Reports Regional Agencies = Department / Agency = Working jointly and in partnership Delivery Agencies Delivery mechanisms Local Community = national levers and covers HCC, CSCI, NICE, Audit Commission and wider agencies e. g. CQC etc including ‘clinicians’ e. g. GMC, NMC, HPC, etc = joint local working relationship Footnote 1, 07/8 programme to co-locate public health and social care presence in the regions. Footnote 2, consider in light of wider regulatory review. 21

Lesson 5: Let Data Drive Delivery and Cost 22

One Version of the Truth 23

Monitor the Impact Know the Impact of Policy on Practice

Strategic Reviews Key Review Concepts Efficiency • Appropriateness • Effectiveness • Efficiency • Integration Policy Alignment Effectiveness • Performance Assessment • Strategic Policy Alignment

The Program Activity Architecture Policy requirements for Strategic Outcomes incl: Performance Measures Policy requirements for each program activity element include: Strategic Outcome* Program Activities* Program title & description Expected results Departmental PAA: reflect the inventory of all the programs of a department depicted in their logical relationship to each other and to the SO(s) to which they contribute Sub-Activity Level** Performance measures Planned & actual spending Target & actual results Governance Accountability levels to Parliament (Estimates & Public Accounts) *require TB approval, incl. major & minor changes Sub-Sub Activity Level** Lowest Level programs **require TBS approval

Lesson 6: Support to deliver improved, more Monitor the Impact efficient and effective services for citizens

A Delivery Unit Performance Policy (ie. Performance Framework, Excellence, cross departmental working) Reporting Prime Minister Secretary of State Senior Officials Unblocking Delivery Obstacles (ie. progress reviews, problem solving, follow-up work with and brokering between departments) Capacity Building & Cross Govt Learning on Delivery (ie. knowledge mgmt, deptl delivery units, networks & training) Performance Monitoring (ie. Data tracking and reporting)

Some final observations • Outcomes Matter • The skills and dynamics of delivery can be analysed and understood • The basics of good delivery are key • Consistency and alignment enhances outcomes • Systematic intervention can improve systems and outcomes • Collaboration pays dividends at all levels • Citizens can co-produce services

Continuing the dialogue…. . Ray Shostak, CBE ray@shostak. uk. com