World Meteorological Organization WMO OMM Working together in

World Meteorological Organization WMO OMM Working together in weather, climate and water 5. 8 Monitoring and Evaluation of Capacity Development WMO www. wmo. int

Linkage between the next SP, OP and RBB SP WMO OMM OP (2012 -15) KOn 11 Del GSN STn ERn 1 Del KOn 13 SECRETARIAT RBB (2012 -15) Budget Act KOn 12 Act Budget

Results-based WMO Operating Plan and Secretariat Budget 2012 -2015 Framework of the WMO OMM (5) ST 1 (8) ER 1 KOs of RA I … VI Indicators, Baselines, Targets Secretariat Departments Activities and Budget KOs of TC 1 … TC 8 Indicators, Baselines Targets ER 1 ER 2 Strategic level For every ER, each RA and TC will define specific Key Outcomes together with Indicators, Baselines and Targets; based on these plans and in close coordination the Secretariat develops deliverables, activities and budget proposals. Operational level

KO’s , KPI’s and Priority Areas WMO OMM • ST 2: Advancing scientific research and application as well as development and implementation of technology • ER 4: Enhanced capabilities of Members to access, develop, implement and use integrated and interoperable surface-based and space-based systems for weather, climate and hydrological observations, as well as related environmental observations, based on world standards set by WMO. • KO 4. 1: WMO Integrated Global Observing System (WIGOS) is implemented • KPI – Completion of WIGOS milestones according to WIGOS implementation plan. , – Increased availability of observations for users • Priority: WIGOS and WIS are priority areas • Total of 24 KOs, most of them have not more than 2 KPIs. • Additional 7 KOs on GFCS and DRR as major initiatives

WMO OMM Strategic planning documents 1. WMO Strategic Plan : High-level statement of strategic directions and priorities expressed through Strategic Thrusts and organization-wide Expected Results and Performance Indicators addressing the Global Societal Needs; 2. WMO Operating Plan: converts the organization-wide ERs into specific outcomes/ deliverables, indicators and targets 3. Secretariat Implementation Plan and budget: converts the organization-wide ERs into Secretariat specific outcomes/ deliverables, indicators and targets and links them to the planned activities and the Secretariat budget; 4. WMO Strategic Executive summary: In a language appealing to those outside WMO and those making decisions to fund NMHSs. and Tool/System for Monitoring & Performance Evaluation – Tracks implementation and provides for control and corrective actions for achieving the ERs.

WMO OMM WMO Strategy for Capacity Development : Why? • Establish a CD Strategy to ensure, in a holistic approach, all actors in Met/Hydro/Climate work towards the same overall objective : Facilitating sustainable development of NMHS Resolution Cg-XVI/B/WP 11. 2 • Congress noted that a coordinated and cohesive approach for capacity development would be needed to enhance capabilities of NMHSs in developing countries to meet growing Societal Needs at different levels. • Congress noted that capacity building is a crosscutting activity and contributes to all Expected Results (ER),

WMO OMM Definitions and Logic of WMO strategic planning approach Societal GSNs : Global Needs STs : Strategic Thrusts WMO Strategic Plan ERs : Expected Results KOs : Key Outcomes Actions and Budget WMO OP 20122015

WMO OMM

WMO OMM Strategic Thrust n 3 Expected Result n 6

WMO OMM Tools to Monitor and Evaluate Progress • Types: Empirical, Qualitative, Index, breadbasket, case histories, Indicators (available economic, social or environmental) • Example Collection tools: – – Self reporting: Surveys, Visits of PR and national staff Country assessments National Records Reports: OPAGS, Regional WGs, Fellowship Evaluations, project evaluation – Country Profile Data Base

WMO OMM Conceptual Framework for Monitoring and Evaluation Feedback and learning M&E M&E Inputs Activities Deliverables Expected Results Key Outcomes WMO governance decisions Activities as stipulated in the WMO Operating Plan based on contributions of the TCs, RA and the Secretariat Deliverables as stipulated in the WMO Operating Plan based on contributions of the TCs, RAs and the Secretariat Expected Results as stipulated in the WMO Strategic Plan Key Outcomes as stipulated in the WMO Operating Plan TCs and RAs human resources Financial & human resources: ·WMO Secretariat ·WMO Joint Programmes ·Members Partner Organizations’ contributions Activities of WMO Joint Programmes Activities of Members Deliverables of WMO Joint Programmes Deliverables of Members WMO Secretariat enabling processes and coordination Expected Results of WMO Joint Programmes Key Outcomes of WMO Joint Programmes

WMO OMM Capacity Development – and ER 6 Expected Result 6: Enhanced capabilities of NMHSs, in particular in developing and least developed countries, to fulfil their mandates. ÞLooking at Key Outcomes (Kos) and Key Performance Indicators (KPIs) ÞHow do we measure?

WMO OMM EC-WG/SOP Questionnaire • The working group reviewed in 2011 a standardized questionnaire to assess the impacts of the achieved results. • Such a survey could help create baselines and targets.

WMO OMM Draft Survey on WMO Impacts of Achieved Results on Members Survey Questions for Capacity Development, ER 6 Key Outcomes 6. 1 -6. 4 KO 6. 1: NMHSs and Regional Centers are improved, particularly in developing and least developed countries

WMO OMM KPI 6. 1. 1: Number of NMHSs and WMO Regional Centers with defined roles in national and regional socioeconomic development activities, in cooperation with international and regional development agencies

WMO OMM KPI 6. 1. 2: Number of NMHSs with programmes and networks for collaborating on service delivery at regional levels

WMO OMM • KO 6. 2: Infrastructure and operational facilities of NMHSs and Regional Centres are improved, particularly in developing and least developed countries

WMO OMM KPI 6. 2. 1: Number of NMHSs with improvement in infrastructure and operational facilities to address identified deficiencies

WMO OMM KPI 6. 2. 2: Number of NMHSs with modernized capacities to develop and disseminate products to their customers

WMO OMM • KO 6. 3: Education and Training Development activities at national and regional levels are improved, especially in developing and least developed countries

WMO OMM KPI 6. 3. 1: The number of RTCs providing education and training support for GFCS related activities • To be completed by WMO Secretariat using other sources.

WMO OMM KPI 6. 3. 2: The degree to which Members are getting value for money from the WMO Fellowship Programme

WMO OMM KPI 6. 3. 3: The degree to which the RTCs support the regional training demands in particular for students from developing and least developed countries

WMO OMM • KO 6. 4: Capacities of NMHSs are enhanced through cooperation and partnerships with other national and regional organizations

WMO OMM KPI 6. 4. 1: Development projects and activities funded through voluntary and extra-budgetary resources • To be completed by WMO Secretariat using other sources.

WMO OMM Your inputs…ideas? • Questions in Survey…. – Are we measuring what needs to be measured to show the capacities of NMHS are being developed? – Are the questions • Too simple (yes/no) to be of use? • Too difficult to answer with enough specificity to show improvement in 4 years? • About right – What other tools does WMO need to develop/improve (if any) to evaluate and monitor CD?

WMO OMM Additional Sources • Survey on the Role and Operation of Meteorological Services (2011) • Regional Surveys • Programmatic Surveys • Information from assessments done on bilateral basis, other development agencies or at sub-regional levels as part of project development

WMO OMM Annex 1 KPI reference sheet to be completed

WMO OMM Annex 2 Example of a duly completed KPI Reference Sheet