Metrics Planning and Reporting (MPAR) WG Breakout Summary H. K. (Rama) Ramapriyan NASA GSFC Clyde Brown NASA La. RC Co-Chairs, MPARWG 8 th Earth Science Data Systems Working Group Joint Working Group Meeting Wilmington, DE October 20 -22, 2009 ESDSWG meeting – 10/21 -23/2008
MPAR Working Group q Attendance – 23 q Topics Discussed Ø Discussions with Martha Maiden Ø Metrics baseline review Ø FY 09 Reporting Ø Core and Community metrics collection ESDSWG meeting – 10/21 -23/2008
MPARWG Breakout Sessions Item Who Day 1 - Session 1: FY 09 Activities and Progress Introduction Rama HQ / Program view of Metrics Martha Maiden's Four Questions and MPARWG responses Maiden Hunolt E-Books vs Metrics - What is Reported Via Each Rama Action / Open Items from Oct 2008 workshop Summary of FY 09 Metrics Reporting Rama Hunolt Day 2 - Session 2: Metrics and MEa. SUREs Telecon Results Metrics Baseline Review Rama Hunolt / All Day 2 - Session 3: Metrics Collection - Core and Community Overview of Metrics Collection MCT & EMS Rama Introduction for MEa. SURES and Changes to MCT Randy Barth Transition to "MCT 2" and EMS for Metrics Reporting Kevin Murphy Day 2 - Session 4: TBD, Action Items and Work Plan Continued discussion of metrics baseline ESDSWG meeting – 10/21 -23/2008 Hunolt/All
Discussions with Martha Maiden q Martha emphasized importance of metrics for her reporting to her management and OMB Ø Metrics from both Community systems and Core systems are important ØUser satisfaction metrics – American Consumer Satisfaction Index is an important metric watched by OMB q q q Aggregates and highlights from core and community metrics are useful Impact metrics are very useful, but a better format is needed Encouraged use of metrics for the benefit of the Projects themselves ESDSWG meeting – 10/21 -23/2008
Actions Planned in Response q q Provide separate aggregates for community systems that include trends, so community systems are not lost in the totals. Provide periodic reports in a form useful to Martha ØProgram level summaries and trends ØHighlights from individual community systems. These could be based on the regular metrics, e. g. noting significant growth in users, distribution, addition of new projects or services, etc. ØProjects could use supplemental comments to highlight / call attention to items. q The form in which this information is provided to Martha could be quad-chart or a short pdf ‘newsletter’. ESDSWG meeting – 10/21 -23/2008
Metrics Baseline Review Products and Services Metrics Common Metrics: ØMetric 1 - Distinct Users – No change ØMetric 2 – Distinct Users by Class – No change ØMetric 3 – Products Distributed – Added option of breaking down by product types ØMetric 4 – Product Types Available ØMetric 5 – Volume Distributed – after considerable discussion, left unchanged ØMetric 6 – Volume Available – after considerable discussion, left unchanged ØMetric 7 – no longer used ØMetric 11 – Services Provided ØMetric 12 – Services Available – need explanatory examples for projects to use this better ESDSWG meeting – 10/21 -23/2008
Metrics Baseline Review q q Project Defined (Project Specific) Metrics (up to 4, defined by each project, metrics 100, 101, 102, 103) Programmatic Metrics: ØMetric 8 – Science Focus Areas Supported ØMetric 9 – Applications Areas Supported ØMetric 10 – Education categories Supported q q Impact Metrics – decided to recommend use of quad chart format, but will develop strawman examples and validate with Martha New Citation Metric ØDeborah Smith (RSS, Inc) presented an approach they have been using to collect citations of their products ØRecommend use of citation metric to be used optionally ESDSWG meeting – 10/21 -23/2008
FY 09 Reporting 22 REASo. N; 15 ACCESS; 8 MEa. SUREs [Many of the REASo. N and ACCESS Projects completed during FY 09 q Impact metrics – 3 from 2 REASo. N and 1 MEa. SUREs Projects q We expect 10 or more MEa. SUREs projects to start reporting in FY 10 q ØGenerally, projects report when they start distributing data to users Last REASo. N and ACCESS Projects are expected to finish in FY 10 q q Upcoming ACCESS reporting - TBD ESDSWG meeting – 10/21 -23/2008
Core and Community Metrics Collection q q Core system metrics ØEOSDIS Data Center metrics collection using EMS was demonstrated ØInternal use of metrics in ESDIS for planning purposes ØNetwork flows vs. requirements are analyzed using periodic “pinging” and measuring actual data flows Community metrics ØCollected using MCT ØFY 09 improvements to MCT discussed (e. g. , 508 compliance) ØTransition to EMS-based collection of community metrics discussed – needs further examination ØRecommended prototyping automated collection with two separate (diverse) projects using log files ESDSWG meeting – 10/21 -23/2008
Скачать презентацию Metrics Planning and Reporting MPAR WG Breakout Summary
c8f9e7db5d664fe0d564e67fb9234d4a.ppt