Federation Past Present and Future a personal view

Federation Past, Present and Future: a personal view John Townshend, Past President of the ESIP Federation, 1999 -2000 Global Land Cover Facility ESIP University of Maryland National Research Council – July 2001

Presentation Outline • The goals, scope and operating mode of the Federation. • A brief history of the Federation. • What is the relationship of the Federation to EOSDIS and New. DISS? • How successful has the Federation been in reaching communities? • Future of the Federation National Research Council – July 2001

What is the Earth Science Information Partnership? • Set up as an experiment in science governance environmental data products and services. • Cross-cutting involving government, academic, commercial and non-profit participants. • Brings together data and rapidly evolving technologies to meet the needs of many Earth Science missions. • Acts as a framework for coordinating and expanding upon pioneering research set forth by the Partners. • Prototype for the next generation Earth Science Enterprise model. National Research Council – July 2001

1) The goals, scope and operating mode of the Federation. • The Goals of the federation are to increase the quality and value of Earth Science products and services throughout their life cycle. • The Beneficiaries are all the Federation’s stakeholders. • Achieving the goals will be by continuously improving all of the science-based processes underpinning its goods and services. National Research Council – July 2001

Federation Scope: Core Activities • Encouraging the use of best science practices to ensure the quality and breadth of data and resultant information, products and services. • Ensuring that data and information can be readily exchanged and integrated to improve Earth science data, information, products, and services. • Contributing to the development of an Earth science information economy through application research and commerce. • Increasing the diversity and breadth of users and uses of Earth science data, information, products and services. National Research Council – July 2001

Federation Partner Types • Type 1 ESIPs (principally the DAACs) – standard data and information whose production, publishing/ distribution, and associated user services requires considerable emphasis on reliability and disciplined adherence to schedules. • Type 2 ESIPs – data and information products and services in support of Earth System Science that are developmental or research in nature or where emphasis on flexibility and creativity is key to meeting the advancing research needs. • Type 3 ESIPs – providing ESE-based data and information products and services to users beyond the Earth System Science research community. • Type 4 ESIPs – Large institutional sustaining members – currently only NASA. National Research Council – July 2001

ESIP Types 2 s Creating and distributing remote sensing data using a variety of innovative tools and interfaces. • • • • The GPS Environmental and Earth Science Information System The Ocean ESIP The Passive Microwave ESIP The Numerical Terradynamic Simulation Group The Global Land Cover Facility Snow. SIP The Seasonal to Inter-annual ESIP IBM Watson Research Center Public Health ESIP The Southwest RESAC The Tropical Rain Forest Information Center Great Plains Regional Earth Science Applications Center National Research Council – July 2001 The Distributed Oceanographic Data System (DODS) Terrestrial Environmental Research The Earth Science Partners Private Network The Earth System Science Workbench

ESIP Types 3 s Working directly with user communities to provide targeted environmental products and services. • Bay Area Shared Information Consortium • California Land Science Information Partnership • The Earth Data Analysis Center • • • • The Environmental Legal Information System Museums Teaching Planet Earth Northeast Regional Earth Science Applications Center The Earth Data Discovery Consortium Reading Information Technology Incorporated Scientific Fishery Systems Terra. SIP Veridian MRJ Technology Solutions The Upper Midwest Aerospace Consortium Stormcenter. com TERC Mid-Atlantic Regional Earth Science Applications Center Southern California Wildfire Hazard Center National Research Council – July 2001

2. A brief history of the Federation NRC meeting held at La Jolla 1995 ESIPs Types 2 and 3 initiated in March 1998 Federation Beginnings: Initial Working Prototype: • NRC recommendation • No Single contractor • USGCRP found it controversial • Exploration of selfgovernance • NASA took on the Federation Experiment • Self-organizing elements, “cluster” • NASA DISS evolves – PI processing, New. DISS • DAACs become part of Federation • Exercise in “Coopetition” National Research Council – July 2001 7 th Federation Meeting, Grand Forks, ND July 2001 Ongoing Evolution: • Technological, scientific and structural innovation • Heterogeneous but highly communicative structure leads to rapidly adaptable systems which can swiftly meet changing ESE needs • Begin Mapping to New. DISS • Foundation initiated

Organizational Structures of the Federation • Development of Organizational Structures – Formal structures (such as Committees and Working Groups, e. g. for Interoperability and developing the information economy). – Informal structures (namely Clusters to allow members of the Federation to organize as they please) – Formal rules of governance set up in January 2000 – Not-for-profit legal structure July 2001 National Research Council – July 2001

3. The Relationship of the Federation to EOSDIS and New. DISS National Research Council – July 2001

DAACs National Research Council – July 2001

Federation Extensions to EOSDIS Terra Satellite Observation Collection Calibration and QA Raw Data Received Assessment HDF-EOS, Integerized Sinusoidal standard products Source Data Entry Data Processing Type 1 EOSDIS National Research Council – July 2001 MODIS Vegetation Continuous Fields Reprojected, Reprocessed products MODIS Land Rapid Response for US Forest Service National Interagency Fire Center Remote Sensing of the Environment Value-added Products Models Type 2 Publications Targeted Applications Type 3 ESIP I’s ensure data availability and long-term archiving ESIP 2’s add value to and ensure the scientific utility of key EOS data and products ESIP 3’s apply EOS data for commercial and nonprofit uses

Proposed New. DISS Characteristics • New. DISS can be regarded as the logical evolution progression from the Federation • Structure is heterogeneous – participants (investigators & institutions) – technological approaches (many capabilities exist and many more to be developed) • • Distributed and flexible, responsive systems Smaller, more manageable pieces Driven by competition Establish a framework to integrate ESIP activities. – Define a core set of interface standards and practices – Utilize community-wide interface standards National Research Council – July 2001

Evolution of New. DISS Individual researchers, no data management coordination Highly structured data system connecting various data repositories National Research Council – July 2001 Inclusion of PI data processing in heterogeneous system processing, archive and distribution

Relationship to New. DISS System Wide Interoperability Layer National Research Council – July 2001

Federation as a Testbed • The Federation by definition is a broad spectrum of Earth science data centers. The Federation is a ready-made laboratory for testing New. DISS concepts and reporting on the next set of “lessons learned”: – Federation focuses on tasks that involve cross center issues that strengthen and build the Federation from within. – Each ESIP is independent - yet they work toward the common good; the whole is greater than the sum of its parts – Federation is lightweight, nimble and thus can conduct rapid prototyping: MODIS cluster – collaboration between backbone centers and science and mission centers to improve dissemination of MODIS-based products Ø LBA cluster – provides assistance to LBA project in acquiring and disseminating products Ø National Research Council – July 2001

Federation Data and Information Interoperability • A critical aspect of the evolution of the Federation focuses resources and expertise toward improving data interoperability. The Federation has adopted and produced solutions including: – Catalog-level data searching via the Internet (Using GCMD and Mercury) – Allowing distributed data exchange using novel systems (DODS) – Numerous working prototypes for data discovery and access (MOCHA, GLCF ESDI, TRFIC Landsat Portal) – The adoption of both ‘core’ (HTTP, XML, FGDC metadata) and ‘community’ (DIF, Open. GIS, Z 39. 50) standards for data exchange. National Research Council – July 2001

4) How successful has the Federation been in reaching communities? September 2000 Quarter Ratio of Downloads/Data Available Average Data Delivered per ESIP 324, 150 Number of New Data Sets Institutions Using ESIP Software 91. 3 % MB 153 537 March 2001 Quarter 139. 3% 339, 305 MB 204 632 Select Metrics to Date — An average of 113. 16 data sets available per ESIP — 204 data sets not available before the Federation — An average of 9. 58 services available per ESIP — 65 services not available before the Federation — — — 1, 910, 591 Products delivered to users 13. 7 Terabytes data provided to users 84, 718 Services provided to users National Research Council – July 2001

How successful has the Federation been in reaching private sector, policy, and general public users? The Federation has been challenged by NASA management to broaden the user base for remote sensing data and value-added derived products. To date there are numerous examples of significant progress in reaching non-traditional user communities. COMMERCIAL DATA & INFO SUPPLIERS • The commercial market place has been a particular target for Federation partners. The Type 3 ESIPs are the Federation partners who move environmental data into new for profit market areas. RITI Marine Solutions offers the Interfaces for viewing remote sensing data are being developed for commercial partners. integration of dynamic resource (vessel) with internal/external databases, visual geographic presentation, and storeforward full length ship tracks (proprietary algorithm for effective storage). Transferring Earth science data to the public sector by Federation partners. Storm. Center, for example, draws from existing and emerging remote sensing data and passes these on to numerous users. COMMERCIAL EDUCATION DEVELOPERS • There are over eleven Federation partners adding value to Earth science data so that new markets for these data are created. The Challenger Center has now been replicated at 29 other sites in the U. S. and Canada. National Research Council – July 2001 Ocean Expeditions: El Niño is the 1998 CODIE award winner for Best Curriculum Software for Secondary/Post-Secondary Schools. The Federation also reaches non traditional commercial interests through consulting and environmental legal services.

How successful has the Federation been in reaching private sector, policy, and general public users? METRICS of REACHING NEW USERS • To assess the impacts and reach the Federation is having on the commercial and general public access and use of environmental data and services the following data is useful: Percentage of web users from known domains reveal for the latest reporting Quarter 2001. These numbers demonstrate how Federation partners are attracting users from both commercial and the general public. • BASIC 20% Commercial users • Snow. SIP 48% General Public access • Passive Microwave 49% General Public access • Ocean ESIP 44% Commercial users Success stories from non-traditional user communities. The Federation collects ‘nuggets’ from our partners that address particular data and information questions. The following demonstrate the number of entries in a few relevant categories since September 2000. • Broadening the Earth science user community to include non-traditional users. • Advancing Earth science data and information to the commercial market. • PR events (e. g. Presentation by ESIP staff at conferences or community events). Thanks from a John Q. Public citizen for real time global satellite data, Passive Microwave ESIP, Third Quarter 2000 National Research Council – July 2001 86 entries 37 entries 35 entries “Thank you so very much for making this type of information so accessible to the public. It's just so wonderful to be able to see the patterns on the "whole" globe in real time. ”

Distribution of data and service users by domain, First Quarter 2001 August 2000 to March 2001 Commercial: 11% 14% Education: 23% 34% National Research Council – July 2001

Federation Services and Applications Commercial Fishing Urban Planning Precision Farming Weather Ranching K-12 Instruction Land Management Application Providers Data and Information Providers The ESIP Federation National Research Council – July 2001 Science-driven Processing Products and Services Providers Energy

The Federation Aids Many User Communities • Development of commercially viable products such as diverse as Palm applications to data brokering. • Provision of products, services and technologies to critical elements of our nation’s economy and infrastructure: – Farming: BASIC’s Vineyard Infestation Defense Map – Ranching: EDAC’s Rangeland Toolkit – Commercial Fishing: Scientific Fishery Systems, Inc. ’s Fish. Trek – Weather: RITI’s Weathe. Route GIS Software – Energy: Snow. SIP’s Hydroelectric Power Flow Estimates – Media: Stormcenter. com’s Meteorological Information for the Media National Research Council – July 2001

UMD, GSFC and USFS Fire Rapid Response • Rapid response provision of Active Fires and Reflectance (1 to 6 hrs) • Operational prototyping of standard MODIS L 2+ products. • Automatic system developed since February 2001 • Daily Reflectance and Active Fire mapped to standard NIFC regions • Builds upon IMAPP software developed by Univ. of Wisconsin • Transfer of Rapid Response software to USFS VCC Burned Area, ID/MT • Remote Sensing Applications Center (Salt Lake City) • Fire Science Lab (Missoula) • 2001 bridging activity to provide MODIS products while DB stations are built • Enhanced VCC Burn Severity and Smoke / Aerosol products 25 km National Research Council – July 2001

MODIS FIRE PRODUCTS Northern Australia, July 29, 2001 National Research Council – July 2001 Sicily, July 23, 2001 500 m

Fires in Eastern Siberia, a. m. July 30 th 2001 Fires National Research Council – July 2001

5) Plans for evolving the Federation • Widening the Federation: several new ESIPs have already joined. • Focussing on the latest EOS Data, e. g. MODIS Cluster. http: //www. modis-cluster. org • Improving interoperability - focussing on the data level rather than directory and catalog levels. • Developing strategic alliances outside of the Federation e. g rapid fire response, links to international programs, e. g TRFIC with GOFC and START. • Setting up a parallel Foundation: a legal entity so that funds can be received by the entity rather than individual ESIPS. National Research Council – July 2001

New sources of funding • Note: all ESIPs leveraged off other funds; ESIP 3 s only 50% funded; cost shares in ESIP 2 s. • Some ESIPs are developing using private capital • Inclusion of NCDC means that NOAA funds are now included. • No figures directly available but all ESIPs are seeking extra funding by multiple routes: other NASA sources, other agencies (e. g. USFS), not-for-profits (e. g. WRI). • But without some core funding the future of the Federation will be difficult and may disappear. – Funds for DISS are difficult to obtain. National Research Council – July 2001

Current Senate Mark Up 1998 1999 National Research Council – July 2001 2000 2001 2002

Current Senate Mark-up University Pork Algorithm Dev. Federation National Research Council – July 2001 “Synergy” ECS Increase ECS

Explore the ESIP Federation www. esipfed. org www. esipfed. net www. modis-cluster. org National Research Council – July 2001

Extra slides National Research Council – July 2001

Bay Area Shared Information Consortium: Vineyard Infestation Defense Map The initial study area National Research Council – July 2001 • Infestation mitigation through species monitoring • Monitoring of glassywinged sharpshooter • Mapping of vineyards and other potential habitats • A partnership between BASIC, county government, academia and Central Coast Pierce’s Disease Task Force

Earth Data Analysis Center: Rangeland Toolkit • A comprehensive rangeland management toolkit, integrating: Examples of data available through the Toolkit: Bareness (top) and drainage (below) National Research Council – July 2001 – Satellite Data – Landscape attributes – Socioeconomic data • Data, metadata and previews are available for download • A testbed for valueadded products based on EOSDIS data and technologies

Scientific Fishery Systems, Inc. : Fish. Trek • A comprehensive software package for mapping fisheries • Integrates bathymetric maps, GPS and automated catch logging and mapping – Inputs largely taken from NOAA ship sounds Fish. Trek Commercial Fishing Software National Research Council – July 2001 • Vector-based datasets increases usability for the novice • Fully commercialized product

Reading Information Technology Inc. : Weathe. Route GIS Software • An Electronic Navigation system (ENC) designed to help mariners avoid heavy weather at sea • Based on NOAA Significant Sea Height forecasts • Provides near real-time forecasts to captains: A value-added GIS with full navigation capability National Research Council – July 2001 – – Up to 5 days in advance Improves safety and efficiency Increases fuel efficiency Decreases hull stress

Snow. SIP: Hydroelectric Power flow Estimates • Population increase and agriculture make water resource management critical, especially for the State of California • SNOWSIP has calculated hydroelectric power flow estimates based on snowmelt • Additional measurements: The use of remotely sensed satellite data allows for the accurate (97%) modeling of snowpack and the corresponding volume of runoff National Research Council – July 2001 – Daily/diurnal variations in radiative inputs to snowpak – Effect of climate variability on snowmelt

Stormcenter. com: Meteorological Information for the Media Examples of services available through Stormcenter. com National Research Council – July 2001 • A subscription service for news stations, nationwide • Current issues in weather and nature addressed with satellite and in-situ imagery: – Visualization of breaking environmental news – Broadcast graphics print, web and broadcast-ready – Local to national-scale imagery – Automatic notification for subscribers

Upper Midwest Aerospace Consortium: Crop and Range Alert System Anomalies reported by the alert system National Research Council – July 2001 • Assessment of forage quantity and quality by tracking rangeland biomass and nitrogen • AVHRR-NDVI used as primary indicator • Near real-time • Provides a more synoptic view than traditional range assessment methods

Planet Earth Science, Inc. : Ocean Expeditions: El Niño One of the resources available to students: a Sea Surface Temperature Anomaly Map National Research Council – July 2001 • Students become the crew on a virtual ship exploring the World’s Oceans • Using virtual reality, students participate in an authentic science project • Real-time satellite data is used to monitor interactions between ocean temperature, currents, winds and atmospheric pressure

Global Land Cover Facility: Provides Imagery for Disney’s Animal Planet • What makes image brokering attractive to the public sector? : – – The Golden Tamarin Habitat as imaged by Landsat 7 National Research Council – July 2001 Speed of delivery (FTP) Cost of product (below market) Custom formats, projections Science and technology support • Animal Planet required a scientifically valid means to map the habitat of the Golden Tamarin but had limited funds • Through a cooperative arrangement imagery was expeditiously provided

MODIS FIRE PRODUCTS Northern Australia, July 26, 2001 National Research Council – July 2001 Sicily, July 25, 2001 500 m

Relationship to EOSDIS: Lessons Learned • Information technology outpaces the time required to build large, operational data systems and services • Data systems and services should leverage off emerging information technology; and not try to drive it • A single data system should not attempt to be all things to all users • A single, large design and development effort stifles creativity • Future information systems will be distributed and heterogeneous in nature National Research Council – July 2001

Distribution of Data and Service Users by Domain, Third Quarter 2000 National Research Council – July 2001

National Research Council – July 2001