ESAC ESTEC PSA status and interactions with

ESAC / ESTEC PSA status and interactions with PDS 26 th March 2010 Dave Heather ESA-ESTEC, dheather@rssd. esa. int D. Heather (22/10/2009) 1

Scope 1. 2. 3. 4. 5. Historical perspective PSA general properties / services Current mission archive status PSA and PDS collaboration / interaction and issues IPDA and interoperability D. Heather (22/10/2009) 2

Definition and Purpose The Planetary Science Archive (PSA) is the initiative, the setup, the process and the implementation to preserve data from ESA’s spacecraft to planetary bodies, as well as supplementary information acquired in laboratories or ground-based observatories. The PSA is the primary resource for all scientific and engineering data from the European Space Agency’s planetary missions. The prime objectives of the PSA are: Ø to support the experimenter teams in the preparation for the spacecraft and ground-based long-term archives Ø to enable and ensure the (long-term) preservation of these archives Ø distribution of scientific useful data to the world wide scientific community Ø provision of supplementary data services aiming to maximize the usage of planetary mission data and ease the scientific data analysis. http: //www. rssd. esa. int/psa D. Heather (22/10/2009) 3

Historical Perspective Ø GIOTTO Science Data Archive – initiated before we had a legal basis in the Agency. NASA participated via the International Halley Watch plus the Mo. Us signed for NASA support (DSN, Navigation). Ø ROSETTA Science Management Plan Ø ROSETTA ESA/NASA Memorandum of Understanding Ø Mars Express and Venus Express Science Management Plans Ø Bebi-Colombo Science Management Plan Rosetta Science Management Plan (ESA/SPC(94)37, page 28): “ 6. Provide the reduced and calibrated scientific data sets from his/her instrument in useable Mars in the Science Management Plan form to the RSOC for inclusion. Express. Mo. U Bepi. Colombo, Science Management Plan ESA/C(2002)148: Rosetta ESA/NASARosetta Science Data Archive. ” Giotto Mission Mars Express Scientific The Science reponsibilities: will prepare the Page. Provide -Operations Centre. DISTRIBUTION final. SCIENTIFIC DATA level. Data 42, RSOC scientific AND (raw data, calibrated data, and higher the data), “ARTICLE 10 RIGHTS IN OF Archive (MESDA) within onelaunched onreceipt 1985 to study comet 1 P/Halley. the The Giotto spacecraft was year of the 2 to the Bepi. Colombo ESA-JAXAfrom Giotto July of the complete data sets archive in including relevant calibrationscience data archiving and products, “the preparationcomet 1 P/Halley on 13 above, Module science- investigations. theteam - to of guidelines for 10. 1 March all scientific and within 596 by Based individual Mars period defined in and Lander 1986 and came supported. Rosetta. PI on Express Orbiter encounteredthat will be agreed with the ESA SOC for application bykm of spacecraft … Following the ancillary the a format create the Rosetta encountered comet ESA Data Library, and NASA’s Small Body Node of Data Archive. ”recommendations on standards July 1992. general current technologydeposited with the 26 P/Grigg-Skjellerup on 10 for scientific data of and IMEWG nucleus. will be data records Giotto science community. future missions to. Data. Scientific(PDS) Archive distributed NASA will place the science data in the U. S. Planetary Mars, the archive would be System Data using PDS format. Page 42: instrument data are archived in co-operation with as a set of CDROMS based on Giotto “ 5. 4 Rosetta NASA’s PDS Small Bodies the NASA Planetary Data System. a public archive in thescience data Node of the U. S. Planetary Data System …” The ESA-JAXA Small Bodydata products from the compatible with the submitted to archive, which will be mission have to Watch … Node this period, the scientific a set of CDROMs (“International Halley be Planetary After and were distributed on Data reduced and calibrated form on and part of the be used by the scientific Campaign”) RSOC in a. System (PDS), will be based such that they can Planetary Science Archive (PSA) RSOC will Mars Express and Rosetta. community. developed for Smart-1, Rosetta Scientific Data Archive within one year of the A close collaborationprepare the Small Bodies node was established and with the PDS receipt of the complete data sets from the individual Rosetta science investigations. Based maintained up to today. on current technology, the archive would be distributed as a set of CD-ROMs. …” D. Heather (22/10/2009) 4

Data Rights and Duties Ø ESA Convention (SP-1271) Ø Resolution on the Rules concerning Information, Data and Intellectual Property ESA Convention, Article V In carrying out its activities under Article V, the Agency shall ensure that any scientific results shall be published or otherwise made widely available after prior use ESA/C/CLV/Rules 5 (Final) by the scientists responsible for the experiments. The resulting reduced data shall be the property of the Agency. “OWNERSHIP, ACCESS, USE AND DISSEMINATION OF RAW AND CALIBRATED DATA RESULTING FROM A PROGRAMME OR ACTIVITY OF THE AGENCY 1. OWNERSHIP OF RAW AND CALIBRATED DATA The Agency shall be the owner of all raw and calibrated data directly resulting from a payload flown in the context of an Agency’s programme or activity (excluding any data which are required for the control of the payload itself) including when the payload is furnished by a Provider. ” D. Heather (22/10/2009) 5

Planetary Science Archive • Available since March 2004: http: //www. rssd. esa. int/PSA • Active development • Datasets received from PI teams Needs to be Peer-reviewed by independent team Needs to be validated before being ingested into the PSA (check PDS format compliance) • Around 13 TB of data on hard disks • Classical User Interface • Map based interface for MEX • Dataset Browser (not searchable) D. Heather (22/10/2009) 6

PSA : one archive, several missions Huygens Giotto Rosetta Smart-1 Venus Express Mars Express D. Heather (22/10/2009) 7

PSA-DH Personnel and Mission Contacts D. Heather (22/10/2009) 8

ESAC Science Archives Team Christophe Arviset ESA 2 FTEs for PSA Pedro Osuna* ESA Data Access Group User Interface Group Jesus Salgado* INSA Inaki Ortiz* SERCO Neil Cheek SERCO D. Heather (22/10/2009) Ignacio Leon INSA Monica Fernandez INSA 9

End Users, Services and Functionalities Offered Ø PSA Scientific Advisory Group Ø Helpdesk, Notification Ø Data Query and Retrieval Ø PDS Standard Support (currently only for Version 3!!) Ø Coordination and chair for Peer Reviews Ø Design, Production and Delivery of auxiliary data Ø SPICE conversion Ø Software consultancy and support Ø ESOC data long-term archive preparation Ø Science planning information, long-term archive preparation D. Heather (22/10/2009) 10

PSA Advanced Interface PSA supports 4 different views: -Query -Latest Results -Shopping Basket -Login/Register -(Request Monitor to come) Selection of Result Display Options Query Panel are ANDed when querying the database D. Heather (22/10/2009) 11

PSA UI – The Map Interface Search Button Select Base Map Type (MOLAInstrument / Select / VIKING…) Detector 1. Click and drag to select your area of interest Zoom function / day/night projection 5. 2. Press ‘Search’ Transfer at any time toand wait for interface the standard to ‘standard’ UI Access footprints the map for. Return to searches. 6. advanced Your search parameters browser, you can view will 3. Clickboundaries be remembered. day/night on/ desired Long Lat information footprint and sub-solar point for selected footprint. 4. Download directly or view label / docs etc. D. Heather (22/10/2009) 12

PSA Browser Interface Opening Screen Click on the instrument that interests you Select the data set you wish to look at Browse through the directories to locate the files you want Right click the product you want to save etc. to bring up the menu Left click to view directly (where possible) D. Heather (22/10/2009) 13

Data Producers, Services and Functionality Offered Ø Data Archive Workshops on request (internal and external) Ø SPICE Workshops (on request – in co-ordination with NAIF) Ø Individual Archive Consultancy from early phase Ø Coordination of Mission Data Archive Working Groups (DAWG) Ø Coordination and support of Mission geometrical parameter information Ø Dataset Support/Development Tools Ø Product generation from telemetry to L 2 (GDP) – used for C 1 Ø Dataset validation Ø Dataset ingestion D. Heather (22/10/2009) 14

PSA Data Producer Services • We offer a PSA Volume Verification Tool (PVV) Data producer » Can verify the structure of the whole data set » Can verify the content of the ODL language » Can verify the references, catalogue structure, etc Database » Can easily ingest data sets (standard delivery within one week) • PVV includes Semi-automatic update of the tool Direct connection to the PSA database (network based) » Always the latest dictionary Support of former dictionary versions • PVV does not include A guarantee of PDS compliance! • PVS: qualitative validation now operational!! D. Heather (22/10/2009) 15

PSA Management Services Ø Statistics Tool (PST) Ø Currently does NOT account for FTData set Browser access. Ø Notification Management Ø Data Set and User Administration (PAT) Ø Dictionary Administration (PDT) D. Heather (22/10/2009) 16

Current Status, Available Datasets Ø GIOTTO: data from comet Halley and Gripp-Skellerup Ø Comet Halley ground-based observations (Halley-Watch) Ø Comet Wirtanen ground-based observations Ø Mars Express: instrument and auxiliary data available (~11 Tb) Ø HRSC Radiometric, Map-Projected (to July 2009) and DTM (to Dec 2006) data. Ø ASPERA NPI and ELS data up to June 2009. Ø OMEGA, SPICAM, MARSIS and Ma. RS data all available and deliveries continue. Ø PFS data only on FTP. Ø Calibrated data from several instruments now coming in. Ø Working group on Mars upper atmosphere D. Heather (22/10/2009) 17

Current Status, Available Datasets Ø Huygens: instrument and housekeeping data available for all instruments Ø New GCMS data has just been sent Ø DISR re-ingestion planned with improved ‘processing’ Ø Venus Express: first data sets released Spring 2009 Ø VMC, SPICAV-SOIR, MAG available and regular deliveries. Ø First SPICAV data now available. Ø VIRTIS still closing out review, but available via ftp. Ø Rosetta: first data release still in preparation, pending PDS/PSA agreement Ø PDS collaborations ongoing Ø Validation is a major issue. Ø Steins fly-by joint PSA / PDS review last October last year. Very successful. D. Heather (22/10/2009) 18

Current Status, Available Datasets Ø SPICE: Rosetta, MEX and VEX data sets released. New versions will be ingested after Easter. Ø Chandrayaan-1: data pipeline running at ESAC for CIXS and SIR 2. Ø SMART-1: AMIE, SIR, D-CIXS and XSM data being prepared for release in May. Ø Bepi. Colombo: data handling and archive support now taking shape. New approach for ESA. Ø Updated PSA WWW area – continuous improvements: Ø Tools Ø Documentation Ø FTP Access Ø http: //www. rssd. esa. int/PSA D. Heather (22/10/2009) 19

PSA and PDS Interactions • It was decided at an early stage that PSA data would comply with PDS Standards: To maximise the cross-compatibility of ESA and NASA data Existing and well established standard in the community • We have good contact and excellent support from several of the PDS Nodes: MEX: Geosciences / PPI. MOU between ESA and NASA Rosetta: Small Bodies Node (primary). International mission. VEX: Atmospheres Node. Interoperability. Review support. Ancillary data: NAIF. SPICE used for all ESA planetary missions. D. Heather (22/10/2009) 20

PSA and PDS collaboration Rosetta issues Rosetta first review took place Feb 2007 but data are still not available Full ingestion (and hence compliance) required at PDS How does one ensure full compliance? VTOOL? LVTOOL? TBTOOL? PVV? Others? REVIEW? FULL / OPEN PSA/PDS INVOLVEMENT IN ARCHIVE DEVELOPMENT CYCLE? ALL OF THE ABOVE… MORE!? D. Heather (22/10/2009) 21

PSA and PDS Interaction -> IPDA / PDS 4 • Development of our validation tool has highlighted several inconsistencies in the PDS Standards and the way in which various experts interpret them » IPDA • PDS and PSA are drawing upon the lessons learned on both sides of this working relationship in order to refine and streamline the Standards. • These ‘lessons learned’ are being used by the IPDA to help develop a refined set of IPDA recommendations for archiving that will allow for international archive interoperability and avoid similar issues in future. • Some issues also input to PDS 2010 (PDS 4) development. • Involved in PDS 2010 System Review and in IPDA assessment of PDS 4 D. Heather (22/10/2009) 22

IPDA - VEX Interoperability Ø VEX - First interoperability studies between PDS and PSA Ø Access to data remotely Ø Full search functionality Ø Transparent access to data on various servers Ø Planned for PDS data access on VEX D. Heather (22/10/2009) 23

Open Issues “Difficulty” to control Dataset delivery schedule Affects PSA-DH schedule Affects PSA-DT schedule Incomplete deliveries to the scientific community as calibrated data deliveries are not available in time by some experimenter teams on the on-going planetary missions Still not fully resolved Insufficient resources within some of the experimenter teams To be solved with new concept in Solar System Science Operations Division - but can only be done partially. Issue was addressed with delegations (SPC) on a number of occasions Parallel activities in member states – we try to coordinate and focus resources D. Heather (22/10/2009) 24

Open Issues Standard evolution. With IPDA (all international partners), and maintaining our well established links with PDS, we follow PDSStandard Evolution. Peer Review handling – data are sometimes slow to be released through the formal procedure Requires time and significant resources to perform review Delay in DS ingestion into PSA Data set ingestion turnaround Manual process Potential Remedy actions Planning & monitoring through Dataset Spreadsheet Try automate ingestion process? Review Peer-review concept? Immediately release the incoming Datasets as “provisional”? D. Heather (22/10/2009) 25