e SDO Algorithms Visualization and Data Access Elizabeth

e. SDO Algorithms, Visualization and Data Access Elizabeth Auden 27 September 2006 HELAS Workshop Nice

What is e. SDO? e. SDO is an e-science project funded by PPARC to make SDO data and algorithms available to the UK solar community using the virtual observatory. e. SDO Duration: 36 months, October 2004 – October 2007 SDO Launch: October 2008 RESEARCH - Phase A: 1 October 2004 – 30 September 2005 IMPLEMENTATION - Phase B: 1 October 2005 – 30 September 2007 Current e. SDO Project Documentation: http: //www. mssl. ucl. ac. uk/twiki/bin/view/SDO 27 September 2006 HELAS Roadmap Workshop Elizabeth Auden

e. SDO Workpackages I. Algorithms • Code: feature recognition, local helioseismology, global helioseismology • Grid integration: Astro. Grid and JSOC II. Data centres • Astro. Grid interface with US JSOC data centre • Astro. Grid interface with UK (or European? ) data centre III. Visualization tools • Catalogues • SDO Streaming tool • On-the-fly image galleries and movies 27 September 2006 HELAS Roadmap Workshop Elizabeth Auden

e. SDO Project Team • MSSL: Coronal Image Analysis – – • Elizabeth Auden, project manager Len Culhane, PI Lidia van Driel-Gesztelyi Mike Smith RAL: Event and Feature Recognition – Richard Harrison – Andrzej Fludra – Vito Graffagnino • JSOC, Joint Operations Centre Sheffield: Local Helioseismology – Mike Thompson – Sergei Zharkov • US Links: Stanford University Lockheed Martin Birmingham: Global Helioseismology – Yvonne Elsworth – Bill Chaplin – Thierry Toutain 27 September 2006 HELAS Roadmap Workshop Elizabeth Auden

Algorithms Code Feature Recognition / Event Analysis • Coronal loop detection, MSSL • Non-linear magnetic field extrapolation, MSSL • Helicity computation, MSSL • Small event detection, RAL • CME dimming region recognition, RAL Local Helioseismology • Perturbation map generation, Sheffield • Subsurface flow analysis, Sheffield • Local helioseismology inversions, Sheffield Global Helioseismology • Mode parameters analysis, Birmingham • Mode asymmetry analysis, Birmingham 27 September 2006 HELAS Roadmap Workshop Solar image from TRACE mission. Coronal loops marked in blue. Elizabeth Auden

Algorithms Grid Integration JSOC • SDO pipeline at Stanford University / Lockheed Martin • e. SDO algorithms deployed as pipeline modules (automated or user-run) • Data will be obtained locally from JSOC archive Astro. Grid CEA • UK hosted algorithms hosted as web services • Accessible through Astro. Grid workflows • Data transferred from My. Space, remote data centre or user’s local storage Solar. Soft • Algorithm C modules wrapped in IDL • Distribution through MSSL Solar. Soft gateway • Data transferred to algorithm from user’s local storage 27 September 2006 HELAS Roadmap Workshop Elizabeth Auden

Algorithms Astro. Grid Deployment 9 algorithms have been deployed as CEA applications 27 September 2006 HELAS Roadmap Workshop Elizabeth Auden

Data Centres US Data Centre Primary US Data Store • Files stored in SUMS (Storage Unit Management System) at Stanford University with metadata in distributed DRMS (Data Resource Management System) • DSA / DRMS integration for grid data access investigations • VSO (Virtual Solar Observatory) primary US access Astro. Grid / VSO Interface • VSO queries in Astro. Grid workflows through STAP (Simple Time Access Protocol) web service US / UK Network Latency Tests • Test expediency of file transfer mechanisms • Test between Stanford, MSSL, UCL, RAL 27 September 2006 HELAS Roadmap Workshop Elizabeth Auden

Data Centres Possible UK Data Centre Storage • 30 TB disc cache • 60 day rolling cache HMI/AIA • True cache for HMI / AIA • DRMS / SUMS Investigation of DRMS / SUMS Transfer • Export formats from JSOC • Data search – DSA • Data transfer - CEA • Test installation MSSL 27 September 2006 HELAS Roadmap Workshop Elizabeth Auden

Data Centres Possible European Data Centre TLAs: • DRMS: Data Resource Management System (JSOC) • • EU Framework 6 support for European helioseismology data and algorithms Possible full mirror of SDO data in Europe? DSA / DRMS interface for Astro. Grid access to full mirror of SDO data in Europe? DRMS SUMS US 27 September 2006 SUMS: Storage Unit Management System (JSOC) DSA: Data. Set Access (Astro. Grid) DRMS SUMS DSA UK UK datasets HELAS Roadmap Workshop Europe Elizabeth Auden

Data Centres e. SDO / Solar-B Prototype: system perspective (Local) Astro. Grid Helio. Scope XML STAP service URLs (Local) UKSSD CADS file system Note: Astro. Grid components (JES, DSA, My. Space, Workbench) may be located anywhere. 27 September 2006 HELAS Roadmap Workshop Elizabeth Auden

Data Centres User access through Astro. Grid workbench: STAP service access through Helio. Scope, DSA access through the Task Launcher: locate Data. Set Access components, submit ADQL queries STAP service will point to URLs in UK, US or Germany. DSA will integrated with DRMS (Data Resource Management System) in UK or Germany. Helio. Scope: solar data results from Simple Time Access Protocol services. 27 September 2006 HELAS Roadmap Workshop Elizabeth Auden

Visualization Tools Catalogues Small Event / CME Dimming catalogue • Generated from continuously running of small event detection / CME dimming region recognition algorithms • Tables of event / CME statistical information • Likely to be generated in UK Helioseismology Mode Parameters catalogue • GONG month of data generated from mode parameters analysis algorithm • Likely to be generated in US Catalogues can be accessed through ADQL (Astronomical Data Query Language) query to DSA. 27 September 2006 HELAS Roadmap Workshop Elizabeth Auden

Visualization Tools SDO streaming tool User Facilities • Pan in space P • Zoom in space P • Pan in time • Zoom in time Data Products • Lo. S Magnetogram • HMI continuum map • AIA images Future Work • FFT / wavelet analysis for coronal wave investigations 27 September 2006 HELAS Roadmap Workshop Elizabeth Auden

Visualization Tools On-the-fly Image Galleries and Movies Thumbnail Extraction P • Extract thumbnails in US from HMI and AIA science products Image Gallery Generation P • User specifies start time, stop time, cadence, data product • Image gallery generated on the fly in web browser Movie Generation • User specifies start time, stop time, cadence, data product • MPEG movie generated on the fly 27 September 2006 HELAS Roadmap Workshop Find data with Astro. Grid Helio. Scope. Use PLASTIC to load data into visualization tools. Elizabeth Auden

e. SDO and the Virtual Observatory e. SDO has links with Astro. Grid, VOTech, VSO, and the IVOA. VOTech: EU Framework 6 funded Technology / infrastructure for European virtual observatory Int’l Virtual Observatory Alliance Defines standards for VO schemas and coordinates interoperable software between virtual observatories UK-funded e-science project Provide access to SDO data, algorithms and visualization tools through Astro. Grid. UK-funded Virtual Observatory Infrastructure for the UK: data access, applications, virtual storage, workflows, computational resources HELAS Roadmap Workshop Elizabeth Auden US-funded Virtual Observatory Infrastructure to search, obtain and analyze solar data 27 September 2006

Future Work Phase B implementation development now in progress: • Summer / Autumn 2006: – Algorithm integration with Astro. Grid; test integration with JSOC – Unit and science testing for algorithms – Add STAP service client to visualization tools – Add movie functionality to streaming tool – Investigate DRMS integration with DSA • 2007: – Completed algorithms – Full algorithm integration with JSOC and Solar. Soft – Coronal wave analysis in SDO streaming tool – Work with HELAS to explore European SDO data centre – Catalogues 27 September 2006 HELAS Roadmap Workshop Elizabeth Auden

Further Information Links e. SDO: http: //www. mssl. ucl. ac. uk/twiki/bin/view/SDO Astrogrid: http: //www. astrogrid. org VOTech: http: //wiki. eurovotech. org/bin/view/VOTech/ IVOA: http: //www. ivoa. net Software to try Astro. Grid Workbench: http: //www 2. astrogrid. org/desktop/index Helio. Scope: included with workbench e. SDO Streaming Tool: http: //www. mssl. ucl. ac. uk/twiki/bin/view/SDO/Streaming. Tool Contacts e. SDO: Elizabeth Auden, eca@mssl. ucl. ac. uk Astrogrid Solar Science: Silvia Dalla, s. dalla@manchester. ac. uk 27 September 2006 HELAS Roadmap Workshop Elizabeth Auden