Combined use of AMDA CAA QSAS and CL

Combined use of AMDA, CAA, QSAS and CL based on time-table exchange. C. Jacquey, V. Génot, E. Budnik, M. Bouchemit, M. Gangloff, CDPP/CESR B. Cecconi, CDPP/LESIA C. Perry, CAA team T. Allen, A. Rochel, QSAS team E. Penou, J. A. Sauvaud, CESR This talk is related to: -Poster (Jacquey et al. ) this evening -Talks on CAA (C. Perry), QSAS (T. Allen) and CL (E. Penou) Cluster Workshop, Tenerife, March 12, 2008

Some aspects of the studies of space physics need to analyse measurements obtained by several observatories We need to analyse multi-instrument data We need to compare the observations to simulation results We We need to extract and treat a lot of data coming from multiple resources This takes time and energy Dynamical processes The time is the very first parameter The studies consists of analysis/characterisation of events treated in case studies or in statistical studies Creating Time-Tables (or event list) is a common step of the work

Time-Tables (Event Lists) ØTime-Table produced by the users ØTime-Tables coming from statistical studies (e. g. magnetopause crossing lists) sometimes made available to the community (e. g. the Frey’s substorm list) ØTime-Table coming from search tool based on orbitographic or conjonction criteria (e. g. the Query tool of the SPDF/SSCWeb) ØTime-Table coming from search tool based on mathematical criteria applied to the content of the data (e. g. AMDA of CDPP) ØTime-Table coming from visual search tool (e. g. AMDA of CDPP) Exchanging Time-Tables between data centres, services and tools? saving time and energy. Increase of the scientific return.

AMDA Automated Mutiple Dataset Analysis External data Parameter editor ØWeb based service ØTransparent access to data ØAMDA local database (CLUSTER, ACE, THEMIS, GEOTAIL, WIND, …, ISEE) ØExternal database (CDAWeb, MAPSKP, THEMIS/CESR, …) Visualisation editor Download data Conditional search Visual search Time-Table manager ØProduces and exploits time-tables AMDA DOES NOT WORK WITH Internet. Explorer and Safari AMDA is public (http: //cdpp-amda. cesr. fr)

Resources: data centres, services, tools CAA CDPP Mirror THEMIS database Validated high resolution CLUSTER data CDPP database Plots and tools QSAS DARTS CL AMDA Time-Tables Others … SSCWeb Time-Tables GAIA Others CDAWeb SPDF SSL THEMIS database + many services Virtual Observatories VSPO, VMO, VHO, VIRBO, …

Science Case: study and characterisation of the local properties of the cross-tail current Ø Based on CLUSTER data when it encircles the neutral sheet. (i) Current density, profile, gradient (ii) Thickness, location, motion (iii) Distribution function (i+ and e-), flows, … (iv) Turbulence/wave level and spectra (v) … Ø Influence factor: (i) Solar Wind pressure (ii) IMF (iii) Geomagnetic activity: - State of the ionosphere. - Oxygen abundance - Ring current intensity

Use Case: the main steps Selection of the periods of NS encircling condition Access and extraction of modelisation or simulation data (Tsyganenko, CCMC, …) Fo rm att ing Access and extraction of ground (AE, AL, SYM, DST) data ic Bas AMDA t nts tme rea s me Ti in hift CDAWeb, CDPP, AMDA, DARTS, … g D Access and extraction of ACE/WIND/GEOTAIL Interplanetary data at a m er gi ng Access and extraction of CLUSTER data Generic tools User tools CAA Specific tools Results Catalogues Maps Models CL, QSAS or AMDA CL

Creation of time-tables on AMDA/CDPP TT 1, time-table listing the periods when the conditions are fulfilled TT 2, extended time-table

Conditional search

Visualisation

Visual search

Managing Time-Tables Text or XML VOTable

Parameter editor

Data dowload Data merging

First step: producing sub-databases corresponding to the time-tables AMDA -Automated search -Visual search Time-Tables CAA Validated high resolution CLUSTER data Time-Tables rs ete m ta, Da ra pa Da ta , p ara me te rs Standardised sub-databases Other? CDAWeb User tools QSAS CL DARTS Other? CDPP/CESR THEMIS database SSL THEMIS database

How to do that? ØStandardisation of the time-tables ØStandardisation of the organisation of the subdatabases to be produced ØInterface implementation on CAA for reading the timetable and producing sub-databases ØInterface implementation on the tools QSAS and CL ØStandardisation of the time-table description

Step 2: catalogues CAA, CDAweb, AMDA, SSCWeb or CDPP or other services data centres QSAS, CL, AMDA or other tools T_St T_E art nd P 1 … PN Q 1 … QM … T_St T_E art nd P 1 … PN Q 1 … QM …

Future: Archiving the catalogues for availability to the community AMDA CAA Time-table: T 1 – T 2 …. Computation of P 1, . . , PN Catalogues and associated tools available at the data centres CDPP CDAweb Catalogue: T 1, T 2, P 1, …, PN DARTS Sub-database USER Catalogue: T 1, T 2, P 1, …, PN, Q 1, …, QN QSAS or CL USER

Conclusion ØExchange of standardised Time-Tables between data centres, services and tools should allow the researcher - to save large amount of time and energy - to exploit more extended data amount - to increase the scientific return of the data ØFirst step: extraction and exploitation of sub-databases corresponding to the Time-Tables ØSecond step: creation and exploitation of catalogues, similarly to astronomy and solar physics ØThis project starts with CAA, QSAS, CL and AMDA, but of course, other new participant is welcome ØThe specifications of this project need the input of the potential users Poster this evening in order to collect suggestions and ideas. AMDA: contact: amda@cesr. fr Stable version: http: //cdpp-amda. cesr. fr Test version: http: //iapetus. cesr. fr/AMDA_PRETEST/

Use cases Use case = simulation of studies which could be performed by the researchers in order to: Øidentify and analyse their needs Øspecify the user requirements ØImagine, define and specify the system to be developed The developers of the system need the input from the community Obvious requirement: standardisation ØStandardisation of the data format CEF, CDF, Net. CDF, (FITS) beyond that, standard way for representing data of different classes But (particle , waves, Time-Tables, …) ØStandardisation of the data description SPASE (Space Physics Archive Search and Extract, http: //www. spase-group. org/ )

Step 2: sub-database extraction CAA, CDAweb, DARTS, CDPP, AMDA, … On Requirements: %Standard format for time-tables %Interface for reading and managing the time-tables %User interface for selecting data %Optionally, basic on the fly treatment (e. g. data merging, SW propagation shifting, …) %Standardised management of the multiple file output Filename Descriptor Product recovering

download

Pi-Pj/(Xi-Xj)

Step 3: plot production on CL, QSAS, AMDA %INPUT: ▪ time table ▪ corresponding sub-database %Manipulation of the time-table %User edited plot %Launch in batch mode Requirements: Standardisation (i) of the time-table and (ii) of the data Plot editor Batch mode Standardised management of the plot files ▪ Filename ▪ Descriptor ▪ Product recovering

AMDA Automated Mutiple Dataset Analysis Created by the CDPP: http: //cdpp. cesr. fr/ Stable version: http: //cdpp-amda. cesr. fr/ Testing version: http: //iapetus. cesr. fr/AMDA_PRETEST/ Contact: amda@cesr. fr

Example: event search I ele SEE ctr on Old fashion: “paper” search ISEE-data AL IMP PLA 8 P 8 IM AG M AE, AL IMP-8 , AE ISEE MAG Current fashion: web CDAWeb SPDF UCLA CAA CDPP And others Event search takes time and energy

AMDA: the goals Automated Mutiple Dataset Analysis Multiple Dataset: the study of the (multi-scale) dynamics of plasma objects requires to perform the integrated analysis of multi-point and multi-instrument data Automated (or semi-automated) Analysis: help for ØSearch, characterisation, classification of events ØExtensive exploitation of the vast database (statistics) Ø“historical” studies (example: through solar cycles) ØBuilding catalogues ØBuilding virtual constellations ØTime-Table (Event. List) production

AMDA, fact sheet • Built in IDL, C, FORTRAN, Javascript • Works on standardised and “simple” data, Net. CDF • Includes models (Tsyganenko, Shock, MP, NS, …) • Uses external data (now from CDAWeb and soon from MAPSKP, IWF, THEMIS, …) • Forum for user feedback • Twiki for information (development, bugs, …) exchange • Already in use for scientific studies AMDA DOES NOT WORK WITH Internet. Explorer and Safari AMDA is public

Functionalities ØAutomated access to data ØAccess to and on-the-flight ingestion of external data (test version only) ØData merging ØData download ØData visualisation ØUser edited parameter computation on the data ØAutomated conditionnal search ØVisual search Production of Time-Tables

AMDA/”External Data” AMDA System Local Database MAPSKP CDPP CDAWeb others

Needed development on AMDA ØTemporal functions (mean, regression, dispersion estimator, …) ØUser pluggins ØContinuous SW propagation shifting ØScatter plots, map plots, … ØYour suggestion Ø….

Conclusion and questions ØCAA, QSAS, CL and CDPP teams are starting a collaboration for providing to the researchers ways to save time and energy for performing their studies ØOther services and database are invited to join this effort ØWe need input, comments, suggestions from the potential users.