Recommendations from the 7 th CAA Operations Review

Recommendations from the 7 th CAA Operations Review H. Laakso 8 th CAA Operations Review 5 June 2013 1

4 th - 6 th Operations Reviews q A few updates done, see documents § CAA-EST-RP-0061_v 13 § CAA-EST RP-0071_v 12 § CAA-EST RP-0081_v 11 q Visualization of particle distribution functions, including angle-angle plots and pitch-angle distributions, for CIS, PEACE, and RAPID § CAA QL tool displays four types of distribution function plots, one at a time § Plot types are: Sauvaud, angle-angle, energy-pitch angle, wheel § The visualization tool is available only on–demand (http: //caa. estec. esa. int/caa_stage/) q Data mining tool § Beta-version installed on the CAA q Determination of boundary crossings § Contract terminated, CAA will develop the algorithms and produce the datasets 8 th CAA Operations Review 5 June 2013 2

New Visualization Tool – Particle distributions (beta) 8 th CAA Operations Review 5 June 2013 3

Example: Sauvaud Plot for Magnetopause Crossing 8 th CAA Operations Review 5 June 2013 4

Example: Wheel Plot for Magnetopause Crossing 8 th CAA Operations Review 5 June 2013 5

4 th Operations Review (2009) q The replies to the recommendations are presented in document CAA-EST-RP 0061_v 13 § 3 recommendations for CAA remain open R 9. The board recommends that the CAA develop data-mining tools to be included in the web interface. This should include the ability to identify time intervals by searching for mean/variance/min/max criteria on a combination of physical and ephemeris data sets. Both a GUI interface and command line interface should be implemented. A standard time-table format should be adopted. A prototype interface should be available for testing by autumn 2009. OPEN. Survey level products have been identified and can be produced by the CAA using the existing datasets. Beta-version of the data mining system has been implemented and is available on the CAA. 8 th CAA Operations Review 5 June 2013 6

4 th Operations Review (2009) R 10. An addition to the data-mining tools could be automated identification of magnetospheric boundaries and regions. The board recommends that the CAA determines if such techniques have already been developed within the scientific community. OPEN. The CAA placed a small contract to FMI to create a complete list of crossings of the magnetopause, bow shock, neutral sheet and electron radiation belt. This work was not feasible and therefore the CAA has taken over the project and will create the datasets in 2013. In addition the ECLAT EU/FP 7 -project lead by Steve Milan will produce some boundary datasets in the tail. This project will be finished in March 2014. 8 th CAA Operations Review 5 June 2013 7

4 th Operations Review (2009) R 15. Double Star, the first bilateral mission between ESA and China, ceased collecting data in 2008, and the existing data currently resides with the individual instrument teams in Europe and China. Without proper archiving of the Double Star data it will eventually be lost to the scientific community as happened many times in the past with previous space missions. The board is aware that activities to ingest Double Star data into the CAA have begun, as recommended in OR 3. However, the board fears that in the absence of additional funds for the CAA to complete this work, this important activity will fail. The quality of the data ingested cannot be guaranteed without proper investment in calibration and data validation activities. Without extra funding, some of the instrument teams will be unable to provide their data to the CAA. The board urges that the CAA seek additional support to continue quality assurance and archiving of the Double Star data. OPEN. All raw data have been reprocessed by the Chinese. However, the files caused more questions than solved them. Ideally the final data products are based on a combination of old and new raw data. Different teams are adopting different scheme in using raw data files. 8 th CAA Operations Review 5 June 2013 8

5 th Operations Review (2010) q The replies to the recommendations are presented in document CAA-EST-RP 0071_v 12 § 5 recommendations for CAA remain open OR 5 -R 7 Users often wish to examine pitch angle distributions for particle energies in specific narrow ranges. The board compliments the CAA and particle teams on their efforts to provide pitch angle distributions for fixed energy ranges and requests that they implement a system that allows the user to select any energy band for pitch angle plots interactively. OPEN. The CAA graphics will be updated to include a number of filtering options per panel. 8 th CAA Operations Review 5 June 2013 9

5 th Operations Review (2010) OR 5 -R 8 Data mining techniques often return lists of intervals when interesting events are occurring. Users will then likely want to download one or more relevant datasets for these intervals. The board requests that the CAA implement a system that enables users to download data from intervals identified from data mining. OPEN. The implementation of this feature is related to another feature where a user can create lists of intervals and save them as user profile. OR 5 -R 9 Some parameters (e. g. density) may be obtained by various methods from different instruments. The board requests that the CAA users guide include a section describing which datasets provide specific physical parameters, e. g. density, electric field etc and which datasets should be selected for various locations and times (see http: //themis. ssl. berkeley. edu/instrument_esa. shtml and http: //themis. ssl. berkeley. edu/flow_diagram. shtml) OPEN. This is a challenging request and no simple judgement can be made between different methods. One could create a section in the CAA user guide that describes different methods and their strengths/weaknesses/ caveats in general. 8 th CAA Operations Review 5 June 2013 10

5 th Operations Review (2010) OR 5 -R 10 The board welcomes the transition of the CAA to a long term permanent archive after the end of the mission. The board notes that the services provided by the CAA result from extensive interactions with the scientific community via a number of peer reviews and interactions with the board itself. The board wishes to ensure that the permanent archive maintain the full range of services currently provided by the CAA. The board therefore recommends that discussions of the transition to a permanent archive should include not only CAA and permanent archive staff, but also members of the science community who can help define requirements for the permanent archive. OPEN. The CFA (Cluster Final Archive) will become operational before the end of Cluster operations. However, the CFA contains only the user interfaces to access the Cluster data. The CAA will stay active and will take care of all the calibration/cross-calibration and data processing issues. When the CFA user interfaces are acceptable, the user access to the CAA will be stopped. The specification review of the CFA will be held after the 6 th Operations Review of the CAA. The CFA will be reviewed similarly to the CAA to ensure that the same functionalities exist. 8 th CAA Operations Review 5 June 2013 11

5 th Operations Review (2010) OR 5 -R 12 The board notes the immense value of Double Star observations in conjunction with Cluster observations. Both the CAA and permanent archive should include and provide the best version of Double Star observations available. Since the final version of Double Star observations may not be available by the time of the transition to the final archive, the final archive must have the ability to incorporate Double Star observations. OPEN. The CFA will be the repository of all CAA files and will be able to include new datasets and files to the CFA database after they were ingested into the CAA database. This includes the Double Star but also EU/FP 7 ECLAT datasets. 8 th CAA Operations Review 5 June 2013 12

6 th Operations Review (2011) q The replies to the recommendations are presented in document CAA-EST-RP 0071_v 12 § 1 recommendations for CAA remain open OR 6 -R 11 The board notes that the data-mining facility is a great success, as is the data-streaming interface, and we thank the team for their hard work. We recommend that these are implemented in their full version as soon as possible. The data-mining facility provides a new level of science exploitation of the archive, and we encourage its implementation in the CFA as well, as noted below. OPEN. The beta version is available on the CAA and it will be implemented on the CFA at a later stage. 8 th CAA Operations Review 5 June 2013 13

6 th Operations Review (2011) OR 6 -R 17 The board recommends that the data-mining facility demonstrated by Chris Perry be implemented within the CFA (need 1, priority 2). OPEN. The beta-version is operational on the CAA, but this may not be implemented on the CFA before closure of the CAA. OR 6 -R 18 The board recommends that data-streaming be implemented in the CFA (need 1, priority 2). OPEN. This interface is used by several users and this needs to be implemented before closing the CAA. OR 6 -R 19 The board recommends that a first implementation of data visualization is supported in the beta version of the CFA (need 1, priority 1). OPEN. CFA will use the CAA IDL graphics. The on-demand plotting has been functional for 6 months and the pre-generated plots have been under development (processing, database, display) 8 th CAA Operations Review 5 June 2013 14

6 th Operations Review (2011) OR 6 -R 21 The board is concerned that during the CAA hibernation period, there will no longer be the opportunity for data ingestion into the CFA in the event of significant reanalysis of data products. The board recommends that data ingestion should still be possible in extraordinary circumstances. Such data ingestion will be at the discretion of the CFA archive scientist. The CAA should investigate options to allow this, including whether the ingestion software should be archived for the usage of PI teams. The CAA should contemplate a scheduled revival of the CAA ingestion system (maybe 2 years after hibernation) to facilitate final ingestion of data. The board notes that anything above cannot be used to facilitate late deliveries. OPEN. A simple system will be implemented. This will likely happen during the next 12 months. OR 6 -R 22 The board recommends that the interoperability of the CFA with other space plasma archives is given a high priority to allow comparison with non. Cluster data sets. OPEN. The CFA allows to download datasets in VOTABLES. 8 th CAA Operations Review 5 June 2013 15

Recommendations from 7 th Operations Review q The replies are presented in document CAA-EST-RP-0091_v 10 q CAA and CFA received 15 recommendations § 4 recommendations for CAA where all are closed § 11 recommendations for CFA where 4 are completed and 6 remains open q Instrument teams received 7 recommendations and the teams will answer to those in their reports § 3 recommendations for ALL teams § 1 for CIS § 1 for PEACE § 1 for STAFF § 1 FOR BWD 8 th CAA Operations Review 5 June 2013 16

7 th Operations Review (2012) 8 th CAA Operations Review 5 June 2013 17