Building the Framework for the National Virtual Observatory: An Information Technology Research Initiative of the National Science Foundation Bob Hanisch and the NVO Collaboration 1 10 June 2002 Hanisch/NVO
The National Virtual Observatory • National Academy of Sciences “Decadal Survey” recommended NVO as highest priority small (<$100 M) project “Several small initiatives recommended by the committee span both ground and space. The first among them—the National Virtual Observatory (NVO)—is the committee’s top priority among the small initiatives. The NVO will provide a “virtual sky” based on the enormous data sets being created now and the even larger ones proposed for the future. It will enable a new mode of research for professional astronomers and will provide to the public an unparalleled opportunity for education and discovery. ” —Astronomy and Astrophysics in the New Millennium, p. 14 2 10 June 2002 Hanisch/NVO
Project Team • NSF ITR project, “Building the Framework for the National Virtual Observatory” is a collaboration of 17 funded and 3 unfunded organizations – – – Astronomy data centers National observatories Supercomputer centers University departments Computer science/information technology specialists • PI and project director: Alex Szalay (JHU) • Co. PI: Roy Williams (Caltech/CACR) 3 10 June 2002 Hanisch/NVO
Proposal Team A. Szalay R. Williams JHU Caltech C. Alcock U. Penn. M. Livny U. Wis. K. Borne ADC/Raytheon C. Lonsdale IPAC T. Cornwell NRAO T. Mc. Glynn HEASARC/USRA D. De. Young NOAO A. Moore CMU G. Fabbiano SAO R. Moore SDSC/UCSD A. Goodman Harvard J. Pier USNO J. Gray Microsoft R. Plante NCSA/UIUC R. Hanisch STSc. I T. Prince Caltech G. Helou IPAC E. Schreier STSc. I S. Kent FNAL N. White GSFC C. Kesselman USC 10 June 2002 4 Hanisch/NVO
Proposal Team - Univ. Depts. A. Szalay JHU R. Williams Caltech C. Alcock U. Penn. M. Livny U. Wis. K. Borne ADC/Raytheon C. Lonsdale IPAC T. Cornwell NRAO T. Mc. Glynn HEASARC/USRA D. De. Young NOAO A. Moore CMU G. Fabbiano SAO R. Moore SDSC/UCSD A. Goodman Harvard J. Pier USNO J. Gray Microsoft R. Plante NCSA/UIUC R. Hanisch STSc. I T. Prince Caltech G. Helou IPAC E. Schreier STSc. I S. Kent FNAL N. White GSFC C. Kesselman USC 5 10 June 2002 Hanisch/NVO
Proposal Team – CS/IT A. Szalay JHU R. Williams Caltech C. Alcock U. Penn. M. Livny U. Wis. K. Borne ADC/Raytheon C. Lonsdale IPAC T. Cornwell NRAO T. Mc. Glynn HEASARC/USRA D. De. Young NOAO A. Moore CMU G. Fabbiano SAO R. Moore SDSC/UCSD A. Goodman Harvard J. Pier USNO J. Gray Microsoft R. Plante NCSA/UIUC R. Hanisch STSc. I T. Prince Caltech G. Helou IPAC E. Schreier STSc. I S. Kent FNAL N. White GSFC C. Kesselman USC 6 10 June 2002 Hanisch/NVO
Proposal Team – Nat’l Obs. A. Szalay JHU R. Williams Caltech C. Alcock U. Penn. M. Livny U. Wis. K. Borne ADC/Raytheon C. Lonsdale IPAC T. Cornwell NRAO T. Mc. Glynn HEASARC/USRA D. De. Young NOAO A. Moore CMU G. Fabbiano SAO R. Moore SDSC/UCSD A. Goodman Harvard J. Pier USNO J. Gray Microsoft R. Plante NCSA/UIUC R. Hanisch STSc. I T. Prince Caltech G. Helou IPAC E. Schreier STSc. I S. Kent FNAL N. White GSFC C. Kesselman USC 7 10 June 2002 Hanisch/NVO
Proposal Team - NASA A. Szalay JHU R. Williams Caltech C. Alcock U. Penn. M. Livny U. Wis. K. Borne ADC/Raytheon C. Lonsdale IPAC T. Cornwell NRAO T. Mc. Glynn HEASARC/USRA D. De. Young NOAO A. Moore CMU G. Fabbiano SAO R. Moore SDSC/UCSD A. Goodman Harvard J. Pier USNO J. Gray Microsoft R. Plante NCSA/UIUC R. Hanisch STSc. I T. Prince Caltech G. Helou IPAC E. Schreier STSc. I S. Kent FNAL N. White GSFC C. Kesselman USC 8 10 June 2002 Hanisch/NVO
Project Management NSF CISE + AST External Review Committee Executive Committee Data Centers PI/Project Director: Szalay Co-PI/Chief Architect: Williams Project Scientist Project Manager es otyp Prot nce Technical Working Group Data Providers Theoretical Astrophysics es tiviti Data. Access/Resources Extra-Solar Planets e. Ac Data Models AGN Census ctur Grid Services/Testbed Rare/Exotic Objects stru Metadata Standards Digital Milky Way Infra Scie E&O Coordinator Portals/Workbenches Local/Distant Universe Science Working Group System Architect 10 10 June 2002 Hanisch/NVO
Team Organization • Executive Committee – A. Szalay, R. Williams, R. Hanisch (PM), D. De Young (PS), R. Moore (SA), G. Helou, E. Schreier • Education & Outreach – M. Voit, Coordinator • First Working Groups established – Metadata (R. Plante/NCSA) – Systems (R. Moore/UCSD) – Science (D. De Young/NOAO) • Project teams established for initial science demonstrations – GRB follow-up (T. Mc. Glynn/HEASARC) – Brown dwarf search (B. Berriman/IPAC) – Cluster galaxy morphologies (R. Plante/NCSA) 11 10 June 2002 Hanisch/NVO
Education & Outreach • Integral part of project • Emphasis is on development of partnerships • Kick-start with a workshop this summer at STSc. I (July 11 -12) – Understand requirements on NVO services from perspective of formal education, informal education, commercial/corporate, and public outreach content developers 12 10 June 2002 Hanisch/NVO
Education/Outreach Partners Association of Science. Technology Centers International Planetarium Society National Air and Space Museum Silicon Graphics (Digital Planetarium) Spitz (Electric Sky) Maryland Space Grant Consortium Gettysburg College (Project CLEA) UC Berkeley (CSE@SSL) American Museum of Natural History 13 10 June 2002 Hanisch/NVO
Management Plan • Wrote and delivered formal management plan to NSF in January • 11 major work breakdown categories, with sub-elements to three levels • All level-two technical WBS areas have designated lead who is responsible for tasks and schedule within that area Building the Framework for the National Virtual Observatory NSF Cooperative Agreement AST 0122449 Management Plan December 2001 The challenge of building a framework to enable the National Virtual Observatory will be met with a management structure that supports distributed research and development. We take optimal advantage of the domain expertise already resident in the organizations supporting the existing archival systems, sky surveys, and source catalogs of the astronomy community and meld this diversity with state-of-the-art information technology. Our structure ensures strong communication and coordination among the distributed, multi-disciplinary, heterogeneous resources, with accountability to both the community and the funding agency. It ensures that astronomy needs drive technology development. 14 10 June 2002 Hanisch/NVO
Work Breakdown Structure 15 10 June 2002 Hanisch/NVO
Milestones • • Nov 2001 – Jan 2002: Established project structure April 15, 2002: VOTable V 1. 0 May 1, 2002: 50+ Cone. Search services registered May 8, 2002: Defined initial science demos June 10, 2002: Form International VO Alliance (IVOA) Nov 15, 2002: Internal testing of science demos January 2003: Initial science demonstrations (AAS) August 2003: Intermediate NVO science demos (IAU) 16 10 June 2002 Hanisch/NVO
Milestones (cont’d) • Major technical initiatives (WBS) – Early standards: VOTable/Cone. Search (3. 2. 6, 3. 3. 2, 3. 4. 1, 3. 4. 4, 5. 3. 3) – Understanding Web Services (3. 6) – Understanding Grid Services (3. 1. 7, 4. 5, 5. 3. 4, 9. 1) – Profile Registrations (3. 4. 2, 3. 6. 3) – Data Models (2. 1, 2. 2) – Resource Management (6. 1. 2) – System Architecture (4. 1) 17 10 June 2002 Hanisch/NVO
Reporting and Communication • Formal Quarterly and Annual Reports to NSF, being cc’d to NASA • Informal monthly reports to project manager • Biweekly project status telecons with level-two WBS leaders • Weekly Executive Committee telecons • Weekly or biweekly working group telecons (Metadata, Systems, Science) • Archived e-mail exploders for all working groups and management discussions 18 10 June 2002 Hanisch/NVO
Development Approach • First year: emphasize prototyping and experimentation, leading to real demos but not necessarily production-level software or system – Many IT tools now available; extensive evaluation through prototypes necessary to refine choices – Will set up framework for more formal software management (baseline, test, revision control) for a distributed development effort so that it is ready for second-year development • NSF ITR project is not expected to define and “deliver” the entire NVO 19 10 June 2002 Hanisch/NVO
Critical Path • Science demonstrations – Identified, scoped and scheduled • Service registry issues – Needs international coordination (Garching) • User interface issues – Need to retrofit existing portals • EPO requirements – Impact on metadata standards 20 10 June 2002 Hanisch/NVO
Role of Science Prototypes • • • Keep focus on user- and science needs Identify most common services Verify standardization efforts Encourage data providers to participate Demonstrate to community that NVO tools will – arrive soon – will be useful for everybody – can evolve incrementally • First science demos planned for January 2003 21 10 June 2002 Hanisch/NVO
Initial Science Prototypes • Brown-Dwarf search – Distributed query across several archives – Correlations with non-detections – Example of typical NVO search • Gamma-Ray burst – Event follow up service – Exercise in standards compliance/interoperabilty • Galaxy evolution in clusters – On-the-fly image analysis and pattern recognition – Exercise in grid computing 22 10 June 2002 Hanisch/NVO
NVO: How Will It Work? • • Define commonly used “atomic” services Build higher level toolboxes/portals on top We do not build “everything for everybody” Use the 90 -10 rule: – – Define the standards and interfaces Build the framework Build the 10% of services that are used by 90% Let the users build the rest from the components 23 10 June 2002 Hanisch/NVO
Atomic Services • Metadata information about resources – Waveband – Sky coverage – Translation of names to universal dictionary (UCD) • Simple search patterns on the resources – Cone Search – Image mosaic – Unit conversions • Simple filtering, counting, histogramming • On-the-fly recalibrations 24 10 June 2002 Hanisch/NVO
Higher Level Services • Built on Atomic Services • Perform more complex tasks • Examples – – – Automated resource discovery Cross-identifications Photometric redshifts Outlier detections Visualization facilities • Expectation: – Build custom portals in matter of days from existing building blocks (like today in IRAF or IDL) 25 10 June 2002 Hanisch/NVO
VOTable • VOTable (R. Williams, F. Ochsenbein+ …) – – – – Proper XML document Supports streaming Handles complex, self descriptive data sets Developed jointly with European effort VOTable V 1. 0 released on April 15, 2002 Currently undergoing testing Reference parser just finished Design of API just started 26 10 June 2002 Hanisch/NVO
Sky. Query • Distributed Query tool using a set of services • Feasibility study, built in 6 weeks from scratch – Tanu Malik (JHU CS grad student) – Tamas Budavari (JHU astro postdoc) • Won 2 nd prize in Microsoft. NET Contest • Allows queries like: SELECT o. obj. Id, o. ra, o. r, o. type, t. obj. Id FROM SDSS: Photo. Primary o, TWOMASS: Photo. Primary t WHERE XMATCH(o, t)<3. 5 AND AREA(181. 3, -0. 76, 6. 5) AND o. type=3 10 June 2002 Hanisch/NVO 27
28 10 June 2002 Hanisch/NVO
Cone. Search • • Search for catalog objects around a point Returns data in VOTable format Requires a registered profile Point of the exercise – A learning experience – Existing archives test and implement VOTable – Understand service description issues • Automated test and verification • In less than two weeks 7 groups, 50 services • Cross-Identification service built on top 29 10 June 2002 Hanisch/NVO
30 10 June 2002 Hanisch/NVO
International Collaboration • European initiatives underway – Astrophysical Virtual Observatory funded by European Commission (€ 3. 3 million, three years) – Astro. Grid, funded by UK e-science program (£ 5 million, three years) • Other international efforts starting: – Canada (C$4 M recently approved), India, Japan, Chile, Germany, Russia, Australia • International VO roadmap being developed, to be announced at Garching VO conference • International VO Alliance • Regular telecons among NVO, AVO, and Astro. Grid leadership • Frequent technical contacts among partners 10 June 2002 Hanisch/NVO 31
International Standards • Active collaboration among NVO, AVO, and Astro. Grid on VOTable – V 1. 0 released on April 15 – Basis for testing metadata models, exchange protocols, encoding mechanisms • Continued development of FITS standard – World Coordinate System definitions • • • Framework definition (Paper I) Celestial coordinates (Paper II) Spectral dispersion relations (Paper III) Distortion functions (Paper IV) Time (TBD) 32 10 June 2002 Hanisch/NVO
Summary • NSF ITR NVO project is one of four major and numerous other small VO-related initiatives now underway world-wide • NVO is adopting, adapting, or developing necessary technology as derived from science requirements • Our project management approach seems to be working based on the first six months experience • NVO project is dealing with many of the management challenges that will face the ultimate VO organization http: //us-vo. org 33 10 June 2002 Hanisch/NVO
Скачать презентацию Building the Framework for the National Virtual Observatory
a464aebad11716cd77bba0eedeedddf0.ppt