NRAO VLA Archive Survey National Radio Astronomy Observatory Jared Crossley Loránt Sjouwerman Edward Fomalont Nicole Radziwill 2008 -06 -25 SPIE: Observatory Operations: Strategies, Processes, and Systems
NRAO VLA Archive Survey (NVAS) • Survey of the VLA Archive generated by an automated data pipeline. • Under development since 2006. • Data products include: images, calibrated data, diagnostics, logs, … • Project ongoing, but data is already available on the Web. http: //archive. nrao. edu/
Very Large Array • Synthesis array of 27 antennas • Located in New Mexico, USA • Operational for 28+ years • • Maximum baseline: 0. 3 to 35 km Frequencies: 0. 07 to 50 GHz Produces over 150 papers per year Outputs “visibility” measurements
“Typical” VLA Data Reduction • Project scientists are responsible for all parts of the data reduction process. • Download VLA raw data from the VLA Archive. • Making images requires: – Careful editing and calibration – Fast Fourier transform processing – Deconvolution methods • To make data more easily accessible we have created the VLA Archive Survey
Present State of Survey Number of Images 72839 Unique Sky Positions 15590 Dates 1991 to 2003 Frequencies 1 to 50 GHz Observing Modes Continuum Array Configurations D, C, B, some A
NVAS Pipeline System • • Acquire Makes use of preexisting components: – – • 1. Raw data archive and interface AIPS synthesis imaging software package Perl and shell scripts connect components to form the pipeline. NVAS data is incorporated in the raw data archive. One interface provides access to raw and processed data. 2. Process 3. Archive 4. Validate
NVAS Pipeline Stages Acquire Data 1. Acquire • Perl script automates communication with Archive server by HTTP requests. • Input start and stop dates for processing 1. 2. 3. 4. Script queries the server Algorithm selects one or more data files Download to local disk Data is loaded into AIPS, split by frequency, and prepared for processing
NVAS Pipeline Stages Data Processing • AIPS “runfile” (script) performs standard synthesis imaging steps (VLARUN) • Output data products: – – – Images Calibrated data Diagnostic plots Log files Metadata for Web interface 1. Acquire 2. Process
Example image and diagnostic plots: Crab Nebula at 5 GHz
NVAS Pipeline Stages Archiving • Shell and Perl scripts • Move data products to archive server • Generate HTML for survey Web interface 1. Acquire 2. Process 3. Archive
NVAS Pipeline Stages Validation • Human-interactive process • A private Web interface is used to remove low quality data. • All NVAS data is reviewed by a human. • Less than 5% of images have been removed during data validation. 1. Acquire 2. Process 3. Archive 4. Validate
Pipeline System Performance • Only data validation requires human interaction. • All other pipeline stages are limited by computer and network data rates. – Data processing requires the most time; highresolution imaging may take many hours • Survey data is 1. 4 TB and growing
Access the survey from http: //archive. nrao. edu/ Data Vault: 7019 -47
Future Development 1. Finish imaging of VLA continuum data At the same time, continue algorithm upgrades: • Self-calibration • Better data editing • Mosaic imaging • Composite imaging of long-timescale projects 2. Apply new algorithms to all survey data 3. Process spectral line data Expect to finish these stages end of 2010.
Summary NRAO VLA Archive Survey is… • Automated survey of the full VLA archive • Makes VLA data more accessible to all astronomers • Provides reference images, calibrated data, diagnostics, logs, … • Ongoing survey, expected completion in 2010 • Currently available at: http: //archive. nrao. edu/
Скачать презентацию NRAO VLA Archive Survey National Radio Astronomy Observatory
c9e63c0bb6812c730ed5d20c1093ca38.ppt