Introduction to Observational Catalogues Databases Duncan Law-Green

Introduction to Observational Catalogues & Databases Duncan Law-Green (LEDAS Archive Scientist) 21 st February 2007

Introduction What this seminar is about Introduction to selection of online astronomical data resources to help with your research. Outline of data formats and catalogue tools. Practical demos. What this seminar is not about Data reduction and analysis techniques. Computer science of databases, SQL programming, how to build your own databases. Introduction to Observational Catalogues & Databases: 21/02/2007

Use of Catalogues/Archives Literature searches Build science case for your proposal Archive retrieval Previous observations of your objects Feasibility studies Is the observation possible with your instrument? Observation planning Source positions, instrument FOV, guide stars etc. Class studies Filter catalogues for interesting sources/outliers

Data & Metadata Data A set of measured parameters for a source. Metadata Data about the dataset, provides vital context info. Examples include: coordinate epoch, date of observations, observing mode, filter bandpass, pipeline processing performed etc. Data without metadata is useless. Some data formats preserve metadata, some don't. Introduction to Observational Catalogues & Databases: 21/02/2007

Data Formats I ASCII (CSV/TSV: Comma/Tab-Separated Variables) Advantages: Easy to generate, easy to read. Will ingest directly into, e. g. Excel Disadvantages: No direct metadata support. No integrity support. No documentation. Bulky.

Data Formats II FITS (NOAO), HDS, NDF (Starlink) Advantages: Structured formats, include metadata. Multi-dimensional (tables, images, datacubes). Well-defined formats, good software support. Binary format, compact. Disadvantages: Varying compatibility between FITS, HDS, NDF. Conversion may affect metadata. No semantics.

Data Formats III VOTable (IVOA, Astro. Grid) Advantages: Structured format, metadata+semantics. Human-readable, supported by modern software. Uses XML, existing tools to generate and check integrity. Disadvantages: Uses XML – very bulky. Multidimensional support awkward. Standards evolving.

Semantics What a data column means in physical terms Unified Content Descriptors (UCDs) Created by IVOA (International Virtual Observatory Alliance) as standard controlled vocabulary of keywords to describe physical nature of table columns. Current system “UCD 1+” R. A. (Main): Source ID: Radio flux ratio: pos. eq. ra; meta. main meta. id; src phot. flux; em. radio; arith. ratio UCDs are feature of well-constructed VOTables. Intended to ease automated data handling, “workflows”

VOTable <? xml version="1. 0"? > <VOTABLE version="1. 1" xmlns: xsi="http: //www. w 3. org/2001/XMLSchema-instance" xsi: no. Namespace. Schema. Location="http: //www. ivoa. net/xml/VOTable/v 1. 1"> <COOSYS ID="J 2000" equinox="J 2000. " epoch="J 2000. " system="eq_FK 5"/> <RESOURCE name="my. Favourite. Galaxies"> <TABLE name="results"> <DESCRIPTION>Velocities and Distance estimations</DESCRIPTION> <PARAM name="Telescope" datatype="float" ucd="phys. size; instr. tel" unit="m" value="3. 6"/> <FIELD name="RA" ID="col 1" ucd="pos. eq. ra; meta. main" ref="J 2000" datatype="float" width="6" precision="2" unit="deg"/> <FIELD name="Dec" ID="col 2" "pos. eq. dec; meta. main" ref="J 2000" datatype="float" width="6" precision="2" unit="deg"/> <FIELD name="Name" ID="col 3" ucd="meta. id; meta. main" datatype="char" arraysize="8*"/> <FIELD name="RVel" ID="col 4" ucd="src. veloc. hc" datatype="int" width="5" unit="km/s"/> <FIELD name="e_RVel" ID="col 5" ucd="stat. error; src. veloc. hc" datatype="int" width="3" unit="km/s"/> <FIELD name="R" ID="col 6" ucd="phys. distance" datatype="float" width="4" precision="1" unit="Mpc"> <DESCRIPTION>Distance of Galaxy, assuming H=75 km/s/Mpc</DESCRIPTION> </FIELD> continued on next slide. . .

VOTable (cont. ) continued from previous slide. . . <DATA> <TABLEDATA> <TR> <TD>010. 68</TD><TD>+41. 27</TD><TD>N 224</TD><TD>297</TD><TD>5</TD><TD>0. 7</TD> </TR> <TD>287. 43</TD><TD>-63. 85</TD><TD>N 6744</TD><TD>839</TD><TD>6</TD><TD>10. 4</TD> </TR> <TD>023. 48</TD><TD>+30. 66</TD><TD>N 598</TD><TD>182</TD><TD>3</TD><TD>0. 7</TD> </TR> </TABLEDATA> </TABLE> </RESOURCE> </VOTABLE>

Treeview File format viewer Can read multiple file formats, display hierarchical structures, expand collapse nodes with click of mouse. Some basic plotting, image, stats routines.

Literature Search ADS: NASA Astrophysics Data Service adsabs. harvard. edu 3 bibliographic databases Astronomy & Astrophysics (1. 2 million) Physics (3. 6 million) Not peer-reviewed! Ar. Xiv e-prints (400, 000) Searchable by author, subject, title, object, abstract text, full-text. . . Links to full PDFs of articles, object catalogues, data tables. My. ADS Update Service: subscribe to updates

“Telegrams” IAU Circulars: Central Bureau for Astronomical Telegrams Central clearinghouse for info on transient events (comets, solar system bodies, novae, supernovae etc. ). Subscribe by email. Search via ADS Astronomers' Telegram Primarily high-energy transient events (GRBs etc. ). Subscribe via email or RSS. Searchable web interface, mirror @ LEDAS

Data Servers Catalogue Servers Surveys and article data. Searchable by position, filter by various parameters, output data in ASCII, FITS, VOTable. (examples: NED, HEASARC, LEDAS, Vi. Zie. R) Image Servers Images of the sky at various wavelengths. Output data in bitmap (GIF, JPG, PNG) or FITS image. May or may not be “science grade”. (examples: DSS-I/II, SDSS, Sky. View, Aladin) Archive Servers Repository of public data from particular observatory or mission. Various formats, may need specialist software or training to interpret. (examples: Hubble, MERLIN, Chandra, LEDAS).

Aladin Advanced image and catalogue search system Plot catalogue search results directly on image Highly versatile, write scripts for repetitive operations. Launch via CDS website, LEDAS site or directly on desktop. RTFM! Introduction to Observational Catalogues & Databases: 21/02/2007

Aladin Button bar Plane stack Main window Preview Object data Introduction to Observational Catalogues & Databases: 21/02/2007

Aladin Multiview option Splitscreen button Introduction to Observational Catalogues & Databases: 21/02/2007

TOPCAT Catalogue plotting, editing and filtering tool Cross-correlations between catalogues Introduction to Observational Catalogues & Databases: 21/02/2007

TOPCAT

Cross-correlation Search for matching positions in 2 or more catalogues (e. g. “does this X-ray source have a radio counterpart? ”) Consider positional uncertainties, statistical probability of chance coincidence Convenient tool for 2, 3, 4 -way catalogue matches in TOPCAT (Joins -> Pair Match etc. ) Introduction to Observational Catalogues & Databases: 21/02/2007

Virtual Observatory (VO) International project to simplify access to astronomical catalogues and archives. Coordinated by IVOA. Standard set of access commands (“protocols”), all databases “appear” the same on the network. UK VO project Astro. Grid, developed additional software for distributed search, distributed storage, workflows etc. Introduction to Observational Catalogues & Databases: 21/02/2007

Astro. Grid Workbench Workbench: Desktop Java application for VO searches and data processing. Check availability of data Execute simultaneous searches across multiple catalogues/servers Construct “workflows”: drag+drop editing of data gathering/reduction/analysis pipelines. Save results to “My. Space”, temp scratch space Introduction to Observational Catalogues & Databases: 21/02/2007

VOPlot Product of VO India project Reading, interactive plotting and filtering tool for catalogue data (primarily VOTables) Introduction to Observational Catalogues & Databases: 21/02/2007

And finally. . . Stellarium (http: //www. stellarium. org/) Free planetarium software, impress your friends!. . . Questions to ledas-help@star. le. ac. uk Seminar slides, URLs to appear on my webspace http: //www. star. le. ac. uk/~dlg/ Introduction to Observational Catalogues & Databases: 21/02/2007