Introduction to EXPRe S — Beyond production e-VLBI

Introduction to EXPRe. S - Beyond production e-VLBI services T. Charles Yun Program Manager EXPRe. S Project, JIVE

Presentation Overview • Introductions: • EXPRe. S • VLBI • Correlation (analysis) • Some lessons and thoughts 2006 November 21 IST 2006 - Helsinki, Finland 2

Introduction to EXPRe. S What is EXPRe. S? • EXPRe. S = Express Production Real-time e-VLBI Service The overall objective of EXPRe. S is to create a productionlevel, real-time, “electronic” VLBI (e-VLBI) service, in which the radio telescopes are reliably connected to the central supercomputer at JIVE in the Netherlands, via a high -speed optical-fibre communication network. . . - or Make e-VLBI routine, reliable and realistic for astronomers 2006 November 21 IST 2006 - Helsinki, Finland 3

Introduction to EXPRe. S Details • EXPRe. S is made possible by the European Commission (DGINFSO), Sixth Framework Programme, Contract #026642 • Project Details • Three year, started March 2006 • International collaboration • Funded at 3. 9 million EUR • Means: high-speed communication networks operating in real-time and connecting some of the largest and most sensitive radio telescopes on the planet 2006 November 21 IST 2006 - Helsinki, Finland 4

Introduction to EXPRe. S Activities in EXPRe. S • Networking Activities • NA 1: Management of I 3 • NA 2: EVN-NREN Forum • NA 3: e-VLBI Science Forum • NA 4: e-VLBI Outreach, Dissemination & Communications • Specific Service Activities • SA 1: Production e-VLBI Service • SA 2: Network Provision for a Global e-VLBI Array • Joint Research Activities • JRA 1: Future Arrays of Broadband Radio Telescopes on Internet Computing 2006 November 21 IST 2006 - Helsinki, Finland 5

Introduction to EXPRe. S Partners • Joint Institute for VLBI in Europe (coordinator), the Netherlands • AARNET Pty Ltd. , Australia • ASTRON, the Netherlands • Centro Nacional de Informacion Geografica, Spain • Chalmers Tekniska Hoegskola Aktiebolag, Sweden • Commonwealth Scientific and Industrial Research Organization (CSIRO), Australia • Cornell University, USA • Delivery of Advanced Network Technology to Europe Ltd. (DANTE), UK • Instituto Nazionale di Astrofisica, Italy • Instytut Chemii Bioorganicznej PAN, Poland • Max Planck Gesellschaft zur Foerderung der Wissenschaften e. V. , Germany • National Research Foundation, South Africa • Shanghai Astronomical Observatory, Chinese Academy of Sciences, China • SURFNet b. v. , The Netherlands • Teknillinen Korkeakoulu, Finland • The University of Manchester, UK • Universidad de Concepcion, Chile • Uniwersytet Mikolaja Kopernika, Poland • Ventspils Augstskola, Latvia 2006 November 21 IST 2006 - Helsinki, Finland 6

Introduction to VLBI Primer- VLBI • A radio telescope looks at an object in the sky and collects data to create an “image” of the source • Multiple telescopes can view the same object. The distance between the telescopes is the baseline. The baseline can be compared to building a single telescope with the diameter of this distance (sort of). • The resolution increases with additional telescopes and longer baselines • Correlation is the process by which data from multiple telescopes is collected and processed to create a more accurate image. The correlator a super computer (interferometry) • The sensitivity of the image increases with the data collection rate at the telescope 2006 November 21 IST 2006 - Helsinki, Finland 7

Introduction to VLBI Once upon a time. . . • Telescopes collected data on tapes… heavy and bulky… postal mail… once all the tapes arrived… tapes were lost/damaged… hard drive arrays slightly improved the situation. . . • It was not unusual for the time between experiment to the beginning of correlation to be multiple weeks. • Today, you can transport the data over the network: e-VLBI - electronic VLBI 2006 November 21 IST 2006 - Helsinki, Finland 8

Introduction to VLBI Why transport data over the network? • Using the network to transport data improves science • Eliminating the need to move physical objects enables: • Real time analysis • Ability to identify minor problems in data collection • Hybrid observations • Responsiveness to transient events • Automated observation (hands-off observing) • Networked data supports flexible analysis 2006 November 21 IST 2006 - Helsinki, Finland 9

Introduction to Correlation Primer- Correlation (Analysis) • Synthesis imaging simulates a very large telescope by measuring Fourier components of sky brightness on each baseline pair • EVN Mk. IV data processor at JIVE • custom silicon, 1024 chips • Input data is 1 Gb/s max • Around 100 T-operations/sec • Dedicated, purpose designed/built hardware 2006 November 21 IST 2006 - Helsinki, Finland 10

Introduction to Correlation Once upon a time… • Cost to build correlator… limited flexibility (majority of computation in custom hardware)… preset data input rates… scheduling of scarce resource (correlator)… upgrade cost forces longer life-cycle than desired 2006 November 21 IST 2006 - Helsinki, Finland 11

Introduction to Correlation Why “Grid-ify” correlation? • Grid computing offers promising possibilities: • keep up with input (e. g. , LOFAR on Blue. Gene) • Higher precision and new applications • Better sensitivity, interference mitigation, spacecraft navigation • Can CPU cycles be found on the Grid? • From 16 antenna @ 1 Gb/s (e. VLBI) To 1000 s at 100 Gb/s (SKA) 2006 November 21 IST 2006 - Helsinki, Finland 12

Reflection. Lessons Learned Each of these bullets is a set of papers, posters and presentations in and of itself… • Networking is coordination • EXPRe. S participants on 6 continents • Connectivity • Networking assumes connectivity, Last mile issues • Saturating the network is hard • End host hardware • End-to-end Network optimization • Designing new applications • Custom software- operational vs. proof of concept • Flexible solutions- address current problems, future needs 2006 November 21 IST 2006 - Helsinki, Finland 13

Reflection. Looking Forward • Much has been done before • Importance of standards, open source • Look at other leaders in the field • Collaboration • Working across disciplines, continents • Partnering to fill gaps (e. g. , cpu hardware, analysis algorithms, visualization, network, storage) • Shared investments 2006 November 21 IST 2006 - Helsinki, Finland 14

Conclusion Questions/Answers • Contact information T. Charles Yun Project Manager EXPRe. S (JIVE) tcyun at jive dot nl • Additional Information http: //expres-eu. org/ http: //www. jive. nl/ • [note: only one “s”] EXPRe. S is made possible through the support of the European Commission (DG-INFSO), Sixth Framework Programme, Contract #026642 2006 November 21 IST 2006 - Helsinki, Finland 15

2006 November 21 IST 2006 - Helsinki, Finland 16