Goals of a New VLBI Data System

Goals of a New VLBI Data System • • • Low cost Based primarily on unmodified COTS components Modular, easily upgradeable Robust operation, low maintenance cost Easy transportability Conformance to VLBI Standard Interface specification (VSI) Compatibility with existing VLBI systems Flexibility to support e-VLBI Minimum of 1 Gbps data rate 24 -hour unattended operation at 1 Gbps

Tape vs. Disc Price Comparison

Advantages of Magnetic Discs • • • Readily available consumer product Standard electrical interface Technology improvements independent of electrical interface Self contained; don’t have to buy expensive tape drives Rapid random access to any data Essentially instant synchronization on playback to correlator (no media -wasting early starts needed) • No headstacks to wear out or replace – ever! Virtually maintenance free.

Mark 5 VLBI Demonstration System – March 2001 3 months start to finish!

Disk-based VLBI Data Systems • Europe – PC/EVN – – • Developed primarily by Metsahovi group Supports partial implementation of VSI-H specification Uses 5 PC’s to support recording and playback at 1 Gbps Has been used for several European experiments Japan – K 5 – Developed by CRL – Uses 4 PC’s to support recording and playback at 512 Mbps – Several experimental units deployed in Japan • Australia – Swinburne group – Based on Apple Xserve RAID array interface – Designed for pulsar work; plan to use for VLBI at 128 Mbps • U. S. – Mark 5 A – Developed at Haystack Observatory with support of international consortium (BKG, EVN, KVN, MPI, NASA, NRAO & USNO) – 1 Gbps in single chassis – Chosen by IVS and EVN to replace Mark 4 and VLBA tape systems – ~30 systems now deployed; expect 60 -70 by end 2003

Mark 5 A VLBI Data System

Mark 5 A Characteristics • • 1 Gbps recording and playback Direct ‘plug-compatible’ replacement for VLBA or Mark 4 tape drive ‘ 8 -pack’ disk modules for ease of disk handling Packaged in single 5 U chassis Supports e-VLBI Supported by NASA Field System Supported by Mark 4 correlators (JIVE, MPI, USNO, Haystack) Available commercially from Conduant Corp for ~$16 K (compared to Mark 4/VLBA tape drive for ~$150 K) • Plan intercompatible support for ATA serial disks • Plan inexpensive ‘expansion chassis’ • Mark 5 B (VSI-compatible) system under development; will allow VLBA to support 1 Gbps; Mark 5 A can be upgraded with interface card replacement

Mark 5 A Experience • Daily UT 1 ‘Intensive’ observations Wettzell-Hawaii have been exclusively Mark 5 for ~10 months – almost no problems • 15 -day geodesy experiment in Oct 02 successfully recorded and processed from Westford antenna; processed one disk set with two missing disks! • Several astronomy experiments, including recent mm-VLBI experiment have successfully used Mark 5 A • Have been some teething problems, particularly learning to properly manage disks, but believed under control

Disk-Media Status • Hard disk price vs capacity/performance will continue to drop rapidly - Now ~$1. 00/GB, expected to drop to ~$0. 50/GB by ~2005 (Mark 4/VLBA tape is ~$2. 00/GB) • 200 GB disks now available – 27 hours @ 256 Mbps unattended • (comparable to ~5 VLBA tapes) • 320 GB disks expected soon – 22 hours @ 512 Mbps unattended (comparable to ~9 VLBA tapes) • 700 GB disks expected ~2005 – 24 hours @ 1 Gbps unattended (comparable to ~19 VLBA tapes)

Strawman Plan for VLBA 36 Mark 5 systems - ~$600 K Storage requirements: 24 hours @ 1 Gbps = 11 TB 300 station-days @ 1 Gbps = 3300 TB Media costs Rate (Mbps) Total (TB) Disk ($/GB) Other ($/GB) 2003 256 825 1 0. 15 1. 0 M$ 2004 512 825 0. 75 0. 15 +0. 8 M$ 2005 1024 1750 0. 5 0. 10 +1. 0 M$ Total 3300 ~Cost 2. 8 M$ Total cost over 3 years ~3. 4 M$ VLBA correlator: • Should be able to keep up with 512 Mbps for up to 20 stations • Can process 1024 Mbps for up to 10 stations

e-VLBI: The Next Step • Bandwidth growth potential for higher sensitivity – VLBI sensitivity (SNR) proportional to square root of Bandwidth resulting in a large increase in number of observable objects (only alternative is bigger antennas – hugely expensive) – e-VLBI bandwidth potential growth far exceeds recording capability (10 Gbps per station is possible today; 100 Gbps in near future!) • Rapid processing turnaround – Astronomy • Ability to study transient phenomena with feedback to steer observations – Geodesy • Higher-precision measurements for geophysical investigations • Better Earth-orientation predictions, particularly UT 1, important for military and civilian navigation

Practical Advantages of ‘e-VLBI’ • Increased Reliability – remove recording equipment out of field – remote performance monitor & control capability in near real-time • Lower Cost – Automated Operation Possible • eliminates manual handling and shipping of storage media – Real-time or near-real-time Processing • forestalls growth of storage-capacity requirements with bandwidth growth – Reduce and perhaps eliminate expensive recording-media pool (millions of $’s!) • Avoid unexpected media-shipping interruptions and losses

Japan has led the way in e-VLBI! • 1998 -2001: Keystone project – 4 antennas around Tokyo area connected in real-time at 256 Mbps • 2000: Gbps ftp e-VLBI demonstration • 2001: ATM-based 1 Gbps real-time VLBI • 2002: 2 Gbps VLBI demonstration

Connecting the Global VLBI Array in the New Era of High-Speed Networks 8 -9 April 2002 MIT Haystack Observatory, Westford, MA USA Sponsored by: International VLBI Service (IVS) Global VLBI Working Group (GVWG) MIT Haystack Observatory With the world increasingly wired for high-speed data communications, the prospects for routine global electronic transmission of VLBI data (dubbed ‘e-VLBI’) become brighter every day. Not only will e-VLBI help eliminate costly and complex recording equipment, but it should eventually lead to data rates and volumes unattainable by traditional recording equipment. This will lead to improved sensitivity, allowing new science to be explored at lower costs. This international two-day meeting at MIT Haystack Observatory is being organized to explore the current state of highspeed astronomy data transmission, concentrating on e-VLBI, but recognizing the synergy with other geodesy/astronomy applications requiring real-time or near-real-time high-speed data transmissions. Among the topics to be discussed: • • • International networking facilities – now and future User requirements for high-speed networking Reports on current and projected e-VLBI and related efforts Networking protocols for real-time data transmission Public vs. dedicated networks Establishing international standards for e-VLBI data transfer Important Dates • • • 3 December 2001 - First announcement 1 February 2002 – Second announcement 24 February 2002 – Deadline for abstract submission 1 March 2002 – Final announcement 10 March 2002 – Deadline for registration 22 March 2002 – Deadline for hotel reservations Program Committee • • Alan Whitney – Haystack (USA) Yasuhiro Koyama – CRL (Japan) Steve Parsley – JIVE (Europe) Jon Romney – NRAO (USA) Registration is required for attendance Registration closes 10 March 2002 Registration is limited to 80, so register early! For complete information see the meeting web site at: http: //web. haystack. mit. edu/e-vlbi/meeting. html

e-VLBI Development at Haystack Observatory • Phase 1: Develop e. VLBI-compatible data system – Mark 5 system • Phase 2: Demonstrate 1 Gbps e-VLBI using Bossnet (w/ DARPA and NASA support) – ~700 km link between Haystack Observatory and NASA/GSFC – First e-VLBI experiment achieved ~788 Mbps transfer rate • Phase 3: Develop adaptive network protocol (newly awarded NSF grant to Haystack Observatory; collaboration with MIT Lab for Computer Science and MIT Lincoln Laboratory); – New IP-based protocol tailored to operate in shared-network ‘background’ to efficiently using available bandwidth – Demonstrate on national and international networks • Phase 4: Extend e-VLBI to national and global VLBI community – ‘Last mile’ problem remains a serious challenge

Mark 5 VLBI Disk-Based Data System (Phase 1)

Bossnet 1 Gbps e-VLBI demonstration experiment (Phase 2) Future Initial experiment

Performance test results – Haystack/GGAO Average sustained rate >900 Mbps over 10 hours

Correlation results of e-VLBI test observation

International e-VLBI experiments • Westford, MA to Kashima, Japan - experiments in Oct 02 and Mar 03 – Files exchanged over Abilene/GEMnet networks • Nominal speed expected to be ~20 Mbps; best achieved so far ~11 Mbps – Correlation on Mark 4 correlator at Haystack and PC Software correlator at Kashima; nominal fringes obtained – Further experiments are scheduled; network tuning is in progress • Kauai, Hawaii to Wettzell, Germany (in progress) – Daily experiments of ~100 GB are ideal candidate for early e-VLBI – Data will be transferred to Haystack Observatory for processing (OC-3 speeds are possible) – Network links are now being brought up

New IP Protocols for e-VLBI (Phase 3) • Based on observed usage statistics of networks such as Abilene, it is clear there is much unused capacity • New protocols are being developed to utilize networks in ‘background’ mode for applications such as e-VLBI – – Will ‘scavenge’ and use ‘secondary’ bandwidth Will give priority to ‘normal’ users Requires a new ‘end-point adaptive strategy’ Should substantially reduce cost of e-VLBI fiber bandwidth • Work being carried out under NSF sponsorship by MIT Haystack Observatory in collaboration with MIT Laboratory for Computer Science and MIT Lincoln Laboratory – 3 -year program; will demonstrate e-VLBI connections both nationally and internationally

Typical bit-rate statistics on Abilene network 1. 0 Usage >20 Mbps less than 1% of the time 0. 1 0. 001 100 500 Mbps Conclusion: Average network usage is only a few % of capacity

Typical distribution of heavy traffic on Abilene 1. 0 0. 9 0. 8 0. 7 <10% of ‘bulk’ transfers exceed ~100 secs 200 400 1000 secs Conclusion: Heavy usage of network tends to occur in bursts of <2 minutes

Extend to national and global community (Phase 4) • Many possibilities for international connections – – – Surfnet – U. S. to Europe at 2. 5 Gbps IEEAF – U. S. to Europe at 10 Gbps Trans. PAC – U. S. to Japan at 655 Mbps (upgrade to 1. 2 Gbps planned) GEMnet – currently ~20 Mbps, planning to upgrade to 2. 2 Gbps Super-Sinet – 2. 5 Gbps Japan-to-U. S. AMPATH – possible connections to telescopes in Chile and Brazil • Not well covered – – Australia Africa China Large parts of South America

But there is a significant problem – ‘ Last-mile’ costs • Most of the world’s telescopes are not well connected • Electronic and electro-optic costs are dropping rapidly – Gig. E switch: 2001 - $15 K 2003 - $1. 2 K – Gig. E transceivers 2001 - $750 2003 - $180 – CWDM transceivers $400 -800 for 50 -100 km reach! • Direct fiber cost is relatively low– $60/fiber-km in 80 -fiber bundle • If you can buy or lease existing fiber, there is no better time! • But – fiber installation cost is still tall pole – Europe: >$20/m (or any populous wide-area) – U. S. : >$10 m (in simplest desert environment) • The upside: there is developing a lot of momentum and support from the greater networking community to get the job done! Also desperately needed: – Modern digital filter banks to replace aging and obsolete analog BBC’s!

10 Gbps

GÉANT: The connectivity at 10 Gbps 2. 5 Gbps EE LV 34 155 UK 155 LU NL 155 BE FR SE IE LT 45 SE - Po. P for Nordunet 155 DE ES PT CZ 0 SK 622 CH IT AT HU 622 34 622 PL GR HR BG 34 45 CY 155 SI IL 34 RO

622 Mbps +10 Gbps l Transoceanic donations to IEEAF (in red)

Conclusions • Modern PC and networking technology has led to a revolution in VLBI data technology • Transition to all disks will be rapid due to technical, economic and practical advantages • e-VLBI is riding an unprecedented wave of global network connectivity and networking community enthusiasm • There is no better time to lease or buy installed fiber than now! • Gradual transition from disks to disk/e-VLBI to all e-VLBI is likely • 10 -100 Gbps/antenna is technically possible with e-VLBI; can VLBI correlators keep up? But big hurdles remain…. . • Can e-VLBI survive the long-term networking costs? • There is momentum gathering in the networking community to provide national and international ultra-high-speed networking as a critical ‘enabling infrastructure’ for U. S. and international science; the astronomy community needs to makes its voice heard loud and clear!

EVN Future Vision