National Network Services LHC Tier 3 Meeting March

National Network Services LHC Tier 3 Meeting March 15 2007 Joe Metzger ESnet Senior Network Engineer Lawrence Berkeley National Laboratory metzger@es. net, www. es. net 1

Why should you care about networks? • The process of science is changing – The Network is now a critical part of many Science Experiments including the LHC • The Network Architectures and service delivery models are changing. – Moving from simple rings to dense meshes – Circuit Based Services • The underlying network technology is changing. – Sonet Circuits purchased from a carrier to mixes of Sonet and Ethernet services – Multi-domain stitched circuits • Wizards Gap: – People who understand the network get 100 times better performance than those who don’t. – http: //e 2 epi. internet 2. edu/btg/ 2

Who are National Networks? • General Educational – Internet 2 – NLR • Mission Oriented – DOE • ESnet – NASA • NREN: NASA Research and Engineering Network • NISN: NASA Integrated Services Network – DOD • DREN 3

Services we provide • • • Traditional IP Services Circuit Services Measurement & Monitoring Colocation NOC Security Federated Trust Conferencing Middleware 4

Traditional IP Services – Basic Network Services • IPv 4 – Advanced Network Services • Multicast • IPv 6 • QOS – Best Effort – Scavenger – Transit • Between Customers • To International Peers • To other Research Networks (Mostly) • To the Internet (Some more than others. . . ) 5

Circuit Services • All of the networks are building or offering some types of Circuit Based services – Physical circuits between customers and other points of interest (partners, exchange points, etc) – Virtual Circuits • Within 1 domain • Across multiple domains • There a lot of different projects working on developing circuit based services. Most of them are coordinating with each other towards to goal of providing on-demand multidomain end-to-end dynamic virtual circuits. – – – OSCARS BRUW HOPI Dragon USN etc 6

Measurement & Monitoring • • • Internet 2 Observatory NLRView Perf. SONAR – A global collaboration involving 30+ networks around the world to develop a web services based multi-domain network measurement infrastructure. • www. perfsonar. net 7

NOC • All of the National Networks have a Network Operations Center that provides – Network Monitoring – Troubleshooting – etc 8

Misc • The National Networks support many different Middleware, Security, AAA, and collaboration activities including: – Federated Trust • ESnet CA • USHER • DOEGrids • I 2 In. Common – Collaboration • ECS • The Commons – Security & AAA • SALSA • Shibboleth 9

Common Backbone Architecture – ESnet, I 2 & NLR have a nation-wide footprint on Level 3 Fiber – Optical layer that provides 32 to 40 10 G lambdas – Multiple different networks built on top of the Optical layer • NLR – Frame. Net – Packet. Net – Transit. Rail • Internet 2 – HOPI – Circuit Network (No official name yet. . ) • ESnet – ESnet IP is on the Internet 2 Optical layer – ESnet SDN is using both the Internet 2 and NLR Optical layers 10

The Backbones Internet 2 11

Expectations for 2008 • The national networks will add capacity so they will still have 5+ Gbps of idle capacity almost all the time. • People using properly designed and tuned storage systems will be able to easily move large data sets between major universities and government labs in the US and Europe over traditional 10 Gbps routed R&E paths: – 400 GB in an hour – 10 TB in a day • You will need to use Circuit Services if you want: – Guaranteed bandwidth – Traffic isolation – Security assertions • Multi-domain Circuit services will be available – Short lived temporary circuits will probably be included in base services, or cheap. – Long term dedicated circuits will cost money. • The Wizards Gap will continue to grow. – Many people will still be having trouble with performance problems in the end hosts, TCP stacks, disks & file systems. 12

Data Transfer Times 10 PB 300, 240. 0 Gbps 25, 020. 0 Gbps 3, 127. 5 Gbps 1, 042. 5 Gbps 148. 9 Gbps 34. 7 Gbps 1 PB 30, 024. 0 Gbps 2, 502. 0 Gbps 312. 7 Gbps 104. 2 Gbps 14. 9 Gbps 3. 5 Gbps 100 TB 2, 932. 0 Gbps 244. 3 Gbps 30. 5 Gbps 10. 2 Gbps 1. 5 Gbps 339. 4 Mbps 10 TB 293. 2 Gbps 24. 4 Gbps 3. 1 Gbps 1. 0 Gbps 145. 4 Mbps 33. 9 Mbps 1 TB 29. 3 Gbps 2. 4 Gbps 305. 4 Mbps 101. 8 Mbps 14. 5 Mbps 3. 4 Mbps 100 GB 2. 9 Gbps 238. 6 Mbps 29. 8 Mbps 9. 9 Mbps 1. 4 Mbps 331. 4 Kbps 10 GB 286. 3 Mbps 23. 9 Mbps 3. 0 Mbps 994. 2 Kbps 142. 0 Kbps 33. 1 Kbps 1 GB 28. 6 Mbps 2. 4 Mbps 298. 3 Kbps 99. 4 Kbps 14. 2 Kbps 3. 3 Kbps 100 MB 2. 8 Mbps 233. 0 Kbps 29. 1 Kbps 9. 7 Kbps 1. 4 Kbps 0. 3 Kbps 5 Minutes 1 Hour 8 Hours 24 Hours 7 Days 30 Days Table 1. Data rates required for data set movement in a given time interval From: http: //www. es. net/pub/esnet-doc/TCP-Bulk-Data-Transfer. pdf 13

Internet 2 and ESnet Optical Node ESnet Internet 2 IP core M 320 ESnet metro-area networks SDN core switch support devices: • measurement • out-of-band access • monitoring • security T 640 dynamically allocated and routed waves (future) grooming Ciena device Core. Director optical interface to R&E Regional Nets Network Testbeds various equipment and experimental control plane management systems support devices: • measurement • out-of-band access • monitoring • ……. Future access to control plane fiber east fiber west Infinera DTN fiber north/south Internet 2/Level 3 National Optical Infrastructure 14