Network Update Steve Cotter Director Network Services Joint

Network Update Steve Cotter Director, Network Services Joint Techs, Fermilab July 16, 2007

Internet 2 Network Infrastructure Overview • Layer 1: Managed wavelengths from Level(3) Communications • Level(3) owns and manages Infinera optical gear: responsible for software upgrades, equipment maintenance, remote hands, sparing, NOC services • Internet 2 NOC has total provisioning control • Layer 2: Internet 2 owned and managed Ciena Core. Directors • Using DRAGON GMPLS control plane • Layer 3: Internet 2 owned and managed Juniper T 640 s • Expanded Observatory • Platform for layer 1/3 network performance data collection, collocation, experimentation • perf. SONAR integration for intra- & inter-network performance analysis • International connectivity • Layer 1 network extended to international exchange points in Seattle, Chicago and New York City • Peering points in Seattle, PAIX, Equinix Chicago, others Slide 2

Pacific Northwest GP 2001 6 th Ave Westin Bldg Portland Oregon GP 707 SW Washington Qwest Seattle 1000 Denny Way Level 3 Rieth Cleveland TFN/MERIT 4000 Chester Chicago CIC/MREN BOREAS Internet 2 710 N Lakeshore Starlight Portland 1335 NW Northrop Level 3 Boise New York 111 8 th Level 3 Detroit Reno Eureka Ogden Omaha Rawlins via 1005 N B St San Francisco Sunnyvale 1380 Kifer Salt Lake Inter-Mountain GP Front Range GP 572 S De. Long Indianapolis IU 1902 S East St Pittsburgh GP 143 S 25 th Los Angeles 818 W 7 th Level 3 Albuquerque New Mexico GP 104 Gold Ave SE Phoenix Tucson Tulsa 18 W Archer Raton New York NYSERNET 32 A of the A Philadelphia MAGPI 401 N Broad Washington MAX 1755 Old Meadow Denver 1850 Pearl San Luis Obispo San Diego Chicago 600 W Chicago Edison Sacramento Cambridge NOX 300 Bent St Syracuse Buffalo Tionesta Los Angeles CENIC 600 W 7 th Equinix Albany 316 N Pearl Level 3 Kansas City St. Louis GPN 1100 Walnut Nashville Univ Memphis 2990 Sidco Dr Louisville Univ Louisville 848 S 8 th St Charlotte Atlanta 180 Peachtree St NW Rancho De La Fe (tentative) Raleigh 5301 Departure Dr Atlanta SLR 345 Courtland Birmingham Dallas El Paso 501 W Overland Regen DTN Core Node Drop DTN Other Level 3 Mobile Valentine Tallahassee Sanderson San Antonio Houston LEARN 1201 N I-45 Baton Rouge LONI 9987 Burbank Jacksonville FAMU&USF 4814 Phillips Hwy

Circuit Services Nodes Availability Status July 15, 2007 Pacific Northwest GP 2001 6 th Ave Westin Bldg Portland Oregon GP 707 SW Washington Qwest Seattle 1000 Denny Way Level 3 Portland 1335 NW Northrop Level 3 Chicago CIC/MREN MERIT BOREAS Internet 2 710 N Lakeshore Starlight Boise Tionesta via 1075 Triangle Ct Reno Ogden Eureka Omaha Rawlins Edison Sacramento via 1005 N B St San Francisco Sunnyvale CENIC 1380 Kifer Level 3 Salt Lake Inter-Mountain GP 572 S De. Long Level 3 Denver Front Range GP 1850 Pearl Level 3 Tulsa One. Net 18 W Archer Level 3 Los Angeles 818 W 7 th Level 3 San Diego Albuquerque New Mexico GP 104 Gold Ave SE Level 3 Phoenix Tucson Raton El Paso 501 W Overland Level 3 New York 111 8 th Level 3 Detroit Kansas City St. Louis GPN 1100 Walnut Level 3 Nashville Tennessee GP 2990 Sidco Dr Level 3 Pittsburgh GP 143 S 25 th Level 3 Louisville 848 S 8 th St Level 3 Charlotte Atlanta SLR 345 Courtland Tallahassee Sanderson San Antonio Raleigh NCREN 5301 Departure Dr Level 3 Jacksonville FLR 4814 Phillips Hwy Level 3 Mobile Valentine New York NYSERNET 32 Ave of the Americas Philadelphia MAGPI 401 N Broad Level 3 Washington MAX 1755 Old Meadow Level 3 Atlanta 180 Peachtree St NW Level 3 MC Birmingham Dallas Cambridge NOX 300 Bent St Level 3 Buffalo Chicago 600 W Chicago Level 3 MC Rancho De La Fe (tentative) Albany 316 N Pearl Level 3 Syracuse Indianapolis 1902 S East St Level 3 San Luis Obispo Los Angeles CENIC 600 W 7 th Equinix Cleveland TFN 4000 Chester Level 3 Rieth Houston LEARN 1201 N I-45 Level 3 Baton Rouge LONI 9987 Burbank Level 3 Fully Operational Installed – not fully turned up Not Installed yet Install Cancelled

Transition Progress ¢ ¢ PHASE 1: New York, Philadelphia, Washington, Pittsburgh, Cleveland, Chicago, Boston 0 0 0 Internet 2 and Connector Site Design Discussions Colocation Planning, Connector Coordination Complete Colocation Availability and Equipment Installation Wave Availability and Backbone Turn Up Connector Cutover and Abilene Node Turn Down Complete PHASE 2: Raleigh, Atlanta, Nashville, Louisville, Indianapolis 0 0 0 Internet 2 and Connector Site Design Discussions Colocation Planning, Connector Coordination Complete Colocation Availability and Equipment Installation Wave Availability and Backbone Turn Up Connector Cutover and Abilene Node Turn Down Complete except USF PHASE 3: Kansas City, Tulsa, Houston, Baton Rouge, Jacksonville, Albuquerque, Denver 0 0 0 Internet 2 and Connector Site Design Discussions Colocation Planning, Connector Coordination Complete Colocation Availability and Equipment Installation Wave Availability and Backbone Turn Up Connector Cutover and Abilene Node Turn Down Complete except HOU, FAMU, BTR PHASE 4: Salt Lake City, Seattle, Portland, Los Angeles 0 0 0 Internet 2 and Connector Site Design Discussions Colocation Planning, Connector Coordination Complete Colocation Availability and Equipment Installation Wave Availability and Backbone Turn Up Connector Cutover and Abilene Node Turn Down Complete In progress; CENIC, SLC, Portland outstanding

Internet 2 Network by the Numbers: • • 13, 500 long haul route miles Deployed and configured over 300 Infinera Network Elements Day 1 capacity of 100 Gbps Built 27 custom collocation suites representing 3, 365 sqft of space including: • 91 Racks - Internet 2, ESnet, third-parties • 60 Individual bulk cables with 48 & 96 fiber count • Deployed 64 metro fiber route miles • Enabled connectivity from Houston-to-El Paso & El Paso-to-Denver for L(3) to Internet 2/GRNOC and ESnet NOC access • Product Summary: Wavelengths, HSIP, Private Lines, Metro Fiber, Collocation, Cross-Connects, Power Controllers, and GPS timing • Backbone alone was 569 individual tasks completed over 261 days! • Developed the Virtual Network Operations Center – Provisioning and Troubleshooting Dashboard

Slide 8

Internet 2 Network Flexible Infrastructure Supporting e-Science, Network Research & Education • Best-Effort High-Speed IP Service • Enables delivery of advanced content, commodity services, etc. • Point-to-Point Wavelength Services • Circuit Service for static or on-demand bandwidth • Point-to-point Ethernet (VLAN) Framed SONET Circuit • Point-to-point SONET Circuit • Bandwidth provisioning available in 50 Mbps increments • Physical Connection • 1 or 10 Gigabit Ethernet • OC-192 SONET Slide 9

Internet 2 Network Commodity Peering Service • The growing list of members taking advantage of this service includes: • • • Great Plains Network (GPN) MERIT through CIC OSCnet University of Iowa through CIC University of New Mexico ● ● ● Indiana Giga. Po. P Northern Crossroads (NOX) Oregon Giga. Po. P University of Louisville University of Texas – Austin • Currently, the CPS offers over 60, 000 routes through peering partnerships with commercial networks in Chicago, Seattle, and Palo Alto. • A connection to NYC is underway, which will further improve and diversify our commercial network peering structure. • Additional peering connections are being investigated. Slide 10

Internet 2 Network Wavelength & Circuit Services • Connection oriented services provide for: • Guaranteed bandwidth and predictable latency (repeatable, dependable performance between collaborating sites) • Traffic segregation (support specific policy or traffic engineering requirements) • Router bypass: Express links created for highbandwidth, limited duration long-haul traffic reducing the need for mid-path L 3 interfaces • Cost efficiency: L 3 router blades cost > L 2 ports > L 1 or L 0 interfaces • Capability tradeoff but could possibly improve performance Slide 11

Internet 2 Network Wavelength & Circuit Services • Automated circuit provisioning enable rapid deployment and efficient utilization of capital investment • Establishing end-to-end lightpaths is a non-trivial task: it is resource intensive and error prone • Automated reservation, allocation, and provisioning enables co-scheduling of network and non-network resources Slide 12

Multi-Service/Domain/Layer/Vendor Provisioning • • • Multi-Domain Provisioning Interdomain ENNI (Web Service and OIF/GMPLS) Multi-domain, multi-stage path computation process AAA Scheduling GEANT TDM GUI XML Internet 2 Network Regional Network AST Dynamic Ethernet TDM ESNet Domain Controller Ctrl Element Ethernet SONET Switch Router Slide 13 Dynamic Ethernet Data Plane Control Plane Adjacency LSP Slide from Tom Lehman, ISI-East IP Network (MPLS, L 2 VPN)

Internet 2 Network Circuit Services Applications • Create private networks between key collaborating sites within a science community or on a project-byproject basis. Examples: • Radio Astronomy community sharing telescopes and correlation facilities • High Energy Physics community distributing large data sets • Smaller projects may link data repositories and computational resources among key participants – e. g. grid portals • Network engineering and operations community can use these capabilities to support traffic engineering objectives • We expect applications will use these circuit services to carry high-bandwidth, limited duration flows and will retain use of IP links for generic Internet access. Slide 14

Collaborations • The DICE group - Dante (GEANT 2), Internet 2, CANARIE, and ESnet • Working closely with ESnet on interfacing OSCARS and HOPI - involves AAA work, using OSCARS interface • “Stitching” project to describe data layer interconnections between segments of a P 2 P path • Reporting back progress to the GLIF and other organizations • For example, Phosphorus, in coordination with the SURFnet and University of Amsterdam participants • Also having discussions with JGN 2 • Coordinating with OGF on various schema - topology, path computation, signaling • Working with the appropriate standards bodies - ITU, IETF, and OIF Slide 15

For more information: http: //www. internet 2. edu/network/ http: //i 2 net. blogspot. com scotter@internet 2. edu Thank you! Slide 16

Slide 17

Slide 18