Giga Po P Transport Options I-WIRE Positioning for

Giga. Po. P Transport Options: I-WIRE Positioning for the Bandwidth Tsunami • Virtual Internet 2 Member Meeting • Oct 4, 2001 • Linda Winkler • Argonne National Laboratory • winkler@mcs. anl. gov

I-WIRE Background • State Funded Dark Fiber Optical Infrastructure to support Networking and Applications Research UIC NU / Starlight Star Tap ANL IIT UC • $7. 5 M Total Funding • $4 M FY 00 -01 (in hand) • $2. 5 M FY 02 (approved 1 -June-01) • Additional $0. 5 M in FY 03, FY 04 • Application Driven NCSA/UIUC • Access Grid: Telepresence & Media • Computational Grids: Internet Computing • Data Grids: Information Analysis • New Technologies Proving Ground • Optical Switching • Dense Wave Division Multiplexing • Advanced middleware infrastructure For more information see www. iwire. org

Dark Fiber • Location, location • Metro • Roughly $2 per meter per strand • Lateral challenges • Regional • May require regeneration; regen, reshape, retime (3 R) is expensive! • Wide area • Expensive due to 3 R requirements • Key is location of carrier runs • $100 K-$1 M/month for 2200 mile OC-192 link

Dark Fiber (cont. ) • Key is the one time up front cost for the purchase of an IRU • Maintenance and management are the buyers problem • Obtain fiber characteristics as soon as possible (SMF vs. NZDSF, OTDR shots) • Rapid provisioning possible allowing more adaptive networks • The fiber industry is immature and underdeveloped, allowing sophisticated customers to negotiate much more attractive deals than would result from a standard RFP pricing exercise.

Indefeasible Right to Use (IRU) Services • Terms 10, 15, 20 yrs • Alternatives • Long term capital lease • Short term lease • Managed service • Considered as a physical asset which can be re-sold, traded or used a collateral. • Cost can be amortized over lifetime which results in a monthly cost substantially below traditional telecommunication services. • Be sure of contract conditions due to shaky nature of some vendors financial situation.

Dense Wavelength Division Multiplexing • To ring or not to ring (ring vs. mesh) • Redundancy (at what cost? ) • Survivability • Protection • Mesh topology benefits • • • Migration, scaling Deployment speed Capacity Utilization Network Restoration Operating Costs • Laser reach-3 R issues and OEO • Large portion of the cost • No standards • Number and spacing of lambdas are design variables • One transponder per wavelength • Beware OC-192/10 Gb. E WAN PHY/10 Gb. E LAN PHY are not all equal

Tera. Grid Backplane Star. Light International Optical Peering Point (see www. startap. net) Abilene ackpl rid B Tera. G Chicago St. Louis Indianapolis Urbana Los Angeles San Diego OC-48 (2. 5 Gb/s, Abilene) Multiple 10 Gb. E (Qwest) Multiple 10 Gb. E (I-WIRE Dark Fiber) • Solid lines in place and/or available by October 2001 • Dashed I-WIRE lines planned for Summer 2002 I-WIRE UIC Starlight / NW Univ Multiple Carrier Hubs ANL Ill Inst of Tech St Louis Giga. Po. P Univ of Chicago Indianapolis (Abilene NOC) NCSA/UIUC Charlie Catlett – Argonne National Laboratory (catlett@mcs. anl. gov)

Tera. Grid Proposed Backplane Architecture One Wilshire (Carrier Fiber Collocation Facility) 455 N. Cityfront Plaza (Qwest Fiber Collocation Facility) 4 x 10 Gb. E Los Angeles Chicago 2200 mi DTF Backbone Core Switch Vendor TBD Long-Haul DWDM (Operated by site) Vendor TBD Metro DWDM (operated by site) 15 mi 115 mi Vendor POP at JPL Qwest San Diego POP 2 mi 140 mi 25 mi Vendor TBD Metro DWDM 20 mi Caltech SDSC Ciena Core. Stream™ Long-Haul DWDM (Operated by Qwest) ANL NCSA Vendor TBD Switch/Router* (256 Gb/s crossbar) Site Border Switch Cluster Aggregation Switch Caltech Cluster (64 p) SDSC Cluster (250 p) NCSA Cluster (2000 p) Charlie Catlett – Argonne National Laboratory (catlett@mcs. anl. gov) ANL Cluster (128 p) DTF Local Site Resources and External Network Connections *Initial Phase using IP Switch/Routers. This design will be evaluated beginning October 2001. Phase 2 (optical mesh) will also be evaluated prior to full DTF cluster deployment in early 2002. .

Wavelength Services • OC 48, OC 192 vs. 10 Gb. E • Be sure of handoff specifications • Management- determine level required • Qwest/Teleglobe/(3)Link Global/Global Crossing service offerings • Benefits • • Lower cost Customer responsible for protection Share cost of electronics-save capital investment Customer empowerment Potential for higher utilization of network Transparency of signal Flexibility

IP Routers and Switches • Interoperability • Which PHY (LAN vs. WAN) interface between DWDM and CPE? • $$$ • WAN PHYs tend to be pricey • Is 10 Gb. E really 10, 000 Mb/s, or 9. 3 Gb/s, or maybe 8 Gb/s? • What is the largest individual stream you must support? • Aggregates vs. large streams • Currently no visibility into the optical layer

Next Steps: Optical Mux / Wavelength Router / Optical Wavelength Cross-connect System 2. 5 / 10 / 40 Gb/s Gig. E / 2. 5 Gb/s 10 Gb/s OTU l 1 Optical Mux/ l-Router/ Cross-Connect Customer Interface ln OTU FDP lx DWDM

Optical Switches • Current tech is O-E-O • • • Incoming signals are converted from Optical to Electrical Signal is switched electrically Outgoing signals are converted back to optical Up to 64 x 64 at 2. 5 Gb/s Smart but slow • Future tech will be O-O-O • • • Operate in the all optical domain Bit rate independent A win at 40 Gb/s Below 10 Gb/s electronic switching will be hard to beat Challenge will be in management Fast but dumb

Optical Internetworking Progress • Optical Internetworking Forum (OIF) • UNI 1. 0 specification in progress • Based on domain services model • Internet Engineering Task Force (IETF) • Generalized Multiprotocol Label Switching (GMPLS) • Incorporates Domain and Peer Models • OIF and IETF are in sync • OIF UNI 1. 0 based on GMPLS • Start with Domain Model and evolve • Stay tuned. The ending of this story has not yet been written.

Wavelengths and the Future • Wavelength services are causing a network revolution: • Core long distance SONET Rings will be replaced by meshed networks using wavelength cross-connects • Re-invention of pre-SONET network architecture • Improved transport infrastructure will exist for IP/packet services • Electrical/Optical grooming switches will emerge at edges • Automated Restoration (algorithm/GMPLS driven) becomes technically feasible. • Operational implementation will take beyond 2003

Star. Light Optical Peering Point & Co-lo Facility • 710 N. Lake Shore Dr. Chicago, IL • Northwestern Campus • Central downtown location • Near carrier services serving Chicago loop • Telephone switch room • Colo space available • Multiple carriers access • Ameritech, AT&T, Qwest, Global Crossing, MCI Worldcom • Other builds possible