Pacific Wave Pacific Light Rail And National Light

Pacific Wave, Pacific Light Rail, And National Light Rail Jacqueline Brown University of Washington, Seattle CANS 2002, Shanghai, 22 August 2002

Pacific Wave is a service of the Pacific Northwest Gigapop. – High speed peering point for regional and international networks – Complement of Abilene and CA*net 3 International Transit Network services – Open peering point for participant-managed bilateral peerings

Pacific Wave Emphasis

Features of Pacific Wave • Close to major Pacific cable landing sites • Located in incumbent telecommunications carrier grade hotel – easy access to collocation services – easy access to fiber-meet-me-room – easy access to full U. S West Coast fiber grid • Gigabit Ethernet is high speed, low cost, low maintenance – No PNWGP staff involvement required to setup your peerings; no ATM PVC’s or mandated routing policy • Switches implement PIM-SM snooping for increased multicast efficiency

Features of Pacific Wave (cont. ) • 24 x 7 Network Operations Center • Redundant environmentals (DC, HVAC) • PNWGP Commodity Internet Services – Engineered and delivered via four diverse tierone vendors – Vendors chosen after exhaustive review of capacity, support, peering, etc. – Diverse geographic connectivity for commodity services – 1. 45 Gbps throughput (under expansion)

N AT I O N A L L I G H T R A I L Networks at Pacific Wave: Spring 2002 • • • AARNet – Australian Academic & Research Network Abilene Network/Internet 2 ATT Broadband Internet CA*net 3/CANARIE DREN – Defense Research and Engineering Network ESNet – Energy Sciences Network Microsoft Corporation Pacific Northwest Gigapop Siemens Medical – Pointshare TANET 2 – Taiwan Research Network Trans. PAC – Asia Pacific Academic Networks (APAN)

Note: Aubrey Bush’s slide RESEARCH = Network research testbeds EXPERIMENTAL = Experimental network infrastructure OPERATIONAL = 24/7 operational high performance networks Chicago 5/DEC/01 Grand Challenges in e-Science

R&E Network Tiers LEADERS NETWORK TYPE NETWORK DISCONTINUITIES CAPABILITIES/USERS Academic Research, papers, lab. Expe Computer Science & Net. spools of fiber In labs & sparse testbe Researchers CS, Comp. Science & ‘Grid’ Bleeding-edge e 2 e services & ’s, facilities researchers, P 2 p fiber, + major experiments (eg DTF) & >=10 g ip … for key experiments, programs & net. experimenters + leading edge projects, researchers & for labs next Experimental & gen. and centers, optics sys. & net. Developmental tech. , architectures, Grids, Industry R&D groups + I-WIRE Networks content, apparatus, etc. ‘Things PACIFIC LIGHTRAIL + I 2 -Abilene, wave projects; Leading-Edge ip Servi that Internet-2 Operational High Performance Giga. Po. Ps; for been done haven’t demanding ap Research Support Networks Fed. ‘NGI’ Nets. & middleware before; with, Internet-2, Advanced Common Denominator, Advanced to, on or over Giga. Po. Ps Services for networks’. Services Research & Education Network R&E users I 2 -K 20 ISP’s, hosting. com’s etc. Commodity Internet Business, . Gov. & General 12/05/01 Public

NATIONAL LIGHT RAIL An Extraordinary Opportunity for a Research & Education Community Owned & Lit, Fiber-Based, National-Scale Experimental & Research, & Production ‘. edu’ Network Infrastructure Revised 8 -15 -02 rj

Optical Fiber Options: Creative Approaches to Extraordinary (but perishable) Opportunities (to make Bandwidth a Resource instead of a Constraint!)

And, Long Term Keep the Experimental Networks tier from sliding down the network development ‘Spiral’ Commercialization Privatization Today’s Internet Research Nets Research and Development Source: Ivan Moura Campos ‘NGI’ efforts Partnerships

N A Real Partnership & Project: ATIONAL L IGHT R AIL • Partners = CENIC & Pacific Wave (aka the ‘Pacific Light. Rail partners) + Distributed Terascale Facility (DTF-ETF) & I-Wire folks + UCAID + NCAR + a few others • Rooted in a lightweight, but coordinated, and opportunistic collaboration of the NLR partners to acquire, provision and ‘operate’ optical networking assets • Now seeking to establish national scale: – owned & lit fiber infrastructure cost-effectively providing many waves – bulletproof dedicated, waves for separate services, projects, experiments – optical infrastructure substrate for e-science projects (proposing to a diverse array of funding agencies) – provide appropriate hooks and support for advanced network measurement and academic research – provide waves for various other R&E metro, regional and national ‘operational’ network etc services – A viable long term option (but full-costed 5 yr amortization for start) – ‘AUP free’! – Meta. Po. Ps & strategic pop’s

Optical Network Landscape • Temporarily open windows of opportunities wrt: – – Optronics kit manufacturers (some will do LAN PHY for us). Fiber Owners Long haul & Metro – great deals available now Po. Ps Co-Lo facilities – available & affordable [But Not Wave Vendors (Instead we are seeing recent rising prices/lessened competition, and unwillingness to: generalize occasional good pricing; to create believable/bulletproof long term price and availability protections; to provide techs like LAN PHY )] • Existing metro, regional (&linked!) & now interregional/national efforts to deploy cost-effective, reasonably enduring, i. e. owned & lit fiber based multi R&E infrastructure: – – Leveraging state & regional K-20 + higher ed transport activities Leveraging ‘production’ networks (on dedicated waves) Research & Experimental network capabilities To/among major research sites including NSF, Do. E, and university facilities and end users

Optical network project differentiation/evolution Distance scale (km) Metro U. S. Examples < 60 UW-Pacific N/W Gigapop (Seattle), USC/ISI (LA) 60 -500 I-WIRE (IL), CENIC ONI, I-LIGHT (IN) > 500 Pacific Light. Rail, Tera. Grid DTF, ‘Next Gen Abilene’ (many cases) Equipment Dark fiber & end terminals 5/31/02 State/ Regional (some linked) Extended Regional/ National Derived from Steve Corbato, Internet 2 Feb. ‘ 02 Add Amps & distant ops Add OEO Regens/adm’s & > O&M $$’s

ADDRESSING NETWORK DISCONTINUITIES PACIFIC LIGHTRAIL LONG LINES: • Lit fiber plus waves • Options for more @ discount • Could buy fiber (nationally) METRO: • Owned Fiber Rings/Segs. in SEA, SF BAY, LA, SD & SAC P/NWGP-Fairbanks U. Alaska ARSC etc Abilene? R SEA P/NWGP R S UW OHSU/OGI? PDX R CENIC ONI Tier 1 & 2 R EMV R Peerings? UCD Peerings UCB & LBL R R UCSB SB R UCLA ANH Meta. Po. Ps! SNY Stanford SLAC UCSF SLO PROVISIONABLE DEDICATED p 2 p WAVES SAC Caltech ESTABLISHING KEY PIECES FOR FLEXIBLE LONG TERM LOW COST PROVISIONING! or Peerings R LA ISI USC R R SD SDSC UCSD R CHI NCAR? ! I-WIRE !!! LANL? ? R UCI DEN NM NY DC 10 gig waves draft 12/4/01

National Light. Rail: cost-effective owned lit R&E fiber fabric Initial footprint 3/03 Canarie fabric Leading-Edge e 2 e services & experimental network facilities via Meta. Po. Ps and inter-gigapop links for research & next gen. tech. , arch. , grids, DTF expansion, content, sensors, apparatus … Up to 40 dedicated 10 gigabit waves with up to hundreds of dedicated gigabit links system-wide. SEA 10 gbs Tycom IEEAF donation BOS CHI SUN CLV? PIT NY DC DEN 10 gbs Tycom IEEAF Donation Raleigh? LA ATL Meta. Po. Ps & Core Nodes ADM sites Desired Expansion Po. Ps/ADM National Light. Rail” (NLR) International Broadband Metro/WAN owned Fiber ‘Rings’connecting strategic R&E endpoints. 8/14/02

Cal. REN DC NLR 10 gig Lambdas (startup NLR plus some CENIC & DTFETF waves on same glass) Seattle § Optical Amps are configured for 40 10 gigabit waves, which sub-dividable into hundreds of one gigabit dedicated paths plus 10 gigs § 4 10 gigabit waves at startup with one subdivided into 8 one gigabit paths. § Inexpensive addition of waves though-out or locally § Easily geographically extensible Portland Davis Via Metro Sacramento Oakland HPR Teragrid HPR & DC Sunnyvale Solidad Chicago Denver San Luis Obispo Los Angeles Santa Barbara Pittsburgh Tustin San Diego D. C. 8/14/02

Initial Configuration: • At least 4 express 10 gig waves on each DWDM span (in some cases there as many as 8) • 8 wdm systems/spans in the initial footprint • Capacity up to 20 express waves and 20 add/drop waves per span • 1 wave terminated in NLR router for a common shared IP service (with the exception of SAN) • 1 wave terminated on NLR switches • 1 10 GE client facing port on routers • 8 1 GE client facing ports on switches for allocatable experimental 1 gig dedicated/private net services • Client router/switch ports and extra waves connect directly at colo and/or across metro WDM – uses of other 10 gig waves under active discussion

Lit National NLR Core Fabric Fiber A. Starting footprint - operational by March 2003. A. 5, 207 miles B. Multi-Loop core for owned fiber R&E net - June 2004? B. 11, 000 miles Dashed lines for regional not costed 7 -28 -02 ronj

NLR Achieves: • National & Extensible Core system of Lit Fiber with Dedicated Waves and/or bandwidth for Experimental Production and Research networks, – Initially providing no fewer than 4 Ten Gigabit Waves Nationwide; – One of which will be subdivided into dedicated 1 gigabit circuits/facilities; – Optical Amplifiers are configured from start with 40 10 gps waves; – Very low cost for bringing up each additional 10 gigabit wave (each can be done point to point, e 2 e, or system-wide), or subdividing a wave; – Max. capacity of 40 separate 10 gigabit waves (400 gb/s total), each of which is easily subdividable into at least 8 1 gigabit dedicated circuits (or > 300 total if system-wide, or many more if point to point in smaller geographies). • Interoperates with, leverages and further enables DTF-ETF & other research & experimental network Endeavors • Easy and Inexpensive Extension to the other critical mass geographies • Establishes an enduring and highly flexible platform for enabling network based e-Science and for enabling NSF and other agency research & experimental net. activities. • Meta. Po. P & strategic physical Po. P infrastructure!

Thank you! Jbrown@cac. washington. edu CANS 2002, Shanghai, 22 August 2002