Скачать презентацию Build vs Buy Internet Datacenter Operations Research INTAP Скачать презентацию Build vs Buy Internet Datacenter Operations Research INTAP

5880770787cda7200b6ddadf7d29e9a1.ppt

  • Количество слайдов: 62

Build vs. Buy: Internet Datacenter Operations Research INTAP March 6, 2001 Tokyo Build vs. Buy: Internet Datacenter Operations Research INTAP March 6, 2001 Tokyo

Agenda • • My Background and Who is Equinix? – 15 minutes Recent Internet Agenda • • My Background and Who is Equinix? – 15 minutes Recent Internet Operations Research – 20 minutes “Build vs. Buy Internet Datacenter” White Paper – 40 minutes Future: Where our Industry is heading – 5 minutes

NSFNET to Post-NSFNET Era Internet NSFNET (1987 -1994) Regional Tech Meetings NWNet Merit Oar. NSFNET to Post-NSFNET Era Internet NSFNET (1987 -1994) Regional Tech Meetings NWNet Merit Oar. Net Nyser. Net NEARNet BARRNet NSFNET –Performance & Operations Regional –Reporting & Coordination SDSC –Routing & Network Planning Regional Sesqui NCAR Oar. Net NWNet Privatization (1994 -Now) North American Network Operators Group Regional Merit ANS Nyser. Net UUNet Sprint MCI (NANOG) Meetings AT&T Regional MY JOB: 5 Years as Chair of NANOG 12 Years in Internet…No Internet Preso… Regional

Internet Statistics • 87. 5% of all Internet statistics are made up. Source: I Internet Statistics • 87. 5% of all Internet statistics are made up. Source: I made it up.

Equinix, Inc. • Scale – size of Internet Business Exchanges, how many, where they Equinix, Inc. • Scale – size of Internet Business Exchanges, how many, where they are, future plans • “Virtual Fly Through” Tour of the IBX - 6 minutes

Industry Leading Security IBX™ Centers • 5 security checkpoints to customer equipment • State Industry Leading Security IBX™ Centers • 5 security checkpoints to customer equipment • State of the art video imaging and archiving • On site security personnel 24 x 365 IP/Carrier Knowledgeable Staff Customer Friendly Spaces – Conducting Business & Equipment Preparation • Designed from customer experience • Ergonomic work spaces Fully Redundant Backup Power • Built to 3 x power capacity • 24 x 365 availability • 3 levels of power backup • Customer troubleshooting • Forward looking design for growing industry • Smart Hands

IBX™ Growth Sites currently open or Under Development IBX™ Growth Sites currently open or Under Development

Equinix Internet Business Exchange Model • Infrastructure Content Providers • Players • Neutrality… Neutral Equinix Internet Business Exchange Model • Infrastructure Content Providers • Players • Neutrality… Neutral Internet Business Exchange Model Internet Service Providers Carriers

Carrier Internet Exchange Model • Infrastructure Carrier Operated Internet Exchange • Players Internet Service Carrier Internet Exchange Model • Infrastructure Carrier Operated Internet Exchange • Players Internet Service Providers Non Carrier-Neutral No Content Support Economic Distortions Carrier Operator

Job Title: Equinix Co-Founder & Chief Technical Liaison • So, my job is to Job Title: Equinix Co-Founder & Chief Technical Liaison • So, my job is to take friends out for lunches and dinners… • Internet Operations Research & • Write White Papers • Summary: What did I learn?

No Internet Operations Documentation To make things more difficult, subtle language differences! No Internet Operations Documentation To make things more difficult, subtle language differences!

The Internet Operators: All about relationships The Internet Operators: All about relationships

My Job: White Papers New Job Title: Equinix Co-Founder & Chief Technical Liaison Document My Job: White Papers New Job Title: Equinix Co-Founder & Chief Technical Liaison Document Internet Operations Research – “Interconnection Strategies for ISPs” – “Internet Service Providers and Peering” – “Peering Case Studies” – “Peering Simulation Game” – “Build vs. Buy Internet Datacenter” today

White Paper: Interconnection Strategies for ISPs Direct circuits replaced with: • OC-12 into Exchange White Paper: Interconnection Strategies for ISPs Direct circuits replaced with: • OC-12 into Exchange • Fiber cross connects Cost Drivers: • Big OC-12 Direct Circuit • Cheap Fiber Cross Connects • Aggregation Efficiency over OC-12 (2: 1, 3: 1) • Rack Fees Exchange Model Cost Formula: Exchange. Model. Cost=Big. Direct. Circuit()+(n-1)*x/2+Rack. Fees() OC-12 $23, 000/mo XC=$200/mo Rack=$1500/mo So does the Exchange-based model make sense?

White Paper: Internet Service Providers Transit & Peering ISP A $$ Transit ISP B White Paper: Internet Service Providers Transit & Peering ISP A $$ Transit ISP B Transit Peering $$ Peering West. Net Routing Table USNet East. Net

White Paper: Case Study: Peering in the Bay Area White Paper: Case Study: Peering in the Bay Area

White Paper: The Peering Simulation Game White Paper: The Peering Simulation Game

White Paper: Build Vs. Buy Internet Data. Center • Modeling: • When does it White Paper: Build Vs. Buy Internet Data. Center • Modeling: • When does it make sense to build your own? • To Outsource? • Issues to consider • Economies of Scale? • Multiple Business Models

Service Level Agreements in Principle • Guarantee a higher level quality • Define Measurement Service Level Agreements in Principle • Guarantee a higher level quality • Define Measurement Criteria for adhering to higher quality service Level • Meeting Performance Level critical to success of business • Failure to meet Service Levels penalty significantly worse than cost of service – e. Bay: $64, 000/minute of downtime

SLAs in Practice • No SLA Pricing – ISPPrice. NO SLA=X • SLA Pricing SLAs in Practice • No SLA Pricing – ISPPrice. NO SLA=X • SLA Pricing – Probability. Of. Missing. SLA=p – ISPPrice. SLA 1=X*(1+p) • Network does not typically change to meet new criteria • CIO Security Audit Story • Equinix Switch SLA: 99. 999% Uptime

SLA-Datacenters • • • State-of-the-art Datacenters Infrastructure Failure unlikely SLAs rare, but… Typically Limited SLA-Datacenters • • • State-of-the-art Datacenters Infrastructure Failure unlikely SLAs rare, but… Typically Limited to free month service (Insufficient to potential loss customer faces) • But what is an Internet Datacenter? • Include hosting? Include Internet Service? Include Caching, Content Distribution? • Models help Define and Frame the Problem…

SLA Datacenter Parameters • If there were Internet Datacenter SLAs • Parameters: – Installation SLA Datacenter Parameters • If there were Internet Datacenter SLAs • Parameters: – Installation Lead Time – Service Availability – 24/7 Customer Support – Provisioning/Responsiveness • Independent of Transport and Internet Service Issues.

Creating the Build vs. Buy Internet Datacenter Whitepaper • Create a community white paper Creating the Build vs. Buy Internet Datacenter Whitepaper • Create a community white paper on issues associated with the building & scaling of Internet Datacenters • Identify the issues that folks should consider when making the build vs. buy decision Today: • Demonstrate the document & models to date and get feedback NOTE WELL: • EARLY STAGE OF BVB WHITE PAPER (v 0. 4)! • This Model may be of help to you in your SLA Discussions

Toured 32 (now) data centers Toured 32 (now) data centers

Some with very Secure Fiber Access… Source: Technology Center of the Americas Tour Some with very Secure Fiber Access… Source: Technology Center of the Americas Tour

Some with less secure fiber entry… Source: Don’t Ask. Some with less secure fiber entry… Source: Don’t Ask.

Some Very Secure… http: //www. thebunker. net Some Very Secure… http: //www. thebunker. net

Some Cat 5 Hurricane proof… Some Cat 5 Hurricane proof…

Some not quite cat proof. Loading Dock Some not quite cat proof. Loading Dock

Great religious debate over ties Velcro vs. Plastic cable ties vs. String Great religious debate over ties Velcro vs. Plastic cable ties vs. String

Time to white paper what I’ve seen & Facilitate BOF Discussion • Datacenter BOF Time to white paper what I’ve seen & Facilitate BOF Discussion • Datacenter BOF I NANOG Oct ’ 98 Atlanta • Build vs. Buy White Paper • Internet Datacenter Model – Definition of Internet Datacenter • Focused discussion of Each Component of the Model

Internet Datacenter Model Dependence Relationship Increased Value Internet Datacenter Model Dependence Relationship Increased Value

Customer Business Model Value Dependence Customer Business Model Value Dependence

Internet Datacenter Model Example IXP Value Dependence Internet Datacenter Model Example IXP Value Dependence

Size Ultimate Goal • Build Financial models that allow an ISP to say: • Size Ultimate Goal • Build Financial models that allow an ISP to say: • “Given our size requirements, it makes sense to build our own. ” Or • “Given our size requirements, it probably makes sense to outsource this to someone who can realize economies of scale. ” Unit cost + + = Buy Build

Flavor of the Real Estate Model • • • Place to put stuff Proximity Flavor of the Real Estate Model • • • Place to put stuff Proximity to Fiber Space Size Utility Power Facilities Yard Scaling of Datacenter Real Estate? Increased Value Relationship

Scaling of the Real Estate Model $/sq. ft. Premium Real Estate Market (Tokyo, NYC, Scaling of the Real Estate Model $/sq. ft. Premium Real Estate Market (Tokyo, NYC, …) Real Estate Market (everywhere else) # Sq. Ft.

Real Estate Size is Important • Choose Too Small: Run out of space, lose Real Estate Size is Important • Choose Too Small: Run out of space, lose economies of scale • Choose Too Big: Cost per Rack very high Vs. • Right of First Refusal (some Internet Datacenters allow this) • Expandability of Space

Commitment to Data. Center • 1 yr contract Vs. • 10 year lease Time Commitment to Data. Center • 1 yr contract Vs. • 10 year lease Time to Market Issue – 6 – 9 months to build

Hidden Charges • Meet Me Rooms • Required products: – Must buy port on Hidden Charges • Meet Me Rooms • Required products: – Must buy port on switch or certain volume of BW • Cross Connect Fees

Size & Finding Real Estate • 100, 000 sq ft in downtown DC? • Size & Finding Real Estate • 100, 000 sq ft in downtown DC? • 10, 000 sq ft in downtown DC • Vs. • Fiber Miles to Internet Datacenter

Flavor of the Infrastructure Model • Heating Venting Air Conditioning (HVAC) • Security Infrastructure Flavor of the Infrastructure Model • Heating Venting Air Conditioning (HVAC) • Security Infrastructure • Power Infrastructure • Fire Suppression

Heating, Venting, Air Conditioning (HVAC) Infrastructure • Controlled Humidity ~45% • Too much Humidity: Heating, Venting, Air Conditioning (HVAC) Infrastructure • Controlled Humidity ~45% • Too much Humidity: System Failure • Too Little Humidity: Static Electricity • Air Temperature: ~72 (IBM: 65 -85) • • Solutions: DX Systems Air Cooled Water-Based • How do these Scale?

HVAC Cost HVAC System Cost: Economies of Scale Direct Expansion HVAC Scaling $1, 000/ton HVAC Cost HVAC System Cost: Economies of Scale Direct Expansion HVAC Scaling $1, 000/ton Air Cooled $478/ton Chilled Water System $250/ton DX Systems: 25 -300 tons of Cooling ~ 10 K sq ft or ~150 -200 racks Terramark Outsource of HVAC to FPL… # Racks

Crossover point where Chilled Water lowest per-rack solution Crossover point where Chilled Water lowest per-rack solution

Trend? Outsourcing of HVAC FPL HVAC (Miami FL) Water Chiller NOTA: 3 Blocks Away Trend? Outsourcing of HVAC FPL HVAC (Miami FL) Water Chiller NOTA: 3 Blocks Away Source: NAP of the Americas Tou

Security Infrastructure • 24 x 7 Manned • Access: Biometrics, card keys • Video Security Infrastructure • 24 x 7 Manned • Access: Biometrics, card keys • Video Surveillance Sliding Glass

Power Infrastructure • Sun Netra 1 U=90 Watts – 40 U Rack=3600 Watts! • Power Infrastructure • Sun Netra 1 U=90 Watts – 40 U Rack=3600 Watts! • Power Source – Power Grid (dual feeds) – Co-Generation – UPS/CPS/Generators • Energy Storage – UPS Batteries – Flywheel (no UPS) – Fuel Cells (service) • Monitoring Issues • Redundancy Issues • TBD: Scaling Issues

Fuel Cells for Generators Fuel Cells for Generators

Fire Suppression Infrastructure • Two Dominant Systems: • VESPA air sampling • Dry pipe, Fire Suppression Infrastructure • Two Dominant Systems: • VESPA air sampling • Dry pipe, pre action water system vs. FM-200 • Smoke & Heat Required to activate region

Infrastructure Operations • Operations of HVAC, Security, Power systems • N+1 Redundancy Infrastructure Operations • Operations of HVAC, Security, Power systems • N+1 Redundancy

Internet Datacenter Model Internet Datacenter Model

Transport Service Model • Dark Fiber Provider(s), Carrier(s), Aggregators (like Universal Access), etc. • Transport Service Model • Dark Fiber Provider(s), Carrier(s), Aggregators (like Universal Access), etc. • Business Model? – Owned by Carrier? Will others come in? – Not owned by Carrier? Will Carriers come in? • What type of services available? • How long to Provision? • Big Problem: Keeping up with BW Demand

Internet Datacenter Model Internet Datacenter Model

Internet Service Model • ISP(s), Aggregator(s) like Inter. NAP? • Business Model – Owned Internet Service Model • ISP(s), Aggregator(s) like Inter. NAP? • Business Model – Owned by ISP? Will others come in? – Not owned by ISP? Will others come in? • Capacity Issues • Latency Issues • Interconnection Strategies Allowed?

Internet Datacenter Model Internet Datacenter Model

Content Provider Model • Hoster Focus: – Installing New Servers, Provisioning Host Services, – Content Provider Model • Hoster Focus: – Installing New Servers, Provisioning Host Services, – Managing Server Farm, SLAs, Tech Support, 24/7 NOC, Back Office Solutions • Hoster owns Datacenter? – – Pro: Can buy Transit Services Con: Expensive w/Bandwidth Hungry Apps Con: Shared bandwidth issues (Yahoo! attack) Con: Local Loop Expensive • Business Model Content Distributor(s), Hoster(s)

Internet Datacenter Model Internet Datacenter Model

Future of the IXP? DSL Access Providers Wireless Access Svc Dial-in Modem Operators USENET Future of the IXP? DSL Access Providers Wireless Access Svc Dial-in Modem Operators USENET Svcs Virtual E-Mail Svcs ISP Web. Host Svcs Search Engines Cache/Content Distribution Sparing Service Providers Internet Service Provider Carrier Carriers Market. Place: Stratification of component operations, fast creation & provisioning of packaged operations services Implication: IXP Content Sources, bandwidth demands

Future SLA Challenges 1. Bundling of Internet Operations Services… 2. Into Cohesive SLA-able services Future SLA Challenges 1. Bundling of Internet Operations Services… 2. Into Cohesive SLA-able services 3. Notion: Trickle-down SLA Bundle Participant Operations Agreements Financial Agreements Example: Bundle Internet Data. Center, Transport, Internet, MSP. What happens when Transport service fails? Who covers the cost of the SLA?

Bv. B Document Availability • All references are in the Bv. B white paper Bv. B Document Availability • All references are in the Bv. B white paper – Hyperlinks work in electronic version of document – Early Stage of document: Comments Welcome! • Send e-mail to [email protected] com Subject: Build vs. Buy White Paper

Thanks to • Sean Donelan (Equinix) • Friends at Exodus, Frontier • Jason Colangelo Thanks to • Sean Donelan (Equinix) • Friends at Exodus, Frontier • Jason Colangelo (APC) Global Center, Level 3, etc. • Daniel Golding (Net. Rail) • Stephen Stuart/Phil Steffora (MFN) • Kenney Huang (Asia. Infra) • All Colo/ISPs tour • Sandip Gupta (Ensim) facilitators • Rob Glenn (Sun) • Jay Adelson (Equinix) • Everyone here! Please e • Josh Horn (NOTA) mail feedback. • Justin Newton (Net. Zero)