Build vs Buy Internet Datacenter William B Norton

Build vs. Buy: Internet Datacenter William B. Norton – Equinix wbn@equinix. com Michael Lucking – Epoch Internet

Goals • Create a community white paper on issues associated with the building & scaling of Internet Datacenters • Identify the issues that folks should consider when making the build vs. buy decision Tonight: • Demonstrate the document & models to date and get feedback

Non-Goals (Out-of-Order) Not to advocate building you own rack space Not to advocate buying rack space (Rather – document the scaling of each and the criteria involved in the decision) (Warning: Tradeoffs are Religious Topics for many. )

Toured a lot of data centers

Some with very Secure Fiber Access… Source: Technology Center of the Americas Tour

Some with less so Source: Don’t Ask.

Some Very Secure… http: //www. thebunker. net

Some Cat 5 Hurricane proof…

Some not quite cat proof. Loading Dock

Great religious debate over ties Velcro vs. Plastic cable ties vs. String

Time to white paper what I’ve seen & Facilitate BOF Discussion • Datacenter BOF I NANOG Oct ’ 98 Atlanta • Build vs. Buy White Paper • Internet Datacenter Model – Definition of Internet Datacenter • Focused discussion of Each Component of the Model

Internet Datacenter Model Dependence Relationship Increased Value

Internet Datacenter Model Example Value Dependence

Ultimate Goal • Build Financial models that allow an ISP to say: • “Given these parameters and our required footprint, it makes sense to build our own. ” Or • “Given these parameters, it probably makes sense to outsource this to someone who can realize these economies of scale. ” + + = Buy Build

Real Estate for Internet Datacenter Issues • • • Place to put stuff Proximity to Fiber Space Size Utility Power Facilities Yard Scaling of Datacenter Real Estate? Increased Value Relationship

Scaling of the Real Estate Model $/sq. ft. Premium Real Estate Market (Tokyo, NYC, …) Real Estate Market (everywhere else) # Sq. Ft.

Real Estate Size is Important • Choose Too Small: Run out of space, lose economies of scale • Choose Too Big: Cost per Rack very high Vs. • Right of First Refusal (some Internet Datacenters allow this) • Expandability of Space

Commitment to Data. Center • 1 yr contract Vs. • 10 year lease Time to Market Issue – 6 – 9 months to build

Hidden Charges • Meet Me Rooms • Required products: – Must buy port on switch or certain volume of BW • Cross Connect Fees

Size & Finding Real Estate • 100, 000 sq ft in downtown DC? • 10, 000 sq ft in downtown DC • Vs. • Fiber Miles to Internet Datacenter

Infrastructure Scaling • Heating Venting Air Conditioning (HVAC) • Security Infrastructure • Power Infrastructure • Fire Suppression

Heating, Venting, Air Conditioning Infrastructure • Controlled Humidity ~45% • Too much Humidity: System Failure • Too Little Humidity: Static Electricity • Air Temperature: ~72 (IBM: 65 -85) • • Solutions: DX Systems Air Cooled Water-Based • How do these Scale?

HVAC Cost HVAC System Cost: Economies of Scale Direct Expansion HVAC Scaling $1, 000/ton Air Cooled $478/ton Chilled Water System $250/ton DX Systems: 25 -300 tons of Cooling ~ 10 K sq ft or ~150 -200 racks # Racks

Crossover point where Chilled Water lowest per-rack solution

Security Infrastructure • 24 x 7 Manned • Access: Biometrics, card keys • Video Surveillance Sliding Glass

Power Infrastructure • Monitoring of UPS, DC Power, PDUs, • N+1 Redundancy • Power Draw? – 20 K=500 Kw – (Jason Colangelo, APC)

Fuel Cells for Generators

Fire Suppression Infrastructure • VESPA air sampling • Dry pipe, pre action water system vs. FM 200

Infrastructure Operations • Operations of HVAC, Security, Power systems • N+1 Redundancy

Transport • Dark Fiber Provider(s), Carrier(s), etc.

Internet Service • ISP(s), Aggregator(s)

Content Providers • Content Distributor(s), Hoster(s), Manager(s)

Sample Crank of the Model: Assumptions • Both: $400/sq foot build out costs for conditioned space • Buy: Equinix retail price $1, 500/rack used for comparison, • Buy: No discount for buying more or larger cages • Build: Building Real Estate comes in 10, 000 sq foot increments • Both: Conditioned space includes infrastructure (power, HVAC, Security systems, etc. ) • Both: Does not include network bandwidth, hosting equipment, managed services, etc. • Build: Financing of build costs is available, • Build: Adequate building is available, and build time (time to market) is not an issue • Build: Sufficient network capacity (transport and transit) is available and not an issue • Both: Facility population is not an issue – ignoring collocation benefits and don’t expect to buy or sell services at the facility.

Sample Crank of the Model: Internet Datacenter Build vs. Buy Note: Fictitious

References – Don’t Squint… • • • • Ripin & Sayles, Insider Strategies for Outsourcing Information Systems, 1999 Oxford University Press Greaver, Strategic Outsourcing: A structured Approach to Outsourcing Decisions and Initiatives, 1999 AMACOM Frenza, J. P. , Buying Web Services, 1999 Wiley Computer Publishing, particularly Chapter 1: An Outsourcing Overview by Frenza and Chapter 5: Infrastructure and Site Hosting by Chris Bryant. Norton, Donelan, Adelson, Newton, Data Center Needs Birds of a Feather meeting at NANOG 14 – Atlanta, Georgia – November 9, 1998 7: 30 -9 PM, http: //www. nanog. org/mtg 9811/data. bof. html Schroepfer, Andrew M. , Communication Service Providers, Equity Research, U. S. Bancorp Piper Jaffray Equity Research, December 1999, particularly Ch 12: Facilities Hosting Providers. “What every CEO needs to know about Electronic Business”, Business Week, e. biz, Marsh 22, 1999 IBM White Paper: Building a shared services data center, http: //www. as. ibm. com/asus/cwpliushrdsvcs. html, focuses on shared services enterprise Data Center Murray, Steve, “Web hosting Strategies: Market Review and Forecast, 1998 -2003”, IDC Report #20953, December 1999. Search for “Data Center Construction” on http: //www. google. com. http: //www. as. ibm. com/asus/nordstrom. html http: //www. gruber. com/compfac/datal. htm http: //www. edgemail. com/whitepapers/ Dialtone Internet: http: //dialtoneinternet. net/tour/page 0. htm

References – cont’d • • • • • Yahoo! Directory of Colocation Providers: http: //dir. yahoo. com/Business_and_Economy/Companies/Internet_Services/Web_Services/Network_Service_Providers/Hosti ng/Colocation/ Yahoo! Directory of web hosting providers: http: //dir. yahoo. com/Business_and_Economy/Companies/Internet_Services/Web_Services/Network_Service_Providers/Hosti ng/Web_Site_Hosting/White. Pages/wp_0. html Search for Internet Data Center on google. com: AT&T Internet Data Centers: http: //www. ipservices. att. com/datatour/idc_dedicated. html Web hosting directory site: http: //www. webhostdir. com/ Exodus Internet Data Center descriptions and virtual tour: http: //www. exodus. com/idcs/ . Equinix Internet Business Exchange (IBX) virtual tour http: //www. equinix. com PSINet Colocation Product: http: //www. psinet. com. hk/hosting/co_location. html Flashhost data centers http: //www. flashhost. com/data_center. html Hosting Internet Data Center in Zaire: http: //www. is. co. za/solution/hosting/IDC 2. html Level 3 colocation facilities: http: //www. level 3. com/us/services/colocation Frontier Global Center (now global crossing) http: //206. 132. 184. 106/solutions/datacenters. shtml Exodus Communications Q 1 2000 Quarterly Earnings Announcement 4/20/2000 2: 00 PM Pacific Time: http: //webevents. broadcast. com/financecalls/event/index. asp? Earnings. ID=766&src=Street. Fusion Houses are designed by Geniuses & built by Gorillas, by Bob Johnson, ISBN: 1 -886110 -48 -4 Facilities Operations & Engineering Reference, The Certified Plant Engineer Reference, R. S. Means Company, Inc. , ISBN: 087629 -462 -X Network Equipment-Building System (NEBS) Requirements: Physical Protection, Bellcore GR-63 -CORE, Issue 1, October 1995. A module of LSSGR, FR-64; TSGR, FR-440; and NEBSFR, FR-2063 Central-Station Alarm Services, UL 827 Standard for Safety, Underwriters Laboratories, Inc. ISBN 0 -7629 -0067 -9 , Sixth Edition © 1997

Document Availability • All references are in the Bv. B white paper • Send e-mail to wbn@equinix. com

Thanks to • Sean Donelan (Equinix) • Friends at Exodus, Frontier • Jason Colangelo (APC) Global Center, Level 3, etc. • Daniel Golding (Net. Rail) • Stephen Stuart/Phil Steffora (MFN) • Everyone here! Please email feedback. • All Colo/ISPs tour facilitators • Jay Adelson (Equinix) • Josh Horn (NOTA) • Justin Newton (Net. Zero)