User-Based Innovation Communities Drive Commercial Systems Software

User-Based Innovation & Communities Drive Commercial Systems Software James Hamilton GM SQL Server Web. Data Development & SQL Security Architect http: //research. microsoft. com/~James. RH@Microsoft. com 2004. 15

Introduction u I’m an unrepentant commercial software guy : -) Ø u System Software Focused Ø Ø u Have done some non-commercial S/W work in spare time Ø Ported g++, gdb, and Taylor UUCP to AIX 1. 3 Ø Ran a UUCP site for years 11 years at IBM Ø Ada & C++ Development Manager Ø Lead Architect DB 2 UDB Database 7 years at Microsoft Ø Windows 2000 Base Development Ø SQL Server development team in various roles –Relational System Development Manager –Security Architect –General Manager Web. Data Development Interested in better understanding & harnessing user community contribution to S/W systems What follows are my observations & opinions and do not necessarily represent a Microsoft position 2

Agenda u Driving innovation: users or manufacturer? Ø Ø u Revolutionary change often not user driven Ø Examples from DB & TP world with which I’m most familiar Ø Redundant Array of Inexpensive Disks (RAID) Ø Relational database Users prime drivers of product evolution Community drives commercial S/W success Ø Ø Ø Commercial S/W has always been community dependent Why not go open source & gain the community contribution? Parallels between commercial & non-commercial s/w efforts 3

Revolutionary change: often not user driven u The fundamental changes are often not user driven “…Well-managed companies that have their competitive antennae up, listen astutely to their customers, invest aggressively in new technologies, and yet still lose market dominance” u Users do drive incremental & evolutionary change Ø u Some examples from DB world: Ø Online index create Ø Online re-org Ø XML datatype Ø Automated multi-system administration Ø … Revolutionary change examples from systems world: Ø RAID & Relational DB 4

Redundant Array of Inexpensive Disks u u u Disks were expensive, IBM dominated, & growing at sub. Moores law rates Commodity disk much less reliable & 1/5 to 1/10 the capacity of enterprise disk “A Case for Redundant Arrays of Inexpensive Disks (RAID)” – Patterson, Gibson, Katz Ø Base observation: Commodity disks with redundancy can be combined to produce larger & more reliable storage SIGMOD, June 1988 5

EMC Symmetrix DMX 3000 u u u 84 TB Storage capacity 576 3 ½” commodity Seagate (usually) SCSI Disks 256 GB Memory & 100 1 Ghz Power. PC CPUs 10 x to 15 x storage cost premium over commodity disk EMC annual revenue: $6. 24 B (2003 10 K filing) RAID overall annual revenue: $13 B (1998 Disk/Trend Report) 6

Relational Database u “A Relational Model of Data for Large Shared Data Banks”– E. F. Codd CACM 13, 6 (June 1970) Ø “Some industrial motivation and then straight to the math” – Irv Traiger u Broadly debated: CODASYL vs Relational Ø u SIGFIDET and SIGMOD conferences First IBM System R relational DB user: Ø MIT Sloan School of Management (System R Ø System R code base later became IBM SQL/DS Phase 0 proto 1975) Ø u Relational DB Theory: E. F. Codd later DB 2 for VM & VSE — product still available Manufacturer research rather than user community pull SQL Language: Don Chamberlin 7

Relational Database Market u u u Overall annual Relational DB market: $8 B Gartner expects 86% of DB revenue to be Relational by 2005 Many DB & TP industry innovations driven by user partnerships Ø IBM TPF: American Airlines SABER Ø Ø Project started 1959 IBM IMS: Rockwell-NASA Apollo Program Ø Released 1969 Ø IBM CICS: Partnership with Public Utilities Ø PUCICS released 1968 The innovation required user involvement but they didn’t drive it Ø Source: Gartner 2001 8

Agenda u Driving innovation: users or manufacturer? Ø Ø u Revolutionary change often not user driven Ø Examples from DB & TP world with which I’m most familiar Ø Redundant Array of Inexpensive Disks (RAID) Ø Relational database Users prime drivers of product evolution Community drives commercial S/W success Ø Ø Ø Commercial S/W has always been community dependent Why not go open source & gain the community contribution? Parallels between commercial & non-commercial s/w efforts? 9

Communities Drive Commercial S/W u Product Support Ø Ø u User community support is the best way to provide scalable, available, high quality product support Ø It’s difficult to invest enough in a dedicated support team to replace a community based program Examples form engineering team in which I work: Ø Participation required of all team members Ø Newsgroups, customer presentations, feedback sessions, service team internships, customer requirements DB, work with MVP community, sample programs, … Marketing and Sales also heavily community driven Ø Ø ISV & reseller sales model common Shareware & community sales sites 10

Community Example: www. boatdiesel. com 11

Why not use Community Development? u S/W business driven by important, difficult to reproduce S/W aggregations Ø Ø Ø u Large S/W aggregations are actively protected Ø u But they are open in many ways Ø Most support interfaces to allow 3 rd party extension Ø Source is typically available in a controlled way Open-source S/W systems also actively protected Ø u Separates lack-luster profit from the truly impressive S/W business cost of entry is very close to zero Ø Profit of most entrants unexciting Large S/W systems with critical mass very valuable: Ø SAP, Windows, Oracle, DB 2, … However, branches are possible and sometimes succeed Commercial S/W open at key interfaces but user source code mods typically not supported Ø Both commercial and non-commercial S/W systems dependent upon community for success 12

Summary u Most product innovation is driven by users Ø Ø u Revolutionary changes often not user driven Ø Ø u Improvements in speeds, feeds, & features Applications of existing technology to new domains Fundamental new approaches to existing problems Especially approaches that are: Ø Not backward compatible Ø Don’t adequately solve the entire breadth of the problem domain All products, whether commercial or not, depend upon user community for support & innovation Ø Ø Ø User driven support only affordable & effective option User Community source of most innovation User development community typically the driver of even non-open source product success 13

Microsoft