There is no magic Bob Martin bobmartin 08008 mac

There is no magic Bob Martin bobmartin 08008@mac. com “I assume you will show the alternative ways to use a cage, a whip, and a chair to manage software teams? ” - Stu Feldman, VP Engineering Google

About Me • Built a lot of software ~100 M lines of code – System software • Operating systems • Programming languages • Transaction control – Network management – Service control points – Embedded systems • Packet switches • Network equipment • Testing equipment • Love technology – EE and CS trained – Life long passion – now neural science, molecular & evolutionary biology • Help me convince Al to work in computational biology! • Not expert in contemporary software technology/tools • Conned into this talk by a great friend of 42+ years

Facts of Life • Failure is the norm – But it needn’t be • Error correction costs increase exponentially during the lifecycle • Programmers are a special breed* – – They like machines not people They have high need for approval/recognition They must respect the leader Beware, complicators and prima donnas lurk in the weeds • Software engineering is complexity management – Race with technology/tools – Complexity is still ahead • Good software engineering is old hat – But is not always used - why? ? “Two things are infinite: the universe and human stupidity; and I'm not sure about the universe” - Einstein * “Psychology of Computer Programmer” - Gerald Weinberg

Why Software Projects Fail * Average overrun: 89. 9% on cost, 121% on schedule, with 61% of content * Barry Boehm - A View of 20 th and 21 st Century Software Engineering

Increase in Software Cost-to-fix vs. Phase (1976) * Relative cost to fix defect 1000 500 200 100 50 20 • 10 Smaller Software Projects 5 2 1 Requirements Design Code Development test Acceptance test Phase in which defect was fixed * Barry Boehm - A View of 20 th and 21 st Century Software Engineering Operation

What Year Was This Conference? NATO Software Engineering Conference - 1968 Design Production • • Guidelines – – • • Completeness, modularity, efficiency Self-monitoring and performance improvement Incremental systems Balance • Security, control, … vs. cost • Limited goals to attain excellent performance Design problems – – – • Deductive or inductive Modularity and interfaces Complexity control Proscriptions – – • – – External function, e. g. , user language Internal function, e. g. , parsers Techniques – – – Data structures on system design Fixed resources Cooperating processes Documentation Organization for producing software • • Number and quality of people Structure of large groups Control and measurement Internal communication Production techniques Monitoring Service • Distribution – • • Acceptance criteria Documentation Adaptation Maintenance – – • • • Media Error detection & reporting Response and distribution Documentation Performance Feedback into design

Impact of Increased Discipline • Low software engineering maturity results in: – Reduced productivity (35%)* – Late detection of defects (22%)* – Longer time to market (19%)* – Higher post-release defect rates (39%)* * median annual improvement 0. 8 CMM** process maturity Industry-wide Company X 0. 7 0. 6 0. 5 0. 4 0. 3 • Takes several years of focused improvement to become a high-performance software team 0. 2 0. 1 0 • Focus on the programmer – Just management: well-controlled junk • Delicate touch to avoid process imprisonment 1 2 3 4 CMM Level ** Software Engineering Institute’s Capability Maturity Model (CMM)

Learning • Disasters – Mine – Others • Really good companies – – – IBM - Discipline & inspections Sun - Tempo Japan - Quality circles & reuse Microsoft - Reuse Apple - User centered design & tool kits, “Design of Everyday Things” - Don Norman • Really smart professionals – – – – Al Aho – Swamp drainer Fred Brooks – “Mythical Man Month, ” “No Silver Bullet” Barry Boehm – Estimation and cost of errors Michael Cusumano – Software factories Edsger Dijkstra – Importance of time in systems Watts Humphrey - Encapsulated team and personal discipline - “TSP, PSP” Martyn Thomas - Formal methods Ken Thompson - The programming craft

Two Troubled Projects • Loop Maintenance System – IBM MVS system – PDP 11/20 backup – 8 Months to go No schedule No performance plan No requirements Weak management Switch Test Head PDP 11/20 Repair Service Bureaus PDP 11/70 IBM 370 • Loop Assignment System – Univac system – Failed trial $250 million down the drain Demoralized, weak team – Senior 18 -month commitment No architecture or requirements No technology infrastructure – Working “temporary” subsystem Unix based “Temporary” Subsystem “Robust Pipe” Univac 1100 with C

Building The Team • All - domain expertise, discipline & simplifiers • Team leads – programming, lateral & motivational skills • Architect – deep system software & computer science – Great ones are really rare “Everything should be made as simple as possible. . . but not simpler” - Einstein • Specification - user-centered design & customer skills • Design/programming – proud craft persons – Read great books to learn to write, read great programs to learn to program “Lion’s Commentary on Unix” - John Lions • • • Testing – sadists & analysts Performance – back of envelopers Quality - superb communicators Tools & environment – geeks emeritus Product manger coordinated extended team – Sales, marketing, pricing, legal, service, … “Executives owe it to the organization and to their fellow workers not to tolerate nonperforming individuals in important jobs” - Drucker

Process is not a four-letter word • Iterative by many names – Build a little, test a little, refine requirements a little – Entrance/exit criteria Iterative Definition/Development b 1 b 2 b 3 b 4 b 5 Integration, Test, & Trial • Templates & best cases R 1 R 2 R 3 – Lightweight – Web • Inspections & reviews – Requirements, design, code, test, etc – Lightweight to heavyweight – Especially on tough stuff and with new people • Lead customers – User-centered design • Screw-up early detection & action – Jeopardies and speaking up – Audits with smart friends Phased Deployment • Early on – Architecture – New technology/risk – Performance – Factory operations Handoffs Automation Configuration control • Later with increasing process discipline – Features – Quality “I had the world’s greatest group, and I should have made the world’s two greatest groups. I didn’t realize there are benefits to having real implementers and real users” - Alan Kay

What’s your favorite bug?

Architecture “Controlling complexity is the essence of computer programming” - Kernighan • Top-down decomposition of a complex problem to allow independent, bottom-up development and change • Goals – Changeability/customizability – Chunk independence • Reduce n 2 effects – Inject huge-payoff CS technology • Tiny languages, protocols, finite state machines, search, algorithms & data structures, formal methods, etc – Performance – Reuse • Check it – Prototype – Audit with smart friends • Crumble with time – “An Architecture History of the Unix System” - Feldman Usenix ‘ 84

Calling the shot • The true art – Best to wait until specification/design done – Domain expertise crucial – Sizing models help, but not magic (still +/- 30%) “Let the numbers do the talking” - Don Reifer • Aggressive but doable • Starvation not gluttony – Reduce communications, complexity & feature creep “Whilst it is true a large programming project might require a large programming team, a large programming team will always build a large project” - J. K. Buckle “Managing Software Projects” • Slips – If you must, do it once – Iterative development provides an out “A customer will always forgive you a slipped schedule or missed feature, they will never forgive you for bad quality or performance” - Buckle

Which Project Used 15 x Staff of the Other? • Loop Maintenance System – IBM MVS system – PDP 11/20 backup – 8 Months to go No schedule No performance plan No requirements Weak management Switch Test Head PDP 11/20 Repair Service Bureaus PDP 11/70 IBM 370 • Loop Assignment System – Univac system – Failed trial $250 million down the drain Demoralized, weak team – Senior 18 -month commitment No architecture or requirements No technology infrastructure – Working “temporary” subsystem Unix based Temporary Subsystem “Robust Pipe” Univac 1100 with C

How many lines of code? Determine the frequency count of the occurrence of words in text, most frequent first • • “ 1” - Six Unix tools, piped together 100 - C, Pascal ~600 - ACM paper on commenting code 1, 000 - Cobol, PL/1 5, 000 - Unisys SW VP 10, 000 - IBM SW VP ? ? ? with very detailed requirements

Tempo “. . the disaster is due to termites, not tornadoes; and the schedule has slipped imperceptibly but inexorably. Indeed, major calamites are easier to handle; one responds with major force…” - Brooks • The plan – Delicate balance of top/down and bottom/up – PERT to plan, GANTT to manage • Critical path control – Don’t multitask – Buffer schedule – 50% estimates • Railroad train feature loading – Leave the station on time – Features left behind • The weekly/biweekly project meeting – Milestones • All milestones are important, particularly early ones and few big ones • Build team morale, tempo, customer confidence “No plan survives contact with the enemy” - Helmuth von Moltke the Elder

The Project Meeting “The Screaming”

Whipping the rowers • Project manager style – Most decisions are 50/50 – Avoid acting like “A glacier with dignity” “An officer in charge on an Indian agency made a requisition in the autumn for a stove costing seven dollars, certifying at the same time that it was needed to keep the infirmary warm during the winter, because the old stove wore out. ” Then, after glacial dignity “The stove is here. So is spring” * * “An Autobiography” - Theodore Roosevelt

Testing “Program testing can be a very effective way to show the presence of bugs, but it is hopelessly inadequate for showing their absence” - Dijkstra • Full lifecycle activity – One day build and test (tempo!) Buy death march whip • Multi-level • What to test – – Requirements & design Coverage Usability Performance Number – Automated – Domain specific tools – Field driven test libraries } Announce new features • Measure with S curves Time Test cases run Bugs found Bugs closed

Get your S-curve You can’t make this stuff up

Communications & Documentation “Schedule disaster, functional misfits, and systems bugs all arise because the left hand doesn’t know what the right hand is doing” - Brooks • Project meetings – Hierarchical – Weekly or biweekly • Jeopardies are good • Documents – “The act of writing forces hundreds of tiny decisions” - Brooks – Few key ones, all template driven, all reviewed/inspected • Requirements, architecture, design, test & project plan • Strong version and configuration control • Code – Structure - aim for one pagers – Comments • “This section of code is cursed” - Stroustrup C++ • “You are not expected to understand this” - Unix

• Quality – Measures • Fault density • Service responsiveness Number Metric Driven Improvement – Improvement 4 3 2 1 • Root cause analysis • Team & individual >6 …. • Customer satisfaction – Huge fault density correlation . . 2 Age . . 4 3 2 1 Severity 1 Project Teaches . . Project Learns Installability Usability Service Performance Functionality Quality Organization Project Good • Productivity 4 3 2 1 Organization Learns

A Project-Specific Metric • Project characteristics – Gluttonously staffed ~ 5, 000 people – Long crumbled architecture - 20+ years – Kryptonite-weight processes • Humongously successful – 99. 9999 uptime • Down 30 seconds a year for all causes – Drug dealer cash flow • Proposed programmer metric Lines of code Meetings attended Improvement objective: exceed 1!

Contrarian Views • Design methodologies – Great designers make great designs – Methodologies make lousy designs look good • Fancy measures – Confuse precision with accuracy

Making It Stick • People support what they invent • Thick dictates are good only for cellophane – Bellcore’s was ten one-pagers (what not how) • Participatory multi-day course – Crusty expert led – Project team takes course • Time it with a new release schedule • Output is their plan & their process • Team mentors • Senior support

My Most Notable Disasters • “Hi ho silver” – Loop maintenance system • Domain ignorance – Customer ordering system “Organizations can only do damage to themselves and to society if they tackle tasks that are beyond their specialized competence, their specialized values, their specialized functions” - Drucker • Second system effect – Switch assignment system “This second is the most dangerous system a man ever designs. …The general tendency is to over-design the second system, using all the ideas and frills that were cautiously sidetracked on the first one. ” - Brooks

Maybe a little magic • High quality, on time useful software can be built – Bellcore after six-years of continuous improvement (1994) • 95% on time/content • 3 -4 x competitive productivity • 10 x competitive quality • It’s all about people – Picking and developing the best and getting rid of the worst – Helping them select and use good software engineering practices – Motivating and rewarding them for • Using and improving these practices • Behaving as teams – Giving them the time and resources to do their jobs • Aggressive but realistic goals – Making and holding them accountable • Crowd control can be fun

Things you might consider trying (Other than shooting Alfie “the shot-caller”) • Development process – Iterative development • Entrance/exit – Design template – User-centered design (pair with another team) – Inspections • Heavy and lightweight – Performance model – Defect metrics & root cause analysis – Tempo control • • Project meeting Milestones Railroad Jeopardy • Outside audit (pair with another team) – Changeability/customization • Automated testing – System & module – S curves • Lots of extra-bold Starbucks – Two shots

Final