A Plan-Driven Process How and Why to Fake

A Plan-Driven Process —How and Why to Fake it Philippe Kruchten @ USC, on March 18 th, 2004

Presenter Philippe Kruchten, Ph. D. , P. Eng. Dept. of Elec. & Comp. Eng. University of British Columbia 2356 Main Mall Vancouver BC V 6 T 1 Z 4 pbk@ece. ubc. ca kruchten@ieee. org 2

Fitting in a Plan-Driven Environment § System engineering “waterfall” approach § Necessity to comply to 4 IEEE Standards 4 ISO 12207 4 some company or industry standard § Intent to certify or remain certified at some CMM level 3

Fake it. § Do the “right thing”, e. g. , use appropriate agile practices § Present enough of an external façade to the outside world to meet the requirements § Present management artifacts ‘as if’ you were doing things in a plan driven fashion. § Tailor the external demands to the maximum extremity feasible. § in particular, tailor: 4 Document templates 4 Level of “precision” 4 Frequency of updates (no early “freezing”) 4 Adjust the jargon § Produce 4 High level plan 4 Plans by iteration (or cycle, or sprint, …) § Inform and set the expectations right for the external world 4

Example: IEEE Standards § Tailored software development plan 1058: 1998 § Section 6. 1: introduce your agile process § Section 7. 4: Test-first § Section 7. 5: pair programming § Section 7. 6: introduce the scrum § Section 7. 8: introduce retrospective § etc… § The standards do not tell you HOW you have to do the work. The plans can be as flexible as you want or need. § But set up the right expectations. 5

Scaling UP the Agile processes? § § § How large can I expand the team and still be agile? How long… ? How many attributes of agility can I drop and still be agile? Which practices can I scale up? Which practices I cannot use on large projects? § Or should we : 4 Understand the ideal conditions of agility 4 Scale down projects in a way that establishes these ideal conditions of agility If the mountain will not go to Mahomet, let Mahomet go to the mountain. (Proverb) 6

Agile Process “Sweet Spot” § High bandwidth communication 4 Small team (10 -15), Collocated, Face to face (as opposed to document based) § Customer on site 4 a customer representative, domain literate and empowered to make decisions § Short lifecycle (weeks or months, not years) § Powerful development environment: 4 Rapid development cycle, though automation 4 Continuous integration (builds) 4 Automated test § Iterative 4 Accommodating change, refactoring § Business applications (rather than technical, embedded, real-time) § New development (rather than maintenance) 4 Availability of test regression suite ? 7

The “Bitter Spot” (? ) § § § Large team Distributed team No empowered customer on site Long development cycle Inefficient development environment (low level of automation) Different cultures § Low communication bandwidth, reliance on documents § Waterfall § Aversion to change § Try to follow a fixed plan 8

Large Project § Large projects are not going away § Tend to : • Heavy early planning (and desperately try to follow) • Relying solely on written artifacts (workproducts), − including in dialog with customer − And in following a defined process § Waterfall approach still dominant paradigm 4 Easy on management 4“comforting”, false sense of rationality, of determinism 4 Similar to other engineering disciplines 9

An Exemplar Large Project § Avoid the marginal conditions (go from 12 to 18) § Do not compare a bad large project with a good small and agile project § § § § § 10 200 people or more Distributed Different organizations/companies 2 ½ years before first delivery Iterative lifecycle Good, skilled people Access to customer Access to efficient programming environment New system, architecture not set

The Ideal Large Project—Agile? § High bandwidth communication 4 How to achieve beyond 15? § Customer 4 How to make it available to 150 people § Focus on results 4 How to focus on something getting out in 2 years? § Some level of planning and explicit documentation will be necessary § Break-down the 200 people in: 410 teams of 15 4 Establish the conditions of optimal agility 4 Use the 50 others to fill the space in between, act as the glue • Communication • Customer role 11

Setting up the Large Project— Inception+Elaboration Inception Elaboration Management team Initial team Architecture team Prototyping team time 12 Agile teams LCA Milestone Superstructure teams

Setting up the Large Project— Construction+Transition § Create 2 types of team 4 Feature teams • User oriented Elaboration Management team Construction & Transition Management team Architecture team 4 Infrastructure teams • Provider of common services to feature teams Architecture team Feature team 11 Feature team Prototyping team Infrastructure Feature team 3 team A Infrastructure team B Feature team 2 integration team 13

Increased level of ceremony Inception § Feature teams § Infrastructure teams § “glue” Elaboration Management team Construction & Transition Management team Architecture team § More planning § More documentation § Less agility 14 Feature team 11 Feature team Prototyping team Initial team Architecture team Infrastructure Feature team 3 team A Infrastructure team B Feature team 2 integration team

Where are we? § Feature teams and infrastructure teams: 4 Organized as “agile projects” § Added a project superstructure to reproduce ideal conditions 4 Architecture team 4 Integration team 4 Project management team § Increase in level of ceremony § More planning involved than with a single agile project 4 Keep feedback loop from iterations, react to change 4 Not a plan first and then execute to plan 15

Issues with a Federation of Teams § Dependencies between teams 4 Teams are customer for each other 4 May need “brokering” by architecture team to avoid too much one-to-one communication § Stovepipes 4 Teams as small fiefdoms, building and empire and reinventing the wheel 4 Architecture team participate in design reviews, planning sessions § Imbalance: staff and skills 4 Identified by architects, or raised up by team lead § Communication breakage § Too much staff too early § Long cycle: lack of feedback loop? § Rotation of staff, and circulation of architect 4“moral” equivalent of pair programming 4 Spreading the culture 16

Dual Rhythm Example § Long cycle 4 Lack of feed back 4 Team lose track of ultimate goal 4 Need intermediate, tangible goals § Dual ‘beat’ Weekly/biweekly scrum System increment beat: 6 months Daily scrum Team increment: 1 month 17

Organizing Project Documentation § Progressively introducing more explicit documentation 4 More efficient use of staff (e. g. , architecture team) 4 Greater visibility § Software (/System) Architecture § Project level backlog, Risks and issues § Requirements: “vision” and key use cases § Key interfaces § Recommended practices, project standards (the actual concrete process) § Reusable elements § Overall project plan 18

Iterations and Demonstrable Progress Uncertainty in Stakeholder Satisfaction Space Initial State Actual Path and precision of artifacts Emergence of Requirements, Architecture, Design, Plans, and Product 19 Initial Plan

Examples § Ship System 2000 (FS 2000) 4 Central software architecture team 4 Iterative development 4 Architecture => blueprint for organization § Canadian Air Traffic Control system (CAATS) 4 Went further than SS 200 4 Customer on-site: large team of actual users of ATC 4 Start prototyping with a small team + arch. team 4 Architecture team “split” at end LCA milestone • Played role of facilitator, communication vertex 4 Integration team 4 Progressive introduction of explicit documentation 20

Examples (cont. ) § Rational Software’s Product Group 4 Team of teams (700 total) 4 Distributed – 7 sites around the globe 4 Varied culture (acquisition) 4 Centralized: • Architecture • Product definition, with “local rep” • Integration (installer, licensing, etc. ) 4 Dual rhythm • Major beat: 6 months • Internal beat: monthly 21

A different kind of project management support § Single prioritization scheme § “Virtual” SCRUM § Bring 4 Requirements (user stories, use cases, etc. ) 4 Problems (bugs reports, etc. ) 4 Issues (tactical things) 4 Tasks (and other time consumers) 4 Risks 4 Business constraints 4 Resources 4 Process under one single tool to support project managers and practitioners § Visibility of results, progress, metrics to all 22

Different project management support § Scheduling and WBS § Earned value Replaced by: § Tactical backlog support § Virtual Scrum Some new players in this space: § Osellus, Toronto: IRIS § Ensemble Systems, Richmond BC: Task. Workbench § F 4 Tech, Boulder, CO: ? ? ? § Big, Seattle area company…. 23

Summary § Large projects set up as as a federation of agile teams § Two levels of: 4 Communication 4 Organization 4 Project ‘beat’ § Software architecture serves as the blueprint for the team structure 4“Emerges” during elaboration with a small team (or two) § Then Architecture + Integration teams add communication “glue” 4 Serve as customers 4 Facilitate communication 4 Balance skills, load 4 Foster reuse 4 Assemble final product § Gradual introduction of explicit documents 4 Where they support effective communication, reduce ambiguity, spread common culture 24

References and further reading 25 1. P. B. Kruchten, The Rational Unified Process: An Introduction, 2 ed. Boston: Addison-Wesley, 2000. 2. B. W. Boehm, D. Port, M. Abi-Antoun, and A. Egyed, "Guidelines for the Life Cycle Objectives (LCO) and the Life Cycle Architecture (LCA) deliverables for Model-Based Architecting and Software Engineering (MBASE), " USC, Los Angeles, USC Technical Report USC-CSE-98 -519, 1999. 3. K. Schwaber and M. Beedle, Agile Software Development with SCRUM. Upper Saddle River, NJ: Prentice-Hall, 2002. 4. L. Brownsword and P. Clements, "A Case Study in Successful Product Line Development, " Software Engineering Institute, CMU/SEI-96 -TR-035, 1996. 5. T. Paine, P. Kruchten, and K. Toth, "Modernizing Air Traffic Control through Modern Software Methods, " in 38 th Annual Air Traffic Control Association Conference. Nashville, Tenn. : ATCA, 1993. 6. P. Kruchten, "Iterative Software Development for Large Ada Programs, " in Proc. Ada-Europe conference, Montreux, Switzerland, vol. LNCS 1088, A. Strohmeier, ed. : Springer-Verlag, 1996, pp. 101 -110. 7. P. Kruchten and C. J. Thompson, "An Object-Oriented, Distributed Architecture for Large Scale Systems, " in Proc. of Tri-Ada'94, Baltimore, November 1994: ACM. 8. P. Kruchten, "The Software Architect, and the Software Architecture Team, " in Software Architecture, P. Donohue, ed. Boston: Kluwer Academic Publishers, 1999, pp. 565 -583.

Thank you 26