Organizational Patterns James O Coplien Bell Laboratories Naperville

Organizational Patterns James O. Coplien Bell Laboratories Naperville, Illinois, USA cope@research. bell-labs. com Ja. CC Software Developers 16 September 1999 11: 00 - 12: 30 Bell Laboratories 1

Organizational Patterns James O. Coplien Bell Laboratories Naperville, Illinois, USA cope@research. bell-labs. com IIT Colloquium Series Tuesday, February 8, 2000 M-50 Auditorium Bell Laboratories 2

Overview What are organizational patterns? Methodology Our model of organizational change Using patterns for change management The patterns themselves can be found at http: //www. bell-labs. com/cgi-user/Org. Patterns Bell Laboratories 3

What is a pattern? A solution to a problem in a context Architectural patterns ideas first published by Alexander in 1977 Look at issues of structure, not just parts Build on proven practice, not “promising theories” Adopted by the software community starting in 1990; started gaining critical mass by 1993 Today, widely used as a design tool, especially in the OO community Bell Laboratories 4

What are organizational patterns? Solutions to organizational problems in a context First appeared in the Alexander + software context at PLo. P in 1994 (Coplien, Whitenack); received with some skepticism Now, a growing body of knowledge Or, a construct from anthropology, Kroeber: » Universal patterns: transcend cultures » Systemic patterns: have a common root in an ancient culture » Total culture patterns: give a culture its identity Patterns define culture Bell Laboratories 5

Why do we care about organizational patterns? Because process-based approaches have serious limitations that hark back to the era of the industrial revolution Because experience with project management, technology transfer, etc. show software to be a primarily social activity Because of the tie to cultures, and the study of cultures is about social activity Because we should care about human comfort in addition to product quality Because they work! Bell Laboratories 6

The Pasteur Process Research Project at Bell Labs Goal: To know how to design highly productive software development organizations Premise: Organization drives process, which drives productivity Strategy: Correlate properties of organizations with order-of-magnitude productivity Technique: Patterns Bell Laboratories 7

A Theory of Organizational Change The idea of an organization as an anthropomorphic entity » Some are intelligent: good reactions, learn from mistakes » Some are stupid: react in insane ways, don’t learn Organizations can learn! Organizations can’t be taught Therefore, learning is experiential » Experience is a hard teacher Can be helped with empirical foundations and introspection Bell Laboratories 8

Empirical foundations “Pattern Mining” Finding recurring structures and practices in healthy organization To a degree, seeking the absence of these structures in dysfunctional organizations A soft science Akin to anthropology and its many branches Bell Laboratories 9

Organizational Pattern Mining Based on social network analysis with informal extensions Organizational role-play to gather social network data Dyadic and triadic sociometric data Uses CRC cards Light facilitation Half-day exercise Bell Laboratories 10

CRC Cards: Classes, Responsibilities, and Collaborators Subsystem coord. Validate MR lists Build group products Administer ENVY Resolve logical deps. Bell Laboratories Subsystem coord. Change committee Designers System test 11

Work-life Role-Play Identify project roles Study subjects play roles Development scenarios drive role-play Capture interaction & coupling on CRC cards Social Network Analysis Tools » Organization Structure Visualization » Organizational Metrics Capture Trends as Generative Patterns Bell Laboratories 12

Distilling the Patterns Sociometric analyses » Pasteur analysis tool » Analytical studies of data Catalog sociograms, sociomatrices, sociometric data The pattern must recur The pattern must solve a problem Document in pattern form Bell Laboratories 13

QPW as a Subject Fourth generation product in one generation Remarkable press reviews The most effective organization we have found » » Productivity Quality Interval Market Used to calibrate our “spectrum” Analysis done at my request I am not a QPW user Bell Laboratories 14

Organization Metrics QPW Bell Laboratories 17

Writers’ Workshops A forum to review patterns Borrowed from the creative literature community Designed to protect the dignity of the author Several basic steps: » » » » Bell Laboratories Moderator introduces author Author reads the work Author becomes a fly on the wall Someone summarizes the pattern The strengths are emphasized Suggestions for improvement Author asks for clarification Thank the author 18

Using the Patterns Let dysfunctional organizations become exposed to the patterns to see what is possible Use organizational role-play as an introspection tool Collect and process sociometric data from the role play Present the data to the organization for a second round of introspection—a “mirror” Learning and growth build on the insights Bell Laboratories 19

The Organizational Role-Play Same technique as used for pattern mining Largely self-running Role-based » Useful in most development organizations Actor-based » Useful for diagnosing specialized team problems » “Ethnological” studies Bell Laboratories 20

Facilitation Guidelines Involve everybody Don’t try solving other peoples’ problems » Other organizations must solve their own problems » Interface issues can be dealt with in another forum » Interfaces generally involve politics Don’t look for “the problem” » Problems are systemic » Solutions are generative » Let the patterns cause the “aha”s Bell Laboratories 21

More principals of organizational change Happens bottom-up, perhaps with the top-down support of a sponsor or patron Patterns are closely tied to value systems—which must be elicited bottom-up Organizational change means building a new culture: reverse anthropology » » » Bell Laboratories New language New normative behaviors New values New symbols New stories (patterns) New rituals 22

What makes organizations excel? No silver bullet Well-known productivity enablers A little luck The right balance of techniques to make a working social (and economic and technical and. . . ) system Bell Laboratories 23

Communication is Key The “Buffalo Mountain” pattern Decentralized, yet a graceful distribution Almost fully connected Anti-schmisogenetic » No splinter groups » No time-serial sub-processes » No “subroutines” Bell Laboratories 24

Communication Intensity QPW Bell Laboratories 25

Size Easy to socialize the architecture Self-directing » Correlates to Gerry Weinberg’s SEI/CMM parallel » The true hallmark of a mature organization Divide and conquer Essentially large organizations need hierarchy Bell Laboratories 26

Architect Also Implements Problem: Preserving the architectural vision through to implementation Context: An organization of developers that needs strategic technical direction. Forces: Totalitarian control is viewed by most development teams as a draconian measure. The right information must flow through the right roles. The architect is one of the few people, or may be the only person, with the big picture. Big picture knowledge is important even during implementation, especially when using iterative development. But the architect usually isn’t around during implementation. Solution: Beyond advising and communicating with developers, architects should also participate in implementation. Resulting Context: A development organization that perceives buy-in from the guiding architects, and that can directly avail itself of architectural expertise. The architect learns from implementation constraints and may modify the architecture accordingly. Rationale: Vitruvius notes: “. . . [A]rchitects who have aimed at acquiring manual skill without scholarship have never been able to reach a position of authority to correspond to their pains, while those who relied only upon theories and scholarship were obviously hunting the shadow, not the substance. But those who have a thorough knowledge of both, like men armed at all points, have the sooner attained their object and carried authority with them. ” Larry Constantine emphasizes the value of “street creds” in an architect. Bell Laboratories 27

Distribute Work Evenly Like QPW adjacency graph Sociometric analysis: low graph centrality Can be remedied by load leveling, staffing, splitting roles, combining roles Bell Laboratories 30

Organization Metrics for Distribute Work Evenly QPW Bell Laboratories 31

Personnel Issues Personal excellence and integrity Highly specialized Balance of teamwork (communication) and individuation (specialization, personal signoffs) How does one assemble such a group? 1. Magic: A secret that only Philippe knows 2. Outside interests? (Gabriel) 3. Self-selecting Team (another pattern) Bell Laboratories 32

Engage Quality Assurance QA is tightly coupled in the social network Large beta program–customer engagement Developers reserve early verification to themselves QA first-in, last-out Bell Laboratories 33

Reward Excellence A sense of contribution » Reinforced in project signoffs » Reinforced in reward system Financial incentives » No first-hand insights into QPW » Controversial, but claims demonstrable results Two cultures with extravagant rewards: 1. The U. S. West Coast 2. Financial trading Overcomes fear of change Bell Laboratories 34

Process? The ability to introspect well The ability to self-manage Organizational structure over task sequencing Effective meetings, and lots of them Bell Laboratories 35

Conclusion Organizational patterns are: » Patterns that capture the “architecture” of organizations » In anthropology, the fabric of social structure Role-playing data gathering and introspection Organizational change is generative Organizations can use patterns as exemplars for improvement A rich body of literature exists and is growing Bell Laboratories 36