Professional Software Development Shorter Schedules Higher Quality Products

Professional Software Development Shorter Schedules, Higher Quality Products, More Successful Projects, Enhanced Careers by Steve Mc. Connell Matt Sawka

About the Author Steve Mc. Connell is has a bachelor's degree Whitman College and a master’s degree in Software Engineering from Seattle University. ► He has been working in the Software industry since 1984, including being a consultant for Microsoft. ► From 1998 -2002, he was Editor and Chief of IEEE Software magazine. ► He is currently on a panel of experts that advises the Software Engineering Body of Knowledge project and is a Vice Chair of the IEEE Professional Practices Committee. ► Currently the CEO and Chief Software Engineer of Contrux Software. ► ► ► http: //www. stevemcconnell. com/ http: //www. contrux. com/

Purpose ► ► ► ► ► What is software engineering? How does software engineering relate to computer science? Why isn’t regular computer programming good enough? Why do we need a profession of software engineering? Why is engineering the best model for a software development profession? In what ways do effective practices vary from project to project (or company to company), and in what ways are they usually the same? What can organizations do to support a professional approach to software development? What can individual software developers do to become full-fledged professionals? What can the software industry as a whole do to create a true profession of software engineering?

Organization The Software Tarpit Wrestling with Dinosaurs Fool’s Gold Cargo Cult Software Engineering Body of Knowledge Novum Organum Individual Professionalism Orphans Preferred Raising Your Software Consciousness Building the Community Programmer Writing Organizational Professionalism Software Gold Rushes Business Case for Better Software Practices Ptolemaic Reasoning Quantifying Personnel Factors Construx’s Professional Development Program Industry Professionalism Engineering a Profession Hard Knocks Stinking Badges The Professional’s Code Alchemy

Part One The Software Tar Pit

Chapter 1: Wrestling with Dinosaurs “He that will not apply new remedies must expect new evils, for time is the greatest innovator. ” –Francis Bacon

Wrestling with Dinosaurs ► “No scene from prehistory is quite so vivid as that of the moral struggles of great beasts in the tar pits. In the mind’s eye one sees dinosaurs, mammoths, and sabertoothed tigers struggling against the grip of the tar. The fiercer the struggle, the more entangling the tar, and no beast is so strong and so skillfully but that he ultimately sinks. Large-system programming has over the past decade been such a tar pit, and many great and powerful beasts have thrashed violently in it Most have emerged with running systems – few have met goals, schedules, and budgets Large and small, massive or wiry, team after team has become entangled in the tar No one thing seems to cause the difficulty – any particular paw can be pulled away. But the accumulation of simultaneous and interacting factors brings slower and slower motion. Everyone seems to have been surprised by the stickiness of the problem, and it is hard to discern the nature of it But we must try to understand it if we are to solve it. -The Mythical Man-Month (p. 3), Fred Brooks. 1975

Wrestling with Dinosaurs ► Has anything changed? § ~75% of all medium-sized projects and 90% of or more of large projects are subject to excessive schedule pressure. § Overtime is the standard. ► “In many companies, programmers faced with deadlines have been known to spend nights in the offices. ” – Fortune, 1967 § Windows NT was projected to take 1500 staff-years to complete. ► OS/360 took more than 3 times this amount in 1966. § The advent of web-development poses the question is delivering software quickly better than delivering quality software? Mc. Connell argues that this is not a new phenomena.

Chapter 2: Fool’s Gold “Hope is a good breakfast, but it is a bad supper. ” –Francis Bacon

Fool’s Gold ► During the California Gold Rush of the late 1800 s, many were deceived by fool’s gold – iron pyrite, which has the same appearance as Gold. ► The last fifty years in software development has produced similar results. Software Engineers seeking solid processes and standards have mostly found fool’s gold, software practices that appear useless on first glance, but are actually flaky, brittle, and virtually valueless.

Fool’s Gold ► Problem: Looking back further in history, how could we build one of the ancient pyramids from ancient Egypt? § The stone blocks need to be moved 10, 000 meters from the river to their final resting place. § You have 100 days to move the blocks. § You have 20 people to move the blocks. § You are allowed to use any method you’d like. ► On average, you must move the blocks 100 meters a day (or have a method that will speed up the process later on). ► How would you accomplish this?

Fool’s Gold ► Option 1: § Immediately start pushing the blocks with brute force. § Result: You move the block 10 meters a day and fall 90 meters/day behind.

Fool’s Gold ► Option 2: § Analyze the problem. You decide to cut down several trees and use them as rollers to move the block. Create a smooth, level roadway to push forward. § Result: Although there is an initial investment to find the trees, it still will pay off.

Fool’s Gold ► Option 3: § Implement option 2, but balance the pushers with pullers and add log-movers, so that a group is always moving extra logs in front of the block. § Result: An improvement on option 2, the block will be continuously moving.

Fool’s Gold ► How does software development relate to ancient Egyptian pyramids? § Change the pyramid block to a software development project. ► You have 100 days to complete a project. This means you either have to complete 1/100 of the source code each day or you need to schedule some parts taking less time than others. § Avoid “last-minute” syndrome (Option 1) ► The development team has little concern near the beginning of the project, but has a frenzied push at the end of the development cycle.

Fool’s Gold ► How does software development relate to ancient Egyptian pyramids? § Also avoid “code-and-fix” development (Option 2) ► Put a brute force of programmers on the project without properly planning or design of the software. The project team will show initial progress but will be unable to finish strong. Most effort will go into defects.

Fool’s Gold ► The Silver Bullet Syndrome § An elephant could be capture to pull block. However, after it is captured, it tramples 2 workers and runs off. The managers then think they should’ve spent time learning to handle the elephant.

Fool’s Gold As project planning matures, the amount of effort spent on later stages should be manageable. ► Despite the downfalls, “code-and-fix” is still heavily used today for two reasons: ► § It provides initial signs of progress § It requires no training

Fool’s Gold ► Focusing on Quality § If an organization can remove ~95% of their defects before a release, they can minimize the amount of effort spent on correcting them later.

Fool’s Gold ► Software isn’t “soft”. ► Let’s suppose you are designing a system to print a set of 5 reports, and eventually 10 reports. § How the reports can be “soft”: ► Is ten an upper limit on thee number of reports? ► Will the future reports be similar to the initial five reports? ► Will all of the reports always be printed? ► Will they always be printed in the same order? ► To what extent will the user be able to customize the reports? ► Will users be allowed to define their own reports? ► Will the reports be customizable and definable on the fly? ► Will the repots be translate to other languages?

Fool’s Gold § How the reports can be “hard”: ► Defining more than 10 reports. ► Defining a new report that is different form the initial set of reports. ► Printing a subset of the reports. ► Printing the reports in a user-defined order. ► Allowing the user to customize reports. ► Translating the repots to another language that users the Latin alphabet. ► Translating the reports to a another language that uses a non-Latin alphabet or reads right-to-left. ► Bottom Line: Flexibility costs money. Limiting flexibility can save money in the short term, but can incur higher costs later on the development life-cycle.

Chapter 3: Cargo Cult Software Engineering “In the South seas there is a cargo cult of people. During the war they saw airplanes with lots of good materials, and they want the same thing to happen now. So they’ve arranged to make things like runaways, to put fires along the sides of runways to make a wooden hut for a man to sit in, with two wooden pieces on his head for headphones and bars of bamboo sticking out like antennas – he’s the controller – and they wait for the airplanes to land They’re doing everything right. The form is perfect. It looks exactly the way it looked before. But it doesn’t work. No airplanes land. So I call these things cargo cult science, because they follow all the apparent precepts and forms of scientific investigation, but they’re missing something essential, because the planes don’t land. ” –Richard Feynman

Cargo Cult Software Engineering ► Process-oriented vs. Commitment-oriented Development styles § Process-oriented ► Relies on a carefully defined process, planning, scheduling, and directly application of Software Engineering best-practices. ► This succeeds because the organization is constantly improving on their best practices. § Commitment-oriented ► Also known as “hero-oriented” or “individual empowerment”, this style relies on hiring the best people and asking them for total commitment to a project. They work with completely autonomy something work 60 -100 hours a week until a project is completed. ► This style succeeds because it utilizes individual motivation.

Cargo Cult Software Engineering ► Imposters: § Process-oriented imposter (Bureaucratic) ► Observes successful companies using best-practices (such as NASA’s Software Engineering Laboratory), see that they have many meetings and documents and emulate the deliverables only. § Commitment-oriented imposter ► Observe successful companies like Microsoft, and emphasize the long hours and large compensation packages, not the fact that the employees of Microsoft love to create software. ► Cargo-cult engineering is simply using a development style “just because” or “because the company requires it”, rather than using the style advantageously.

Chapter 4: Software Engineering, Not Computer Science “A scientist builds in order to learn; an engineer learns in order to build. ” –Fred Brooks

SE not CS ► Like other engineering disciplines, Software Engineering can be tailored for several objectives: § § § § ► Minimal defects Maximum user satisfaction Minimal response time Good maintainability Good extendibility High robustness High correctness. § § § Short schedule Predictable delivery date Low cost Small team size Flexibility to make mid-project feature-set changes. Unlike other disciplines in which risk relies on the physical materials, Software Engineering carries risk in optimizing the project goals.

SE not CS ► What is the best way to think of software development? Is it a science? Art? Craft? Something else? § ~40% of developers today have degrees in Computer Science. § Scientists learn what is true, how to test hypotheses, and how to extend their knowledge. § Engineers learn what is true, useful, and how to apply their knowledge to solve a problem. ► Is Software Engineering just a buzzword? § Some object to Software development being classified as engineering because the commercial market doesn’t allow for the careful, time-consuming engineering process to be completed.

Chapter 5: Body of Knowledge “Truth will sooner come out of error than from confusion. ” –Francis Bacon

Body of Knowledge ► To be an expert in a field, a person needs to know around 50, 000 pieces of information. ► But with software engineering evolving, how can anyone know enough to be considered an expert? ► Essence vs. Accident § No Silver Bullet – Essence and Accidents of Software Engineering – Fred Brooks, 1987. § Essence – properties that something must have. ► Wheels on a car ► Software engineering: Specification, Design, and Verification. § Accident – optional properties. ► Air-Conditioning ► Software Engineering: Coding and testing.

Body of Knowledge ► Computer programs are complex. § At the center, the goal is to define underlying realworld concepts and debugging that understanding. ► Software must be flexible and have the ability to change. § If a program is successful, more people will use it and it will be adapted to be used outside of its original scope. ► Software is “invisible. ” § It’s not possible to create a 2 -d or 3 -d geometric model of the system.

Body of Knowledge ► In 1968, NATO held its first conference on Software Engineering. ► Mc. Connell estimates that in 1968, the average half-life of knowledge was about 10 years, with only about 20% of that knowledge at the Stable Core.

Body of Knowledge ► By 2003, Mc. Connell estimates that the average half-life of knowledge was about 30 years, with about 50% of that knowledge at the Stable Core.

Body of Knowledge ► How can we categorize the Software Engineering “body of knowledge” (SWEBOK)?

Body of Knowledge ► What sources contribute to the SWEBOK?

Body of Knowledge ► What information is contained within SWEBOK? ► SWEBOK Knowledge Areas § Software Requirements ► The discovery, documentation, and analysis of the functions to be implemented in software. § Software Design ► Definition of the basic structure of the system at the architectural and detailed levels, division into modules, definition of interfaces for modules, and choice of algorithms within modules. § Software Construction ► Implementation of the software including detailed design, coding, debugging, unit testing, technical reviews, and performance optimization. This area overlaps somewhat with Software Design and Software Testing.

Body of Knowledge ► SWEBOK Knowledge Areas § Software Testing ► All activities associated with executing software to detect defects and evaluate features. This includes planning, testcase design as well as the tests themselves: development, unit, component, integration, system, regression, stress, and acceptance. § Software Maintenance ► Revision and enhancement of existing software, related documentation, and tests. § Software Configuration Management ► Identification, documentation, change control of all deliverables generated on a project (source code, content, requirements, etc…). § Software Quality ► All activities associated with providing confidence that a software item conforms or will conform to technical requirements.

Body of Knowledge ► SWEBOK Knowledge Areas § Software Engineering Management Plan ►Planning, tracking and controlling of software projects, work, and organizations. § Software Engineering Tools and Methods ►Tolls and methodologies support (CASE tools, reusable libraries, formal methods and practices, etc…). § Software Engineering Process ►Activities related to improving development quality, timeliness, productivity, and other characteristics. § http: //www. swebok. org/

Chapter 6: Novum Organum “A prudent question is one-half of wisdom. ” –Francis Bacon

Novum Organum ► In the 1620 s, Francis Bacon published Instauratio Magna which attempted to redefine scientific inquiry. Within this work is an essay, Novum Organum which challenges his colleagues to focus on scientific methodologies rather than deductive reasoning when studying the world. His view of the scientific method had three steps: § Purge your mind of prejudices (superstition) § Collect observations and experiences systematically. § Stop, survey what you’ve seen, and draw an initial conclusion.

Novum Organum ► What does it mean to have a Software Engineering “profession”? § According to the Code of Federal Regulations, a profession has: ► A requirement for extensive learning and training ► A code of ethics imposing standards higher than those normally tolerated in the marketplace. ► A disciplinary system for professionals who breach the code ► A primary emphasis on social responsibility over strictly individual gain, and a corresponding duty of its members to behave as members f a disciplined and honorable profession. ► A prerequisite of a license priori to admission to practice.

Novum Organum ► Is Software Engineering a profession? § The Software Engineering Institute (SEI) has identified 8 elements of a mature profession.

Novum Organum ► Elements to a mature profession: § Initial Professional Education ► Completing a university program in their field. § Accreditation ► The university program is accredited by an oversight body that determines if the program provides adequate education. Accreditation Board for Engineering and Technology (ABET) oversees American engineering programs. § Skills Development ► Education is not enough to develop full professional capabilities, some type of further experience is needed to perform the job individually with a satisfactory level of competence.

Novum Organum ► Elements to a mature profession: § Certification ► A professional is requirements to pass one or more exams that ensures they have a minimum level of knowledge. § Licensing ► Similar to certification, this is a mandatory exam administered by a overrunning authority. § Professional Development ► Ongoing professional education to maintain or improve a worker’s knowledge and skills.

Novum Organum ► Elements to a mature profession: § Professional Societies ► A Community of like-minded individuals who put their professional standards above their self-interest. § Code of Ethics ► Ensures that a profession’s practitioners behave responsibly. § *Organizational Certification ► Organizations must also be certified.

Novum Organum ► Maturity Levels for the elements § Nonexistence ► The element simply doesn’t exist. § Ad Hoc ► The element exists, but only in isolated instances § Established ► The element exists and is clearly identifiable. § Maturing ► The element has existed for many years and is being maintained and improved.

Novum Organum ► How does Software Engineering rank? § § § § Initial Professional Education – Ad Hoc to Established Accreditation – Established Skills Development – Established Licensing – Ad Hoc Professional Development – Ad Hoc Professional Societies – Established Code of Ethics – Established Organizational Certification - Established

The Software Tar Pit Summary Wrestling with Dinosaurs § Has anything changed? ► Fool’s Gold § Building the Egyptian Pyramids § Code-And-Fix, Last-Minute, Silver-Bullet § Software isn’t “soft. ” ► Cargo Cult Software Engineering § Process-Oriented vs. Commitment-oriented ► Software Engineering Not Computer Science ► Body of Knowledge § SWEBOK ► Novum Organum § What is a profession? ►

Part Two Individual Professionalism

Chapter 7: Orphans Preferred “Wanted: Young, skinny, wirey fellows not over 18. Must be expert riders willing to risk death daily. Orphans preferred. Wages $25/week. ” -Pony Express, 1860 “We realize the skills, intellect, and personality we seek are rare, and our compensation plan reflects that. In return, we expect TOTAL AND ABSOLUTE COMMITMENT to project success – overcoming all obstacles to create applications on time and within budget. ” –Jobs Rated Almanac, 1995 for an SE posting.

Orphans Preferred ► “The stereotypical programmer is a shy young man who works in a darkened room, intensely concentrating on magical incantations that make the computer do his bidding. He can concentrate for 12 to 16 hours at a time, often working through the night to make his artistic vision a reality. He subsists on pizza and Twinkies. When interrupted, the programming creature responds violently, hurling strings of cryptic acronyms at his interrupter – “TCP/IP, RPC, RCS, ACM, and IEEE!” he yells. The programmer breaks his intense concentration only to attend Star Trek conventions and watch Monty Python reruns. He is sometimes regarded as an indispensable genius, sometimes as an eccentric artist. Vital information is stored in his head and his head alone. He is secure in his job, knowing that, valuable as he is, precious few people compete for his job. ”

Orphans Preferred ► Meyers-Briggs Type Indicator § Extroversion (E) vs. Introversion (I) ► Extroverts are focused on the outside world, people. Introverts focus on the world of ideas. § Sensing (S) vs. Intuition (N) ► How a person deals with decision-making data. Sensing persons focus on facts, concrete data and experience. Intuitive people look for possibilities and focus on conceptual theories. § Thinking (T) vs. Feeling (F) ► How a person makes a decision. The thinkers make objective, analytic decisions, where as feelers rely on emotions and feelings. § Perceiving (P) vs. Judging (J) ► The perceiving person is flexible and likes open-ended possibilities, where as the judging person prefers control and order.

Orphans Preferred ► Meyers-Briggs and Software Developers § Most Software Developers (25 -40%) are ISTJ. ► 50 -75% of programmers are introverts, compared to 25% in the general population. § About 60% of software developers have at least a Bachelors, compared to 30% of the general population. ► 80 -90% of programmers are Thinking (T) compared to 50% of the general population. ► There’s a 50 -50% split of programmers between Sensing (S) and Intuition (N).

Orphans Preferred ► Meyers-Briggs and Designers § Great Designers… ► can general move in-between categories. ► have a mastery of common tools. ► aren’t afraid of complexity. ► seek out constructive criticism. ► have experienced failed projects. ► are not afraid of the brute-force approach. ► must be creative. ► have a restless desire to create.

Orphans Preferred ► Total and Absolute Commitment § Although working 12 to 16 hours may seem extreme, it’s not out of the realm of possibility. Many Microsoft programmers working this or more during the development of Windows NT. ► “Work pervades their existence. Friends fade into the background. The ties of marriage fray or rip apart. Children are neglected or deferred. Hobbies wither. Computer code comes to mean everything. If private dreams are nursed at al, it is only to ease the pain of creating NT. ” – P. Zachary § Programmers tend to show a loyalty to the project, even if their loyalty to the company is waning. § Some programmers aim to be “hero” programmers, who take on mountains of work and hours.

Orphans Preferred ► Demographics § The average programmer age peaks between 30 -35 years old (which is about 10 years younger than most professions). § 72% of computer and information science BSs and 83% of Ph. D’s were men. § Only 17% of high-schoolers taking the AP Computer Science test were female. This is the lowest % of all AP tests. Highest level of % of Education developers High school or less 11. 8% Some college, no degree 17. 2% Associate’s degree 11. 0% Bachelor’s degree 47. 4% Graduate degree 12. 8%

Orphans Preferred ► Job Prospects Job breakdown for software workers Current Software prsnl in US Computer and information scientists, research 28, 000 Computer programmers 585, 000 Computer software engineers, applications 380, 000 Computer software engineers, systems software 317, 000 Computer systems analysts 431, 000 Network systems and data comm. analysts 119, 000 Other computer specialists 203, 000 Total 2, 063, 000

Chapter 8: Raising Your Software Consciousness “If a man will begin with certainties, he shall end in doubts; but if he will be content to begin with doubts, he shall end in certainties” -Francis Bacon

Raising Your Software Consciousness ► In the 1970 s, Charles Reich published, The Greening of America, which identified 3 types of awareness § Consciousness I (Con I): Pioneer mentality ► Great emphasis on independence and self-satisfaction § Consciousness II (Con II): Gray flannel suit mentality ► The corporate man. Knowing how to get along with other and playing by the rules § Consciousness III (Con III): Enlightened Independence ► Operates on the basis of principles, with little regard for the rules of Con II and without the selfishness of Con I.

Raising Your Software Consciousness ► In the 1970 s, Charles Reich published, The Greening of America, which identified 3 types of awareness § Consciousness I (Con I): Pioneer mentality ► Great emphasis on independence and self-satisfaction § Consciousness II (Con II): Gray flannel suit mentality ► The corporate man. Knowing how to get along with other and playing by the rules § Consciousness III (Con III): Enlightened Independence ► Operates on the basis of principles, with little regard for the rules of Con II and without the selfishness of Con I.

Raising Your Software Consciousness ► Consciousness and Software Development § Con I – Self Reliance ► Software developers who are “Lone Rangers”. They have little tolerance for other ideas. ► Little training is needed. This approach works adequately in small projects. § Con II – Rules ► Rules allow programmers to work with others. § Con III – Principles ► Programmers understand that the rules from Con II are based on principle. Programmers focus on the underlying effective of their actions on software development.

Chapter 9: Building the Community “If any human being earnestly desires to push on the new discoveries instead of just retaining and using the old; to win victories over Nature as a worker rather than over hostile critics as a disputant; to attain, in fact, clear and demonstrative knowledge instead of attractive and probable theory; we invite him as a true son of Science to join our ranks. ” -Francis Bacon

Building the Community ► It’s important to remember that we’re not simply lone-ranger programmers, or even programmers for one company, but rather a community of professionals. § IEEE § ACM ► These organizations can provide solutions to common problems or articles to further your understanding of the field.

Chapter 10: Architects and Carpenters “Engineers produce plans Builders implement the plans to produce a product. ” -Terri Maginnis

Architects and Carpenters ► As Software Engineering continues to develop, a wider range of abilities will begin to distinguish itself. Average software developers Highly skilled software developers Unlicensed software engineers/certified software technologists (coders). § Professional Software Engineers § § § The average person who earns a professional degree earns 50% more than someone without. ► The average person who earns a master’s degree will earn 25% more than someone without. ►

Architects and Carpenters ► Job Specialization § The Surgical Team, proposed by Fred Brooks in The Mythical Man. Month

Architects and Carpenters ► Job specializations today § Technology ► Software Technologist § Knowledge of specific technologies (Microsoft, Novell, Oracle, Apple) § Software Engineering ► Software Engineers § Architecture, configuration control, cost estimation, customer support, database administration, education and training, function point counting, human factors, information systems, integration, maintenance and enhancement, measurement, network, package acquisition, performance, planning, process improvement, quality assurance, requirements, reusability, standards, systems software support, technical writing, testing, tools development.

Architects and Carpenters ► Team Specializations § § § Construction lead Design lead Planning and tracking lead Project business manager Quality assurance lead Requirements lead

Chapter 11: Programmer Writing “Read not to contradict and confute, nor to believe and tae for granted, nor to find talk and discourse, but to weigh and consider. ” -Francis Bacon

Programmer Writing “The gap between the best software engineering practice and the average practice is very wide – perhaps wider than in any other engineering discipline. A tool that disseminates good practice would be important. ” – No Silver Bullet, Fred Brooks ► According to Mc. Connell, there are six types of authors: ► § § § ► Recent retirees University professors Seminar instructors Consultants Think-tank developers Developers working on production software. Who should be writing our documentation?

Programmer Writing ► “In this distribution of functions, the scholar is the delegated intellect. In the right state, he is, Man Thinking. In the degenerate state, when the victim of society, he tends to become a mere thinker, or, still worse, the parrot of other men's thinking. …I learn immediately from any speaker how much he has already lived, through the poverty or the splendor of his speech. Life lies behind us as the quarry from whence we get tiles and copestones for the masonry of today. This is the way to learn grammar. Colleges and books only copy the language which the field and the work-yard made. “ -Excepts from ’The American Scholar’ by Ralph Waldo Emerson (1837), http: //www. emersoncentral. com/amscholar. htm ► Is there a disconnect between academia and the work-force? Are we still thinking for ourselves? Or merely “thinking” of what others have thought through?

Programmer Writing ► “James Fenimore Cooper syndrome” § In The Deerslayer, Cooper writes that 6 Indians climbed onto sapling to board a scow coming downstream. § Mark Twain described this as a fantasy situation. Because of his knowledge of riverboat piloting, he knew this situation was not plausible with the sapling or with the dimensions of the scow. § This syndrome represents a practitioner calling into the question the writings of a scholar. ► Does the Software Engineering literature suffer from James Fenimore Cooper Syndrome?

Individual Professionalism ► Orphans Preferred § Meyers Briggs and Software Engineering demographics ► Raising your Software Consciousness § The Greening of America Building the Community § IEEE and ACM ► Architects and Carpenters ► § Specializations ► Programmer Writing § ‘The American Scholar’ § James Fenimore Cooper Syndrome

Part Three Organizational Professionalism

Chapter 12: Software Gold Rushes “Prosperity doth best discover vice, but adversity doth best discover virtue. ” -Francis Bacon “The root of all superstition is that men observe when a thing hits, but not when it misses. ” -Francis Bacon

Software Gold Rushes In 1848, gold was discovered in riverbeds in California. Many self-proclaimed entrepreneurs set out to make their fortune. This made up the California Gold Rush. But, by mid -1849, the majority of the easily found gold was collected. Many miners would spent hours a day digging through freezing water looking for any traces of the metal. By the 1850 s, most miners had joined corporations to continue their hunts. ► Software development experienced a similar phenomena. There a few success stories (Bill Gates and Paul Allen of Microsoft; Steve Jobs and Steve Wozniak of Apple, Bob Frankston and Dan Bricklin of Visi. Calc), but many failures as well. ►

Software Gold Rushes Much of the “software gold rush” is characterized by highrisk projects, long hours, hacking, informal or no processes, little-to-no documentation, and very little quality assurance. ► Post-gold rush development has en emphasis on lower-risk, high capital projects, larger teams, formal processes, and general standards. There isn’t an effort to rush projects out the door, but rather do thorough testing. ► Post-gold rush consumers are generally more demanding on the products that they are looking to buy. ► Many companies that survive a gold-rush would not be able to survive a second. ►

Chapter 13: Business Case for Better Software Practices “When you can measure what you are speaking about, and express it in numbers, you know something about it; but when you cannot measure it, when you cannot express it in numbers, your knowledge is of a meager and unsatisfactory kind. ” -Lord Kelvin, 1893

Business Case for Better Software Practices ► Development practices pay off § In 1994 James Herbsleb prepared a study of 13 organizations’ “business value” (similar to Return On Investment). ► In 1995, systemic improvements increased the ROI anywhere from 500 -900%. § In 1997, Rini van Solingen found an increase ranging from 700 -1900%. § In 2000, Caspers Jones found that the ROI could easily be in the double digits (over 1000%). § In 2001, Watts Humphrey found an ROI increase of 500%.

Business Case for Better Software Practices ► Current organizational effectiveness § A study funded by the SEI found that those organizations who implemented a change to their development practice found an average of 35% gain in productivity, 19% schedule reduction, and post-release defects were reduced by 39%. § The same study found that for the best-case scenarios, 58% productivity could be gained over 4 years, a compounded gain of 500% was achieved, a 23%/year schedule reduction (with a 91% reduction over 6 years), and postrelease defects were reduced by 99%.

Business Case for Better Software Practices Organization Results BDN International ROI 300% Boeing Information System Estimates within ~20%, $5. 5 million saved in 1 year, ROI 775% Computer Sciences Corporation 65% reduction in error rate General Dynamics Decision Systems 70% reduction in rework; 94% defect rate reduction (drr), 2. 9*productivity gain Harris ISD DPL 90% drr, 2. 5*productivity gain, ROI 900% Hughes $2 million reduction in costs, ROI 500% IBM Toronto 90% drr, 80% reduction in rework Motorola GED 2 -3*productivity gain, 2 -7*cycle reduction time, ROI 677% Philips ROI 750% Raytheon ROI 770% Schlumberger 4*reduction in released defects

Business Case for Better Software Practices Organization Results Siemens 90% reduction in released defects Telcorida Defect 1/10 industry aver, customer satisfaction up from 60% to 91% Texas Instruments – Systems 90% reduction in delivered defects Thomson CSF ROI 360% U. S. Navy ROI 410% USAF Ogden Air Logistics Center ROI 1, 900% USAF Oklahoma City Air Logistics ROI 635% USAF Tinker Air Force Base ROI 600%

Business Case for Better Software Practices Practice 12 -month ROI 36 -month ROI Formal code inspections 250% 1, 200% Formal design inspections 350% 1, 000% Long-range technology planning 100% 1, 000% Cost and quality estimation tools 250% 1, 200% Productivity measurements 150% 600% Process assessments 150% 600% Management training 120% 550% Technical staff training 90% 550%

Business Case for Better Software Practices ► Performance improvements with the Capability Maturity Model (CMM)

Business Case for Better Software Practices ► Performance improvements with the Capability Maturity Model (CMM) § In general for average organizations, Effort = 2. 94 * (KLOC)^1. 10 § NASA’s Software Engineering Laboratory’s (SEL) effort calculation has Effort = 1. 27 * (KLOC)^. 986 § The. 986 is especially important because it means they are achieving a slight economy of scale. ► COCOMO II § Only three of the 22 factors used to calculate effort were changeable at the individual project management level: Level of Documentation, Architecture and Risk Resolution, and Development for Reuse. Most of these factors are at the organizational level and cannot be easily changed.

Business Case for Better Software Practices 10 questions to ask about software activities How much are you spending on software development? 2. What percentage of your projects are currently on time and on budget? 3. What is the average schedule and budget overrun for your projects? 4. Which of your current projects are most likely to fail outright? 5. What percentage of your project costs arises from avoidable rework? 6. How satisfied (quantitatively) are users of your software? 7. How do the skills of your staff compare to the industry averages? 8. How do the capabilities of your organization compare to similar organizations? 9. How much (quantitatively) has your productivity improved in the past 12 months? 10. What is your plan for improving the skills of your staff and the effectiveness of your organization? 1.

Chapter 14: Ptolemaic Reasoning “All models are wrong; some models are useful. ” -George Box “Knowledge itself is power. ” -Francis Bacon

Ptolemaic Reasoning ► Ptolemy was an astronomer who lived around A. D. 100 and theorized that the sun revolved around the earth. ► His theory was eventually replaced by Copernicus when his data showed findings that contradicted Ptolemy. ► In the same way, real-world data supports an object-oriented development process.

Ptolemaic Reasoning ► Capability Maturity Model § Developed by the Software Engineering Institute. § There are five software levels: ► Level 1: Initial § Software development is chaotic. This is the default level where the organization generally relies on the “code-and-fix development” model. ► Level 2: Repeatable § Basic project management practices are established on a project -by-project basis. The strength of the organization lies on its ability to repeated experiences.

Ptolemaic Reasoning ► Capability Maturity Model ► Level 3: Defined § The organization adopts standardized technical and management processes across the organization. A group is assigned to monitor the software process. ► Level 4: Managed § Project outcomes become highly predictable. A standard process is identified and variations can be detected. ► Level 5: Optimizing § The focus of the organization is on proactive identification of process improvements. It has the ability to measure changes to the process and identify their strengths and weaknesses. § Conway’s Law: The structure of a computer program reflects the structure of the organization that built it.

Ptolemaic Reasoning ► Navigating the Capability Maturity Model 1991 2002

Ptolemaic Reasoning ► Navigating the Capability Maturity Model

Ptolemaic Reasoning ► CMM and Risk Management § Cheyenne Mountain ATAMS project ► The project team was able to… § complete the project in 1/5 th of the projected time. § complete the project in ½ of the estimated time. ► Overall, they were able to deliver the software one month ahead of schedule and within budget. ► Eighteen months after release, only two defects were discovered. ► Would developers like CMM? § In a survey of 50 organizations, 20% of level 1 organization rated staff morale as “good” or “excellent”. § At Level 2, 50% of the staff rated morale as “good” or “excellent”. § At Level 3, 60% of the staff rated morale as “good” or “excellent”. § Most developers working in a level 5 environment don’t want to work for an organization that is less than level 5.

Ptolemaic Reasoning ► What does it take to implement CMM? § Commitment from top management (and followthrough) § Establishment of a Software Engineering process group (sometimes multiple groups). § Appropriate training in middle management and various technical positions. ► What is success? § Success = Planning * Execution § As an organization moves up CMM levels, they will find that lower levels seem valuable and less effective.

Chapter 15: Quantifying Personnel Factors “Personnel attributes and human relations activities provide by far the largest source of opportunity for improving software productivity. -Barry W. Boehm

Quantifying Personnel Factors ► Personnel Experience ► COCOMO II § In a study conducted by Sackman, Eriskon, and Grant, they found that the variance in time-to-completion for a programming problem varied almost 20 -to-1 among a programmers with 7+ years of experience. § Other studies have found similar findings ranging anywhere from 5 -to-1 to 10 -to-1. § In some cases, the programmers were unable to complete the problem. § In a group of 7 random programmers, there’s a 50/50 change that at least one of the programmers will produce negative productivity. § There are 7 factors that are influenced by experience: Application experience (1. 51), Communications factors (1. 53), Language and tool experience (1. 43), Personnel continuity (1. 51), Platform experience (1. 40), Programmer capability (1. 76), Analyst capability (2. 00).

Quantifying Personnel Factors ► Physical Environment § In these war-games, the environment played a factor. Of the top 25% of programmers, most have bigger, private offices (~78 ft 2) with fewer interruptions ► Motivation § Microsoft provides each group with a “morale budget” which the team can spend on anything from plaques, to t-shirts, to dinner and movies. § Microsoft allows programmers to accommodate family situations. ► Seniority § Because programming experience plays such a large factor in development, many companies have found success in have senior personnel help with each project.

Chapter 16: Construx’s Professional Development Program

Construx’s Professional Development Program ► Construx’s Software Development objectives § Skills enhancement ► Improve the skills of the employees § Career pathing ► Provides a structured path for improvement and career choice. § Support for common software job titles ► Have a full list of titles: software developers, testers, business analysts, project managers, architects, etc… § Consistency ► Provide a consistent means for evaluation § Generlizability beyond Contrux ► Provide a generic framework for other companies to model.

Construx’s Professional Development Program ► Construx’s Knowledge Areas (SWEBOK) § Software Requirements § Software Design § Software Construction § Software Testing § Software Maintenance § Software Configuration Management § Software Quality § Software Engineering Management § Software Engineering Tools and Methods § Software Engineering Process

Construx’s Professional Development Program ► Construx’s Capability Levels § Introductory ► Employee can perform basic functions in an area under supervision. Professional development is occurring. § Competency ► Employ can perform effective, independent work and serve as a role model for the less experienced. Occasionally mentors § Leadership ► Employee performs exemplary work and regularly coaches employees and provides some leadership direction. § Mastery ► Employee can perform reference work and has deep experience. Usually, the employee has taught classes or written papers. They are recognized outside of Construx.

Construx’s Professional Development Program ► Construx’s Capability Levels Experience Introductory Competency Leadership Introductory Knowledge Mastery Introductory Competency - Competency - Leadership Competency Leadership Mastery - Mastery

Construx’s Professional Development Program ► Construx’s Professional Development Ladder § Each engineer is giving a rating between 9 -15, based on their knowledge and experience. A 12 is considered to by a full professional. § Most engineers don’t go beyond because it requires career-long commitments to the company to the Software Engineering field. ► Construx’s Culture § Professional Development Plan, Mentoring Program, Professional Development Plaques, Training Program, Salary Structure, SE Discussion groups, Level-12 recognition.

Organizational Professionalism ► Software Gold Rushes § Gold Rush vs. Post-Gold Rush software ► Business Case for Better Software Practices § Process improvements and their effect on business ► Ptolemaic Reasoning § CMM ► Quantifying Personnel Factors § § ► COCOMO II Environment, Motivation, etc… Contrux’s Professional Development Program § § Development objectives CKAs Capability Levels Ladder-based Careers

Part Four Industry Professionalism

Chapter 17: Engineering a Profession “Engineering can provide a life of genuine satisfaction in many ways, especially through ministering in a practical manner to the needs and welfare of mankind. ” -Vannevar Bush

Engineering a Profession ► Engineering feats of the past Reims Cathedral Sydney Opera House

Engineering a Profession ► Maturation Cycle of an Engineering Discipline § There are 3 stages in the development of a discipline ► Craft § Work is performed by talented amateurs. There is little-to-no concept of scalability. ► Commercial § Workers are more economically driven. The focus is on quality in the products and production processes. ► Professional Engineering § A corresponding science is developed to better understand the engineering problem and this science is then applied on a wider scale to many of the common engineering problems of the discipline. § As a discipline matures, solutions to common problems are codified (equations, models, components) so that they can be reused.

Engineering a Profession ► The science behind software § Unlike Physics, the science behind the Civil Engineering that built the Reims Cathedral, Software Engineering doesn’t have an exact science behind it. § Software Engineering artifacts ► Architectures, software design procedures ► Design patterns ► Requirements ► User Interface elements and procedures ► Estimates ► Planning data (project plans, procedures) ► Test plans, cases, data, procedures ► Technical review procedures ► Source code, construction procedures, integration procedures ► Software configuration management procedures ► Post-project reports ► Organizational structures

Chapter 18: Hard Knocks “Natural abilities like natural plants, that need pruning by study; and studies themselves do give forth directions too much at large, except they be bounded in by experience. ” -Francis Bacon

Hard Knocks

Hard Knocks ► Is Computer Science relevant to Software Engineering? § Notice the decline of Computer Science and IS degrees from 1985 -1997. ► The decline was due to Computer Science curriculum's becoming irrelevant to the workplace. § There’s an upsurge from 1998 -2000. ► The Gold Rush and dot-com booms fueled a resurgence interested for incoming freshmen.

Hard Knocks ► Should Software Engineering follow the same path as other engineering disciplines?

Hard Knocks ► Academic development of Software Engineering ► Accreditation § The first Master’s degree in Software Engineering was offered by Seattle University in 1982. § University of Sheffield’s Computer Science department (UK) offered an undergraduate Software Engineering degree in 1988. § Rochester Institute of Technology offered the first US undergraduate course in 1996. § 25% of Software Engineering programs are offered in the US. § In 2003, ~24 institutions offered an undergraduate Software Engineering degree. § Should Software Engineering focus on software or engineering? § The Computer Science accrediting board requires that their students make a ”scholarly contribution” to computer science, but no industry experience. § ABET requires “non-academic engineering experience. ”

Chapter 19: Stinking Badges “Badges? We ain’t got no badges. We don’t need no badges. I don’t have to show you any stinking badges. ” -Gold Hat Bandito, The Treasure of the Sierra Madre

Stinking Badges ► Certification § A voluntary process culminating in an exam that is administered by a professional society. § Typically, both education and experience are required. § There are some certifications that are available. ► Institute for Certification of Computing Professionals’ Associate Computing Professional and Certified Computing Professional ► Microsoft’s Microsoft Certified Professional ► Novel’s Certified Network Engineer ► Oracle’s Oracle Certified Professional ► Apple’s Apple Certified Server Engineer ► IEEE’s Certified Software Development Professional.

Stinking Badges ► Licensing § A mandatory legal process culminating in an exam that is administered by a governing agency. § Typically, this is required to protect the public (doctors, architects, lawyers). ► Can Software Engineering be licensed? § ‘There is no generally agreed upon body of knowledge for software engineering. ’ ► SWEBOK § ‘Knowledge is software engineering changes so quickly that exams will be out of date by the time they’re offered. ’ ► Review the contents of SWEBOK.

Stinking Badges ► Can Software Engineering be licensed? § ‘No reasonable test for software engineering skill could be put into a multiple-choice format. Indeed, no exam-based practices could adequately ensure competency of software engineers. ’ ► Other complex disciplines already have exams (medicine, law, etc…). Why is this any different? § ‘The breadth of sub-disciplines involved in software would make licensing all of the sub-disciplines impractical. ’ ► The medical profession has several exams for different specialties. Only software development that will impact the public’s health and/or welfare would need to be explicitly licensed. § ‘The fundamentals of Engineering Exam required for licensing existing professional engineers is inappropriate for those receiving a computer science degree. ’ ► Some undergraduate degree programs focus on the engineering discipline, but more work would need to be done to adopt this exam.

Stinking Badges ► Is licensing a bad idea? § ‘Licenses would unduly restrict the number of people who could practice software engineering at a time when demand for software engineers is increasing. ’ ► Not all Software Engineers would be required to have a license. § ‘When an engineer receives a license, it will be good for life, which is inappropriate considering that software engineering’s body of knowledge is changing so rapidly. ’ ► Review SWEBOK and note that other licenses help keep professionals up-to-date. § ‘Licensing can’t guarantee that every individual who’s licensed will actually be competent. Licensing will give the public a false sense of security. ’ ► Like other disciplines, only certain Software Engineers would need to be licensed.

Stinking Badges ► More on Licensing § Texas currently offers a “bootstrap” Professional Engineering license for Software Engineers. § One side-effect of being licensed is that you’re legally accountable for your work. § Licensed engineers would have the final say on methodology choices. § 3 paths to general licensing ► Specialty in engineering that focus on software ► Create a software engineering specialty exam ► Create a Professional Software Engineering credential. ► Engineering students in Canada receive an iron ring to symbolize their commitment to society.

Chapter 20: The Professional’s Code “Although there is a little bit of Peter Pan in each of us, maturity brings with it the desire to contribute to the communal welfare. The fulfillment of this yearning, I repeat, provides the engineer with his primary existential pleasure. ” -Samuel C. Florman

The Professional’s Code ► A code of ethics is required for all professions. ► Software Engineering Code of Ethics and Professional Practice (joint venture with ACM/IEEE) § http: //www. acm. org/serving/se/code. htm § There are two guiding philosophies… ► “Software engineers shall commit themselves to making the analysis, specification, design, development, testing and maintenance of software a beneficial and respected profession. In accordance with their commitment to the health, safety and welfare of the public, software engineers shall adhere to the following Eight Principles: “ (Preamble short-version)

The Professional’s Code ► Software Engineering Code of Ethics and Professional Practice § There are eight guiding principles… Public: Software engineers shall act consistently with the public interest. 2. Client and Employer: Software engineers shall act in a manner that is in the best interests of their client and employer, consistent with the public interest. 3. Product: Software engineers shall ensure that their products and related modifications meet the highest professional standards possible. 4. Judgment: Software engineers shall maintain integrity and independence in their professional judgment. 1.

The Professional’s Code ► Software Engineering Code of Ethics and Professional Practice § There are eight guiding principles… Management: Software engineering managers and leaders shall subscribe to and promote an ethical approach to the management of software development and maintenance. 6. Profession: Software engineers shall advance the integrity and reputation of the profession consistent with the public interest. 7. Colleagues: Software engineers shall be fair to and supportive of their colleagues. 8. Self: Software engineers shall participate in lifelong learning regarding the practice of their profession and shall promote an ethical approach to the practice of the profession. 5.

The Professional’s Code ► Why is the Code needed? § All professions are required to have a Code. § Death-march projects ► Developers debate unachievable schedules. § Low-ball bidding ► Developers should provide realistic estimates. § Code-and-fix development ► Inconsistent with developing quality code. § Knowledge stagnation ► Developers cannot perform professionally without keeping upto-date on professional issues.

Chapter 22: Alchemy “Q: What are the most exciting/promising software engineering ideas or techniques on the horizon? A: I don’t think that the most promising ideas are on the horizen. They are already here and have been here for years but are not being used properly. ” -David L. Parnas

Alchemy Best Practices Year Described Automated estimation tools 1973 Evolutionary delivery 1988 Measurement 1977 Productivity environments 1984 Risk management planing 1981 Change board 1978 Throwaway user interface prototyping 1975 JAD sessions 1985 Information hiding 1972 Design for change 1979 Source code control 1980 Incremental integration 1979 Branch-coverage testing 1979 Inspections 1976 SEI’s CMM 1987 Software Engineering Process Groups 1989

Alchemy ► If so many of the best-practices have been in existing for 15+ years, why haven’t they been implemented on a wide -spread scale? § § § Many organizations fear changing what’s worked previously. Not all best-practices work in practice. Many companies fear the risk involved in changing.

Alchemy ► How can we diffuse information? § The Agricultural Extension ► Research Subsystem § Research professors producing innovative results that are later diffused to the group. ► State Extension Agents § Connect the Research Subsystem to the County agents. ► County Extension Agents § Persons who help local farmers choose what innovations that should use. ► SEI acts as software’s Research Subsystem § NASA’s SEL producing guidebooks ad training courses.

Industry Professionalism ► Engineering a Profession § What is a profession? § Is Software Engineering a profession? ► Hard Knocks § History of Software Engineering ► Stinking Badges § Certification vs. Licensing ► The Professional’s Code § Software Engineering Code of Ethics and Professional Practice ► Alchemy § Best-practices § Diffusing information

Overview The Software Tarpit Wrestling with Dinosaurs Fool’s Gold Cargo Cult Software Engineering Body of Knowledge Novum Organum Individual Professionalism Orphans Preferred Raising Your Software Consciousness Building the Community Programmer Writing Organizational Professionalism Software Gold Rushes Business Case for Better Software Practices Ptolemaic Reasoning Quantifying Personnel Factors Construx’s Professional Development Program Industry Professionalism Engineering a Profession Hard Knocks Stinking Badges The Professional’s Code Alchemy