Evolution of Software System Quality Second Asia-Pacific Conference on

Evolution of Software/System Quality Second Asia-Pacific Conference on QUALITY SOFTWARE Dr. Raymond A. Paul December 11, 2001 NOV 2000

Definition QUALITY is the characteristic that distinguishes the grade of excellence or superiority of a process, product, or service. • In general usage, QUALITY means different things. Meaning of QUALITY varies considerably across specific disciplines and applications. • What is Quality? 2

Historical Aspects of QUALITY Evolution • Edward Deming – Postulated Statistical QUALITY Control Principles – Famous 14 Points of QUALITY Management – A Subset of these Principles successfully adapted by Japanese Manufactures • William Crosby – Emphasized Humanistic Behavioral Aspects of QUALITY Improvement – Becoming More Important Now 3

Juran TOTAL STATISTICAL QUALITY MANAGEMENT • QUALITY Trilogy A. QUALITY Planning • Set of QUALITY Goals • Set Plans for Operations Based on these Goals B. QUALITY Control • Responsible for Meeting QUALITY Goals • Prevent Adverse Changes • Set and Observe Performance Measures Compare with Industry Standards Benchmarking Compare with Deming, CMM C. QUALITY Improvement • Moving from Current Level to the Next Higher Level • Organize Teams, Train Operators to identify and Correct QUALITY Problems 4

Six Sigma Initiatives • GE and Motorola • Recognized Widely by Manufacturing Industry – Adopted in Various Ways by Software Industry • Major Results – Total QUALITY Management (TQM) – ISO Certification – Malcolm Baldridge Nation of QUALITY Award (Dept. of Commerce) Six Sigma Means no more that 3. 4 defects per million opportunities (six standard deviations away from the average of an assumed normal distribution) • IBM (late Harlan Mills) Clean Room “follows” this Metric 5

Perspectives on QUALITY (Garvin 1984) VIEWS • Transcendented View – Something we recognize but cannot define EXAMPLE: Beauty, Aristotle's Concept of Form • User View – Fitness of Purpose • Manufacturing (Design) View – Conformance to Specification • Product View – Tied to inherent product functions and characteristics • Value-Based View – Depends on the price the customer is willing to pay • What is Quality? 6

QUALITY Models (A LOOSE INTERPRETATION) • Mc. Call’s QUALITY Model (1977) – Shows relationships between external QUALITY Factors such as correctness, reliability, usability, testability, etc. and Product QUALITY Criteria such as traceability, completeness, error tolerance, etc. • PROBLEM: Difficult to measure these QUALITY Factors – No Standards, No Methods, No Tools • What is Quality? 7

Boehm's QUALITY Model (1986) • Asserts that QUALITY software satisfies the needs of the users, designers, testers, and maintainers • Relates software’s general utility to maintainability, reliability, testability, etc. , to device independence, completeness, accessibility, etc. • Very difficult to apply in practice, but easy to understand learn • Parts of it can be supported by tools; therefore, useful in a restricted sense • What is Quality? 8

ISO 9126 QUALITY Model (1991) • Consolidates many views of software QUALITY • ISO hierarchy is strict and non-overlapping unlike Mc. Call’s and Boehm's – EXAMPLE: In the model, maintainability depends ONLY on analyzability, changeability, and stability. Whereas, in Boehm’s model, maintainability and understandability depend on structuredness, legibility, etc. , in an overlapping way. • Like Boehm’s model, it does not tell us how to measure characteristics such as security, accuracy, and interoperability. • What is Quality? 9

Software QUALITY Characteristics Desired (MOST MODELS INCLUDE THESE) • Functional Correctness Functions and their specified properties • Reliability Attributes maintained by software under stated conditions • Usability Effort needed for users • Efficiency Software performance with a given set of resources • Maintainability Effort needed to make specified modifications • Portability Effort needed to transfer software from one environment to another specified environment • What is Quality? 10

Comments on the Software QUALITY Models • Some important characteristics such as evolvability, safety, and security are left out • Dromey’s Model (1996) tries to address and serve some of these issues but not satisfactory • What is Quality? 11

MATURITY Models (MM) • Specification Process (Design and Implementation) Product • Complete MM emphasizes – Process MATURITY • What is MATURITY? – Methods of continuously improving a procedure, activity, or process by planned measurements and feedbacks. – In essence, it emphasizes incremental but positive improvement of an activity via repeated use and measurement. • Measuring Quality 12

Analysis of Current MATURITY Models – One Dimensional – Does Not Include Items such as • • • – – Maintenance Application Customer Satisfaction Correctness with Respect to Specifications Ease of Use, Ease of Change (maintenance) No Common Set of Assumptions No Common Set of Measurements (no standardization) No Financial Business Considerations No Interoperability and Scalability Considerations The emphasis in all these models INCREMENTAL! Incremental advances based on current experience. NO LEAP FROGGING!!! • Measuring Quality 13

Paul’s Call MATURITY means growing older and wiser • Collecting and retrieving experiences • Measure, observe, gather feedback from the product and process, update lessons learned from the product and process • Maturity techniques continuously strive to improve processes, procedures, and activities by repeated use, planned measurements, and feedback In essence, maturity emphasizes incremental but positive improvement of an activity via repeated use, measurement, analysis of results of the product, and feedback refinement • Measuring Quality 14

Questions What to measure? How to measure? When to measure? Where to measure? How to analyze and interpret results? Remember the problems of measurement. . . – Assumptions – Assessment – Analysis …have to be standardized One Method goal-primary-same GQM Paradigm ( goal-primary-different ) Goal, Question, Metric (BASILI, et al. ) Not Very Effective – Have to customize the product design goal from designers that could be different from users • Measuring Quality 15

Hierarchy of Maturity Models • We Propose a Hierarchy of MATURITY Models for Improvement of IT Processes and Products IT Maturity Model (ITMM) Computing MM Hardware MM Communications MM Software MM • Measuring Quality 16

Assistive Technology to Improve QUALITY of Life • These technologies assist users with physical disabilities. Quality of Life Improvement • 47 million Americans are in some form disabled • 75% of Americans with major disabilities are unemployed (President Clinton) Major Disability – Vision Impairment “World is so Visual” • Key: Augmenting Human Productivity • Designers do not have people with disabilities in mind when they develop their products • Philosophy – Design products that work with more people • Methods of Design – Universal Designs – Specialized Designs • Measuring Quality 17

QUALITY Parameters • Usage Based – Utilitarian – Satisfaction of User’s Needs • Functionality • Ease of Use and Handling • Troubleshooting – Restart and Recovery – Maintenance and Analyzability • Technology-Based Aspects – Design Complexity – Maintainability – Evolutionary Flexibility • Reconfigurability • Agility • Growth Enhancement • Measuring Quality 18

QUALITY Parameters (cont. ) • Economic Considerations – Cost to the Seller, and Price to the Buyer – Competitiveness with Similar Products – Ease of Manufacture, Reuse, and Maintenance • Aesthetic Considerations – – – – Form and Function Appearance and Content Appropriateness for the Purpose Color, Packaging, Appearance Attractiveness to Users Attention-Grabbing Features Entertaining, Interesting, Attention Keeping, e. g. TV/Websites • Mass Appeal • Class/Culture Appeal • Individual Appeal • Measuring Quality 19

QUALITY Parameters (cont. ) • Personality/Individualization – To support and help physical preferences and disabilities – EXAMPLES • • Right vs. Left Handedness Poor Vision Hearing Impairment Web Browsing – Personalization My YAHOO My EXCITE • Personalization for: News Junkies Financial Analysts Customized Information • Measuring Quality 20

Another QUALITY Life Cycle Model Life-Cycle Make it Work Built to Last Demand Goodies New Functions/ Enhancements Activities Processes Products Services Enterprises Life-Cycle Objectives of a Designer Works the First Time Works Many Times Use Becomes Habit Forming • Achieving Quality Establishes User’s Dependency and Brand Loyalty (SONY) 21

Designing Methodologies for Embedding QUALITY-Based (QUALITY SUPPORTING) Functions • Major Characteristics of These Designs – Reliability, Fault Tolerance via: • Redundancy • Highly Reliable Proven Components/Parts, Interconnections, Networks • Design Methods – Flexibility - Reconfigurability – Dynamic Introduction of: • Standby Optional Functions • Universal Designs • Component-Based, Easy-Assembly Design: – Building Block Based – Lego like – Assembly – Disassembly Principle • Achieving Quality 22

Designing Methodologies (cont. ) • The Methods Presented Give Easy-Assembly (Disassembable) Subsystems – Good for Maintenance (Repair, Modification) – Subsystems and their integral components are loosely coupled. This helps system understandability, analyzability, and reuse – Helps to add on or enhance functions • Complex Systems are Synthesized by: – Layered Architectures – each layer assembled from components – Ring-Based Architectures – Good for Security • Achieving Quality 23

Designing Methodologies (cont. ) To Improve Performance Characteristics (in real-time) the System can be Tightly Coupled SYSTEM A Components (Building Blocks) Assemblers Subsystems System (made of loosely coupled subsystems) For Better System Performance, Real-Time Applications use Tight-Coupling Principles (SYSTEM B). SYSTEM A is used as a Prototype. SYSTEM B A more compact high performance design can be generalized by coalescing the layers (rings), i. e. , tight coupling of some layers – SYSTEM B In coalescing several small components into a bigger element, we reduce the inter-component overhead. More importantly, we eliminate or reduce the conflicts among assumptions made during the design of the components. • Achieving Quality 24

Distributed System and Reconfiguration Design • QUALITY implied here includes: – – Fault Tolerance Control (Concurrency and Event Control) Reconfigurability (Part of Control) Heterogeneity EXAMPLES: – CORBA Components and Distributed Control Technologies – Object-Oriented Design Components – NASA Space Shuttle Avionics • Hardware and Software Structured (Preferred) Reconfiguration Philosophy Very Simple Software Reconfigurations • Achieving Quality 25

Universal Designs • • Primarily for Mass-Produced, Batch-Processed Hardware Systems Holds True for Software Systems as well PRINCIPLE The System Includes Many Functions – Primary, Auxiliary, and Supporting Functions • User/Supplier triggers as activities those supporting and auxiliary functions needed for that specific application or circumstance along with the primary functions EXAMPLE Pentium II MMX – Has 57 supporting instructions (functions) to improve multimedia application processing Underlying Philosophy Design products that work for most applications (or people) INSIGHT: Easier to mask out unwanted functions from a universal design (cheaper) than to build specialized functions from scratch (more expensive, time consuming, and labor intensive) • Achieving Quality 26

QUALITY Manager Concept • • QUALITY embraces many aspects, for example: – Dependability, Fault Tolerance – Maintainability – Information Hiding QUALITY is a dynamic property at anytime. Different aspects of quality become more important depending on the circumstance and needs of the USER at that moment. QUALITY manager is a watch dog function that “detects and supplies” the quality aspect needed by the system at that moment. A method of implementing the quality manager function – Remember that computer power is like cash. It can be used to buy: • Fault Tolerance • Multi-Precision Computation • Support or Take Over Failed Computing Functions Use a high-performance computing function whose “excess” computing power can be used for quality manager functions that can invoke and support real-time quality needs. • Achieving Quality 27

Projected Improvement in QUALITY EFFORT AUTOMATION Industry Growth Time • • • Service Industry Growing Very Rapidly (80% of Workers in US) Service Industry’s Growth can be Traced to Increased Automation and Decreasing Labor-Intensive (manual) Functions Increased Automation means better QUALITY of Products and Services • Evolution of Quality to a User Perspective 28

ENTERPRISE SERVICES PRODUCTS PROCESSESS Quality Input Increasing Complexity QUALITY Pyramid ACTIVITIES • Evolution of Quality to a User Perspective 29

Customer Priorities Make it Work A Legacy Dependency - Afraid to Change B Works the First Time – All the Time C Work Horse User’s Dependency D Make it Last Failure – No Option E Part of Lifestyle Legacy Usage History and QUALITY is Assessed Customer is Prepared to Change A Customer is Reluctant to Change B Reliability is Taken for Granted C Maintenance and Cost are No Issue D E Afraid of Anything New – Not Similar to the One Used Usage Life Cycle C D Make it Last Forever E Demand • Evolution of Quality to a User Perspective • Changes in Quality 30

Evolution of QUALITY Consciousness and Measures (HARDWARE, SOFTWARE, INFORMATION TECHNOLGY, AND SERVICE) A Productivity & Performance B Reliability C SQC D Behavioral Considerations E Total Quality Management and Control INDUSTRIAL REVOLUTION: Assembly Line and Mass Production, Productivity Consciousness - Time studies and ELI Whitney’s Standards and Interchangeable Parts, Ford’s Assembly Line (Productivity Emphasis) Early Operations Research Machines can be produced rapidly, and they are complex -need for improved reliability. EXAMPLE: Steam Engines, Automobiles, Airplanes Statistical Quality Control- Edward Deming, Shewhert Man is not a machine. Quality can be improved by focusing on human behavior and industrial psychology. Rise of industrial unions, human factors, and ergonomic emphasis improve quality and productivity by creating better working environments. ( Crosby) TQM & C – J. Juran – Trilogy of Quality Management, Most Popular and Important • Evolution of Quality to a User Perspective 31

Evolution of QUALITY Consciousness and Measures F SIX SIGMA Quality Initiatives G Software Quality Initiatives and Methodologies H IBM Clean Room Methods I CMM of SEI J Qo. S K IT MM (cont. ) GE and Motorola - Moved from mostly mechanical parts to electronic parts and assemblies - Jack Welch of GE Structured Programming, High-level languages, object-oriented design, and implementation languages Structured Programming and Statistical Testing – Harlan Mills, Mike Fagan Inspection Procedures Watts Humphrey, Bill Curtis Initiative of DARPA for Communications Networks To Be Developed • Evolution of Quality to a User Perspective 32

QUALITY Concerns Change with • User Needs • Technology Evolution • Product and Service Sophistication EXAMPLE: HARDWARE & SOFTWARE & COMMUNICATION EMBEDDED & DISTRIBUTED SYSTEMS • Evolution of Quality to a User Perspective 33

QUALITY Concerns with Change HIGH Degradation of QUALITY Due to Change LOW Complexity of Change • Evolution of Quality to a User Perspective HIGH 34

Service Triplets PROCESS PRODUCT SERVICE • Current Software Quality FADs Emphasize Process Aspects Only • CMM Emphasizes Continuous Improvement in Software Development Processes (Well-Defined Activities) • Specification Process Software Product • CMM Emphasizes Process Maturity • Evolution of Quality to a User Perspective 35

Change of Customer QUALITY Priorities with Product (Software) Life Cycle Make it Work (Works the First Time and All the Time) User’s Dependency on Product User’s Expectation (Product Work Horse Ethic) Built to Last User Afraid to Change Legacy Mentality PRODUCT USAGE LIFE CYCLE Customer Expectation Increasing Dependency on the Product The Ultimate Milestone of QUALITY • Evolution of Quality to a User Perspective 36

QUALITY – Customer Priorities (EVOLUTION OF QUALITY CONSCIOUSNESS) Usage Life Cycle – How QUALITY is assessed by the customer A B Make it Works the First Time – EVERYTIME A B C D E User’s Dependency C D E Work Horse Ethic Built to Last User Afraid to Change, Legacy Mentality = Reliability and Maintainability = Failure No Option = Refining, Enhancing User Preferred Features • Evolution of Quality to a User Perspective 37

Quality of Databases • • • Databases are used widely and extensively in all kinds of enterprises and their use will only increase. But what is the view of database quality? From a database manager point of view, database quality is terms of database size, transaction speed, data reliability, scalability and data integrity. However, from a software designer point of view, quality may mean the maintainability, code reliability, and software architecture attributes. However, from the enterprise point of view, quality may mean the contribution of a database to the enterprise mission. • Evolution of Quality to a User Perspective 38

Quality from a Higher Level Perspective • We need to think in terms of a high level perspective from the user point of view rather than a design or process points of view – Design view such as redundancy for fault tolerance, buffers for scalability. – Process views such as following certain techniques, work flow, or using specific kinds of tools such as configuration control. • Furthermore, we should not think in terms of system requirements only, we need to think about the overall mission of the enterprise, operation, enterprise capabilities, and enterprise evolution point of view. • Evolution of Quality to a User Perspective 39

Enterprise Evolution • Technology is changing rapidly at the speed of the Internet. • Enterprise mission also changes, but at a much slower speed. • Enterprise capability also needs to change with knowledge and technology changes. – This is the trade-off between budget growths at X% per year and Moore’s law [IT technical capability grows at 67% per year]. • An orderly transition of enterprise capabilities, while taking advantage of technology change, is badly needed. • This also assumes that an enterprise has a sound strategy to know what is needed and how to incorporate new technologies and new knowledge growth plans into the existing infrastructure and superstucture. • Evolution of Quality to a User Perspective 40

Summary • QUALITY - particularly software quality - has to be more adaptive • Products and processes grow with their application and human environments. Quality criteria have to change with the situation, circumstance, and the environment. These are dynamic characteristics. Therefore, quality has to be dynamic and adaptive. • Quality aspects change with application and product life cycles. Software is the right place to put quality adoption and evolution. • The new IT environment will be multidisciplinary, fast changing with rapid diversification and globalization. • THE QUALITY JOURNEY CONTINUES, BE PREPARED • Summary 41

LEADERSHIP • The position or office of a leader • Capacity or ability to lead • A continuum of good decisions • Provide direction • To influence

HOPE • Michelangelo lived a few days shy of 89 years, still sculpting, painting, writing, and designing in an age when 90 was about 60 years beyond normal life expectancy was speaking to this idea of having high hopes and aims in this quotation (next slide) of his. • I recall being in front of the statue of David in Florence and being transfixed. The size, the majesty, the spirit that seemed to jump right out of the marble was Michelangelo saying to all of us, “Aim high. ” • High aim indeed. Look at the Sistine Chapel where Michelangelo painted the ceiling by lying on his back working every day for 4 years. Lesser artists considered the task impossible, yet Michelangelo took it on and many more and aimed high 43

HOPE The greater danger for most of us is not that our aim is too high and we miss it, but that it is too low and we reach it. Michelangelo (1475 -1564) 44

HOPE cont. • No one knows enough to be a pessimist!!! • Michelangelo’s advice is just as applicable today as it was almost 500 years ago. Never listen to those who try to influence you with their pessimism. Follow these guidelines: – Refuse to listen to or internalize the proclamations of those who point to your limitations. You must always remember: • when you argue for your limitations, the only thing you get are limitations • imagine Michelangelo telling you that you can create the quality you desire, and the great danger is not in having too much hope, but in reaching what you have perceived as hopeless 45