Formalising usability as part of ISO standards Nigel Bevan nbevan@usability. serco. com www. usabilitynet. org Serco Usability Services UK ©Serco Reproduction permitted provided the source is acknowledged 1
Topics What is usability? Definitions of usability? ISO 9126 -1 and ISO 9241 -11 Specifying usability as quality in use: ISO 9126 -4, the Common Industry Format and ISO 25030 Usability of everyday products: ISO 20282 User interface usability: ISO 9241 User centred design process: ISO 13407 Usability capability: ISO TR 18529 ©Serco Reproduction permitted provided the source is acknowledged 2
What is usability? what do you think usability is. . . ? ©Serco Reproduction permitted provided the source is acknowledged 3
IBM “IBM's focus on ease of use has led to technology breakthroughs and advances in design research” “For developers and manufacturers, the advantages of creating usable products far outweigh the costs” “Every dollar invested in ease of use returns $10 to $100” http: //www. ibm. com/ibm/easy/ ©Serco Reproduction permitted provided the source is acknowledged 4
Microsoft “The Usability Group is an integral part of the product design process at Microsoft” Over 120 usability engineers Over 25 usability labs http: //www. microsoft. com/usability/ ©Serco Reproduction permitted provided the source is acknowledged 5
Why is usability important? Focus on user and organisational needs reduce development times less training and support and documentation is required Improve productivity simpler interface, fewer user errors Improve the competitive edge increasing expectations for ease of use increasing usability of competitive products high profile of usability in advertising Improve the quality of life less stress, users are more satisfied lower staff turnover Health and safety legislation European Directive on Display Screen Equipment ©Serco Reproduction permitted provided the source is acknowledged 6
Case study: Hewlett Packard User-centred design methods were applied to redesign software used for identifying network problems: Old New time to finish task 9. 4 min 4. 1 min problems identified 16% 78% average length of call 30 min 10 min size of manual 25 pages 4 pages people needing the manual 53% 3% user satisfaction rating 3. 5 6. 8 In addition to the benefits to customers, HP recovered their costs in 18 months ©Serco Reproduction permitted provided the source is acknowledged 7
ISO/IEC 9126 Software Product Quality Model functionality reliability accuracy suitability interoperability security maturity fault tolerance recoverability availability usability efficiency understandability learnability operability time behaviour resource utilisation maintainabili ty portability analysability changeability stability testability adaptability installability co-existence replaceability ©Serco Reproduction permitted provided the source is acknowledged 8
ISO/IEC 9126 -1 Definitions Functionality The capability of the software product to provide functions which meet stated and implied needs when the software is used under specified conditions. Reliability The capability of the software product to maintain a specified level of performance when used under specified conditions Usability The capability of the software product to be understood, learned, used and attractive to the user, when used under specified conditions. Efficiency The capability of the software product to provide appropriate performance, relative to the amount of resources used, under stated conditions. Maintainability The capability of the software product to be modified. Modifications may include corrections, improvements or adaptation of the software to changes in environment, and in requirements and functional specifications. Portability The capability ©Serco Reproduction permitted provided the source is acknowledged of the software product to be transferred from one environment to 9
ISO 9241 -11 Guidance on Usability : The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use. Effectiveness : The accuracy and completeness with which users achieve specified goals. Efficiency: The resources expended in relation to the accuracy and completeness with which users achieve goals. Satisfaction : The comfort and acceptability of use. ©Serco Reproduction permitted provided the source is acknowledged 10
ISO/IEC 9126 -1 Software Product Quality Model qualit y in use functionality reliability accuracy suitability interoperability security maturity fault tolerance recoverability availability usability efficiency understandability learnability operability attractiveness time behaviour resource utilisation maintainability portability analysability changeability stability testability adaptability installability co-existence replaceability ©Serco Reproduction permitted provided the source is acknowledged 11
Quality in Use ISO/IEC 9126 -1 (2001) Software Engineering Product quality - Part 1: Quality model quality in use the capability of the software product to enable specified users to achieve specified goals with effectiveness, productivity, safety and satisfaction in specified contexts of use Quality in use is the user’s view of the quality of a system containing software, and is measured in terms of the result of using the software, rather than properties of the software itself Quality in use measures the combined effect of ease of use, functionality, efficiency and reliability. ©Serco Reproduction permitted provided the source is acknowledged 12
Approaches to achieving usability organisational capability process quality product quality in use effect of the product life cycle processes development process product usability capability user centred process interface and interaction ISOTR 18529 ISO 13407 ISO 9241 parts 10, 12 -17 ISO/IEC 9126 -2/3 ©Serco Reproduction permitted provided the source is acknowledged usability in context ISO 9241 -11 ISO 14598 -1 ISO/IEC 9126 -4 ISO 20282 -2 13
Approaches to achieving usability 1 Design for quality in use ISO 9241 -11, ISO 9126 -4: effectiveness, efficiency, (safety), satisfaction ISO 20282 -2 Usability of everyday products 2 Design the user interface ISO 9241 parts 12 to 17 Ergonomic principles 3 Use user-centred methods in development ISO 13407 Human centred design processes 4 Establish organisational processes ISO TR 18529 Human-centred lifecycle process descriptions ©Serco Reproduction permitted provided the source is acknowledged 14
What can be specified and measured? requirements product Quality in use metrics ISO/IEC 9126 -4 User requirements Deliver External usability metrics ISO/IEC 9126 -2 Functional requirements Prototype Internal usability metrics ISO/IEC 9126 -3 Detailed specification Code ©Serco Reproduction permitted provided the source is acknowledged evaluation Test user goal Test user function inspection 15
1. Design for quality in use Usability/quality in use measures: Effectiveness and efficiency MUSi. C Performance Measurement Method http: //www. usability. serco. com/publs. html Satisfaction SUMI questionnaire www. ucc. ie/hfrg/questionnaires/sumi Reporting usability Common industry format for usability test results includes intended context of use and actual context of evaluation www. nist. gov/iusr ©Serco Reproduction permitted provided the source is acknowledged 16
ISO/IEC CD 25030: Software quality requirements and evaluation – Quality requirements Requirements can be categorised as: functional requirements quality requirements other non-functional requirements Software quality requirements are categorised using the ISO/IEC 9126 -1 software quality model into requirements for: quality in use (the effectiveness, productivity, safety and satisfaction with the software applied in its context of use) external quality (the behaviour of the finished software product) internal quality (static properties during development, such as adherence to coding standards, and meeting size and complexity thresholds) ©Serco Reproduction permitted provided the source is acknowledged 17
Requirements life cycle model Definition Formalisation Evaluation Functional requirements Quality requirements Developed system Quality in use Stated requirements Stakeholder needs Measurable requirements Measured quality External quality Stated requirements Measurable requirements Measured quality Internal quality Stated requirements Measurable requirements Measured quality Other non-functional requirements ©Serco Reproduction permitted provided the source is acknowledged 18
Quality requirements specification Define quality requirements Identify stakeholders end users, developers, producers, trainers, maintainers, disposers, acquirer, supplier organisations and regulatory bodies Elicit requirements from stakeholders Quality in use, External and Internal Quality Analyse the set of requirements Resolve problems Confirm requirements Record requirements Formalise quality requirements Specify target values for measures Demonstrate traceability Maintain requirements ©Serco Reproduction permitted provided the source is acknowledged 19
Elicit requirements from stakeholders Include the needs, wants, desires, expectations and perceived constraints Identify requirements Quality in use External Internal Identify constraints Define scenarios of use ©Serco Reproduction permitted provided the source is acknowledged 20
Quality in use Extent to which a product used by specified users meets their needs to achieve specified goals with effectiveness, productivity and satisfaction in specified contexts of use Effectiveness: accuracy and completeness Productivity: the resources expended by users in relation to the effectiveness achieved ©Serco Reproduction permitted provided the source is acknowledged 21
Why is quality in use important? It is about meeting business and user needs Effectiveness: success in achieving goals Productivity: staff time Satisfaction: willingness to use the system ©Serco Reproduction permitted provided the source is acknowledged 22
Quality in use: context of Use The usability of a product is affected not only by the features of the product itself but also by its Context of Use Context is the characteristics of: the users of the product the tasks they carry out the technical, organisational and physical environment in which the product is used the date and time when the product is being used ©Serco Reproduction permitted provided the source is acknowledged 23
Example: A bank ATM Context description: The users The user characteristics The tasks users perform The technical environment (hardware and software to support system) The physical environment The social or organisational environment ©Serco Reproduction permitted provided the source is acknowledged 24
New design features to meet contextual needs Recess for wheelchair access. Speech output for visually impaired users. Customisation features for rapid access. Finger print for identification. Visor appears during sunny weather. Buttons light during darkness. Alarm button for security alert. ©Serco Reproduction permitted provided the source is acknowledged 25
Efficiency: mean transaction times ©Serco Reproduction permitted provided the source is acknowledged 26
User satisfaction: SUMI questionnaire 60 50 40 30 New System 20 Old System 10 0 Global Efficiency Affect Helpfulness Control Learnability ©Serco Reproduction permitted provided the source is acknowledged 27
The danger of not setting quality in use requirements New software for issuing UK passports Installed in passport issuing offices Took operators twice as long Caused delays of up to 3 months in obtaining a passport Huge cost of additional clerical staff ©Serco Reproduction permitted provided the source is acknowledged 28
Example of quality in use requirements User performance “all data entry clerks will be able to complete the task with at least 95% accuracy in under 10 minutes” User satisfaction “the mean score on the SUMI scale will be greater than 50” More information on quality in use requirements ©Serco Reproduction permitted provided the source is acknowledged 29
Common Industry Format usability tests NIST initiative National Institute of Standards and Technology Objective: raise the profile of usability in procurement Suppliers provide standard test reports to purchasers Suppliers include: IBM, Microsoft, HP, Sun, Kodak, Oracle, Compaq Purchasers include: Boeing, Northwest Mutual Life, State Farm Insurance, Fidelity Reports provided in confidence Could permit comparisons Agreed as US standard ANSI/NCITS 354, to be submitted to ISO http: //www. nist. gov/iusr ©Serco Reproduction permitted provided the source is acknowledged 30
Report format - technical Test objectives Participants Total number and user groups Key characteristics and capabilities expected How selected: with the key characteristics and capabilities? Context of evaluation Task scenarios and task data used Any performance criteria used Physical and computer environment ©Serco Reproduction permitted provided the source is acknowledged 31
Report format - results Effectiveness % completely and correctly achieving their goals Separate data where assistance was given Efficiency Mean time taken to complete the task Standard deviation (and standard error) Satisfaction Optional: own scales or standard questionnaire ©Serco Reproduction permitted provided the source is acknowledged 32
ISO 20282: Usability of everyday products Part 1: Universal user profile Design should take account of Strength and Biomechanical Abilities Handedness Body Dimensions. Visual Abilities. Auditory Abilities Cognitive Abilities Language and literacy Culture Age Gender ©Serco Reproduction permitted provided the source is acknowledged 33
ISO WD 20282 -2: Test method A summative test method for measuring usability based on ISO 9241 -11 (Guidance on usability) ISO 9126 -4 (Quality in use metrics) Common Industry Format for usability test reports Identify intended context of use and user groups Measures Success rate (ease of operation) Task time (efficiency) Satisfaction scale (being developed) ©Serco Reproduction permitted provided the source is acknowledged 34
Approaches to achieving usability organisational capability process quality product quality in use effect of the product life cycle processes development process product usability capability user centred process interface and interaction ISOTR 18529 ISO 13407 ISO 9241 parts 10, 12 -17 ISO/IEC 9126 -2/3 ©Serco Reproduction permitted provided the source is acknowledged usability in context ISO 9241 -11 ISO 14598 -1 ISO/IEC 9126 -4 ISO 20282 -2 35
2. Design the user interface ISO 9241: Ergonomic requirements for office work with visual display terminals Parts 10 to 17 give guidance on software 10 Dialogue principles 11 Guidance on usability 12 Presentation of information 13 User guidance 14 Menu dialogues 15 Command Language dialogues 16 Direct Manipulation dialogues 17 Form-filling dialogues ©Serco Reproduction permitted provided the source is acknowledged 36
Approaches to achieving usability process quality product quality in use life cycle processes development process product effect of the product usability capability user centred process interface and interaction organisational capability ISOTR 18529 ISO 13407 ISO 9241 parts 10, 12 -17 ISO/IEC 9126 -2/3 ©Serco Reproduction permitted provided the source is acknowledged usability in context ISO 9241 -11 ISO 14598 -1 ISO/IEC 9126 -4 ISO 20282 -2 37
3. ISO 13407: Human centred design process for interactive systems 1. Plan the human centred process Meets requirements 2. Specify the context of use 5. Evaluate designs against user requirements 3. Specify user and organisational requirements 4. Produce design solutions ©Serco Reproduction permitted provided the source is acknowledged 38
ISO 13407 Scope Computer-based interactive systems The detailed recommendations are applicable to computer hardware and software Can apply the same principles to the design of any system for use by humans A pen A chair A meeting room A conference ©Serco Reproduction permitted provided the source is acknowledged 39
TRUMP: Trial Usability Maturity Process EU-funded trial application of user-centred design methods developed in previous research projects (INUSE and RESPECT) Serco: apply the methods Lloyds Register: Usability Maturity Assessment Inland Revenue/EDS - IT for 60, 000 staff RAD methodology Israel Aircraft Industries - aerospace systems traditional methodology ©Serco Reproduction permitted provided the source is acknowledged 40
Typical approach to usability feasibility Usability group requirements user and task analysis design prototyping implement release usability test ©Serco Reproduction permitted provided the source is acknowledged 41
TRUMP approach: “ubiquitous usability” Plan Process Specify Context of Use Specify Requirements Design Solution s Evaluate against Requirements System lifecycle feasibility 1. Stake -holder meeting requirements 2. Context of use 4. Usability requirements 3. Scenarios 5. Evaluate existing system design 6. Prototyping 7. Style guide implement release 8. Evaluation 10. Collect feedback 9. Usability testing www. usability. serco. com/trump/ucdmethods ©Serco Reproduction permitted provided the source is acknowledged 42
Conventional design process How do we incorporate user centred design? requirements product User requirements Deliver Functional requirements Prototype Detailed specification evaluation Function test Code ©Serco Reproduction permitted provided the source is acknowledged 43
5. Evaluate existing system 1. Plans Context used for evaluation 2. Intended context of use 3. Intended scenarios of use requirements 4. Quality in use requirements User requirements 4. External usability requirements 4. Internal usability requirements Scenarios used for evaluation product 6. Mock up Use in context Functional requirements 6. Paper prototype Prototype Detailed specification 6. Paper specificatio n 7. Style guide Code evaluation ©Serco Reproduction permitted provided the source is acknowledged Function test Inspection 7. Style guide 44
Methods: feasibility 1. Stakeholder meeting A half-day meeting to identify and agree on the role of usability, broadly identifying the intended context of use and usability goals, and how these relate to the business objectives and success criteria for the system. 2. Context of use A half-day workshop to collect and agree detailed information about the intended users, their tasks, and the technical and environmental constraints. 3. Scenarios of use A half day workshop to document examples of how users are expected carry out key tasks in a specified contexts, to provide an input to design and a basis for subsequent usability testing. ©Serco Reproduction permitted provided the source is acknowledged 45
Methods: requirements 4. Evaluate an existing system Evaluate an earlier version or competitor system to identify usability problems and obtain measures of usability as an input to usability requirements. 5. Usability requirements A half-day workshop to establish requirements for effectiveness, efficiency and satisfaction for the user groups and tasks identified in the context of use analysis and in the scenarios. ©Serco Reproduction permitted provided the source is acknowledged 46
Methods: design 6. Paper prototyping Evaluation by users of quick low fidelity prototypes (using paper or other materials) to clarify requirements and enable draft interaction designs and screen designs to be rapidly simulated and tested. 7. Style guide Identify, document and adhere to industry, corporate or project conventions for screen and page design. ©Serco Reproduction permitted provided the source is acknowledged 47
Methods: evaluation 8. Evaluation of machine prototypes Informal usability testing with 3 -5 representative users carrying out key tasks to provide rapid feedback on the usability of prototypes. 9. Usability testing Formal usability testing with 8 representatives of a user group carrying out key tasks to identify any remaining usability problems and evaluate whether usability objectives have been achieved. 10. Collect feedback from users Collect information from sources such as usability surveys, help lines and support services to identify any problems that should be fixed in future versions. ©Serco Reproduction permitted provided the source is acknowledged 48
Cost benefits at IAI Development cost/benefits A all the methods used (except Style Guide) resulted in savings in development costs of between $5 K and $70 K for each method, with a total saving of $330 K. The cost of using the methods was only $22 K, giving a cost-benefit ration of 1: 15. Sales cost/benefits Increased sales because customers would be more likely to buy the product were estimated to be $400 K Support cost/benefits Reduced costs of training and support were estimated at £ 50 K. Overall cost benefits The overall costs of the maturity assessments and use of methods was $27 K. The total estimated savings and increased sales is $780 K, giving a cost-benefit ration of 1: 29. ©Serco Reproduction permitted provided the source is acknowledged 49
Trial Application at IAI Bottom Line Positive Feedback from Participants A Definite Improvement in the Development Process Very Cost Effective and Low Cost Mostly Intuitive however tailoring sometimes required Expert Guidance needed in few techniques LAHAV decided to incorporate TRUMP techniques in it’s standard development process ©Serco Reproduction permitted provided the source is acknowledged 50
IAI Conclusions Low cost simple methods for improving systems usability are here The methods can be easily integrated into any software development process The responsibility for quality in use should be transferred to development organization The Methods and supporting means are available on the Web No more Excuses ©Serco Reproduction permitted provided the source is acknowledged 51
Approaches to achieving usability organisational capability process quality product quality in use effect of the product life cycle processes development process product usability capability user centred process interface and interaction ISOTR 18529 ISO 13407 ISO 9241 parts 10, 12 -17 ISO/IEC 9126 -2/3 ©Serco Reproduction permitted provided the source is acknowledged usability in context ISO 9241 -11 ISO 14598 -1 ISO/IEC 9126 -4 ISO 20282 -2 52
4. Establish usability capability ISO TR 18529: Human-centred lifecycle process descriptions HCD. 1 Ensure HCD content in system strategy HCD. 2 Plan and manage the HCD process HCD. 3 Specify the user and organisational requirements HCD. 4 Understand specify the context of use HCD. 5 Produce design solutions HCD. 6 Evaluate designs against requirements HCD. 7 Introduce and operate the system ©Serco Reproduction permitted provided the source is acknowledged 53
Process improvement overview 1. Identify needs for usability process improvement • Use the usability maturity model in ISO TR 18529 to assess the current capability of each organisation 2. Make the identified improvements to the software development processes • Introduce simple user-based methods implementing ISO 13407 3. Reassess the usability capability the organisation • Assess the extent of the improvement 4. Identify the cost-benefits of the improvements • Integrate the methods into the documented processes ©Serco Reproduction permitted provided the source is acknowledged 54
Usability Maturity Model HCD. 2 Plan and manage the HCD process Consult stakeholders. Identify and plan user involvement. Select human-centred methods and techniques. Ensure a human-centred approach within the project team. Plan human-centred design process. Manage human-centred process. Champion human-centred activities. Provide support for human-centred design ©Serco Reproduction permitted provided the source is acknowledged 55
Usability Maturity Model HCD. 3 Specify the user and organisational requirements Clarify and document the system goals. Define stakeholders. Assess risk to stakeholders. Define the system. Generate the user and organisational requirements. Set usability objectives. ©Serco Reproduction permitted provided the source is acknowledged 56
Usability Maturity Model HCD. 4 Understand specify the context of use Identify and document user’s tasks. significant user attributes. organisational environment. technical environment. physical environment. ©Serco Reproduction permitted provided the source is acknowledged 57
Usability Maturity Model HCD. 5 Produce design solutions Allocate functions Produce composite task model Produce system design Use existing knowledge to develop design solutions Specify system Develop prototypes Provide user training Provide user support ©Serco Reproduction permitted provided the source is acknowledged 58
Usability Maturity Model HCD. 6 Evaluate designs against requirements Specify and validate context of evaluation. Evaluate early prototypes in order to define the requirements for the system. Evaluate prototypes in order to improve the design. Evaluate the system in order to check that the system requirements have been met. Evaluate the system in order to check that the required practice has been followed. Evaluate the system in use in order to ensure that it continues to meet organisational and user needs. ©Serco Reproduction permitted provided the source is acknowledged 59
l United Kingdom’s tax collection department l Employ over 60, 000 staff in more than 600 local offices l l l Partnership with EDS for software development now running for more than 5 years JAD: Joint Application Development RAD: Rapid Application Development ©Serco Reproduction permitted provided the source is acknowledged 60
Usability capability assessment Formal CMM/SPICE style assessment by Lloyds Register Interviewed 13 stakeholders in twelve 3 -hour sessions over one week. A conventional software process assessment procedure based on SPICE (ISO 15504) was used. Produced a detailed profile and rich information about where improvements would be beneficial. At a feedback meeting on the last day, it provided the basis for an agreed set of improvement activities A second assessment a year later showed significant improvements ©Serco Reproduction permitted provided the source is acknowledged 61
Inland Revenue/EDS Conclusions 4 Significant improvement in all areas of usability maturity 4 IR/EDS now strong in the “core” areas of context, requirements, design and evaluation 4 Evaluation activity rated as outstanding 4 The methods are very cost effective and will be incorporated into the documented development process 4 “A step change in the awareness and practice of human centred design within IR/EDS”. ©Serco Reproduction permitted provided the source is acknowledged 62
IAI LAHAV Background Expertise in Military Aircraft “Avionics Upgrade Programs” Customers Worldwide User needs addressed by a group of Pilots Trial Application: PC-based Mission Planning system ©Serco Reproduction permitted provided the source is acknowledged 63
Usability capability assessment workshop Process improvement workshop One-day workshop Review each process and activity in ISO TR 18529 in detail Score each process as: Not, Partly, Largely or Fully performed Note whether the process is managed a documented part of normal procedures Establish a target level for each process on the same scales A second assessment a year later showed significant improvements ©Serco Reproduction permitted provided the source is acknowledged 64
Trial Application at IAI Bottom Line Positive Feedback from Participants A Definite Improvement in the Development Process Very Cost Effective and Low Cost methods LAHAV decided to incorporate TRUMP techniques in it’s standard development process ©Serco Reproduction permitted provided the source is acknowledged 65
Process improvement conclusions The usability maturity model was a valuable tool for identifying needs for process improvement Inland Revenue: summative assessment requiring three person weeks effort IAI: a simpler formative one-day assessment A very promising way to make major process improvements www. usability. serco. com/trump ©Serco Reproduction permitted provided the source is acknowledged 66
ISO 13407 Conclusions ISO 13407 and ISO TR 18529 provide an internationally-agreed description of good practice in user-centred design The content can be used to specify or evaluate: How usability should be incorporated into the development process Whether a project has applied user centred design Organisational capability in user-centred design Basis for process improvement Appropriate content for education and training courses in user centred design Necessary areas of professional competence: professional accreditation ©Serco Reproduction permitted provided the source is acknowledged 67
www. usabilitynet. org ©Serco Reproduction permitted provided the source is acknowledged 68
Скачать презентацию Formalising usability as part of ISO standards Nigel
9586d9a70d4d4fa9c2756cef30330b5f.ppt