Excellence Around the Globe Test Factory Services Your

Agenda • Why care about quality? • Professional testing in a factory? • How do I to get there? • Why HP? • Who has done it?

Why care about quality?

Why care about quality? If all that can happen is a project delay… • What is the impact of a project delay? − Project impact • A four week delay for a 75 person project = $1. 3 M in cost − 75 people x 900/person/day x 20 days = $1. 3 M − Business impact • In September 2004, HP struggled with SAP consolidation project resulting in a three week delay for their supply-chain software. Results: HP's ordering system became chaotic • $400 M loss in Q 3 revenue • Damaged reputation with customers • Three executives were dismissed • HP's shares dropped 15% on the news

Success of IT projects Source: CHAOS Report 2007, Standish Group International, Inc.

The gap between the specification. . . And the user accceptance test 8 3/18/2018 HP CONFIDENTIAL

The gap between improvements. . . And the retest 9 3/18/2018 HP CONFIDENTIAL

Understanding defects Introduction of defects • • 10 Majority of defects are introduced during the requirements and design phase! However majority of defects are actually detected during user acceptance testing and in production! November 1, 2006 Detection of defects Source: NIST 2002 RTI Project 7007. 011 HP Confidential

Assessing cost of defects This industry average is used as a baseline to calculate the cost savings Industry References: 3 B. Boehm and V. Basili, "Software Defect Reduction Top 10 List, " IEEE Computer, IEEE Computer Society, Vol. 34, No. 1, January 2001, pp. 135 -137.

The HP Quality Methodology (STLC) Idea Business Requirement Verification Design Test preparation Deployed Application Implement Developer Test Functional Test / E 2 E Test System Test User Acc. Test Operation Production Effort and Cost Traditional Cost Planned Cost time

Defect detection rate & cost 120 100 Defect (ISTQB): 80 60 A flaw in a component or system that can cause the component or system to fail to perform its required function, e. g. an incorrect statement or data definition. A defect, if encountered during execution, may cause a failure of the component or system. 20 n pe ra tio O U AT Sy st em Te st st Te eg ra tio n ui re m en In t ts Ve U rif ic ni t. T at io es t n 0 R eq Inserted defect: 40 Defects left without HPQM Defects left with HPQM Defect created (inserted) in a phase R eq u ire TO TA L n tio O pe ra Te AT U em Te st st st tio st n Te Sy m Finding of an Introduced defect ra en t s Ve rif Te st i. . . Detected defect: eg Defect not found in the phase it originated In t Introduced defect: ni t Defect found in the phase it originated U Contained defect: $ 700, 000. 00 $ 600, 000. 00 $ 500, 000. 00 $ 400, 000. 00 $ 300, 000. 00 $ 200, 000. 00 $ 100, 000. 00 $- Correction Costs without HPQM Correction Costs with HPQM

Profesional testing in a factory?

Introducing the HP Quality Model Customer success roadmap defined Level 4 Level 3 Quality Savvy Level 2 Level 1 Level 0 Quality Initialising Quality Agnostic Quality Conscious Product Utility Most Centralised organizations technology, Project-based No documented QA processes or recognition of the need for them Testing, if done at all, is manual Service Utility technology, processes, & personnel The majority of testing, if not all, is still manual licenses, admin and support Centralized technology Centralized service bureau for QA personnel QA process in place, but mainly in individual projects Increasing Maturity 11. 50 – 12. 00 10 m 1 of 2 Centralized methodology and best practices for QA procedures Quality Expert Centre of Excellence Centralized technology & personnel QA processes govern testing and quality initiatives QA is thought leader and has enterprise influence

Testing in the Lifecycle - today IT Supplier Requirements Business Analyst Design Business User Test Execution Production Test Environment Implement Unit Test

Professional Testing IT Supplier Business Analyst Requirements Design Business User Implement Unit Test Quality Factory Acceptance Criteria Manual Test execution Test definition Test data specification Production Roll Out Test automation Test environment

Test Factory Service Catalog (1) Lifecycle Phase Idea • Analysis • Business Impact Analysis • Test Effort calculation • Design 18 Service • Acceptance criteria / Test rule definition • Test environment design • Performance test scenario definition • 3/18/2018 Impact analysis KPI Number of businesses Functions affected • Function change rate • Test Case Change effort • Regression Test effort Number of issues found • Number of issues accepted • Functional coverage per business function

Test Factory Services Catalog (2) Lifecycle Phase Service Implementation • Functional Test – Test environment setup – Driver and Stub KPI • • development – Test data defintion – Test script development • / maintenance – Test execution • Non Functional Test • – Test environment setup – Driver and Stub development • – Test data defintion – Test script development • / maintenance – Test execution 19 3/18/2018 Number of defects found Number of defects accepted Functional coverage per business function Number of concurrent users during performance test System response times Number of concurrent users during Load test

Test Factory Services Catalog (3) Lifecycle Phase Deployment • • • Go Live test AD hoc regression Training environment provisioning • Maintenance 20 Service • • • Impact Analysis Test Case maintance Regression Test • 3/18/2018 KPI Functional coverage during ad hoc regression Change Rate per function • Number of regression test cases • Function coverage during regression test • Average system response time during performance / load test

How to get there?

Building a Test Factory 1 -2 days Time Envisioning Workshop • Present HP as solution Partner • Agreement on milestones for engagement Build blueprint for first test factory line Set up first factory line and run one cycle Application Quality Roll-out Service 10 -20 days • Solution Blueprint & Business Case 30 -100 days Pilot installation >100 days Development

Sample Timeline Q 308 Q 408 Q 109 Q 209 Q 309 Q 409 Phase 1: Preparation Phase 2: Setup the first line Phase 3: Operate the first line Phase 4: Evaluate the results and adjust Phase 5: Setup the next lines

Providing real time status information Dashboards as Program “Cockpit” Application Management Dashboard - Eri. Doc Demand / Project Status Functional Test Status Performance Test Status

Controlling the Factory Consolidated top-down planning & bottom-up execution Controls Flexibility Audit trail Notifications Escalations Governance structures Business rules Visibility Workflow Budgeting/costing Project proposals Issues, risks, scope changes Change management Resource capacity analysis Project scheduling Time tracking HP PPM Center™ Testing issues (defects) Operational work Service desk or 3 rd party integrations

Why HP?

HP Quality Services tri-dimensional approach aligns business with IT • Improve performance Reduce backlog • Ensure quality • Quality Simplify Standardize Optimize Strategic quality methodology 27 3/18/2018 HP CONFIDENTIAL Marketleading HP Software Global delivery capabilities

Simply with HP Quality Methodology • • • 28 Supports the application lifecycle, requirement to deployment Integrates with every development methodology Supports reuse of existing artifacts Incorporates the business Bridges the communication gap between IT and business 3/18/2018 HP CONFIDENTIAL Quality Simplify Standardize Optimize Strategic quality methodology Marketleading HP Software Global delivery capabilities

Blueprint - Based on real world experience

Standardize with HP Software Tools - • • • 30 Market leading product (60% +) Requirements management, riskbased testing, and cycle management First real-time link between business requirements and quality in a single enterprise platform. Delivers risk-based release decisions and end-to-end traceability. Introduces dimension of time and release cycles for agile quality Web 2. 0 Performance validation in a world of AJAX and FLEX Accelerated testing SAP “TAO” HP Quality Management Ecosystem™ 3/18/2018 HP CONFIDENTIAL Quality Simplify Standardize Optimize Strategic quality methodology Marketleading HP Software Global delivery capabilities

Optimize with HP Global Delivery 400+ plus testing professionals across GDAS Centers • Skilled professionals: Test Managers, Test Architects, Test consultants, Test Leads, Manual Test Engineers, Automated Test Engineers and Performance Test engineers, etc… • 150+ certified professionals - CSTE, CSTM, CSTP ISTQB, Brain bench. etc… • Strong people retention & development practices • 31 3/18/2018 HP CONFIDENTIAL Quality Simplify Standardize Optimize Strategic quality methodology Marketleading HP Software Global delivery capabilities

Qualification • Learning and Development Program (Grow@hp) − Certified Tester Foundation Level − Certified Tester Advanced Level (Test Management) − Certified Professional for Requirements Engineering • Certification Program

Who has done it?

Testing Addendum Total Cost of Defect Detection Project Example I – Automative Industrie (Experience) HP approach Cost $/ Defect Requirements Unit Testing Functional Testing Integration Testing System Testing UAT Production Defect Costs (total) Savings (total) 34 Defect % absolute $ 139, 00 65% Classic approach Defect % Cost 5 250 $ 34. 750, 00 $ 977, 00 28% 1500 $ 1. 465. 500, 00 15 750 $ 732. 750, 00 200 $ 195. 400, 00 2% 50 $ 356. 800, 00 60 3000 $21. 408. 000, 00 1% 40 $ 564. 080, 00 20 1000 $14. 102. 000, 00 $ 2. 929. 280, 00 $36. 277. 500, 00 $ 7. 136, 00 $14. 102, 0 0 5% 2500 $ 347. 500, 00 Defect absolute $33. 348. 220, 00 Average Defect Number in comparable Projects (Experience) Standard Software (SAP/Siebel) /high Complexity due to Functionality / Interfaces

KMD Information Technology Services Objective Approach Results • To remain one of Denmark’s • 2004: Initiated IT lifecycle • 96% customer satisfaction with top three IT service providers management plan • Created a Quality Center of • To assure customer satisfaction by improving operational quality Excellence built on HP Quality Center software • Now accommodates 1600+ users and 150 test projects • Established an Operational testing services has led to referrals and new business • 18% revenue growth: 2006 IT improvements • Reduced time to repair from hours to minutes • Reduced risk through − Functional and performance Quality Center powered by HP testing and monitoring Performance Center software − 99. 5% - 100% availability and HP Business Availability Business Outcomes • Rapid root cause analysis Center software saves ~$300 k, or 2200 work • HP Universal CMDB hours, per year integration for 3 rd party help • Able to offer new services to its desk solution and all HP customers management software 35 3/18/2018

Technology for better business outcomes