Portraits in Practice Software Engineering Research Development

Portraits in Practice Software Engineering Research & Development Challenges at Siemens Juergen Kazmeier Brian Berenbach Dan Paulish Marlon Vieira Page 1 Copyright 2007 © Siemens Corporate Research

Siemens has a long tradition of technological innovations. Page 2 Copyright 2007 © Siemens Corporate Research

The rate of innovations is increasing. Share of sales with products… … 5 years and younger … 6 to 10 years old 55% 48% 75% 30% 29% 19% 22% 16% … more than 10 years old 1980 Page 3 1985 6% 2005 Copyright 2007 © Siemens Corporate Research

Siemens is one of the world’s largest software companies. ► ► 60% of Siemens’ business is based on software. ► Page 4 Siemens has more than 30, 000 software developers. Siemens spends more than 3 billion euros per year on software development. Copyright 2007 © Siemens Corporate Research

But, Siemens is not recognized as a software company, since most of our software is embedded. Examples: Automation devices Industrial control systems Automotive components Communication systems Rail systems Medical devices Page 5 Copyright 2007 © Siemens Corporate Research

Siemens at a Glance Information and Communications Automation and Control Power Transportation Medical Lighting Communications Automation and Drives Power Generation Transportation Systems Medical Solutions Sylvania Siemens Business Services Industrial Solutions and Services Power Transmission and Distribution Siemens VDO Automotive Annual Worldwide Sales Siemens Building Technologies Net Income: $2. 8 B World’s 21 st largest company Siemens AG: Worldwide figures for fiscal 20051 (U. S. GAAP 2) 1 Fiscal Year October 1 – September 30 2 Average annual exchange rate for FY 2005: € 1. 00 =$1. 273 Page 6 47, 000 employees $98. 2 billion $6. 6 B dedicated to global R&D 75% of total sales are from products and services developed in the last five years Copyright 2007 © Siemens Corporate Research

Global presence of Research and Development More than 47, 000 R&D employees at 150 locations Page 7 Copyright 2007 © Siemens Corporate Research

Corporate Technology: About 2, 300 Researchers and Developers Worldwide Berlin St. Petersburg Moscow Romsey (RMR) Princeton Siemens Corporate Technology Beijing A Global Network Shanghai Berkeley Tokyo Bangalore Erlangen Munich Page 8 Copyright 2007 © Siemens Corporate Research

Much of our research is done in collaboration with universities. Page 9 Copyright 2007 © Siemens Corporate Research

Software Engineering Challenges at Siemens § Functionality previously realized in electrical or electro-mechanical systems is now being realized in software => bigger, more complex, & more software projects (hundreds of developers, millions of lines of code). § Meeting functional and non-functional requirements is important to business success => restricted hardware resources, real-time performance, safety critical applications. § Multisite development projects. § High quality (i. e. , thoroughly tested, reliable) software is important to business success. Our software engineering methods and technologies must address the increasing scale and complexity of emerging software systems. Page 10 Copyright 2007 © Siemens Corporate Research

There are three major levers for optimizing software development. Processes • Simplification • Standardization • Stabilization Synergies • Architecture • Platforms Structure • Offshoring/ Outsourcing • Site strategy Page 11 Copyright 2007 © Siemens Corporate Research

We address software engineering challenges in R&D programs. Today’s Portraits in Practice focus: Requirements Engineering, Architecture Practices, Testing Processes & Project Management Requirements Engineering Assess and improve software development processes and project management Process From business needs to software design faster, with better quality and transparency for project control and Assessment & oversight Improvement Requirements Engineering (Brian) Project Management SCR SE Quality Engineering Ensure high quality Testing software based (Marlon) systems by Testing, Code Quality Management and Performance Engineering System & Software Quality Page 12 Architecture Practices, GSD (Dan) SW Platform Technologies Building coherent, interoperable and future-proof systems Architecture & Platform Technologies Copyright 2007 © Siemens Corporate Research

Thank you! Contact: Juergen Kazmeier Department Head, Software & Systems Engineering juergen. kazmeier@siemens. com Phone +1 (609) 734 -3610 Page 13 Copyright 2007 © Siemens Corporate Research

Requirements Engineering @ Siemens Brian Berenbach Brian. Berenbach@Siemens. com Page 14 Copyright 2007 © Siemens Corporate Research

The Challenge To provide effective requirements development and management processes for a set of extraordinarily disparate Siemens companies. To raise the level of expertise in requirements engineering throughout Siemens. Page 15 Copyright 2007 © Siemens Corporate Research

For Example… Siemens Automotive Chassis & Car body Fleet Management Infotainment Interior Marine Solutions Special OEM Solutions Power Train Public Transport Solutions Replacement Parts Page 16 Copyright 2007 © Siemens Corporate Research

Customers Provide the Requirements Stakeholder requests from customers in PDF documents & Excel tables Manual Process es Difficulty Managing Change! Page 17 Requirements Database Copyright 2007 © Siemens Corporate Research

Requirements as State Tables Page 18 Again, Difficulty Managing Change! Copyright 2007 © Siemens Corporate Research

Customer Siemens Page 19 Copyright 2007 © Siemens Corporate Research

Unique OEM Problems Customers deliver their requests in varying media Everything is priority 1 Customers feel under no obligation to limit change requests Change requests come in the form of new documents (PDF) and state tables (Excel) with no change matrix. How do you manage change and traces? Requests may be vague because customers expect that “Siemens will figure it out”. Page 20 Copyright 2007 © Siemens Corporate Research

Providing Integrated Medical Solutions Leads to Many Requirement Dependencies Page 21 Copyright 2007 © Siemens Corporate Research

“V” Model for Traceability Impact analysis System requirements Subsystem requirements Component requirements Derivation analysis Stakeholder requirements Are all requirements covered by the layer below? Acceptance test plan Derivation analysis Impact analysis System test plan Integration test plan Component test plan Are all requirements covered by tests? Page 22 Copyright 2007 © Siemens Corporate Research

“V” Model leads to tag based tracing Page 23 Copyright 2007 © Siemens Corporate Research

Tag based training has issues No rationale for traces Difficult to cluster traces Effort rises geometrically with project size Difficulty managing cascading or crosscutting changes Page 24 Copyright 2007 © Siemens Corporate Research

Companies such as Siemens Building Technology build products with distributed (worldwide) stakeholders and development teams. Page 25 Copyright 2007 © Siemens Corporate Research

How to capture functional and non-functional requirements? Desired Single Customer View but… Ø Multiple Regions Ø Multiple Business Units Ø Everything must plug and play TOGETHER Page 26 Copyright 2007 © Siemens Corporate Research

Global Challenges for Siemens Conversion of products to product lines Capturing requirements across regions and across business units Raising the RE skill level of staff world-wide Page 27 Copyright 2007 © Siemens Corporate Research

Challenges One size does NOT fit all! Effectively Controlling Change Managing Scale Managing Customers Page 28 Copyright 2007 © Siemens Corporate Research

…and Some Possible Solutions Traceability Dynamic Tracing Rich Traces New Modeling Languages Complexity, Scale (>10 K Requirements) Model-Driven RE RE Skills Basic RE Courses Advanced Tools. Skill Certification Page 29 Copyright 2007 © Siemens Corporate Research

Thank you! Contact: Brian Berenbach RE Center of Competence Head brian. berenbach@siemens. com Phone +1 (609) 734 -3395 Page 30 Copyright 2007 © Siemens Corporate Research

Methods & Processes for Global Software Development Dan Paulish daniel. paulish@siemens. com Page 31 Copyright 2007 © Siemens Corporate Research

Agenda Motivation for Global Software Development (GSD) Challenges An Approach Lessons Learned Open Issues and Research Questions Page 32 Copyright 2007 © Siemens Corporate Research

Global Software Development (GSD): Motivation Lower development costs • Low-wage countries Capitalizing on a pool of trained workforce • Quest for talent Increased output, reduced time • Improve time-to-market • Round-the-clock development Original motivation was reduced development cost, but there are other reasons for GSD. Market proximity • Specific local expertise • Market acquisition effort Governmental policies and incentives Page 33 Copyright 2007 © Siemens Corporate Research

But, Global Software Development is Difficult. Physical distance Time Difference: 11. 5 hours Page 34 Copyright 2007 © Siemens Corporate Research

Global Software Development Siemens has more than 15 years experience developing software products globally. Today, most Siemens business groups develop software in low-cost countries. Organization and process model for global development divides responsibility between a central headquarters organization & remote development teams. Requirements Domain expertise Management & Process expertise Software Technology engineering expertise Architecture & design Supplier Managers Software engineering expertise Development experience Development plan Acceptance tests Central team Verified by Remote development teams Component requirements Component design Integrated by Verified by Page 35 Code Tests Copyright 2007 © Siemens Corporate Research

Global Studio Project (GSP) Experimental global software development project using university student teams and researchers. Shadows real Siemens global development project, process, & organization. Document processes, best practices, and understanding of how to successfully execute global projects. Page 36 Copyright 2007 © Siemens Corporate Research

GSP Research Goals Identify and document best (and worst) practices for global software development. Identify the prerequisites for successful global software development. Test a global software development reference process. Determine artifacts for commissioning a remote development team. Identify communications necessary for effective global development. Page 37 Copyright 2007 © Siemens Corporate Research

Social Network Analysis (SNA) Affiliations (Sites) SCR Central Team Tech. Univ. Munich CMU Sapphire IIITB Monmouth #5 Monmouth Codicons Monmouth TCT Limerick Ardna. Croise Roles Developer Team Leader Function Manager Remote Supp. Manager Central Supp. Manager Executive Page 38 Copyright 2007 © Siemens Corporate Research

Some Lessons Learned Amount of Required Communications Cultural Differences among Teams Start-up Time for Remote Teams Supplier Managers as the interface to Remote Teams Team Size Agility Page 39 Copyright 2007 © Siemens Corporate Research

Open Issues and Research Questions 1. Given that the technical artifacts that are delivered to the remote component development teams are not adequate in specifying the precise work to be done, in what ways are they deficient? 2. What strategies do the remote teams employ to compensate for the deficiencies found in the received technical artifacts? 3. What are the early warning signs that an issue is imminent? Can communication patterns, for example, between the central and remote teams be used to predict future component integration problems? Page 40 Copyright 2007 © Siemens Corporate Research

Thank you! Contact: Dan Paulish Distinguished Member of Technical Staff daniel. paulish@siemens. com Phone +1 (609) 734 -6579 Page 41 Copyright 2007 © Siemens Corporate Research

Issues and Best Practices in Testing Automation - An Industrial View - Marlon E. Vieira marlon. vieira@siemens. com Page 42 Copyright © Siemens Corporate Research Copyright 2007 © Siemens AG 2007. All rights reserved.

Agenda § Motivation and Some Challenges § Observations on Automation of Testing for Diverse Siemens Projects § Some Observable Best Practices § Our Research Focus § Suggestions on Research Topics § Final Comments Page 43 Copyright 2007 © Siemens Corporate Research

Automation of Testing: Motivation • Pressure to release software to the market (with quality and on time) • Some tests cannot be run without automation • Test teams have time to address bigger test issues • Improve tester job satisfaction Page 44 Copyright 2007 © Siemens Corporate Research

Automation of Testing: Some Challenges to Adopt It in Practice Some Challenges • Global Development • Investment in the automation itself (ROI calculation) • Testers with development skills • Industrial scale solutions Page 45 Copyright 2007 © Siemens Corporate Research

Automation of Testing: Observations (Diverse Siemens Projects) The test process and its automation in practice (Tools) Plan tests plan test Design tests Test Management Tools MBT Tools Execute tests Capture-Replay Simulators/emulator s Custom Frameworks Unit High * Page 46 Component Medium * * Automation level (usage) Integration Medium * Analyze results Test Analysis Tools Defect Classification Metrics Analysis Validation Low * Copyright 2007 © Siemens Corporate Research

Other Observations: Best Practices § Observable Best Practices • Dealing with testing automation as a software development process • Designing tests first before deciding to automate – Test creation (or generation) separated from execution • Applying an incremental approach – Start small and grow - Provide frameworks (libraries) and exercise tests with different data – Not everything is automated (allow exploratory tests) Page 47 Copyright 2007 © Siemens Corporate Research

Other Observations: Emerging Strategies § Emerging Industrial Strategies • Test-driven development (Unit Testing Frameworks) • Model-based testing • Keyword-driven automation Page 48 Copyright 2007 © Siemens Corporate Research

Our Research Focus on Automation of Testing Model Based Testing • Test case generation based on UML Analysis Models • UML 2. 0 Models – Use Case, Activity, Sequence and Statechart diagrams • U 2 TP (UML 2 Testing Profile) Data driven test case generation • Category-Partition Method Architectural issues on flexible/robust testing frameworks • Customizable framework to execute tests in a specific domain Page 49 Copyright 2007 © Siemens Corporate Research

Suggestions for Research: Improvements for Industrial Scale Problems Testability • Design changes (dynamic) • Different environments • Failure treatment (restart and remote test execution) • Visibility and Control (stimulus and result verification) Event-Triggered Real-time Systems • Non-deterministic execution orders • Impact of the execution environment Page 50 Copyright 2007 © Siemens Corporate Research

Final Comments Automation of Testing must fit within project and organizational context. The industrial automation approach needs to be affordable. 1. Usable and Scalable 2. Allow early results 3. Easy Evolution A broad range of academic research is currently being used in the software industry. Examples: – Assertions – Model-Based Testing – Model Checking Page 51 Copyright 2007 © Siemens Corporate Research

Thank you! Contact: Marlon Vieira Program Manager, Software Testing marlon. vieira@siemens. com Phone +1 (609) 734 -3313 Page 52 Copyright 2007 © Siemens Corporate Research