A Software Engineering Institute for the Victorian Software

A Software Engineering Institute for the Victorian Software Industry A Re-useable Case Study*. . . by Assoc. Prof. Karl Reed, FACS, FIE-Aust. , MSc, ARMIT Chair IEEE-Computer Society Tech. Council on Software Engineering Governor, IEEE-Computer Society(1997 -1999, 2000 -2002), Director, Computer Sys. & Software Engineering Board, ACS, Department of Computer Science & Computer Engineering, La Trobe University Hon. Visiting Professor, Middlesex University *A summary presented to the Sheffield-Hallam Univ. Feb. 2001 1

Why Were We Proposing VSEI? Promote government policy in information technology ensuring support for an already successful industry Ensure that the industry has a major research institute Ensure ‘world’s best practice’ is understood and adopted here Twenty years from now a permanent structure delivering an increased competitive edge to a major exporting industry! 2

Australian Political Reality Information Technology Sector --> Primary Industry share of GDP Extensive Gov’t R &D for Primary Sector. . 10 Divisions of CSIRO, State R & D inst. , CRC’s ~15 Govt. -Industry Funded R & D Corps for Primary Industry Estimated relative short-fall ~A$700 M. p. a (£ 300 M. p. a) Few Public sector large-scale “professional” research groups Universities regarding R & D as their domain. . means of cross-subsidising teaching(? ) How does this compare with the UK? 3

What was our strategy? “Forcing” argument based on comparison with Primary Industry and Govt. rhetoric Linkage with industry Credible proposal …. to deliver world’s best practice Twenty years from now a permanent structure delivering an increased competitive edge to a major exporting industry! 4

Goals Guarantee the long term technical viability of a major IT industry sector the software and services industries Provide equivalent R & D support as that available to overseas competitors Provide professional research capability advancing the discipline 5

Industry-Academia Working Party Meeting since May 1997 Chairman Karl Reed, La Trobe University Deputy Chairman Paul Radford, Managing Director Charismatek Working party Silvio Salom, Managing Director Adacel Laurie Lock Lee, Manager Planning and Development BHPIT Robyn Lawrie, Technical Director Charismatek Alex Sawicki, Department of State Development/Multimedia Victoria Gary Stoneham, Marketing Manager MITS Sally Duncan, Project Manager Megatec Barbara O’Brien, Quality Manager Megatec Bill Jacobs, Technical Director TUSC Computer Systems David Cleary, La Trobe University Special advisor Dan Marantz, Cobol Digital Observers The Preston Group Whittle Programming KCS Computer Services Clive Finkelstein, Information Engineering 6

Academic Partners with industry links Karl Reed, Amdahl Project (Case, Metrics, Evolvable Programming) Tharam Dillon, La Trobe University OO, Relibility, Metrics T. Y. Chen, Swinburne Univ. Tech. , IVE/STC HK Testing and Software Quality Paul Bailes, Univ. Queensland. Software Maintenance Centre HK opportunity for participation 7

Not Seeking Silver Bullets We’ve all heard of goto-less programming, structured analysis and design, CASE, object orientation, CMM, SPIN ? ? ? all trumpeted as ‘the solution’ This proposal “a considered holistic response to a series of difficult problems” not a “quick fix”! 8

An Appropriate Solution A Small to Medium Enterprise Bias Only one of its kind in the world, other SEIs client domains generally not the software industry focus on embedded systems, telecom and defense aerospace pre-occupied with process issues Our objectives commercial outcomes, deliverables to industry not just academic Funding for technology transfer 9

World’s Best Practice Aggregated research teams ~ researchers 40 + ~20 PG's Funded technology transfer Industry driven collaboration (including management) Cash driven budget (mostly Government) Single point of control 10

Scale Comparable to World's SEI's Overall A$2 M pa to A$30 M pa (combined Esprit is much larger) Fraunhofer Institute for Experimental Software Engineering DM 20 Mpa Centre (goal) de Recherche de Montreal (CRIM) C$18 M pa (total) C$3 M pa (software engineering) 11

International Funding Models Overall typically less than 50% non-government funding, often as low as 30%, sometimes zero funds often obtained from government sources other than granting agency funding models dominant mode; core funding (up to 70%) guaranteed by government agency with allowances for ramp-up 100% government funding grant based funding 12

International Funding Models Fraunhofer Institute for Experimental Software Engineering Fraunhofer Gesellschaft 40% regional government 10% future goal; non-Fraunhofer funding to be 70% (ramp up conditions apply) joint projects; seek 50% involvement (people preferred) 13

SEIs Around the World a Brief Summary Client domain generally not targeted at the software industry mainly ‘embedded’ systems, telecom and defense aerospace (Korea and Taiwan and some Esprit projects are software industry) Modus Operandi networks, multi-project granting (CNRC, CRIM), academic driven, research driven, captive-client (SEI), in house, large-scale (Fraunhofer), etc 14

SEIs Around the World a Brief Summary Areas all of investigation areas of software engineering. . . Technology transfer seminars to technology trials and ‘leverage’ activities (often funded by agency), experiments (PIEs), intellectual property handover Research styles single project (Esprit), short, medium, longterm, team, consortia, distributed consortia 15

SEIs Around the World a Brief Summary Outcomes process improvement, methodology, tools, some product (Esprit varies enormously), intellectual property Intellectual all property models; shared by partners, client owned 16

SEIs Around the World a Brief Summary Participants (primary funding sources) single agencies (SEI, Fruanhofer, etc), multiple agencies (European SEI), state governments (Italy, Canada, Germany), multi-national programs (Europe), very few consortia with upfront industry funding 17

SEIs Around the World a Brief Summary Research agenda determination ‘agenda setting’; agency attempts to inject new technology and best of breed practice (NSF, SEI, DARPA) ‘responsive’; industry set research agendas ‘academic interventionist’; academia propose research projects ‘joint’; academia and industry ‘empiricist’; agendas derived from study of practice 18

SEIs Around the World Fraunhofer Institute for Experimental Software Engineering Funding Fraunhofer Gesellschaft 40% regional government 10% future goal; non-Fraunhofer funding to be 70% local government 50% of new building special ramp-up conditions apply Scale goal DM 20 M pa 19

SEIs Around the World, Centre de Recherche de Montreal (CRIM) Software Development Tools and Methods (SDTM) Funding about C$3 M pa for software engineering Quebec Scale total CRIM C$18 M pa Client domain embedded systems, aerospace, telecom, electricity supply, software tool builders 20

SEIs Around the World, Centre de Recherche de Montreal (CRIM) Software Development Tools and Methods (SDTM) Modus operandi joint academic-industry projects studies of tools, methods etc for clients specific research projects. participation sought technology monitoring projects seem small 21

SEIs Around the World, Centre de Recherche de Montreal (CRIM) Software Development Tools and Methods (SDTM) Areas of investigation methodology improvement re- and reverse engineering improvement of practice, including metrics, standards and SQA object orientation domain-specific architectures, reuse real-time systems, including formal methods 22

Some ESPRIT Research Projects Legacy areas Assessment Workbench (LAW) (aerospace, nuclear) reengineering, metrics workbench for C legacy code (safety critical systems) symbolic execution, formal methods outcomes prototypes for evaluation and adoption 23

Where Are We? Traditionalist’s View R ! E G ST IN SA IT I D 16 year A wunder-kinder throwing systems / Wold together IN / poorly designed functionality S OView LEM Modernist’s N B 4 rapidly deployed RO functionality ! P systems Emeet customer needs 4 rapid evolution of Tto A being left behind 4 conventional approaches M / Bowsers that are limited / Time-To-Market web-application deployment 4 the old do not understand 24 new the

To many surprises…. !!!(nsf report on s/w research 1998) “F 1. Current software has too many surprises. The sources of surprise are poorly understood. ” “F 2. Key sources of software surprise include immature or poorly integrated software domain sciences, construction (product) principles, and engineering processes. Software research emphases have swung from process to product research, with weak coverage of domain sciences and integration. ” Sources of surprises. . . Real and apparent ambiguity in the means of representation of systems, e. i. Languages (cf 3 pages of c++ with 3 pages of government regulations)(Reed, 2000) 25

No surprises…. !!!(nsf report on s/w research 1998) “F 1. Current software has too many surprises. The sources of surprise are poorly understood. ” Sources of surprises. . . Real and apparent unpredictability in behaviour. . . “Teenagers have less trouble with PC software because they are adept at playing computer games” Charles Wright, editor Melbourne Age “green pages” computer section 2000 “Building ‘bots’ that play computer games with near human competence is not that hard” US researcher in AI…. 26

By way of Illustration. . . Some Contradictions…… and confusion 1. Software Architecture. . ‘not immutable, not always determinable a’priori, multiple versions in one artefact, retrofitable…. Analog with “built” systems not clear. 2. Software Process. . CMM vs fine-grained process independent, Time To Market vs Planned Process, Phase incompletedness, Extreme Programming. 3. Software Process. . . Often mandated, but not followed… few detailed studies similar to production engineering (see Hess) 4. Re-use… not successful, yet components industry emerging 5. Engineering & SE. . Poor choices of analogues from traditional domains, e. g. “immutable components” 27

Some Contradictions…… and confusion (cont’d) 6. SWEBOK. . Organised body of knowledge opposed by leading SE players. 7. Prescriptive Design processes. . . only slowly beginning to appear, perhaps via UML. 8. Requirements Engineering. . . Cannot always be completed in advance. . may be continuous part of the implementation process. . . 9. Software Crisis… yet increasingly, successful large-scale applications are ubiquitous 10. High Quality training for 30 yrs. . Yet each new s/w development wave starts with a blank mind, e. g. web-based computing 11. Documentation matters but. . It’s seldom actually done 28

Approaching Software Developers… / Technico-Commercial Drivers… the linkage è / The goal is to find a high-level, one-line statement of pressing commercial issue that maps directly on to a “technology acquisition” (research) agenda (map idea to common concept base accessible to highest management) Show an economic benefit Be able to show ROI after adoption costs (equipment + training) and productivity losses due to learning curves after adoption. (improved profit) è Show resolution of competitive advantage problems (beat off competitors, maintain market share) è Show new market opportunities due to new products/services è 29

R esearch-Commercial Mapping… Defining Relevance Typical SE Research Agenda Australia ~ 1997 ¶ Impact of developments in 1. Re-engineering and Empirical run -time platforms Studies of s/w Practice, Technico-commercial Drivers ¶ Low-cost and evolving software 2. Tools and Methodologies, and Design Representation, ¶ User Interface Development 3. Re-Use, ¶ Software Productivity 4. Evolving Software, ¶ Performance Predictability 6. Object Oriented Dev. ¶ Software Product Quality Certification 7. Product Quality Measurement 8. Time-to-Market ¶ Time to Market 30 9. Testing

The ANSEI Technico. Comercial Driver to Research agenda mapping / 31

Organisational Proposal Stakeholders Government Research Experience Technical Board CEO Industry Experience Industry Bus Management Board Technical Support Academia Personnel 32

Research Agendas Providing Solutions Over-Arching Goals Our research outcomes are methodologies with the following properties, these in fact become research objectives performance-based, predictable, development of systems to stated performance requirements fine-grained-prescriptive, provide precise prescriptions for steps in development improved technology and expertise for reengineering of existing systems study of existing systems and projects, using re -engineering, design recording 33

Research Agendas Providing Solutions Over-Arching Goals Issues of re-use, re-use intensive and -use based methodologies evolving re software (current agenda of DARPA) object oriented methodologies, will influence, and be influenced engineering of interfaces, part of the methodology program Plus, tools that reflect this! 34

Characterising Time to Market Delivery schedules severely truncated compared to ‘normal’ (less than 50%) How would we deal with this currently? RAD/JAD, What prototyping, ‘super programmers’ would the product be like? slow, unreliable, incomplete expensive! Research goal convert this to a methodology capable of delivering quality products 35

Characterising Time to Market Research for problems a given project a minimum feasible delivery time tdmin -n cost = k tdmin quality greatly reduced competitive position increased! 36

Time to Market Project Attributes Attribute. Standard Development (under control) Time to Market (current situation) Time to Market (target situation) Schedule controlled acceptable high truncated unpredictable uncontrollable high usually low truncated predictable acceptable predictable high Customer satisfaction high usually resentful high Runtime resources Design quality minimal excessive predictable high poor high Cost Quality 37

Research Questions Schedule What is an optimal/reasonable project schedule? Is it feasible to attempt a project within a specific timeframe? Resources How can available human resources best be allocated to ensure projects succeed? 38

Research Questions Factors determining schedules estimating staff quality and experience methodology re-use tools 39

Research Activities & Outcomes Determine existing best practices collaborative work with institute partners from industry and academia Develop models, methods, tools and methodologies Assess models, methods and tools field assessments using real projects internal assessments using institute projects Disseminate knowledge gained field application of methods and tools with institute partners, publication 40

Research Overview Issue Schedule Factors Impact of schedule reduction Risk assessment Approach Data collection Process recording exemplar projects Codification of known models (eg Microsoft) Outcomes Calibrated new estimating techniques Identification of risk indicators Tool assessment Resource allocation Impact on project Data collection Improved planning Process recording methods Impact of decomposition models and parallel implementation Staff Quality Skill identification, fine Improved selection grained classification techniques Experience. Identification and recording Training need of experience identification Project structure 41

Research Overview Issue Methodology Factors Improved productivity Re-use Areas of application Tools CASE versus lightweight Project management and planning Languages Process/design recording Approach Analysis RAD/JAD Conversion of prototyping to product development Identify essential features of usability Application generators Ultra high skill levels How to achieve this? What is re-use? Assess current re-user practices Importance of experience Outcomes High quality prototyping What do we really need here? Tool integration Analysis of existing tools Tool set selection New tools Choice Identify processes Review and recommendation 42 Components Plans Designs Test plans

5. Current State of Knowledge and Practice-MS vs the Rest. . (cont’d) history Dijkstra and THE OS in the 70’s (the lesson? ). . . “Five people as smart as Edgar Dijkstra can do anything” Reed, 1981 The first Unix effort…(but what did it take for product versions) OS/360, PL/I (the lesson? ) (60’s). . Very large teams can build large systems very quickly. . ~ x 1000 person years Total volumes of functionality (e. g. OS/360) may allow partitioning. . TTM issue obscured. . 43

Waterfall S/W Process Model Optimal task allocation, observed <1970 one or two people Feasibility Study Requirements Analysis Systems Analysis / No need for ‘third- party” readable work products! Program Design Programming / “Extreme Unit Test programming”? / Private s/w process? System Integration (Pe. SP compliant? ) System Test 44

Time to Market Conclusion Massive competitive advantage from time to market products with traditional high quality Necessary elements of time to market focussed processes may be accessible Detailed ‘research’ by experienced staff with access to current practice is most likely to be successful 45

The role of re-engineering. . S/W Archaeology and S/W Architecture. . / recovery of standard architectures / identification of s/w construction practices, e. g. shifts from one programming style to another / development of maintainability and evolvability classifications for -§ design methodologies § architectural styles / development of maintainability and evolvability classifications for architectural styles 46

component semantics and concept extraction. . The role of re-engineering. . Architecture issues for the S/W Archaeologist / identification of design approaches which ensure that conceptual architectures are transferred to implementation / identification of standard mappings from conceptual to actual architectures which occur using different design approaches on different problems 47

Conclusion The proposal didn’t win. . beaten by SEA. . (Problem ridden. . Consortium) Is there a need for an SEI in the UK? Surely more than one) Do the arguments “map”? How will the academic community react? (SE research in the US did not stop because CMU got the SEI…) 48

Technico-Commercial Problems Facing the Software Industry Hardware-software run-time environments-adding GUI etc to legacy systems ‘Evolving’ software dynamically ‘Engineered’ user interfaces Software productivity Design to predicted performance Time to market Software quality Testing Object orientation ‘Net-centric’ systems- autonomous 49 communicatings "agents"

Technico-Commercial Problems Facing the Software Industry Developments in hardware-software runtime environments continuing improvements in price/performance + GUI need to migrate legacy systems to these platforms need to add functionality to legacy systems on existing platforms, e. g GUI, client-server Re-engineering of legacy systems to conform to these constraints a major business opportunity 50

Technico-Commercial Problems Facing the Software Industry ‘Evolving’ software dynamically software capable of rapid change to meet existing customers changing needs software capable of rapid change to meet needs of new customers New development methodologies which yield ‘evolving’ software 51

Technico-Commercial Problems Facing the Software Industry ‘Engineered’ user interfaces UI design derived from data presentation and processing + ergonomics Methodology supporting improved UI design 52

Technico-Commercial Problems Facing the Software Industry Software productivity increase productivity to produce PC shrinkwrapped software reduce costs for NC/style delivery platforms, reduce maintenance costs New diagramming systems design recording new methodologies enforcing re-use 53

Technico-Commercial Problems Facing the Software Industry Software Quality Testing and Certification Improve efficiency of specfication-based testing Develop means of code-quality control and asessment Improve design to simplify "testablity" of code 54

Research Agenda Providing Solutions Main Items Our research agendas address the technicocommercial problems reengineering and empirical studies of software practice tools, methodologies and design representations re-use evolving software engineering of user interfaces object oriented development product quality measures time to market 55