Extreme Hubris Martyn Thomas martyn thomas-associates co uk email

Extreme Hubris Martyn Thomas martyn@thomas-associates. co. uk email me if you want evidence or references for any of the points made. 1 © Martyn Thomas Associates 2005

A typical development …. Internal review Initial requirements Use Cases (consultants) Invitation to tender Ideas from bidders Contractual requirements Final System late, over budget, reduced benefits Changes 2 © Martyn Thomas Associates 2005

A truth universally acknowledged. . . • …. . requirements always change. • Some people argue that this means it isn’t worth spending effort getting precise requirements at the start of a project • The latest manifestation of this absurdity comes as part of e. Xtreme Programming. – “We value responding to change over following a plan” The Agile Manifesto 3 © Martyn Thomas Associates 2005

Requirements and XP 4 © Martyn Thomas Associates 2005

Requirements always change • Some changes are extrinsic: the external needs really change. – For example, the hardware can’t be built as specified. Or certification rules are changed. • Many changes are intrinsic: the client discovers new or different requirements. – often in response to a question from the developers about an omission or ambiguity. 5 © Martyn Thomas Associates 2005

What to do? “Embrace Change: be agile” Ö Break the development into small releases Ö Involve a user representative throughout the project X define each release through user stories, test cases, and metaphors X keep all the documentation in the code Ö rebuild the system often and run regression tests X program in pairs. Group ownership: change anything X don’t write any code that is not needed to pass the current test set. Refactor when needed. – Y 2 K as “re-factoring? ” 6 © Martyn Thomas Associates 2005

No! Complex systems need system-level analysis and design • In XP, systems evolve. Evolution is slow, and succeeds only through widespread failure. Darwin. • Prototyping is a valuable way of exploring and developing user requirements, but prototypes are not deliverable products! • Successful, deterministic development of complex artefacts can only be achieved through – abstract specification – analysis to identify and remove the causes of intrinsic change – strong management of extrinsic change 7 © Martyn Thomas Associates 2005

Abstraction requires precision! • The two most important characteristics of a specification notation are (1) that it should permit problem-oriented abstractions to be expressed … not test cases! • … and (2) that it should have rigorous semantics so that specifications can be analysed for anomalies and to explore system properties. Not stories or metaphors!! “In this connection it might be worthwhile to point out that the purpose of abstracting is not to be vague, but to create a new semantic level in which one can be absolutely precise”. Dijkstra 1972 8 © Martyn Thomas Associates 2005

Strong Change Management 9 © Martyn Thomas Associates 2005

Dependability requires Evidence • A dependable system is one which we can justifiably trust to have the properties we require. • Build a system that you know will have the required properties • Provide evidence that you have succeeded – it is rarely cost-effective to rely on testing for evidence of dependability 10 © Martyn Thomas Associates 2005

MANIFESTO * for Dependable Software Development We hold these truths to be self-evident: 1 The major barrier to dependable software development is irreducible complexity 2 Our predominant tool for mastering complexity is abstraction 3 The purpose of abstraction is “not to be vague, but to create a new semantic level in which one can be absolutely precise” 11 * from manifest = obvious; hence: a statement of the obvious © Martyn Thomas Associates 2005

MANIFESTO for Dependable Software Development 4 We need to be precise to support accurate communication and precise reasoning. 5 Formal specification is therefore fundamental to Dependable Software Development (DSD). 6 Without a formal specification, we cannot know what we are trying to develop, nor can we assess the implications of proposed changes. 12 © Martyn Thomas Associates 2005

MANIFESTO for Dependable Software Development 7 Errors are inevitable in any human activity; DSD therefore requires notations, methods and tools that reduce the probability of making errors, detect any errors as soon as possible after they are made, and support rigorous arguments that major classes of errors have been eliminated. In this way, DSD reduces costs, reduces risks, and enhances quality. 13 © Martyn Thomas Associates 2005

MANIFESTO for Dependable Software Development 8 Implementation languages should be strongly typed, unambiguous, analysable, and supported by deep static analysis tools. Lowry 1969. 9 DFD requires mature engineering processes, competent and experienced staff, and rigorous quality management. 10 The evidence for dependability that can be provided by testing is generally weak and 14 almost never cost-effective. © Martyn Thomas Associates 2005

MANIFESTO for Dependable Software Development 11 At 99% confidence it is almost always absurd to claim a lower pfh than 10 -4 for any system containing software. With one exception: zero. 12 The Safety Integrity Levels in IEC 61508 and elsewhere are therefore absurd. We need new DSD standards based on formal reasoning about specified properties, and incorporating the principles of this Manifesto. 15 © Martyn Thomas Associates 2005

The Path to DSD • Establish mature, auditable processes (ISO 9001, CMM Level 3 etc). • Introduce formal specification of all important system properties. • Introduce rigorous design leading to implementation in a language that supports strong static analysis. • Write reasoned dependability arguments (eg safety cases) based on the evidence from analysis. • Do not tolerate defects: investigate root causes and improve the process. 16 © Martyn Thomas Associates 2005