Software Engineering Chapter 3 Critical systems Ku-Yaw Chang

Software Engineering Chapter 3 Critical systems Ku-Yaw Chang canseco@mail. dyu. edu. tw Assistant Professor Department of Computer Science and Information Engineering Da-Yeh University Ku-Yaw Chang Software Engineering

Objectives Understand that in a critical system, system failure can have severe human or economic consequences Understand four dimensions of system dependability: availability, reliability, safety and security Understand that to achieve dependability you need to avoid mistakes during the development of a system, to detect and remove errors when the system is in use and to limit the damage caused by operational failures Ku-Yaw Chang Software Engineering 2

Preamble System failures n n Cause inconvenience but no serious, long-term damage Result in significant economic losses, physical damage or threats to human life Critical systems Three main types n Safety-critical systems Injury, loss of life, serious environmental damage e. g. chemical manufacturing plant n Mission-critical systems Failure of goal-directed activity e. g. navigational system for a spacecraft n Business-critical systems Very high costs e. g. customer accounting system in a bank Ku-Yaw Chang Software Engineering 3

Preamble Dependability n Cover related system attributes Availability Reliability n Safety Security The most important emergent property of a critical system Systems that are unreliable, unsafe or insecure are often rejected by their users System failures may be enormous Untrustworthy systems may cause information loss Ku-Yaw Chang Software Engineering 4

Preamble Trusted methods and techniques must be used n n Well-tried techniques rather than new techniques Developers are naturally conservative Expensive software engineering techniques may sometimes be used n Costs of verification and validation are usually very high – more than 50% of the total cost Most are socio-technical systems n System operators can Help recover from problems Cause problems if they make mistakes Ku-Yaw Chang Software Engineering 5

Contents 3. 1 A simple safety-critical system 3. 2 System dependability 3. 3 Availability and reliability 3. 4 Safety 3. 5 Security 3. 6 Exercises Ku-Yaw Chang Software Engineering 6

Diabetes A common condition where the human pancreas is unable to produce sufficient quantities of a hormone called insulin Insulin metabolizes glucose in the blood n Low levels of blood glucose (too much insulin) Temporary brain malfunctioning, unconsciousness and death n High levels of blood glucose (too little insulin) Eye damage, kidney damage and heart problems Miniaturized sensors n Automated insulin delivery systems Monitor blood sugar level Deliver appropriate dose of insulin when required Ku-Yaw Chang Software Engineering 7

A software-controlled insulin pump Ku-Yaw Chang Software Engineering 8

Data-flow model of the insulin pump Ku-Yaw Chang Software Engineering 9

Contents 3. 1 A simple safety-critical system 3. 2 System dependability 3. 3 Availability and reliability 3. 4 Safety 3. 5 Security 3. 6 Exercises Ku-Yaw Chang Software Engineering 10

System dependability A property equating to its trustworthiness n The degree of user confidence that the system Will operate as they expect Will not ‘fail’ in normal use Four principal dimensions to dependability n Availability To deliver services when requested n Reliability To delver services as specified n Safety To operate without catastrophic failure n Security To protect itself against accidental or deliberate intrusion Ku-Yaw Chang Software Engineering 11

Dimensions of dependability Ku-Yaw Chang Software Engineering 12

Other dependability properties Reparability n Reflects the extent to which the system can be repaired in the event of a failure Maintainability n Reflects the extent to which the system can be adapted to new requirements Survivability n Reflects the extent to which the system can deliver services while under hostile attack Error tolerance n Reflects the extent to which user input errors can be avoided and tolerated Ku-Yaw Chang Software Engineering 13

System dependability A trade-off between system performance and system dependability n n High dependability can only be achieved at the expense of system performance Dependable software includes extra, often redundant, code Increasing the dependability can significantly increase development costs n Additional design, implementation and validation costs Ku-Yaw Chang Software Engineering 14

Cost/dependability curve Ku-Yaw Chang Software Engineering 15

Contents 3. 1 A simple safety-critical system 3. 2 System dependability 3. 3 Availability and reliability 3. 4 Safety 3. 5 Security 3. 6 Exercises Ku-Yaw Chang Software Engineering 16

Availability and reliability Reliability n The probability that the system’s services will be correctly delivered as specified Availability n The probability that the system will be up and running to deliver these services to users when they request them Both properties are closely related n Availability is more critical than reliability A telephone exchange switch n Availability depends on The system itself The time needed to repair the faults Ku-Yaw Chang Software Engineering 17

Practical problems Different environments n an office environment vs. a university environment Human perceptions and patterns n Unreliable windscreen wipers in a car may be irrelevant in a dry climate Severity of failure or consequences of unavailability n n A failure of initialization in the engine management software An engine that cuts out while they are driving Ku-Yaw Chang Software Engineering 18

Reliability terminology Ku-Yaw Chang Software Engineering 19

Faults and failures Failures n Usually a result of system errors Derived from system faults Errors n n Do not necessarily lead to system failures Can be corrected by built-in error detection and recovery Faults n n Do not necessarily result in system errors May be transient and ‘corrected’ before an error arises Ku-Yaw Chang Software Engineering 20

Reliability achievement Fault avoidance n Development techniques are used that either minimize the possibility of mistakes or trap mistakes before they result in the introduction of system faults Fault detection and removal n Verification and validation techniques that increase the probability of detecting and correcting errors before the system goes into service are used Fault tolerance n Run-time techniques are used to ensure that system faults do not result in system errors and/or that system errors do not lead to system failures Ku-Yaw Chang Software Engineering 21

Reliability modeling Model a software system as an input-output mapping n Some inputs will result in erroneous outputs The reliability of the system n The probability that a particular input will lie in the set of inputs that cause erroneous outputs Different people will use the system in different ways n The probability is not a static system attribute but depends on the system’s environment Ku-Yaw Chang Software Engineering 22

Input/output mapping Ku-Yaw Chang Software Engineering 23

Reliability perception Ku-Yaw Chang Software Engineering 24

Reliability improvement Removing X% of the faults in a system will not necessarily improve the reliability by X%. n A study at IBM showed that removing 60% of product defects resulted in a 3% improvement in reliability Program defects may be in rarely executed sections of the code so may never be encountered by users. n Removing these does not affect the perceived reliability A program with known faults may therefore still be seen as reliable by its users. n Deliberately avoid using system features that can cause problems Ku-Yaw Chang Software Engineering 25

Contents 3. 1 A simple safety-critical system 3. 2 System dependability 3. 3 Availability and reliability 3. 4 Safety 3. 5 Security 3. 6 Exercises Ku-Yaw Chang Software Engineering 26

Safety A property of a system n The ability to operate, normally or abnormally, without danger of causing human injury or death and without damage to the system’s environment Increasingly important to consider software safety n More and more devices incorporate software-based control systems Safety requirements are exclusive requirements n They exclude undesirable situations rather than specify required system services Ku-Yaw Chang Software Engineering 27

Safety-critical systems n Systems where it is essential that system operation is always safe Two classes n Primary safety-critical software Software embedded as a controller in a system Malfunctioning cause a hardware malfunction n n Result in human injury or environmental damage Secondary safety-critical software Indirectly result in injury n Ku-Yaw Chang e. g. , a medical database holding details of drugs administered to patients Software Engineering 28

Safety and reliability are related but distinct n In general, reliability and availability are necessary but not sufficient conditions for system safety Reliability n Bs concerned with conformance to a given specification and delivery of service Safety n Be concerned with ensuring system cannot cause damage Irrespective of whether or not it conforms to its specification Ku-Yaw Chang Software Engineering 29

Unsafe reliable systems Specification errors n If the system specification is incorrect then the system can behave as specified but still cause an accident Hardware failures generating spurious inputs n Hard to anticipate in the specification Context-sensitive commands i. e. issuing the right command at the wrong time n Often the result of operator errors Ku-Yaw Chang Software Engineering 30

Safety terminology Ku-Yaw Chang Software Engineering 31

Safety achievement Hazard avoidance n The system is designed so that some classes of hazard simply cannot arise. Hazard detection and removal n The system is designed so that hazards are detected and removed before they result in an accident Damage limitation n The system includes protection features that minimize the damage that may result from an accident Ku-Yaw Chang Software Engineering 32

Normal accidents Accidents in complex systems n n Rarely have a single cause A fundamental principle of safe systems design Be resilient to a single point of failure Impossible to anticipate all possible combinations of system malfunction n Accidents are an inevitable part of using complex systems Ku-Yaw Chang Software Engineering 33

Contents 3. 1 A simple safety-critical system 3. 2 System dependability 3. 3 Availability and reliability 3. 4 Safety 3. 5 Security 3. 6 Exercises Ku-Yaw Chang Software Engineering 34

Security A system attribute n The ability to protect itself from external attacks that may be accident or deliberate Becoming increasingly important as more and more systems are connected to the Internet An essential pre-requisite for availability, reliability and safety Errors can lead to security loopholes n n Not respond to unexpected inputs Array bounds are not checked Programs in C Ku-Yaw Chang Software Engineering 35

Three types of damage Denial of service n Normal services become unavailable Corruption of programs or data n Be altered in an unauthorized way Disclosure of confidential information n Confidential information may be exposed to unauthorized people Ku-Yaw Chang Software Engineering 36

Security terminology Ku-Yaw Chang Software Engineering 37

Security assurance Vulnerability avoidance n n The system is designed so that vulnerabilities do not occur e. g. No external network connection Attack detection and neutralization n n Attacks on vulnerabilities are detected and removed before they result in an exposure e. g. Find and remove viruses before they infect a system Exposure limitation n n Adverse consequences of a successful attack are minimized e. g. A backup policy allows damaged information to be restored Ku-Yaw Chang Software Engineering 38

Security Vast majority of vulnerabilities n Human failings (rather than technical problems) Easy-to-guess passwords Write down passwords in places where they can be found System administrators make errors n n Setting up access control or configuration files Forget to install or use protection software Take a socio-technical perspective n Not just about their technical characteristics Ku-Yaw Chang Software Engineering 39

Contents 3. 1 A simple safety-critical system 3. 2 System dependability 3. 3 Availability and reliability 3. 4 Safety 3. 5 Security 3. 6 Exercises Ku-Yaw Chang Software Engineering 40

Exercises 3. 1 3. 8 3. 10 Ku-Yaw Chang Software Engineering 41

The End Ku-Yaw Chang Software Engineering