a047d127f554ea13b006c58f43480261.ppt
- Количество слайдов: 9
Prepared for the 2005 Software Assurance Symposium (SAS) DS 1 MSL EO 1 Verifying Autonomous Planning Systems Even the best laid plans need to be verified Gordon Cucullu Gerard Holzmann Rajeev Joshi Benjamin Smith Affiliation: Jet Propulsion Laboratory Margaret Smith (PI)
Importance Autonomous Planning Systems (APSs) determine what the spacecraft / rover / installation should do. Compared to conventional software, they are able to determine this in a wide range of circumstances. As a result, • no need for continual oversight (save on 24/7 operations staff) • more science is done (avoid delay of calling back to Earth) • improved safety (more proactive than just “safe mode”) But because APSs must operate in a wide range of circumstances – far too many to test, even if you could predict them all, how can you trust them to do the right thing? ? ? This work is pursuing a solution! SAS_05_Verifying_Autonomous_Planners_Smith
Consequences of a bad plan Wasted Resources How to get from A to B missed science goal ? out of resourc es SAS_05_Verifying_Autonomous_Planners_Smith
Consequences of a bad plan: Loss of Mission How to get from A to B ? SAS_05_Verifying_Autonomous_Planners_Smith
Solution Challenge: Solution: Assure that all plans generated by the APS are safe for the spacecraft. Replace current empirical testing with model checking. The current empirical testing approach is insufficient because it lacks coverage. Model checking offers exhaustive or measurable test coverage leading to greater confidence in correctness. SPIN Model Checker • • • Logic Model Checker used to formally verify distributed software systems. Development began in 1980 at Bell Labs – publicly distributed source code since 1991 Most widely used logic model checker with over 10, 000 users. Recipient of 2002 System Software Award for 2001 from the Association for Computing Machinery (ACM) Verifies software using a meta language called Promela – requires that system being verified be expressed in Promela SPIN flags deadlocks, unspecified receptions, incompleteness, race conditions and unwarranted assumptions about relative speeds of processes SAS_05_Verifying_Autonomous_Planners_Smith
Approach Empirical Testing (current approach) Testing with the SPIN Model Checker (our work) requirements input model Promela Model Testing plans ~100 plans Manually inspect plans to identify undesirable plans undesirable plan Adjust model to exclude undesirable plan all desirable plans end testing properties of desirable plans Testing y time ited b to lim ed requir mple t sa inspec ns pla nly by ited o nd analyzes lim ory a billions mem essor proc of plans speed undesirable plan (error trace) Adjust model to exclude undesirable plan SAS_05_Verifying_Autonomous_Planners_Smith no errors end testing
Relevance to NASA testi ng software complexity Testing methods must keep pace with the highly complex, autonomous systems we need and are developing. • APS are needed by NASA projects to reduce operations costs and meet science return requirements. • Our work retires an important class of risks inherent to all missions using APS. – we replace an inadequate testing method with a method that has greatly improved and measurable test coverage. SAS_05_Verifying_Autonomous_Planners_Smith
Accomplishments • For DS 4 / Champollion APS model, used model checking to find a deadlock error – 10 activities = exploration of ~ 3 million plans lock: dead memory f out o sample 1 sample 2 image 1 compress data • sample 2 image 2 compress uplink oven 1 off-cool off-warm on oven 2 off-cool on off-warm off-cool • off-cool camera drill location off on hole 1 oven 1 off hole 7 oven 1 • power use memory use Selected Earth Observer 1 as a target mission for application of our work. – 100+ activities = more plans than atoms in the universe!!! Current empirical method of where ~100 plans are tested is woefully inadequate. Our approach: Use model checking to greatly improve testing coverage = billions of plans. – prune the search space through the use of constraints • Currently working on a set of automated tools for automatically converting APS for model checking SAS_05_Verifying_Autonomous_Planners_Smith
Where we are Going • Our goal: to improve APS testing capabilities which have been an impediment to the acceptance of APS for other than experimental use. • How we will get there: – complete implementation of a set of tools to fully automate model checking of APS models – improve coverage from hundreds of test cases to billions of test cases. SAS_05_Verifying_ Autonomous_Planners_Smith