The Past Present and Future of Search-Based Software

The Past, Present and Future of Search-Based Software Engineering for the Next 30 Minutes “There’s plenty of room at the top” John A Clark Dept of Computer Science University of York jac@cs. york. ac. uk SBSE Workshop. Cumberland Lodge, Windsor 15 -16 September 2003

Talk n n n Take a step back and see where there has been action in SBSE. Provide some categories to think about problems. Ask some questions about the subject: n n What is software? (Not as ridiculous as it sounds) What gaps are there? Requirements Specification Architecture Design Code Object Code Traditional stages

Cross-lifecycle activities Types of Activity Level n Design Refinement Reverse Engineering verification Level (n+1) Description Inter-level Activities Validation Intra-level Improvement Activities

How low can you go? n n Object code. Intra-level. People have investigated heuristic search as a means of code optimisation. n n Timing performance. Register usage. Space. Various ‘meaning preserving’ transformations (OK, usually ignoring precision). 1 0 0 1 1 1 0 1 1

Going Down, Around, Up n n n Refine. Code to object code - refinement step is generally referred to as compilation –fully automated. Similarly for Field Programmable Gate Arrays, netlists can be produced automatically from VHDL or Handel-C etc. But room for optimisation here, route and place is a hard task. n n Related working hardware proper – but HW/SW software distinction is increasingly blurred. Reverse. Have seen attempts to map object code graphs onto source code graphs (but no use of heuristic search).

Code Level n Testing – a very considerable amount of automated testing generation n n Significant amount in UK (various); Germany (Daimler Chrysler); And various instances around the world; Also note – some parallel work from the EC community. Code transformation - testability transformations

Architectural Level n n n Components and how they work together. Clustering/modularity work considered architectural level. What are criteria for a good architecture? n n Not an easy question. Model based testing now becoming high profile. n Automated testing form architectures is an opportunity to repeat code based successes at this level?

Specification to Code n Genetic Programming! n n n When the actual solution is a program you want to run. Suspect most of the SBSE community’s perception of computation and software functional correctness is a traditional one. See little reason why the above should not count as automatic `refinement’.

Specification to Architecture n Actually some architectural problems can only be solved practically using heuristic search – e. g. allocating tasks to processors in real-time distributed systems (multiple criteria) n n n May need some task replication on different processors. Timing deadlines to met. Communications overheads. T 1, T 5, T 7 T 2, T 8, T 9 T 3, T 4, T 8 T 1, T 2, T 5, T 7

Specification to Design n n Very few applications here. Security protocols n n Goals of the protocols formalised as statements of beliefs Messages containing beliefs evolved as a (provably correct) refinement. Can we do automated refinement of process algebraic descriptions? Are there possibilities for automated refinement in say Z or B?

Specification to Specification n Are there possibilities for transforming specifications automatically? What would the language for transformation look like? What would ‘fit’ specs look like? n n Needs further software engineering work? Other specification matters: n n Some work on using genetic algorithms to explore large state spaces (deadlock detection and security examples) Small exploratory tests on trying to break synthesised specs (Michael Ernst’s technique to generate specifications and optimisation based code level tests to break them. )

Requirements n n n Not a great deal of work here. Next release problem addressed (how to prioritise incorporation of requirements into releases). And? ? ? ?

Management Tools n n Some work on effort/cost prediction More general data description of more general application.

Issues and Opportunities

Non-functional properties n Software executes on some ‘processor’ n n Timing is an issue of course. n Worst, average, jitter. Memory usage is an issue. Power? n Devices may be highly resource constrained, the power to carry out a task may actually be a crucial factor (pervasive environments) and tradeoffs may be in order Precision? ? ?

Generalised Diagnostics n n Testing – showing there is a fault Debugging – showing where there is a fault. Ideas generalise In huge systems how can you track down what a problem is? n n n What data needs to be collected? What are characteristics signatures of failures? Software engineering? Why not? n n We may be heading for self-diagnosing, selfhealing, …. . systems. Autonomic computing. Run time ‘diagnostics’ clearly of significant importance.

Generalised Stress Testing n Testing – you are trying to ‘break the system’ n n n Lots of work at code level falls into this category: falsification testing, exception generation, safety testing, reuse precondition breaking, timing. Little or no work has been carried out for system level properties. Yet in may ways big systems is where the actions is at these days. n Can we attack quality of service claims?

Generalised Robustness n m-out-of-n schemes are a standard model in the safety domain n n N-version programming also used n n n Protect against hardware failure Get separate teams to develop versions and then run them all, take a vote on results. Protect against implementation error. Diversity is a key concept – but N-version programming is controversial n Lack of independence.

Generalised Robustness n n But diversity seems too good to ditch. Embrace notions of populations (Xin) giving solutions: n n Can 100 poor solutions combine to give good results? How can we enforce diversity? n Severely limit program size. n Enforce different function symbol sets, etc. n More standard ways too. (POOR)100=GOOD ?

General Tools Improvement n n Software is developed in an environment. SBSE include the derivation of better tools: n n Verification tools: n ‘Testing’ tools obviously. n Counter-example generators. n Algebraic simplifiers. n Variable re-orderings for model checkers etc. Better prediction methods and tools.

On the Horizon n n Pervasive computing networks. Late binding service provision n Service provision negociated at run-time. (current SBSE work seems static/off-line) Very large scale IT. Meanwhile in a (several) universe(s) far away….

Quantum and Nanotech n n “There’s plenty of room at the bottom. ” Richard Feynman There are many worlds…. . n n Quantum algorithms: we have only two major high level procedures: n Grover’s search n Shor’s Quantum Discrete Fourier Transform Can we discover others by simulation and GP? n A few researchers have published in this area. Real nano-tech involves computer scientists! Programming nanites is software engineering! General point is there alternative models of computation and other architectures that we should not wholly ignore (even if more standard computing platforms are the principal target).

Conclusions n n Lots of good work. But there are gaps n n n As we go large. As we go small. As we give up! As we go dynamic. As we go higher. “There’s plenty of room at the top. ”