Lakatos-style Methods in Automated Reasoning Alison Pease University

Lakatos-style Methods in Automated Reasoning Alison Pease University of Edinburgh Simon Colton Imperial College, London

Spin n Mathematics textbooks are neat n n But mathematical research is scruffy After abortive attempts to prove conjecture n Option 1: show independent of axioms n n E. g. , parallel postulate, axiom of choice Option 2: modify conjecture in order to prove n n Often in the light of a counterexample E. g. , Euler’s theorem, even perfect numbers

Spin 2 n Automated theorem provers n n Not particularly robust or flexible Usually expect to be given a true theorem Always expect theorem to be as intended Very simple example: n n n Child asks ATP: “Are all primes odd? ” ATP says “no” A more flexible ATP would say: n “If you mean all primes except 2 are odd, then yes”

Proofs and Refutations n Imre Lakatos’ famous book n n Gives methods for fixing faulty theorems Uses Euler’s theorem as a running example n n In polyhedra, V – F + E = 2 Has a social setting n n Imagined situation of a teacher and class Respond to new counterexamples encountered n Calls into question what we mean by polyhedra

Ph. D Project of Alison Pease n To implement a model of reasoning n n Has a social aspect n n Implemented as a multi-agent system Has a reasoning aspect n n Based on the notions in proofs and refutations How to fix faulty conjectures Aims: n n Cognitive modelling Philosophical look at creativity Enhance automated theory formation Suggest possible applications to AI techniques

Automated Theory Formation in the HR System n Cycle of invention, induction & deduction: n n Uses generic & specific concept production rules Conjectures are made empirically Uses a heuristic search n n n Form concepts, make conjectures relating the concepts, prove conjectures, assess concepts New concepts are built from best old ones Concrete measures of interestingness Interacts with third party mathematical software n Otter, MACE, Maple, Gap, Math. Web, System. On. TPTP

Enhanced Theory Formation n In other domains of science: n n An 80% conjecture is really quite interesting (just plain wrong in mathematics) HR now makes near conjectures n n Near equivalences: A B Near implications: A B User sets percentage threshold Includes both negative and positive for concept n Working on positives only

Lakatos-Enhanced Automated Theory Formation n n Cycle: invent, induce, repair, prove Additional Implementation n n Lakatos-inspired methods n n Progress measured on two axes What to do in the event of a counterexample Sophistication of the Agency n Teacher and student communication and actions

Lakatos-inspired Methods #1 n Surrender n n n Too many counterexamples to continue Until recently, only option for HR Important to know which ones to surrender n n Number and importance (size? ) of examples which are counterexamples to conjecture considered “Small” counterexamples may be OK

Lakatos-inspired Methods #2 n n Concept barring Given near-equivalence P Q n If all counterexamples, C, are positive for P n n If some counterexamples are positive for P and Q n n n Then find concept X which exactly covers C Negate X and force the concept P ¬ X This produces the exact conjecture: P ¬ X Q Find concept X covering positive counters for P Find concept Y covering positive counters for Q Force production of concepts (i) P ¬ X (ii) Q ¬ Y This leads to conjecture P ¬ X Q ¬ Y Very similar for near-implications n Working on more flexibility in finding concepts

Lakatos-inspired Methods #3 n n Counterexample Barring for P Q If an agent cannot find a concept to cover the (small number of) counters n n Then use entity_disjunct production rule Suppose [a, b] are counters to conjecture n n Invent concept X = n for which n=a n=b Use negate to invent concept P ¬ X Leads to the conjecture P ¬ X Q Split near-equivalences into two near-imps

Lakatos-inspired Methods #4 n n Strategic Withdrawal for P Q If all counters are positives for P n n n Find a concept X which covers most of the positives for both P and Q (different from Q) Make conjectures X Q; X P If some counters positive for P, others for Q n n Find a concept Z which covers most of the examples which are positive for both Make conjectures Z P; Z Q

Other Lakatos Methods n Not implemented yet: n Monster adjusting n n n Lemma incorporation n n Counterexample is re-interpreted Domains like number theory not good for this Looking at steps in proofs to identify problem Proofs and refutations n n Use proofs to suggest counterexamples Fix them using lemma incorporation

Levels of Agent Sophistication n A single teacher, multiple student agents n n All have copies of HR, but different objects of interest Communicate via sockets Build up a global theory via local theories Characterisation of agency n n n Problem: model social process of fixing conjs Knowledge: background to each copy of HR Motivation: to accept/reject/modify conjs Actions: Lakatos methods Communication: sending concepts, conjectures, examples Discussion: directed by teacher keeping group agenda

Complexity of Interactions n Axis of complexity n Max user input n CRC n Teacher uses students as references n Current? n Teacher still decides which conjectures to look at Students decide group agenda and global theory n n Dictatorship – teacher sets problems, specifies methods Students make decisions on method to apply n n Puppet show Democracy via voting Students allowed class discussion n Can ignore the teacher, send requests themselves, etc.

Illustrative Example (Concept Barring) n Three students and teacher n n Integers: 1 -10; 11 -50; 51 -60; Divisors, multiplication; Forced: squares, even num divisors Teacher asks students for best equivalence conjecture Third student supplied this: n n is an integer has even number of divisors n n Has no counterexamples between 51 and 60 Other students supply counters: 1, 4, 9, 16, 25, … n n n And find concept of squares to cover these Formed concept of non-squares Fixed conjecture to be: n n is non-square is has an even number of divisors

Illustrative Example (Counterexample Barring) n n n Two sessions with two students Session 1: All primes except two are odd Session 2: n User forced the concept of n n n One agent said this was true of all integers Conjecture was fixed to: n n n Integers which are the sum of two primes All integers except 2 are the sum of two primes Goldbach’s conjecture Done in anger: see ECAI’ 02 workshop paper

Illustrative Example (Strategic Withdrawal) n Two students given integers 1 to 10 n n Less that or equal to; divisors; digit; *; +; Forced: prime numbers, odd numbers Asked to make near-equivs up to 60% First student: n is prime n is odd n n Second student: 2 (prime, not odd), 1, 9 Found concept for 3, 5, 7 (pos for both sides) n n Odd non-squares Modified conjectures: n n n N is odd n is non-square n is odd [tautology] N is odd n is non-square n is prime [false] Surrender probably better here!!

Conclusions n Reparation techniques may be answer to: n n n Partial model implemented n n n Lack of robustness in ATP (notion of noise) Model given by Lakatos is social in nature Consideration one: reparation methods Consideration two: agency sophistication Lakatos-enhanced theory formation n Advantages over normal theory formation

Further Work n n n Improve along the two axes of development Get examples during autonomous sessions Evaluate the system n n Does it model Lakatos’ ideas? Does it improve theory formation? n n Richer set of conjectures/concepts: is this an improvement? Apply the techniques n n Machine learning: fix faulty learned hypotheses Automated reasoning: fix and prove near-theorems