Скачать презентацию Automating the Processes of Inference and Inquiry Kevin Скачать презентацию Automating the Processes of Inference and Inquiry Kevin

87eb0a77f7cf680ce7637c30c9ef4156.ppt

  • Количество слайдов: 45

Automating the Processes of Inference and Inquiry Kevin H. Knuth University at Albany Kevin Automating the Processes of Inference and Inquiry Kevin H. Knuth University at Albany Kevin H Knuth Game Theory 2009

Describing the World Kevin H Knuth Game Theory 2009 Describing the World Kevin H Knuth Game Theory 2009

States apple banana cherry states of a piece of fruit picked from my grocery States apple banana cherry states of a piece of fruit picked from my grocery basket Kevin H Knuth Game Theory 2009

Statements (States of Knowledge) { a, b, c } powerset { a, b } Statements (States of Knowledge) { a, b, c } powerset { a, b } { a, c } { b, csubset } inclusion a b states of a piece of fruit c {a} {b} {c} statements about a piece of fruit statements describe potential states Kevin H Knuth Game Theory 2009

Implication { a, b, c } { a, b } { a, c } Implication { a, b, c } { a, b } { a, c } {a} { b, c } implies {b} {c} statements about a piece of fruit ordering encodes implication Kevin H Knuth Game Theory 2009

Inference { a, b, c } { a, b } { a, c } Inference { a, b, c } { a, b } { a, c } {a} {b} Quantify to what degree { b, knowing that the system is in c} one of three states {a, b, c} { c }implies knowing that it is in some other set of states statements about a piece of fruit inference works backwards Kevin H Knuth Game Theory 2009

Quantification Kevin H Knuth Game Theory 2009 Quantification Kevin H Knuth Game Theory 2009

Quantification quantify the partial order = assign real numbers to the elements { a, Quantification quantify the partial order = assign real numbers to the elements { a, b, c } { a, b } { a, c } {a} {b} { b, c } {c} Any quantification must be consistent with the lattice structure. Otherwise, it does not quantify the partial order! Kevin H Knuth Game Theory 2009

Local Consistency Any general rule must hold for special case Look at special cases Local Consistency Any general rule must hold for special case Look at special cases to constrain general ru We enforce local consistency x y This implies that: I Kevin H Knuth Game Theory 2009

Associativity of Join V Write the same element two different ways This implies that: Associativity of Join V Write the same element two different ways This implies that: Kevin H Knuth Game Theory 2009

Associativity of Join V Write the same element two different ways This implies that: Associativity of Join V Write the same element two different ways This implies that: The general solution (Aczel) is: DERIVATION OF THE SUMMATION AXIOM IN MEASURE THEORY! (Knuth, 2003, 2009) Kevin H Knuth Game Theory 2009

Valuation I VALUATION x y Kevin H Knuth Game Theory 2009 Valuation I VALUATION x y Kevin H Knuth Game Theory 2009

General Case x y x y z Kevin H Knuth Game Theory 2009 General Case x y x y z Kevin H Knuth Game Theory 2009

General Case x y x y z Kevin H Knuth Game Theory 2009 General Case x y x y z Kevin H Knuth Game Theory 2009

General Case x y x y z Kevin H Knuth Game Theory 2009 General Case x y x y z Kevin H Knuth Game Theory 2009

General Case x y x y z Kevin H Knuth Game Theory 2009 General Case x y x y z Kevin H Knuth Game Theory 2009

SUM RULE symmetric form (self-dual) Kevin H Knuth Game Theory 2009 SUM RULE symmetric form (self-dual) Kevin H Knuth Game Theory 2009

Lattice Products x = Direct (Cartesian) product of two spaces Kevin H Knuth Game Lattice Products x = Direct (Cartesian) product of two spaces Kevin H Knuth Game Theory 2009

DIRECT PRODUCT RULE The lattice product is associative After the sum rule, the only DIRECT PRODUCT RULE The lattice product is associative After the sum rule, the only freedom left is rescaling Kevin H Knuth Game Theory 2009

Context and Bi-Valuations BI-VALUATION I Valuation Bi-Valuation Context i is explicit Measure of x Context and Bi-Valuations BI-VALUATION I Valuation Bi-Valuation Context i is explicit Measure of x with respect to Context i is implicit Bi-valuations generalize lattice inclusion to degrees of inclusion Kevin H Knuth Game Theory 2009

Context Explicit Sum Rule Direct Product Rule Kevin H Knuth Game Theory 2009 Context Explicit Sum Rule Direct Product Rule Kevin H Knuth Game Theory 2009

Associativity of Context = Kevin H Knuth Game Theory 2009 Associativity of Context = Kevin H Knuth Game Theory 2009

CHAIN RULE c b a Kevin H Knuth Game Theory 2009 CHAIN RULE c b a Kevin H Knuth Game Theory 2009

Extending the Chain Rule Since x x and x x y, w(x|x)=1 and w(x Extending the Chain Rule Since x x and x x y, w(x|x)=1 and w(x y |x)=1 x y y x x y Kevin H Knuth Game Theory 2009

Extending the Chain Rule y x x y z x y z Kevin H Extending the Chain Rule y x x y z x y z Kevin H Knuth Game Theory 2009

Extending the Chain Rule y x x y z x y z Kevin H Extending the Chain Rule y x x y z x y z Kevin H Knuth Game Theory 2009

Extending the Chain Rule y x x y z x y z Kevin H Extending the Chain Rule y x x y z x y z Kevin H Knuth Game Theory 2009

Extending the Chain Rule y x x y z x y z Kevin H Extending the Chain Rule y x x y z x y z Kevin H Knuth Game Theory 2009

Extending the Chain Rule y x x y z x y z Kevin H Extending the Chain Rule y x x y z x y z Kevin H Knuth Game Theory 2009

Constraint Equations (Knuth, Max. Ent 2009) Sum Rule Direct Product Rule Kevin H Knuth Constraint Equations (Knuth, Max. Ent 2009) Sum Rule Direct Product Rule Kevin H Knuth Game Theory 2009

Commutativity leads to Bayes Theorem… Bayes Theorem involves a change of context Kevin H Commutativity leads to Bayes Theorem… Bayes Theorem involves a change of context Kevin H Knuth Game Theory 2009

Automated Learning Kevin H Knuth Game Theory 2009 Automated Learning Kevin H Knuth Game Theory 2009

Application to Statements Applied to the lattice of statements our bi-valuation quantifies degrees of Application to Statements Applied to the lattice of statements our bi-valuation quantifies degrees of implication M represents a statement about our MODEL D represents a statement about our observed DATA T is the TRUISM (what we assume to be true) Kevin H Knuth Game Theory 2009

Change of Context = Learning Re-arranging the terms highlights the learning process Updated state Change of Context = Learning Re-arranging the terms highlights the learning process Updated state of knowledge DATA dependent term about the MODEL Initial state of knowledge about the MODEL Kevin H Knuth Game Theory 2009

Information Gain Kevin H Knuth Game Theory 2009 Information Gain Kevin H Knuth Game Theory 2009

Predict a Measurement Value Predict the measurement value De we would expect to obtain Predict a Measurement Value Predict the measurement value De we would expect to obtain by measuring at some position (xe, ye). We rely on our previous data D, and hypothesized model M: Using the product rule Kevin H Knuth Game Theory 2009

Select an Experiment Probability theory is not sufficient to select an optimal experiment. Instead, Select an Experiment Probability theory is not sufficient to select an optimal experiment. Instead, we rely on decision theory, where U(. ) is an utility function Using the Shannon Information as the Utility function Kevin H Knuth Game Theory 2009

Maximum Information Gain By writing the joint entropy of the model M and the Maximum Information Gain By writing the joint entropy of the model M and the predicted measurement De, two different ways, one can show that (Loredo 2003) We choose the experiment that maximizes the entropy of the distribution of predicted measurements. Other cost functions will lead to other results Knuth Kevin H Game Theory 2009

Robotic Scientists This robot is equipped with a light sensor. It is to locate Robotic Scientists This robot is equipped with a light sensor. It is to locate and characterize a white circle on a black playing field with as few measurements as possible. Kevin H Knuth Game Theory 2009

Initial Stage BLUE: Inference Engine generates samples from space of polygons / c COPPER: Initial Stage BLUE: Inference Engine generates samples from space of polygons / c COPPER: Inquiry Engine computes entropy map of predicted measur With little data, the hypothesized shapes are extremely varied and it is good to look just about Knuth Kevin H Game Theory 2009 anywhere

After Several Black Measurements With several black measurements, the hypothesized shapes become smaller. Exploration After Several Black Measurements With several black measurements, the hypothesized shapes become smaller. Exploration is naturally H Knuth Kevin Game Theory 2009 focused on unexplored regions

After One White Measurement A positive result naturally focuses exploration around promising region Kevin After One White Measurement A positive result naturally focuses exploration around promising region Kevin H Knuth Game Theory 2009

After Two White Measurements A second positive result naturally focuses exploration around the edges After Two White Measurements A second positive result naturally focuses exploration around the edges Kevin H Knuth Game Theory 2009

After Many Measurements Edge exploration becomes more pronounced as data accumulates. Kevin H Knuth After Many Measurements Edge exploration becomes more pronounced as data accumulates. Kevin H Knuth This is all handled naturally by the entropy! Game Theory 2009

Special Thanks to: John Skilling Janos Aczél Ariel Caticha Keith Earle Philip Goyal Steve Special Thanks to: John Skilling Janos Aczél Ariel Caticha Keith Earle Philip Goyal Steve Gull Jeffrey Jewell Carlos Rodriguez Phil Erner Scott Frasso Rotem Gutman Nabin Malakar Kevin H Knuth Game Theory 2009