963076e1a330445c8e451dcc7a8b8fbe.ppt

- Количество слайдов: 46

Evolution, Brains and Multiple Objectives By Jacob Schrum schrum [email protected] utexas. edu

About Me n B. S. from S. U. in 2006 ¨ Majors: Math, Computer Science and German ¨ Honors Thesis w/ Walt Potter: Genetic Algorithms and Neural Networks n Currently Ph. D. student at U. T. Austin ¨ Received M. S. C. S. in 2009 ¨ Neural Networks Research Group: Genetic Algorithms and Neural Networks

Evolution n n Change in allele frequencies in population Alleles = variant gene forms Genes ⇨ traits Traits affect: ¨ Survival ¨ Reproduction n Natural selection favors good traits

Genetic Algorithms n Abstraction of evolution ¨ Genes = bits, integers, reals ¨ Natural selection = fitness function ¨ Mutation = bit flip, integer swap, random perturbation, … ¨ Crossover = parents swap substrings ¨ Other representations, mutation ops, crossover ops, …

Applications Boolean Satisfiability K. A. De Jong and W. M. Spears, “Using Genetic Algorithms to Solve NP-Complete Problems” ICGA 1989

Applications Magic Squares T. Xie and L. Kang, "An evolutionary algorithm for magic squares" CEC 2003

Applications Circuit Design J. D. Lohn and S. P. Colombano, "A circuit representation technique for automated circuit design" EC 3: 3 Sep. 1999

Applications Wing Design/Cost Optimization J. L. Rogers and J. A. Samareh, "Cost Optimization with a Genetic Algorithm" NASA Langley Research Center, RTA 705 -03 -11 -03, October 2000

Applications Traveling Salesman Problem P. Jog, J. Y. Suh, and D. van Gucht. "The effects of population size, heuristic crossover and local improvement on a genetic algorithm for the traveling salesman problem" ICGA 1989.

Applications Resource-Constrained Scheduling S. Hartmann, "A competitive genetic algorithm for resource-constrained project scheduling" NRL 45 1998

Applications Lens Design X. Chen and K. Yamamoto, "Genetic algorithm and its application in lens design", SPIE 1996

Applications Weight Selection for Fixed Neural Networks F. H. F. Leung, H. K. Lam, S. H. Ling and P. K. S. Tam, "Tuning of the structure and parameters of a neural network using an improved genetic algorithm" NN 14: 1 Jan. 2003

Applications What Are Neural Networks?

Artificial Neural Networks Brain = network of neurons n ANN = simple model of brain n ¨ Neurons organized into layers

What Can Neural Networks Do? n In theory, anything! ¨ Universal n Approximation Theorem NNs are function approximators ¨ n In practice, learning is hard ¨ Supervised: Backpropagation ¨ Unsupervised: Self-organizing maps ¨ Reinforcement Learning: Temporal -difference learning and Evolutionary computation

Neuro-Evolution Genetic Algorithms + Neural Networks n Many different network representations n Fixed length string Subpopulations for each hidden layer neuron [1] Evolve topology and weights [2] [1] F. Gomez and R. Miikkulainen, "Incremental Evolution Of Complex General Behavior" Adaptive Behavior 5, 1997. [2] K. O. Stanley and R. Miikkulainen, "Evolving Neural Networks Through Augmenting Topologies" EC 10: 2, 2002.

Constructive Neuroevolution Population of networks w/ no hidden nodes n Random weights and connections n

Constructive Neuroevolution Evaluate, assign fitness n Select the fittest to survive n

Constructive Neuroevolution Fill out population n Crossover and/or cloning n Crossover Clone

Constructive Neuroevolution Random mutations n Perturb weight, add link, splice neuron n No mutation Perturb weight Add link Splice neuron

Constructive Neuroevolution Can add recurrent links as well n Provide a form of memory n

Neuroevolution Applications Double Pole Balancing F. Gomex and R. Miikkulainen, “ 2 -D Pole Balancing With Recurrent Evolutionary Networks” ICANN 1998

Neuroevolution Applications Robot Duel K. O. Stanley and R. Miikkulainen, "Competitive Coevolution through Evolutionary Complexification" JAIR 21, 2004

Neuroevolution Applications Vehicle Crash Warning System N. Kohl, K. Stanley, R. Miikkulainen, M. Samples, and R. Sherony, "Evolving a Real-World Vehicle Warning System" GECCO 2006

Neuroevolution Applications http: //nerogame. org/ Training Video Game Agents K. O. Stanley, B. D. Bryant, I. Karpov, R. Miikkulainen, "Real-Time Evolution of Neural Networks in the NERO Video Game" AAAI 2006

What I Do With Neuroevolution n Discover complex behavior ¨ Multiagent domains ¨ Simulations, robotics, video games Support for multiple modes of behavior n Multiobjective optimization n

n Mutiobjective Optimization Pareto dominance: iff ¨ ¨ n Assumes maximization Want nondominated points n NSGA-II [3] used n ¨ Popular Nondominated EMO method [3] K. Deb, S. Agrawal, A. Pratap, T. Meyarivan, "A Fast Elitist Non-dominated sorting genetic algorithm for multi-objective optimization: NSGA-II" PPSN VI, 2000

Non-dominated Sorting Genetic Algorithm II n n Population P with size N; Evaluate P Use mutation to get P´ size N; Evaluate P´ Calculate non-dominated fronts of {P È P´} size 2 N New population size N from highest fronts of {P È P´}

Evolve Game AI n Game where opponents have multiple objectives ¨ Inflict damage as a group ¨ Avoid damage individually ¨ Stay alive individually n Objectives are contradictory and distinct Opponents take damage from bat Player is knocked back by NPC

Intelligent Baiting Behavior

How to avoid stagnation Some trade-offs are too easy to reach n Focus on difficult objectives n TUG: Targeting Unachieved Goals n ¨ Avoids need for incremental evolution Evolution Hard Objectives

Smaller Team w/ Expert Timing

Multitask Domains Perform separate tasks n Predator/Prey n ¨ Prey: run away ¨ Pred: prevent escape n Front/Back Ramming ¨ Attack with ram on front ¨ Attack with ram on back

Multimodal Networks n One network, multiple policies ¨ Multitask [4] = one mode per task ¨ Mode mutation = network chooses mode to use Multitask Two tasks, two modes Appropriate mode used for task Mode Mutation Start with one mode, mutation adds another Preference neurons control mode choice [4] R. A. Caruana, "Multitask learning: A knowledge-based source of inductive bias" ICML 1993

Multimodal Predator/Prey Behavior Learned with Mode Mutation Runs away in Prey task Corralling behavior in Predator task

Multimodal Front/Back Ramming Behavior Learned with Multitask Efficient front ramming Immediately turn around to attack with back ram

What about “real” domains? n Unreal Tournament 2004 ¨ Commercial video game ¨ Basis for Bot. Prize competition: Bot Turing Test n Placed 2 nd with our bot: UT^2

UT^2 Behavior/Judging Game

Summary n n Neural networks can represent complex behavior Neuroevolution = way to discover this behavior Multiobjective evolution needed in complex domains Success in challenging designed/commercial domains

Questions? E-mail: schrum [email protected] utexas. edu Webpage: http: //www. cs. utexas. edu/~schrum 2/

Auxiliary Slides n Empirical results

Differences for Alternating and Chasing significant with p <. 05