Evolution, Brains and Multiple Objectives By Jacob Schrum schrum 2@cs. utexas. edu
About Me n B. S. from S. U. in 2006 ¨ Majors: Math, Computer Science and German ¨ Honors Thesis w/ Walt Potter: Genetic Algorithms and Neural Networks n Currently Ph. D. student at U. T. Austin ¨ Received M. S. C. S. in 2009 ¨ Neural Networks Research Group: Genetic Algorithms and Neural Networks
Evolution n n Change in allele frequencies in population Alleles = variant gene forms Genes ⇨ traits Traits affect: ¨ Survival ¨ Reproduction n Natural selection favors good traits
Genetic Algorithms n Abstraction of evolution ¨ Genes = bits, integers, reals ¨ Natural selection = fitness function ¨ Mutation = bit flip, integer swap, random perturbation, … ¨ Crossover = parents swap substrings ¨ Other representations, mutation ops, crossover ops, …
Applications Boolean Satisfiability K. A. De Jong and W. M. Spears, “Using Genetic Algorithms to Solve NP-Complete Problems” ICGA 1989
Applications Magic Squares T. Xie and L. Kang, "An evolutionary algorithm for magic squares" CEC 2003
Applications Circuit Design J. D. Lohn and S. P. Colombano, "A circuit representation technique for automated circuit design" EC 3: 3 Sep. 1999
Applications Wing Design/Cost Optimization J. L. Rogers and J. A. Samareh, "Cost Optimization with a Genetic Algorithm" NASA Langley Research Center, RTA 705 -03 -11 -03, October 2000
Applications Traveling Salesman Problem P. Jog, J. Y. Suh, and D. van Gucht. "The effects of population size, heuristic crossover and local improvement on a genetic algorithm for the traveling salesman problem" ICGA 1989.
Applications Resource-Constrained Scheduling S. Hartmann, "A competitive genetic algorithm for resource-constrained project scheduling" NRL 45 1998
Applications Lens Design X. Chen and K. Yamamoto, "Genetic algorithm and its application in lens design", SPIE 1996
Applications Weight Selection for Fixed Neural Networks F. H. F. Leung, H. K. Lam, S. H. Ling and P. K. S. Tam, "Tuning of the structure and parameters of a neural network using an improved genetic algorithm" NN 14: 1 Jan. 2003
Applications What Are Neural Networks?
Artificial Neural Networks Brain = network of neurons n ANN = simple model of brain n ¨ Neurons organized into layers
What Can Neural Networks Do? n In theory, anything! ¨ Universal n Approximation Theorem NNs are function approximators ¨ n In practice, learning is hard ¨ Supervised: Backpropagation ¨ Unsupervised: Self-organizing maps ¨ Reinforcement Learning: Temporal -difference learning and Evolutionary computation
Neuro-Evolution Genetic Algorithms + Neural Networks n Many different network representations n Fixed length string Subpopulations for each hidden layer neuron [1] Evolve topology and weights [2] [1] F. Gomez and R. Miikkulainen, "Incremental Evolution Of Complex General Behavior" Adaptive Behavior 5, 1997. [2] K. O. Stanley and R. Miikkulainen, "Evolving Neural Networks Through Augmenting Topologies" EC 10: 2, 2002.
Constructive Neuroevolution Population of networks w/ no hidden nodes n Random weights and connections n
Constructive Neuroevolution Evaluate, assign fitness n Select the fittest to survive n
Constructive Neuroevolution Fill out population n Crossover and/or cloning n Crossover Clone
Constructive Neuroevolution Random mutations n Perturb weight, add link, splice neuron n No mutation Perturb weight Add link Splice neuron
Constructive Neuroevolution Can add recurrent links as well n Provide a form of memory n
Neuroevolution Applications Double Pole Balancing F. Gomex and R. Miikkulainen, “ 2 -D Pole Balancing With Recurrent Evolutionary Networks” ICANN 1998
Neuroevolution Applications Robot Duel K. O. Stanley and R. Miikkulainen, "Competitive Coevolution through Evolutionary Complexification" JAIR 21, 2004
Neuroevolution Applications Vehicle Crash Warning System N. Kohl, K. Stanley, R. Miikkulainen, M. Samples, and R. Sherony, "Evolving a Real-World Vehicle Warning System" GECCO 2006
Neuroevolution Applications http: //nerogame. org/ Training Video Game Agents K. O. Stanley, B. D. Bryant, I. Karpov, R. Miikkulainen, "Real-Time Evolution of Neural Networks in the NERO Video Game" AAAI 2006
What I Do With Neuroevolution n Discover complex behavior ¨ Multiagent domains ¨ Simulations, robotics, video games Support for multiple modes of behavior n Multiobjective optimization n
n Mutiobjective Optimization Pareto dominance: iff ¨ ¨ n Assumes maximization Want nondominated points n NSGA-II [3] used n ¨ Popular Nondominated EMO method [3] K. Deb, S. Agrawal, A. Pratap, T. Meyarivan, "A Fast Elitist Non-dominated sorting genetic algorithm for multi-objective optimization: NSGA-II" PPSN VI, 2000
Non-dominated Sorting Genetic Algorithm II n n Population P with size N; Evaluate P Use mutation to get P´ size N; Evaluate P´ Calculate non-dominated fronts of {P È P´} size 2 N New population size N from highest fronts of {P È P´}
Evolve Game AI n Game where opponents have multiple objectives ¨ Inflict damage as a group ¨ Avoid damage individually ¨ Stay alive individually n Objectives are contradictory and distinct Opponents take damage from bat Player is knocked back by NPC
Intelligent Baiting Behavior
How to avoid stagnation Some trade-offs are too easy to reach n Focus on difficult objectives n TUG: Targeting Unachieved Goals n ¨ Avoids need for incremental evolution Evolution Hard Objectives
Smaller Team w/ Expert Timing
Multitask Domains Perform separate tasks n Predator/Prey n ¨ Prey: run away ¨ Pred: prevent escape n Front/Back Ramming ¨ Attack with ram on front ¨ Attack with ram on back
![Multimodal Networks n One network, multiple policies ¨ Multitask [4] = one mode per](https://present5.com/presentation/963076e1a330445c8e451dcc7a8b8fbe/image-34.jpg "Multimodal Networks n One network, multiple policies ¨ Multitask [4] = one mode per")
Multimodal Networks n One network, multiple policies ¨ Multitask [4] = one mode per task ¨ Mode mutation = network chooses mode to use Multitask Two tasks, two modes Appropriate mode used for task Mode Mutation Start with one mode, mutation adds another Preference neurons control mode choice [4] R. A. Caruana, "Multitask learning: A knowledge-based source of inductive bias" ICML 1993
Multimodal Predator/Prey Behavior Learned with Mode Mutation Runs away in Prey task Corralling behavior in Predator task
Multimodal Front/Back Ramming Behavior Learned with Multitask Efficient front ramming Immediately turn around to attack with back ram
What about “real” domains? n Unreal Tournament 2004 ¨ Commercial video game ¨ Basis for Bot. Prize competition: Bot Turing Test n Placed 2 nd with our bot: UT^2
UT^2 Behavior/Judging Game
Summary n n Neural networks can represent complex behavior Neuroevolution = way to discover this behavior Multiobjective evolution needed in complex domains Success in challenging designed/commercial domains
Questions? E-mail: schrum 2@cs. utexas. edu Webpage: http: //www. cs. utexas. edu/~schrum 2/
Auxiliary Slides n Empirical results
Differences for Alternating and Chasing significant with p <. 05
Скачать презентацию Evolution Brains and Multiple Objectives By Jacob Schrum
963076e1a330445c8e451dcc7a8b8fbe.ppt