From the Ground Up A I Architecture and

From the Ground Up: A. I. Architecture and Design Patterns Dr Brett Laming Rockstar Leeds

Contents • • Basic Principles Think/Act - The I/O Divide An Algorithmic Approach A Functional Approach Movement Animation Conclusion

Basic Principles

Why Do We Care? • It’s a Competitive Market… – – Higher Expectations Tighter Schedules Multiple Platforms Simultaneous Titles • …But Patterns Are Everywhere – Identify, generalise, reuse, evolve – Reliability – Production Speed More Fun Stuff

Inspirations Marvin Minsky Different representations for different views Noam Chomsky Hierarchical decomposition Structure vs Meaning “Colorless green ideas sleep furiously” Chris Hecker Style vs Structure [2008] What is the textured triangle of A. I. Damian Isla Cognitive Maps Spatial Relations Semantics Craig Reynolds Simple rules Complex behaviour Daniel Dennet Behaviour can be viewed at the physical, design and intentional levels David Marr Computational, algorithmic and implementational

What Am I Looking For? Algorithms Reusability Components Hierarchy Concepts Commutability Best Practice

How Do I Find Them? • Observation – How might it work? • Introspection – What would I do? • Generalisation – I’ve done this before – They do the same • Bad Experience – Lets not do that again • Background – I studied this once? – Could I apply this?

Best Practice

Best Practice • Prototype new ideas where possible – Get visual and design direction • Mock-ups – Prove (or disprove) the concept • Quick and dirty programming • Play to peoples strengths – Maths Physics Guys – Navigation Mesh Collision Guys

Best Practice • Program Defensively – Assert and Unit Test – Automated Scripts as soak tests – One co-ordinate system and S. I. units • Maximise Workflow – Cater tools to their needs – Put new functionality on bypass • Think of the man-hour cost! – Minimise potential for human error

Best Practice • Build A Debugging Suite – – Instant Pause Flyable Camera Layered Information Action Histories • Maintain Player Immersion – A. I. should not be too bad or too good – Limit ourselves to what the player would know – Constrain them to the same actions

The Algorithmic Approach

The Think-Act Loop Player Sensory Receptors Brain A. I. • Key tenants Sensory Data – Use only what we. Think know might – Mimic the player Controller Game Act

Sensory Input Sensory Data Think Controller Act

Sensory Data • Detail level – Depends on genre and perceived communication • Thief vs Medieval Total War II • General Model – Visual component an arc – Auditory component a radius • Auditory targets less official – Occlusion too expensive? • Shoot a weapon to get info – Theorise using ghost images

Blackboards • Used To Share Information – Static blackboard stores defined types of info – Dynamic blackboard stores arbitrary data • Agents write to the board – Generally read it as well Agent A Agent B {10, 20, 15} {-30, 20, 13} {-10, 15, 12} {17, 11, 5} Scout {3, 17, 10} Cover {17, 11, 5}

Control Output c. Sensor. Cone Think Controller Act

Virtual Controller (yoke) • Purpose – Carries control instructions – Provides a strict I/O Divide Process() c. Virtual. Yoke yoke • Notice the const correctness – Unifies player and A. I. • Control mapping for the player • A. I. fills it in from Think() • Key Properties – Never stored in it’s entirety – Created on the stack – Lifetime of a single Process() Think(yoke&) const Controller Act(const yoke&)

… So Actually Process() c. Blackboard c. Virtual. Yoke yoke Think(yoke&) const Act(const yoke&) c. Sensor. Cone

Virtual Yoke • Composite c. Virtual. Yoke – Collection of smaller yokes – Allows selective storage c. Locomotive. Yoke fp 32 m. Gas; m. Steering; m. Pitch; c. Entity *mp. Dont. Avoid; c. Locomotion. Yoke m. Locomotion; c. Weapon. Yoke m. Weapons; c. Anim. Yoke m. Animation; • Context Based Controller Input • Logic Flow Control • Conditional signals c. Weapon. Yoke enum e. Fire { F_PRESSED, F_WHEN_IN_CONE, F_WHEN_LOCKED }; e. Fire m. Fire. Button;

Taking Action Process() c. Blackboard c. Virtual. Yoke yoke Think(yoke&) const Act(const yoke&) c. Sensor. Cone

Act • Applies Yoke Commands – Composite yokes Subsystems • Object Models – Supply common interfaces • Could be a turret mounted weapon or my pistol. • Could be driving a car, a plane, a boat or myself! Act() yoke i. Weapon. Mgr* i. Locomotive*

Object Model • Self Contained – Instructions for Think – Actions for Act – Commutable • Plug and Play • Downloadable Content – Maybe broadcast use • Think Sims 2! i. Locomotive // Fills in yoke Compute. Motion(const c. Target &, c. Locomotive. Yoke&) // Computes forces Apply. Motion(const c. Target &, const c. Locomotive. Yoke&, v 3 &force, v 3 &torque)

More Coolness Apply. Motion(const c. Target& targ, c. Locomotive. Yoke& yoke) { v 3 force(Zero); v 3 torque(Zero); if (Drivable()) { Drivable()->Apply. Motion( targ, yoke, force, torque); } if (Floatable()) { Floatable()->Apply. Motion( targ, yoke, force, torque); } Apply. Force. Torque(force, torque); } Compute Apply

Frame Time Process() c. Blackboard c. Virtual. Yoke yoke dt() Think(yoke&, dt) const c. Sensor. Cone Update(dt) Act(const yoke&, dt)

dt() • Work into everything – Including fixed time steps – No more s+=v, s+=v*dt() smoothing • Benefits – Integration • Implicit forward Euler – – – Rough Smoothing Closed Feedback Pause dt=0 Level Of Detail dt=2 dt Special Effects feedback control dt()

The Functional Approach

Problem Domain Needs a problem • Examine the Terminology – Feeling, Knowledge, Goals, Beliefs, Needs • Examine the Concepts – Decisions, Facts, Uncertainty, Exploration, Verb-Noun Actions, Repetition, Sequencing

What Are We Doing? • Goal based reasoning – – Working to solve a goal Thinking about and realising smaller tasks Taking a hierarchical approach Using a limited number of short verb-noun pairings to form a plan • We’ve seen this before – An old pattern • Colossal Cave Adventure and MUDs • Verb-noun actions like “get axe, wield axe” separated by movement “n, e, e, s, e” – We use a container object model

Applying The Pattern Taking the computational to the algorithmic Know of “Mine, Smithy” 1. Goto “Mine” Know of “Wall” 2. Get Ore From “Wall” 3. Goto “Smithy” Know of “Door” 4. Use “Door” 5. Play “Open Door” 6. Warp Inside Know of “Owner”, “Forge” 7. Play “Close Door” 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. Goto “Owner” Put 10 gp In “Owner” Get Time From “Owner” Goto “Forge” Put Time In “Forge” Put Iron In “Forge” Use Enchant Use “Forge” Get Sword From “Anvil”

Easy Questions • Why did I choose to do this again? – Because we were driven to it by personality and need • What happens when I get the treasure? – I’ll probably choose to do something else depending on my mood • Why stop at get iron? – Because its reached the atomic level there are no more questions, just results

Putting It Together Ambient Controller Explicit Order “Get treasure” “Script here, Please greet the player” Planning “Get magic sword”

Explicit Orders • Script commands Ambient Controller – Script on A. I. • Autopilot – Player on Player • Player instructions – Player on A. I. • Squad Commander – A. I. on A. I. Explicit Order “Get treasure” “Script here, Please greet the player” Planning “Get magic sword”

Scripting Notes • Don’t mix styles – Script has immediate control – Script waits for an opportunity • Keep common properties separate – No sharing memory locations • A script global population density • A code global population density – Maintain a set order of calculation • Generally consistent with style

Ambient Controller • Generates sensible actions autonomously – Maybe Idle – Maybe Full Daily Routine • Daily Routines – Character properties – Needs/Drives – Scheduling – Time of day nice emergent behaviour Ambient Controller Explicit Order “Get treasure” “Script here, Please greet the player” Planning “Get magic sword”

Daily Routine Time Of Day Sleep Goto Work Character Housework 5% Tavern 30% Shopping 5% Libido Work Leave Work 10% Study Hunger dt() 50% Brothel Drive Model Relax Go Home Schedule

Planning Ambient Controller Explicit Order Goto Position “Get treasure” “Goto Forge” “Script here, Please greet the player” Perform Action “Use Forge” Planning “Get magic sword” Continuous bar dt() Defines Think()

Plan Components • Play “Animation” – Waits on dt() • Use “Object” • Get/Take “Object” From “Container” – Primary world manipulation – Contents determine state – Object model again • Goto “Location” – Broadcast actions. – Waits on dt() – Change world based on – Key A. I. output state – Complex – May wait on dt() – Warrants special attention later

Search Based Planning • Traditional academic approach – See STRIPS, Hierarchical Task Networks, Bratko • The Good – Mimics our introspective reasoning – Seeks to fully realise a plan to the goal • Directed search for optimal solutions • Post processing even more so • The Bad – Knowledge representation • Scalability - difficult for video games • Lots of storage

Procedural Planning • • Industry preferred approach Hierarchical Easy to comprehend Limited Language – – Goals Sub-goals Conditions Actions Get Object Short of money? Thief skills? Make Steal • Transitions – Sequential – Decision Based • Powerful Results Goto Mine Get Ore Buy

Procedural Planning - Issues • Competing children – We have to make a best guess from the options – A* might help • But we could still end up with case of a basic sword being bought but not affording the forge. • Incomplete plan means no post process – Not good for player supporting A. I. – More action for generic A. I.

The Curve Ball • Task Interruption – I’m returning to my gang hideout – I see an enemy • I engage the enemy – I roll out of the way of a car – I recover to my feet • I re-engage the enemy – I continue to return to my gang hideout • … Is A Key Requirement of our A. I.

Finite State Machines • No plan history No idea of context No generalised exit. Hideous state history workarounds • Don’t scale well Death Examine Kill Examine Death Gain Range – Many transitions • HFSM came along – Eased transitions – But history still an issue Look At Attack

Behaviour Trees Get Object Selection Create Low money? Get Material Buy Ok money? Sequencing Goto Source Take Material Actions Decorators Crime Allowed? Steal Thief Class? Preconditions

Modelling Interruption? Ambient Behaviour Return Home Locate Hideout Goto Location Parallel? Kill Enemy Gain Range Fire Weapon

Behaviour Trees • Simple and powerful – Limited vocabulary – Most situations handled • Highly flexible – – Plug and play Customisable Nice design tools Handy child evaluation • Lends itself to directed decision making • Issues – Interruption handling • Where to return – Amount of flexibility • Trees get complicated

MARPO Get Object Not enough money? Create Get Material Goto Source Take Material Money? Buy Thief class? Thieving allowed? Steal

MARPO Long term Get Object Create Get Material Take Material Reactive Immediate

MARPO Long term Reactive Get Object Kill Enemy Create Gain Distance Get Material Take Material Immediate

MARPO Long term Reactive Immediate Get Object Kill Enemy Roll Gain Distance

MARPO • Behavioural Bamboo Forest – Stack based tasks • Suppression model – Multiple threads of execution • Keeps only one stack in memory down to current task – Decision logic lies with the parent • Higher level parameterised building blocks – Authoring is by script not tool. • Winding Ability – Allows auto-recovery from any state – Respects script orders on an immediate basis

Movement

Goto Position • What Is Position? – – – A world co-ordinate An entity A navigation point An offset off an entity A radius off an offset, off an entity • Still World Positions! – So generalise c. Target void Set(…); void Set. Arrival. Conditions(…); v 3 World. Pos() const; v 3 Intercept. Pos(…) const; bool Has. Arrived(…) const; • Lots for free – Entity intercept – Completion checks

The Problem • • Static geometry Dynamic obstacles … on the move Terrain type • Usable surfaces

Stage 1 - Navigation • Navigation Mesh – Industry standard [Tozour, 2008] – Handles static geometry • Searching with A* – Optimise search space, not A* • Improve Data Format – Make it Hierarchical – Allow for reference spaces [Isla, 2005]

Navigation • Quick To Search Live Updates • Zones Spatial Reasoning [Isla, 2005]

Components c. Navigator Pos() Final. Target() Current. Target() Has. Arrived() Update( c. Target ) c. Nav. Pos m. Final c. Nav. Pos m. Current c. Nav. Pos m. Pos c. Nav. Poly c. Nav. Pos … indices … edge info c. Nav. Id m. Parent c. Nav. Id m. First. Child m. Poly. Id; m. Poly. Source; m. Child. Count c. Poly. Coords … local co-ords

Stage 2 - Optimisation • String Pulling – Bevel uncrossable edges to a radius – Tighten points

Stage 3 - Dynamic Avoidance • At rest – Mesh binding • On move – Intercept calculation

Stage 4 - Locomotion • Differing approach – Vehicle – Humanoid • Can we generalise?

Locomotion • • Generalised by i. Locomotion Same interface Different yoke instructions Vehicles i. Locomotion Compute. Motion(…) Apply. Motion(…) – Vehicles supply gas, steering – Difficulty is in mapping target to gas and steering • Actors – Actors supply ideal position, velocity, direction – Difficulty is in animation to hit position.

Locomotion - Vehicles Target Position Speed For Deceleration Angle To Steering Yoke Maths PID Controller Top Speed For Steering Min Speed to Gas Maths PID Controller Yoke

Some Hints Speed to Gas Angle To Steering Speed For Deceleration Manual Shift vmax a u d s 1 s 2 v • Equations of Motion • PID • Response Curves

Animation

Locomotion - Character • Animation Position • Challenges – Sensible manoeuvre choice • Animation Graphs • Navigation Mark Up – Natural fluidity • Foot positioning • Hand positioning Idle Crouch Walk Run Crouch Run

Goal Based • Directed Search of Animation Graph – Incorporates interesting A. I. along the way – Think Gears of War, A* • Interested Champandard • Personal Issues – Need to find a full animation plan to goal • This would need to sit with path finding • Makes it more expensive – Painful Heuristic Balancing • Switching to the time domain probably partially solves it • But how do we prevent repeated actions

“Carrot On A Stick” Method • Keep i. Locomotion interface • Compute. Motion() computes velocity • Apply. Motion() animates for velocity • dt() helps minimise error • Want to use a cover point? – Explicit decision a new target i. Locomotion Compute. Motion(…) Apply. Motion(…) • Given the animation properties • We can solve by mathematics

“Carrot On A Stick” Method Under fire? Kill Route to cover Goto Anchor Point Set piece Nav. mesh Cover Find cover?

“Carrot On A Stick” Method Direction for Animation • Offset based on , s • Changes our target – Hierarchical changes off=ƒ( , s) s • Changing over dt • Smoothes our arrival – Think high jump! – Never perfect – But mistakes made are human – Constrained by navigation mesh

“Carrot On A Stick” Method vmax a vaim V = 0? s u dt u s d v • Equations of Motion – They’re back! • Give us our aim velocity vaim • What about a and d?

“Carrot On A Stick” Method • Animations – distance (s) in time (t) an average velocity – Assume velocity bands – Look up v for animation – Dead zones indirectly determine blending v Crouch run Crouch walk • a and d now character properties • Actively correct for v v Crouch run – Use tricks like light I. K. • Implicitly correct by dt() Crouch walk t

“Carrot On A Stick” Method • Animation graph – Used for information query only • Locomotive cycle driven by vaim • Free things! – Foot positioning • Obtained through cycle alteration and dt() – Blending – Character movement properties • Obtained through a and d – Swim, crawl, crouch, walk • Same procedure, different velocity bands – Human like arrival mistakes

Conclusion

Conclusion Good reusable AI is about – Best Practices – Strict interfaces – Multiple takes on problems – Concepts – Commutability – Components – Algorithms – Hierarchy – Pattern Recognition – Human Sciences

Conclusion • Style vs Structure - Hecker 2008 – “Texture Mapped A. I. Triangle” – Style • No idea where it is – annoyingly – Structure • • Not one single triangle But many triangle like pieces Represented differently But all essentially the same piece

References • Key References – – – • Laming, B. [04] “The Art of Surviving A Simulation Title”, A. I. Wisdom 2, Charles River Media Laming, B. [08] “The MARPO Methodology: Planning And Orders”, A. I. Wisdom 4, Charles River Media Isla, D. [05], “Dude, where’s my Warthog? ”, www. aiide. org/aiide 2005/talks/isla. ppt, 13. 3. 08 Tozour, P. [08] “Fixing Pathfinding Once and For All”, http: //www. ai-blog. net/archives/000152. html, 13. 3. 08 Champandard, A. , http: //aigamedev. com, 13. 3. 08 A. I. Wisdom Books in general Introduction – • Reynolds, C. [87] “Boids”, http: //www. red 3 d. com/cwr/boids/, 13. 3. 08 Intercept Calculation and Dynamic Avoidance – – • Stein, N. [02] “Intercepting a Ball”, A. I. Wisdom 1, Charles River Media. [HINT ] Tozour, P. [02] “Building a Near-Optimal Navigation Mesh”, A. I. Wisdom 1, Charles River Media Best Practice – • Tozour, P. [02] “Building an AI Diagnostic Toolkit”, A. I. Wisdom 1, Charles River Media Sensory and Blackboards – • Isla, D. and Blumberg, B. [02] “Blackboard Architectures”, A. I. Wisdom 1, Charles River Media Planning – – – • Champandard, A [08], “Getting Started With Decision Making and Control Systems”, A. I. Wisdom 4, Charles River Media Champandard, A. , “Understanding Behaviour Trees”, http: //aigamedev. com/hierarchical-logic/bt-overview, 13. 3. 08 Yiskis, E. [04] “A Subsumption Architecture for Character-Based Games”, A. I. Wisdom 2, Charles River Media Navigation – • Numerous references, almost all A. I. Wisdom books. Locomotion – Vehicle – – • Alexander, B. [02] “The Beauty of Response Curves”, A. I. Wisdom 1, Charles River Media Forrester, E. [04] “Intelligent Steering Using PID Controllers”, A. I. Wisdom 2, Charles River Media Conclusion – Hecker, C. [09] “Structure vs Style”, http: //chrishecker. com/Structure_vs_Style, 13. 3. 08