You are on page 1of 18


AI  Architectures  
Gustavo  Patow  
IMAE  -­‐  UdG  
Par7ally  based  on  
AI  Architectures:  A  Culinary  Guide  

Mexican  Food  Analogy  

The  outside  is  a  content  delivery  mechanism  


 every  Bme  you  take  a  bite  of  your   content.  you  never  know  when  the  enBre  plaKorm  is   going  to  simply  fall  apart   The  most  basic  architecture   Adding  a  liLle  bit  of  structure  to  a   bunch  of  otherwise  disjointed  rules   maps  over  somewhat  to  the  most  basic   of  AI  architectures   2   .3/28/15   Bunch  of  rules   •  Simply  adding  rules  here  or  there  around  our  code   that  change  the  direcBon  of  things  in  a  fairly   haphazard  manner   •  Obviously.  the  problem  with  that  is  you  can  only  get   so  much  content  before  things  become  unstable   •  It’s  a  bit  unwieldy  as  well   •  Most  importantly.

 are  listed  in  the   body  of  that  state.     3   .  for  example.  an  agent   can  only  be  in  one  state   at  a  Bme  (although  other   implementaBons  exist)   Unaware Combat Investigate Search The  finite  state  machine  (FSM)   •  This  organizes  the  agent  behavior  beLer     •  Because  everything  the  agent  needs  to   know  about  what  it  is  doing  is  contained   in  the  code  for  the  state  that  it  is  in   –  The  animaBons  it  needs  to  play  to  act  out  a   certain  state.3/28/15   The  finite  state  machine  (FSM)   The  most  basic  part  of  a   FSM  is  a  state:   –  an  AI  agent  is  doing  or   being  something  at  a   given  point  in  Bme   –  It  is  said  to  be  “in”  a  state   –  TheoreBcally.

 “if  [the  player   enters  the  room]  and  [is  holding  the  Taco  of  Power]  and  [I  have  the   Salsa  of  SmiBng].  the  result  of  this  is  that  I  would  transiBon  from  “guard”  to   “flee”  instead   4   .3/28/15   An  example   •  The  transiBon  logic  in  any  given  state  may  be  as  simple  or  as   complex  as  needed   •  For  example.   Another  one   •  More  commonly.  State  A  might  say  that.  it  may  simply  involve  a  countdown  Bmer  that   says  to  switch  to  a  new  state  aVer  a  designated  amount  of   Bme   –  It  may  be  a  simple  random  chance  that  a  new  state  should  be   entered   –  For  example.  every  Bme  we  check.  then  flee”     –  Obviously.  state  machines  employ  elaborate  trigger   mechanisms  that  involve  the  game  logic  and  situaBon   •  For  instance  our  “guard”  state  may  have  the  logic.     –  Note  the  three  individual  criteria  in  the  statement   •  We  could  certainly  have  a  different  statement  that  says.  “if  [the   player  enters  the  room]  and  [is  holding  the  Taco  of  Power]  and  [I   DO  NOT  have  the  Salsa  of  SmiBng].  then  aLack  the  player”     –  at  which  point  my  state  changes  from  “guard”  to  “a>ack”.  there   is  a  10%  chance  of  transiBoning  to  State  B   –  We  could  even  elect  to  make  the  new  state  that  we  will   transiBon  to  a  result  of  a  random  selecBon  as  well—say  a  1/3   chance  of  State  B  and  a  2/3  chance  of  State  C.

 go  and   touch  every  single  other  state  that   could  potenBally  transiBon  to  it   •  To  add  a  State  E.  we  would  have  to   edit  A-­‐D  to  add  the  transiBons  to  E   •  EdiBng  a  state’s  logic  invokes  the   same  problem   •  You  have  to  remember  what  other   states  may  be  involved  and  revisit   each  one   5   .  as  the  number   of  states  increases.   the  number  of   potenBal  transiBons   increases  as  well   •  at  an  alarming  rate!   Workload  for  each  new  state   •  To  have  that  state  accessible.3/28/15   Problems   •  First.

3/28/15   Workload   •  AddiBonally.  it  really  doesn’t  maLer  what  I’m  doing  at  the  Bme—I’m  going  to   flinch.  what  they  should  change  to   –  That  logic  oVen  has  very  li>le  to  do  with  the  state  contained  in  and  more   to  do  with  what  is  going  on  outside  the  state  or  even  the  agent  itself   •  If  I  hear  a  gunshot.  or  any  number  of  other  appropriate  responses.  our  agents  were  in  one  state  at  a  Bme   –  they  were  “doing  something”  at  any  given  moment   •  even  if  that  something  was  “doing  nothing”   –  Inside  each  state  was  decision  logic  that  told  them  if  they  should  change   to  something  else  and.  any  logic  that   would  be  involved  in  that   transiBon  must  also  be   interworked  into  the  other   state-­‐specific  logic  that  may   already  be  there   •  Suffers  from  some  of  the  same   fragility  of  the  ad  hoc  bunch  of   rules  we  menBoned  earlier   The  Behavior  Tree   •  It  is  useful  to  point  out  the  differences     between  an  ac7on  and  a  decision   –  In  the  FSM.  wet  myself.  duck  for  cover.   •  Therefore.  why  would  I  need  to  have  the  decision  logic  for  “React  to  Gunshot”  in  each   and  every  other  state  I  could  have  been  in  at  the  Bme?   6   .  in  fact.

 changing  values  in  the   world.   7   .3/28/15   The  Behavior  Tree   Separates  the  states  from  the  decision  logic   The  Behavior  Tree   •  States  and  decision  logic  sBll  exist  in  the   AI  code   –  but  they  are  not  arranged  so  that  the   decision  logic  is  in  the  actual  state  code   •  Instead.  the  decision  logic  is  removed  to  a   stand-­‐alone  architecture   –  This  allows  it  to  run  by  itself—either   conBnuously  or  as  needed—where  it   selects  what  state  the  agent  should  be  in   –  All  the  state  code  is  responsible  for  is   doing  things  that  are  specific  to  that  state   such  as  animaBng.  etc.

3/28/15   The  Behavior  Tree   All  the  decision  logic   is  in  a  single  place!!!   Hierarchical  Finite  State  Machine   some  states  contain  other  related  states   making  the  organizaBon  more  manageable   8   .

3/28/15   Hierarchical  Finite  State  Machine   •  There  are  mulBple  levels  of  states   •  Higher-­‐level  states  will  only  be  concerned   with  transiBoning  to  other  states  on  the   same  level   •  On  the  other  hand.  how  it  gets  to  that  state  is  significantly   different   •  A  planner  compares  its  situaBon—the  state  of  the  world  at   the  moment—and  compares  it  to  a  collecBon  of  individual   atomic  acBons  that  it  could  do   •  It  then  assembles  one  or  more  of  these  tasks  into  a  sequence   (the  “plan”)  so  that  its  current  goal  is  met   9   .  lower-­‐level  states   inside  the  parent  state  can  only  transiBon   to  each  other   •  This  Bered  separaBon  of  responsibility   helps  to  provide  a  li>le  structural   organiza7on  to  a  flat  FSM  and  helps  to   keep  some  of  the  complexity  under   control   Planner   •  Like  a  behavior  tree.  the  reasoning  architecture  behind  a   planner  is  separate  from  the  code  that  “does  stuff”     •  While  the  end  result  of  a  planner  is  a  state  (just  like  the  FSM   and  behavior  tree).

 it  would  have  to  pick  one  up.   10   .  if  the  goal  is  “kill  player”.  if  another  method  of  saBsfying  the  “kill  player”   goal  is  to  throw  a  Taco  of  Power  at  it.  a  planner  might   discover  that  one  method  of  saBsfying  that  goal  is  to  “shoot   player”.  and  the  agent  already   has  one  in  hand.     Of  course.  this  requires  having  a  gun.  it  would  have  to  move  to  one  it  knows  exists.  it  would  likely  elect  to  take  the  shorter  plan   and  just  hurl  said  taco.     If  one  is  not  nearby.     If  it  doesn’t  know  where  one  is.  it  may  have  to  search  for  one.     If  the  agent  doesn’t  have  a  gun.3/28/15   Planner   A  planner  actually  works   backwards  from  its  goal     Planner   •  For  example.     –  –  –  –  –  •  Of  course.     The  result  of  searching  backwards  is  a  plan  that  can  be  executed   forwards.

 a  drawback  of  this  is  that  authorial  control  is  diminished.  predictable  moments  are  the  excepBon.  “no…  I  really  want  you  to  do   this  exact  thing  at  this  moment”   11   .     –  In  a  FSM  or  BT.  “outside  the  box”  soluBons  were  the  excepBon  from  the   predictable.  you  must   specifically  override  or  trick  the  planning  system  to  say.  the  scripted.  hand-­‐authored  systems   –  In  a  planner.  “here  are  the  potenBal  things  you  could  do…  go  forth  and  do   things”   •  Of  course.  a  major  plus  of  the  planner  is  that  a   new  acBon  can  be  dropped  into  the  game  and  the  planner  architecture  will   know  how  to  use  it   –  This  speeds  up  development  Bme  markedly   –  All  the  author  says  is.  creaBve.3/28/15   Differences   •  The  planner  diverges  from  the  FSM  and  BT  in  that   it  isn’t  specifically  hand-­‐authored   –  Therein  lies  the  difference  in  planners—they  actually   solve  situaBons  based  on  what  is  available  to  do  and   how  those  available  acBons  can  be  chained  together   –  One  of  the  benefits  of  this  sort  of  structure  is  that  it   can  oVen  come  up  with  soluBons  to  novel  situaBons   that  the  designer  or  programmer  didn’t  necessarily   account  for  and  handle  directly  in  code   Advantages  and  Disadvantages   •  From  an  implementaBon  standpoint.

com/open/coverage/htn-­‐planning-­‐discussion/     “uBlity-­‐based”  method   •  Instead  of  assembling  a  plan  like   the  planner.   •  You  can  select  it  based  on  what   you  have  a  taste  for  or  what  is   most  accessible  at  the  moment.  however.3/28/15   Example   •  A  more  recent  flavor  of  planner  is  the   hierarchical  task  network  (or  HTN)  planner   such  as  was  used  to  great  effect  in  Guerilla’s   Killzone  2   hLp://aigamedev.  you  simply  select   what  it  is  that  would  like  to   poke  at  and  execute.  the   uBlity-­‐based  system  simply   selects  the  single  next  acBon   •  All  the  ingredients  are  in  the   mix  and  available  at  all  Bmes.     •  However.   12   .

 the  mathemaBcal  approach  that  uBlity-­‐based   systems  employ  is  necessary  to  ferret  out  what  the  most   reasonable  acBon  to  take  is   •  Aside  from  The  Sims.  other  common  areas  where  uBlity-­‐ based  systems  are  appropriate  are  in  RPGs.  and   simulaBons.   13   .  they  are  more  appropriate  in  situaBons  where  there   are  a  large  number  of  potenBally  compeBng  acBons  the  AI   can  take—oVen  with  no  obvious  “right  answer”   •  In  those  Bmes.  RTS.3/28/15   “uBlity-­‐based”  method   •  •  •  •  The  progression  of  AI  architectures   throughout  The  Sims  franchise  is  well   documented     Each  potenBal  acBon  in  the  game  is   scored  based  on  a  combinaBon  of  an   agent’s  current  needs  and  the  ability  of   that  acBon  or  item  to  saBsfy  that  need   The  agent  then  uses  an  approach   common  in  uBlity-­‐based  methods  and   constructs  a  weighted  sum  of  the   consideraBons  to  determine  which  acBon   is  “the  best”  at  that  moment   The  acBon  with  the  highest  score  wins   “uBlity-­‐based”  method   •  While  uBlity-­‐based  systems  can  be  used  in  many  types  of   games.

 uBlity  systems  offer  a  deep  level  of  control   The  difference  is  that  rather  than  telling  the  system  exactly  what  to  do  in  a   situaBon.  determining   how  the  acBons  stack  up  is  oVen  more  opaque   That’s  not  to  say  that  a  uBlity-­‐based  AI  is  not  controllable  or  configurable   In  fact.  the  uBlity-­‐based  AI  code  is  a  reasoner   Once  an  acBon  is  decided  up.  like  a  planner.  the  agent  sBll  must  transiBon  to  a  state   The  uBlity  system  simply  is  selecBng  what  state  to  go  to  next   In  the  same  way.  just  like  those  other  systems.  the  system  is  providing  suggesBons  as  to  what  might  be  a  good   idea   In  that  respect.  ediBng.  it  is  a  relaBvely  simple  exercise  to  traverse  the  tree   and  find  the  branches  and  nodes  that  would  be  acBve  in  a  parBcular  situaBon   •  •  •  •  •  Because  a  uBlity  system  is  inherently  more  fuzzy  than  binary.  adding  acBons  to  the  system  is  fairly  straight  forward   By  simply  adding  the  acBon  with  the  appropriate  weights.  the  reasoning  code  is   all  in  a  single  place   This  makes  building.  tuning  and  tweaking  the  system  much  more   compartmentalized   Also.   14   .   Drawback   •  There  isn’t  always  a  good  way  to  intuit  what  will  happen  in  a  given  situaBon   –  In  a  behavior  tree.  then.  the  AI  will   automaBcally  take  it  into  account  and  being  using  it  in  relevant  situaBons   This  is  one  of  the  reasons  that  games  such  as  The  Sims  were  as  expandable   as  they  were—the  agents  simply  included  any  new  object  into  their   decision  system  without  any  changes  to  the  underlying  code.  a  uBlity  system  shares  some  of  the  adaptable  aspects  of   planners—the  AI  simply  looks  at  its  available  opBons  and  then  decides  what   is  most  appropriate.  for  example.3/28/15   ImplementaBon   •  •  •  •  •  •  •  •  Like  behavior  trees  and  planners.

  “It  looks  preLy  cool.  looks  over  your  shoulder  and  says.  “This   is  what  I  have”   •  If  a  designer  wanders  in.”  there’s  really  nothing   you  can  do  to  change  it   •  About  all  you  can  do  is  try  to  retrain  the  NN  and  hope  for  the   best    Neural  Network  (NN)   •  Lack  of  designer  control  aVer  the   fact   •  Unfortunately.  this  tends  to   disqualify  NNs  and  other  machine-­‐ learning  soluBons  from   consideraBon  in  the  game  AI   environment  where  that  level  of   control  is  not  only  valuable  but   oVen  a  requirement   15   .  neural  nets  need  to  be  trained  with   test  or  live  performance  data   •  At  some  point  you  have  to  wrap  up  the  training  and  say.  but  in  [this  situaBon]  I  would  like  it  to  do   [this  acBon]  a  liLle  a  liLle  more  oVen.3/28/15    Neural  Network  (NN)   •  This  is  the  caveat  emptor  of  the  NN-­‐based  AI  soluBon   •  As  a  type  of  “learning”  AI.

3/28/15   Wrap  up   •  This  has  by  no  means  been  an  exhausBve   treatment  of  AI  architectures   •  The  purpose  was  simply  to  expose  you  to  the   opBons  that  are  out  there  and  why  you  may   or  may  not  want  to  select  each  for  the   parBcular  tastes  and  needs  of  your  project   •  To  sum  up.  let’s  go  through  the   opBons  once  again…   Wrap  up   16   .  though.

 2009."  ArBficial  Intelligence  and  InteracBve   Digital  Entertainment  Conference  (Stanford   University).  and   decide  what  you  like   Homework  (HFSM)   Michael  Booth.3/28/15   Conclusions   •  there  is  no  “one  size  fits  all”  solu7on  to  AI   architectures   –  You  also  don’t  have  to  limit  yourself  to  a  single   architecture  in  a  game   •  Depending  on  your  needs.  any  of  the  techniques  may  be  The  Right   Way  for  you  to  organize  your  AI   •  As  with  any  technical  decision.  your  team.  the  secret  is  to   research  what  you  can.   (On  Moodle)   17   .  and  your  prior   experience.  even  try  a  few  things  out.  "The  AI  Systems  of  LeV  4   Dead.

3/28/15   Thanks!   18   .