Roguelike Intelligence - Stateless

AIs
From RogueBasin

Since I'm an AI engineer, and since I'm unemployed right now, and since I'm
interested in roguelike games, I thought I'd write a series of articles on game
AI for roguelikes as a contribution to the community.

Some of these techniques (Stateless AI's and State Machines in particular) are
properly "pseudointelligence" rather than "artificial intelligence", because they
don't involve any actual learning. But "AI" is how game designers refer to the
decision-making code for their artificial antagonists, regardless of its actual
status, so I'm going to use it.

The ground I plan to cover here includes roughly these articles; I may add an
article or two to this or change the order of articles depending on what people
are interested in reading.

1. Stateless AI
2. State Machine AI
3. "Evolving" stateless AI's or state machine AI's
4. Modeling the player
5. Minimaxing in application to roguelike domains
6. Neural Networks and training them with backprop
7. "Evolving" neural networks.
8. Recurrent Neural Networks (NN's with memory).

Roguelike Game AI, part 1: Stateless AI's

The classic "zombie" AI is easy to code: It goes like this:

ZOMBIE AI
if can-move-toward-player
move-toward-player
else if can-attack-player
attack-player
else stand-still

And off your zombie goes, closing on the player as fast as it can and beating
the player as hard as possible once it gets there. It's the simplest possible
monster AI, the first monster we all implement when we're testing monster
movement code for the first time.

Suppose you want to have a different kind of brain dead monster called a
"Golem", which uses ranged attacks at range. Consider this:
 GOLEM AI
If can-attack-player
attack-player
else if can-move-toward-player
move-toward-player
else stand-still

This is the same as the zombie if it's carrying a sword, but if you give it a
ranged weapon its behavior is dramatically different. If it has a ranged attack,
it will stop dead as soon as it catches sight of the player character and stand
there shooting until the player kills it or gets out of range or it runs out of
ammo. If it's not dead and still has ammo, it will run after the player until it's in
range again and start shooting some more. If it has both a ranged and hand-to-
hand attack, it shoots till it runs out of ammo, then switches to hand-to-hand
and behaves like a zombie.

This seems like a smarter, more versatile monster, and it's no harder AI work
than the simple, stupid zombie. It can really mess up the way the dungeon
works though, if other monsters can't get past it in corridors.

These are (very) simple examples of a stateless AI. It's called stateless because
every time it's the monster's turn to act, it starts in the same 'state' -- ready to
just start on the if statements until it finds an action it can do. Angband is a
popular game whose monsters use (nearly) stateless AI.

Not all stateless AI's are as stupid as those given above. You can make a simple
modification of the zombie AI for another kind of AI for slightly-less stupid
monsters.

GHOUL AI
If can-move-toward-player
AND (random < move-probability
OR can't-attack-player)
move-toward-player
else if can-attack-player
attack-player
else stand-still

Now, ghouls behave a lot like a zombie or a golem, except that its move/attack
decision has a random element not predictable by the player. It may pause on
the way to the player to fire a ranged weapon, but it won't just stand there
firing and being a roadblock to other monsters, and it won't just run toward the
player allowing him to ignore it till it gets there. Its move probability can be set
in monster configuration so you could make different types of monsters using
the ghoul AI more or less likely to move, which will distinguish your monsters
from each other a little bit more.

Now, one more complication and your monsters will have just about the same
intelligence as Angband monsters.

ANIMAL AI
 If damage-taken > morale
if can-run-away-from-player
run-away-from-player
else if can-attack-player
attack-player
else if can-move-toward-player
AND (random < move-probability
OR can't-attack-player)
move-toward-player
else if can-attack-player
attack-player
else stand-still

The complication was the damage check in the first line. Now you can assign
your monsters different morale as well as different move probabilities, and the
timid ones with morale less than one percent of their hitpoints will run away
when they take even a tiny wound while the maniacal ones with morale equal
to or higher than hitpoints will fight to the death. Monsters that run away will
come back once they heal a little bit, fight when cornered, and generally
behave about like you'd expect a bunch of really stupid animals to behave. The
damage check is sort-of a mode, since damage carries from one round to the
next. But this is still a modeless AI, because we make the decision every round.
The random-state is also sort-of a mode, but the decision made is effectively
purely random, so it doesn't count.

Now let's look at code smarter than Angband's, introducing the idea of a
monster with a ranged attack and a preferred range. That will go like this:

ARCHER AI
if can-move-away-from-player
AND (damage-taken > morale
OR too-close-to-player)
move-away-from-player
else if can-move-toward-player
AND damage-taken < morale
AND too-far-from-player
move-toward-player
else if can-attack-player
attack-player
else stand-still

This creature will use a "gun-n-run" strategy, trying to use a ranged attack at
range while staying out of the player's reach. If it's faster than the player
character, or if a bunch of them are encountered together, it can be very
dangerous to a character with no ranged attacks, even if it's ridiculously weak.

If it has no ranged attacks, it becomes a silly "spectator" in the dungeon,

walking around and keeping an eye on the player character but not doing much
of anything else. Give it a morale of 1, lots of movement and hitpoints, and
name it The "Dungeon Survivor Camera Crew". Your players will report intense
satisfaction from finally maneuvering it into a deadend and killing it.

Now, you can take this a lot further in terms of dreaming up stateless AI's that
pursue different strategies in the dungeon, including AI's that stay in packs,
trying not to get too widely separated from others of their kind. But before you
do all of that, there's another opportunity to make your monsters smarter using
these simple, stateless AIs.

You can increase the basic pseudointelligence of stateless monsters, and

increase the differentiation between monsters, by making the various
primitives involved in the stateless AI's more sophisticated and different on a
per-monster basis.

For example, run-away-from-player can be stupid (moving to the adjacent

square that maximizes distance to the player) or smarter (heading for a room
exit - by preference, one that the monster is closer to than the player) or smart
(heading for an exit, but avoiding dead ends).

Move-toward-player can be stupid (stepping to the adjacent square closest to

the player) or smarter (following the player's scent track if the player's not in
sight) or smart (doing pathfinding to find a way around obstacles when the
player's not in sight) or exhibit both brains and teamwork (pathfinding if
player's not in sight, trading places with a more badly-damaged monster of the
same type if one is closer to the player).

Attack-player can be stupid (using whatever the monster's only attack is) or
smarter (picking among several attack forms based on what's available, such
as switching to meelee weapons when out of ammo) or smart (picking among
different attacks based on what's available and known about the player
character).

"stand-still" routines can do some monster-specific thing like throwing a

healing spell at the most badly wounded monster in the room, which is another
kind of monster teamwork, or casting a teleport spell, which will frequently get
the monster out of whatever "stymied" situation it's in, or telling the player a
rumor or insulting him.

If you have a few dozen different stateless AI's, and then a few different
versions of most of the primitives they use, combinatorics is on your side. You
will be able to make a menagerie of "simple" monsters that exhibit hundreds of
different kinds of behavior.

That said, this type of pre-coded, stateless AI is still very limited; it is in fact
the very simplest form of pseudointelligence for implementing game
adversaries

Some of you will by now have noticed something. I first introduced the "zombie
AI" which defaults to movement if possible, then I introduced the "golem AI"
which defaults to attack if possible. Then I introduced the "ghoul AI" which
makes a random choice between movement and attack when both choices are
valid.

What I didn't point out in the first article was that the "ghoul AI" can be used to
imitate both the "zombie AI" and the "golem AI" -- all you need to do is set the
move-probability parameter to one or zero respectively.

I went on to introduce the "animal AI" -- but that can be used to imitate the
"ghoul AI." All you need to do is set the morale to some value greater than the
creature's hitpoints and it will never retreat.

Then I introduced a run-n-gun monster with the "archer AI". But here I wasn't
strictly generalizing; The "animal AI" can't be imitated strictly within the code
of the "archer AI".

This may not seem important, but bear with me for a little bit; here's an AI
pseudocode that can be used to implement both the "archer AI" and the
"animal AI".

TYPICAL AI
If damage > morale
if can-run-away-from-player
run-away-from-player
else if can-attack-player
attack-player
else if too-far-from-player
AND can-attack-player
AND can-move-toward-player
if random < charge-probability
move-toward-player
else attack-player
else if too-close-to-character
AND can-attack-player
AND can-move-away-from-player
if random < retreat-probability
move-away-from-player
else attack-player
else if can-attack-player
attack-player
else if too-far-from-player
AND can-move-toward-player
move-toward-player
else if too-close-to-player
AND can-move-away-from-player
move-away-from-player
else stand-still

Now, if we want an "archer AI" we set retreat-probability to 1 and charge-

probability to 1.

If we want an "animal AI" we give it a too-close-to-player function that is never

true, a too-far-from-player function that is always true, and copy the "animal
AI's" move-probability parameter into the "typical AI's" charge-probability
parameter.

The separation of charge-probability and retreat-probability means that the

creature may pause while moving toward the player to fire a ranged weapon,
or pause while moving away from the player to fire a ranged weapon. But since
these are different tactical situations it seemed reasonable that the
probabilities should be different.
One other thing I did in the above AI was to separate the functions run-away-
from-player and move-away-from-player. The first is for panicked situations
when the monster doesn't want to be anywhere close to the player; it could be
implemented in a smart, spell-using monster as casting a long-range teleport
spell. The second is tactical; it means the monster wishes to be further away
from the player for tactical reasons. The smart, spell-casting monster could
implement this as a classic "blink" - a limited teleport that moves it, but only
ten to fifteen squares away. Less gifted creatures will implement this as
stepping to an adjacent square further from the player.

What I wanted to demonstrate by folding both "animal" and "archer" into the
same AI was this; if you are using stateless AI's you can have a universal AI
routine that's shared between all monsters. The distinction between monster
AI's is then reduced to a simple array of parameters and methods.

Even if no worthwhile combination of the two is possible, you can still combine
any two stateless AI's into the same stateless AI code by just making an
additional parameter that says which branch of the decision tree to follow and
putting each of the original stateless AI's on one branch of the decision tree.

Instead of the more sensible code above, for example I could have just written

TYPICAL AI:
if am-I-an-archer?
{...archer AI ....}
else
{...animal AI ....}

But I do want to make a point: this kind of combination should not be done
automatically or thoughtlessly; If I had done it without thinking, for example,
then I would have redundant branches for "zombie" and "golem" and "ghoul" in
my code (or redundant objects all inheriting from "monster", which is exactly
the same kind of mental spaghetti) when all of those behaviors can easily be
modeled by the "animal" code. Also, I wouldn't have the possibility of firing-
while-advancing for archers, and without doing that I wouldn't have thought of
the case of firing-while-retreating.

If you use OO a lot, or without seriously considering what is redundant or what

would be better combined, it's probably worth taking a good long look at your
code to see how much redundant stuff you have built in. This happens a lot
with behavioral modeling code; frequently you'll create something more
complex, adding capabilities incrementally, without noticing that it's made a
bunch of other things redundant.

Another risk is that you might create something different, and miss out on
other benefits or different behaviors you could model if instead you made the
routine you had more general. This is what happened when I implemented the
"golem" AI, and later the "archer" AI; in both cases there was a more general
AI that included both kinds of behavior where I was drawing a line between
them, and in both cases the more general AI also allowed new behaviors that
neither of the previous two allowed.

The take-home lesson is that, as a design principle, it's always better to have a
stateless AI that's more general than it is to have more different stateless AI's.
OO programming can be handy, but it allows you to very easily miss benefits,
redundancies, and synergies if you're not being particularly alert for them. It's
equally valid to say that it helps you by keeping such issues outof your hair, but
the fact is that your design decisions in response to those issues are things
your game can benefit from.

Now I think I've said enough about stateless AI's and it's time to move on to
state-machine AI's.

Reference section
Here is a list of the primitives I've used in building stateless AI's in these first
two articles.

can-move-toward-player
move-toward-player
can-move-away-from-player
move-away-from-player
can-run-away-from-player
run-away-from-player
can-attack-player
attack-player
too-far-from-player
too-close-to-player
stand-still

Here are some other primitives you may find handy in building a good stateless
AI:

player-is-friend
player-has-food
player-too-powerful
player-is-same-alignment
player-is-opposed-alignment
enough-buddies-in-pack
too-far-from-pack-center
can-move-toward-pack-center
move-toward-pack-center
too-close-to-pack-center
can-move-away-from-pack-center
move-away-from-pack-center

Stateless AI's aren't really general enough to model dynamic monster/monster

relationships, but if you want to try it you'll need a lot more primitives than just
these.

Ray Dillinger

Retrieved from "http://www.roguebasin.com

/index.php?title=Roguelike_Intelligence_-_Stateless_AIs&oldid=22340"
Categories: Articles AI

This page was last modified on 30 September 2010, at 16:28.