Getting these characters to behave in a way that is both interesting and (from the player point of view) logically consistent with the game world is a tremendous challenge.

For the moment, we'll divide control of the AI characters into two broad areas:

• control of movement, and
• control of behaviour: actions, speech, etc

Character Movement

We will consider several approaches to controlling character movement:

• Random movement

• Pattern based movement

• Path finding

Random movement

The easiest of approaches from a code/logic point of view, random movement periodically sets or changes the speed and direction of travel for the AI.

Speed/direction might be reset at fixed intervals, and/or in response to specific triggers (e.g. when the AI reaches an intersection, finds its way blocked, etc).

This can be handy for wandering characters, adding a level of unpredictability and variation to the game, but care must be taken to avoid having the character do things that detract from believability or gameplay, e.g.

• if the AI movement is totally random, it might not synch with its behaviour (e.g. moving away from the player while attacking)
• random movement might not be beleivable, e.g. a guard that wanders away from player characters it should probably be intercepting
• random movement can cause impossible situations for the player, e.g. a variety of random AI all converging on the player at once, or blocking a position or item the player is required to use
• random movement can also interrupt scripted storylines, e.g. a random AI blundering into one of the scripted showdowns or scenarios in the game, disrupting the intended pattern of behaviour or altering the intended difficulty of the situation

Of course, for the character to change speed/direction in response to triggers (e.g. intersections, dead ends, etc) then you must check for such events.

Pattern based movement

Another simple approach to AI movement is to establish fixed paths for the AI to follow.

This might be a circular path the AI repeats over and over, or a point-to-point path, dictating where it starts, where it ends, and how it moves in between.

Path patterns can be time based or point based, e.g.

• point based
  move towards point (x1,y1) at speed s1,
  when that point is reached, move towards point (x2,y2) at speed s2,
  etc.
  This requires checking at regular intervals to see if the character has reached its curent target
• time based
  move in direction d1 at speed s1 for time t1,
  then move in direction d2 at speed s2 for time t2,
  etc.
  this typically requires setting timers to indicate when the next change is scheduled

As with random movement, care must be taken to ensure the planned paths do not create illogical or unduly difficult situations for the player.

Path finding

The idea in path finding is to have the AI assess where it is, where it wants to be, and to have it rationally choose how to get there.

Again, there are a variety of ways to achieve this, with a few of them outlined below.

• right hand avoidance
  If an AI needs to get from point A to point B, one approach is as follows:
  • whenever it is possible, the AI moves in a direction directly towards point B
  • whenever the AI finds its way blocked, it turns to the right (clockwise) and moves forward until it has the opportunity to turn to the left (counter clockwise) and get around the obstruction
  This is relatively simple to program, but often leads to very poor choices on the part of the AI.
• forward avoidance
  In a slightly different approach, have the AI look ahead along a line from its current position to its objective.

  If there is an obstacle in the way, have the AI pick a new path to the edge of the object, and once it reaches the edge of the object have it recalculate a path to its objective.

  	       AI
  	       |\
  	       | \
  	       |  \
  	    ****** \
  	    ****** /
  	       |  /
  	       | /
  	       |/
  	     Objective
  	 

  Two questions the designer must ask themselves at this point are "How far ahead can the AI look?", and "Can it see around corners?"

  e.g. should the AI be able to plan/react to things that a player wouldn't be able to see in the same situation.

  Which obstacles and openings the AI can actually 'see' has a significant impact on the effectiveness of its path planning.

• best path
  'Best' path algorithms are used to evaluate how easy/fast/safe different routes are for getting from point A to point B.

  Typically the AI treats the map as a grid, with fixed locations it can move into.

  The AI looks at where it is and where it wants to be, and assigns every position in the grid a numeric value, with lower numbers indicating more desirable positions along the way (e.g. safer, easier to move through, etc)

  The AI then looks at each possible path (sequence of grid positions) from A to B, tallying up the numbers along that path.

  The path with the lowest total is designated the best path, and the one the AI will follow.

  The trick is finding an efficient way to evaluate every possible path from A to B. One algorithm is outlined below:

  • Assign each grid position its value (based on how easy it is to move through, how safe it is, etc) with lower numbers indicating faster/safer positions
  • Assign each grid position a path value - this is the 'cost' to get to that position from A. Initially set all the costs to any ridiculously large number.
  • Create a queue to hold a collection of grid positions
  • Give position A a path cost of 0 (since the AI is already there) and put position A into the queue
  • While the queue isn't empty:
    • Extract the next position, let's call it P, from the front of the queue
    • For every position you can move to directly from P:
      - compute the cost of reaching it from P as the cost of reaching P plus the value of the other position
      - if the cost is lower than the cost previously associated with the position, change that position's cost (you've found a new, cheaper, way to get there by going through P)
      - if the other position isn't currently in the queue then add it to the back of the queue

  You can improve efficiency by adding heuristics (rules of thumb) to try and eliminate paths/positions that are obviously off track, but the core of the algorithm remains the same.

• pursuit/avoidance
  A special case of path planning is where one character is chasing another, or trying to flee from another character.

  You could simply use the path planning ideas above to have a character try and move directly towards or directly away from its rival (basic chase/flee strategies) or you could try to project where your rival is going and move towards or away from that point (intercept courses/avoidance).

  The prediction aspects can be made as simple or as complex as desired: you could simply assume your rival will keep going in the same direction at the same speed and compute your best speed and direction to arrange an intercept, or you could also try and predict changes in your opponent's speed and direction (though this requires some understanding of their likely behaviour).

Even with path finding algorithms, care must still be taken to ensure the paths do not create illogical or unduly difficult situations for the player.

Lab exercises on AI path planning and movement control

Character Behaviour

Their actions will determine, in large part, how believable the game world is and how interesting and challenging the gameplay is.

If these characters existed in the real world, they would be constantly evaluating the world around them, observing what is taking place, planning for the future, etc. However, we lack the processing power (not to mention the coding skills) to emulate every aspect of this in a game.

As a result, we need some shorthand control approaches that make the character look like they're really 'thinking', but require less code and less processing power to evaluate.

This usually means identifying a small number of key actions, stimuli, goals, and events RELEVANT TO THE IN-GAME SITUATION that will be used to determine each character's actions.

There are several common techniques applied to determine character actions. We'll consider four approaches briefly, using each of them to model AI poker players.

The four techniques we'll look at are (pseudo) random behaviour, neural networks, rule based systems, and state machines. Before we look at the four techniques, let's discuss example a bit.

The idea is to have different AI characters that play poker against the human player(s), hopefully creating an interesting and challenging game for the player.

The general idea of the gameplay is that each hand consists of multiple rounds of drawing or revealing cards and betting based on the information available. (Details would of course depend on the specific flavour of poker being played.)

At each betting stage, the AI has five actions to choose from:

• fold
• check/call
• small raise
• medium raise
• big raise

If the AI was another human, the information it would have available would be:

• knowledge of all upcards so far
• knowledge of its own discards so far
• knowledge of its own hole cards so far
• knowledge of the amount bet on the current hand so far
• knowledge of each player's current cash holdings
• knowledge of the rules of the game
• some knowledge of the odds associated with different card combinations

Because it is an AI, we could give the AI additional information (e.g. much more detailed calculations of the odds, possibly even knowledge about the player's hand - giving the AI a cheat to increase its difficulty).

Based on the informations and actions available, let's consider four approaches to determining AI behaviour.

Random behaviour

One of the easiest approaches to program that gives some degree of interest to AI actions is random behaviour.

We could assign each AI player different odds of picking each of the five available actions. For each AI player, generate a random value in the range 0-99 and consult the following table.

Action	AI 1	AI 2	AI 3
Fold	<=10	<=15	<=20
Call/check	11-30	16-30	21-59
Small raise	31-55	31-60	60-80
Medium raise	56-80	61-89	80-94
Large raise	81+	90+	95+

This would make AI 1 appear to be the more aggressive of the three characters, and AI 3 to be the more conservative. Unfortunately, since it ignores the information available to the AI, it will sometimes result in very poor decisions. (For example, an AI with an outstanding hand may fold.)

We could improve on this a bit by adding modifiers to the behaviour, e.g.:

• for each character's hand that is visibly better than your own, subtract 5 from the random value generated
• if your own hand is better than every showing hand, add 10 to the random value generated
• for every character still "in" on the current hand, if they have more cash than you then subtract 3 from the value generated otherwise add three to the value generated
• etc

Rule based systems

Rule based systems specify precise conditions for character behaviour.

This tends to give them more logical and consistent behaviour, but can involve intense coding and debugging and can also make the AI too predictable over long periods of gameplay.

A very simple example of rule-based behaviour might look something like:

• If your hand is better than everything showing on the table and if your cash on hand is at least five times the maximum bet allowed, then raise the maximum.
• Otherwise, if your hand is as good as everything showing on the table and if your cash on hand is at least three times the maximum bet allowed, then make a medium raise.
• Otherwise, if your hand is as good as everything showing on the table, then make a minimal raise.
• Otherwise, calculate your odds of drawing/turning over a hand that is better than everything showing on the table. If those odds are better than 50% then call/check.
• Otherwise fold.

Note that if you wanted different AI gambling personalities then you would need to either create seperate rule sets for each or include modifiers to the general behaviour rules, and set the modifiers differently for each AI.

In fact, a combination of rule based and random behaviour is often used - giving the AI reasonably logical behaviour but still providing a significant degree of unpredictability. (The modifiers used in the random behaviour section are an example of this blended approach.)

Neural networks

Another approach to try to simplify the decision making process (and code) associated with the AI, yet still create flexibility among AI personas, is to use a simplified neural net approach.

In its simplest form, this works as follows:

• map each element of AI information into a numeric value
• assign a weight to each element of AI information
• associate a range of values with each potential AI action
• multiply each information element by its weight, add all the values together, and see which action range the result falls into.

For example, the first table below lists the information the AI will take into account, and how it will weight that information.

The second table shows what range of final values will be associated with each action.

Information Weighting

Information	AI 1	AI 2	AI 3
Number of players "in"	-1	1	0
Number of players with better showing hands	-2	-1	-3
Cash on hand divided by maximum bet	2	1	0
Number of betting rounds remaining	2	1	0
Amount of cash in the pot	3	2	3

Action Determination

Action	AI 1	AI 2	AI 3
Fold	< 0	<1	< -1
Call/check	0..2	1..4	-1..4
Small raise	2..5	4..6	4..8
Medium raise	5..8	6..10	8..16
Large raise	8+	10+	16+

For example, to determine the action for AI 1:

• take the number of players still "in" and multiply by -1
• take the number of players with better showing hands and multiply by -2
• take your cash on hand divided by the maximum bet and multiply by 2
• take the number of betting rounds remaining in the hand and multiply by 2
• take the amount of cash in the pot and multiply by 3
• sum all these values together, then
  • if the value is less than 0 then fold
  • if the value is between 0 and 2 then call/check
  • if the value is between 2 and 5 then make a small raise
  • if the value is between 5 and 8 then make a medium raise
  • if the value is greater than 8 then make the maximum raise

State based systems