04 - Adversarial Search AI

C S E 4 8 1 : A RT I F I C I A L I N T E L L I G E N C E
CSE 481
ARTIFICIAL INTELLIGENCE
ADVERSARIAL SEARCH AND GAMES
Conventional Search
In conventional search
There is no adversary to consider

A solution is a method for finding goal
Heuristic techniques
Could find an optimal solution but not always
Games: Adversarial Search
Games are adversarial
A solution involves strategy that specifies a move for every

possible opponent reply
We can find an exact solution
Using search with alternating player moves and assuming an
opponent playing perfectly
The search explores the game tree to the conclusion and selects
the move with the guaranteed best outcome
Time limits may force an approximate solution

A heuristic evaluation function
Replaces the utility function
It evaluates the goodness of a game position

Examples: chess, checkers etc.
The Game Prototype
Formulating games as search problems

1. Initial State
The board position + the player to move
2. Successor Function
A list of (move, state) pairs specifying legal moves
3. Terminal Test
Detects a terminal state (i.e., detects if the game is finished)
4. Utility Function
Returns a numerical value for terminal states
Possibly as simple as 1 for win, 0 for lose, and -1 for draw
Game Search & Game Trees
From the initial state and successor function
We generate the game tree

Each turn is 1 ply (a layer of nodes representing states)
We use the search tree to determine the next move:

1. Search to terminal states
2. Use their utility function values + assumptions about opponent
moves
3. Pick the optimal next move
Example: a complete tree for an example game called

NIM (http://illuminations.nctm.org/ActivityDetail.aspx?ID=140)
A very small game: NIM Game Rules
Consider a game called NIM
It is played by two players
In this game a number of stones are placed in a pile
In each players turn, player can remove either 1 or 2

stones from the pile
The player who removes the last stone wins
A very small game: NIM Game Representation
Consider the game tree for

the 5 stone version of NIM.
[VAL] represents the first

player (We) node or MAX
node
(VAL)
represents
the
Opponent node or MIN
node
VAL is the current number

of stones in the pile
We start the game
A very small game: NIM Node Evaluation
If a node corresponds to MAX

move and the value is [0], then
that node indicates a Lose for
MAX and a utility value of 0 is
assigned to this node
Similarly a node with (0) is a

Win for MAX and a utility value
of 1 is assigned to this node
To propagate these values back

up the list from bottom up as
follow:
if we are evaluating a MAX node, find

the maximum of the utility function of
all branches going out from this node
if we are evaluating a MIN node find
the minimum of the utility function of
all branches going out from this node
Optimal Strategies: minimax
minimax values for a simple 2-ply example
here's the game tree

with terminal values from a utility function
10
now we back up the values for min nodes
11
then we back up values for max node
12
The optimal play assumption
Optimal play for MAX

Assumes that MIN also plays optimally
i.e. maximizes the worst-case outcome for MAX
What if the assumption doesn't hold

What if MIN does not play optimally?
In this case, MAX will do even better
Extending minimax
13
Pruning the game search tree
The number of game states is

Exponential in the number of moves
Thus, for real non-trivial games

There are just too many nodes
So, how to avoid examining every node in the tree?

We can still compute the correct minimax decision without
expanding all the nodes
We can prune the game tree by not expanding portions that

wouldn't make any difference (- pruning algorithm)
14
- pruning example
15
- pruning example
16
- pruning example
17
- pruning example
18
- pruning example
- Pruning Algorithm
19
Idea: if current path is already worse than some

other known path, stop expanding it.
Improves the speed of the mini-max search

algorithm
is the max lower bound of all solutions. i.e. the value of

the best move so far at any choice point on MAX path
is the min upper bound of all solutions. i.e. value for the
best move so far at any choice point on MIN path
20
- Pruning Algorithm Example

=-
=+
=-
=+
=-
=+
=-
=+
MAX
MIN
MAX
MIN
MAX
1) Setup phase: Assign to each left-most internal node of the tree,

variables: = -, = +
21

=-
=+
=-
=+
=-
=+
=-
=3
3
MAX
MIN
MAX
MIN
MAX
2) Look at first computed final configuration value. Its 3. Parent is a MIN node, so
set the (min) value to 3
22

=-
=+
=-
=+
MIN
=3
=+
=3
=+
=-
= 33
3
MAX
MAX
MIN
MAX
3) Look at next value, 5. Since parent is a MIN node, we want the minimum of
3 and 5 which is 3. Parent MIN node is done fill (max) value of its parent MAX
node. Always set for MAX nodes and for MIN nodes. Copy the state of the MAX parent
node into the second unevaluated MIN child
23

=-
=+
=-
=+
MIN
=3
=+
=3
=2
=-
= 33
3
MAX
MAX
MIN
MAX
4) Look at next value, 2. Since parent node is MIN with =+, 2 is smaller, change
24

=-
=+
=-
=+
=3
=+ 3
=-
= 33
3
MAX
MIN
=3
=2
2
2
MAX
MIN
MAX
5) Now, the MIN parent node has a max value of 3 and MIN value of 2. The value of the 2nd child
does not matter. if it is >2, 2 will be selected for MIN node. If it is <2, it will be selected for MIN node,
but since it is <3 it will not get selected for the parent max node. Thus, we prune the right sub-tree of
the MIN node. Propagate MAX value up the tree
25

=-
=+
=-
=3
=3
=+ 3
=-
= 33
3
MIN
=-
=3
=3
=2
2
2
MAX
=-
=3
MAX
MIN
MAX
6) MAX node is now done and we can set the value of its parent and propagate node
state to sibling sub-trees left-most path
26

=-
=+
=-
=3
=3
=+ 3
=-
= 33
3
MIN
=-
=3
=3
=2
2
2
MAX
=-
=3
10
MAX
MIN
MAX
7) The next node is 10. 10 is not smaller than 3, so state of parent does not change. We
still have to look at the 2nd child since is still
27

=-
=+
MAX
=-
=3
=3
=+ 3
=-
= 33
3
=3
=2
2
2
MIN
=4
=3
4
=-
=3
4
MIN
10
MAX
MAX
8) The next node is 4. Smallest value goes to the parent MIN node. MIN sub-tree is
done, so the parent MAX node gets the (max) value from the child. Note that if the
MAX node had a 2nd sub-tree, we can prune it since >
28

=3
=+
MAX
=-
=3 3
=3
=+ 3
=-
= 33
3
=3
=2
2
2
=3
=+
=4
=3
4
=-
4 = 3
10
=3
=+
=3
=+
MIN
MAX
MIN
MAX
9) Continue propagating value up the tree, modifying the corresponding and values.
Also, propagate the state of root node down the left-most path of the right sub-tree
29

=3
=+
MAX
=-
=3 3
=3
=+ 3
=-
= 33
3
=3
=2
2
2
=3
=+
MIN
=4
=3
4
=-
4 = 3
=3
=+
2
=3
2 = 2
MIN
10
MAX
MAX
10) Next value is a 2. We set the (min) value of the MIN parent to 2. Since no other
children exist, we propagate the value up the tree
30

=3
=+
MAX
=-
=3 3
=3
=+ 3
=-
= 33
3
=3
=2
2
2
=3
=2
MIN
=4
=3
4
=-
4 = 3
=3
=+
2
=3
2 = 2
MIN
10
MAX
MAX
11) We have a value for the 3rd level MAX node, now we can modify the (min) value of
the MIN parent to 2. Now, we have a situation that > and thus the value of the
rightmost sub-tree of the MIN node does not matter, so we prune the whole sub-tree
31

=3
=+
=-
=3 3
=3
=+ 3
=-
= 33
3
=3
=2
2
2
MAX
=3
2=2
=4
=3
4
=-
4 = 3
10
=3
=+
2
=3
2 = 2
2
MIN
MAX
MIN
MAX
12) Finally, no more nodes remain, we propagate values up the tree. The root has a value of 3 that
comes from the left-most child. Thus, the player should choose the left-most childs move in order to
maximize his/her winnings. The result is the same as with the original minimax algorithm, but we did not
visit all nodes of the tree
Comments on - Pruning Algorithm
32
Pruning does not change the eventual results though

entire sub-trees can be pruned
Its effectiveness depends partly on
The order of examining successors

Perfect ordering would be best successors first which allows
earliest pruning
With "perfect ordering" time complexity is O(bm/2)
The branching factor reduces to sqrt(b)

From an alternative view, given - pruning and a fixed
amount of time we can look twice as far ahead as with simple
minimax
33
Dealing with games with large game trees
What to do when the game tree is too large
Complete search is not feasible

We can't guarantee optimal play
Replace the utility function with a heuristic evaluation
function that assigns an estimate to intermediate nodes
without going all the way down to the terminal nodes
Non-Optimal Game Trees
34
Heuristic evaluation function
Evaluation function should

1. order terminal states as the utility function would
2. be economical to compute
3. when applied to non-terminals
Be highly correlated with the actual chance of winning
This introduces uncertainty about the outcome
This uncertainty is due to computational limitations
The game is still characterized by complete information
Non-Optimal Game Trees
35
Cutoff depth and heuristic evaluation function
The simplest cutoff is a fixed depth limit

Chosen to allow a move choice within the time limit
A more robust approach

Uses iterative deepening
Selects its move from deepest search completed in time

04 - Adversarial Search AI

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

04 - Adversarial Search AI

Uploaded by

Copyright:

Available Formats

C S E 4 8 1 : A RT I F I C I A L I N T E L L I G E N C E

There is no adversary to consider

Games: Adversarial Search

Games are adversarial

A solution involves strategy that specifies a move for every

Time limits may force an approximate solution

It evaluates the goodness of a game position

The Game Prototype

Formulating games as search problems

Game Search & Game Trees

From the initial state and successor function

We generate the game tree

We use the search tree to determine the next move:

Example: a complete tree for an example game called

A very small game: NIM Game Rules

Consider a game called NIM

It is played by two players

In this game a number of stones are placed in a pile

In each players turn, player can remove either 1 or 2

The player who removes the last stone wins

A very small game: NIM Game Representation

Consider the game tree for

[VAL] represents the first

VAL is the current number

We start the game

A very small game: NIM Node Evaluation

If a node corresponds to MAX

Similarly a node with (0) is a

To propagate these values back

if we are evaluating a MAX node, find

Optimal Strategies: minimax

minimax values for a simple 2-ply example

here's the game tree

Optimal Strategies: minimax

minimax values for a simple 2-ply example

now we back up the values for min nodes

Optimal Strategies: minimax

minimax values for a simple 2-ply example

then we back up values for max node

Optimal Strategies: minimax

The optimal play assumption

Optimal play for MAX

What if the assumption doesn't hold

Pruning the game search tree

The number of game states is

Thus, for real non-trivial games

So, how to avoid examining every node in the tree?

We can prune the game tree by not expanding portions that

Idea: if current path is already worse than some

Improves the speed of the mini-max search

is the max lower bound of all solutions. i.e. the value of

- Pruning Algorithm Example

1) Setup phase: Assign to each left-most internal node of the tree,

- Pruning Algorithm Example

- Pruning Algorithm Example

- Pruning Algorithm Example

- Pruning Algorithm Example

- Pruning Algorithm Example

- Pruning Algorithm Example

- Pruning Algorithm Example

- Pruning Algorithm Example

- Pruning Algorithm Example

- Pruning Algorithm Example