PLANNING

ELAINE RICH, KEVIN KNIGHT, SHIVASHANKAR B NAIR

CHAPTER 13
Pg No 247 - 262
 What does planning involve?
• Planning problems are hard problems:
• They are certainly non-trivial.
• Solutions involve many aspects that we have studied
so far:
– Search and problem solving strategies
– Knowledge Representation schemes
– Problem decomposition -- breaking problem into
smaller pieces and trying to solve these first.

5/14/2022 Unit 3 - Planning - R.Jayabhaduri 2

 Blocks World Planning Examples
• The Blocks World consists of:
– A flat surface such as a tabletop
– An adequate set of identical blocks
which are identified by letters.
– The blocks can be stacked one on
one to form towers of apparently
unlimited height.
– The stacking is achieved using a
robot arm which has fundamental
operations and states which can be
assessed using logic and combined
using logical operations.
– The robot can hold one block at a
time and only one block can be
moved at a time.

5/14/2022 Unit 3 - Planning - R.Jayabhaduri 3

 SOLAR
• University of California, Berkeley Artificial
Intelligence Research (BAIR) has introduced
a new reinforcement learning (RL)
method, Stochastic Optimal Control with
Latent Representations (SOLAR), which can
help robots quickly learn tasks such as
stacking blocks or pushing objects from
visual inputs.

Lego Block Stacking

Our robot learns to stack a Lego block and push a mug onto a coaster with only
inputs from a camera pointed at the robot. Each task takes an hour or less of
interaction to learn.
5/14/2022 Unit 3 - Planning - R.Jayabhaduri 4
 Mug Pushing
5/14/2022 Unit 3 - Planning - R.Jayabhaduri 5
5/14/2022
UC Berkeley’s RL-Powered SOLAR Accelerates Robotic Learning
Unit 3 - Planning - R.Jayabhaduri 6
 Teaching robots to see with Unity

simulate the complete pick-and-place system

on objects with unknown and arbitrary poses.

5/14/2022 Unit 3 - Planning - R.Jayabhaduri 7

 Four Actions & Five Predicates

• UNSTACK(A,B)-- pick up ON(A,B)-- block A is on block B.

clear block A from block B; ONTABLE(A)-- block A is on the
• STACK(A,B)-- place block A table.
using the arm onto clear CLEAR(A)-- block A has nothing
block B; on it.
• PICKUP(A)-- lift clear block HOLDING(A)-- the arm holds
A with the empty arm; block A.
ARMEMPTY-- the arm holds
• PUTDOWN(A)-- place the
nothing.
held block A onto a free
space on the table

5/14/2022 Unit 3 - Planning - R.Jayabhaduri 8

 Why Use the Blocks world as an example?

• sufficiently simple and well behaved.

• easily understood
• yet still provides a good sample environment to study
planning:
– problems can be broken into nearly distinct subproblems
– we can show how partial solutions need to be combined to
form a realistic complete solution.

5/14/2022 Unit 3 - Planning - R.Jayabhaduri 9

 PLANNING SYSTEM COMPONENTS

5/14/2022 Unit 3 - Planning - R.Jayabhaduri 10

 Planning System Components

• Simple problem solving tasks basically involve the

following tasks:
1. Choose the best rule based upon heuristics.
2. Apply this rule to create a new state.
3. Detect when a solution is found.
4. Detect dead ends so that they can be avoided.
5. More complex problem solvers often add a fifth task:
Detect when a nearly solved state occurs and use special
methods to make it a solved state.

5/14/2022 Unit 3 - Planning - R.Jayabhaduri 11

 I. Choice of best rule
• Methods used involve
– finding the differences between the current states
and the goal states and then
– choosing the rules that reduce these differences
most effectively.
– Means end analysis good example of this

5/14/2022 Unit 3 - Planning - R.Jayabhaduri 12

 II. Rule application

• Previously rules could be applied without any difficulty as

complete systems were specified and rules enabled the
system to progress from one state to the next.
• Now we must be able to handle rules which only cover parts
of systems.
• A number of approaches to this task have been used.
– STRIPS

5/14/2022 Unit 3 - Planning - R.Jayabhaduri 13

 III. Detecting Progress
• The final solution can be detected if
– we can devise a predicate that is true when the
solution is found and is false otherwise.
– requires a great deal of thought and requires a
proof.

5/14/2022 Unit 3 - Planning - R.Jayabhaduri 14

 Planning
• Operators are represented by three lists:
– [Preconditions:] specify the applicability of the operator
– [delete list:] specifies what does not hold any longer
– [add list:] specifies what newly holds

5/14/2022 Unit 3 - Planning - R.Jayabhaduri 15

 STRIPS Planning in the Blocks World
• Operator Pickup(x) can be represented as

5/14/2022 Unit 3 - Planning - R.Jayabhaduri 16

 Represent the operators
Operator(Unstack(x, y))
• Precondition : [on(X,Y), clear(X), handempty]
• Add : [holding(X),clear(Y)]
• Delete : [handempty,clear(X),on(X,Y)]

Operator(stack(X,Y))
• Precond : [holding(X),clear(Y)]
• Add : [handempty,on(X,Y),clear(X)],
• Delete : [holding(X),clear(Y)],

Operator(putdown(X))
• Precond : [holding(X)]
• Add : [ontable(X),handempty,clear(X)]
• Delete : [holding(X)]

5/14/2022 Unit 3 - Planning - R.Jayabhaduri 17

5/14/2022 Unit 3 - Planning - R.Jayabhaduri 18
 GOAL STACK PLANNING

5/14/2022 Unit 3 - Planning - R.Jayabhaduri 19

 Goal Stack Planning
• Basic Idea  : To handle interactive compound
goals uses goal stacks
Here the stack contains :
– goals,
– operators -- ADD, DELETE and PREREQUISITE lists
– a database maintaining the current situation for
each operator used.

5/14/2022 Unit 3 - Planning - R.Jayabhaduri 20

 The Planning System STRIPS
• Fikes and Nilsson
1971: STanford Research Institute Problem Solver
– initial state Init (given by predicate logical formulae)
– goal state goal (given by predicate logical formulae)
– set of rules (given by name, preconditions, delete list,
& add list )

5/14/2022 Unit 3 - Planning - R.Jayabhaduri 21

 A Simple Blocks World Problem
• Consider the following where wish to proceed
from the start to goal state.

5/14/2022 Unit 3 - Planning - R.Jayabhaduri 22

 Solution : A Simple Blocks World Problem
• Initially the goal stack is the goal state.
• Split the problem into four subproblems
• Two are solved as they already are true in the initial state --
ONTABLE(A), ONTABLE(D).
Method 1
ON(C,A)
ON(B,D)
ON(C,A) ^ ON(B,D) ^ ONTABLE(A) ^ ONTABLE(D)

Method 2
ON(B,D)
ON(C,A)
ON(C,A) ^ ON(B,D) ^ ONTABLE(A) ^ ONTABLE(D)

5/14/2022 Unit 3 - Planning - R.Jayabhaduri 23

5/14/2022 Unit 3 - Planning - R.Jayabhaduri 24
 Search Tree to Goal Stack

Method 1
ON(C,A)
ON(B,D)
ON(C,A) ^ ON(B,D) ^ ONTABLE(A) ^ ONTABLE(D)

First check whether ON(C,A) is true in the current state. Here false. How to make it true?
Out of 4 operators, STACK can make it true. Put STACK (C,A) on stack in the place of
ON(C,A) which yields as
STACK(C,A)
ON(B,D)
ON(C,A) ^ ON(B,D) ^ ONTABLE(A) ^ ONTABLE(D)

Precondition : CLEAR(A) ^ HOLDING(C). Now goal stack becomes

CLEAR(A)
HOLDING(C )
CLEAR(A) ^ HOLDING(C )
STACK(C,A)
ON(B,D)
ON(C,A) ^ ON(B,D) ^ ONTABLE(A) ^Unit
5/14/2022 ONTABLE(D)
3 - Planning - R.Jayabhaduri 25
• Check whether CLEAR(A) is true. False. UNSTACK(B,A) alone
can make it true resulting in the goal stack

ON(B,A)
CLEAR(B)
ARMEMPTY
ON(B,A) ^ CLEAR(B) ^ ARMEMPTY
UNSTACK (B,A)
HOLDING(C )
CLEAR(A) ^ HOLDING(C )
STACK(C,A)
ON(B,D)
ON(C,A) ^ ON(B,D) ^ ONTABLE(A) ^ ONTABLE(D)

5/14/2022 Unit 3 - Planning - R.Jayabhaduri 26

 Compare the top element of the goal stack ON(B,A) to the world
model. It is satisfied. Pop it off and consider the next goal
CLEAR(B). True again.
ON(B,A)
CLEAR(B)
ARMEMPTY
ON(B,A) ^ CLEAR(B) ^ ARMEMPTY
Now the first goal is true, the second is universally true, and the arm is empty.
Thus all top three goals are true means that we can apply the operator
UNSTACK(B,A) as all prerequisites are met. This gives us the first node in
database
ONTABLE(A)^ ONTABLE(C) ^ ONTABLE(D)^HOLDING(B) ^ CLEAR(A)
HOLDING(C )
CLEAR(A) ^ HOLDING(C )
STACK(C,A)
ON(B,D)
ON(C,A) ^ ON(B,D) ^ ONTABLE(A) ^ ONTABLE(D)

5/14/2022 Unit 3 - Planning - R.Jayabhaduri 27

• To satisfy HOLDING ( C ) – to make it true, PICKUP(C ) and
UNSTACK (C,x) where x could be any block from which C could be
unstacked.
• Create 2 branches of the search tree, corresponding to the
following goal stacks
Goal Stack 1 PICKUP(C ) Goal Stack 2 UNSTACK (C,x)
• ONTABLE(C ) ON(C,x)
• CLEAR(C ) CLEAR(C )
• ARMEMPTY ARMEMPTY
• ONTABLE(C ) ^ CLEAR(C ) ^ ARMEMPTY ONTABLE(C ) ^ CLEAR(C ) ^ ARMEMPTY
• PICKUP (C ) UNSTACK (C,x)
• CLEAR(A) ^ HOLDING(C ) CLEAR(A) ^ HOLDING(C )
• STACK(C,A) STACK(C,A)
• ON(B,D) ON(B,D)
• ON(C,A) ^ ON(B,D) ^ONTABLE(A) ^ ONTABLE(D) ON(C,A) ^ ON(B,D) ^ONTABLE(A) ^ ONTABLE(D)

5/14/2022 Unit 3 - Planning - R.Jayabhaduri 28

• Consider Goal stack 2 which contains a variable x, which appears
in 3 places.
• To satisfy ON(C,x), we have to stack C onto some block x.
• Then the goal stack will be as
CLEAR(x)
HOLDING(C )
CLEAR(x) ^ HOLDING(C )
STACK(C,x)
CLEAR(C )
ARMEMPTY
ON(C,x) ^ CLEAR(C ) ^ ARMEMPTY
UNSTACK(C, x)
CLEAR(A) ^ HOLDING(C)
STACK(C,A)
ON(B,D)
ON(C,A) ^ ON(B,D) ^ ONTABLE(A) ^ ONTABLE(D)

Strategy seems to be unproductive, hence it can be terminated..

5/14/2022 Unit 3 - Planning - R.Jayabhaduri 29

• Choose Method 1 - PICKUP(C )
• Top elements ONTABLE(C ) , CLEAR(C ) are satisfied – true
• ARMEMPTY is not true as HOLDING(B) is true.
• Two operators that can make ARMEMPTY true are - STACK(B,x) and
PUTDOWN(B)
• Choose STACK(B,D) by binding D to x in STACK operator, which makes the goal
stack as
CLEAR(D)
HOLDING(B)
CLEAR(D) ^ HOLDING(B)
STACK(B,D)
ONTABLE(C) ^ CLEAR(C ) ^ ARMEMPTY
PICKUP(C )
CLEAR(A) ^ HOLDING (C)
STACK(C,A)
ON(B,D)
ON(C,A) ^ ON(B,D) ^ ONTABLE(A) ^ ONTABLE(D)

5/14/2022 Unit 3 - Planning - R.Jayabhaduri 30

CLEAR(D) and HOLDING(B) are both true. Now the operation STACK(B,D) can
be performed using the world model
ONTABLE(A) ^ ONTABLE(C ) ^ ONTABLE(D) ^ ON(B,D) ^ ARMEMPTY

All the preconditions are satisfied, hence executed..

Solution
1. UNSTACK(B,A)
2. STACK(B,D)
3. PICKUP(C)
4. STACK(C,A)

5/14/2022 Unit 3 - Planning - R.Jayabhaduri 31

 Example 2 : Blocks World
https://www.cs.bham.ac.uk/~mmk/Teaching/AI/l9.html

Ontable(A) Ontable(B)
Ontable(B) Ontable(C)
On(D,A) On(D,B)
On(C,D) On(A,D)
Clear(C) Clear(A)
Clear(B) Clear(C)
Handempty
5/14/2022
Handempty
Unit 3 - Planning - R.Jayabhaduri 32
 Operators of the Blocks World

5/14/2022 Unit 3 - Planning - R.Jayabhaduri 33

 Example Plan Execution

Ontable(A) Ontable(A) Ontable(A)

Ontable(B) Ontable(B) Ontable(B)
On(D,A) On(D,A) On(D,A)
On(C,D) UNSTACK(C,D)---> On(C,D) PUTDOWN(C)-🡪 Clear(B)
Clear(C) Clear(C) Holds(C) UNSTACK(D,A)--->
Clear(B) Clear(B) Clear(D)
Handempty Handempty Clear(C)
Hold(C) Ontable(C)
Clear(D) Handempty
Ontable(A)
Ontable(B)
On(D,A)
CLEAR(B)
CLEAR(D)
CLEAR(C )
Ontable(C)
Handempty
Holds(D)
Clear(A)
5/14/2022 Unit 3 - Planning - R.Jayabhaduri 34
5/14/2022 Unit 3 - Planning - R.Jayabhaduri 35