Principles of Robot Autonomy II

Fundamentals of Grasping – Jeannette Bohg

What is so important about autonomous
grasping and manipulation?

2/1/23 AA 274B | Lecture 7 2

Beyond Mobility
Contact Interaction
Industrial Robotics
Medical Robotics
Service Robotics
Warehouse Robotics
Field & Disaster Relief Robotics
Ocean One
Human Robot Interaction
Why is grasping and manipulation
challenging?

2/1/23 AA 274B | Lecture 7 12

Pixels Policy

Torques
Kids Making Robots Jealous
 Learning Outcome for next four Lectures

Use Manipulation to Perceive better

Modeling and Evaluating Grasps

Apply Learning to Grasping and Manipulation

Modeling and Executing Manipulation

2/1/23 AA 274B | Lecture 7 19

Today’s itinerary
• What is a Grasp?
• Why is grasping hard?
• How to generate a grasp?
• How to evaluate a grasp?
• Analytical Approach to Modeling a Grasp
• Terminology
• Modeling of a Grasp
• Stability Analysis of a Grasp
• Form Closure
• Force Closure
• Generating a Grasp

2/1/23 AA 274B | Lecture 7 20

What is a grasp?
• Restraining an object’s motion through
application of forces and torques at a set of
contact points

The Allegro Hand

2/1/23 AA 274B | Lecture 7 http://www.simlab.co.kr/ 21
Why is grasping hard?
• High-dimensional search problem

The Allegro Hand

2/1/23 AA 274B | Lecture 7 http://www.simlab.co.kr/ 22
Why is grasping hard?
• High-dimensional search problem

All possible hand-object poses

Contact with object

Approach does not

collide
Grasps that can be
maintained The Allegro Hand
2/1/23 AA 274B | Lecture 7 http://www.simlab.co.kr/ 23
Why is grasping hard?
• Grasp Planning / Synthesis / Generation
• Searching the space of grasp parameters
• Grasp Acquisition
• Executing reaching and grasping to acquire the grasp
on the object
• Grasp Maintenance
• Maintaining the grasp while lifting the object or
manipulating something with it
• Grasp Evaluation
• Objective function for guiding the search or reacting
online during grasp acquisition/maintenance
The Allegro Hand
2/1/23 AA 274B | Lecture 7 http://www.simlab.co.kr/ 24
How to generate a grasp?
• How do robots parameterize their grasp
plan?
• Approach vector
• Wrist orientation of hand
• Initial finger configuration
• Points of contact
• How do you search this space?

http://openrave.org/docs/0.8.0/openravepy/ex
amples.graspplanning/

2/1/23 AA 274B | Lecture 7 25

How to evaluate a potential grasp?
What are good grasp characteristics?

Grasp Maintenance:
contact forces applied by the hand are
such that they prevent contact
separation and unwanted contact
sliding

Closure:
Grasps that can be maintained for every
possible disturbance load

2/1/23 AA 274B | Lecture 7 26

 B659: Principles of Intelligent Robot Motion. David Tidd, Duke University

Let’s model a grasp analytically!

Restraining an object’s motion through application of forces
and torques at a set of contact points

c1 c1

COM COM

c3 c3 c2

2/1/23 AA 274B | Lecture 7 27

 B659: Principles of Intelligent Robot Motion. David Tidd, Duke University

Terminology - Contact Types

• Point: point on plane (stable), point on point or line (unstable)
• Line: line on plane or nonparallel line (stable), line on parallel line
(unstable)
• Plane: plane on plane

Point-Plane Point-Line
Point-Point

2/1/23 AA 274B | Lecture 7 28

 B659: Principles of Intelligent Robot Motion. David Tidd, Duke University

Everything as a Point Contact

• Line contact -> 2 points
• Plane contact -> convex hull of points
• Any distribution of normal forces across a region can be represented
as a weighted sum of point forces along that region’s convex hull

2/1/23 AA 274B | Lecture 7 29

 B659: Principles of Intelligent Robot Motion. David Tidd, Duke University

Everything as a Point Contact

• Line contact -> 2 points
• Plane contact -> convex hull of points
• Any distribution of normal forces across a region can be represented
as a weighted sum of point forces along that region’s convex hull

2/1/23 AA 274B | Lecture 7 30

Terminology - Contact Models
• The kind of contact determines what components of contact force 𝑓
and moment 𝜏 are transmitted through contact
• Local reference frame at contact point, z aligned with normal pointing
inward

• Three common contact models

• Frictionless Point Contact
• Point Contact with Friction
• Soft-finger Contact

2/1/23 AA 274B | Lecture 7 31

Terminology - Frictionless Point Contact
• Forces can only be applied in direction normal to the surface of the
object
c1

COM

c3

2/1/23 AA 274B | Lecture 7 34

Terminology - Point Contact with Friction
• A point contact with friction can apply more than just a normal force
• “Friction cone” is the vector space of all possible/admissable forces a
point can apply due to friction (w/o slipping)
• Static Coefficient of friction μs

2/1/23 AA 274B | Lecture 7 35

Terminology - Linearized Friction Cone
• Pyramidal approximation converts vector space to finite set of vectors
• Space of possible applied wrenches

Linearized Friction Cone. Adopted from

Chapter 28, Fig 28.10 in Springer Handbook of Robotics.

2/1/23 AA 274B | Lecture 7 36

Terminology - Soft-finger Contact Model
• Also allows for torque around the normal
• Space of possible applied wrenches:

𝛾 Torsional friction coefficient

2/1/23 AA 274B | Lecture 7 37

Summary of Contact Models in Wrench Space
Wrench is 6D vector of forces and torques
applied at a contact.

𝒘 = 𝑊𝑟𝑒𝑛𝑐ℎ 𝑏𝑎𝑠𝑖𝑠 ∗ ℱ

Murray, Li, Sastry, A Mathematical Introduction to Robotic Manipulation, Chapter 5

2/1/23 AA 274B | Lecture 7 38

Terminology – Object Wrench
• Each contact point applies forces and torques (a wrench ) to the
object frame
• Stacked as a 6D vector

2/1/23 AA 274B | Lecture 7 39

Terminology – Object Wrench
• Forces 𝑓 ∈ ℝ! and torques 𝜏 ∈ ℝ! applied at a contact that act on the
object center of mass
• Stacked as 6D Vector written wrt to frame in object
c1

COM

𝑖 Index of contact c
𝑑 Vector defining position of
ith contact wrt to COM
𝑓 Force c3
𝜆 Arbitrary scaling of torque component

2/1/23 AA 274B | Lecture 7 40

Terminology - Grasp
• = set of all possible wrenches that can be achieved through the contact
points

• 𝐺! = wrench basis matrix including transformation from local contact-

reference frame to global object-centric reference frame
• G=[G1 ...Gk] =grasp map
• Transforming all applied forces and torques to achievable wrenches

2/1/23 AA 274B | Lecture 7 42

 B659: Principles of Intelligent Robot Motion. David Tidd, Duke University

Terminology - Grasp Wrench Space

• = the set of all possible wrenches that can be applied to an object
through admissible forces at k contact points

• For 3D objects, grasp wrench space

c1 is 6D fy
• 3D for force, 3D for torque
• For 2D objects, it’s 3D
COM

fx

τz
c2
2/1/23 AA 274B | Lecture 7 43
 B659: Principles of Intelligent Robot Motion. David Tidd, Duke University

Terminology - Grasp Wrench Hull

• = Convex hull of all the wrenches from the contact points
• Grasp Wrench hull ≤ Grasp Wrench Space
• Hull easier to compute
c1 fy

COM

fx

τz
c2
2/1/23 AA 274B | Lecture 7 44
Visualization of Hull from GraspIt! Simulator

2/1/23 AA 274B | Lecture 7 45

How to evaluate a grasp? Stability Analysis
What are good grasp characteristics?

Grasp Maintenance:
contact forces applied by the hand are
such that they prevent contact
separation and unwanted contact
sliding

Closure:
Grasps that can be maintained for every
possible disturbance load

2/1/23 AA 274B | Lecture 7 46

Form Closure
- Joint angles locked
- Palm fixed in space
- Impossible to move the object even
infinitesimally under arbitrary
external wrenches
- No wiggle room
- Power grasps, enveloping grasps

2/1/23 AA 274B | Lecture 7 47

Force Closure
- Grasp can be maintained under any
object wrench
- Forces can be applied at the
contact points to withstand the
external wrench
- Friction forces help balance the
wrench
- Fewer contacts needed compared
to Form Closure

2/1/23 AA 274B | Lecture 7 48

How to evaluate a grasp?
- Quantify:
- How many external wrenches can a grasp
withstand?

2/1/23 AA 274B | Lecture 7 49

Let’s break this down!
• Force direction closure with frictionless contacts
• When will set of point contact forces resist arbitrary translation?

2/1/23 AA 274B | Lecture 7 50

How many contacts needed? C-Space Analysis

2/1/23 AA 274B | Lecture 7 51

What does a point contact imply in C-Space?

2/1/23 AA 274B | Lecture 7 52

Condition for Force-Direction Closure

𝛼𝑓! + 𝛽𝑓" + 𝛾𝑓# = 0

2/1/23 AA 274B | Lecture 7 53

Condition for Force-Direction Closure

𝛼𝐩 + 𝛽𝐪 + 𝛾𝐫 = 0

2/1/23 AA 274B | Lecture 7 54

What is the minimum number of contacts for
force-direction closure on 3D object?

2/1/23 AA 274B | Lecture 7 55

Grasp Closure Analysis

2/1/23 AA 274B | Lecture 7 57

Grasp Analysis –Force Closure
• A grasp is a force-closure grasp IF for any external wrench 𝐹" there
exist contact forces 𝑓# ∈ 𝐹$ such that

𝐺 𝑓# = −𝐹"

i.e., if able to apply sufficient force at each contact, every external

wrench can be compensated for.

Practical test? Algebraic condition and using the Wrench hull

2/1/23 AA 274B | Lecture 7 58

 B659: Principles of Intelligent Robot Motion. David Tidd, Duke University

c1
Let’s go through an example!
• 2-point contacts

COM

c2

2/1/23 AA 274B | Lecture 7 59

 B659: Principles of Intelligent Robot Motion. David Tidd, Duke University

c1
Let’s go through an example!
• 2-point contacts d1

• 4 wrenches
f1,1 f1,2 COM
f2,1 f2,2
• Is this grasp stable?
d2

c2

2/1/23 AA 274B | Lecture 7 60

 B659: Principles of Intelligent Robot Motion. David Tidd, Duke University

c1
Let’s go through an example!
𝑓$
τout
d1

f1,1 f1,2 𝑓%
-fy fy COM
f2,1 f2,2

d2

τin
• Ignore for now 𝑓% c2

2/1/23 AA 274B | Lecture 7 61

Cross Product for Torque

The cross product a × b (vertical, in purple) changes as the

angle between the vectors a (blue) and b (red) changes. The
cross product is always orthogonal to both vectors, and has
magnitude zero when the vectors are parallel and maximum
magnitude ‖a‖‖b‖ when they are orthogonal.
2/1/23 AA 274B | Lecture 7 62
 B659: Principles of Intelligent Robot Motion. David Tidd, Duke University

c1
How do you make the grasp stable?
𝑓$
τout
w1,1 d1
w2,2

Wrench hull
w1,2 w2,1 f1,1 f1,2 𝑓%
-fy fy f2,1 f2,2

Does not d2
contain origin,
not stable

τin
• Ignore for now 𝑓% c2

2/1/23 AA 274B | Lecture 7 63

Algebraic Condition

𝛼𝑓! + 𝛽𝑓" + 𝛾𝑓# = 0

2/1/23 AA 274B | Lecture 7 64

 B659: Principles of Intelligent Robot Motion. David Tidd, Duke University

c1

What if there was a third point?

τout d1
w1,1 w2,2

Wrench hull f1,1 f1,2 COM

w1,2 w2,1 f3,1 f3,2 f2,1 f2,2
-fy

w3,2 d2
Does contain d3
origin, stable
w3,1

τin
c3 c2

2/1/23 AA 274B | Lecture 7 65

 B659: Principles of Intelligent Robot Motion. David Tidd, Duke University

Both grasps are stable. Which one is better?

c1 c1

d1 d1

f1,1 f1,2 COM f1,1 f1,2 COM

f3,1 f3,2 f3,1 f3,2 f2,1 f2,2

d2
d3 d3

c3 c3 c2

2/1/23 AA 274B | Lecture 7 66

 B659: Principles of Intelligent Robot Motion. David Tidd, Duke University

Grasp Quality
• Quality is how well a grip can resist disturbances
• Worst case scenario
• How efficiently can a grip resist disturbance wrenches at its weakest point?
• Weakest means the direction (in wrench space) at which the sum
normal force is converted to the desired wrench least efficiently

2/1/23 AA 274B | Lecture 7 67

 B659: Principles of Intelligent Robot Motion. David Tidd, Duke University

Worst Case Scenario

• The point on the wrench hull that w1,1
τout
w2,2
is closest to the origin is the
weakest point Hard to resist
w1,2 w2,1
• Disturbances in the opposite -fy +fy
direction are hardest to resist
ε w3,2
• Metric ε = The radius of the
largest ball that can be enclosed in
the wrench hull w3,1
• Varies from 0 to 1 due to normalization of τin
wrenches

2/1/23 AA 274B | Lecture 7 68

 B659: Principles of Intelligent Robot Motion. David Tidd, Duke University

Physical Meaning
• In the worst case, the sum magnitude of the contact wrenches would
need to be 1/ε times the disturbance wrench

2/1/23 AA 274B | Lecture 7 69

 B659: Principles of Intelligent Robot Motion. David Tidd, Duke University

Are these grasps equally good?

τout τout
w1,1 w1,1 w2,2

w1,2 w1,2 w2,1

-fy +fy -fy +fy

ε w3,2 ε w3,2

w3,1 w3,1
τin τin

2/1/23 AA 274B | Lecture 7 70

 B659: Principles of Intelligent Robot Motion. David Tidd, Duke University

Average Case Scenario

• How efficiently can a grip resist a disturbance wrench on average?
• Metric ν = Volume of the convex hull in wrench space
• The three-point contact has more volume, so it is more stable on
average

2/1/23 AA 274B | Lecture 7 71

Form Closure versus Force Closure
• Both are in a contact configuration that resists all
external disturbances.
• Note: Every form closure grasp is also in force closure

• Why do I need less contact points to be in Force

closure?

2/1/23 AA 274B | Lecture 7 72

How do we generate a grasp?

2/1/23 AA 274B | Lecture 7 73

Suggested Reading
• Lecture Notes!
• Constructing Force Closure Grasps.
Van-Duc Nguyen. IJRR 1988
• Planning Optimal Grasps. Carlo Ferrari
& John Canny. ICRA 1992
• Springer Handbook of Robotics.
Chapter - Grasping. 2016.
• Check out graphical method for
homework assignment

2/1/23 AA 274B | Lecture 5 74

Next time
• Fundamentals of Manipulation

2/1/23 AA 274B | Lecture 5 75