[Grab your reader’s attention with a great quote from the document or use this space to emphasize a key

point. To place this text

box anywhere on the page, just drag it.]

ROBOT DEXTERITY
T E C H N O L O G I C A L
A D V A N C E M E N T
IF ROBOTS ARE REALLY TO HELP US OUT AROUND THE HOUSE OR CARE FOR OUR INJURED
AND ELDERLY, THEY’RE GOING TO WANT TWO HANDS… AT LEAST. BUT USING TWO HANDS
IS HARDER THAN WE MAKE IT LOOK — SO THIS ROBOTIC CONTROL SYSTEM LEARNS FROM
HUMANS BEFORE ATTEMPTING TO DO THE SAME.
THE IDEA BEHIND THE RESEARCH, FROM THE UNIVERSITY OF WISCONSIN-MADISON, ISN’T TO
BUILD A TWO-HANDED ROBOT FROM SCRATCH, BUT SIMPLY TO CREATE A SYSTEM THAT
UNDERSTANDS AND EXECUTES THE SAME TYPE OF MANIPULATIONS THAT WE HUMANS DO
WITHOUT THINKING ABOUT THEM.
FOR INSTANCE, WHEN YOU NEED TO OPEN A JAR, YOU GRIP IT WITH ONE HAND AND MOVE
IT INTO POSITION, THEN TIGHTEN THAT GRIP AS THE OTHER HAND TAKES HOLD OF THE LID
AND TWISTS OR POPS IT OFF. THERE’S SO MUCH GOING ON IN THIS ELEMENTARY TWO-
HANDED ACTION THAT IT WOULD BE HOPELESS TO ASK A ROBOT TO DO IT AUTONOMOUSLY
RIGHT NOW. BUT THAT ROBOT COULD STILL HAVE A GENERAL IDEA OF WHY THIS TYPE OF
MANIPULATION IS DONE ON THIS OCCASION, AND DO WHAT IT CAN TO PURSUE IT.
THE RESEARCHERS FIRST HAD HUMANS WEARING MOTION CAPTURE EQUIPMENT PERFORM
A VARIETY OF SIMULATED EVERYDAY TASKS, LIKE STACKING CUPS, OPENING CONTAINERS
AND POURING OUT THE CONTENTS, AND PICKING UP ITEMS WITH OTHER THINGS BALANCED
ON TOP. ALL THIS DATA — WHERE THE HANDS GO, HOW THEY INTERACT AND SO ON — WAS
CHEWED UP AND RUMINATED ON BY A MACHINE LEARNING SYSTEM, WHICH FOUND THAT
PEOPLE TENDED TO DO ONE OF FOUR THINGS WITH THEIR HANDS:
  SELF-HANDOVER: THIS IS WHERE YOU PICK UP AN OBJECT AND PUT IT IN THE OTHER HAND SO
IT’S EASIER TO PUT IT WHERE IT’S GOING, OR TO FREE UP THE FIRST HAND TO DO SOMETHING
ELSE.

 ONE HAND FIXED: AN OBJECT IS HELD STEADY BY ONE HAND PROVIDING A STRONG, RIGID
GRIP, WHILE THE OTHER PERFORMS AN OPERATION ON IT LIKE REMOVING A LID OR STIRRING
THE CONTENTS.

 FIXED OFFSET: BOTH HANDS WORK TOGETHER TO PICK SOMETHING UP AND ROTATE OR
MOVE IT.

 ONE HAND SEEKING: NOT ACTUALLY A TWO-HANDED ACTION, BUT THE PRINCIPLE OF
DELIBERATELY KEEPING ONE HAND OUT OF ACTION WHILE THE OTHER FINDS THE OBJECT
REQUIRED OR PERFORMS ITS OWN TASK.

THE ROBOT PUT THIS KNOWLEDGE TO WORK NOT IN DOING THE ACTIONS ITSELF — AGAIN, THESE ARE
EXTREMELY COMPLEX MOTIONS THAT CURRENT AIS ARE INCAPABLE OF EXECUTING — BUT IN ITS
INTERPRETATIONS OF MOVEMENTS MADE BY A HUMAN CONTROLLER.

YOU WOULD THINK THAT WHEN A PERSON IS REMOTELY CONTROLLING A ROBOT, IT WOULD JUST
MIRROR THE PERSON’S MOVEMENTS EXACTLY. AND IN THE TESTS, THE ROBOT DOES SO TO PROVIDE A
BASELINE OF HOW WITHOUT KNOWLEDGE ABOUT THESE “BIMANUAL ACTIONS,” BUT MANY OF THEM
ARE SIMPLY IMPOSSIBLE.

THINK OF THE JAR-OPENING EXAMPLE. WE KNOW THAT WHEN WE’RE OPENING THE JAR, WE HAVE TO
HOLD ONE SIDE STEADY WITH A STRONGER GRIP AND MAY EVEN HAVE TO PUSH BACK WITH THE JAR
HAND AGAINST THE MOVEMENT OF THE OPENING HAND. IF YOU TRIED TO DO THIS REMOTELY WITH
ROBOTIC ARMS, THAT INFORMATION IS NOT PRESENT ANY MORE, AND THE ONE HAND WILL LIKELY
KNOCK THE JAR OUT OF THE GRIP OF THE OTHER, OR FAIL TO GRIP IT PROPERLY BECAUSE THE OTHER
ISN’T HELPING OUT.

THE SYSTEM CREATED BY THE RESEARCHERS RECOGNIZES WHEN ONE OF THE FOUR ACTIONS ABOVE IS
HAPPENING, AND TAKES MEASURES TO MAKE SURE THAT THEY’RE A SUCCESS. THAT MEANS, FOR
INSTANCE, BEING AWARE OF THE PRESSURES EXERTED ON EACH ARM BY THE OTHER WHEN THEY PICK
UP A BUCKET TOGETHER. OR PROVIDING EXTRA RIGIDITY TO THE ARM HOLDING AN OBJECT WHILE
THE OTHER INTERACTS WITH THE LID. EVEN WHEN ONLY ONE HAND IS BEING USED (“SEEKING”), THE
SYSTEM KNOWS THAT IT CAN DEPRIORITIZE THE MOVEMENTS OF THE UNUSED HAND AND DEDICATE
MORE RESOURCES (BE IT BODY MOVEMENTS OR COMPUTATIONAL POWER) TO THE WORKING HAND.