2014 VL CogSys 5

Fakultät
für Informa0k
der Technischen Universität München
Cogni&ve Systems
Robotics and Embedded Systems (Informatik VI, TUM)

fortiss GmbH
Cogni0ve Systems
SS 2014 1
Lehrstuhl Informa0k VI – Robo0cs and Embedded Systems

Fakultät für Informa0k
Contents of the Lecture

18/08/2014 – 22/08/2014
18.8. 19.8. 20.8. 21.8. 22.8.
Neural
09:00-‐10:30 Introduc0on Grasping Percep0on Natural
Modelling
Language
Cogni0ve Cogni0ve Interfaces
11:00-‐12:30 SoU Robots ***
Architectures Architectures
13:30-‐15:00 Signal
Sensors + Excursion
ROS + Robo6cs processing +
Object Manipula6on For&ss
Library graphical
15:30-‐17:00 detec6on
models
Cogni0ve Systems
SS 2014 2

Basic Architecture of Robot System
Communica0on and Adapta0on between A. Knoll et al.,

IEEE-ICRA 1997
•  Robot ↔ Human instructor/co-‐worker
•  Robot ↔ Environment
Cogni0ve Systems 3
SS 2014

The natural language processing pipeline

Bird, Klein & Loper (2009)
Cogni0ve Systems
SS 2014 4

Cognitive Systems:
Interaction and Language
Cogni0ve Systems
SS 2014 5

Human-‐Robot Interac&on
To build a collabora0ve robot we need:
§  Input modali0es
§  Reasoning
§  Output channels
§  Robot architecture
§  Safety for the human
§  “Socialness”
§  ... and a lot more
Cogni0ve Systems
SS 2014 6

Evalua&on of Dialogue Systems
§  Dialogue efficiency

§  Overall 0me taken, number of system turns
§  Mean response 0me (system and user)
§  Dialogue quality
§  Number of 0mes the robot gave assembly instruc0ons
§  User gaze behaviour before and aUer system-‐generated references
§  Task success
§  Propor0on of target objects assembled correctly
Cogni0ve Systems
SS 2014 7

JAMES Project: Joint Ac&on for Mul&modal Embodied Social Systems
Social robots that work

in public spaces:
Drink serving in bar as
running example
hkp://www.james-‐project.eu
Cogni0ve Systems 8
SS 2014

User modelling and social signal processing

Mo&va&on
•  For the robot it is most important to be able to recognise when a customer
wants to start and end an interac&on
•  University of Bielefeld recorded and annotated interac0ons in real bars
•  Focus on analysis of ordering requests
•  Very detailed manual analysis of 108 interac0ons

Cogni0ve Systems
SS 2014

Cogni0ve Systems 10
SS 2014

Automaton of bar interac&on script

According to video transcrip0ons by Bielefeld University
Cogni0ve Systems
SS 2014

User modelling and social signal processing

Recorded data show that humans use three cues to indicate that they want to
start an interac0on
•  Body posture (posi0on and orienta0on)
•  Head pose
•  Face-‐to-‐Face-‐forma0on

N-‐forma0on H-‐forma0on V-‐forma0on L-‐forma0on
Cogni0ve Systems
SS 2014

Training of Hidden Markov Model (HMM)
§  Body posture and f-‐forma0ons from Kinect data

§  Head pose es0ma0on algorithm by
Vatahska / Bennewitz / Behnke
§  Recordings of more than 200 scenes in front of the JAMES
robot
§  Labelling of 1,720 states
–  1,010 training
–  319 cross valida0on
–  391 tes0ng
Cogni0ve Systems
SS 2014

HMM states and transi&ons
Cogni0ve Systems
SS 2014

Cogni0ve Systems 15
SS 2014

Human-‐robot collabora&on
Cogni0ve Systems
SS 2014 16

Robots in industrial produc&on today

§  Highly automated produc0on
§  welding, glueing, …
§  stacking, sor0ng, …
§  Off-‐line programmed robots work on
their own
§  It is s0ll an “art” to design and program the robots
§  mostly not adap0ve to environmental changes
§  Strict separa0on of workspaces
between robots and humans
§  lack of percep0on
§  safety aspects
Cogni0ve Systems
SS 2014 17

Humans in industrial produc&on
§  Human dexterity needed for:

§  cabling
§  moun0ng of filigree parts
§  sure ins0nct
§  high quality goods

§  Human eyes needed for:
§  inspec0on and patching
§  fix assembly mistakes
§  controlling complex products

§  Human flexibility needed for:
§  precise work with unknown parts
§  handling of limb parts
§  customized assembly
Cogni0ve Systems
SS 2014 18

Challenge: combining the skills
§  Human
§  easy handling of filigree tasks
(sleight of hand)
§  fast adap0on to changes in the environment
§  fast adjustment to new jobs
§  Industrial robot

§  precise repe00on of movements
§  0me op0mized execu0on of tasks that are
op0mized for the robot
§  non-‐stop working without exhaus0on
Cogni0ve Systems
SS 2014 19

Further Reading
R. Pieraccini, The Voice in the Machine. Cambridge, MA: MIT Press, 2012.

D. Ferrucci, E. Brown, J. Chu-‐Carroll, and J. Fan, “Building Watson: An overview of the DeepQA
project,” AI Magazine, pp. 59–79, 2010

S. Bird, E. Klein, and E. Loper, Natural Language Processing with Python: Analyzing Text with the
Natural Language Toolkit. Sebastopol, CA: O’Reilly, 2009

A. Knoll & I. Glöckner. A Basic System for Mul6modal Robot Instruc6on. In: P. Kühnlein, H.
Rieser, H. Zeevat (eds.) Perspec0ves on Dialogue in the New Millenium, Pragma0cs and Beyond
New Series, Vol. 114, John Benjamins, 2003

A. Knoll, B. Hildebrandt, and J. Zhang. Instruc6ng coopera6ng assembly robots through situated
dialogues in natural language. In Proc. 1997 IEEE Conf. on Robo6cs and Automa6on,
Albuquerque, New Mexico. IEEE, April 1997
Cogni0ve Systems
SS 2014 20


•  Robot ↔ Human instructor/co-‐worker IEEE-ICRA 1997
Cogni0ve Systems 21
SS 2014

Cognitive Systems:
Planning
Cogni0ve Systems
SS 2014 22

What is planning?

Many different planning tasks for cogn0ve systems:
•  Path/mo0on planning
•  Assembly planning
•  …

“Planning is the art and prac0ce of thinking before ac0ng.“ Patrik Haslum, ANU
Cogni0ve Systems
SS 2014 23

Descrip&on of a planning task

Elements:
•  Goals Planning task:
•  State descrip0ons Given an ini0al state, find a sequence of
ac0ons which is supposed to lead me to a goal
•  Possible ac0ons
Different levels of abstrac0on, example: drive from A to B

State descrip&ons Ac&ons Required knowledge
Topological posi0on descrip0on Choose road at a crossing Schema0c road map
Posi0on on map Drive, change road geometrical road map
Actual posi0on on road Accelerate, break, steer Precise map, environment model
Cogni0ve Systems
SS 2014 24

Planning for cogni&ve systems

•  Task planning
•  State descrip0on: akributes, proper0es
-  Symbolic, discrete
-  Measurable?
-  Intermediate states?
•  Ac0ons: state transi0ons with precondi0ons
and state changes
-  Assumed to be atomic
-  Time for ac0on execu0on?
•  (Low level) mo0on planning

•  State descrip0on: posi0on, joint angles
•  Ac0ons: control commands
In real life, both have to be combined.

Cogni0ve Systems
SS 2014 25

Classical symbolic planning language: STRIPS
Representa0on of states

•  Decompose the world in logical condi0ons and represent a state
as a conjunc0on of posi0ve literals.
•  Proposi0onal literals: Poor ∧ Unknown
•  First Order-‐literals (grounded and func0on-‐free):
At(Plane1, Melbourne) ∧ At(Plane2, Sydney)
•  Closed world assump0on
Representa0on of goals

•  Par0ally specified state and represented as a conjunc0on of posi0ve ground literals
•  A goal is sa0sfied if the state contains all literals in goal

Cogni0ve Systems
SS 2014 26

Classical symbolic planning language: STRIPS (II)
Representa0ons of ac0ons

Ac0on = PRECOND + EFFECT
Ac0on(Fly(p,from, to),
PRECOND: At(p,from) ∧ Plane(p) ∧ Airport(from) ∧ Airport(to)
EFFECT: ¬At(p,from) ∧ At(p,to))

= ac0on schema (p, from, and to need to be instan0ated)

Ac0on name and parameter list
Precondi0on (conjunc0on of func0on-‐free literals)
Effect (conjunc0on of func0on-‐free literals)

Add-‐list vs. delete-‐list in EFFECT

Cogni0ve Systems
SS 2014 27

Classical symbolic planning language: STRIPS (III)
The result of execu0ng ac0on a in state s is the state s’
•  s’ is same as s except
•  Any posi0ve literal P in the effect of a is added to s’
•  Any nega0ve literal ¬P is removed from s’
STRIPS assump0on: (avoids representa0onal frame problem)
every literal NOT in the effect remains unchanged

Cogni0ve Systems
SS 2014 28

Classical symbolic planning language: STRIPS (IV)
Init(On(A, Table) ∧ On(B,Table) ∧ On(C,A) ∧ Block(A) ∧ Block(B) ∧ Block(C) ∧ Clear(B) ∧

Clear(C))

Goal(On(A,B) ∧ On(B,C))

Ac6on(Move(b,x,y)
PRECOND: On(b,x) ∧ Clear(b) ∧ Clear(y) ∧ Block(b) ∧ (b≠ x) ∧ (b≠ y) ∧ (x≠ y)
EFFECT: On(b,y) ∧ Clear(x) ∧ ¬ On(b,x) ∧ ¬ Clear(y))

Ac6on(MoveToTable(b,x)
PRECOND: On(b,x) ∧ Clear(b) ∧ Block(b) ∧ (b≠ x)
EFFECT: On(b,Table) ∧ Clear(x) ∧ ¬ On(b,x))

Image source: Russel/Norvig
Cogni0ve Systems
SS 2014 29

Requirements for logical planning approaches
Observable
•  The agent can determine the current
state
Are these proper0es always fulfilled?
Discrete
•  Only a finite set of possible ac0ons in
each state
•  n ways at each crossing
Environment fully known

•  Knowing the result of each ac0on
•  Next town known for each way

Determinis0c
http://wiki.einsatzleiter.net/images/Baum_auf_Stra%C3%9Fe.jpg
•  Exactly one result for each ac0on

Cogni0ve Systems
SS 2014 30

Planning in the Real World

Un0l now: planning considered as search for goal states

In real-‐world applica0ons, addi0onal constraints apply
Time
• Execu0on 0mes are relevant, especially in concurrent plans
Resources
• Availability of reusable resources, e.g. machines, robots, …
• Availability of consumable resources, e.g. fuel, screws, …
In addi0on to finding a valid plan, scheduling is important

• Execu0on order in concurrent plans determines overall execu0on
0me

SS 2014 Cogni0ve Systems

Hierarchical planning
Several levels of abstrac0on for the plans

Detailing out the plan during the planning steps, can be deferred to plan execu0on phase
for the sake of flexibility
•  Off-‐line vs. on-‐line planning
•  E.g. in driving, route with ci0es planned off-‐line, exact steering parameters on-‐line based
on sensor informa0on

pull push
Plan refinement
•  Simplest form: Descrip0on as tuple
(original plan, refined plan)
Cogni0ve Systems
SS 2014 32

Hierarchical planning example
Plan refinement example with resource limita0ons

Formal descrip0on of the refinement by replace operator
replace(original , substitute, precondition, aditional _ property )

In the example (off-‐line variant)

⎛ (move), ⎞
⎜ ⎟
⎜ (grasp(manipulator ), pull(manipulator )), ⎟
replace⎜ ⎟
⎜
(available(manipulator )),
⎟
⎜ (addcost(rentalfee)) ⎟
⎝ ⎠
Cogni0ve Systems
SS 2014 33

Planning for the real world
Task: “Bring the pa0ent a drink”

Needs to deal with
•  Alterna0ves for sub-‐plans
•  Know where the drink is
•  Search for drink
•  Concurrent execu0on of sub-‐plans
•  Driving and searching
•  Track pa0ent’s hand and move bokle
•  Real-‐0me requirements
•  Failure in ac0on execu0on

Cogni0ve Systems
SS 2014 34

Main aspects for real-‐world planning
•  Single vs. mul0 agent plans

•  Centralized vs. decentralized planning
•  Plan coordina0on
•  Fault-‐tolerance aspects
•  Concurrent alterna0ves, e.g. for op0miza0on
•  Con0ngency planning, plan with e.g. sensor data dependent alterna0ves
•  Plan repair techniques

In addi0on to these high-‐level planning, mo0on planning techniques are always needed in
real systems, see module IN2138 Robot Mo0on Planning

Cogni0ve Systems
SS 2014 35

References:
•  Ghallab et al., Automated Planning – theory and prac0ce, Morgan Kaufmann
Publishers, 2004
•  Steven M. LaValle, Planning Algorithms, Cambridge University Press, 2006, online
available hkp://planning.cs.uiuc.edu/
•  J. Hoffmann and B. Nebel (2001) "The FF Planning System: Fast Plan Genera0on
Through Heuris0c Search", Journal of Ar0ficial Intelligence Research, Volume 14,
pages 253-‐302
•  M. Helmert (2006) "The Fast Downward Planning System", Journal of Ar0ficial
Intelligence Research, Volume 26, pages 191-‐246
•  Russel/Norvig, Ar0ficial Intelligence: A Modern Approach, 3rd ed. Chapters 10+11
Cogni0ve Systems
SS 2014 36


Cogni0ve Systems 37
SS 2014

Cognitive Systems:
State Estimation
Cogni0ve Systems
SS 2014 38

Applica&ons
All autonomous systems need to locate themselves in space
Cogni0ve Systems
SS 2014 39

Mo&va&on
Robot localiza0on
•  A robot is in an unknown environment

•  Each loca0on in the environment has a
certain probability of the robot being
there
•  The probability of the robot being at a
certain is equal before the robot has
any knowledge
•  The robot uses its sensors to find out
where it is; however, the sensor data
might be noisy or otherwise inaccurate
Cogni0ve Systems
SS 2014 40

Basics
Vocabuary for talking about state es0ma0on
•  Percep&on is sensor data

•  Ac&ons are in this case movements
•  States are variables
•  Beliefs are probabili6es
•  Priors are beliefs before a sensor input
•  Posteriors are beliefs aUer the sensor input
Cogni0ve Systems
SS 2014 41

Cogni&on: Percep&on-‐Ac&on Loop
PERCEPTION COGNITION ACTION
Cogni0ve Systems
SS 2014 42

Cogni&on: State es&ma&on
SENSE FILTER MOVE
•  Each 0me a robot senses it gains informa0on

•  Each 0me the robot moves it looses informa0on
•  Filters try to infer informa0on about the robot state through different probabilis0c
methods
Cogni0ve Systems
SS 2014 43

Three doors problem

An ar0ficial cogni0ve system tries to assess in front of which door it is.
1D-location
Cogni0ve Systems
SS 2014 44

Probability of being in front of a door

Before the first measurement the probability for being in
Probability front of any door or the wall is equal.
p(l)
1D-location l
The state of maximum confusion is when all locations have equal probability.
Cogni0ve Systems
SS 2014 45


The robot realizes that it is in front of any door aUer the first measurement.
p(l)
After one measurement the probabilities change. This is the posterior belief.
Cogni0ve Systems
SS 2014 46


The robot realizes that it is in front of any door aUer the first measurement.
p(l) First door. Second door.
Third door.
p = .33 p = .33 p = .33 In case of discrete intervals
After one measurement the probabilities change. This is the posterior belief.
One says the belief function is transformed through a measurement.
Cogni0ve Systems
SS 2014 47


Now the robot has moved. It realizes that it is in front of
p(l) another door aUer the second measurement.
?
Applet and Java code:
http://www.cs.ubc.ca/~poole/
demos/localization/
localization.html
A second measurement will update the probabilities again. If the robot had started at the
second door, if will now sense the wall instead of a door. This lowers the probability.
Cogni0ve Systems
SS 2014 48

Methods for Localiza&on / Tracking
Methods
HISTOGRAM KALMAN PARTICLE

FILTER FILTER FILTER
Filter State Space Belief Efficiency

Histogram Discrete Multimodal Exponential
Kalman Continuous Unimodal Quadratic
Particle Continuous Multimodal ?
Cogni0ve Systems
SS 2014 49

Key Publica&ons
Main Source:
Thrun, S., Burgard, W., & Fox, D. Probabilis6c robo6cs. Cambridge: MIT press, 2005.

Mathema&cal Methods:
Bishop, C. M. PaXern Recogni6on and Machine Learning. New York: Springer, 2006.
Doucet, A., de Freitas, N., Gordon, N. Eds., Sequen0al Monte Carlo Methods in Prac0ce, Sta0s0cs for
Engineering and Informa0on Science, Springer, New York, NY, USA, 2001.
Maybeck, P. S. Stochas0c models, es0ma0on, and control, Mathema0cs in Science and Engineering 141, 1979
Kalman, R.E. A new approach to linear filtering and predic0on problems. Transac6ons of the American Society
for Mechanical Engineering, Series D, Journal of Basic Engineering, 82, 35 – 45, 1960

Applica&ons:
Thrun, S., Montemerlo, M., Dahlkamp, H., Stavens, D., Aron, A., Diebel, J., … Mahoney, P. (2006). Stanley: The
Robot that Won the DARPA Grand Challenge. Journal of Field Robo0cs, 23(April), 661–692. doi:10.1002/rob

Cogni0ve Systems
SS 2014 50

Cogni&ve Systems:
Percep&on
Cogni0ve Systems 51
SS 2014


Cogni0ve Systems 52
SS 2014

Cogni0ve Systems
SS 2014

Cogni0ve Systems
SS 2014

Cogni0ve Systems
SS 2014

Cogni0ve Systems
SS 2014

Natural Scene Sta&s&cs
Cogni0ve Systems
SS 2014

2nd-‐order Sta&s&cs (acf, psd)
Conner et al.,

Proc. IEEE, 1972.
Cogni0ve Systems
SS 2014

Cogni0ve Systems
SS 2014

Cogni0ve Systems
SS 2014

The Recep&ve Field
Cogni0ve Systems
SS 2014

Cogni0ve Systems
SS 2014

Technical Bandpass Coders
Cogni0ve Systems
SS 2014

Gaussian Distribu0on Subtrac0ng 2 Gaussians
Filter Output
Cogni0ve Systems
SS 2014

Biological Spa&al Frequency Channels

Tuning characteris0cs
of cor0cal simple cells
Hubel, Wiesel, 1958

De Valois et al., 1982
How can this specific par00oning in be explained within an informa0on
theore0c approach?
Cogni0ve Systems
SS 2014

Fourier Space
t = 0:.1:10;!
y = sin(t);!
plot(t,y);!
y = sin(t) + sin(3*t)/3;!
plot(t,y);!
y = sin(t) + sin(3*t)/3 + sin(5*t)/5 + sin(7*t)/7 + sin(9*t)/9;

!
plot(t,y);!
Cogni0ve Systems
SS 2014

Cogni0ve Systems
SS 2014

PP
PP
Where ?
MT IT
MT V3
?
Dorsal
V2
Ventral
PP
What
V3 V4 IT V4 V1
MT V3
V2 V2
IT V4 V1
V1
Lateral Geniculate Nucleus

(Thalamus)
Superior Colliculus
Retina
Cogni0ve Systems
SS 2014

The Riesenhuber et al. Model
Cogni0ve Systems
SS 2014

Speech production Speech understanding

Broca’s Wernicke’s
Area Area
A1
Broca Wernicke A2
A2
A1
Medial Geniculate Nucleus

(Thalamus)
Inferior Colliculus
Nucleus of the
Lateral Lemniscus
Superior Olivary
Nucleus
Cogni0ve Systems
SS 2014

Informa&on Fusion in Technical Systems
Abidi, Gonzalez, 1992
Cogni0ve Systems
SS 2014

Cogni0ve Systems
SS 2014
Lehrstuhl Informa0k VI – Robo0cs and Embedded Systems Calvert, Lewis, 2004

Auditory-‐Visual Covariances
Cogni0ve Systems
SS 2014

Method: determining 2D visual

component
incremental thresholds
auditory
component
Cogni0ve Systems
SS 2014

Ideal psychometric func6on:

Step func6on (fixed threshold)
100
Real psychometric
Propor0on s0muli detected (%) func6on:
S-‐shaped (sigmoid or
logis6c) func6on
50 50%
threshold
0
S0mulus intensity
Cogni0ve Systems
SS 2014

Decision Fusion Model
Predicted 2D-‐Thresholds:
Cogni0ve Systems
SS 2014

Data Fusion Model
Predicted 2D-‐Thresholds:
Cogni0ve Systems
SS 2014

Measured 2D-‐Thresholds
”Selec0ve Fusion Model”
Cogni0ve Systems
SS 2014

Aristoteles: De Anima.
book II, chapter 7, 418a."
Cogni0ve Systems
SS 2014

Key Publica&ons
Hyvärinen, A., Hurri, J., & Hoyer, P.O. Natural Image Sta6s6cs -‐ A probabilis6c Approach
to Early Computa6onal Vision. Springer, 2009

Chen, S., Li, Y., & Kwok, N.M. Ac6ve vision in robo6c systems: A survey of recent
developments. MIT press, 2005.

Tsotsos, J.K.: A Computa6onal Perspec6ve on Visual AXen6on. MIT press, 2011

Corke, P. Robo6cs, Vision and Control: Fundamental Algorithms in MATLAB. Springer,
2013

Cogni0ve Systems
SS 2014 80


Cogni0ve Systems 81
SS 2014

Thanks for your aZen&on!

... and send your questions, comments etc. to
florian.roehrbein@in.tum.de
Cogni0ve Systems
SS 2014 82

2014 VL CogSys 5

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

2014 VL CogSys 5

Uploaded by

Copyright:

Available Formats

Fakultät

Robotics and Embedded Systems (Informatik VI, TUM)

Contents of the Lecture

18.8. 19.8. 20.8. 21.8. 22.8.

Basic Architecture of Robot System

Communica0on and Adapta0on between A. Knoll et al.,

The natural language processing pipeline

§ Output channels

§ Robot architecture

§ Safety for the human

§ ... and a lot more

Evalua&on of Dialogue Systems

§ Dialogue eﬃciency

JAMES Project: Joint Ac&on for Mul&modal Embodied Social Systems

Social robots that work

User modelling and social signal processing

Automaton of bar interac&on script

User modelling and social signal processing

N-­‐forma0on H-­‐forma0on V-­‐forma0on L-­‐forma0on

Training of Hidden Markov Model (HMM)

§ Body posture and f-­‐forma0ons from Kinect data

HMM states and transi&ons

Robots in industrial produc&on today

Humans in industrial produc&on

§ Human dexterity needed for:

Challenge: combining the skills

§ Industrial robot

Basic Architecture of Robot System

Communica0on and Adapta0on between A. Knoll et al.,

What is planning?

Descrip&on of a planning task

Diﬀerent levels of abstrac0on, example: drive from A to B

Posi0on on map Drive, change road geometrical road map

Planning for cogni&ve systems

• (Low level) mo0on planning

In real life, both have to be combined.

Classical symbolic planning language: STRIPS

Representa0on of states

Representa0on of goals

Classical symbolic planning language: STRIPS (II)

Representa0ons of ac0ons

Classical symbolic planning language: STRIPS (III)

Classical symbolic planning language: STRIPS (IV)

Init(On(A, Table) ∧ On(B,Table) ∧ On(C,A) ∧ Block(A) ∧ Block(B) ∧ Block(C) ∧ Clear(B) ∧

Image source: Russel/Norvig

Requirements for logical planning approaches

Environment fully known

• Exactly one result for each ac0on

Planning in the Real World

In addi0on to ﬁnding a valid plan, scheduling is important

Lehrstuhl Informa0k VI – Robo0cs and Embedded Systems

Several levels of abstrac0on for the plans

Hierarchical planning example

Plan reﬁnement example with resource limita0ons

Planning for the real world

Task: “Bring the pa0ent a drink”

Main aspects for real-­‐world planning

• Single vs. mul0 agent plans

Basic Architecture of Robot System

Communica0on and Adapta0on between A. Knoll et al.,

• A robot is in an unknown environment

• Percep&on is sensor data

Cogni&on: Percep&on-­‐Ac&on Loop

§  Output channels

§  Robot architecture

§  Safety for the human

§  ... and a lot more

§  Dialogue eﬃciency

N-‐forma0on H-‐forma0on V-‐forma0on L-‐forma0on

§  Body posture and f-‐forma0ons from Kinect data

§  Human dexterity needed for:

§  Industrial robot

•  (Low level) mo0on planning

•  Exactly one result for each ac0on

Main aspects for real-‐world planning

•  Single vs. mul0 agent plans

•  A robot is in an unknown environment

•  Percep&on is sensor data

Cogni&on: Percep&on-‐Ac&on Loop

•  Each 0me a robot senses it gains informa0on

2nd-‐order Sta&s&cs (acf, psd)

y = sin(t) + sin(3t)/3 + sin(5t)/5 + sin(7t)/7 + sin(9t)/9;