Scribd Logo
Upload

Welcome to Scribd!

  • Bartolo ZZ I

    Uploaded by

    Filippo Costanzo
    14 views23 pages
    The document discusses event-driven sensing approaches for a humanoid robot. It focuses on event-driven touch and vision sensors. For touch sensing, it describes the iCub robot's capacitive tactile sensors covering its fingertips and skin with around 4500 taxel sensors. The touch sensors use an event-driven approach where taxels trigger events based on exceeding noise and level thresholds. For vision, it describes a spatial-temporal dynamic vision sensor that provides a stream of per-pixel brightness change events, as well as its use for optical flow computation. The event-driven approaches aim to mimic biological sensing and processing in a more efficient way compared to traditional sampling-based methods.

    Original Description:

    Original Title

    Bartolo Zz i

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Report this Document
    The document discusses event-driven sensing approaches for a humanoid robot. It focuses on event-driven touch and vision sensors. For touch sensing, it describes the iCub robot's capacitive tactile sensors covering its fingertips and skin with around 4500 taxel sensors. The touch sensors use an event-driven approach where taxels trigger events based on exceeding noise and level thresholds. For vision, it describes a spatial-temporal dynamic vision sensor that provides a stream of per-pixel brightness change events, as well as its use for optical flow computation. The event-driven approaches aim to mimic biological sensing and processing in a more efficient way compared to traditional sampling-based methods.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Flag for inappropriate content
    14 views23 pages

    Bartolo ZZ I

    Uploaded by

    Filippo Costanzo
    The document discusses event-driven sensing approaches for a humanoid robot. It focuses on event-driven touch and vision sensors. For touch sensing, it describes the iCub robot's capacitive tactile sensors covering its fingertips and skin with around 4500 taxel sensors. The touch sensors use an event-driven approach where taxels trigger events based on exceeding noise and level thresholds. For vision, it describes a spatial-temporal dynamic vision sensor that provides a stream of per-pixel brightness change events, as well as its use for optical flow computation. The event-driven approaches aim to mimic biological sensing and processing in a more efficient way compared to traditional sampling-based methods.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Flag for inappropriate content
    of 23

    Event-Driven Sensing for a

    Humanoid Robot
    Chiara Bartolozzi
    Our goal…
    Smart Autonomous Systems
    Event-Driven iCub
    Vision Computation
    Touch

    Communication
    Touch
    ED Sensors — Touch

    POSFET “Sustained” taxel

    “Transient” taxel
    ED Sensors — Touch

    Sustained

    Transient

    Caviglia et al. ISCAS 2016


    iCub Touch

    • Fingertips & Skin

    • Capacitive Sensors

    • ~4500 Sensors (taxels)


    ED Sensors — Touch Taxel
    Taxel Taxel Taxel
    Taxel Taxel Taxel
    Taxel

    Tactile sensing
    Taxel

    digital converters
    Off-the-shelf
    RH
    Taxel
    Taxel Taxel Taxel
    Taxel Taxel Taxel
    Taxel Taxel

    Taxel
    Taxel ADC
    ADC Taxel
    Taxel ADC Taxel
    Taxel

    Local custom I2C bus


    T
    T Clock-driven
    T H Clock-driven
    Clock-driven
    samples
    samples
    samples
    T
    T
    T H H T
    T
    T

    Artix7 FPGA
    soft-core
    Acquisition controller

    Tactile hub
    T
    T H T
    T
    T H T
    Events generator
    Daisy

    Asynchronous
    H H chained

    events
    T T
    Events arbiter
    tactile
    T T hubs
    T H T
    T T
    T T T
    T H T
    T
    T T

    infrastructure
    AS-AER body
    Asynchronus

    sensor
    events

    Robotic head
    T
    T
    T H H T
    T Events timestamping

    ZynQ7000
    T

    platform
    Events acquisition

    T
    T H T
    T
    T H T
    Event based Sample based
    RH Robotic head
    ED Sensors — Touch

    Noise Threshold - θ
    Level Threshold - Δ
    Fingertip - 12 taxels, #7 active
    ED Sensors — iCub Vision

    SENSORE
    Figura 25 Dettagli geometrici e tolleranze ottiche per il sistema
    Python CMOS
    spazio-variante per robotica umanoide
    ante attività ha interessato la progettazione di un obiettivo per l’occhio
    iCub. Attualmente quest’ultimo impiega una lente grandangolare
    coppiata ad un sensore matriciale del formato 1/3”, con una Back Focal
    Spatial 1,3Mpxl
    lto bassa. Dal momento che si vogliono disporre due sensori dietro la stessa
    aumentare la BFL per accomodare un elemento di separazione (Beam-
    fronte d’onda (Fig. 26).
    Res 1/2”
    Temporal 42 fps
    Res (24ms)
    AFOV 42°
    ED Sensors — Vision
    n t s
    V !
    E v e
    e n Ct o n
    o pio n ATIS t io CMOS
    N o
    V is m a m
    g e
    t !
    e r fo r t re a n
    I m a
    p u U T l I n s S Spatial
    i s io 240x304
    o o m B o ra e n t V
    Res 2/3”
    N
    t C m p E v i r e!d
    d ap T e o u s s p
    i m e Temporal 1μs
    A l o i t i nu l I
    d n T Res
    x p o n t i c a
    a n
    E C log AFOV 61°
    B i o
    ED Vision — Optical Flow
    ED Vision — Optical Flow

    Benosman et al. TNNLS 2014


    ED Vision — Corner Detection
    9 9

    8 8

    7 7

    Binary Image Patch


    6 6

    5 5

    4 4

    - spatial derivatives 3

    2
    3

    1 1
    1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9

    ED Harris Score — R 1.5

    - flat: two small eigenvalues 1


    R small
    0.5
    - edge: one small and one big

    Score
    R negative 0
    - corner: two large eigenvalues
    R big -0.5

    -1
    -10 -5 0 5 10
    Distance from Ground Truth (px)
    ED Vision — Corner Detection
    ED Vision — Corner Detection
    3 3

    2
    Error < 2pxl 2

    Error (px)
    C1
    120 C11 1 1

    C2
    100 C3
    C12

    C13
    80 0 0
    C8 10 25 40 55 70 85 100 0 15 30 45 60 75 90
    y (px)

    C4 Speed magnitude (deg / s) Corner orientation (deg)


    60
    #10 5
    C5

    Number of Harris processing (proc / s)


    6 350
    C9
    40 0

    50%{

    Number of detections (det / s)


    C10
    300 15
    5 30
    20 C6 45

    3x{
    250 60
    C7 4
    75
    0
    0 20 40 60 80 100 120 200 90
    x (px) 3
    150
    2
    100
    1
    Vasco et al. IROS 2016 50

    0 0
    0 10 25 40 55 70 85 100 0 10 25 40 55 70 85 100
    Speed (deg / s) Speed (deg / s)
    ED Vision — Disparity and Vergence
    LEFT RIGHT

    Disparity:
    Phase-Shift
    Model

    Vergence Control:
    Proportional Velocity
    ED Vision — Vergence Control

    FRAME-BASED Vs. EVENT-BASED


    without normalization No need to normalize
    office light
    Stimulus depth (mm)

    with normalization
    dark environment

    Time (s) Time (s)

    Vasco et al. Humanoids 2016


    Event-Driven Vision — Ball Tracking & Gaze
    Control

    Glover et al. IROS 2016


    Event-driven POSFET
    S. Caviglia Event-driven iCub and Vision
    M. Valle iCub Facility crew
    L. Pinna G. Metta
    A. Abou H. Akolkar
    V. Vasco
    A. Glover
    F. Solari
    P. Motto Ros
    R. Benosman

    Thanks!

    You might also like