20131102-Towards ECV-輔大

Wang, Yuan-Kai()
Electrical Engineering,
Fu Jen University ()
Email: ykwang@mail.fju.edu.tw,
URL: http://www.ykwang.tw
Slideshare: Search "Towards Embedded Computer Vision"
Towards
Embedded
Computer Vision
Towards Embedded Computer Vision p.
Fu Jen University Department of Electrical Engineering Wang, Yuan-Kai
2
MIT Oxygen
http://oxygen.lcs.mit.edu/ 1999
3
The Aware Home
http://awarehome.imtc.gatech.edu/
Ambient Intelligence v.s.
Object Intelligence
Embedded system + Sensors
4
Wearable Computing
Smart Home, Smart
5 5
Embedded Systems
"Without" Sensors
MIC ITIS
6 6
Embedded Systems
"With" Sensors
GPS
Navigation
Wii
Roomba
Exoskeleton
7
Driver assist
Front view
Night vision
Blind corner
Blind spot
Rear view
Embedded Systems
"With" Image Sensors
8 8
Embedded Systems
Google Glass
Smart Watch
9
Microsoft
SenseCam
Embedded Systems
10
Observation
Miniaturization

High Resolution

3D Capability
11
12
Embedded Computer Vision
! Embedded System + Camera
+ Computer Vision Algorithm
Image
Processing
Image
Recognition
Image
Capturing
13
Embedded Computer Vision
! Embedded compute vision
" Implements computer vision algorithms
on low-cost, low-power,
constrained hardware
! Constrained hardware
" Low-speed CPU
" Low capacity memory
" No floating-point (FPU)
" Low-resolution image sensor
A B
Research Status (1/3)
14
H
K
K
15
16 16
DARPA
Grand
Challenge
Google
Car
Google
Taxi
17
International Activities (1/2)
! Special conferences
" IEEE Int. Workshop on Embedded Computer
Vision
" IEEE Int. Conf. Distributed Smart Cameras
! Special journal issues
" IEEE Journal of Selected Topics in Signal
Processing, vol. 2, no. 4, Aug. 2008
" EURASIP Journal on Embedded Systems
! Short courses in important CV conferences
" CVPR07&08: Distributed vision processing in
smart camera networks
" ESC07: Embedded CV and smart cameras
" ICASSP09: Distributed processing in smart
cameras
18
International Activities (2/2)
! Research projects and Lab.
" Princeton Univ./Georgia Tech.:
Embedded Systems Lab., Wayne Wolf
" Stanford Univ.
Wireless sensor networks Lab.
" UCLA, CMU, MIT
" Delft Univ. of Technology
SmartCam Project
" Graz Univ. of Technology
19
Three Ways for ECV
! CPU (Central Processing Unit)
" ARM, PowerPC
! DSP (Digital Signal Processor)
" TI, ADI, NXP
! FPGA (Field Programmable Gate Array)
" Altera, Xilinx
20
CPU
! Stanford MeshEye
" ARM 7 (55MHz), ZigBee node
" 3 image sensors
" 30x30 grayscale x 2
" 640x480 color x 1
Object detection
(http://wsnl.stanford.edu/smartcam.html)
"MeshEye:a hybrid-resolution
smart camera mote for applications
in distributed intelligent
surveillance", IPSN-SPOTS, 2007
21
DSP
! "Visual surveillance on DSP-based
embedded platform," Graz Univ. of
Technology, 2008(Phd. dissertation)
" TI C6414 (600MHz)
" Applications: Adaboost face detection, vehicle
detection, license plate detection
22
FPGA (2/3)
"Hardware, Design and
Implementation Issues on a
FPGA-Based Smart Camera,"
ICDSC, 2007
23
Hybrid: CPU+DSP
! "Distributed Embedded Smart Cameras
for Surveillance Applications,"
IEEE Computer, 2006.
Developed for traffic surveillance
Hybrid: CPU+DSP by SoC
! TI OMAP
24
25
Hybrid: CPU+FPGA
! UCLA Cyclops
" MCU: Atmega128
" CPLD: image processing
" ECV app.: Hand gesture recognition
http://research.cens.ucla.edu/
projects/2007/
Multiscaled_Actuated_Sensing/
Cyclops/
Hybrid: CPU+FPGA by SoC
! Xilinx Zynq
26
27
Challenges for ECV
! Algorithm refinement
" Parallel computation
" Function partition, Multi-threading
" Stream processing
" Memory hierarchy optimization
! Hardware design
" Pipeline, SIMD, board design
! Optimized programming skills
" Fixed-point arithmetic
" Memory management
" Intrinsic commands
28
Our Experiences
Entertainment Robot (CPU)
Vision Sensor (CPU)
Video Surveillance (FPGA)
Wearable Vision (DSP)
29
Case 1
Entertainment
Robot
CPU
30
Sony AIBO
Rolling Dice
31
32
Face Detection
33
Face Recognition
34
Facial Expression Recognition
35
Case 2
Vision Sensor
CPU
36
FJUCam
Power
consumption:
1W
Power sources:
5V DC current
5V Cell Battery
Weight: 35gm
Size:
6 x 4.5 x 5 (cm)
(W x H x D)
Three Modules
1. Main board, 2. Lens module
3. Storage module
37
Face Detection
FJUCam
Face Detection
Image
Display
Vision Sensor Network
38
Cyclops Robot
! Color tracking
39
Case 3
Video Surveillance
FPGA
40
Mobile Video Surveillance
FPGA
(3G/GPRS)
(3G) GSM
Wap Push
PC
Background subtraction, ...
2.8 GHz Intel CPU
Software: C/C++
Frame rate: 10 fps
41
Background Subtraction
Current
Frame
-
Background
Image Update
Post Processing
Motion Object Image
Background Image
1 + k B
k B
1 + k P
1 + k M
) , ( ) , (
) , (
1
1
y x B y x P
y x M
k k
k
! =
+
+
) , ( ) 1 ( ) , (
) , (
1
1
y x P y x B
y x B
k k
k
+
+
! + = " "
Speed up by (1) Circuit design, (2) Paralization
42
FPGA Speed Up
! Comparisons
" PC: 2.8GHz CPU, C implementation
" FPGA can speed up 500 times
6lock(Hz) FS
60
F0A
2.8G
25M
51
10
43
Case 4
Wearable Vision
X-EYE
ARM + DSP
44
Goal
44
Demo Video
45
46
Development

SD USB-WIFI

USB-RS232

USB

BeagleBoard
Camera
USB
Hub
47
Hardware
47 47
Peripheral I/O

USB Host
JTAG
DVI-D video out
S-Video out
SD/MMC+
Stereo in/out
RS-232 serial1
Alternate power
USB 2.0 HS OTG
USER
OMAP3530 Processor
600MHz Cortex-A8
NEON+VFPv3
16KB/16KB L1
256KB L2
430MHz C64x+ DSP
32K/32K L1
48K L1D
32K L2
Power VR SGX GPU
64K on-chip RAM
POP Memory
256MB LPDDR RAM
256MB NAND flash
RESET
7.6 cm
48
System
2010.04.25
48
49
Software
FJUCam2
50
Next : UAV
51
Next : Sleep Monitoring
52
Next : Vehicular
Object Detection
53
Heterogeneous Multicore
! Xilinx Zynq: ARM+FPGA
54
55
Future of
Embedded Vision
56
Summary
! Embedded CPU is not appropriate for
ECV
" Pros
" Great flexibility
" Fast development
" Cons
" No float-point arithmatics
" Interrupt-based architecture
! Smart phone with 4/8 cores is OK?
57
I Think !
! Embedded compute vision
" Low-cost, low-power,
minimal hardware
! High-Performance
computer vision
" Fast speed without cost,
power, and hardware
constraints
From contradiction to convergence ?!
Constrained
Resource
Abundant
Resource
Yes by multicore
Contradiction
58
Challenges (1/2)
! Algorithm decomposition
" Function decomposition
" Partition serial part and parallel part
" Data flow analysis and
data dependency analysis
" Parallelism
" Loop unrolling
" Multithreading
" Pipeline
59
Challenges (2/2)
! Performance analysis method
" For efficiency improvement
! Implementation efforts
" Choose a good embedded platform for
computer vision
" Software issues
" Hardware issues
" Programming skills
" Multi-threading
60
Future Research Directions
Heterogeneous
Multicore
ARM
+ DSP
ARM +
FPGA
ARM +
GPGPU
I Think Again !
! Is there a development platform for
embedded computer vision with
" Built-in camera
" AE, AF, AWB
" High-performance processor
" Heterogeneous multicore
" Software development environment:
" C, Matlab
" OpenCV
" Sample codes
" Labs
61
Embedded Vision Alliance
62
! Ramesh Raskar
MIT Platform
63
! Andrew Adams
Stanford Platform
64
!
Creative Commons - 3.0

!
(1) (2)
!

66

20131102-Towards ECV-輔大

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

20131102-Towards ECV-輔大

Uploaded by

Copyright:

Available Formats

Wang, Yuan-Kai()

You might also like