E-Board Presentation

E-board Control
Analysis and development of a Bluetooth remote control application, that interacts with an
Android smartphone accelerometer
Project delivery meeting - Milano, 28 Luglio 2010
Massimo Luraschi
matr. 739979
Department of
Electronic and
Information
http://move.dei.polimi.it/
Massimo Luraschi matr. 739979
Index
2
1. Software design principles
2. GUI
3. Sensors
4. Gestures
5. Bluetooth
Software design principles
3
4
Development on Android
platform (2.1 version)
Choice: SDK (Software
Development Kit) vs NDK (Native
Development Kit)
Programming language: Java
GUI
Bluetooth
Sensors
Background services
Control and user interaction
Model View Controller design
pattern:
GUI
Graphical User Interface
5
GUI
Welcome screen
Driving options menu
Exit button (shuts off eventual
background services and
bluetooth connection)
Safe behavior: difference
between closing and minimizing
E.g.: incoming call
6
BT Connection state notification area
GUI
Welcome screen
E.g.: incoming call
7
Debug mode ON/OFF
GUI
Welcome screen
E.g.: incoming call
8
Selects accelerometer driving mode
Selects E-board pressure sensors driving mode
Closes the application and Bluetooth connection
Debug mode ON/OFF
Selects Touch Screen (Mixed) driving mode
GUI
Connection screen
Presents a list of already paired
devices
Scan for devices button
Touch device name starts the
connection
9
GUI
Connection screen
Presents a list of already paired
devices
Scan for devices button
Touch device name starts the
connection
10
GUI
Drive with Accelerometer screen
11
Visual feedback of commands
sent to E-board
Start/Stop buttons
Back button
Puts an end to the driving and gets back to
home screen
First touch of the button starts sensors
acquisition service and begins driving
experience
Second tuch of the button, shuts down sensor
acquisition service and stops driving
The sphere moves along the screens upper
portion to notify the user of the directions sent
to E-board by smartphones movements.
E.g.: til smartphone foreward, sphere moves
upward
GUI
Drive with Accelerometer screen (2)
12
If debug mode is active,
accelerometer screen reports real
time data from sensors
Accelerometer data are logged
on a file to analyse sensors
acquisition characteristics
(frequency, accuracy, )
GUI
E-board pressure sensors driving screen
13
In this mode, driving is left to E-
board
Bluetooth connection has to
remain active
For safety reasons mode
command is periodically sent to
E-board
E-board sends back a message
containing weight distribution on
feet pressure sensors
home screen
Feet are coloured differently, depending on the
pressure detected by the sensors
GUI
14
board
remain active
E-board
home screen
GUI
15
board
remain active
E-board
home screen
GUI
16
board
remain active
E-board
home screen
GUI
17
board
remain active
E-board
home screen
GUI
18
board
remain active
E-board
home screen
GUI
19
board
remain active
E-board
home screen
GUI
Schermata di guida con Touch Screen
20
Hybrid driving mode
Smartphone indicates a speed
setpoint using the SeekBar
Steering is left to the E-board by
using the pressure sensors
home screen
One touch in the Widget area allows to
indicate desired speed
After first touch is possible to drag the bar
Sensors
Sensors
21
Sensors
22
Accelerometer
Sway (along phone X axis)
Surge (along phone Y axis)
Heave (along phone Z axis)
Magnetometer + Inclinometer
Yaw (rotation on Z axis)
Pitch (rotation on X axis)
Roll (rotation on Y axis)
Nexus One axis reference
Smartphone uses blue axis
system in the picture
From now on, usual aero-
naval axis reference will be
used (red axis in the picture)
X
Y
Z
Z
Y
X
Sensors
Characteristics
23
Range: [-2g; +2g]
Sampling frequency
Noise
0 1 2 3 4 5 6 7
-20
-15
-10
-5
0
5
10
15
20
aZ
time [s]
a
c
c
e
l
e
r
a
t
i
o
n

[
m
/
s
2
]
0 1 2 3 4 5 6 7
-20
-15
-10
-5
0
5
10
15
20
aY
time [s]
a
c
c
e
l
e
r
a
t
i
o
n

[
m
/
s
2
]
0 1 2 3 4 5 6 7
-20
-15
-10
-5
0
5
10
15
20
aX
time [s]
a
c
c
e
l
e
r
a
t
i
o
n

[
m
/
s
2
]
Sensors
Sampling frequency
Not constant
Unpredictable
In Android there is no way to predefine sampling frequency
Sensors are not sampled upon request, it is sensors themselves
who notify changes in signal values to sensors subscribers
24
Sensors
Sampling timestamps histogram
25
0 10 20 30 40 50 60 70 80 90 100
0
5
10
15
20
25
30
35
40
45
50
Histogram of
t between timestamps
t [ms]
%

o
f

o
c
c
u
r
e
n
c
i
e
s
nacquisizioni: 1418
Sensors
Noise and signals filtering
26
Noisy signals
Excessive signal variations
Non-constant frequency
Chosen filter: Moving Average
window width: 5
0 2 4 6 8 10 12
-12
-10
-8
-6
-4
-2
0
2
4
6
aX
time [s]
a
c
c
e
l
e
r
a
t
i
o
n

[
m
/
s
2
]
0 2 4 6 8 10 12
-15
-10
-5
0
5
10
15
aY
time [s]
a
c
c
e
l
e
r
a
t
i
o
n

[
m
/
s
2
]
0 2 4 6 8 10 12
-10
-5
0
5
10
15
aZ
time [s]
a
c
c
e
l
e
r
a
t
i
o
n

[
m
/
s
2
]
Sensors
Noise and signals filtering
27
5.5 6 6.5 7 7.5 8
6
7
8
9
10
11
12
aZ
time [s]
a
c
c
e
l
e
r
a
t
i
o
n

[
m
/
s
2
]
Gestures
Gestures
28
Gestures
Requirements
29
Univocal movements
Easy recognition
Low complexity
Speed Steering
Gestures
Signal Analysis
30
Need to analyse signals to associate them with gestures
Different analysis for Magnetometer and Inclinometer
Magnetometer is noisy and inaccurate
Inclinometer signals are based on biaxial accelerometer
variations
Gravity acceleration projection on a plane brings to estimation
errors in some orientations
Gestures
Signal Analysis: Yaw
31
Yaw signal comes from
Magnetometer
Inaccurate
Susceptible of magnetic field
sources
Unreliable
0 2 4 6 8 10 12 14 16 18
10
20
30
40
50
60
70
80
90
Yaw
time [s]
r
o
t
a
t
i
o
n

[
]
0 5 10 15 20 25 30 35
0
50
100
150
200
250
300
350
Yaw
time [s]
r
o
t
a
t
i
o
n

[
]
Slow movements along X, Y, Z axis, with
no rotations
Close (~25cm) to a magnetic field
source (audio speakers)
Gestures
Signal Analysis: Pitch and Roll
32
Inclinometer is a dummy
sensor: it calculates Pitch
and Roll from Accelerometer
signals
In some cases calculations
are wrong
Example: rotation around Z
axis with phone in vertical
position (Pitch expected
value: null)
Correlation between
detected accelerations and
rotations
Errors in correlating
accelerations and rotations?
0 5 10 15 20 25
-90
-85
-80
-75
-70
-65
-60
-55
-50
-45
-40
Pitch
time [s]
A
n
g
l
e

[
]
0 5 10 15 20 25
-50
-40
-30
-20
-10
0
10
20
30
40
50
Roll
time [s]
A
n
g
l
e

[
]
Gestures
Signal Analysis: Pitch and Roll (2)
33
0 5 10 15 20 25
-8
-6
-4
-2
0
2
4
6
8
aY
time [s]
a
c
c
e
l
e
r
a
t
i
o
n

[
m
/
s
2
]
0 5 10 15 20 25
-90
-85
-80
-75
-70
-65
-60
-55
-50
-45
-40
Pitch
time [s]
A
n
g
l
e

[
]
0 5 10 15 20 25
-50
-40
-30
-20
-10
0
10
20
30
40
50
Roll
time [s]
A
n
g
l
e

[
]
0 5 10 15 20 25
6
6.5
7
7.5
8
8.5
9
9.5
10
aX
time [s]
a
c
c
e
l
e
r
a
t
i
o
n

[
m
/
s
2
]
Gestures
34
0 5 10 15 20 25
-8
-6
-4
-2
0
2
4
6
8
aY
time [s]
a
c
c
e
l
e
r
a
t
i
o
n

[
m
/
s
2
]
0 5 10 15 20 25
-90
-85
-80
-75
-70
-65
-60
-55
-50
-45
-40
Pitch
time [s]
A
n
g
l
e

[
]
0 5 10 15 20 25
-50
-40
-30
-20
-10
0
10
20
30
40
50
Roll
time [s]
A
n
g
l
e

[
]
0 5 10 15 20 25
6
6.5
7
7.5
8
8.5
9
9.5
10
aX
time [s]
a
c
c
e
l
e
r
a
t
i
o
n

[
m
/
s
2
]
Gesture
35
Definition of accelerations model
=
=
=
g c c a
g s a
g s c a
z
y
x

Result
( ) ( )
|
|
\
|

=
x z
y
a a
a

sin cos
arctan
( ) ( ) ( ) ( )
( ) [ ]

, min arg : ,
,
0 0
2 2 2
g
z y x g
J
g c c a g s a g s c a J + + + =
Angles calculation by minimizing a cost function
|
|
\
|
=
z
x
a
a
arctan
Proposal: Re-calculate rotations as angles of conversion from
smartphone reference system and the inertial reference system
Gestures
36
Confrontation of analytic
results and smartphone
signals
Same situation as previous
charts
Problem seems solved
|
|
\
|

=
z
x
a
a
arctan
( ) ( )
|
|
\
|

=
x z
y
a a
a

sin cos
arctan
0 5 10 15 20 25
-50
-40
-30
-20
-10
0
10
20
30
40
50
Roll
time [s]
A
n
g
l
e

[
]
0 5 10 15 20 25
-90
-85
-80
-75
-70
-65
-60
-55
-50
-45
-40
Pitch
time [s]
A
n
g
l
e

[
]
Gestures
37
Test performed by acquiring signals while performing all
gestures at users disposal, one by one
Analysis of the signals in each position
Gesture-signal correlation
Pitch signal with smartphone in vertical position is correctly
detected
New problem with smartphon in half tilted
0 10 20 30 40 50 60 70
-100
-50
0
Pitch
time [s]
A
n
g
l
e

[
]
0 10 20 30 40 50 60 70
-100
-50
0
50
Roll
time [s]
A
n
g
l
e

[
]
Gestures
38
/
|
/
_
Smartphone movements:
Phone pitch
Gestures
39
New issue is complementar to smartphone inclinometer problem
discussed earlier
Cause: contemporary variation of a
z
and a
x
leads to unexpected
solution ( )
Proposed solution: usage of both signals (analitical and
inclinometer)
In normal conditions both signals are identical and correct
When they diverge, the correct one is the invariant one
Best signal selection: lowest differential signal
Possible quick & dirty solution: best signal selection based
on detected Pitch (if > -80 : =
smartphone
)
|
|
\
|
=
z
x
a
a
arctan
Gestures
40
0 5 10 15 20 25 30
-90
-80
-70
-60
-50
-40
-30
-20
-10
0
Pitch
time [s]
A
n
g
l
e

[
]
Single differential based Pitch selection
Gestures
41
Average of five differentials based Pitch selection
0 5 10 15 20 25 30
-90
-80
-70
-60
-50
-40
-30
-20
-10
0
Pitch
time [s]
A
n
g
l
e

[
]
Gestures
42
Quick and dirty Pitch selection
0 5 10 15 20 25 30
-90
-80
-70
-60
-50
-40
-30
-20
-10
0
Pitch
time [s]
A
n
g
l
e

[
]
Gestures
43
Spikes presence, removed with non-linear low pass filter based
on s(t) s(t-1) difference
Moving Average on differential helps
Quick and Dirty solution has good, but could lead to strong
discontinuities in unpredicted case
Proposal: instead of filtering the input (differential) with Moving
Average, filter the output signal (Pitch)
Gestures
44
N.L.F. and M.A.(5) on output solution
0 5 10 15 20 25 30
-100
-90
-80
-70
-60
-50
-40
-30
-20
-10
0
Pitch
time [s]
A
n
g
l
e

[
]
Gestures
Recap
45
Data processing recap graph
a
x
a
y
a
z
M.A. (5)
0
C
o
m
p
a
r
e
N.L.F. M.A.(5)

f
E
s
t
i
m
a
t
e
f
Gestures
Conclusions
Yaw is not usable for driving
Inclinometer Pitch e Roll signals have problem in some
conditions
Estimate rotations using accelerations
Estimated signals correctly detect the gesture causing previous
problem
Estimated signals suffer from a different and complementary
problem
Use
Inclinometer
and
estimated
depending on the situation
With this processing, Pitch and Roll signals satisfy easy
recognition and low complexity requisites
Two signals (actually four) are enough to command speed and
steering of the E-board
46
Bluetooth
Bluetooth
47
Bluetooth
E-board communication protocol
48
Protocol
Serial Port on RFCOMM channel (slave mode)
Messages to E-board
#MODE,Command1,Command2\n
Driving mode
p: drive with
plantar pressure
mode
a: drive with
accelerometer
mode
t: drive with
touch mode
Command 1
Speed
[0,100] integer
values
0: still
100: max speed
Command 2
Steering
[0,100] integer
values
0: steer left
100: steer right
Bluetooth
E-board communication protocol
49
Protocol
Serial Port on RFCOMM channel (slave mode)
Messages from E-board
#MODE,Command1,Command2\n
Identifier
f: weight
distribution on
plantar pressure
sensors
Command 1
Left foot
[0,100] integer
values
0: unloaded sensor
100: full load (E-
board scales basing
on medium weight)
Command 2
Right foot
[0,100] integer
values
0: unloaded sensor
100: full load (E-
board scales basing
on medium weight)

E-Board Presentation

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

E-Board Presentation

Uploaded by

Copyright:

Available Formats

E-board Control

You might also like