Professional Documents
Culture Documents
59 - A Neural Network Implementation On An Inexpensive Eight Bit Microcontroller PDF
59 - A Neural Network Implementation On An Inexpensive Eight Bit Microcontroller PDF
III. PSEUDO FLOATING POINT error occurs without the pseudo floating point calculations.
During the neural network training process precise This network was trained to the surface in Figure 3. To help
weights are created to generate a series of outputs. Due to show the importance of these scaling factors Figure 4 and
the importance of the weights in the output of the neural Figure 6 were created. These Figures are an example of a
network, a pseudo floating point system is implemented. control surface with the pseudo floating point arithmetics
The weights of each neuron are scaled in order to use the disabled and the error created. Notice the distortion around
maximum number of bits available in each calculation. This the edges of the surface compared to the ideal surface in
way unnecessary round-off error is avoided. The weights Figure 3. The errors between the two surfaces were plotted
are scaled as necessary by the user as they are placed into in Figure 5. Now the surface in Figure 6 shows the same
the weight array. In addition to storing the weights the surface, and Figure 7 shows the error when the pseudo
location of the decimal point is also stored by the user. floating point arithmetics is enabled.
Without this floating point structure when the product of
very small numbers is calculated the error is unnecessarily
large. In some cases the weights are out of operating range
of the microcontroller and an overflow condition can occur.
In this case the surface becomes completely unrecognizable.
Using the pseudo floating point arithmetics insures that all
0.5
values remain within the operating range and overflow does
not occur.
A similar technique is also used on the inputs in order to 0
help smooth the surface. The inputs are also scaled to
maximize the number of bits used for the calculations. The
scaling is done by factors of two only so no actual -0.5
multiplication or division is required but simply shifts. In
20
order to properly take care of the shifts, the inputs can be
15 20
shifted at the beginning of the calculation and the weights 15
10
are already scaled. The bias, which is normally one, is 5
10
5
replaced with the input scale factor see equation 2.
Fig. 3. Required surface.
§ (input1 ww1 input2 ww2 bias ww3) ·
Net Act¨¨ ¸¸ (2)
© Weight Scale Input Scale ¹
0.5
-0.5
20
15 20
10 15
10
5 5
Fig. 4. Neural network surface with pseudo floating point
arithmetics disabled.
0.05
-0.05
-0.1
yB
B
0
-0.5
yA
20 A yB
15 20 B
10 15
10 yB y A
5 5 y1 ( x) yA x
2'x
Fig. 6. Surface with pseudo floating point arithmetics yA
enabled. A
§ § x · ·
2
'y¨1 ¨ 1 ¸
¨ © 'x ¸¹ ¸
y2 ( x)
'y © ¹
A 'x B y2 ( x)
'y
'x 2
'x 2 x 'x
2
0.05
'x 128
0 B
-0.05 y
y ( x) y1 ( x) y2 ( x)
-0.1 x
A0 'x 2'x
20
15 20
15
10
10
Fig. 8. Tanh approximations.
5 5
The approximation of tanh is calculated by reading the
Fig. 7. Difference between Figure 6 and 3. values of yA, yB and 'y from memory and then the first
linear approximation is calculated using yA and yB.
3
-3
x 10
5
1
4
3
5
2
2 1
0
-1
30
Input 4 20 30
Scale 20
10 10
0.6
VI. MATLAB SIMULATION
0.4
To help test the multiplication and activation algorithm, a
0.2
Matlab simulator was built. This was absolutely necessary
0 because the microcontroller introduces round-off errors
-0.2
while computing its 16-bit fixed point multiplication. In
order to simulate this round-off error, routines were written
-0.4 that round every calculation to its 8-bit counterpart. The
-0.6 round-off routine is given in equation 6.
-0.8
y floor x 256 256 (6)
-1
0.4
0.2 The multiply routine is organized as described above in the
0
0 0.05 0.1 0.15 0.2 0.25 Multiply section. Handling negatives needs special care
when modeled in Matlab. At the beginning of the Matlab
Fig. 12. Required surface using Tanh and IEE 754 floating routine the negative numbers are converted to the two’s
point calculations complement, shown in equation 7. Then the multiplication
0.6 routine is carried out normally. At the end of the routine,
0.4
the numbers are converted back to standard positive and
negative decimals to allow normal addition and subtraction
0.2 in the other steps. This conversion from two’s complement
0
is done with few subtractions. The final result is tested to
see if it underflows because when calculated on the
-0.2
microcontroller it would rollover to a positive number.
-0.4 Matlab must simulate this rollover with an extra addition as
seen in equation 8.
-0.6
-0.8
-1
a.b c.d p
0.4
0.2 a.b 0 p p c d 256
(7)
c.d 0 p p a b 256
0 0.1 0.15 0.2 0.25
0 0.05
Fig. 13. Surface using approximation of tanh and PIC based p 65536 p p 65536
mathematics with pseudo floating point.
y 256 x (8)
113
INES 2008 • 12th International Conference on Intelligent Engineering Systems • February 25–29, 2008 Miami, Florida
114