Unit Ii

www.Vidyarthiplus.
com
UNIT-2
NEURAL NETWORKS
FOR
CONTROL
www.Vidyarthiplus.com
Hopfield-Tank model
TSP must be mapped, in some way, onto the neural network
structure
Each row corresponds to a particular city and each column to
a particular position in the tour
Mapping TSP to Hopfield neural net

There is a connection between each pair of
units
The signal sent along a connection from i to t
j is equal to the weight Tij if i is activated. It is
equal to 0 otherwise.
A negative weight defines inhibitory
connection between the two units
It is unlikely that two units with negative
weigh will be active or on at the same time
Discrete Hopfield Model

connection weights are not learned
Hopfield network evolves by updating the activation of each
unit in turn
In final state, all units are stable according to the update rule
The units are updated at random, one unit at a time
{Vi}i=1,...,L,
L :number of units
Vi :activation level of unit i
Tij: connection weight between units i and j

tetai: threshold of unit i.
Discrete Hopfield Model (Cont.)

Energy function
Units changes its activation level if and only if the energy of

the network decreases by doing so:
Since the energy can only decrease over time and the number
configuration is finite
the network must converge (but not necessarily the minimum
energy state)
Continuous Hopfield-Tank
Neuron function is continuous
(Sigmoid function)
The evolution of the units over
time is now characterized by the
following differential equation :
Ui, Ii and Vi are the input, input

bias, and activation level of unit I,
respectively
Continuous Hopfield-Tank
Energy function
Discrete time approximation is applied to the

equations of motion
Application of the Hopfield-Tank

Model to the TSP

model to the TSP
(1)The TSP is represented as an N*N matrix
(2) Energy function
(3)Bias and connection weights are derived

model to the TSP
Results of Hopfield-Tank
Hopfield and Tank were able to solve a
randomly generated 10-city,with parameter
value :A=B=500,C=200,N=15.
They reported for 20 trails, network converge
16 times to feasible tours.
Half of those tours were one of two optimal
tours
The size of each black square

indicates the value of the
output of the corresponding
neuron
The main weaknesses of the original

Hopfield-Tank model

Hopfield-Tank model
(d) Model plagued with the limitation of hillclimbing approaches
(e) Model does not guarantee feasibility

Hopfield-Tank model
The positive points:
Can easily implemented in hardware
Can be applied to non-Euclidean TSPs
PROCESS IDENTIFICATION
Pattern Classification Network
5 output nodes
16 output nodes
100 input nodes
Character Recognition
Learning Curve for Training the Pattern Classification Network
The Learning Curve for Training the Pattern Classification Network
1200
800
600
Cumulative Iterations
1000
400
200
0
0.01
0.009
0.008
0.007
0.006
0.005
0.004
0.003
0.002
0.001
Error Tolerance
The chart depicts the learning curve, as the error tolerance was successively lowered
through the values 0.01, 0.005, 0.0025, 0.001, 0.0005, 0.00025, and finally 0.0001.
The vertical axis shows the cumulative iterations needed to achieve the perfect 5 out of
5 training performance at each level of error tolerance.
Training the Network

Tips for Training
Training Tip #1
Start with a relatively large error tolerance, and incrementally lower it to the desired level
as training is achieved at each succeeding level. This usually results in fewer training
iterations than starting out with the desired final error tolerance.
Training Tip #2
If the network fails to train at a certain error tolerance, try successively lowering the
learning rate.

Tips for Training
Training Tip #3
In a system to be used for a real-world application, such as character recognition, you
would want the network to be able to handle not only pixel noise, but also size variance
(slightly smaller letters), some rotation, as well as some variations in font style.
To produce such a robust network classifier, you would need to add representative
samples to the training set. For example, samples of rotated versions of a character could
be added to the training set, etc.
Training a Network
The Momentum Term
Smooth out the effect of weight adjustments over time.
Formula:
Weight Change = learning rate * input * error output +
momentum_parameter * previous_weight_change
Momentum term can be disabled by setting it to zero.
Warning! Setting the momentum term and learning rate too large can overshoot a good
minimum as it takes large steps!
Using Neural Networks in a Control Problem

Inverted Pendulum Problem
1 output node
20 output nodes
5 input nodes
Input:
x, v, theta, angular velocity
Output: Angle

Inverted Pendulum Problem (Dynamics of the Problem)
Formulas:
mg sin( (t )) cos( (t ))[ F (t ) + mbl ( ' (t )) 2 sin( (t ))]
' ' (t ) =
(4 / 3)ml mb l cos 2 ( (t ))
x' ' (t ) = {F (t ) + mb l ( ' (t )) 2 sin( (t )) ' (t ) cos( (t ))]} / m
(t ) = broom angle(with vertical) at time t (in radians)

' (t ) = angular velocity
Input:
x, v, theta, angular
velocity
Output: Angle
x(t) = cart position at time t ( in meters)

x(t) = cart velecity
F(t) = force applied to cart at time t (Newtons)
m = combined mass of cart and broom (1.1 kg)
See FFBRM.EXE
mb = mass of broom (0.1 kg)

l = length of broom (pivot point to center of mass, 0.5 meters)

Inverted Pendulum Problem (Limits of Network Training)
The equations weve seen ignore the effects of

friction. Control system failure occurs when the
cart hits either end of the track (at x= 2.4 meters),
or the angle reaches radians (180 degrees)
For practical purposes, though, unrecoverable

failure occurs when the angle reaches 12
degrees in magnitude so the Neural Network
training will be restricted to this range.
State of the Cart-Broom System:

x, x, theta , angular velocity -dot

SIMULATION ISSUES
How can we simulate the complete behaviour of the system now that we know how to derive
all the state variables describing the system?
What is the cart-broom state given any time t?
Eulers Method
Eulers method relates a variable and its derivative via the
simple approximation equation:
x(t + h) = x(t ) + h x' (t )

Eulers method has only its simplicity to recommend it, and is
normally not used when any amount of accuracy is required of
a numerical solution. However, it is adequate for our needs.
State of the Cart-Broom System:
x, x, theta , angular velocity -dot

SIMULATION ISSUES
Heres an appplication of Eulers method
mg sin( (t )) cos( (t ))[ F (t ) + mbl ( ' (t )) 2 sin( (t ))]
' ' (t ) =
(4 / 3)ml mb l cos 2 ( (t ))
Eulers Method
x(t + h) = x(t ) + h x' (t )
float f_theta (float frce,float th1,float th2)

{
float denom,numer,cost,sint;
cost = cos (th1);
sint = sin (th1);
denom = four_thirds * m * l - mb * l * cost * cost; /* Always > 0 */
numer = m * g * sint cost * (frce + mb * l * th2 * th2 * sint);
return numer/denom;
}
float f_x (float frce,float th1,float th2,float th3)
{
float cost,sint,term;
cost = cos (th1);
sint = sin (th1);
term = mb * l * (th2 * th2 * sint - th3 * cost);
return (frce + term)/m;
}
x' ' (t ) = {F (t ) + mbl ( ' (t )) 2 sin( (t )) ' (t ) cos( (t ))]} / m

SIMULATION ISSUES
Heres an appplication of Eulers method
Eulers Method
x(t + h) = x(t ) + h x' (t )
void new_broom_state (float frce,state_rec old_state,state_rec

*new_state)
{
const float h = 0.02;
float th3;
/* Euler's method applied to system of equations */
/* (not known for accuracy, but good enough here !) */
new_state->theta = old_state.theta + h * old_state.theta_dot;
th3 = f_theta (frce,old_state.theta,old_state.theta_dot);
new_state->theta_dot = old_state.theta_dot + h * th3;
new_state->x_pos = old_state.x_pos + h * old_state.x_dot;
new_state->x_dot = old_state.x_dot +
h * f_x (frce,old_state.theta,old_state.theta_dot,th3);
return;
}

Feed-Forward Controller System
How can we teach the Network to learn broom balancing?
Analogy to a Human Controller

Suppose that a human controller has no idea how the cart-broom system is going to respond to
input forces. By randomly applying a force and observing whether or not that force helps in
balancing the broom, eventually, the controller may notice that pushing the cart in the same
direction that the broom is leaning will slide the cart back under the broom and tends to restore
a vertical angle.
With enough repetitions, the controller learns how much force to supply and how often. An
expert controller can anticipate which way the broom is going to fall and can apply a corrective
force before the broom goes far in that direction.

Feed-Forward Controller System
The approach in teaching the network is similar to the process of teaching a human controller.
The networks will learn the dynamics of the cart-broom system by observing numerous
repetitions of random-force applications at different broom states.
The trained network will then be a model of the cart-broom dynamics. Like the human expert,
the network can predict the next broom state,
state and with this knowledge the correct force can be
applied.
applied
Collection of Training Data
Run the cart-broom system through 100 random force
applications and collect max-min data.
All input data should be normalized. Each parameter will
have its own [min, max] values.

Feed-Forward Controller System (BANG-BANG Control)
Running the Network
The controller operates by looking ahead, like an experienced controller would. The trained
network emulates the broom dynamics, so that the controller can ask: What will happen if I do
nothing (zero force)?
The trained network answers this question by supplying the broom angle that would result on
the next iteration if zero force were applied. Once the angle is predicted, the appropriate
action can be taken.

Alternative Solution to the Inverted Pendulum Problem
2 output nodes
16 (4+4)
hidden nodes
4 input nodes
Input:
x, v, theta, angular velocity
Output: Force, direction
Noise factor could be added during training (e.g. 0.05) to make the
network more robust.
See FFBRM.EXE

Network Inputs
The following could be done after collecting the raw data for Neural Net
training.
1.
Calculate max, min values of each input parameter.

[min, max] for each variable (x, x-dot, theta, theta-dot)

Network Inputs
The following could be done after collecting the raw data for Neural Net
training.
1.
Calculate max, min values of each input parameter.

[min, max] for each variable (x, x-dot, theta, theta-dot)
2.
3.
Calculate xFactor.
see pdf file for more details on this.

Unit Ii

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Unit Ii

Uploaded by

Copyright:

Available Formats

www.Vidyarthiplus.

Mapping TSP to Hopfield neural net

Discrete Hopfield Model

Tij: connection weight between units i and j

Discrete Hopfield Model (Cont.)

Units changes its activation level if and only if the energy of

Ui, Ii and Vi are the input, input

Discrete time approximation is applied to the

Application of the Hopfield-Tank

Application of the Hopfield-Tank

(3)Bias and connection weights are derived

Application of the Hopfield-Tank

The size of each black square

The main weaknesses of the original

The main weaknesses of the original

The main weaknesses of the original

Training the Network

Training the Network

Using Neural Networks in a Control Problem

x, v, theta, angular velocity

Using Neural Networks in a Control Problem

x' ' (t ) = {F (t ) + mb l ( ' (t )) 2 sin( (t )) ' (t ) cos( (t ))]} / m

(t ) = broom angle(with vertical) at time t (in radians)

x(t) = cart position at time t ( in meters)

mb = mass of broom (0.1 kg)

Using Neural Networks in a Control Problem

The equations weve seen ignore the effects of

For practical purposes, though, unrecoverable

State of the Cart-Broom System:

Using Neural Networks in a Control Problem

x(t + h) = x(t ) + h x' (t )

Using Neural Networks in a Control Problem

x(t + h) = x(t ) + h x' (t )

float f_theta (float frce,float th1,float th2)

x' ' (t ) = {F (t ) + mbl ( ' (t )) 2 sin( (t )) ' (t ) cos( (t ))]} / m

Using Neural Networks in a Control Problem

x(t + h) = x(t ) + h x' (t )

void new_broom_state (float frce,state_rec old_state,state_rec

Using Neural Networks in a Control Problem

Analogy to a Human Controller

Using Neural Networks in a Control Problem

Using Neural Networks in a Control Problem

Using Neural Networks in a Control Problem

x, v, theta, angular velocity

Output: Force, direction

Using Neural Networks in a Control Problem

Calculate max, min values of each input parameter.

Using Neural Networks in a Control Problem

Calculate max, min values of each input parameter.

You might also like