42 views

Uploaded by Ogunranti Rasaq

neural network algorithm

save

You are on page 1of 37

**Artificial Neural Networks
**

Part 1

(with English Narrations)

http://scholastictutors.webs.com

(http://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part01-2013-02-15.pdf)

1

©Scholastic Tutors (Feb, 2013)

ISVT 911-0-20-130215-1

International Baccalaureate (IB)

2

Artificial Neural Networks - #1

Classification using Single Layer Perceptron Model

http://scholastictutors.webs.com

(ANN-001)

http://youtu.be/S3iQgcoQVbc

Click here to see the video

3

Classification Example

• The following table shows sample data obtained from two

different fruits.

• Train a single layer perceptron model using the above parameters

to classify the two fruits.

• Using the model parameters you have obtained classify the fruit

with weight 140gm and length 17.9cm.

Weight (grams) Length (cm)

Fruit 1 (Class

C1)

121 16.8

114 15.2

Fruit 2 (Class

C2)

210 9.4

195 8.1

http://scholastictutors.webs.com

4

8

12

16

20

50 100 150 200 250

(121,16.8)

(114,15.2)

(195,8.1)

(210,9.4)

X

1

(Weight

in grams)

X

2

(Length in cm)

Class C

1

Class C

2

Weight (grams) Length (cm)

Fruit 1 (Class

C1)

121 16.8

114 15.2

Fruit 2 (Class

C2)

210 9.4

195 8.1

Classification Example

http://scholastictutors.webs.com

Artificial Neural Networks Theory

5

This space is left for

sponsors/advertisements

from

US$ 9

99

per advert

(Multi-book & Multi-page adverts with

web links to divert traffic to your web

sites/videos are also available)

http://scholastictutors.webs.com

http://youtu.be/S3iQgcoQVbc

Click here to see the video

6

• Artificial Neural Networks (ANN) are computers

whose architecture is modeled after the brain.

• They typically consist of many hundreds of simple

processing units which are wired together in a

complex communication network.

• Each unit or node is a simplified model of a real

neuron which fires (sends off a new signal) if it

receives a sufficiently strong input signal from the

other nodes to which it is connected.

Artificial Neural Networks Theory

http://scholastictutors.webs.com

7

• The strength of these connections may be

varied in order for the network to perform

different tasks corresponding to different

patterns of node firing activity.

• This structure is very different from traditional

computers and hence make way to Intelligent

Machines.

Intelligent Machine

http://scholastictutors.webs.com

8

• The model consists of a set of synapses each of

which is characterized by a weight or strength of

its own.

• An adder, an activation function and a bias.

I

n

p

u

t

S

i

g

n

a

l

s

¿

x

1

x

2

x

3

x

4

w

k1

w

k2

w

k3

w

k4

Synaptic Weights

Summing

Junction

Bias

b

k

Activation

Function

(.) ¢

y

k

Output v

k

Model of a Neuron

http://scholastictutors.webs.com

9

• In mathematical terms, a neuron k can be

described by:

and

where u

k

is the linear combiner output due to

input signals.

• Also .

1

m

k k j j

j

u w x

=

=

¿

( )

k k k

y u b ¢ = +

k k k

v u b = +

Some Mathematics

http://scholastictutors.webs.com

10

• The bias is an external parameter of artificial

neuron and can be included into the equations as

follows:

• and

• Note the change of limits of j from 1 to 0.

0

m

k k j j

j

v w x

=

=

¿

( )

k k

y v ¢ =

Bias as an Input

http://scholastictutors.webs.com

11

w

k0

=b

k

(bias)

¿

x

1

x

2

x

3

x

4

w

k1

w

k2

w

k3

w

k4

Synaptic Weights

(including bias)

Summing

Junction

Activation

Function

(.) ¢

y

k

Output

v

k

w

k0

Fixed Input

x

0

= +1

Modified Neuron Structure

http://scholastictutors.webs.com

12

• Threshold Function or Heaviside Function:

1 if 0

( )

0 if 0

v

v

v

¢

>

¦

=

´

<

¹

0

0.2

0.4

0.6

0.8

1.0

-2.0 -1.5 -1.0 -0.5 0 0.5 1.0 1.5 2.0

( ) v ¢

Types of Activation Functions

http://scholastictutors.webs.com

13

• Piecewise-Linear Function:

1

1,

2

1 1

( ) ,

2 2

1

0,

2

v

v v v

v

¢

¦

> +

¦

¦

¦

= + > > ÷

´

¦

¦

s ÷

¦

¹

0

0.2

0.4

0.6

0.8

1.0

-2.0 -1.5 -1.0 -0.5 0 0.5 1.0 1.5 2.0

( ) v ¢

Types of Activation Functions (cont…)

http://scholastictutors.webs.com

14

• Sigmoid Function:

where a is the slope parameter of the sigmoid

function.

( )

1

( )

1 exp

v

av

¢ =

+ ÷

0

0.2

0.4

0.6

0.8

1.0

-8.0 -6.0 -4.0 -2.0 0 2.0 4.0 6.0 8.0

( ) v ¢

Increasing a

Types of Activation Functions (cont…)

http://scholastictutors.webs.com

• The neuronal model we have just discussed is

also known as a perceptron.

• The perceptron is the simplest form of a

neural network used for the classification of

patterns said to be linearly separable.

• Basically, it consists of a single neuron with

adjustable synaptic weights and bias.

• Now we will look at a method of achieving

learning in our model we have formulated.

Single Layer Perceptron

15

http://scholastictutors.webs.com

16

• Variables and Parameters

| |

| |

1 2

1 2

1 1 input vector

1

1 1 weight vector

bias

actual response

desired response

learning-rate parameter, a postive constant less

T

m

T

m

n m

x n x n x n

n m

b n w n w n w n

b n

y n

d n

q

= + ×

= +

= + ×

=

=

=

=

=

x( ) ( )

, ( ), ( ), , ( )

w( ) ( )

( ), ( ), ( ), , ( )

( )

( )

( )

than unity

Perceptron Convergence (Learning)

Algorithm

http://scholastictutors.webs.com

17

1. Initialization. Set . Then perform the following

computations for time step n = 1,2, …

2. Activation. At time step n, activate the perceptron by

applying input vector and desired response .

3. Computation of Actual Response. Compute the actual

response of the perceptron:

where is the signum function.

0 = w( ) 0

n x( )

d n ( )

sgn

T

y n n n = ( ) [w ( )x( )]

sgn(.)

1 if 0

sgn

1 if 0

,

( )

,

x

x

x

+ > ¦

=

´

÷ <

¹

Perceptron Convergence (Learning)

Algorithm (cont…)

http://scholastictutors.webs.com

18

4. Adaptation of Weight Vector. Update the

weight vector of the perceptron:

where

5. Continuation. Increment time step n by one

and go back to step 2.

| |

1 n n d n y n n q + = + ÷ w( ) w( ) ( ) ( ) x( )

1

2

1 if belongs to class

1 if belongs to class

n C

d n

n C

+

¦

=

´

÷

¹

x( )

( )

x( )

Perceptron Convergence (Learning)

Algorithm (cont…)

http://scholastictutors.webs.com

19

• The hyper-plane

is the decision

boundary for a two

class classification

problem.

1

1 1 2 2

0

or

0

m

i i

i

w x b

w x w x b

=

+ =

+ + =

¿

x

2

x

1

0

Class C

2

Class C

1

Decision Boundary

http://scholastictutors.webs.com

Solution to the Example Question

(with correct initial weights and bias)

20

This space is left for

sponsors/advertisements

from

US$ 9

99

per advert

(Multi-book & Multi-page adverts with

web links to divert traffic to your web

sites/videos are also available)

http://scholastictutors.webs.com

http://youtu.be/S3iQgcoQVbc

Click here to see the video

1 2

0 30 0 300

0 50 0 01

( ) , ( ) ,

( ) , .

w w

given

b q

= ÷ =

¹

`

= =

)

¿

x

1

x

2

50

-30

300

Activation

Function

(.) sig

y

(n)

Output

Fixed Input

x

0

= +1

With correct initial

weights and bias

1 if 0

sgn

1 if 0

,

( )

,

x

x

x

+ > ¦

=

´

÷ <

¹

Solution to the Example

(with correct initial weights and bias)

21

http://scholastictutors.webs.com

Therefore the Initial Decision

Boundary for this example is:

1 2

30 300 50 0 x x ÷ + + =

4

8

12

16

20

50 100 150 200 250

(121,16.8)

(114,15.2)

(195,8.1)

(210,9.4) (100,9.83)

(200,19.83)

Class C

1

Class C

2

x

1

x

2

Initial hyper-plane does separate the two classes.

22

Solution to the Example

(with correct initial weights and bias) (cont…)

1 2

30 100 50

100 9 83

300

, . x x

× ÷

= = =

1 2

30 200 50

200 19 83

300

, . x x

× ÷

= = =

1 1 2 2

0 wx w x b + + =

1 2

0 30 0 300

0 50 0 01

( ) , ( ) ,

( ) , .

w w

given

b q

= ÷ =

¹

`

= =

)

http://scholastictutors.webs.com

Class Unknown

x

1

= 140

x

2

= 17.9

x

0

= +1

Now use the above model to classify the unknown fruit.

| |

| |

( )

( )

1

unknown 1 140 17 9

3 50 30 300

unknown sgn 3 unknown sgn 50 1 30 140 300 17 9

sgn 1220 1

this unkown fruit belonge to the class .

x( ) , , .

w( ) , ,

( ) w ( )x( ) .

( )

T

T

T

y

C

= +

= ÷

= = × ÷ × + ×

= = +

¿

50

-30

300

Activation

Function

(.) sig

y

(n)

=+1

Output

Fixed Input

+1220

For Class C1,

Output = +1

Classification of the Unknown Fruit

23

this unknown fruit belongs to the class

http://scholastictutors.webs.com

Solution to the Example Question

(with unknown initial weights and bias)

24

This space is left for

sponsors/advertisements

from

US$ 9

99

per advert

(Multi-book & Multi-page adverts with

web links to divert traffic to your web

sites/videos are also available)

http://scholastictutors.webs.com

http://www.youtube.com/watch?feature=player_profilepage&v=S3iQgcoQVbc#t=1220s Click here to see the video

25

1 2

0 30 0 300

0 1230 0 01

( ) , ( ) ,

( ) , .

w w

given

b q

= ÷ =

¹

`

= ÷ =

)

1 2

30 300 1230 0 x x ÷ + ÷ =

1 2

30 100 1230

100 14 1

300

, . x x

× +

= = =

1 2

30 200 1230

200 24 1

300

, . x x

× +

= = =

Initial hyperplane does not separate the two classes.

Therefore we need to Train the Neural Network

4

8

12

16

20

50 100 150 200 250

(121,16.8)

(114,15.2)

(195,8.1)

(210,9.4)

(100,14.1)

(200,24.1)

x

1

x

2

With unknown initial

weights and bias

Solution to the Example

(with unknown initial weights and bias)

Therefore the Decision Boundary for this

case:

http://scholastictutors.webs.com

Class C1

x

1

= 121

x

2

= 16.8

x

0

= +1

| |

| |

( )

( )

| |

0 1 121 16 8 and 0 1

0 1230 30 300

0 sgn 0 0

sgn 1230 1 30 121 300 16 8

sgn 180 1 0

Hence no need to recalculate the weights.

1 1 1230 30 300

x( ) x( ) , , . ( )

w( ) w( ) , ,

( ) ( ) w ( )x( )

.

( ) ( )

w( ) w( ) , ,

T

T

T

T

n d

n

y n y

d

n

= = + = +

= = ÷ ÷

= =

= ÷ × ÷ × + ×

= = + =

+ = = ÷ ÷

¿

300

-30

-1230

Activation

Function

(.) sig

y

(n)

=+1

Output

Fixed Input

+180

For Class C1,

Output = +1

Training with known Fruit

(121,16.8)

26

http://scholastictutors.webs.com

Class C1

x

1

= 114

x

2

= 15.2

x

0

= +1

| |

| |

( ) ( )

1 1 114 15 2 and 1 1

1 1230 30 300

1 sgn 1 1 sgn 1230 1 30 114 300 15 2

sgn 90 1 1

Hence recalculate the weights.

x( ) , , . ( )

w( ) , ,

( ) w ( )x( ) .

( ) ( )

we have to

T

T

T

d

y

d

= + = +

= ÷ ÷

= = ÷ × ÷ × + ×

= ÷ = ÷ =

¿

300

-30

-1230

Activation

Function

(.) sig

y

(n)

=-1

Output

Fixed Input

-90

For Class C1, the

output should be +1,

but what we get is -1.

Hence we have to

recalculate the weights.

Training with known Fruit

(114,15.2)

http://scholastictutors.webs.com

28

Here we use Adaptation of Weight Vector (Step 4) to update the weight

vector of the perceptron.

| |

| |

| |

| | ( ) | |

| | | |

| |

1

1 1230 30 300

1 1 114 15 2

1 1 1 1 0 01

1 1 2 1230 30 300 0 01 1 1 1 114 15 2

1230 30 300 0 02 2 28 0 304

2 1229 08 27 72 300 304

w( ) w( ) ( ) ( ) x( )

w( ) , ,

x( ) , , .

( ) , ( ) , .

w( ) w( ) , , . , , .

, , . , . , .

w( ) . , . , .

T

T

T T

T T

T

n n d n y n n

d y

q

q

+ = + ÷

= ÷ ÷

= +

= + = ÷ =

( + = = ÷ ÷ + + ÷ ÷ +

¸ ¸

= ÷ ÷ + +

= ÷ ÷

Adaptation of Weight Vector

http://scholastictutors.webs.com

Class C2

x

1

= 210

x

2

= 9.4

x

0

= +1

| |

| |

( ) ( )

2 1229 08 27 72 300 304

2 1 210 9 4 and 2 1

2 sgn 2 2 sgn 1229 08 1 27 72 210 300 304 9 4

sgn 4227 4224 1 2

Hence no need to recalculate the weights.

1 3 1229 08 27 7

w( ) . , . , .

x( ) , , . ( )

y( ) w ( )x( ) . . . .

( . ) ( )

w( ) w( ) . , .

T

T

T

d

d

n

= ÷ ÷

= + = ÷

= = ÷ × ÷ × + ×

= ÷ = ÷ =

+ = = ÷ ÷

| |

2 300 304 , .

T

¿

300.304

-27.72

-1229.08

Activation

Function

(.) sig

y

(n)

=-1

Output

Fixed Input

-4227.4224

For Class C2,

Output = -1

Training with known Fruit

(210,9.4)

29

http://scholastictutors.webs.com

Class C2

| |

| |

( ) ( )

3 1229 08 27 72 300 304

3 1 195 8 1 and 3 1

3 sgn 3 3 sgn 1229 08 1 27 72 195 300 304 8 1

sgn 4202 0176 1 3

Hence no need to recalculate the weights.

1 4 1229 08 27 7

w( ) . , . , .

x( ) , , . ( )

y( ) w ( )x( ) . . . .

( . ) ( )

w( ) w( ) . , .

T

T

T

d

d

n

= ÷ ÷

= + = ÷

= = ÷ × ÷ × + ×

= ÷ = ÷ =

+ = = ÷ ÷

| |

2 300 304 , .

T

¿

x

1

= 195

x

2

= 8.1

300.304

-27.72

-1229.08

Activation

Function

(.) sig

y

(n)

=-1

Output

Fixed Input

x

0

= +1

-4202.0176

For Class C2,

Output = -1

Training with known Fruit

(195,8.1)

30

http://scholastictutors.webs.com

Class C1

| |

| |

( ) ( )

4 1229 08 27 72 300 304

4 1 121 16 8 and 4 1

4 sgn 4 4 sgn 1229 08 1 27 72 121 300 304 16 8

sgn 461 91 1 4

Hence no need to recalculate the weights.

1 5 1229 08 27 72

w( ) . , . , .

x( ) , , . ( )

y( ) w ( )x( ) . . . .

( . ) ( )

w( ) w( ) . , . ,

T

T

T

d

d

n

= ÷ ÷

= + = +

= = ÷ × ÷ × + ×

= = + =

+ = = ÷ ÷

| |

300 304 .

T

¿

x

1

= 121

x

2

= 16.8

300.304

-27.72

-1229.08

Activation

Function

(.) sig

y

(n)

=+1

Output

Fixed Input

x

0

= +1

461.91

For Class C1,

Output = +1

31

Training with known Fruit

(121,16.8)

http://scholastictutors.webs.com

| |

| |

( ) ( )

5 1229 08 27 72 300 304

5 1 114 15 2 and 5 1

5 sgn 5 5 sgn 1229 08 1 27 72 114 300 304 15 2

sgn 175 46 1 5

Hence no need to recalculate the weights.

1 6 1229 08 27 72

w( ) . , . , .

x( ) , , . ( )

y( ) w ( )x( ) . . . .

( . ) ( )

w( ) w( ) . , . ,

T

T

T

d

d

n

= ÷ ÷

= + = +

= = ÷ × ÷ × + ×

= = + =

+ = = ÷ ÷

| |

300 304 .

T

Class C1

¿

x

1

= 114

x

2

= 15.2

300.304

-27.72

-1229.08

Activation

Function

(.) sig

y

(n)

=+1

Output

Fixed Input

x

0

= +1

175.46

For Class C1,

Output = +1

32

Training with known Fruit

(114,15.2)

http://scholastictutors.webs.com

1 2

27 72 300 304 1229 08 0 . . . x x ÷ + ÷ =

1 2

27 72 100 1229 08

100 13 32

300 30

. .

, .

.

x x

× +

= = =

Therefore the decision boundary obtained

after Neural Network training is:

1 2

27 72 200 1229 08

200 22 55

300 30

. .

, .

.

x x

× +

= = =

4

8

12

16

20

50 100 150 200 250

(121,16.8)

(114,15.2)

(195,8.1)

(210,9.4)

(100,13.32)

(200,22.55)

x

1

x

2

Class C

1

Class C

2

The new decision boundary

can now be used to classify

any unknown item.

Decision Boundary After Training

33

http://scholastictutors.webs.com

Classification of the Unknown Fruit using the

New Decision Boundary

34

This space is left for

sponsors/advertisements

from

US$ 9

99

per advert

(Multi-book & Multi-page adverts with

web links to divert traffic to your web

sites/videos are also available)

http://scholastictutors.webs.com

http://www.youtube.com/watch?feature=player_detailpage&v=S3iQgcoQVbc#t=1889s Click here to see the video

Class Unknown

x

1

= 140

x

2

= 17.9

x

0

= +1

Now use the above model to classify the unknown fruit.

For Class C1,

Output = +1

Classification of the Unknown Fruit

using the New Decision Boundary

35

¿

300.304

-27.72

-1229.08

Activation

Function

(.) sig

y

(n)

=+1

Output

Fixed Input

175.46

unknown = +1,140,17.9

5 = −1229.08, −27.72,300.304

y unkown = sgn

(5)(unknown)

= sgn −1229.08 × 1 − 27.72 × 140 + 300.304 × 17.9

= sgn 265.56 = +1

∴ this unknown fruit belongs to the class

1

http://scholastictutors.webs.com

International Baccalaureate (IB)

36

Artificial Neural Networks - #1

Classification using Single Layer Perceptron Model

http://scholastictutors.webs.com

(ANN-001)

END of the Book

If you like to see similar solutions to any Mathematics problems please

contact us at: home.video.tutor@gmail.com with your request.

http://youtu.be/S3iQgcoQVbc

Click here to see the video

Videos at: http://www.youtube.com/user/homevideotutor

37

(http://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part01-2013-02-15.pdf)

Scholastic Video Book Series

Artificial Neural Networks

Part 1

(with English Narrations)

(END)

©Scholastic Tutors (Feb, 2013)

ISVT 911-0-20-130215-1

- MultilayerUploaded byMd Qutubuddin
- 131473648-Neural-and-Fuzzy-Logic.pdfUploaded byApoorvagiri
- ch6PUploaded byAmrutha Ghali
- Application of Artificial Neural Network (ANN) Technique for the Measurement of Voltage Stability Using FACTS Controllers - ITUploaded bypradeep9007879
- ann4-3s.pdf_7oct.pdfUploaded byveena
- softcomputing_1Uploaded byAmodita Roy
- Backpropagation for Continuous Theta Neural NetworksUploaded bydavid fan
- New Microsoft Word DocumentUploaded byAguz Munandar
- NNFLUploaded bySri Lalitha
- A Survey of Machine LearningUploaded byAnonymous lVQ83F8mC
- vectorisationUploaded byIlie Stefan
- COMP 4211 - Machine LearningUploaded byLinh Ta
- ME SyllabusUploaded byjainshines
- Artificial IntelligenceUploaded bysudheer1044
- Chapter 6Uploaded byarmin2200
- SCT modelUploaded byKannan Tindivanam
- Artificial Neural Network Model WithUploaded bymsuid

- CalculationsUploaded byOgunranti Rasaq
- LOOP DIAGRAMUploaded byOgunranti Rasaq
- 10.5923.j.eee.20130302.04Uploaded byOgunranti Rasaq
- Guide to the Nigerian Power SectorUploaded byOgunranti Rasaq
- Full Text Analysis of Power Flow of Nigerian 330kv Grid SystemUploaded byOgunranti Rasaq
- 70Uploaded byOgunranti Rasaq
- Transmission Company of Nigeria Investor Forum PresentationUploaded byOgunranti Rasaq