Neural Networks CSE425

Moin Mostakim Assignment 1 January 30, 2020

Basic Perceptron

Build a neuron with Bias

Try to add bias in the programme and perform your programme in any binary
classification data
Understand the given code and explain your code after adding bias. Try to show data
representation code and explain how bias helps to neuron.

1. Introduction Of basic Neuron

When we say Neural Networks, we mean artificial Neural Networks (ANN). The idea of
ANN is based on biological neural networks like the brain.
The basic structure of a neural network is the neuron. A neuron in biology consists of
three major parts: the soma (cell body), the dendrites, and the axon.

Figure 1: Biological Neuron

The dendrites branch of from the soma in a tree-like way and getting thinner with
every branch. They receive signals (impulses) from other neurons at synapses. The
axon - there is always only one - also leaves the soma and usually tend to extend for
longer distances than the dentrites. The axon is used for sending the output of the
neuron to other neurons or better to the synapsis of other neurons.

1
 Neural Networks CSE425
Moin Mostakim Assignment 1 January 30, 2020

Figure 2: Graphical representation of biological neuron

A perceptron of artificial neural networks is simulating a biological neuron.

Figure 3: Mathematical process of simple neuron

When a signal comes in, it gets multiplied by a weight value that is assigned to this
particular input. That is, if a neuron has three inputs, then it has three weights that
can be adjusted individually. The weights usually get adjusted during the learn phase.
After this the modified input signals are summed up. It is also possible to add addi-
tionally a so-called bias b to this sum. The bias is a value which can also be adjusted
during the learn phase.

2
 Neural Networks CSE425
Moin Mostakim Assignment 1 January 30, 2020

Figure 4: Perceptron

Finally, the actual output has to be determined. For this purpose an activation or
step function φ is applied to weighted sum of the input values.

2. A simple neural network implementing AND function:

You could imagine that you have two attributes describing am eddible object like a
fruit for example: sweetness and sourness
We could describe this by points in a two-dimensional space. The x axis for the
sweetness and the y axis for the sourness. Imagine now that we have two fruits as
points in this space, i.e. an orange at position (3.5, 1.8) and a lemon at (1.1, 3.9).
We could define dividing lines to define the points which are more lemon-like and
which are more orange-like. The following program calculates and renders a bunch of
lines. The red ones are completely unusable for this purpose, because they are not
separating the classes. Yet, it is obvious that even the green ones are not all useful.
Listing 1: Plotting lines Python script!
import numpy a s np
import m a t p l o t l i b . p y p l o t a s p l t

def c r e a t e _ d i s t a n c e _ f u n c t i o n ( a , b , c ) :
""" 0 = ax + by + c """
def d i s t a n c e ( x , y ) :
""" r e t u r n s t u p l e ( d , pos )
d is the distance
I f pos == −1 p o i n t i s b e l o w t h e l i n e ,
0 on t h e l i n e and +1 i f ab o ve t h e l i n e
"""
nom = a ∗ x + b ∗ y + c
i f nom == 0 :
pos = 0
e l i f (nom<0 and b<0) or (nom>0 and b >0):
pos = −1

3
 Neural Networks CSE425
Moin Mostakim Assignment 1 January 30, 2020

else :
pos = 1
return ( np . a b s o l u t e (nom) / np . s q r t ( a ∗∗ 2 + b ∗∗ 2 ) , pos )
return d i s t a n c e

points = [ (3.5 , 1.8) , (1.1 , 3.9) ]

f i g , ax = p l t . s u b p l o t s ( )
ax . s e t _ x l a b e l ( " s w e e t n e s s " )
ax . s e t _ y l a b e l ( " s o u r n e s s " )
ax . set_xlim ([ −1 , 6 ] )
ax . set_ylim ([ −1 , 8 ] )
X = np . a r a n g e ( −0.5 , 5 , 0 . 1 )

c o l o r s = [ " r " , "" ] # f o r the samples

s i z e = 10
f o r ( index , ( x , y ) ) in enumerate ( p o i n t s ) :
i f i n d e x== 0 :
ax . p l o t ( x , y , " o " ,
c o l o r=" d a r k o r a n g e " ,
m a r k e r s i z e=s i z e )
else :
ax . p l o t ( x , y , " oy " ,
m a r k e r s i z e=s i z e )

step = 0.05
f o r x in np . a r a n g e ( 0 , 1+s t e p , s t e p ) :
s l o p e = np . tan ( np . a r c c o s ( x ) )
d i s t 4 l i n e 1 = c r e a t e _ d i s t a n c e _ f u n c t i o n ( s l o p e , −1, 0 )
#p r i n t (" x : " , x , " s l o p e : " , s l o p e )
Y = slope ∗ X

results = []
f o r p o i n t in p o i n t s :
r e s u l t s . append ( d i s t 4 l i n e 1 ( ∗ p o i n t ) )
#p r i n t ( s l o p e , r e s u l t s )
i f ( r e s u l t s [ 0 ] [ 1 ] != r e s u l t s [ 1 ] [ 1 ] ) :
ax . p l o t (X, Y, "g−" )
else :
ax . p l o t (X, Y, " r−" )

p l t . show ( )

4
 Neural Networks CSE425
Moin Mostakim Assignment 1 January 30, 2020

Figure 5: Neural Network With out Bias

In the following program, we train a neural network to classify two clusters in a 2-

dimensional space. We show this in the following diagram with the two classes class1
and class2. We will create those points randomly with the help of a line, the points of
class2 will be above the line and the points of class1 will be below the line.
Listing 2: Perceptron Python script!
import numpy a s np

class Perceptron :

def __init__ ( s e l f , input_length , w e i g h t s=None ) :

i f w e i g h t s i s None :
s e l f . w e i g h t s = np . o n e s ( i n p u t _ l e n g t h ) ∗ 0 . 5
else :
s e l f . weights = weights

@staticmethod
def u n i t _ s t e p _ f u n c t i o n ( x ) :
if x > 0.5:
return 1
return 0

5
 Neural Networks CSE425
Moin Mostakim Assignment 1 January 30, 2020

def __call__ ( s e l f , in_data ) :

weighted_input = s e l f . w e i g h t s ∗ in_data
weighted_sum = weighted_input .sum( )
return P e r c e p t r o n . u n i t _ s t e p _ f u n c t i o n ( weighted_sum )

p = P e r c e p t r o n ( 2 , np . a r r a y ( [ 0 . 5 , 0 . 5 ] ) )

data_in = np . empty ( ( 2 , ) )
f o r i n 1 in range ( 2 ) :
f o r i n 2 in range ( 2 ) :
data_in = ( in1 , i n 2 )
data_out = p ( data_in )
print ( data_in , data_out )

Figure 6: Data divided in classes

We will see that the neural network will find a line that separates the two classes.
This line should not be mistaken for the line, which we used to create the points.

6
 Neural Networks CSE425
Moin Mostakim Assignment 1 January 30, 2020

This line is called a decision boundary.

Figure 7: Line identification through Neural Network