Data Mining and Neural Networks

Danny Leung CS157B, Spring 2006 Professor Sin-Min Lee

Artificial Intelligence for Data Mining

Neural networks are useful for data mining and decision-support applications.
People are good at generalizing from experience. Computers excel at following explicit instructions over and over.

Neural networks bridge this gap by modeling, on a computer, the neural behavior of human brains.
2

Neural Network Characteristics

Neural networks are useful for pattern recognition or data classification, through a learning process. Neural networks simulate biological systems, where learning involves adjustments to the synaptic connections between neurons
3

Anatomy of a Neural Network

Neural Networks map a set of input-nodes to a set of output-nodes Number of inputs/outputs is variable The Network itself is composed of an arbitrary number of nodes with an arbitrary topology

Input 0

Input 1

...

Input n

Neural Network

Output 0

Output 1

...

Output m

Biological Background

A neuron: many-inputs / one-output unit

Output can be excited or not excited Incoming signals from other neurons determine if the neuron shall excite ("fire")

Output subject to attenuation in the synapses, which are junction parts of the neuron
5

Basics of a Node
A node is an element which performs a function
Wb Input 0 Input 1

... ...

Input n

W0

W1

Wn

y = fH((wixi) + Wb)

+
fH(x)

Connection
Output

Node
6

A Simple Preceptron

Binary logic application

fH(x) [linear threshold] Wi = random(-1,1) Y = u(W0X0 + W1X1 + Wb)
Wb

Input 0

Input 1

W0

W1

+
fH(x)

Output
7

Preceptron Training

Its a single-unit network

Adjust weights based on a how well the current weights match an objective

Perceptron Learning Rule

Wi = * (D-Y).Ii
= Learning Rate D = Desired Output

Neural Network Learning

From experience: examples / training data

Strength of connection between the neurons is stored as a weight-value for the specific connection

Learning the solution to a problem = changing the connection weights

9

Neural Network Learning

Continuous Learning Process

Evaluate output Adapt weights Take new inputs Learning causes stable state of the weights
10

Learning Performance

Supervised

Need to be trained ahead of time with lots of data

Unsupervised networks adapt to the input

Applications in Clustering and reducing dimensionality Learning may be very slow No help from the outside No training data, no information available on the desired output Learning by doing Used to pick out structure in the input:
Clustering Compression

11

Topologies BackPropogated Networks

Inputs are put through a Hidden Layer before the output layer
All nodes connected between layers

Input 0

Input 1

...

Input n

H0

H1

...

Hm

Hidden Layer

O0

O1

... ...

Oo

Output 0

Output 1

Output o

12

BP Network Supervised Training

Desired output of the training examples Error = difference between actual & desired output

Change weight relative to error size

Calculate output layer error , then propagate back to previous layer

Hidden weights updated

Improved performance
13

Neural Network Topology Characteristics

Set of inputs Set of hidden nodes Set of outputs

Increasing nodes makes network more difficult to train

14

Applications of Neural Networks

Prediction weather, stocks, disease

Classification financial risk assessment, image processing Data association Text Recognition (OCR) Data conceptualization Customer purchasing habits Filtering Normalizing telephone signals (static)
15

Overview

Advantages
Adapt to unknown situations Robustness: fault tolerance due to network redundancy Autonomous learning and generalization

Disadvantages
Not exact Large complexity of the network structure
16

Referenced Work

Intro to Neural Networks - Computer Vision Applications and Training Techniques. Doug Gray. www.soe.ucsc.edu/~taoswap/ GroupMeeting/NN_Doug_2004_12_1.ppt
Introduction to Artificial Neural Networks. Nicolas Galoppo von Borries. www.cs.unc.edu/~nico/courses/ comp290-58/nn-presentation/ann-intro.ppt

17