Contents

1. Neural Network Background 2. Implementation 3. Testing and Result

 

1. Neural Network Background

1.1 What is Neural Network

The idea of Neural Network came from the neural structure of the human brain. For example, when a young boy sees and remembers an image of a tiger. His brain collects the information of the image and once he has a chance to see a real tiger in a zoo, he can recognize the image and specify this type of animal correctly even though he has never seen a real tiger before. This amazing brain network composes of small cells called "neurons" that connected together as a network.

A biological neuron works by receiving electric signals from other neurons through connections called synapses. If the combination of these signals is higher than a threshold value, it will send a signal on to other neurons connected to it. There are approximately one hundred billion neurons each connected to as many as one thousand others the human brain. When a human learns, the threshold value in each neuron is adjusted.

Neural networks are models of biological structures. A model neuron consists of multiple inputs and a single out put. Each input is modified a weight, which multiplies with the input value. The neuron will combine these weighted inputs and use these to determine its output.

A Model Neuron

Artificial Neural Network (ANN) or simply called Neural Network is good for recognizing the pattern. For example, hand writing recognition. First, we train the network to recognize character "A". Assume that "A" is translated to binary code "1000". For input, let's say we have 16 inputs. When we train the network, we feed many hand-written "A" that translated to 16-bit binary (e.g. 1010101001001000) to the network and tell it the output as "1000". This process is called "Training Process". While training, each neuron in the network will be adjusted its weight value. Until the network can give us the correct answer for all "A" that we feed, the training process is done. When there is a new hand-written "A" feed into the network, if it looks similar to one of the trained "A", the network will give the correct output.

There are many kinds of algorithm in neural networks. The most popular one is BackPropagation. For my project, I applied this concept to use it for face identification application program.

1.2 BackPropagation

3-level BackPropagation network

Figure 4. The Sigmoid or Squashing Function

Now I will show the algorithm of this network and then I will give an example how this network operates.

1.3 An Example of BackPropagation

This example has 2 input neurons, 2 hidden neurons and 1 output neuron. Suppose the next training example has inputs and desired output of
u1 = 1, u2 = 0, u5 = 1.
Compute Forward pass:

S1 = -, f(S1) = 1; because it's the input neuron f(S1) = u1 = 1 S2 = -, f(S2) = 0 ;because it's the input neuron f(S2) = u2 = 0 S3 = (w3,0u0) + (w3,1u1) + (w3,2u2) ; u0 always has +1 activation = (1*1) + (1*3) + (4*0) = 4 f(S3) = f(4) = 0.9820 S4 = (w4,0u0) + (w4,1u1) + (w4,2u2) = (-6*1) + (6*1) + (5*0) = 0 f(S4) = f(0) = 0.5000 S5 = (w5,0u0) + (w5,3u3) + (w5,4u4) = (-3.92*1) + (2*0.9820) + (4*0.5000) = 0.044 f(S5) = f(0.044) = 0.5110

Copyright 2003. All Rights Reserved.