*Perceptron Simulation Homework *
*(Learning and Memory PSYCH)*
Perceptron is a linear classifier devised by Frank Rosenblatt. It consists of input and output vectors, a weight vector, an activation function and some rules that define the steps of operation. The simplest example of Perceptron is binary classification where its weight vector can be adjusted to solve classification problems.
It is a form of supervised learning and requires a set of inputs and outputs (training set) to learn a mapping function. The amount and quality of the training set determines the final performance of the perceptron.
The figure above can be translated into the following operational steps that could be implemented in any programming language or even in simple spreadsheet programs.
Step 1: Prepare your training set.
Step 2: Set your initial weight to be a small nonzero value between 0 and 1.
Step 3: Compute weighted sum of inputs. In other words, calculate w0x0 + w1x 1 + w2x2 + ….. wnxn. Here w0x0 is a bias (with w0 = θ and x0 is always 1).
Setp 4: Calculate the output by computing the activation function *f(**W**·* *X**).* Output Y= 1 if W·X > 0*, *otherwise Y= -1.
Step 5: Compare the calculated output and the teaching output. Adjust the weight according to the following formula.
w*i *(new) = w*i *(old) + *α *(T-Y) x*i*
Here *α* is a learning parameter and takes the value between 0 and 1 (the larger the faster the learning).
Repeat Step 3-5 until Y matches the training set.
A computation example: Simulating logical operation AND (*α* = 0.1; *θ =* 0.5)
X0
X1
X2
Output (T)
W0
W1
W2
X·W
Y
α(T-Y)X0
α(T-Y)X1
α(T-Y)X2
alpha
1
-1
-1
-1
0.5
0.1
0.1
0.3
1
-0.2
0.2
0.2
0.1
1
1
-1
-1
0.3
0.3
0.3
0.3
1
-0.2
-0.2
0.2
0.1
1
-1
1
-1
0.1
0.1
0.5
0.5
1
-0.2
0.2
-0.2
0.1
1
1
1
1
-0.1
0.3
0.3
0.5
1
0
0
0
0.1
1
-1
-1
-1
-0.1
0.3
0.3
-0.7
-1
0
0
0
0.1
1
1
-1
-1
-0.1
0.3
0.3
-0.1
-1
0
0
0
0.1
1
-1
1
-1
-0.1
0.3
0.3
-0.1
-1
0
0
0
0.1
1
1
1
1
-0.1
0.3
0.3
0.5
1
0
0
0
0.1
1
-1
-1
-1
-0.1
0.3
0.3
-0.7
-1
0
0
0
0.1
1
1
-1
-1
-0.1
0.3
0.3
-0.1
-1
0
0
0
0.1
1
-1
1
-1
-0.1
0.3
0.3
-0.1
-1
0
0
0
0.1
1
1
1
1
-0.1
0.3
0.3
0.5
1
0
0
0
0.1
*Question 1:* Simulate logic operations OR, NAND, NOR, XOR. Did you solve all of the logic operations? Why or why not?* Note. Use Different α and **θ **from the example above with two decimal places. Generate a random initial weight with two decimal places. For example, W1 = 0.99. With these values, almost everyone should have different end results.*
Input
AND
OR
NAND
NOR
XOR
-1
-1
-1
-1
1
1
-1
1
-1
-1
1
1
-1
1
-1
1
-1
1
1
-1
1
1
1
1
1
-1
-1
-1
*Question 2:* Create a perceptron with more than 2 input nodes and more than 1 output nodes. Generate your own training set and train your perceptron network until no error. Come up with a way of using the perceptron for a practical purpose such as the one shown below.
*Optional Pattern Recognition Problem (extra 2 points)*
Train a Perceptron network to recognize the number 0, 1 ,2, 3 … 9 in a 5 by 3 matrix. You will need 15 input nodes and 10 output nodes (representing numbers 0-9). Upon finishing training, try to recognize the test pattern. You can still use Exel although it might take a long laborious hours to complete. Alternatively, use a programming language such as Python.
Training pattern
Input = [1 0 1 1 1 1 1 1 1 1;
1 1 1 1 0 1 1 1 1 1;
1 0 1 1 1 1 1 1 1 1;
1 1 0 0 1 1 1 0 1 1;
0 1 0 0 0 0 0 0 0 0;
1 0 1 1 1 0 0 1 1 1;
1 0 1 1 1 1 1 0 1 1;
0 1 1 1 1 1 1 0 1 1;
1 0 1 1 1 1 1 1 1 1;
1 0 1 0 0 0 1 0 1 0;
0 1 0 0 0 0 0 0 0 0;
1 0 0 1 1 1 1 1 1 1;
1 1 1 1 0 1 1 0 1 1;
1 1 1 1 0 1 1 0 1 1;
1 1 1 1 1 1 1 1 1 1;]
Output = [ 1 0 0 0 0 0 0 0 0 0;
0 1 0 0 0 0 0 0 0 0;
0 0 1 0 0 0 0 0 0 0;
0 0 0 1 0 0 0 0 0 0;
0 0 0 0 1 0 0 0 0 0;
0 0 0 0 0 1 0 0 0 0;
0 0 0 0 0 0 1 0 0 0;
0 0 0 0 0 0 0 1 0 0;
0 0 0 0 0 0 0 0 1 0;
0 0 0 0 0 0 0 0 0 1;]
Test_pattern = [ 1 0 1 1 1;
1 1 1 1 0;
1 1 1 1 1;
1 1 0 0 1;
0 1 0 0 1;
1 1 1 1 1;
1 0 1 1 1;
0 1 1 1 1;
1 0 1 1 1;
1 1 1 0 0;
0 1 0 1 0;
1 0 0 1 1;
1 1 1 1 1;
1 1 1 1 0;
1 1 1 1 1;]
*What needs to be submitted: Hard copy of your simulation results *
Your results should include (Question 1 and 2)
1. Initial and final weight sets
2. Changes in weights on every trial
For the optional question,
1. Initial and final weight sets
2. Results for the test pattern
3. Changes in weights using a graphical notation (e.g., # of errors on every nth trial).
