Introduction

This applet demonstrates a simple form of supervised learning entitled the perceptron learning algorithm. Using this applet, you can train the perceptron to act as a binary logic unit. It can compute or approximate most 2-input Boolean functions. However, a problem arises when trying to train the perceptron on the XOR (or XNOR) function. The applet provides a "work-around" for this problem by introducing an extra input.

Theory

Single-Layer Perceptron Neural Networks
A single-layer perceptron network consists of 1 or more artificial neurons in parallel. Each neuron in the single layer provides one network output, and is usually connected to all of the external (or environmental) inputs. The applet shown below is an example of a single-neuron, single-layer perceptron network.

Single-layer perceptron networks have many limitations and are not computationally complete. As mentioned in the introduction, 2 input networks can not approximate the XOR (or XNOR) functions. Of the (2²)ⁿ or 16 possible functions, a 2 input perceptron can only perform 14 functions. As the number of inputs, n, increases, the number of functions that can be computed decreases rapidly.

Perceptron Learning Algorithm

The perceptron learning algorithm was originally developed by Frank Rosenblatt in the late 1950s. The perceptron learning algorithm is as follows:

1. Initialize the weights and threshold to small random numbers.
2. Present a vector to the neuron inputs and calculate the output.
3. Update the weights according to: W_j(t+1) = W_j(t)+η(d-y)x_j
   • d is the desired output
   • t is the iteration number
   • η is the gain or step size, where 0.0 < n < 1.0
4. Repeat steps 2 and 3 until:
   • the iteration error is less than a user-specified error threshold or
   • a predetermined number of iterations have been completed.

Notice that learning only occurs when an error is made, otherwise the weights are left unchanged. During training, it is often useful to measure the performance of the network as it attempts to find the optimal weight set. A common error measure or cost function used is sum-squared error. It is computed over all of the input vector / output vector pairs in the training set and is given by the equation below:

        p
E = 0,5 Σ yi-di2
        i=1

where p is the number of input/output vector pairs in the training set.

Instructions

To Train The Perceptron:

1. Select the desired Boolean function in the truth table.
2. Select the desired activation function by clicking the activation function image. You can also change some function parameters by holding down the <Shift> key and then clicking the image with your mouse.
3. Adjust the training parameters as desired. Legal values are as follows:
   • Learning Rate: 0.0 to 1.0
   • Iterations: 1 to 10000 (I wouldn't want to try this)
   • Error Threshold: 0.0 to 0.5
4. Click the Train button to begin a normal training session OR
5. Click the Step button repeatedly to single-step through the training session.

During Training:

1. To stop a training session (normal or single-step) in progress, click the Stop button.
2. A few notes:
   • The progress of the training session is displayed in the progress and status bars.
   • The error for the current input vector is displayed in the Current-Error text field.
   • The sum-squared error over all the input vectors is displayed in the Sum-Squared Error text-field
   • The perceptron's ability to classify the inputs into two classes (0 and 1) is shown in a graph in the top-right corner of the applet. Currently, this only works for the 2D graph.

To Test The Perceptron:

1. After training Click the Test button repeatedly to cycle through all four input vectors.
2. Some notes:
   • If the neuron output is correct (within the error threshold specified), the neuron output text-field is painted green otherwise it is red.
   • The current error and sum-squared error are updated and displayed in their text fields.

To Solve the Exclusive-OR (XOR) Problem:

1. Click the Show XOR Solution check box.
2. Set the truth table to the XOR function
3. Click the Train or Step button to start a training session.
4. Once the training is complete, click the Test button to check the results.

Source Code

Please feel free to read, copy, and/or modify any portion of the source files for non-commercial purposes. If you do decide to use any of this source code in your own work, please send a message to Fred at fcorbett@ee.umanitoba.ca. Comments, questions, and bug reports would also be appreciated.

 

Package ANNs	3D Routines	Applet Source
ActivationFunction.java ArtificialNeuron.java NeuralNetwork.java SingleLayerPerceptron.java	Matrix3D.java Model3D.java SphereModel.java WireFrameModel.java PatternSpaceCanvas3D.java	SimpleNeuronApplet.java PatternSpaceCanvas.java TrainingPanel.java TruthTablePanel.java PerceptronApplet.java

Note: The 3D routines employ modified versions of the applets XYZApp and ThreeD distributed as samples in the Sun Microsystems Java Development Kit 1.02. This source code is Copyright (c) 1994-1996 Sun Microsystems, Inc. All Rights Reserved.

---

This applet was made by Fred Corbett. For information and questions about the applet, please contact him. The applet was kindly mailed to me by him. It originally comes from http://home.cc.umanitoba.ca/~umcorbe9/.