GestureApp is made by Konstantin Boukreev

GestureApp recognises mouse gestures (not 2d-images!)

Download executable (158 Kb)
Download source (101 Kb)

The error graph after training

Mouse gesture application

This application can regognise gestures you make with your mouse. For a list of gestures, see on your right.

For training, the application uses a multilayer perceptron and standard back-propagation algorithm. The main problem was in the representation of an input data for neural network.

The best result Konstantin found was in the transformation of a mouse path into a vector of cosines and sines.

For example:

   path {170:82 172:83 175:85 177:86 ...}
   transformed into
   vector {0.45 0.55 0.45 0.71 ... 0.89 0.83 0.89 0.71 ...}

Recognition algorithm

1. record a mouse path
2. smooth a path to a base points
3. transform points to angles' vector
4. compute sines and cosines
5. pass values (cosines and sines) to network's inputs
6. apply softmax function on an output network vector
7. find and verify a winner

Neural network architecture

• input layers : 32 sinapses
• hidden layer : 32 neurons
• output layer : 29 axons (one for each gesture)
• fully connected layers
• transfer function : log-sigmoid
• incremental training algorithm, standard back-propagation method
• momentum, variable learning rate (slowly reduced)
• input noise