Gesture Recognition

(Byte The Chip)

Understanding human motions can be posed as a pattern recognition problem. Humans express time varying motion patterns (gestures), such as a wave, in order to convey a message to a recipient. If a computer can detect and distinguish these human motion patterns, the desired message can be reconstructed, and the computer can respond appropriately. Constellation method and Hidden Markov Model (HMM) are some of the different methods of gesture recognition. Both methods have their special features and used according to appropriate situations.

What is Gesture Recognition?

Interfacing with computer using gestures of human body, typically hand movements is known as gesture recognition by computers. For example, a person clapping his hands together in front of a camera can produce the sound of cymbals being crashed together when the gesture is fed through a computer.

Basic Hand Gestures of American Sign Language

One way gesture recognition is being used is to help the physically impaired to interact with computers, such as interpreting sign language. The technology also has the potential to change the way users interact with computers by eliminating input devices such as joysticks, mice and keyboards and allowing the unencumbered body to give signals to the computer through gestures such as finger pointing. Gesture recognition technology also can be used to read facial and speech expressions (i.e., lip reading), and eye movements.

A basic Block Diagram for Gesture Recognition Systems

Gesture Recognition Techniques:

1)  The Constellation Model:

The constellation model was developed by Markus Weber for distinguishing between types of objects, e.g. faces vs. cars in 2-D images. When trained on multiple images of a certain object, for example on human faces. Weber’s algorithm learned a constellation model for the object type a model based on the arrangement of important features, such as the eyes and nose for human faces, and how they co-vary spatially. For any test image, the algorithm would extract the features from the image and compute a probabilistic score using all its learnt models. The highest scoring model was declared a match.

Gesture Recognition using the constellation method:

The basic approach is same as that of Weber. Effectively, each gesture sample datum is processed and its features are extracted. The features can then be used to learn a new model (in conjunction with many more samples of the same kind) or can be classified as one of the models the system has learnt. The main features of this technique are as follows.
a.Unlike Weber, our task is to recognize different movements of the same object, the hand. We consider the motion of the silhouette of the hand ignoring fine details like knuckles etc.
b.The motion of the hand can be in any of the 3 dimensions. Our approach is far more expanded than that of Weber.
c.Since we don’t know which part of the hand will generate the gesture, we develop clusters for various detections with each cluster referring to a particular distinct feature. These features are used to compare the learnt gestures of training period and test input by assigning a probabilistic score. The gesture with closest score is the one recognized.

      2)     The Hidden Markov Model

Hidden Markov Models (HMM) are one of the most common forms of probabilistic graphical models. They are used to model time-invariant models that have both an underlying mechanism (hidden states) and an observable consequence. They have been extremely successful in language modelling and speech recognition systems and are still the most widely used technique in these domains. HMM is basically a probabilistic method. Such types of methods are required in signal processing. There have been other fields of interest that HMM are utilized for computational biology, bio-medical signal interpretation and also for image classification, segmentation and de-noising.

What is HMM?

An HMM is a Markov chain (a mathematical system that undergoes transitions from one state to another, between a finite or countable number of possible states. It is a random process usually characterized as memory-less: the next state depends only on the current state and not on the sequence of events that preceded it) where each state generates an observation.

HMM is a finite-state machine that changes state once every discrete time. At discrete instants of time, the process is assumed to be in some state and an observation is generated by the random function corresponding to current state. The underlying Markov chain then changes states according to its transition probability matrix (a matrix used to describe the transitions of a Markov chain. Each of its entries is a non-negative real number representing a probability). Only the observation of the random function is known, but the underlying Markov chain is hidden, hence, it is called hidden Markov model. Although, HMM is statistically strong and computationally efficient, it requires large amount of training data and many calculations.

Future:

In the real world, applications abound. Gesture recognition systems can be used for much more intuitive, rapid interaction with machines. Some versions, especially electronic glove-based ones, are already being used in virtual environments, such as 3-D modeling of molecules. Gesture recognizer(s) can provide a great interface for the vocally challenged by responding efficiently to sign language. Furthermore, we might even be able to create a system that responds to voice and gesture cues, making it even more effective at interacting with humans. 

- The Processor