Hernandez-Rebollar - Accelerometers and Decision Tree for ASL

Hernandez-Rebollar, J.L.; Lindeman, R.W.; Kyriakopoulos, N., "A multi-class pattern recognition system for practical finger spelling translation," Multimodal Interfaces, 2002. Proceedings. Fourth IEEE International Conference on , vol., no., pp. 185-190, 2002

Summary

Rebollar et al. present a new algorithm for classification of ASL finger spelling letters (J and Z, the only letters that move, are statically signed at the ending posture of the gesture). They create their own glove so they don't have to sink a lot of money in to the expensive options currently available. Their gloves uses 5 accelerometers, one per finger, that measure in two axes. The y axis is aligned to point at the tip of each finger, and measures flexion and pitch. The x axis gives an idea about roll, yaw, and abduction.

They take the ten measurement values (two axes per finger, 5 fingers) and convert them to a 3D vector. The first dim is the sum of the x-axis values, the second is the y-axis, and the third is the y-axis value of the index finger, which they claim is adequate for describing the bentness of the palm.

The 3D vector is fed into a decision tree. For 21/26 letters, 5 signers doing 10 reps of each letter, they get 100% accuracy. For the I and Y, they get 96%. For U,V, and R, the accuracy is 90%, 78%, and 96%.

Discussion

Again, another paper where they sum all their values to get a global picture. This is a horrible idea as fingers will mask each other. At least sum the square of the values, so you can see if some are really high compared to others. Or, better, yet, use the 10 dimensions for the decision tree. It's really not that hard.

It was nice to see something besides an HMM, and they do get pretty good results. However, I'm ready for J and Z to move.

I also like their hardware approach. Seems simple and a lot less expensive than dropping 10-30K on a CyberGlove.

BibTeX

@ARTICLE{rebollar2002multiClassFingerSpelling,
title={A multi-class pattern recognition system for practical finger spelling translation},
author={Hernandez-Rebollar, J.L. and Lindeman, R.W. and Kyriakopoulos, N.},
journal={Multimodal Interfaces, 2002. Proceedings. Fourth IEEE International Conference on},
year={2002},
volume={},
number={},
pages={ 185-190},
doi={10.1109/ICMI.2002.1166990},
}