Marsh - Shape your imagination

Marsh, T.; Watt, A., "Shape your imagination: iconic gestural-based interaction," Virtual Reality Annual International Symposium, 1998. Proceedings., IEEE 1998 , vol., no., pp.122-125, 18-18 1998

Summary

Marsh and Watt present findings on a user study where they examine how gestures are used to describe objects in a non-verbal fashion. They describe how iconic gestures (those that immediately and clearly stand for something) fall into two camps: substitutive (hands match the shape or form of object) and virtual (outline or trace a picture of the shape/object).

For their study, they used 12 subjects of varying backgrounds. They had 15 shapes from two categories: primitive (circle, triangle, sphere, cube, etc) and complex (table, car, French baguette, etc). The shapes were written on index cards and presented in the same order to each user. The users then were told to describe the shapes using non-verbal communication. Of all the gestures, 75% were virtual. For the 2D shapes, 72% of people used one hand, while the 3D objects were all describe with 2 hands. For the complex shapes, iconic gestures were either replaced or accompanied by pantomimic (how the object was used) or deictic (pointing to something) gestures, rather than iconic ones. Some complex shapes were too difficult for users to express (4 for chair, 1 each for football, table, and baguette). They also discovered 2D is easier than 3D.

Discussion

I really liked this paper. While it was a little short, I think it was neat that they were able to break up the gestures that people made. This reminded me a lot of Alvarado et al's paper where they performed the user study about how people draw. I think it's especially useful to see that if we want to do anything useful with haptics, we have to enable the users to use /both/ hands.

Some things:

• How did they pick their shapes, especially the complex ones? I mean, come on, French baguette? Although, this is a really good example because it's friggin hard to mime.


• They note that most of the complex objects are too difficult to express with iconic gestures alone. That's why sign languages aren't that simple to learn. Not everything can be expressed easily with just iconic gestures. This paper was good that it pointed this out and made it clear, even though it seems obvious. It also seems to drive the need for multi-modal input for complex recognition domains.


• They remark that 3D is harder than 2D. Besides the fact that this claim is obvious and almost a bit silly to make, it does seem that there are 2D shapes that would be very difficult to express. For example: Idaho. I wonder if their comparison between 2D and 3D here is a fair one. Obviously adding another dimension to things is going to make it exponentially more difficult, but they're comparing things like circle to things like French baguette.


• Finally, who decides if a gesture is iconic or not? Isn't this shaped by experience and perception?

BibTeX

@ARTICLE{658465,
title={Shape your imagination: iconic gestural-based interaction},
author={Marsh, T. and Watt, A.},
journal={Virtual Reality Annual International Symposium, 1998. Proceedings., IEEE 1998},
year={18-18 1998},
volume={},
number={},
pages={122-125},
keywords={computer graphics, graphical user interfaces3D computer generated graphical environments, 3D spatial information, human computer interaction, iconic gestural-based interaction, iconic hand gestures, object manipulation, shape manipulation, spatial information},
doi={10.1109/VRAIS.1998.658465},
ISSN={1}, }