Lapides - 3D Tractus

Lapides, P., Sharlin, E., Sousa, M. C., and Streit, L. 2006. The 3D Tractus: A Three-Dimensional Drawing Board. In Proceedings of the First IEEE international Workshop on Horizontal interactive Human-Computer Systems (January 05 - 07, 2006). TABLETOP. IEEE Computer Society, Washington, DC, 169-176. DOI= http://dx.doi.org/10.1109/TABLETOP.2006.33

Summary

I want to draw in 3D, but all I have is this tablet PC. I know, I'll put the table PC on a table that moves up and down, simulating the third dimension. I'll make a drawing program with a user interface that shows the 3D drawing from a few angles so users won't get too confused. We'll use visual cues like depth with line width. Also, we'll use perspective projection so the user knows which things are 'below' the current plane of the tablet PC.

People used it and said it was neat.

Discussion

This drawing/3D modeling approach is a little more realistic than other things we've read (Holosketch or the superellipsoid clay thing) since all you need is a tablet and one of their funky elevator things. So I'll give it kudos there.

I'm not sure how simple or intuitive it is, however, to have to move your tablet pc up and down to draw in the third dimension. I question the accuracy, especially if you're trying to line things up one on top of another, etc, since you don't really have a good idea of where things are in the Z axis. This is especially hard if what you're trying to draw exists above the current plane of the tablet PC, since their software only shows you what's below the current plane.

Neat idea, but a bit klunky. Hope someone doesn't get their legs cut off if the tractus goes berserk.

Krahnstoever - Activity Recognition with Vision and RFID

Krahnstoever, N.; Rittscher, J.; Tu, P.; Chean, K.; Tomlinson, T., "Activity Recognition using Visual Tracking and RFID," Application of Computer Vision, 2005. WACV/MOTIONS '05 Volume 1. Seventh IEEE Workshops on , vol.1, no., pp.494-500, 5-7 Jan. 2005

Summary

Person in an office or warehouse with cameras on them. Track their movements with a monte carlo model examining the image frames. Augment this with RFID tags embedded in all the objects the human can interact with. Do activity recognition by examining how the person is moving (vision) and what they are interacting with (rfid). RFID helps augment visual tracking for the purposes of activity recognition.

Discussion

So they take an existing Monte Carlo visual tracking algorithm and magically throw RFID in to the jar. They say this does better. Sort of a "duh" moment. Why do we let Civil Josh pick papers?

Bernardin - Grasping HMMs

Bernardin, K., K. Ogawara, et al. (2005). "A sensor fusion approach for recognizing continuous human grasping sequences using hidden Markov models." Robotics, IEEE Transactions on [see also Robotics and Automation, IEEE Transactions on] 21(1): 47-57.

Summary

I have a robot that I want to teach to grab things. I can teach it by example. I have 14 different types of grips that I use everyday. I'll put pressure sensors in a glove under the CyberGlove. I will grab something, and then let it go. All of this data will be fed into an HMM in the HTK speech recognition toolkit. The HMM will tell me which grasp I am making with up to about 90% accuracy.

Discussion

Pretty neat. If you know what you're grasping, you can do things like activity recognition and such. Especially helpful when you start using smart rooms and offices, etc. Maybe even Information Oriented Programming (IOP)!

I think the pressure sensors really helped augment the CyberGlove, especially since there were so many grasp categories.

Nishino - Object modelling with gestures

Nishino, H., Utsumiya, K., and Korida, K. 1998. 3D object modeling using spatial and pictographic gestures. In Proceedings of the ACM Symposium on Virtual Reality Software and Technology (Taipei, Taiwan, November 02 - 05, 1998). VRST '98. ACM, New York, NY, 51-58. DOI= http://doi.acm.org/10.1145/293701.293708

Summary

Put on special glasses to get a 3d stereoscopic image from a curved screen, and put glove/motion tracker on your hands to track them. Have some virtual clay modelled by a superellipsoid (it's mathematically easy to work with, relatively). Create a blob, deform it, mash it, pinch it, stretch it, put it in a pan, bake it up as fast as you can. Combine a bunch of blobs to make things like teapots, vases, and bigger blobs.

Discussion

Good for professional sculptors who might want to fashion something without wasting real clay. But, since clay is easy to recycle (just add water), who cares. If you're not a sculptor, are you good enough with your hands to make your blobs of junk look like things in real life? How accurate are the hands, so a noisy spike doesn't accidentally mash your teapot into oblivion?

Pretty neat idea, just not sure of its usefulness.

Campbell - Invariant Features, Tai-Chi

Campbell L W, Becker D A, Azarbayejani A, Bobick A F, and Pentland A, Invariant Features for 3-D Gesture Recognition, Proc. of FG'96 (1996) 157-162.

Summary

Look at a series of gestures occurring in Tai-Chi captured by video. Extract a lot of features about the gestures, including plain (x,y,z) coordinates, velocities for these coords, polar coordinates, polar velocities. Do each of these with and without head data (always with hand data). Plug all the different sets of features into an HMM and see which feature set does the best. Polar velocity with no head does the best at about 95% accuracy overall. Plain (x,y,z) does the worst at about 34% overall.

Discussion

Just take a bunch of features and string them all together. Perform a standard feature extraction/selection algorithm. Get a set of features that probably outperforms all your sets.

Win.

This paper isn't interesting, really, as it just shotguns a bunch of features into an HMM and see who wins.

Fail.

Wesche - FreeDrawer

Wesche, G. and Seidel, H. 2001. FreeDrawer: a free-form sketching system on the responsive workbench. In Proceedings of the ACM Symposium on Virtual Reality Software and Technology (Baniff, Alberta, Canada, November 15 - 17, 2001). VRST '01. ACM, New York, NY, 167-174. DOI= http://doi.acm.org/10.1145/505008.505041

Summary

Electronic pen you can use to draw in 3D space. To make things simpler for their algorithm, you're restricted to spline curves. You trace out the general curve with the pen and the computer calculates the parameters of the spline. You can draw curves, modify them, connect curves together to form a network, fill in surfaces between curves. You wear wonky VR goggles to see what you're drawing.

Discussion

Tradeoff between user freedom (virtual clay) and performance--they choose performance by limiting a user's drawing style (restricted to splines). They claim this is easy because it has closed form representation, is easily transferable (just the parameters of the splines and not every voxel need to be transmitted), and computationally cheap (storing every voxel for virtual clay is expensive).

They admit you need an artistic flair and a little bit of training to get used to using the splines. Well then why not just train on a CAD system? Isn't the point to offer an intuitive interface with no need for training or restrictions? Plus, if you use CAD, you don't have to use /just/ splines, can be precise and exact, and don't have to wear wonky 3D goggles.

Poddar - Gesture Spech, Weatherman

I. Poddar, Y. Sethi, E. Ozyildiz, R. Sharma. Toward Natural Gesture/Speech HCI: A Case Study of Weather Narration. Proc. Workshops on Perceptual User Interfaces, pages 1-6, November, 1998.

Summary

Three categories of gestures: pointing, area, and contour, each with three phases: preparing, making the gesture, and retraction. Use features that measure distances/angles between the face and hands and plug into an HMM. Get 50-60% accuracy on four test sequences.

Now add speech to the gesture data. Compute co-occurrences of marker words with different gestures and use the data to help the HMM classify gestures. Accuracy goes up about 10%.

Discussion

Adding speech to gesture data improves the accuracy. This is fairly obvious, and they've shown that it does a little bit. The one thing I don't like is the manual labeling of speech data.

I wish they would have done more gestures, and their accuracies weren't great. But at least it was a fusion of contextual data.