Alexandra Stefan
ABSTRACT
This paper describes work towards designing a computer vision system for helping users look up the meaning of a sign. Sign lookup is treated as a video database retrieval problem. A video database is utilized that contains one or more video examples for each sign, for a large number of signs (close to 1000 in our current experiments). The emphasis of this paper is on evaluating the trade-offs between a non-automated approach, where the user manually specifies hand locations in the input video, and a fully automated approach, where hand locations are determined using a computer vision module, thus introducing inaccuracies into the sign retrieval process. We experimentally evaluate both approaches and present their respective advantages and disadvantages.
- J. Alon, V. Athitsos, Q. Yuan, and S. Sclaroff. Simultaneous localization and recognition of dynamic hand gestures. In IEEE Motion Workshop, pages 254--260, 2005. Google Scholar
Digital Library
- V. Athitsos, C. Neidle, S. Sclaroff, J. Nash, A. Stefan, Q. Yuan, and A. Thangali. The American Sign Language lexicon video dataset. In IEEE Workshop on Computer Vision and Pattern Recognition for Human Communicative Behavior Analysis (CVPR4HB), 2008.Google ScholarCross Ref
- B. Bauer and K. Kraiss. Towards an automatic sign language recognition system using subunits. In Gesture Workshop, pages 64--75, 2001. Google ScholarDigital Library
- M. Black and A. Jepson. Recognizing temporal trajectories using the condensation algorithm. In Face and Gesture Recognition, pages 16--21, 1998. Google ScholarDigital Library
- F. Chen, C. Fu, and C. Huang. Hand gesture recognition using a real-time tracking method and Hidden Markov Models. Image and Video Computing, 21(8):745--758, August 2003.Google ScholarCross Ref
- A. Corradini. Dynamic time warping for off-line recognition of a small gesture vocabulary. In Recognition, Analysis and Tracking of Faces and Gestures in Real-time Systems (RATFG-RTS), pages 82--89, 2001. Google ScholarDigital Library
- Y. Cui and J. Weng. Appearance-based hand sign recognition from intensity image sequences. Computer Vision and Image Understanding, 78(2):157--176, 2000. Google ScholarDigital Library
- R. Cutler and M. Turk. View-based interpretation of real-time optical flow for gesture recognition. In Face and Gesture Recognition, pages 416--421, 1998. Google ScholarDigital Library
- T. Darrell, I. Essa, and A. Pentland. Task-specific gesture analysis in real-time using interpolated views. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 18(12):1236--1242, 1996. Google ScholarDigital Library
- J. Deng and H.-T. Tsui. A PCA/MDA scheme for hand posture recognition. In Automatic Face and Gesture Recognition, pages 294--299, 2002. Google ScholarDigital Library
- K. Fujimura and X. Liu. Sign recognition using depth image streams. In Automatic Face and Gesture Recognition, pages 381--386, 2006. Google ScholarDigital Library
- W. Gao, G. Fang, D. Zhao, and Y. Chen. Transition movement models for large vocabulary continuous sign language recognition. In Automatic Face and Gesture Recognition, pages 553--558, 2004. Google ScholarDigital Library
- T. Kadir, R. Bowden, E. Ong, and A. Zisserman. Minimal training, large lexicon, unconstrained sign language recognition. In British Machine Vision Conference (BMVC), volume 2, pages 939--948, 2004.Google ScholarCross Ref
- E. Keogh. Exact indexing of dynamic time warping. In International Conference on Very Large Data Bases, pages 406--417, 2002. Google ScholarDigital Library
- J. B. Kruskall and M. Liberman. The symmetric time warping algorithm: From continuous to discrete. In Time Warps. Addison-Wesley, 1983.Google Scholar
- J. Ma, W. Gao, J. Wu, and C. Wang. A continuous Chinese Sign Language recognition system. In Automatic Face and Gesture Recognition, pages 428--433, 2000. Google ScholarDigital Library
- J. Martin, V. Devin, and J. Crowley. Active hand tracking. In Face and Gesture Recognition, pages 573--578, 1998. Google ScholarDigital Library
- S. C. W. Ong and S. Ranganath. Automatic sign language analysis: A survey and the future beyond lexical meaning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(6):873--891, 2005. Google ScholarDigital Library
- V. Pavlovic, R. Sharma, and T. Huang. Visual interpretation of hand gestures for human-computer interaction: A review. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 19(7), 1997. Google ScholarDigital Library
- L. Rabiner. A tutorial on hidden markov models and selected applications in speech recognition. In Proceedings of the IEEE, volume 77:2, 1989.Google ScholarCross Ref
- L. Rabiner and B. Juang. Fundamentals of speech recognition. Prentice Hall, 1993. Google ScholarDigital Library
- H. Rowley, S. Baluja, and T. Kanade. Rotation invariant neural network-based face detection. In CVPR, pages 38--44, 1998. Google ScholarDigital Library
- H. Sagawa and M. Takeuchi. A method for recognizing a sequence of sign language words represented in a Japanese Sign Language sentence. In Automatic Face and Gesture Recognition, pages 434--439, 2000. Google ScholarDigital Library
- H. Sakoe and S. Chiba. Dynamic programming algorithm optimization for spoken word recognition. In IEEE Transactions on Acoustics, Speech, and Signal Processing, volume 34(1), pages 43--49, 1978.Google ScholarCross Ref
- T. Starner and A. Pentland. Real-time American Sign Language recognition using desk and wearable computer based video. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(12):1371--1375, 1998. Google ScholarDigital Library
- C. Valli, editor. The Gallaudet Dictionary of American Sign Language. Gallaudet U. Press, Washington, DC, 2006.Google Scholar
- P. Viola and M. Jones. Rapid object detection using a boosted cascade of simple features. In IEEE Conference on Computer Vision and Pattern Recognition, volume 1, pages 511--518, 2001.Google ScholarCross Ref
- C. Vogler and D. N. Metaxas. Parallel hidden markov models for american sign language recognition. In IEEE International Conference on Computer Vision (ICCV), pages 116--122, 1999.Google ScholarCross Ref
- C. Vogler and D. N. Metaxas. Handshapes and movements: Multiple-channel american sign language recognition. In Gesture Workshop, pages 247--258, 2003.Google Scholar
- C. Wang, S. Shan, and W. Gao. An approach based on phonemes to large vocabulary Chinese Sign Language recognition. In Automatic Face and Gesture Recognition, pages 411--416, 2002. Google ScholarDigital Library
- M. Yang and N. Ahuja. Recognizing hand gesture using motion trajectories. In IEEE Conference on Computer Vision and Pattern Recognition, volume 1, pages 466--472, 1999.Google Scholar
- G. Yao, H. Yao, X. Liu, and F. Jiang. Real time large vocabulary continuous sign language recognition based on OP/Viterbi algorithm. In International Conference on Pattern Recognition, volume 3, pages 312--315, 2006. Google ScholarDigital Library
Index Terms
- Towards automated large vocabulary gesture search
Recommendations
A system for large vocabulary sign search
ECCV'10: Proceedings of the 11th European conference on Trends and Topics in Computer Vision - Volume Part IA method is presented to help users look up the meaning of an unknown sign from American Sign Language (ASL). The user submits a video of the unknown sign as a query, and the system retrieves the most similar signs from a database of sign videos. The ...
TransASL: A Smart Glass based Comprehensive ASL Recognizer in Daily Life
IUI '23: Proceedings of the 28th International Conference on Intelligent User InterfacesSign language is a primary language used by deaf and hard-of-hearing (DHH) communities. However, existing sign language translation solutions primarily focus on recognizing manual markers. The non-manual markers, such as negative head shaking, question ...
Multi-scenario gesture recognition using Kinect
CGAMES '12: Proceedings of the 2012 17th International Conference on Computer Games: AI, Animation, Mobile, Interactive Multimedia, Educational & Serious Games (CGAMES)Hand gesture recognition (HGR) is an important research topic because some situations require silent communication with sign languages. Computational HGR systems assist silent communication, and help people learn a sign language. In this article, a ...
Comments