Abstract
Researches on signed languages still strongly dissociate linguistic issues related on phonological and phonetic aspects, and gesture studies for recognition and synthesis purposes. This paper focuses on the imbrication of motion and meaning for the analysis, synthesis and evaluation of sign language gestures. We discuss the relevance and interest of a motor theory of perception in sign language communication. According to this theory, we consider that linguistic knowledge is mapped on sensory-motor processes, and propose a methodology based on the principle of a synthesis-by-analysis approach, guided by an evaluation process that aims to validate some hypothesis and concepts of this theory. Examples from existing studies illustrate the different concepts and provide avenues for future work.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
van der Hulst, H.: The phonological structure of words. An introduction. Cambridge University Press, Cambridge (2001) (with C. Ewen)
Johnson, R.E., Liddell, S.K.: Toward a phonetic representation of signs: Sequentiality and contrast, vol. 11. Gallaudet University Press (2010)
Gibet, S., Kamp, J.F., Poirier, F.: Gesture Analysis: Invariant Laws in Movement. In: Camurri, A., Volpe, G. (eds.) GW 2003. LNCS (LNAI), vol. 2915, pp. 1–9. Springer, Heidelberg (2004)
Kennaway, R.: Experience with and Requirements for a Gesture Description Language for Synthetic Animation. In: Camurri, A., Volpe, G. (eds.) GW 2003. LNCS (LNAI), vol. 2915, pp. 300–311. Springer, Heidelberg (2004)
Prillwitz, S., Leven, R., Zienert, H., Hanke, T., Henning, J.: Hamburg Notation System for Sign Languages - An Introductory Guide. University of Hamburg Press (1989)
Gibet, S., Courty, N., Duarte, K., Le Naour, T.: The SignCom system for data-driven animation of interactive virtual signers: Methodology and evaluation. ACM Transactions on Interactive Intelligent Systems, TiiS 1(1), 6 (2011)
Courty, N., Gibet, S.: Why Is the Creation of a Virtual Signer Challenging Computer Animation? In: Boulic, R., Chrysanthou, Y., Komura, T. (eds.) MIG 2010. LNCS, vol. 6459, pp. 290–300. Springer, Heidelberg (2010)
Goldman, A.: Simulating Minds: The Philosophy, Psychology, and Neuroscience of Mindreading. Oxford University Press (2006)
Arbib, M.: From grasp to language: embodied concepts and the challenge of abstraction. Journal of Physiology 102(1-3), 4–20 (2008)
Liberman, A., Cooper, F., Shankweiler, D., Studdert-Kennedy, M.: Perception of the speech code. Psychological Review 74, 431–461 (2005)
Gibet, S., Marteau, P.F.: A self-organized model for the control, planning and learning of nonlinear multi-dimensional systems using a sensory feedback. Applied Intelligence 4(4), 337–349 (1994)
Lebourque, T., Gibet, S.: A Complete System for the Specification and the Generation of Sign Language Gestures. In: Braffort, A., Gibet, S., Teil, D., Gherbi, R., Richardson, J. (eds.) GW 1999. LNCS (LNAI), vol. 1739, pp. 227–238. Springer, Heidelberg (2000)
Gibet, S., Marteau, P.F.: Expressive gesture animation based on non parametric learning of sensory-motor models. In: CASA, 79–85 (2003)
Duarte, K., Gibet, S.: Corpus design for signing avatars. In: Proceedings of the 4th Workshop on Representation and Processing of Sign Languages: Corpora and Sign Language Technologies, Valletta, Malta (2010)
Duarte, K., Gibet, S.: Heterogeneous data sources for signed language analysis and synthesis: The signcom project. In: Proceedings of the Seventh Conference on International Language Resources and Evaluation, Valletta, Malta. European Language Resources Association (2010)
Stokoe, W.C.: Sign language structure: an outline of the communication systems of the American deaf. Journal of Deaf Studies and Deaf Education 10(1), 3–37 (2005)
Héloir, A., Gibet, S., Multon, F., Courty, N.: Captured Motion Data Processing for Real Time Synthesis of Sign Language. In: Gibet, S., Courty, N., Kamp, J.-F. (eds.) GW 2005. LNCS (LNAI), vol. 3881, pp. 168–171. Springer, Heidelberg (2006)
Gibet, S., Marteau, P.-F.: Analysis of Human Motion, Based on the Reduction of Multidimensional Captured Data – Application to Hand Gesture Compression, Segmentation and Synthesis. In: Perales, F.J., Fisher, R.B. (eds.) AMDO 2008. LNCS, vol. 5098, pp. 72–81. Springer, Heidelberg (2008)
Héloir, A., Gibet, S.: A Qualitative and Quantitative Characterisation of Style in Sign Language Gestures. In: Sales Dias, M., Gibet, S., Wanderley, M.M., Bastos, R. (eds.) GW 2007. LNCS (LNAI), vol. 5085, pp. 122–133. Springer, Heidelberg (2009)
Duarte, K., Gibet, S.: Reading between the signs: How are transitions built in signed languages? In: Proceedings of the 10th International Conference on Theoretical Issues in Sign Language Research, West Lafayette, Indiana USA (2010)
Attina, V., Beautemps, D., Cathiard, M.A., Odisio, M.: A pilot study of temporal organization in cued speech production of french syllables: rules for a cued speech synthesizer. Speech Communication 44(1-4), 197–214 (2004)
Awad, C., Courty, N., Duarte, K., Le Naour, T., Gibet, S.: A Combined Semantic and Motion Capture Database for Real-Time Sign Language Synthesis. In: Ruttkay, Z., Kipp, M., Nijholt, A., Vilhjálmsson, H.H. (eds.) IVA 2009. LNCS, vol. 5773, pp. 432–438. Springer, Heidelberg (2009)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Gibet, S., Marteau, PF., Duarte, K. (2012). Toward a Motor Theory of Sign Language Perception. In: Efthimiou, E., Kouroupetroglou, G., Fotinea, SE. (eds) Gesture and Sign Language in Human-Computer Interaction and Embodied Communication. GW 2011. Lecture Notes in Computer Science(), vol 7206. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-34182-3_15
Download citation
DOI: https://doi.org/10.1007/978-3-642-34182-3_15
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-34181-6
Online ISBN: 978-3-642-34182-3
eBook Packages: Computer ScienceComputer Science (R0)