Abstract
We investigate the question of how co-speech iconic gestures are used to convey visuo-spatial information in an interdisciplinary way, starting with a corpus-based empirical and theoretical perspective on how a typology of gesture form and a partial ontology of gesture meaning are related. Results provide the basis for a computational modeling approach that allows us to simulate the production of speaker-specific gesture forms to be realized with virtual agents. An evaluation of our simulation results and our methodology shows that the model is able to successfully approximate human gestural behavior use of iconic gestures, and moreover, that gestural behavior can improve how humans rate a virtual agent in terms of eloquence, competence, human-likeness, or likeability.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bavelas, J., Gerwing, J., Sutton, C., Prevost, D.: Gesturing on the telephone: Independent effects of dialogue and visibility. Journal of Memory and Language 58, 495–520 (2008)
Beattie, G., Shovelton, H.: An experimental investigation of the role of different types of iconic gesture in communication: A semantic feature approach. Gesture 1, 129–149 (2001)
Bergmann, K., Kopp, S.: GNetIc – Using Bayesian Decision Networks for Iconic Gesture Generation. In: Ruttkay, Z., Kipp, M., Nijholt, A., Vilhjálmsson, H.H. (eds.) IVA 2009. LNCS, vol. 5773, pp. 76–89. Springer, Heidelberg (2009)
Bergmann, K., Kopp, S., Eyssel, F.: Individualized gesturing outperforms average gesturing–evaluating gesture production in virtual humans. In: Proceedings of IVA 2010, pp. 104–117. Springer, Heidelberg (2010)
Bergmann, K., Rieser, H., Kopp, S.: Regulating dialogue with gestures—towards an empirically grounded simulation with virtual agents. In: Proceedings of SigDial 2011. ACL, Portland (2011)
Hahn, F., Rieser, H.: Explaining speech gesture alignment in mm dialogue using gesture typology. In: Lupowski, P., Purver, M. (eds.) Proceedings of SemDial, pp. 99–111. Polish Society for Cognitive Science (2010)
Holler, J., Beattie, G.: How iconic gestures and speech interact in the representation of meaning: Are both aspects really integral to the process? Semiotica 146/1, 81–116 (2003)
Kendon, A.: Gesture—Visible Action as Utterance. Cambridge Univ. Press (2004)
Lücking, A., Bergmann, K., Hahn, F., Kopp, S., Rieser, H.: The Bielefeld speech and gesture alignment corpus (SaGA). In: Proceedings of the LREC 2010 Workshop on Multimodal Corpora (2010)
Rieser, H.: On Factoring Out a Gesture Typology from the Bielefeld Speech-and-Gesture-Alignment Corpus (SAGA). In: Kopp, S., Wachsmuth, I. (eds.) GW 2009. LNCS (LNAI), vol. 5934, pp. 47–60. Springer, Heidelberg (2010)
Rieser, H.: How to disagree on a church-window’s shape using gesture. In: Hölker, K., Marello, C. (eds.) Dimensionen der Analyse von Texten und Diskursen, pp. 231–247. LIT Verlag, Münster (2011)
Sowa, T., Wachsmuth, I.: A model for the representation and processing of shape in coverbal iconic gestures. In: Proceedings of KogWis 2005, pp. 183–188 (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Rieser, H., Bergmann, K., Kopp, S. (2012). How Do Iconic Gestures Convey Visuo-Spatial Information? Bringing Together Empirical, Theoretical, and Simulation Studies. In: Efthimiou, E., Kouroupetroglou, G., Fotinea, SE. (eds) Gesture and Sign Language in Human-Computer Interaction and Embodied Communication. GW 2011. Lecture Notes in Computer Science(), vol 7206. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-34182-3_13
Download citation
DOI: https://doi.org/10.1007/978-3-642-34182-3_13
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-34181-6
Online ISBN: 978-3-642-34182-3
eBook Packages: Computer ScienceComputer Science (R0)