EDWARD TSE
SAUL GREENBERG
CHIA SHEN
Abstract
There is a large disparity between the rich physical interfaces of co-located arcade games and the generic input devices seen in most home console systems. In this article we argue that a digital table is a conducive form factor for general co-located home gaming as it affords: (a) seating in collaboratively relevant positions that give all equal opportunity to reach into the surface and share a common view; (b) rich whole-handed gesture input usually seen only when handling physical objects; (c) the ability to monitor how others use space and access objects on the surface; and (d) the ability to communicate with each other and interact on top of the surface via gestures and verbal utterance. Our thesis is that multimodal gesture and speech input benefits collaborative interaction over such a digital table. To investigate this thesis, we designed a multimodal, multiplayer gaming environment that allows players to interact directly atop a digital table via speech and rich whole-hand gestures. We transform two commercial single-player computer games, representing a strategy and simulation game genre, to work within this setting.
- BOLT, R.A. 1980. Put-that-there: Voice and gesture at the graphics interface. In Proceedings of the ACM Conference on Computer Graphics and Interactive Techniques (Seattle, WA), ACM, New York, 262--270. Google Scholar
Digital Library
- CLARK, H. 1996. Using Language. Cambridge University Press.Google Scholar
- COHEN, P.R., COULSTON, R., AND KROUT, K. 2002. Multimodal interaction during multiparty dialogues: Initial results. In Proceedings of the IEEE International Conference on Multimodal Interfaces, IEEE, Piscataway, NJ, 448--452. Google ScholarDigital Library
- COHEN, P.R, 2000. Speech can't do everything: A case for multimodal systems. Speech Technol. Mag. 5, 4.Google Scholar
- COHEN, P.R., JOHNSTON, M., MCGEE, D., OVIATT, S., PITTMAN, J., SMITH, I., CHEN, L., AND CLOW, J. 1997. QuickSet: Multimodal interaction for distributed applications. In Proceedings of the ACM Multimedia Conference, ACM, New York, 31--40. Google ScholarDigital Library
- DIETZ, P.H. AND LEIGH, D.L. 2001. DiamondTouch: A multi-user touch technology. In Proceedings of the ACM Symposium on User Interface Software and Technology (UIST), ACM, New York, 219--226. Google ScholarDigital Library
- GREEBBERG, S. 1999. Designing computers as public artifacts. Int. J. Design Computing: Special Issue on Design Computing on the Net (Nov.30--Dec.3). University of Sydney.Google Scholar
- GREENBERG, S. 1991. Personalizable groupware: Accommodating individual roles and group differences. In Proceedings of the ECSCW Conference,17--32. Google ScholarDigital Library
- GREENBERG, S. 1990. Sharing views and interactions with single-user applications. In Proceedings of the ACM COIS Conference, ACM, New York, 227--237. Google ScholarDigital Library
- GUTWIN, C. AND GREENBERG, S. 2004. The importance of awareness for team cognition in distributed collaboration. In Team Cognition: Understanding the Factors that Drive Process and Performance, E. Salas and S. Fiore (eds.), APA Press, 177--201.Google Scholar
- HEATH, C.C. AND LUFF, P. 1991. Collaborative activity and technological design: Task coordination in London Underground control rooms. In Proceedings of the ECSCW Conference, 65--80. Google ScholarDigital Library
- MAAGERKURTH, C., MEMISOGLU, M., ENGELKE, T., AND STREITZ, N. 2004. Towards the next generation of tabletop gaming experiences. In Proceedings of the Graphics Interface Conference, 73--80. Google ScholarDigital Library
- OVIATT, S. 1999. Ten myths of multimodal interaction, Commun. ACM 42, 11 (Nov.), 74-81. Google ScholarDigital Library
- OVIATT, S. 1997. .Multimodal interactive maps: Designing for human performance. Human-Computer Interaction 12. Google ScholarDigital Library
- PINELLE, D., GUTWIN, C., AND GREENBERG, S. 2003. Task analysis for groupware usability evaluation: Modeling shared-workspace tasks with the mechanics of collaboration. ACM Trans. Computer-Human Interaction 10, 4 (Dec.), 281-311. Google ScholarDigital Library
- SEGAL, L. 1994. Effects of checklist interface on non-verbal crew communications. NASA Ames Research Center, Contractor Rep.177639.Google Scholar
- TANG, J. 1991. Findings from observational studies of collaborative work. Int. J. Man-Machine Studies 34, 2. Google ScholarDigital Library
- TSE, E., SHEN, C., GREENBERG, S., AND FORLINES, C. 2005. Enabling interaction with single user applications through speech and gestures on a multi-user tabletop. In Proceedings of Advanced Visual Interfaces (AVI'06), ACM Press, 336-343. Google ScholarDigital Library
Index Terms
- Multimodal multiplayer tabletop gaming
Recommendations
Enabling interaction with single user applications through speech and gestures on a multi-user tabletop
AVI '06: Proceedings of the working conference on Advanced visual interfacesCo-located collaborators often work over physical tabletops with rich geospatial information. Previous research shows that people use gestures and speech as they interact with artefacts on the table and communicate with one another. With the advent of ...
Exploring true multi-user multimodal interaction over a digital table
DIS '08: Proceedings of the 7th ACM conference on Designing interactive systemsTrue multi-user, multimodal interaction over a digital table lets co-located people simultaneously gesture and speak commands to control an application. We explore this design space through a case study, where we implemented an application that supports ...
Multimodal feedback for tabletop interactions
ITS '11: Proceedings of the ACM International Conference on Interactive Tabletops and SurfacesThis paper presents a study into the use of different modalities in providing per contact feedback for interactions on tabletop computers. We replicate the study by Wigdor et al. [3] and confirm their results, and extend the study to examine not just ...
Comments