Lisa Anthony
Jie Yang
Kenneth R. Koedinger
ABSTRACT
Current standard interfaces for entering mathematical equations on computers are arguably limited and cumbersome. Mathematics notations have evolved to aid visual thinking and yet text-based interfaces relying on keyboard-and-mouse input do not take advantage of the natural two-dimensional aspects of math. Due to its similarities to paper-based mathematics, pen-based handwriting input may be faster, more efficient, and more preferable for entering mathematics on computers. This paper presents an empirical study that tests this hypothesis. We also explored a multimodal input method combining handwriting and speech because we hypothesize that it may enhance computer recognition and aid user cognition. Novice users were indeed faster, more efficient and enjoyed the handwriting modality more than a standard keyboard-and-mouse mathematics interface, especially as equation length and complexity increased. The multimodal handwriting-plus-speech method was faster and better liked than the keyboard-and-mouse method and was not much worse than handwriting alone.
- Anderson, J.R., Corbett, A.T., Koedinger, K.R., and Pelletier, R.: Cognitive Tutors: Lessons Learned. The Journal of the Learning Sciences 4 (1995) 167--207.]]Google Scholar
Cross Ref
- Blostein, D. and Grbavec, A. Recognition of Mathematical Notation. In Handbook on Optical Character Recognition and Document Analysis, Wang, P.S.P. and Bunke, H. (eds) (1996) 557--582.]]Google Scholar
- Brown, C.M.L.: Comparison of Typing and Handwriting in "Two-Finger Typists." Proceedings of the Human Factors Society (1988) 381--385.]]Google ScholarCross Ref
- Locke, J.L. & Fehr, F.S. Subvocalization of Heard or Seen Words Prior to Spoken or Written Recall. American Journal of Psychology 85 (1972) 63--68.]]Google ScholarCross Ref
- Microsoft. Microsoft Word User's Guide Version 6.0 (1993), Microsoft Press.]] Google ScholarDigital Library
- Oviatt, S. Mutual Disambiguation of Recognition Errors in a Multimodal Architecture. Proceedings of the CHI Conference (1999) 576--583.]] Google ScholarDigital Library
- Smithies, S., Novins, K., and Arvo, J.: Equation Entry and Editing via Handwriting and Gesture Recognition. Behaviour and Information Technology 20 (2001) 53--67.]]Google ScholarCross Ref
- Sweller, J.: Cognitive Load During Problem Solving: Effects on Learning. Cognitive Science 12 (1988) 257--285.]]Google ScholarCross Ref
Index Terms
- Evaluation of multimodal input for entering mathematical equations on the computer
Recommendations
Reliable Evaluation of Multimodal Dialogue Systems
Proceedings of the 13th International Conference on Human-Computer Interaction. Part II: Novel Interaction Methods and TechniquesUsability evaluation is an indispensable issue during the development of new interfaces and interaction paradigms [1]. Although a wide range of reliable usability evaluation methods exists for graphical user interfaces, mature methods are rarely ...
Adaptive multimodal fusion
UAHCI'11: Proceedings of the 6th international conference on Universal access in human-computer interaction: design for all and eInclusion - Volume Part IMultimodal interfaces offer its users the possibility of interacting with computers, in a transparent, natural way, by means of various modalities. Fusion engines are key components in multimodal systems, responsible for combining information from ...
Multimodal slideshow: demonstration of the openinterface interaction development environment
ICMI '08: Proceedings of the 10th international conference on Multimodal interfacesIn this paper, we illustrate the OpenInterface Interaction Development Environment (OIDE) that addresses the design and development of multimodal interfaces. Multimodal interaction software development presents a particular challenge because of the ever ...
Comments