Irene Sturm
ABSTRACT
Human-machine interaction by gestures helps to improve communication: it utilizes means of communication which are common to humans but alien to machines. We consider ergonomic, functional and semantic issues in gesture-based interfaces without visual feedback. Such situations arise with the unobtrusive usage of wearable devices but, more importantly, with interfaces for visually impaired persons. We consider the latter scenario -- that of blind users -- in view of forthcoming haptic interface technology and investigate properties of gestural interaction models.
- S. Brewster, J. Lumsden, M. Bell, M. Hall, S. Tasker: Multimodal 'eyes-free' interaction techniques for wearable devices. In G. Cockton, P. Korhonen (editors): Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2003, April 5--10, 2003, Ft. Lauderdale, Florida. 473--480, ACM Press, New York, NY, 2003. Google Scholar
Digital Library
- J. Carter, J. van Erp, D. Fourney, S. Fukuzumi, J. A. Gardner, Y. Horiuchi, G. Jansson, H. Jürgensen, R. Kadefors, T. Kobayashi, M. G. Kwok, M. Miyagi, K. V. Nesbitt: The GOTHI model of tactile and haptic interactions. In Carter and Fourney {3}, 93--95. Available at http://userlab.usask.ca.Google Scholar
- J. Carter, D. Fourney (editors): Proceedings of GOTHI '05, Guidelines on Tactile and Haptic Interactions, October 24--26, 2005, Saskatoon, Saskatchewan, Canada. USERLab, University of Saskatchewan, Saskatoon, 2005. Available at http://userlab.usask.ca.Google Scholar
- A. Cooper, R. Reimann, D. Cronin: About Face 3: The Essentials of Interaction Design. John Wiley & Sons, Indianapolis, 2007. Google ScholarDigital Library
- M. Eramian, H. Jürgensen, H. Li, C. Power: Talking tactile diagrams. In C. Stephanidis (editor): Proceedings of HCI International 2003, Volume 4: Universal Access in HCI, Inclusive Design in the Information Society. 1377--1381, Lawrence Erlbaum Associates, Mahwah, New Jersey, 2003.Google Scholar
- J. B. F. van Erp, T. A. Kern: ISO's work on guidance for haptic and tactile interactions. In M. Ferre (editor): Haptics: Perception, Devices and Scenarios. 6th International Conference, EuroHaptics 2008, Madrid, Spain, June 10--13, 2008, Proceedings. Lecture Notes in Computer Science 5024, 936--940, Springer-Verlag, Berlin, 2008. Google ScholarDigital Library
- M. Fröhlich: Ein wissensbasiertes Rahmensystem zur merkmalsbasierten Gestenerkennung für multimediale Anwendungen. Dissertation, Universität Bielefeld, 1999.Google Scholar
- HyperBraille®. http://www.hyperbraille.de/.Google Scholar
- ISO 9241-9xx: Ergonomics of human-system interaction. Part 910: ISO/WD 9241--910, Framework for tactile and haptic interaction (working draft); part 920: ISO 9241--920, Guidance on tactile and haptic interactions (to appear); part 940: ISO/WD 9241--940, Evaluation of tactile/haptic interactions (working draft). Prepared by ISO TC 159/SC 4/WG9; for the general contents of these documents see {2}, 2009.Google Scholar
- D. Katz: Der Aufbau der Tastwelt. Zeitschrift für Psychologie, Ergänzungsband 11 (1925). Reprinted by Wissenschaftliche Buchgesellschaft, Darmstadt, 1969.Google Scholar
- D. Katz: Studien zur experimentellen Psychologie. Benno Schwabe & Co, Verlag, Basel, 1953.Google Scholar
- A. Kendon: Current issues in the study of gesture. In Nespoulos et al. {24}, 23--47.Google Scholar
- S. Kipke: Sensitive Braille displays with ATC technology (active tactile control) as a tool for learning Braille. In Miesenberger et al. {18}, 843--850. Google ScholarDigital Library
- P. Kolesnik, M. Wanderley: Recognition, analysis and performance with expressive conducting gestures. In Proceedings of the 2004 International Computer Music Conference, ICMC '04, Miami, Florida. 572--575, ICMA, San Francisco, 2004.Google Scholar
- G. Kurtenbach, W. Buxton: The limits of expert performance using hierarchic marking menus. In B. Arnold, G. van der Veer, T. White (editors): Proceedings of the INTERACT '93 and CHI '93 Conference on Human Factors in Computing Systems, April 24--29, 1993, Amsterdam, The Netherlands. 482--487, ACM Press, New York, NY, 1993. Google ScholarDigital Library
- S. Landau: Tactile graphics and strategies for non-visual seeing. In Thresholds 19. 78--82. MIT School of Architecture, 1999.Google Scholar
- S. Landau, K. Gourgey: Development of a talking tactile tablet. Information Technology and Disabilities 7(2) (2001).Google Scholar
- K. Miesenberger, J. Klaus, W. Zagler, A. Karshmer (editors): Computer Helping People with Special Needs, 11th International Conference, ICCHP 2008, Linz, Austria, July 9--11, 2008, Proceedings. Lecture Notes in Computer Science 5105. Springer-Verlag, Berlin, 2008. Google ScholarDigital Library
- M. Miyagi, M. Nishida, Y. Horiuchi: Conference system using finger Braille. In Carter and Fourney {3}, 28--31. Available at http://userlab.usask.ca.Google Scholar
- M. Miyagi, M. Nishida, Y. Horiuchi, A. Ichikawa: Investigation on effect of prosody in finger Braille. In K. Miesenberger, J. Klaus, W. Zagler, A. Karshmer (editors): Computer Helping People with Special Needs, 10th International Conference, ICCHP 2006, Linz, Austria, July 11--13, 2006, Proceedings. Lecture Notes in Computer Science 4061, 863--869, Springer-Verlag, Berlin, 2006. Google ScholarDigital Library
- K. V. Nesbitt: A framework to support the designers of haptic, visual and auditory displays. In Carter and Fourney {3}, 54--64. Available at http://userlab.usask.ca.Google Scholar
- K. V. Nesbitt: Structured guidelines to support the design of haptic displays. In Carter and Fourney {3}, 65--74. Available at http://userlab.usask.ca.Google Scholar
- J.-L. Nespoulos, A. R. Lecours: Gestures: Nature and function. In Nespoulos et al. {24}, 49--62.Google Scholar
- J.-L. Nespoulos, P. Perron, A. R. Lecours (editors): The Biological Foundations of Gestures: Motor and Semiotic Aspects. Lawrence Erlbaum Associates, Hillsdale, NJ, 1986.Google Scholar
- M. Nielsen, T. B. Moeslund, M. Störring, E. Granum: Gesture interfaces. In P. Kortum (editor): HCI Beyond the GUI: Design for Haptic, Speech, Olfactory, and Other Nontraditional Interfaces. 75--106. Elsevier, Amsterdam, 2008.Google Scholar
- A. Pirhonen, S. Brewster, C. Holguin: Gestural and audio metaphors as a means of control for mobile devices. In D. Wixton (editor): Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2002, April 20--25, 2002, Minneapolis, USA. 291--298, ACM Press, New York, NY, 2002. Google ScholarDigital Library
- S.-P. Poh: Talking diagrams. Master's thesis, The University of Western Ontario, 1995. Report 459, Department of Computer Science, The University of Western Ontario.Google Scholar
- M. Potamias, V. Athitsos: Nearest neighbor search methods for handshape recognition. In F. Makedon, L. Baillie (editors): The 1st ACM International Conference on PErvasive Technologies Related to Assistive Environments, PETRA 2008, July 16--18, 2008, Athens, Greece. ACM International Conference Proceedings Series, 8 pp., ACM Press, New York, 2008. Google ScholarDigital Library
- C. Power: Multi-Modal Exploration. PhD thesis, The University of Western Ontario, 2006. Google ScholarDigital Library
- C. Power, H. Jürgensen: Accessible presentation of information for people with visual disabilities. Univ. Access Inf. Soc. (2009), 49 pages. To Appear. Google ScholarDigital Library
- M. Schiewe, I. Sturm: Gesten in HyperBraille, Test 1; Auswertung des Tests 1, 5+2 pp. Manuscript, Potsdam, April 2008.Google Scholar
- M. Schmidt, H. Weber: Multitouch haptic interaction. In HCI International 2009, 13th International Conference on Human-Computer Interaction, 19--24 July, 2009, San Diego, California, USA. Lecture Notes in Computer Science, To Appear, Springer-Verlag, Berlin, 2009. Google ScholarDigital Library
- I. Sturm: Non-visual communication with gestures, 9 pp. Manuscript, Potsdam, 2008.Google Scholar
- H. Thimbleby: Press on: Principles of Interaction Programming. MIT Press, Cambridge, Mass., 2007. Google ScholarDigital Library
- D. Venolia, F. Neiberg: T-Cube: A fast, self-disclosing pen-based alphabet. In B. Adelson, S. Dumais, J. Olson (editors): Proceedings of the SIGCHI Conference on Human Factors in Computing Systems: Celebrating Interdependence, CHI '94, April 24--28, 1994, Boston, Mass. 265--270, ACM Press, New York, 1994. Google ScholarDigital Library
- T. Völkel, G. Weber, U. Baumann: Tactile graphics revised: The novel BrailleDis 9000 pin-matrix device with multitouch input. In Miesenberger et al. {18}, 835--842. Google ScholarDigital Library
- G. Weber: Gestures as a means for the blind to interact with a computer. In H.-J. Bullinger, B. Shackel (editors): Human-Computer Interaction -- INTERACT '87, Proceedings of the Second IFIP Conference on Human-Computer Interaction, held at the University of Stuttgart, Federal Republic of Germany, 1--4 September, 1987. 593--595, North-Holland, Amsterdam, 1987.Google Scholar
- G. Weber: Interaktive Dialogtechniken für blinde Rechnerbenutzer. Dissertation, Universität Stuttgart, 1989.Google Scholar
- G. Weber: Reading and pointing -- modes of interaction for blind users. In G. Ritter (editor): Information Processing 89, Proceedings of the IFIP 11th World Computer Congress, August 28--September 1, 1989, San Francisco, USA. 535--540, North-Holland, Amsterdam, 1989.Google Scholar
- G. Weber: FINGER -- A language for gesture recognition. In D. Diaper, D. J. Gilmore, G. Cockton, B. Shackel (editors): Human-Computer Interaction -- INTERACT '90: Proceedings of the IFIP TC 13 Third International Conference on Human-Computer Interaction, 27--31 August, 1990, Cambridge, UK. 689--694, North-Holland, Amsterdam, 1990. Google ScholarDigital Library
- P. Wetzel: Erkennen von Gesten im Dialog blinder Benutzer mit dem Rechner. Diplomarbeit, Institut für Informatik, Universität Stuttgart, 1988.Google Scholar
- G. Yfantidis, G. Evreinov: Adaptive blind interaction technique for touchscreens. Univ. Access Inf. Soc. 4 (2006), 328--337. Google ScholarDigital Library
- S. Zhao, P. Dragicevic, M. Chignell, R. Balakrishnan, P. Baudisch: earPod: Eyes-free menu selection using touch input and reactive audio feedback. In M. B. Rosson, D. Gilmore (editors): Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI '07, April 28--May 3, 2007, San Jose, California. 1395--1404, ACM Press, New York, NY, 2007. Google ScholarDigital Library
Index Terms
- Communicating through gestures without visual feedback
Recommendations
Haptic emulation of games: haptic Sudoku for the blind
PETRA '10: Proceedings of the 3rd International Conference on PErvasive Technologies Related to Assistive EnvironmentsFor blind persons, information is often presented through substitute media. We explore the possibility of emulating visual perception through haptic perception completely using a new haptic display and using games as a paradigm. We decsribe an ...
HELF (Haptic Encoded Language Framework): a digital script for deaf-blind and visually impaired
AbstractPurposeDigital media has brought a revolution, making the world a global village. For people who are visually impaired and people with visual and hearing impairment, navigating through the digital world can be as precarious as moving through the ...
Exploring feasibility of wrist gestures for non-visual interactions with wearables
W4A '19: Proceedings of the 16th International Web for All ConferenceThe emergence of wearable devices such as smartwatches is spurring new user interface designs and interaction modalities for these devices. One such new input modality for interacting with smartwatches is wrist gestures. Smartwatches are beginning to ...
Comments