D. Scott McCrickard
Skip Abstract Section
Abstract
Addressing the need to tailor usability evaluation methods (UEMs) and promote effective reuse of HCI knowledge for computing activities undertaken in divided-attention situations, we present the foundations of a unifying model that can guide evaluation efforts for notification systems. Often implemented as ubiquitous systems or within a small portion of the traditional desktop, notification systems typically deliver information of interest in a parallel, multitasking approach, extraneous or supplemental to a user's attention priority. Such systems represent a difficult challenge to evaluate meaningfully. We introduce a design model of user goals based on blends of three critical parameters---interruption, reaction, and comprehension. Categorization possibilities form a logical, descriptive design space for notification systems, rooted in human information processing theory. This model allows conceptualization of distinct action models for at least eight classes of notification systems, which we describe and analyze with a human information processing model. System classification regions immediately suggest useful empirical and analytical evaluation metrics from related literature. We present a case study that demonstrates how these techniques can assist an evaluator in adapting traditional UEMs for notification and other multitasking systems. We explain why using the design model categorization scheme enabled us to generate evaluation results that are more relevant for the system redesign than the results of the original exploration done by the system's designers.
- Abowd, G. D. and Mynatt, E. D. 2000. Charting past, present, and future research in ubiquitous computing. ACM Trans. Comput.-Hum. Inter. 7, 1, 29--58. Google Scholar
- Anderson, J. R. and Liebiere, C. 1998. The Atomic Components of Thought. Lawrence Erlbaum Associates, Inc., Mahwah, New Jersey.Google Scholar
- Baddelly, A. 1996. Exploring the central executive. The Quarterly Journal of Experimental Psychology 49A, 5--28.Google Scholar
- Ballas, J. A., Heitmeyer, C. L., and Perez, M. A. 1992. Evaluating two aspects of direct manipulation in advanced cockpits. In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI '92). Monterey, CA, 127--134. Google Scholar
- Barnard, P. and May, J. 1999. Representing cognitive activity in complex tasks. Human-Computer Interaction 14, 93--158. Google Scholar
- Barnard, P. and May, J. 2000. Systems, interactions, and macrotheory. ACM Trans. Comput.-Hum. Inter. 7, 2 (June), 222--262. Google Scholar
- Bartram, L. 1998. Enhancing visualizations with motion. In Proceedings of the IEEE Symposium on Information Visualization (InfoVis '98). Raleigh, NC, 13--16.Google Scholar
- Bartram, L., Ware, C., and Calvert, T. 2001. Moving icons: Detection and distraction. In Proceedings of the IFIP TC.13 International Conference on Human-Computer Interaction (INTERACT 2001). Tokyo, Japan.Google Scholar
- Bartram, L. R. 2001. Enhancing information visualization with motion. Ph.D. thesis, Simon Fraser University, Canada.Google Scholar
- Cadiz, J., Venolia, G. D., Jancke, G., and Gupta, A. 2001. Sideshow: Providing peripheral awareness of important information. Tech. Rep. MSR-TR-2001-83, Microsoft Research, Collaboration, and Multimedia Group. Sept.Google Scholar
- Chewar, C. M., McCrickard, D. S., Ndiwalana, A., North, C., Pryor, J., and Tessendorf, D. 2002. Secondary task display attributes: Optimizing visualizations for cognitive task suitability and interference avoidance. In Proceedings of the Symposium on Data Visualization (VisSym '02). Eurographics Association, Barcelona, Spain, 165--171. Google Scholar
- Cutrell, E., Czerwinski, M., and Horvitz, E. 2001. Notification, disruption, and memory: Effects of messaging interruptions on memory and performance. In Proceedings of the IFIP TC.13 International Conference on Human-Computer Interaction (INTERACT 2001). Tokyo, Japan, 263--269.Google Scholar
- Dahley, A., Wisneski, C., and Ishii, H. 1998. Water lamp and pinwheels: Ambient projection of digital information into architectural space. In Proceedings of the Conference on CHI 98 Summary: Human Factors in Computing Systems. ACM Press, 269--270. Google Scholar
- Duchnicky, R. L. and Kolers, P. A. 1983. Readability of text scrolled on visual display terminals as a function of window size. Human Factors 25, 6, 683--692.Google Scholar
- Enns, J. T. and Rensink, R. A. 1991. Preattentive recovery of three-dimensional orientation from line drawings. Psychological Review 98, 335--351.Google Scholar
- Ericsson, K. A. and Kintsch, W. 1995. Long term working memory. Psychological Review 102, 2, 211--245.Google Scholar
- Foster, K. I. 1970. Visual perception of rapidly presented word sequences of varying complexity. Perception and Psychophysics 8, 215--221.Google Scholar
- Granaas, M. M., McKay, T. D., Laham, R. D., Hurt, L. D., and Juola, J. F. 1984. Reading moving text on a CRT screen. Human Factors 26, 1, 97--104.Google Scholar
- Gray, W. D. and Saltzman, M. C. 1998. Damaged merchandise? A review of experiments that compare usability evaluation methods. Human-Computer Interaction 13, 3, 203--261. Google Scholar
- Green, P. 1999. The 15-second rule for driver information systems. In Proceedings of the ITS America Ninth Annual Meeting. Washington, DC, CD--ROM.Google Scholar
- Greenberg, S. and Fitchett, C. 2001. Phidgets: Easy development of physical interfaces through physical widgets. In Proceedings of the ACM Conference on User Interface Software and Technology (UIST '01). Orlando, FL. Google Scholar
- Harrison, B. L., Ishii, H., Vicente, K. J., and Buxton, W. A. S. 1995. Transparent layered user interfaces: An evaluation of a display design to enhance focused and divided attention. In Conference Proceedings on Human Factors in Computing Systems (CHI '95). ACM Press/Addison-Wesley Publishing Co., 317--324. Google Scholar
- Healey, C. G., Booth, K. S., and Enns, J. T. 1996. High-speed visual estimation using preattentive processing. ACM Trans. Hum. Comput. Inter. 3, 2, 107--135. Google Scholar
- Healey, C. G. and Enns, J. T. 1999. Large datasets at a glance: Combining textures and colors in scientific visualization. IEEE Transactions on Visualization and Computer Graphics 5, 2, 145--167. Google Scholar
- Heiner, J. M., Hudson, S. E., and Tanaka, K. 1999. The information percolator: Ambient information display in a decorative object. In Proceedings of the ACM Symposium on User Interface Software and Technology (UIST '99). Asheville, NC, 141--148. Google Scholar
- Horvitz, E. 1999. Principles of mixed-initiative user interfaces. In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI '99). Pittsburgh, PA, 159--166. Google Scholar
- Horvitz, E., Jacobs, A., and Hovel, D. 1999. Attention-sensitive alerting. In Conference on Uncertainty and Artificial Intelligence (UAI '99). Stockholm, Sweden, 305--313. Google Scholar
- Hudson, J. M., Christensen, J., Kellogg, W. A., and Erickson, T. 2002. "I'd be overwhelmed, but it's just one more thing to do": Availability and interruption in research management. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '02). ACM Press, 97--104. Google Scholar
- Ishii, H., Wisneski, C., Brave, S., Dahley, A., Gorbet, M., Ullmer, B., and Yarin, P. 1998. ambient-ROOM: Integrating ambient media with architectural space. In Proceedings of the Conference on CHI 98 Summary: Human Factors in Computing Systems (CHI '98). ACM Press, 173--174. Google Scholar
- Juola, J. F., Ward, N. J., and McNamara, T. 1982. Visual search and reading of rapid serial presentations of letter strings, words, and text. J. Exper. Psych. General 111, 2, 208--227.Google Scholar
- Kang, T. J. and Muter, P. 1989. Reading dynamically displayed text. Behaviour and Information Technology 8, 1, 33--42.Google Scholar
- Kieras, D. E. and Meyer, D. E. 1997. An overview of the EPIC architecture for cognition and performance with application to human-computer interaction. Human-Computer Interaction 12, 4, 391--438. Google Scholar
- MacIntyre, B., Mynatt, E. D., Voida, S., Hansen, K. M., Tullio, J., and Corso, G. M. 2001. Support for multitasking and background awareness using interactive peripheral displays. In Proceedings of the 14th Annual ACM Symposium on User Interface Software and Technology (UIST '01). ACM Press, 41--50. Google Scholar
- Mamykina, L., Mynatt, E., and Terry, M. A. 2001. Time aura: Interfaces for pacing. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '01). ACM Press, 144--151. Google Scholar
- McCrickard, D. S. 1999. Maintaining information awareness with Irwin. In Proceedings of the World Conference on Educational Multimedia/Hypermedia and Educational Telecommunications (ED-MEDIA '99). Seattle, WA.Google Scholar
- McCrickard, D. S., Catrambone, R., and Stasko, J. T. 2001. Evaluating animation in the periphery as a mechanism for maintaining awareness. In Proceedings of the IFIP TC.13 International Conference on Human-Computer Interaction (INTERACT 2001). Tokyo, Japan, 148--156.Google Scholar
- McCrickard, D. S. and Chewar, C. M. 2003. Attuning notification design to user goals and attention costs. Comm. ACM 46, 3, 67--72. Google Scholar
- McCrickard, D. S., Czerwinski, M., and Bartram, L. 2003. Introduction: Design and evaluation of notification system interfaces. Inter. J. Hum.-Comput. Studies 8, 5, 509--514. Google Scholar
- McFarlane, D. C. 1998. Interruption of people in human-computer interaction. Ph.D. thesis, George Washington University, Washington DC. Google Scholar
- McFarlane, D. C. 2002. Comparision of four primary methods for coordinating the interruption of people in human-computer interaction. Human Computer Interaction 17, 3. Google Scholar
- Newell, A. 1990. Unified Theories of Cognition. Harvard University Press, Cambridge, MA. Google Scholar
- Newman, W. M. 1997. Better or just different? On the benefits of designing interactive systems in terms of critical parameters. In Proceedings of the Conference on Designing Interactive Systems: Processes, Practices, Methods, and Techniques (DIS '97). ACM Press, 239--245. Google Scholar
- Norman, D. A. 1986. Cognitive engineering. In User Centered System Design: New Perspectives on Human Computer Interaction, D. A. Norman and S. W. Draper, Eds. Lawrence Erlbaum Associates, 31--62.Google Scholar
- Redström, J., Skog, T., and Hallnäs, L. 2000. Informative art: Using amplified artworks as information displays. In Proceedings of DARE 2000 on Designing Augmented Reality Environments. ACM Press, 103--114. Google Scholar
- Rhodes, B. and Maes, P. 2000. Just-in-time information retrieval agents. IBM Syst. J. 39, 3--4, 685--704. Google Scholar
- Rogers, R. D. and Monsell, S. 1995. Cost of a predictable switch between simple cognitive tasks. J. Exper. Psych. General 124, 2, 207--231.Google Scholar
- Rosson, M. B. and Carroll, J. M. 2002. Usability Engineering: Scenario-Based Development of Human-Computer Interaction. Morgan-Kaufman, New York, NY. Google Scholar
- See, J. E., Howe, S. R., Warm, J. S., and Dember, W. 1995. Meta analysis of the sensitivity decrement in vigilance. Psych. Bull. 117, 2, 230--249.Google Scholar
- Sheridan, T. 1991. Human factors of driver-vehicle interaction in the IVHS environment. Tech. Rep. DOT HS 807 837, National Technical Information Service, Springfield MA.Google Scholar
- Sutcliffe, A. 2000. On the effective use and reuse of HCI knowledge. ACM Trans. Comput.-Hum. Inter. 7, 2 (June), 197--221. Google Scholar
- Tufano, D., Knee, H., and Spelt, P. 1996. In-vehicle signing functions and systems concepts. In Proceedings of the 29th International Symposium on Automotive Technology and Automation (ISATA) Dedicated Conference on Global Deployment of Advanced Transportation Telematics/ ITS. Florence Italy, 97--104.Google Scholar
- van Dantzich, M., Robbins, D., Horvitz, E., and Czerwinski, M. 2002. Scope: Providing awareness of multiple notifications at a glance. In Proceedings of the 6th International Working Conference on Advanced Visual Interfaces (AVI '02). ACM Press. Google Scholar
- Weiser, M. and Brown, J. S. 1996. Designing calm technology. PowerGrid Journal 1.01.Google Scholar
- Whittaker, S., Terveen, L., and Nardi, B. A. 2000. Let's stop pushing the envelope and start addressing it: A reference task agenda for HCI. Human-Computer Interaction 15, 75--106. Google Scholar
- Wickens, C. D. and Hollands, J. G. 2000. Engineering Psychology and Human Performance, Third ed. Prentice Hall, Upper Saddle River, NJ.Google Scholar
Index Terms
- A model for notification systems evaluation—assessing user goals for multitasking activity
Recommendations
Usability Evaluation of Model-Driven Cross-Device Web User Interfaces
Human-Centered Software EngineeringAbstractUser Interface (UI) development is a challenging task as modern UIs are expected to be available across a wide range of diverse platforms while assuring high usability for heterogeneous users. Model-driven engineering principles have been applied ...
Testing strategies for evaluation of user interfaces in SOA-based systems
MIDI '13: Proceedings of the International Conference on Multimedia, Interaction, Design and InnovationThis paper presents a research that was conducted to determine the best techniques and testing strategies for user interface evaluation of SOA-based systems. Although we can find many articles on, for example, the design of web services, few guidelines ...
Towards a human-computer interaction model for voice user interfaces
CLIHC '19: Proceedings of the IX Latin American Conference on Human Computer InteractionThe user interaction with computer systems has evolved over the years, from Command Line Interfaces (CLIs), Graphics User Interfaces (GUIs), Natural User Interfaces (NUIs) and actually Voice User Interfaces (VUIs). The use of VUIs is increasingly common,...
Comments