Skip to main content
SpringerLink
Log in

Speak to Me: Interacting with a Spoken Language Interface

  • Conference paper
  • First Online:
Design, User Experience, and Usability. Design for Contemporary Interactive Environments (HCII 2020)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 12201))

Included in the following conference series:

Abstract

This study addresses the importance of encouraging effective interaction of users with information systems. We examined the interaction between users and a spoken language interface through a user-centered Wizard of Oz experiment, in which the user behavior of a spoken language search interface which allows only spoken query and touch behavior, was compared with a generic keyboard baseline interface. Forty-eight participants joined the experiment, with each searching on 12 different topics of three types. Results showed that using the spoken interface resulted in significantly less interaction measured by the number of iterations, number of viewed webpages, and number of clicks/touches, than using the baseline interface. In terms of query length, participants using the spoken interface issued significantly longer queries than those using the baseline interface. Results also indicated that participants spent significantly more time in completing tasks, dwelling on each document, and making decisions on each document using the spoken language interface than using the baseline interface. Design criteria and implications for spoken language systems were discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    http://lucene.apache.org/.

  2. 2.

    http://www.techsmith.com/morae.html.

References

  1. Jansen, B.J., Spink, A., Saracevic, T.: Real life, real users, and real needs: a study and analysis of user queries on the web. Inf. Process. Manag. 36, 207–227 (2000)

    Google Scholar 

  2. Belkin, N.J.: Anomalous states of knowledge as a basis for information retrieval. Can. J. Inf. Sci. 5, 133–143 (1980)

    Google Scholar 

  3. Karlgren, J., Franzén, K.: Verbosity and interface design (1997). http://www.ling.su.se/staff/franzen/irinterface.html

  4. Kelly, D., Dollu, V.J., Fu, X.: The loquacious user: a document-independent source of terms for query expansion. In: Proceedings of the 28th Annual ACM International Conference on Research and Development in Information Retrieval (SIGIR 2005), Salvador, Brazil, pp. 457–464 (2005)

    Google Scholar 

  5. Efthimiadis, E.N.: Query expansion. In: Williams, M.E. (ed.) Annual Review of Information Systems and Technology, vol. 31, pp. 121–187 (1996)

    Google Scholar 

  6. Belkin, C.C., Kelly, D., Lin, S.-J., Park, S.Y., Perez-Carballo, J., Sikora, C.: Iterative exploration, design and evaluation of support for query reformulation in interactive information retrieval. Inf. Process. Manag. 37(3), 403–434 (2001)

    Google Scholar 

  7. Begany, G.M., Sa, N., Yuan, X.: Factors affecting user perception of a spoken language vs. textual search interface: a content analysis. Interacting with Computers, 28(2), 170–180 (2015). https://doi.org/10.1093/iwc/iwv029

  8. Yuan, X., Sa, N.: User query behaviour in different task types in a spoken language vs. textual interface: A wizard of Oz experiment. In: The Information Behaviour Conference (ISIC), Zadar, Croatia (2016)

    Google Scholar 

  9. Belkin, N.J., Kwasnik, B.H.: Using structural representations of anomalous states of knowledge for choosing document retrieval strategies. In: SIGIR 1986: Proceedings of the 1986 ACM SIGIR International Conference on Research and Development in Information Retrieval, Pisa, Italy, pp. 11–22. ACM (1986)

    Google Scholar 

  10. Belkin, N.J., et al.: Query length in interactive information retrieval. In: SIGIR 2003. Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 205–212. ACM, New York (2003)

    Google Scholar 

  11. Stone, B.: Google Adds Live Updates to Results. New York Times, December 8 Issue (2009). http://www.nytimes.com/2009/12/08/technology/companies/08google.html

  12. Xiong, W., et al.: Toward human parity in conversational speech recognition. In: 2017. IEEE/ACM Trans. Audio Speech Lang. Process. 25(12), 2410–2423 (2017). https://doi.org/10.1109/TASLP.2017.2756440

  13. Taylor, R.S.: Question negotiation and information seeking in libraries. Coll. Res. Libr. 29, 178–194 (1968)

    Article  Google Scholar 

  14. Belkin, N.J., Oddy, R.N., Brooks, H.M.: ASK for information retrieval. Part I: background and theory; part II: results of a design study. J. Documentation 38(2&3), 61–71; 145–164 (1982)

    Google Scholar 

  15. Saracevic, T., Spink, A., Wu, M.-M.: Users and intermediaries in information retrieval: what are they talking about? In: Jameson, A., Paris, C., Tasso, C. (eds.) User Modeling. ICMS, vol. 383, pp. 43–54. Springer, Vienna (1997). https://doi.org/10.1007/978-3-7091-2670-7_6

    Chapter  Google Scholar 

  16. Trippas, J.R., Spina, D., Cavedon, L., Joho, H., Sanderson, M.: Informing the design of spoken conversational search: perspective paper. In: Proceedings of the 2018 Conference on Human Information Interaction & Retrieval, pp. 32–41. https://doi.org/10.1145/3176349.3176387

  17. Crestani, F., Du, H.: Written versus spoken queries: a qualitative and quantitative comparative analysis. J. Am. Soc. Inf. Sci. Technol. 57(7), 881–890 (2006)

    Article  Google Scholar 

  18. Du, H., Crestani, F.: Spoken versus written queries for mobile information access. In: Crestani, F., Dunlop, M., Mizzaro, S. (eds.) MUIA 2003. LNCS, vol. 2954, pp. 67–78. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-24641-1_6

    Chapter  Google Scholar 

  19. Du, H., Crestani, F.: Retrieval effectiveness of written and spoken queries: an experimental evaluation. In: Christiansen, H., Hacid, M.-S., Andreasen, T., Larsen, H.L. (eds.) FQAS 2004. LNCS (LNAI), vol. 3055, pp. 376–389. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-25957-2_30

    Chapter  Google Scholar 

  20. Du, H., Crestani, F.: Spoken versus written queries for mobile information access: an experiment on Mandarin Chinese. In: Su, K.-Y., Tsujii, J., Lee, J.-H., Kwong, O.Y. (eds.) IJCNLP 2004. LNCS (LNAI), vol. 3248, pp. 745–754. Springer, Heidelberg (2005). https://doi.org/10.1007/978-3-540-30211-7_79

    Chapter  Google Scholar 

  21. Fujii, A., Itou, K., Ishikawa, T.: Speech-driven text retrieval: using target IR collections for statistical language model adaptation in speech recognition. In: Coden, Anni R., Brown, E.W., Srinivasan, S. (eds.) IRTSA 2001. LNCS, vol. 2273, pp. 94–104. Springer, Heidelberg (2002). https://doi.org/10.1007/3-540-45637-6_9

    Chapter  MATH  Google Scholar 

  22. Zue, V., et al.: JUPITER: a telephone-based conversational interface for weather information. IEEE Trans. Speech Audio Process. 8(1) (2000)

    Google Scholar 

  23. Arguello, J., Avula, S., Diaz, F.: Using query performance predictors to reduce spoken queries. In: Jose, J.M., et al. (eds.) ECIR 2017. LNCS, vol. 10193, pp. 27–39. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-56608-5_3

    Chapter  Google Scholar 

  24. Schalkwyk, J., et al.: “Your word is my command”: Google search by voice: a case study. In: Neustein, A. (ed.) Advances in Speech Recognition, pp. 61–90. Springer, Heidelberg (2010). https://doi.org/10.1007/978-1-4419-5951-5_4

  25. Guy, I.: Searching by talking: analysis of voice queries on mobile web search. In: Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 35–44 (2016). https://doi.org/10.1145/2911451.2911525

  26. Kamvar, M., Beeferman, D.: Say what? Why users choose to speak their web queries? In: Interspeech (2010)

    Google Scholar 

  27. Google. Teens Use Voice Search Most, Even in Bathroom, Google’s Mobile Voice Study Finds (2014). http://www.prnewswire.com/news-releases/teens-use-voice-search-most-even-in-bathroom-googles-mobile-voice-study-finds-279106351.html. Accessed 23 Oct 2015

  28. Sa, N., Yuan, X.: Improving voice interaction through examining user voice search behavior: an empirical study. Presented at the Chinese CHI 2017 (2017)

    Google Scholar 

  29. Berg, M.M.: Survey on spoken dialogue systems: user expectations regarding style and usability. In XIV International Ph.D. Workshop, Wisla, Poland, October 2012 (2012)

    Google Scholar 

  30. Johnsen, M., Svendsen, T., Amble, T., Holter, T., Harborg, E.: TABOR – a Norwegian spoken dialogue system for bus travel information. In: Proceedings of 6th International Conference of Spoken Language Processing (ICSLP 2000), Beijing, China (2000)

    Google Scholar 

  31. Gustafson, J., et al.: AdApt-a multimodal conversational dialogue system in an apartment domain. In: Proceedings of 6th International Conference of Spoken Language Processing (ICSLP 2000), Beijing, China (2000)

    Google Scholar 

  32. Salber, D., Coutaz, J.: Applying the Wizard of Oz technique to the study of multimodal systems. In: Bass, Leonard J., Gornostaev, J., Unger, C. (eds.) EWHCI 1993. LNCS, vol. 753, pp. 219–230. Springer, Heidelberg (1993). https://doi.org/10.1007/3-540-57433-6_51

    Chapter  Google Scholar 

  33. Bernsen, N.O., Dybkjær, H., Dybkjær, L.: Designing Interactive Speech Systems, pp. 127–160. Springer, London (1999). https://doi.org/10.1007/978-1-4471-0897-9

    Book  Google Scholar 

  34. Akers, D.: Wizard of Oz for participatory design: inventing a gestural interface for 3D selection of neural pathway estimates. In: CHI’06 Extended Abstracts on Human Factors in Computing Systems (Montréal, Québec, Canada, 22––27 April 2006), pp. 454–459. ACM Press, New York (2006)

    Google Scholar 

  35. Klein, A., Schwank, I., Généreux, M., Trost, H.: Evaluating multimodal input modes in a Wizard-of-Oz study for the domain of web search. In: Blandford, A., Vanderdonckt, J., Gray, P. (eds.) People and Computer XV - Interaction without Frontiers: Joint Proceedings of HCI 2001 and IHM 2001, pp. 475–483. Springer, Heidelberg (2001). https://doi.org/10.1007/978-1-4471-0353-0_29

  36. Kim, J.-Y.: Task as a Predictable Indicator for Information Seeking Behavior on the Web. Unpublished Ph.D. dissertation, Rutgers University, New Brunswick (2006)

    Google Scholar 

  37. Denoyer, L., Gallinari, P.: The Wikipedia XML corpus. In: Fuhr, N., Lalmas, M., Trotman, A. (eds.) INEX 2006. LNCS, vol. 4518, pp. 12–19. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-73888-6_2

    Chapter  Google Scholar 

  38. Liu, J., Belkin, N.J.: Personalizing information retrieval for multi-session tasks: the roles of task stage and task type. In: Proceedings of the 33rd Annual International ACM SIGIR Conference on Research & Development on Information Retrieval (SIGIR 2010). Geneva, Switzerland, 19–23 July 2010

    Google Scholar 

  39. White, R.W., Muresan, G., Marchionini, G.: Evaluating exploratory search systems: introduction to a special topic issue. Inf. Process. Manag. 44(2), 433–436 (2007)

    Article  Google Scholar 

  40. Buttcher, S., Clarke, C.L., Soboroff, I.: The TREC 2006 terabyte track. In: TREC, vol. 6, p. 39 (2006)

    Google Scholar 

  41. White, R.W., Kules, B., Drucker, S., Schraefel, M.C.: Supporting exploratory search. Spec. Sect. Commun. ACM 49(4), 36–39 (2006)

    Google Scholar 

  42. Murdock, V., Kelly, D., Croft, W.B., Belkin, N.J., Yuan, X.-J.: Identifying and improving retrieval for procedural questions. Inf. Process. Manag. 43(1), 181–203 (2007)

    Article  Google Scholar 

Download references

Acknowledgments

We thank the Institute of Museum and Library Services (IMLS) grant #RE-04-10-0053-10.

Author information

Authors and Affiliations

  1. University at Albany, State University of New York, Albany, NY, USA

    Xiaojun Yuan & Ning Sa

Authors
  1. Xiaojun Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ning Sa
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaojun Yuan .

Editor information

Editors and Affiliations

  1. Aaron Marcus and Associates, Berkeley, CA, USA

    Aaron Marcus

  2. World Usability Day and Bentley User Experience Center, Newton Center, MA, USA

    Elizabeth Rosenzweig

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Yuan, X., Sa, N. (2020). Speak to Me: Interacting with a Spoken Language Interface. In: Marcus, A., Rosenzweig, E. (eds) Design, User Experience, and Usability. Design for Contemporary Interactive Environments. HCII 2020. Lecture Notes in Computer Science(), vol 12201. Springer, Cham. https://doi.org/10.1007/978-3-030-49760-6_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-49760-6_13

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-49759-0

  • Online ISBN: 978-3-030-49760-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation