Skip to main content
SpringerLink
Log in

A Literature Review of AR-Based Remote Guidance Tasks with User Studies

  • Conference paper
  • First Online:
Virtual, Augmented and Mixed Reality. Industrial and Everyday Life Applications (HCII 2020)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 12191))

Included in the following conference series:

Abstract

The future of work is increasingly mobile and distributed across space and time. Institutions and individuals are phasing out desktops in favor of laptops, tablets and/or smart phones as much work (assessment, technical support, etc.) is done in the field and not at a desk. There will be a need for systems that support remote collaborations such as remote guidance. Augmented reality (AR) is praised for its ability to show the task at hand within an immersive environment, allowing for spatial clarity and greater efficiency, thereby showing great promise for collaborative and remote guidance tasks; however, there are no systematic reviews of AR based remote guidance systems. This paper reviews the literature describing AR-based remote guidance tasks and discusses the task settings, technical requirements and user groups within the literature, followed by a discussion of further areas of interest for the application of this technology combined with artificial intelligence (AI) algorithms to increase the efficiency of applied tasks.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Bonnet, P., Ducher, P., Kubiak, A.: A brief introduction to augmented reality. In: Koelle, M., Linderman, P., Stockinger, T., Kranz, M. (eds.) Human-Computer Interaction with Augmented Reality. Advances in Embedded Interactive Systems Technical Report, vol. 2, no. 4, 30 p. (2014)

    Google Scholar 

  2. Aameer, R.W., Sofi, S., Roohie, N.: Augmented reality for fire & emergency services. In: Proceedings of International Conference on Recent Trends in Communications and Computer Networks, pp. 32–41 (2013)

    Google Scholar 

  3. Yang, L., Liang, Y., Wu, D., Gault, J.: Train and equip firefighters with cognitive virtual and augmented reality. In: Proceedings of the IEEE 4th International Conference on Collaboration and Internet Computing, pp. 453–459 (2018)

    Google Scholar 

  4. Yun, K., Lu, T., Chow, E.: Occluded object reconstruction for first responders with augmented reality glasses using conditional generative adversarial networks. In: Proceedings of SPIE 2018, vol. 10649, 7 p. (2018)

    Google Scholar 

  5. Lochhead, I., Hedley, N.: Mixed reality emergency management: bringing virtual evacuation simulations into real-world built environments. Int. J. Digit. Earth 12(2), 190–208 (2019)

    Article  Google Scholar 

  6. Caudell, T.P., Mizell, D.W.: Augmented reality: an application of heads-up display technology to manual manufacturing processes. In: Proceedings of Hawaii International Conference on System Sciences, pp. 659–669 (1992)

    Google Scholar 

  7. Azuma, R.T.: A survey of augmented reality. Presence Teleoperators Virtual Environ. 6(4), 355–385 (1997)

    Article  Google Scholar 

  8. Billinghurst, M., Clark, A., Lee, G.: A survey of augmented reality. Found. Trends Hum.-Comput. Interact. 8(2–3), 73–272 (2014)

    Google Scholar 

  9. Milgram, P.: A taxonomy of mixed reality visual displays. IEICE Trans. Inf. Syst. E77-D(12), 1321–1329 (1994)

    Google Scholar 

  10. Biseria, A., Rao, A.: Human computer interface-augmented reality. Int. J. Eng. Sci. Comput. 6(8), 2594–2595 (2016)

    Google Scholar 

  11. Chen, P.H.C., et al.: An augmented reality microscope with real-time artificial intelligence integration for cancer diagnosis. Nature Med. 25, 1453–1457 (2019)

    Article  Google Scholar 

  12. Schlueter, J.A.: Remote maintenance assistance using real-time augmented reality authoring. Master thesis, Iowa State University, 72 p. (2018)

    Google Scholar 

  13. Kim, Y., Hong, S., Kim, G.J.: Augmented reality based remote coaching system. In: Proceedings of the 22nd ACM Conference on Virtual Reality Software and Technology, VRST 2016, Munchen, Germany, pp. 311–312 (2016)

    Google Scholar 

  14. Yamamoto, T., Otsuki, M., Kuzuoka, H., Suzuki, Y.: Tele-guidance system to support anticipation during communication. Multimodal Technol. Interact. 2(3), 55 (2018)

    Article  Google Scholar 

  15. Huang, W., Alem, L., Tecchia, F.: HandsIn3D: supporting remote guidance with immersive virtual environments. In: Kotzé, P., Marsden, G., Lindgaard, G., Wesson, J., Winckler, M. (eds.) INTERACT 2013. LNCS, vol. 8117, pp. 70–77. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-40483-2_5

    Chapter  Google Scholar 

  16. Rao, H., Fu, W.-T.: Combining schematic and augmented reality representations in a remote spatial assistance system. In: IEEE ISMAR 2013 Workshop on Collaboration in Merging Realities, 6 p. (2013)

    Google Scholar 

  17. Hoover, M.: An evaluation of the Microsoft HoloLens for a manufacturing-guided assembly task. Graduate theses and dissertations, Iowa State University (2018). https://lib.dr.iastate.edu/etd/16378

  18. Augestad, K.M., et al.: Educational implications for surgical telemonitoring: a current review with recommendations for future practice, policy, and research. Surg. Endosc. 31, 3836–3846 (2017). https://doi.org/10.1007/s00464-017-5690-y

    Article  Google Scholar 

  19. Jones, B., et al.: Elevating communication, collaboration, and shared experiences in mobile video through drones. In: Proceedings of the 2016 ACM Conference on Designing Interactive Systems (DIS 2016), Brisbane, Australia, pp. 1123–1135 (2016)

    Google Scholar 

  20. Kuzuoka, H.: Spatial workspace collaboration: a SharedView video support system for remote collaboration capability. In: Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems (CHI 1992), Monterey, California, USA, pp. 533–540 (1992)

    Google Scholar 

  21. Ou, J., Fussell, S.R., Chen, X., Setlock, L.D., Yang, J.: Gestural communication over video stream: supporting multimodal interaction for remote collaborative physical tasks. In: Proceedings of the ACM 5th Conference on Multimodal Interfaces (ICMI 2003), Vancouver, British Columbia, Canada, pp. 242–249 (2003)

    Google Scholar 

  22. Kuzuoka, H., et al.: Mediating dual ecologies. In: Proceedings of the ACM Conference on Computer Supported Cooperative Work, Chicago, Illinois, USA, pp. 477–486 (2004)

    Google Scholar 

  23. Fussell, S.R., Setlock, L.D., Yang, J., Ou, J., Mauer, E., Kramer, A.D.L.: Gestures over video streams to support remote collaboration on physical tasks. Hum.-Comput. Interact. 19, 273–309 (2004)

    Article  Google Scholar 

  24. Huang, W., Alem, L., Tecchia, F., Duh, H.B.-L.: Augmented 3D hands: a gesture-based mixed reality system for distributed collaboration. J. Multimodal User Interfaces 12(2), 77–89 (2018). https://doi.org/10.1007/s12193-017-0250-2

    Article  Google Scholar 

  25. Yamashita, N., Kaji, K., Kuzuoka, H., Hirata, K.: Improving visibility of remote gestures in distributed tabletop collaboration. In: Proceedings of the ACM Conference on Computer Supported Cooperative Work (CSCW 2011), Hangzhou, China, pp. 95–104 (2011)

    Google Scholar 

  26. Kirk, S.D., Fraser, D.S.: Comparing remote gesture technologies for supporting collaborative physical tasks. In: Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems (CHI 2006). Montréal, Québec, Canada, pp. 1191–1200 (2006)

    Google Scholar 

  27. Alem, L., Tecchia, F., Huang, W.: HandsOnVideo: towards a gesture based mobile AR system for remote collaboration. In: Alem, L., Huang, W. (eds.) Recent Trends of Mobile Collaborative Augmented Reality, pp. 135–148. Springer, New York (2011). https://doi.org/10.1007/978-1-4419-9845-3_11

    Chapter  Google Scholar 

  28. Herskovitz, J., Ofek, E., Lasecki, W.S., Fourney, A.: Opportunities for in-home augmented reality guidance. In: Proceedings of the ACM Conference on Human Factors in Computing Systems CHI Extended Abstracts, Glasgow, UK, pp. 1–6 (2019)

    Google Scholar 

  29. Gurevich, P., Lanir, J., Cohen, B., Stone, R.: TeleAdvisor: a versatile augmented reality tool for remote assistance. In: Proceedings of the ACM Conference on Human Factors in Computing Systems CHI, Austin, Texas, USA, pp. 619–622 (2012)

    Google Scholar 

  30. Zubizarreta, J., Aguinaga, I., Amundarain, A.: A framework for augmented reality guidance in industry. Int. J. Adv. Manuf. Technol. 102(9–12), 4095–4108 (2019). https://doi.org/10.1007/s00170-019-03527-2

    Article  Google Scholar 

  31. Didier, J.-Y., et al.: AMRA: augmented reality assistance for train maintenance tasks. In: Proceedings of the 4th ACM/IEEE International Symposium on Mixed and Augmented Reality (ISMAR 2005): Workshop Industrial Augmented Reality, Vienna, Austria, pp. 1–10 (2005)

    Google Scholar 

  32. Schröder, M., Ritter, H.J.: Deep learning for action recognition in augmented reality assistance systems. In: Proceedings of SIGGRAPH, Los Angeles, California, USA, pp. 1–2 (2017)

    Google Scholar 

  33. Hu, R., Dollár, P., He, K., Darrell, T., Girshick, R.B.: Learning to segment every thing. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, Utah, USA, pp. 4233–4241 (2018)

    Google Scholar 

  34. Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1137–1149 (2017)

    Article  Google Scholar 

  35. Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 779–788 (2016)

    Google Scholar 

  36. Lin, T.-Y., Goyal, P., Girshick, R.B., He, H., Dollar, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), Venice, Italy, pp. 2999–3007 (2017)

    Google Scholar 

  37. Ro, H., Park, Y.-J., Byun, J., Han, T.-D.: Display methods of projection augmented reality based on deep learning pose estimation. In: Proceedings of the ACM SIGGRAPH 2019 Posters, Los Angeles, California, USA, pp. 1–2 (2019)

    Google Scholar 

  38. Laib, L., Allili, M.-S., Ait-Aoudia, S.: A probabilistic topic model for event-based image classification and multi-label annotation. Sig. Process. Image Commun. 76, 283–294 (2019)

    Article  Google Scholar 

  39. Ouyed, O., Allili, M.-S.: Feature weighting for multinomial kernel logistic regression and application to action recognition. Neurocomputing 275, 1752–1768 (2018)

    Article  Google Scholar 

  40. Akgul, O., Penekli, H.I., Genc, Y.: Applying deep learning in augmented reality tracking. In: Proceedings of the 12th International Conference on Signal-Image Technology & Internet-Based Systems (SITIS), Naples, Italy, pp. 47–54 (2016)

    Google Scholar 

  41. Zhu, J., Ong, S.K., Nee, A.Y.C.: An authorable context-aware augmented reality system to assist the maintenance technicians. Int. J. Adv. Manuf. Technol. 66, 1699–1714 (2013). https://doi.org/10.1007/s00170-012-4451-2

    Article  Google Scholar 

  42. Essig, K., Strenge, B., Schack, T.: ADAMAAS: towards smart glasses for mobile and personalized action assistance. In: Proceedings of the 9th ACM International Conference on Pervasive Technologies Related to Assistive Environments (PETRA), Corfu, Greece, pp. 1–4 (2016)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. National Research Council of Canada, Ottawa, Canada

    Jean-François Lapointe & Heather Molyneaux

  2. University of Quebec in Outaouais, Gatineau, Canada

    Mohand Saïd Allili

Authors
  1. Jean-François Lapointe
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Heather Molyneaux
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohand Saïd Allili
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jean-François Lapointe .

Editor information

Editors and Affiliations

  1. U.S. Army Research Laboratory, Aberdeen Proving Ground, MD, USA

    Jessie Y. C. Chen

  2. U.S. Army Combat Capabilities Development Command, Orlando, FL, USA

    Gino Fragomeni

Rights and permissions

Reprints and permissions

Copyright information

© 2020 NRC Canada

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Lapointe, JF., Molyneaux, H., Allili, M.S. (2020). A Literature Review of AR-Based Remote Guidance Tasks with User Studies. In: Chen, J.Y.C., Fragomeni, G. (eds) Virtual, Augmented and Mixed Reality. Industrial and Everyday Life Applications. HCII 2020. Lecture Notes in Computer Science(), vol 12191. Springer, Cham. https://doi.org/10.1007/978-3-030-49698-2_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-49698-2_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-49697-5

  • Online ISBN: 978-3-030-49698-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation