Skip to main content
SpringerLink
Log in

Assessing the Human Factor of Cybersecurity: Can Surveys Tell the Truth?

  • Conference paper
  • First Online:
HCI International 2020 - Late Breaking Papers: User Experience Design and Case Studies (HCII 2020)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 12423))

Included in the following conference series:

Abstract

Survey-based measuring plays an important role in exploring human behavior. In organizational context, self-reporting of behaviors, attitudes, norms etc. can often lead to people responding in line with expectations rather than reality. Particularly when answering sensitive questions, respondents can disguise the truth for various reasons. This is called a social desirability effect (SDE) and poses a key problem in the field of behavioral studies because it can significantly bias the findings of research. A number of methods to prevent or detect SDE exist. The aim of the paper is to test selected techniques for decreasing SDE in survey-based measuring of information security behavior and to propose an improved scale, minimally susceptible to SDE. We used a cross-sectional survey design with a split-ballot experiment across three companies of critical infrastructure in Slovenia (n = 414). Four groups of employees received versions of information security behavior scale with different combinations of negative, positive and forgiving item wording. No universal group and item type effect of forgiving and alternating item wording was found with testing of the Balanced Inventory of Desirable Responding (BIDR) scale. However, it turns out that the content of items matters because one of methods perform differently for different types of behavioral items. Moreover, the part of analysis showed that combination of forgiving and alternating item wording might be effective in minimizing SDE. Items with best properties were chosen to establish new information security behavior scale. The majority of items were chosen from groups with alternating item wording, especially the one combining positive and forgiving items.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The methodological literature in the field of behavioral sciences suggests that criterion validity coefficient is expected to be up to 0.6 [12]. Although, according to some newest literature, the criterion validity is achieved if the coefficient is above 0.7 [35] or at least above 0.5 [41]. Anyway the majority max out at around 0.3 [6, 41].

References

  1. Belli, R.F., Moore, S.E., Van Hoewyk, J.: An experimental comparison of question forms used to reduce vote overreporting. Electoral. Stud. 25(4), 751–759 (2006). https://doi.org/10.1016/j.electstud.2006.01.001

    Article  Google Scholar 

  2. Bradburn, N.M., Sudman, S., Wansink, B.: Asking questions: the definitive guide to questionnaire design–for market research, political polls, and social and health questionnaires, 2nd edn. Wiley, San Francisco (2004)

    Google Scholar 

  3. Briggs, S.R., Cheek, J.M.: The role of factor analysis in the development and evaluation of personality scales. J. Pers. 54(1), 106–148 (1986). https://doi.org/10.1111/j.1467-6494.1986.tb00391.x

    Article  Google Scholar 

  4. Catania, J.A., Binson, D., Canchola, J., Pollack, L.M., Hauck, W., Coates, T.J.: Effects of interviewer gender, interviewer choice, and item wording on responses to questions concerning sexual behavior. Pub. Opin. Q. 60(3), 345–375 (1996). https://doi.org/10.1086/297758

    Article  Google Scholar 

  5. Chou, H.L., Chou, C.: An analysis of multiple factors relating to teachers’ problematic information security behavior. Comput. Hum. Behav. 65, 334–345 (2016). https://doi.org/10.1016/j.chb.2016.08.034

    Article  Google Scholar 

  6. Cohen, J.: Statistical Power Analysis for the Behavioral Sciences, 2nd edn. Academic Press, New York (2013)

    Book  Google Scholar 

  7. Cox, J.: Information systems user security: a structured model of the knowing–doing gap. Comput. Hum. Behav. 28(5), 1849–1858 (2012). https://doi.org/10.1016/j.chb.2012.05.003

    Article  Google Scholar 

  8. DePoy, E., Gitlin, L.N.: Introduction to Research: Understanding and Applying Multiple Strategies, 6th edn. Elsevier, St. Louis (2019)

    Google Scholar 

  9. Fisher, R.J.: Social desirability bias and the validity of indirect questioning. J. Consum. Res. 20(2), 303–315 (1993). https://doi.org/10.1086/209351

    Article  Google Scholar 

  10. Floyd, J., Fowler, Jr.: Improving Survey Questions: Design and Evaluation. SAGE, Thousand Oaks (2005)

    Google Scholar 

  11. Groves, R.M., Fowler Jr., F.J., Couper, M.P., Lepkowski, J.M., Singer, E., Tourangeau, R.: Survey Methodology, 2nd edn. Wiley, San Francisco (2009)

    MATH  Google Scholar 

  12. Guilford, J.P.: Fundamental Statistics in Psychology and Education, 5th edn. McGraw-Hill, New-York (1973)

    MATH  Google Scholar 

  13. Guo, K.H.: Security-related behavior in using information systems in the workplace: a review and synthesis. Comput. Secur. 32, 242–251 (2013). https://doi.org/10.1016/j.cose.2012.10.003

    Article  Google Scholar 

  14. Hart, C.M., Ritchie, T.D., Hepper, E.G., Gebauer, J.E.: The balanced inventory of desirable responding short form (BIDR-16). Sage Open 5(4), 1–9 (2015). https://doi.org/10.1177/2158244015621113

    Article  Google Scholar 

  15. Herath, T., Rao, H.R.: Encouraging information security behaviors in organizations: role of penalties, pressures and perceived effectiveness. Decis. Support Syst. 47(2), 154–165 (2009). https://doi.org/10.1016/j.dss.2009.02.005

    Article  Google Scholar 

  16. Hu, Q., Dinev, T., Hart, P., Cooke, D.: Managing employee compliance with information security policies: the critical role of top management and organizational culture. Decis. Sci. 43(4), 615–660 (2012). https://doi.org/10.1111/j.1540-5915.2012.00361.x

    Article  Google Scholar 

  17. Johnston, A.C., Warkentin, M., McBride, M., Carter, L.: Dispositional and situational factors: influences on information security policy violations. Eur. J. Inf. Syst. 25(3), 231–251 (2016). https://doi.org/10.1057/ejis.2015.15

    Article  Google Scholar 

  18. Kaminska, O., Foulsham, T.: Understanding sources of social desirability bias in different modes: evidence from eye-tracking. In: ISER Working Paper Series 2013-04, pp. 2–11. Institute for social and economic research, Essex (2013)

    Google Scholar 

  19. Karjalainen, M., Siponen, M., Sarker, S.: Toward a stage theory of the development of employees’ information security behavior. Comput. Secur. 93, 1–12 (2020). https://doi.org/10.1016/j.cose.2020.101782

    Article  Google Scholar 

  20. Kaur, J., Mustafa, N.: Examining the effects of knowledge, attitude and behaviour on information security awareness: a case on SME. In: 3rd International Conference on Research and Innovation in Information Systems – 2013 (ICRIIS 2013), pp. 286–290. IEEE (2013). https://doi.org/10.1109/icriis.2013.6716723

  21. Kim, S.S., Kim, Y.J.: The effect of compliance knowledge and compliance support systems on information security compliance behavior. J. Knowl. Manag. 21(4), 986–1010 (2017). https://doi.org/10.1108/jkm-08-2016-0353

    Article  Google Scholar 

  22. Kruger, H.A., Kearney, W.D.: A prototype for assessing information security awareness. Comput. Secur. 25(4), 289–296 (2006). https://doi.org/10.1016/j.cose.2006.02.008

    Article  Google Scholar 

  23. Kwak, D.H., Holtkamp, P., Kim, S.S.: Measuring and controlling social desirability bias: applications in information systems research. J. Assoc. Inf. Syst. 20(4), 317–345 (2019). https://doi.org/10.17005/1.jais.00537

  24. Lebek, B., Uffen, J., Neumann, M., Hohler, B., Breitner, M.H.: Information security awareness and behavior: a theory-based literature review. Manag. Res. Rev. 37(12), 1049–1092 (2014). https://doi.org/10.1108/mrr-04-2013-0085

    Article  Google Scholar 

  25. Leite, W.L., Beretvas, S.N.: Validation of scores on the marlowe-crowne social desirability scale and the balanced inventory of desirable responding. Educ. Psychol. Measur. 65(1), 140–154 (2005). https://doi.org/10.1177/0013164404267285

    Article  MathSciNet  Google Scholar 

  26. McCormac, A., Calic, D., Butavicius, M., Parsons, K., Zwaans, T., Pattinson, M.: A reliable measure of information security awareness and the identification of bias in responses. Australas. J. Inf. Syst. 21, 1–12 (2017). https://doi.org/10.3127/ajis.v21i0.1697

    Article  Google Scholar 

  27. Menard, P., Warkentin, M., Lowry, P.B.: The impact of collectivism and psychological ownership on protection motivation: a cross-cultural examination. Comput. Secur. 75, 147–166 (2018). https://doi.org/10.1016/j.cose.2018.01.020

    Article  Google Scholar 

  28. Moody, G.D., Siponen, M., Pahnila, S.: Toward a unified model of information security policy compliance. MIS Q. 42(1), 285–311 (2018). https://doi.org/10.25300/misq/2018/13853

  29. Mullins, L.: Essentials of Organisational Behaviour, 2nd edn. Pearson Education, Harlow (2008)

    Google Scholar 

  30. Nederhof, A.J.: Methods of coping with social desirability bias: a review. Eur. J. Soc. Psychol. 15(3), 263–280 (1985). https://doi.org/10.1002/ejsp.2420150303

    Article  Google Scholar 

  31. Nuno, A., John, F.A.S.: How to ask sensitive questions in conservation: a review of specialized questioning techniques. Biol. Conserv. 189, 5–15 (2015). https://doi.org/10.1016/j.biocon.2014.09.047

    Article  Google Scholar 

  32. Padayachee, K.: Taxonomy of compliant information security behavior. Comput. Secur. 31(5), 673–680 (2012). https://doi.org/10.1016/j.cose.2012.04.004

    Article  Google Scholar 

  33. Parsons, K., McCormac, A., Butavicius, M., Pattinson, M., Jerram, C.: Determining employee awareness using the Human Aspects of Information Security Questionnaire (HAIS-Q). Comput. Secur. 42, 165–176 (2014). https://doi.org/10.1016/j.cose.2013.12.003

    Article  Google Scholar 

  34. Paulhus, D.L.: Measurement and control of response bias. In Robinson, J.P., Shaver, P.R., Wrightsman, L.S. (eds.) Measures of Personality and Social Psychological Attitudes, pp. 17–59. Academic Press, San Diego (1991). https://doi.org/10.1016/b978-0-12-590241-0.50006-x

  35. Polit, D.E., Beck, C.T.: Essentials of Nursing Research, 6th edn. Lippincott Williams & Wilkins, Philadelphia (2006)

    Google Scholar 

  36. Rhodes-Ousley, M.: Information Security: the Complete Reference, 2nd edn. McGraw-Hill, New York (2013)

    Google Scholar 

  37. Robbins, S.P.: Organizational Behavior, 9th edn. Prentice-Hall International, Upper Saddle River (2001)

    Google Scholar 

  38. Rocha Flores, W., Ekstedt, M.: Shaping intention to resist social engineering through transformational leadership, information security culture and awareness. Comput. Secur. 59, 26–44 (2016). https://doi.org/10.1016/j.cose.2016.01.004

    Article  Google Scholar 

  39. Safa, N.S., Sookhak, M., Von Solms, R., Furnell, S., Ghani, N.A., Herawan, T.: Information security conscious care behaviour formation in organizations. Comput. Secur. 53, 65–78 (2015). https://doi.org/10.1016/j.cose.2015.05.012

    Article  Google Scholar 

  40. Safa, N.S., Von Solms, R., Furnell, S.: Information security policy compliance model in organizations. Comput. Secur. 56, 70–82 (2016). https://doi.org/10.1016/j.cose.2015.10.006

    Article  Google Scholar 

  41. Salkind, N.J.: Tests & Measurement for People Who (Think They) Hate Tests & Measurement, 3rd edn. SAGE, Los Angeles (2017)

    Google Scholar 

  42. Sauro, J., Lewis, J.R.: When designing usability questionnaires, does it hurt to be positive? In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 2215–2224. Association for Computing Machinery, New York (2011)

    Google Scholar 

  43. Shoemaker, P.J., Eichholz, M., Skewes, E.A.: Item nonresponse: distinguishing between don’t know and refuse. Int. J. Public Opin. Res. 14(2), 193–201 (2002). https://doi.org/10.1093/ijpor/14.2.193

    Article  Google Scholar 

  44. Šinigoj, J.: Informacijska varnostna kultura v izbranih energetskih družbah (in Slovene) (Information security culture in the selected energy companies). Master thesis, University of Ljubljana, Ljubljana (2020)

    Google Scholar 

  45. Tourangeau, R., Yan, T.: Sensitive questions in surveys. Psychol. Bull. 133(5), 859–883 (2007). https://doi.org/10.1037/0033-2909.133.5.859

    Article  Google Scholar 

  46. Tsohou, A., Karyda, M., Kokolakis, S.: Analyzing the role of cognitive and cultural biases in the internalization of information security policies: recommendations for information security awareness programs. Comput. Secur. 52, 128–141 (2015). https://doi.org/10.1016/j.cose.2015.04.006

    Article  Google Scholar 

  47. Vance, A., Siponen, M., Pahnila, S.: Motivating IS security compliance: insights from habit and protection motivation theory. Inf. Manag. 49(3–4), 190–198 (2012). https://doi.org/10.1016/j.im.2012.04.002

    Article  Google Scholar 

  48. Yazdanmehr, A., Wang, J.: Employees’ information security policy compliance: a norm activation perspective. Decis. Support Syst. 92, 36–46 (2016). https://doi.org/10.1016/j.dss.2016.09.009

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported by the Slovenian Research Agency within the “Young researchers” program [grant number P5-0168].

Author information

Authors and Affiliations

  1. Faculty of Social Sciences, University of Ljubljana, Ljubljana, Slovenia

    Špela Orehek & Gregor Petrič

  2. Plinovodi d.o.o., Ljubljana, Slovenia

    Jan Šinigoj

Authors
  1. Špela Orehek
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gregor Petrič
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jan Šinigoj
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Špela Orehek .

Editor information

Editors and Affiliations

  1. University of Crete and Foundation for Research and Technology – Hellas (FORTH), Heraklion, Crete, Greece

    Constantine Stephanidis

  2. Aaron Marcus and Associates, Berkeley, CA, USA

    Aaron Marcus

  3. World Usability Day and Bentley User Experience Center, Newton Center, MA, USA

    Elizabeth Rosenzweig

  4. Tsinghua University, Beijing, China

    Pei-Luen Patrick Rau

  5. San Jose State University, San Jose, CA, USA

    Abbas Moallem

  6. Eindhoven Univesity of Technology, Eindhoven, Noord-Brabant, The Netherlands

    Matthias Rauterberg

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Orehek, Š., Petrič, G., Šinigoj, J. (2020). Assessing the Human Factor of Cybersecurity: Can Surveys Tell the Truth?. In: Stephanidis, C., Marcus, A., Rosenzweig, E., Rau, PL.P., Moallem, A., Rauterberg, M. (eds) HCI International 2020 - Late Breaking Papers: User Experience Design and Case Studies. HCII 2020. Lecture Notes in Computer Science(), vol 12423. Springer, Cham. https://doi.org/10.1007/978-3-030-60114-0_18

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-60114-0_18

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-60113-3

  • Online ISBN: 978-3-030-60114-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation