Acessibilidade / Reportar erro

The use of serious games as an innovative educational strategy for learning cardiopulmonary resuscitation: an integrative review

Uso del juego serio como una estrategia educativa innovadora para aprender la reanimación cardiopulmonar: revisión integradora

ABSTRACT

Objective:

To analyze the scientific evidence available in literature on the use of serious games for health students to learn about cardiopulmonary resuscitation.

Methods:

Integrative review in PubMed / MEDLINE®, LILACS, Scopus and CINAHL databases. Included primary studies, in Portuguese, English or Spanish, from January 2009 to August 2019. The Rayyan software was used for study selection.

Results:

We identified 115 studies, and 8 composed the sample, categorizing two domains: : Intention of the study with regard to cardiopulmonary resuscitation through the serious game, and method used by the serious game to teach CPR.

Conclusion:

Medical students are the main audience for serious game learning, and the main intentions of the games are to compare their effectiveness with traditional methodologies and knowledge retention. The method adopted is the description of a cardiorespiratory arrest for training. The serious game proved effective for learning cardiopulmonary resuscitation.

Keyword
Cardiopulmonary resuscitation; Students; Education; Video games

RESUMEN

Objetivo:

Analizar la evidencia científica disponible en la literatura sobre el uso de juegos serios para aprender sobre la reanimación cardiopulmonar de estudiantes de salud.

Métodos:

Revisión integradora en bases de datos PubMed / MEDLINE®, LILACS, Scopus y CINAHL. Incluyeron estudios primarios, en portugués, inglés o español, de enero de 2009 a agosto de 2019. La aplicación Rayyan se utilizó para la selección.

Resultados:

Identificamos 115 estudios y 8 compusieron la muestra, categorizando: Intencionalidad del aprendizaje a través del juego serio para la Reanimación Cardiopulmonar y Método utilizado por el juego serio para el aprendizaje.

Conclusión:

El estudiante de medicina es el público principal para el aprendizaje serio del juego, y las principales intenciones son comparar su efectividad con las metodologías tradicionales y la retención de conocimientos. El método adoptado es la descripción de un paro cardiorrespiratorio para entrenamiento. El juego serio demostró ser efectivo para aprender la reanimación cardiopulmonar.

Palabra clave
Reanimación cardiopulmonar; Estudiantes; Educación; Juegos de video

RESUMO

Objetivo:

Analisar as evidências científicas disponíveis na literatura sobre a utilização de serious game para a aprendizagem sobre ressuscitação cardiopulmonar de estudantes na área da saúde.

Métodos:

Revisão integrativa nas bases de dados PubMed/MEDLINE®, LILACS, Scopus e CINAHL. Incluídos estudos primários, em português, inglês ou espanhol, de janeiro de 2009 a agosto de 2019. Utilizou-se o aplicativo Rayyan para seleção.

Resultados:

Identificou-se 115 estudos e 8 compuseram a amostra, categorizando-se dois domínios: Intencionalidade do estudo quanto a aprendizagem por meio do serious game para a Ressuscitação Cardiopulmonar e Método utilizado pelo serious game para aprendizagem.

Conclusão:

O graduando de medicina é o principal público para aprendizagem com serious game, e as principias intenções são a comparação da sua efetividade com metodologias tradicionais e a retenção de conhecimento. O método adotado é a descrição de uma parada cardiorrespiratória para treinamento. O serious game demonstrou-se efetivo para aprendizagem da ressuscitação cardiopulmonar.

Palavras-chave
Reanimação cardiopulmonar; Estudantes; Educação; Jogos de vídeo

INTRODUCTION

Recently, a new generation was identified: the "Y" generation, capable of naturally handling technology and, as a result, classified as a group of "digital natives", standing out due to their creativity, curiosity, and restlessness with regards to learning11. Decesaro L, Severo EA, Guimarães JC, Girardi G. A influência da geração x e y na inovação de produto e processo na indústria metalmecânica do Rio Grande do Sul. Rev Bras Gestão Inov. 2018 [cited 2019 Jul 25];5(2):30-58. Available from: Available from: http://ucs.br/etc/revistas/index.php/RBGI/article/view/5160/3105
http://ucs.br/etc/revistas/index.php/RBG...
. In order to change the traditional teaching methods that are still used and guide this generation, active teaching strategies have emerged in the 20th century which have an effective participation of the students (22. Oliveira CM, Marques VF, Schreck RS. Aplicação de metodologia ativa no processo de ensino-aprendizagem: relato de experiência. Rev Eletrôn Pesquiseduca, 2017 [cited 2019 Jul 25];9(19):674-84. Available from: Available from: http://periodicos.unisantos.br/index.php/pesquiseduca/article/view/633
http://periodicos.unisantos.br/index.php...
-44. Freire P. Pedagogia da autonomia: saberes necessários à prática educativa. 51. ed. Rio de Janeiro: Paz e Terra; 2015..

One of the technological strategies for learning, which has been standing out, involves the serious games55. Lewis ZH, Swartz MC, Lyons EJ. What’s the point?: a review of reward systems implemented in gamification interventions. Games Health J. 2016;5(2):93-9. doi: https://doi.org/10.1089/g4h.2015.0078
https://doi.org/10.1089/g4h.2015.0078...
-66. Saunder L , Berridge, EJ. Immersive simulated reality scenarios for enhancing students’ experience of people with learning disabilities across all fields of nurse education. Nurse Educ Pract. 2015;15(6):397-402. doi: https://doi.org/10.1016/j.nepr.2015.04.007
https://doi.org/10.1016/j.nepr.2015.04.0...
, especially in the field of health. These are educational games, developed to stimulate learning in an interactive and intuitive way, accelerating critical thought to deal with clinical complications and plan assistance77. Romero M, Usart M, Ott M. Can serious games contribute to developing and sustaining 21st century skills? Games Culture. 2014;10(2):148-77. doi: https://doi.org/10.1177/1555412014548919
https://doi.org/10.1177/1555412014548919...
-99. Kapralos B, Moussa F, Dubrowski A. An overview of virtual simulation and serious gaming for surgical education and training. In: Anthony LB, Sheryl B, Lakhmi CJ, editors. Technologies of inclusive well-being. Berlin: Heidelberg: Springer; 2014. p. 289-306. Studies in Computational Intelligence, vol. 536. doi: https://doi.org/10.1007/978-3-642-45432-5_14
https://doi.org/10.1007/978-3-642-45432-...
. A variety of themes in health, which can be taught using serious games, can be identified, among which cardiopulmonary resuscitation (CPR) stands out1010. Ricciardi F, De Paolis LT. A comprehensive review of serious games in health professions. Int J Comput Game Technol. 2014:ID787968. doi: https://doi.org/10.1155/2014/787968
https://doi.org/10.1155/2014/787968 ...
.

The worldwide incidence of cardiopulmonary arrest (CPA) varies from 20 to 140 per 100 thousand people, and nearly 1.6 in each thousand hospital admissions progress into CPA1111. Mozaffarian D, Benjamin EJ, Go AS, Arnett DK, Blaha MJ, Cushman M, et al. Executive summary: Heart disease and stroke statistics-2016 update: a report from the American Heart Association. Circulation. 2016;133(4):447-54. doi: https://doi.org/10.1161/CIR.0000000000000366
https://doi.org/10.1161/CIR.000000000000...
-1212. Aranzábal-Alegría G, Verastegui-Diaz A, Quiñones-Laveriano DM, Quintana-Mendoza LY, Vilchez-Cornejo J, Espejo CB, et al. Factors influencing the level of knowledge of cardiopulmonary resuscitation in hospitals in Peru. Rev Colomb Anestesiol. 2017 [cited 2019 Jul 25];45(2):114-21. Available from: Available from: http://www.scielo.org.co/pdf/rca/v45n2/v45n2a07.pdf
http://www.scielo.org.co/pdf/rca/v45n2/v...
. The survival rate post CPA varies from 2% to 11% in hospitals, which shows that the processes and the implementation of new teaching and learning strategies is necessary1212. Aranzábal-Alegría G, Verastegui-Diaz A, Quiñones-Laveriano DM, Quintana-Mendoza LY, Vilchez-Cornejo J, Espejo CB, et al. Factors influencing the level of knowledge of cardiopulmonary resuscitation in hospitals in Peru. Rev Colomb Anestesiol. 2017 [cited 2019 Jul 25];45(2):114-21. Available from: Available from: http://www.scielo.org.co/pdf/rca/v45n2/v45n2a07.pdf
http://www.scielo.org.co/pdf/rca/v45n2/v...
.

Different studies evaluated the learning of CPR, and the results indicate that the abilities become less sharp within 3 to 6 months after the initial training1313. Sena DP , Fabrício DD, Silva VD , Bodanese LC, Franco AR. Comparative evaluation of video-based on-line course versus serious game for training medical students in cardiopulmonary resuscitation: a randomised trial. PLoS One. 2019;14(4):e0214722. doi: https://doi.org/10.1371/journal.pone.0214722
https://doi.org/10.1371/journal.pone.021...
-1515. Kavelak HL, Hollands JM, Bingham AL. Student-led cardiopulmonary resuscitation education to lay providers results in successful knowledge acquisition and skill performance. J Allied Health. 2019;48(1):18-21.. Therefore, developing techniques to update and maintain the knowledge related to CPR is as important as teaching it. In this context, the serious games can be adequate tools, since they are not only used to complement learning, but also to expose students to new situations during the teaching of CPR, thus updating the knowledge1616. Drummond D, Delval P, Abdenouri S, Truchot J, Ceccaldi PF, Plaisance P, et al. Serious game versus online course for pretraining medical students before a simulation-based mastery learning course on cardiopulmonary resuscitation. Eur J Anaesthesiol. 2017;34(12):836-44. doi: https://doi.org/10.1097/eja.0000000000000675
https://doi.org/10.1097/eja.000000000000...
.

Considering the important of teaching and learning CPR, and the use of innovative technologies for the development of professional competence in the field of health with regards to this science1717. Youngquist ST , Kenneth Scheppke A , Pepe PE. Supportive technology in the resuscitation of out-of-hospital cardiac arrest patients. Curr Opin Crit Care. 2017;23(3):209-14. doi: https://doi.org/10.1097/MCC.0000000000000409
https://doi.org/10.1097/MCC.000000000000...
-1919. Silva AC, Bernardes A, Évora YD, Dalri MC, Silva AR, Sampaio CS. Development of a virtual learning environment for cardiorespiratory arrest training. Rev Esc Enferm USP. 2016;50(6):990-7. doi: https://doi.org/10.1590/s0080-623420160000700016
https://doi.org/10.1590/s0080-6234201600...
, and understanding, from the above, that serious games are educational technologies that can promote this articulation (innovation/teaching of CPR), studying the theme in depth can contribute for the understanding of this tool, for the understanding of its use in the current settings, and its adoption by health educators. This premise instigated us to ask the following guiding questions: What are the main intentions of serious games, when targeted at cardiopulmonary resuscitation, considering the teaching and learning process, and what is the teaching method used to reach this objective? Considering this context, the objective of this study is analyzing scientific evidences available in literature about the use of serious games for health students to learn about CPR?

METHODS

This is an integrative literature review about the application of a digital, proactive, and innovative technology, the serious game, targeted at teaching CPR. To reach this objective, the following stages were followed: selection of the research question, definition of the sample, definition of the characteristics of the primary research, analysis of the findings, interpretation of the results, and reproduction of the review2020. Ganong LH. Integrative reviews of nursing research. Res Nurs Health. 1987;10(1):1-11. doi: https://doi.org/10.1002/nur.4770100103
https://doi.org/10.1002/nur.4770100103...
.

The searches were carried from February to April 2019. Primarily, the guiding question of the review was elaborated through the use of the PICO strategy (Patient-Intervention-Comparison-Outcomes). The P, in the acronym, refers to the patients, that is, to the population addressed, made up by health students; the letter I refers to the intervention, that is, the use of the serious games; the letter O, in turn, referred to the outcome, which here is the learning of CPR. The letter C in the acronym was not used in this study. As a result, the following question was elaborated: What are the evidences available in literature with regard to the use of serious games to teach CPR to health students?

With regard to the stages described above, more specifically, the second and third ones, sample definition and definition of the characteristics of the primary research, the search strategies were specified for the databases selected, which were PubMed® and Scopus. The descriptors used, which were from the Medical Subjects Headings (MESH), were “Cardiopulmonary Resuscitation”; “Students, Health Occupations”, and the keyword was “Serious Game”. The search strategy use was: (Students OR Student OR “School Enrollment” OR “Enrollment, School” OR “Enrollments, School” OR “School Enrollments”) AND (“Video games” OR “Game, Video” OR “Games, Video” OR “Video Game” OR “Computer Games” OR “Computer Game” OR “Game, Computer” OR “Games, Computer”) AND (“Serious Game”) (“Video games” OR “Game, Video” OR “Games, Video” OR “Video Game” OR “Computer Games” OR “Computer Game” OR “Game, Computer” OR “Games, Computer” AND “Serious Game”) AND (Learning OR Phenomenography OR “Memory Training” OR “Training, Memory”) AND (“Cardiopulmonary Resuscitation” OR “Resuscitation, Cardiopulmonary” OR CPR OR “Cardio-Pulmonary Resuscitation” OR “Resuscitation, Cardio-Pulmonary” OR “Code Blue” OR “Mouth-to-Mouth Resuscitation” OR “Mouth to Mouth Resuscitation” OR “Mouth-to-Mouth Resuscitations” OR “Resuscitation, Mouth-to-Mouth” OR “Resuscitations, Mouth-to-Mouth” OR “Basic Cardiac Life Support” OR “Life Support, Basic Cardiac”).

In the databases Cumulative Index to Nursing and Allied Health Literature (CINAHL), the descriptors, identified in “Títulos” (Titles), were: “Resuscitation, Cardiopulmonary”; “Students, Health Occupations”; “Education, Health Sciences” and the keyword was “Serious Game”, with the following strategy: (Students OR “Students, College”) AND (“Video Games” OR “Exergames”) AND (“Serious Game”) (“Video Games” OR “Exergames”) AND (“Serious Game”) AND (Knowledge OR “Health Knowledge”) AND (“Resuscitation, Cardiopulmonary”).

For the research in the database Literatura Latino-Americana e do Caribe em Ciências da Saúde (LILACS), descriptors came from the Descritores em Saúde (Health Descriptors - DeCS), and were: “Reanimação Cardiopulmonar”, “Estudante”. The keyword was “Serious game”, coupled with its equivalent terms in English and Spanish. The following search strategy was used: (Estudante) AND (“Jogos de Video”) AND (“Serious Game”) AND (Aprendizagem) AND (“Ressuscitação Cardiopulmonar”) (Estudiantes) AND (“Juegos de Video”) AND (“Serious game”) AND (Aprendizaje) AND (“Reanimación Cardiopulmonar”) (Students) AND (“Jogos de Video”) AND (“Serious game”) AND (Learning) AND (“Cardiopulmonary Resuscitation”)

The term "serious game" was used as a keyword so that the results of the research would be intrinsically related to games specific for learning, since finding researches that were related to the objective of this study proved to be challenging.

All scientific researches found were included, including quantitative or qualitative ones, published in any discipline connected to health, and containing data regarding teaching strategies, learning, serious games, and cardiopulmonary reanimation, published from 2009 to May 2019, a time period selected because 2009 was the year with the first scientific publications about serious games in the teaching of CPR, be it in Portuguese, English, or Spanish, considering those that were published in scientific journals and available in electronic form. Literature reviews, editorials, reviews, experience reports, case studies, theoretical reflections, dissertations, theses, monographs, abstracts from the annals of events, publications in websites, ads, articles, and abstracts that were not available online were excluded. The same was done to all gray literature, documents produced by governments, universities, companies, and industries, in print and without the control of editors.

There were three stages in the selection of the studies. The first involved the triage of the articles. Titles and abstracts were evaluated by two professionals using the Rayyan software, which makes the initial triage process of abstract and title reading faster, using a semi-automated process and incorporating a high level of usability in this process2121. Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan - a web and mobile app for systematic reviews. Syst Rev. 2016;5(1):210. doi: https://doi.org/10.1186/s13643-016-0384-4
https://doi.org/10.1186/s13643-016-0384-...
. During the second stage of the selection, it was verified whether the studies selected by the researchers were the same, which led 15 investigations to be sent to a third party, responsible to make the final decision about their inclusion or exclusion. In the last stage, the works were read in full, to determine the final sample. An instrument2222. Ursi ES, Galvão CM. Prevenção de lesões de pele no perioperatório: revisão integrativa da literatura. Rev Latino-Am Enfermagem. 2006;14(1):124-31. doi: https://doi.org/10.1590/S0104-11692006000100017
https://doi.org/10.1590/S0104-1169200600...
was used, specifying the following items: identification of the article, title, authors, country of origin, language, and year of publication, objectives, method, and results. The method for the development of the serious game was also identified, as well as its target-audience and the effectiveness of the game for the teaching-learning process of CPR. Finally, the level of evidence of the studies was classified2323. Galvão CM. Níveis de evidência [Editorial]. Acta Paul Enferm. 2006; [cited 2019 Oct 29];19(2):V. Available from: Available from: http://www.scielo.br/pdf/ape/v19n2/a01v19n2.pdf
http://www.scielo.br/pdf/ape/v19n2/a01v1...
. The articles were selected according to Figure 1 2424. Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Int J Surg. 2010;8(5):336-41. doi: https://doi.org//10.1016/j.ijsu.2010.02.007
https://doi.org//10.1016/j.ijsu.2010.02....
.

Figure 1 -
Flowchart of the process of identification, selection, and inclusion of the studies, elaborated according to the PRISMA recommendations.

RESULTS

The next stages were the interpretation of results and review reproduction. Eight primary studies were considered eligible to make up the sample of this integrative review. Chart 1, below, characterizes the studies according to their identification number, country of origin, language, and classification of the level of evidence.

Chart 1 -
Characterization of studies that made up the sample

Chart 2 shows the objective, methodological design, effectiveness, and the teaching-learning method adopted by the serious game, considering the studies that were part of the sample of this research.

Chart 2 -
Characterization of the studies with regards to their intentions for learning Cardiopulmonary Resuscitation through the serious game, methodological path, effectiveness, and method used by the serious game for the process of CPR teaching-learning.

DISCUSSION

The scientific production on the use of serious games as innovative strategies to learn CPR in the field of health is still incipient, but has grown in the period under study, though mostly distributed in the international educational setting.

The serious games found in the study were, in most cases, directed at medicine students, followed by those targeted at nursing students. A smaller number was targeted at other health professionals. This could be explained due to the increase in the use of educational technologies for teaching in medicine graduation schools, when compared to traditional methods, coupled with the expressive growth in the number of computer strategies available for medical education throughout the world3333. Kapralos B, Fisher S, Clarkson J, Oostveen R. A course on serious game design and development using an online problem-based learning approach. Interactive Technol Smart Educ. 2015;12(2):116-36. doi: https://doi.org/10.1108/ITSE-10-2014-0033
https://doi.org/10.1108/ITSE-10-2014-003...
-3434. Gorbanev I, Agudelo-Londoño S, González RA, Cortes A, Pomares A, Delgadillo V, et al. A systematic review of serious games in medical education: quality of evidence and pedagogical strategy. Med Educ Online. 2018;23(1):1438718. doi: https://doi.org/10.1080/10872981.2018.1438718
https://doi.org/10.1080/10872981.2018.14...
.

The experimental and quasi-experimental designs were the most frequent in this review, standing out with levels of evidence of 2 and 3, respectively. This could be related to the need to compare the effectiveness of different methodological teaching strategies regarding CPR, for the choice of better practices3535. McCoy EC, Rahman A, Rendon JC, Anderson CL, Langdorf, Lotfipour S, et al. Randomized controlled trial of simulation vs. standard training for teaching medical students high-quality cardiopulmonary resuscitation. West J Emerg Med. 2019;20(1):15-22. doi: https://doi.org/10.5811/westjem.2018.11.39040
https://doi.org/10.5811/westjem.2018.11....
.

The findings were categorized in two domains: Intention of the study with regard to Cardiopulmonary Resuscitation through the serious game, and method used by the serious game to teach CPR.

The first category took into account the main objectives of the studies that used serious games to teach CPR, among which stood out the need to compare traditional teaching strategies2525. Boada I, Rodriguez-Benitez A, Garcia-Gonzalez JM, Olivet J, Carreras V, Sbert M. Using a serious game to complement CPR instruction in a nurse faculty. Comput Methods Programs Biomed. 2015;122(2):282-91. doi: https://doi.org/10.1016/j.cmpb.2015.08.006
https://doi.org/10.1016/j.cmpb.2015.08.0...
,2929. Cook NF, McAloon T, O'Neill P, Beggs R. Impact of a web based interactive simulation game (PULSE) on nursing students' experience and performance in life support training: a pilot study. Nurse Educ Today. 2012;32(6):714-20. doi: https://doi.org/10.1016/j.nedt.2011.09.013
https://doi.org/10.1016/j.nedt.2011.09.0...
-3131. Sena DP, Fabrício DD, Silva VD, Bodanese LC, Franco AR. Comparative evaluation of video-based online course versus serious game for training medical students in cardiopulmonar resuscitation: a randomised trial. PloS One. 2019;14(4):e0214722. doi: https://doi.org/10.1371/journal.pone.0214722
https://doi.org/10.1371/journal.pone.021...
. The identification of the positive results of the serious game for learning CPR2525. Boada I, Rodriguez-Benitez A, Garcia-Gonzalez JM, Olivet J, Carreras V, Sbert M. Using a serious game to complement CPR instruction in a nurse faculty. Comput Methods Programs Biomed. 2015;122(2):282-91. doi: https://doi.org/10.1016/j.cmpb.2015.08.006
https://doi.org/10.1016/j.cmpb.2015.08.0...
,2929. Cook NF, McAloon T, O'Neill P, Beggs R. Impact of a web based interactive simulation game (PULSE) on nursing students' experience and performance in life support training: a pilot study. Nurse Educ Today. 2012;32(6):714-20. doi: https://doi.org/10.1016/j.nedt.2011.09.013
https://doi.org/10.1016/j.nedt.2011.09.0...
and in uncertain results with regards to their effectiveness in the retention of knowledge3030. Drummond D, Delval P, Abdenouri S, Truchot J, Ceccaldi PF, Plaisance P, et al. Serious game versus online course for pretraining medical students before a simulation-basedmastery learning course on cardiopulmonary resuscitation: a randomised controlled study. Eur J Anaesthesiol. 2017;34(12):836-44. doi: https://doi.org/10.1097/EJA.0000000000000675
https://doi.org/10.1097/EJA.000000000000...
,3131. Sena DP, Fabrício DD, Silva VD, Bodanese LC, Franco AR. Comparative evaluation of video-based online course versus serious game for training medical students in cardiopulmonar resuscitation: a randomised trial. PloS One. 2019;14(4):e0214722. doi: https://doi.org/10.1371/journal.pone.0214722
https://doi.org/10.1371/journal.pone.021...
, showing variability in the outcomes.

A study carried out in the Descartes University, in Paris, aiming to compare a traditional CPR course and the effectiveness of a serious game with 82 medicine students, did not find the game to have statistically superior results when compared to the traditional course3030. Drummond D, Delval P, Abdenouri S, Truchot J, Ceccaldi PF, Plaisance P, et al. Serious game versus online course for pretraining medical students before a simulation-basedmastery learning course on cardiopulmonary resuscitation: a randomised controlled study. Eur J Anaesthesiol. 2017;34(12):836-44. doi: https://doi.org/10.1097/EJA.0000000000000675
https://doi.org/10.1097/EJA.000000000000...
. On the other hand, a study with 109 undergraduate nursing students in the University of Girona, in Spain, compared the use of a serious game to teach CPR with a classic teaching methodology, made up by self-directed learning, and found statistically significant differences between the groups tested, with a better performance of the students who used the serious game2525. Boada I, Rodriguez-Benitez A, Garcia-Gonzalez JM, Olivet J, Carreras V, Sbert M. Using a serious game to complement CPR instruction in a nurse faculty. Comput Methods Programs Biomed. 2015;122(2):282-91. doi: https://doi.org/10.1016/j.cmpb.2015.08.006
https://doi.org/10.1016/j.cmpb.2015.08.0...
.

Further investigation on this subject is clearly needed, through well-designed clinical trials that can offer more conclusive outcomes about the adoption of serious games for the learning of CPR to be effective3636. Middeke A, Anders S, Schuelper M, Raupach T, Schuelper N. Training of clinical reasoning with a Serious Game versus small-group problem-based learning: a prospective study. PLoS One. 2018 Sep 11;13(9):e0203851. doi: https://doi.org/10.1371/journal.pone.0203851
https://doi.org/10.1371/journal.pone.020...
.

Still in this context, this research strongly showed another objective of using serious games to teach CPR, that is, the retention of cognitive knowledge2629. Creutzfeldt J, Hedman L, Felländer-Tsai L. Effects of pre-training using serious game technology on CPR performance--an exploratory quasi-experimental transfer study. Scand J Trauma Resusc Emerg Med. 2012;20:79. doi: https://doi.org/10.1186/1757-7241-20-79
https://doi.org/10.1186/1757-7241-20-79...
,2727. Creutzfeldt J, Hedman L, Medin C, Heinrichs WL, Felländer-Tsai L. Exploring virtual worlds for scenario-based repeated team training of cardiopulmonary ressuscitation in medical students. J Med Internet Res. 2010;12(3):e38. doi: https://doi.org/10.2196/jmir.1426
https://doi.org/10.2196/jmir.1426...
,3232. Cutumisu M, Brown MRG, Fray C, Schmölzer GM. Growth mindset moderates the effect of the neonatal resuscitation program on performance in a computer-based game training simulation. Front Pediatr. 2018;6:195. doi: https://doi.org/10.3389/fped.2018.00195
https://doi.org/10.3389/fped.2018.00195 ...
. In the case of most serious games identified, the knowledge was not retained by the players2629. Creutzfeldt J, Hedman L, Felländer-Tsai L. Effects of pre-training using serious game technology on CPR performance--an exploratory quasi-experimental transfer study. Scand J Trauma Resusc Emerg Med. 2012;20:79. doi: https://doi.org/10.1186/1757-7241-20-79
https://doi.org/10.1186/1757-7241-20-79...
,3232. Cutumisu M, Brown MRG, Fray C, Schmölzer GM. Growth mindset moderates the effect of the neonatal resuscitation program on performance in a computer-based game training simulation. Front Pediatr. 2018;6:195. doi: https://doi.org/10.3389/fped.2018.00195
https://doi.org/10.3389/fped.2018.00195 ...
. Corroborating this statement, a study carried out in Sweden with 12 medicine students, to investigate the influence of the use of a serious game to teach CPR, showed that there was a loss in the cognitive performance related to the retention of knowledge as time progressed2629. Creutzfeldt J, Hedman L, Felländer-Tsai L. Effects of pre-training using serious game technology on CPR performance--an exploratory quasi-experimental transfer study. Scand J Trauma Resusc Emerg Med. 2012;20:79. doi: https://doi.org/10.1186/1757-7241-20-79
https://doi.org/10.1186/1757-7241-20-79...
.

A recent systematic review concluded that, to promote the retention of knowledge in the teaching-learning of CPR, different validated teaching strategies must be combined, preferably with the inclusion of popular music and feedback, and applied every 3 to 6 months, although further investigations are necessary to determine an exact period3737. Riggs M, Franklin R, Saylany L. Associations between cardiopulmonary resuscitation (CPR) knowledge, self-efficacy, training history and willingness to perform CPR and CPR psychomotor skills: a systematic review. Resuscitation. 2019;138:259-72. doi: https://doi.org/10.1016/j.resuscitation.2019.03.019
https://doi.org/10.1016/j.resuscitation....
.

The method used by the serious game for CPR learning was also addressed as a category for this study. The extra-hospital setting for CPA attention was addressed in most studies2525. Boada I, Rodriguez-Benitez A, Garcia-Gonzalez JM, Olivet J, Carreras V, Sbert M. Using a serious game to complement CPR instruction in a nurse faculty. Comput Methods Programs Biomed. 2015;122(2):282-91. doi: https://doi.org/10.1016/j.cmpb.2015.08.006
https://doi.org/10.1016/j.cmpb.2015.08.0...
-2828. Buttussi F, Pellis T, Cabas Vidani A, Pausler D, Carchietti E, Chittaro L. Evaluation of a 3D serious game for advanced life support retraining. Int J Med Inform. 2013;82(9):798-809. doi: https://doi.org/10.1016/j.ijmedinf.2013.05.007
https://doi.org/10.1016/j.ijmedinf.2013....
,3131. Sena DP, Fabrício DD, Silva VD, Bodanese LC, Franco AR. Comparative evaluation of video-based online course versus serious game for training medical students in cardiopulmonar resuscitation: a randomised trial. PloS One. 2019;14(4):e0214722. doi: https://doi.org/10.1371/journal.pone.0214722
https://doi.org/10.1371/journal.pone.021...
, also emphasizing the teaching of Basic Life Support2525. Boada I, Rodriguez-Benitez A, Garcia-Gonzalez JM, Olivet J, Carreras V, Sbert M. Using a serious game to complement CPR instruction in a nurse faculty. Comput Methods Programs Biomed. 2015;122(2):282-91. doi: https://doi.org/10.1016/j.cmpb.2015.08.006
https://doi.org/10.1016/j.cmpb.2015.08.0...
-2727. Creutzfeldt J, Hedman L, Medin C, Heinrichs WL, Felländer-Tsai L. Exploring virtual worlds for scenario-based repeated team training of cardiopulmonary ressuscitation in medical students. J Med Internet Res. 2010;12(3):e38. doi: https://doi.org/10.2196/jmir.1426
https://doi.org/10.2196/jmir.1426...
,3030. Drummond D, Delval P, Abdenouri S, Truchot J, Ceccaldi PF, Plaisance P, et al. Serious game versus online course for pretraining medical students before a simulation-basedmastery learning course on cardiopulmonary resuscitation: a randomised controlled study. Eur J Anaesthesiol. 2017;34(12):836-44. doi: https://doi.org/10.1097/EJA.0000000000000675
https://doi.org/10.1097/EJA.000000000000...
,3131. Sena DP, Fabrício DD, Silva VD, Bodanese LC, Franco AR. Comparative evaluation of video-based online course versus serious game for training medical students in cardiopulmonar resuscitation: a randomised trial. PloS One. 2019;14(4):e0214722. doi: https://doi.org/10.1371/journal.pone.0214722
https://doi.org/10.1371/journal.pone.021...
.

The choice of this environment may be justified by two main reasons: first, two thirds of the total CPA cases take place outside the hospital, requiring a fast attention from the Basic Life Support, meaning this primordial stage of CPR should be valued; and, secondly, due to how complex it is to include all peculiarities of Advanced Life Support in the serious game, which is a shortcoming in the education about this subject3838. Monteiro MJ, Pereira MC, Carvalho RM, Carri ES, Carril MF, Rodrigues VM. Capacitação de trabalhadores em suporte básico de vida. Rev Cuid. 2018;9(2):2117-26. doi: https://doi.org/10.15649/cuidarte.v9i2.505
https://doi.org/10.15649/cuidarte.v9i2.5...
.

Only one study addressed the teaching of neonate CPR3232. Cutumisu M, Brown MRG, Fray C, Schmölzer GM. Growth mindset moderates the effect of the neonatal resuscitation program on performance in a computer-based game training simulation. Front Pediatr. 2018;6:195. doi: https://doi.org/10.3389/fped.2018.00195
https://doi.org/10.3389/fped.2018.00195 ...
. Although many serious games have been developed to make the teaching of adult CPR easier, those focused on the CPR of neonates are still scarce3232. Cutumisu M, Brown MRG, Fray C, Schmölzer GM. Growth mindset moderates the effect of the neonatal resuscitation program on performance in a computer-based game training simulation. Front Pediatr. 2018;6:195. doi: https://doi.org/10.3389/fped.2018.00195
https://doi.org/10.3389/fped.2018.00195 ...
.

Didactically, most serious games presented in this review2525. Boada I, Rodriguez-Benitez A, Garcia-Gonzalez JM, Olivet J, Carreras V, Sbert M. Using a serious game to complement CPR instruction in a nurse faculty. Comput Methods Programs Biomed. 2015;122(2):282-91. doi: https://doi.org/10.1016/j.cmpb.2015.08.006
https://doi.org/10.1016/j.cmpb.2015.08.0...
,2828. Buttussi F, Pellis T, Cabas Vidani A, Pausler D, Carchietti E, Chittaro L. Evaluation of a 3D serious game for advanced life support retraining. Int J Med Inform. 2013;82(9):798-809. doi: https://doi.org/10.1016/j.ijmedinf.2013.05.007
https://doi.org/10.1016/j.ijmedinf.2013....
-3232. Cutumisu M, Brown MRG, Fray C, Schmölzer GM. Growth mindset moderates the effect of the neonatal resuscitation program on performance in a computer-based game training simulation. Front Pediatr. 2018;6:195. doi: https://doi.org/10.3389/fped.2018.00195
https://doi.org/10.3389/fped.2018.00195 ...
describe a CPA situation, which leads the player to attend to the victim by applying CPR. The player starts with 100 points and loses points with every wrong decision. They only go to the next stage when the student answers the question correctly. Following this methodology, only two serious games were not more effective than the traditional teaching strategy to which they were compared1717. Youngquist ST , Kenneth Scheppke A , Pepe PE. Supportive technology in the resuscitation of out-of-hospital cardiac arrest patients. Curr Opin Crit Care. 2017;23(3):209-14. doi: https://doi.org/10.1097/MCC.0000000000000409
https://doi.org/10.1097/MCC.000000000000...
,3131. Sena DP, Fabrício DD, Silva VD, Bodanese LC, Franco AR. Comparative evaluation of video-based online course versus serious game for training medical students in cardiopulmonar resuscitation: a randomised trial. PloS One. 2019;14(4):e0214722. doi: https://doi.org/10.1371/journal.pone.0214722
https://doi.org/10.1371/journal.pone.021...
. Although serious games have been proved useful as educational tools in many domains, there is still a lack of reliable, automated, and repetitive methodologies to measure their efficacy, and most of them are evaluated by questionnaires, going against the current trends in the videogame industry3939. Serrano-Laguna A, Manero B, Freire M, Fernández-Manjón B. A methodology for assessing the effectiveness of serious games and for inferring player learning outcomes. Multimed Tools Appl. 2017;77:2849-71. doi: https://doi.org/10.1007/s11042-017-4467-6
https://doi.org/10.1007/s11042-017-4467-...
.

CONCLUSION

The scientific evidences found in literature with regard to using the serious game in the teaching-learning process of students in the field of health, with regard to cardiopulmonary resuscitation, indicated that the main target audience of these games are medicine undergraduate students. The main objectives of the researches were comparing the effectiveness of these games with traditional teaching methodologies and evaluating how well the cognitive knowledge is retained. The method that most serious games use to teach CPR is describing CPA scenarios, asking the player to attend to the victim. The players start with 100 initial points and, as they make mistakes, they lose points. They only go to the next stage when the student answers the question correctly. Serious games were shown to be effective to teach cardiopulmonary resuscitation, but further scientific studies still need to be made so that more conclusive results may be found, due to the variety in the outcomes found with regards to retaining knowledge. Extra-hospital environments and Basic Life Support were addressed widely, and few studies showed intra-hospital scenarios or taught Advanced Life Support.

The main limitations of this study were the scarcity of up-to-date studies that could answer the guiding question, and the lack of methodological specifications with regard to the use of the serious game, which made it more difficult to analyze its effectiveness. It stands out that this study contributes for the teaching, research, and assistance in the field of cardiopulmonary resuscitation, since it identifies the contribution of serious games as innovative CPR teaching-learning strategies, as well as their effectiveness and pedagogical methodology, delineating a framework of knowledge that can give support to the adoption of this praxis in the teaching of health.

REFERENCES

  • 1. Decesaro L, Severo EA, Guimarães JC, Girardi G. A influência da geração x e y na inovação de produto e processo na indústria metalmecânica do Rio Grande do Sul. Rev Bras Gestão Inov. 2018 [cited 2019 Jul 25];5(2):30-58. Available from: Available from: http://ucs.br/etc/revistas/index.php/RBGI/article/view/5160/3105
    » http://ucs.br/etc/revistas/index.php/RBGI/article/view/5160/3105
  • 2. Oliveira CM, Marques VF, Schreck RS. Aplicação de metodologia ativa no processo de ensino-aprendizagem: relato de experiência. Rev Eletrôn Pesquiseduca, 2017 [cited 2019 Jul 25];9(19):674-84. Available from: Available from: http://periodicos.unisantos.br/index.php/pesquiseduca/article/view/633
    » http://periodicos.unisantos.br/index.php/pesquiseduca/article/view/633
  • 3. Diesel A, Baldez AL, Martins SN. Os princípios das metodologias ativas de ensino: uma abordagem teórica. Rev Thema. 2017;14(1):268-88. doi: https://doi.org/10.15536/thema.14.2017.268-288.404
    » https://doi.org/10.15536/thema.14.2017.268-288.404
  • 4. Freire P. Pedagogia da autonomia: saberes necessários à prática educativa. 51. ed. Rio de Janeiro: Paz e Terra; 2015.
  • 5. Lewis ZH, Swartz MC, Lyons EJ. What’s the point?: a review of reward systems implemented in gamification interventions. Games Health J. 2016;5(2):93-9. doi: https://doi.org/10.1089/g4h.2015.0078
    » https://doi.org/10.1089/g4h.2015.0078
  • 6. Saunder L , Berridge, EJ. Immersive simulated reality scenarios for enhancing students’ experience of people with learning disabilities across all fields of nurse education. Nurse Educ Pract. 2015;15(6):397-402. doi: https://doi.org/10.1016/j.nepr.2015.04.007
    » https://doi.org/10.1016/j.nepr.2015.04.007
  • 7. Romero M, Usart M, Ott M. Can serious games contribute to developing and sustaining 21st century skills? Games Culture. 2014;10(2):148-77. doi: https://doi.org/10.1177/1555412014548919
    » https://doi.org/10.1177/1555412014548919
  • 8. Johnsen HM, Fossum M, Vivekananda-Schmidt P, Fruhling A, Slettebo A. A serious game for teaching nursing students clinical reasoning and decision-making skills. Stud Health Technol Inform. 2016;225:905-6. doi: https://doi.org/10.3233/978-1-61499-658-3-905
    » https://doi.org/10.3233/978-1-61499-658-3-905
  • 9. Kapralos B, Moussa F, Dubrowski A. An overview of virtual simulation and serious gaming for surgical education and training. In: Anthony LB, Sheryl B, Lakhmi CJ, editors. Technologies of inclusive well-being. Berlin: Heidelberg: Springer; 2014. p. 289-306. Studies in Computational Intelligence, vol. 536. doi: https://doi.org/10.1007/978-3-642-45432-5_14
    » https://doi.org/10.1007/978-3-642-45432-5_14
  • 10. Ricciardi F, De Paolis LT. A comprehensive review of serious games in health professions. Int J Comput Game Technol. 2014:ID787968. doi: https://doi.org/10.1155/2014/787968
    » https://doi.org/10.1155/2014/787968
  • 11. Mozaffarian D, Benjamin EJ, Go AS, Arnett DK, Blaha MJ, Cushman M, et al. Executive summary: Heart disease and stroke statistics-2016 update: a report from the American Heart Association. Circulation. 2016;133(4):447-54. doi: https://doi.org/10.1161/CIR.0000000000000366
    » https://doi.org/10.1161/CIR.0000000000000366
  • 12. Aranzábal-Alegría G, Verastegui-Diaz A, Quiñones-Laveriano DM, Quintana-Mendoza LY, Vilchez-Cornejo J, Espejo CB, et al. Factors influencing the level of knowledge of cardiopulmonary resuscitation in hospitals in Peru. Rev Colomb Anestesiol. 2017 [cited 2019 Jul 25];45(2):114-21. Available from: Available from: http://www.scielo.org.co/pdf/rca/v45n2/v45n2a07.pdf
    » http://www.scielo.org.co/pdf/rca/v45n2/v45n2a07.pdf
  • 13. Sena DP , Fabrício DD, Silva VD , Bodanese LC, Franco AR. Comparative evaluation of video-based on-line course versus serious game for training medical students in cardiopulmonary resuscitation: a randomised trial. PLoS One. 2019;14(4):e0214722. doi: https://doi.org/10.1371/journal.pone.0214722
    » https://doi.org/10.1371/journal.pone.0214722
  • 14. Aksoy E. Comparing the effects on learning outcomes of tablet-based and virtual reality-based serious gaming modules for basic life support training: randomized trial. JMIR Serious Games. 2019;7(2):e13442. doi: https://doi.org//10.2196/13442
    » https://doi.org//10.2196/13442
  • 15. Kavelak HL, Hollands JM, Bingham AL. Student-led cardiopulmonary resuscitation education to lay providers results in successful knowledge acquisition and skill performance. J Allied Health. 2019;48(1):18-21.
  • 16. Drummond D, Delval P, Abdenouri S, Truchot J, Ceccaldi PF, Plaisance P, et al. Serious game versus online course for pretraining medical students before a simulation-based mastery learning course on cardiopulmonary resuscitation. Eur J Anaesthesiol. 2017;34(12):836-44. doi: https://doi.org/10.1097/eja.0000000000000675
    » https://doi.org/10.1097/eja.0000000000000675
  • 17. Youngquist ST , Kenneth Scheppke A , Pepe PE. Supportive technology in the resuscitation of out-of-hospital cardiac arrest patients. Curr Opin Crit Care. 2017;23(3):209-14. doi: https://doi.org/10.1097/MCC.0000000000000409
    » https://doi.org/10.1097/MCC.0000000000000409
  • 18. Sjöberg F, Schönning E, Salzmann-Erikson M. Nurses' experiences of performing cardiopulmonary resuscitation in intensive care units: a qualitative study. J Clin Nurs. 2015;(17-18):2522-8. doi: https://doi.org/10.1111/jocn.12844
    » https://doi.org/10.1111/jocn.12844
  • 19. Silva AC, Bernardes A, Évora YD, Dalri MC, Silva AR, Sampaio CS. Development of a virtual learning environment for cardiorespiratory arrest training. Rev Esc Enferm USP. 2016;50(6):990-7. doi: https://doi.org/10.1590/s0080-623420160000700016
    » https://doi.org/10.1590/s0080-623420160000700016
  • 20. Ganong LH. Integrative reviews of nursing research. Res Nurs Health. 1987;10(1):1-11. doi: https://doi.org/10.1002/nur.4770100103
    » https://doi.org/10.1002/nur.4770100103
  • 21. Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan - a web and mobile app for systematic reviews. Syst Rev. 2016;5(1):210. doi: https://doi.org/10.1186/s13643-016-0384-4
    » https://doi.org/10.1186/s13643-016-0384-4
  • 22. Ursi ES, Galvão CM. Prevenção de lesões de pele no perioperatório: revisão integrativa da literatura. Rev Latino-Am Enfermagem. 2006;14(1):124-31. doi: https://doi.org/10.1590/S0104-11692006000100017
    » https://doi.org/10.1590/S0104-11692006000100017
  • 23. Galvão CM. Níveis de evidência [Editorial]. Acta Paul Enferm. 2006; [cited 2019 Oct 29];19(2):V. Available from: Available from: http://www.scielo.br/pdf/ape/v19n2/a01v19n2.pdf
    » http://www.scielo.br/pdf/ape/v19n2/a01v19n2.pdf
  • 24. Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Int J Surg. 2010;8(5):336-41. doi: https://doi.org//10.1016/j.ijsu.2010.02.007
    » https://doi.org//10.1016/j.ijsu.2010.02.007
  • 25. Boada I, Rodriguez-Benitez A, Garcia-Gonzalez JM, Olivet J, Carreras V, Sbert M. Using a serious game to complement CPR instruction in a nurse faculty. Comput Methods Programs Biomed. 2015;122(2):282-91. doi: https://doi.org/10.1016/j.cmpb.2015.08.006
    » https://doi.org/10.1016/j.cmpb.2015.08.006
  • 29. Creutzfeldt J, Hedman L, Felländer-Tsai L. Effects of pre-training using serious game technology on CPR performance--an exploratory quasi-experimental transfer study. Scand J Trauma Resusc Emerg Med. 2012;20:79. doi: https://doi.org/10.1186/1757-7241-20-79
    » https://doi.org/10.1186/1757-7241-20-79
  • 27. Creutzfeldt J, Hedman L, Medin C, Heinrichs WL, Felländer-Tsai L. Exploring virtual worlds for scenario-based repeated team training of cardiopulmonary ressuscitation in medical students. J Med Internet Res. 2010;12(3):e38. doi: https://doi.org/10.2196/jmir.1426
    » https://doi.org/10.2196/jmir.1426
  • 28. Buttussi F, Pellis T, Cabas Vidani A, Pausler D, Carchietti E, Chittaro L. Evaluation of a 3D serious game for advanced life support retraining. Int J Med Inform. 2013;82(9):798-809. doi: https://doi.org/10.1016/j.ijmedinf.2013.05.007
    » https://doi.org/10.1016/j.ijmedinf.2013.05.007
  • 29. Cook NF, McAloon T, O'Neill P, Beggs R. Impact of a web based interactive simulation game (PULSE) on nursing students' experience and performance in life support training: a pilot study. Nurse Educ Today. 2012;32(6):714-20. doi: https://doi.org/10.1016/j.nedt.2011.09.013
    » https://doi.org/10.1016/j.nedt.2011.09.013
  • 30. Drummond D, Delval P, Abdenouri S, Truchot J, Ceccaldi PF, Plaisance P, et al. Serious game versus online course for pretraining medical students before a simulation-basedmastery learning course on cardiopulmonary resuscitation: a randomised controlled study. Eur J Anaesthesiol. 2017;34(12):836-44. doi: https://doi.org/10.1097/EJA.0000000000000675
    » https://doi.org/10.1097/EJA.0000000000000675
  • 31. Sena DP, Fabrício DD, Silva VD, Bodanese LC, Franco AR. Comparative evaluation of video-based online course versus serious game for training medical students in cardiopulmonar resuscitation: a randomised trial. PloS One. 2019;14(4):e0214722. doi: https://doi.org/10.1371/journal.pone.0214722
    » https://doi.org/10.1371/journal.pone.0214722
  • 32. Cutumisu M, Brown MRG, Fray C, Schmölzer GM. Growth mindset moderates the effect of the neonatal resuscitation program on performance in a computer-based game training simulation. Front Pediatr. 2018;6:195. doi: https://doi.org/10.3389/fped.2018.00195
    » https://doi.org/10.3389/fped.2018.00195
  • 33. Kapralos B, Fisher S, Clarkson J, Oostveen R. A course on serious game design and development using an online problem-based learning approach. Interactive Technol Smart Educ. 2015;12(2):116-36. doi: https://doi.org/10.1108/ITSE-10-2014-0033
    » https://doi.org/10.1108/ITSE-10-2014-0033
  • 34. Gorbanev I, Agudelo-Londoño S, González RA, Cortes A, Pomares A, Delgadillo V, et al. A systematic review of serious games in medical education: quality of evidence and pedagogical strategy. Med Educ Online. 2018;23(1):1438718. doi: https://doi.org/10.1080/10872981.2018.1438718
    » https://doi.org/10.1080/10872981.2018.1438718
  • 35. McCoy EC, Rahman A, Rendon JC, Anderson CL, Langdorf, Lotfipour S, et al. Randomized controlled trial of simulation vs. standard training for teaching medical students high-quality cardiopulmonary resuscitation. West J Emerg Med. 2019;20(1):15-22. doi: https://doi.org/10.5811/westjem.2018.11.39040
    » https://doi.org/10.5811/westjem.2018.11.39040
  • 36. Middeke A, Anders S, Schuelper M, Raupach T, Schuelper N. Training of clinical reasoning with a Serious Game versus small-group problem-based learning: a prospective study. PLoS One. 2018 Sep 11;13(9):e0203851. doi: https://doi.org/10.1371/journal.pone.0203851
    » https://doi.org/10.1371/journal.pone.0203851
  • 37. Riggs M, Franklin R, Saylany L. Associations between cardiopulmonary resuscitation (CPR) knowledge, self-efficacy, training history and willingness to perform CPR and CPR psychomotor skills: a systematic review. Resuscitation. 2019;138:259-72. doi: https://doi.org/10.1016/j.resuscitation.2019.03.019
    » https://doi.org/10.1016/j.resuscitation.2019.03.019
  • 38. Monteiro MJ, Pereira MC, Carvalho RM, Carri ES, Carril MF, Rodrigues VM. Capacitação de trabalhadores em suporte básico de vida. Rev Cuid. 2018;9(2):2117-26. doi: https://doi.org/10.15649/cuidarte.v9i2.505
    » https://doi.org/10.15649/cuidarte.v9i2.505
  • 39. Serrano-Laguna A, Manero B, Freire M, Fernández-Manjón B. A methodology for assessing the effectiveness of serious games and for inferring player learning outcomes. Multimed Tools Appl. 2017;77:2849-71. doi: https://doi.org/10.1007/s11042-017-4467-6
    » https://doi.org/10.1007/s11042-017-4467-6

Edited by

Associate editor:

Graziella Badin Aliti

Editor-in-chief:

Maria da Graça Oliveira Crossetti

Publication Dates

  • Publication in this collection
    26 Oct 2020
  • Date of issue
    2020

History

  • Received
    30 Aug 2019
  • Accepted
    06 Mar 2020
Universidade Federal do Rio Grande do Sul. Escola de Enfermagem Rua São Manoel, 963 -Campus da Saúde , 90.620-110 - Porto Alegre - RS - Brasil, Fone: (55 51) 3308-5242 / Fax: (55 51) 3308-5436 - Porto Alegre - RS - Brazil
E-mail: revista@enf.ufrgs.br