Presentation
Ensuring Excellence: Validating Cardiac Care Through Medical Simulation
DescriptionHigh-fidelity simulation is becoming a gold-standard for medical skills training, from medical students learning the basics of cardiac failure management, to operating room personnel learning team communication skills, to emergency response teams learning to triage victims after mass-shooting incidents. The value and benefits of simulation training are well-established and include enhanced learning and integration of basic principles, and improved performance and confidence in routine and critical medical decision making, amongst others.
Nevertheless, simulations such as those involving high fidelity mannikins and a team of technicians to choreograph and run are expensive and time-consuming. Therefore, they should be well-designed and validated to support learning objectives. For undergraduate medical education, this is especially critical for program accreditation. The LCME’s competence-based requirements have prompted a re-examination of simulation as an integral part of the curriculum.
At the Carle College of Medicine, we have developed a pipeline to evaluate and validate large scale high-fidelity simulations that allow us to specifically map learning objectives to LCME competency requirements. This was accomplished by implementing pre- and post-simulation questionnaires.
Incorporating pre and post surveys in medical simulation is a valuable tool for enhancing the learning experience, validating knowledge transfer, improving patient safety, and fostering a culture of continuous improvement in healthcare education. It allows educators to tailor their training programs and curricula to better address the specific needs and challenges faced by healthcare professionals in real-world clinical practice.
Here is how you can use pre and post surveys in medical simulation for the purposes:
• Pre-Simulation Survey: Administer a pre-simulation survey to assess learners' existing knowledge, skills, and attitudes related to the specific medical scenario or procedure you plan to simulate. This provides a baseline against which you can measure improvements.
• Human Factors Assessment: Include questions that evaluate learners' understanding of human factors and situation awareness concepts. Ask about their familiarity with the principles of communication, teamwork, situational awareness, and stress management.
• Simulation Scenario: Design a realistic simulation scenario that closely resembles the clinical environment. This should include challenging situations where human factors and situation awareness play a critical role in patient safety.
• Structured Learning: Ensure that the simulation experience is structured to include teamwork, effective communication, and critical thinking to address human factors and situation awareness issues.
• Post-Simulation Survey: Administer a post-simulation survey immediately after the simulation to evaluate the knowledge and skills acquired during the session. This can include specific questions related to the simulation scenario and its associated learning objectives. Ask learners about their awareness of human factors and their ability to manage them during the simulation. Encourage them to reflect on teamwork, communication, and situational awareness aspects of the scenario.
• Feedback and Debriefing: After the simulation, conduct a thorough debriefing session. Discuss the survey results and compare them to the pre-simulation data. Identify areas of improvement and encourage learners to reflect on their performance regarding human factors and situation awareness.
• Iterative improvement: Use the survey results and debriefing feedback to make improvements in subsequent simulations. Tailor future scenarios to address specific knowledge gaps or areas of weakness identified in the surveys.
• Data Analysis: Analyze the survey data to identify trends, common areas of improvement, and individual performance variations. Use this information to inform curriculum development and continuous quality improvement.
• Documentation and Reporting: Maintain records of pre- and post-simulation surveys, debriefing sessions, and other relevant data to demonstrate the effectiveness of your medical simulation program to stakeholders, accrediting bodies, and regulatory agencies.
Applying human factors approach to assessing and improving medical simulation scenarios, we can understand how clinicians interact with patients, technology, and their environment to optimize safety and performance.
Background:
Even though simulation training is not a required component of a medical undergraduate curriculum, simulation training is a cornerstone of the undergraduate education program Carle Illinois College of Medicine (CIMED). Simulation spans the spectrum of standardized patients, part-task trainers, virtual reality and augmented reality simulators, and high-fidelity full-scale simulations. The Jump Simulation Center Urbana in CIMED provides students with an opportunity to transfer theory to practice in an integrated learning environment and serves as an efficient way to practice skills, applying knowledge gained through lectures and/or reading assessments. (Alanazi et al., 2017)
Facilitators and simulation staff designed successful targeted simulation scenarios by breaking down complex medical procedures or disease pathologies into their constituent steps and cognitive processes. Incorporating human factors principles into the simulation scenario design and evaluation, effectiveness of the training would ultimately contribute to better healthcare outcomes and improved learner confidence and competence.
Knowing our learners would be participating within a multidisciplinary team, collaborating with subject matter experts on the simulation scenario design, this enabled us to identify requirements and specific skills that needed to be reinforced through the simulation scenario. It was also crucial that the acquisition of relevant clinical skills knowledge matched didactic teaching within the curriculum. Aligning the learning objectives ensured transfer of knowledge from the classroom to the bedside.
Measuring outcomes with clear performance metrics and assessment criteria based upon the learning objectives validated and supported the effectiveness and educational benefit of the simulation experience and center. There are many aspects to an evaluation in healthcare simulation that range from educating new learners and training current professionals, to a systematic review of programs to improve outcomes. Validation of these assessment tools is essential to ensure that they are valid and reliable. Validity refers to whether any measuring instrument measures what it is intended to measure. Additionally, reliability is part of the validity assessment and refers to the consistent or reproducible results of an assessment tool (Urbina., 2020). Using pre and post assessments within a medical simulation offers benefits for both the learner and educator. Pre-assessments evaluate a baseline measurement of the learners' knowledge, skills, and confidence before engaging within the simulation scenario. The pre-assessment also establishes clear learning objectives allowing the learner to mentally set specific goals for their participation. Post-assessments serve as a valuable tool for assessing competencies gained or improved, evidence of learning, progress and growth that was achieved from the simulation scenario. This is valuable data for educators, institutions, and accreditations bodies. Data from pre and post assessment can be used to evaluate effectiveness of the curriculum and the specific simulation scenarios enable continuous improvement in medical education.
Jump Simulation- Urbana's data clearly validated evidence of learning, progress and growth and competency gained after the simulation scenario.
Pre- Assessment Survey results: 61 participants/ responses. Likert scale 1-5. Overall score of level of confidence and competence 2.82
Post- Assessment Survey results: 58 participants/ responses. Likert scale 1-5. Overall score of level of confidence and competence 3.75
Takeaway Points:
• Using these scores will allow staff and faculty to tailor future simulations to address specific gaps in knowledge, identify trends or common areas of improvement and develop a continuous improvement quality plan.
• Applying the principles of human factors into medical simulation enhances the effectiveness of medical education, leads to better retention of knowledge and skill development to significantly enhance patient safety and healthcare quality.
Alanazi, A., Nicholson, N., & Thomas, S. (2017). The use of simulation training to improve knowledge, skills, and confidence among healthcare students: A systematic review. Internet Journal of Allied Health Sciences and Practice. https://doi.org/10.46743/1540-580x/2017.1666
Lawson, S., Reid, J., Morrow, M., & Gardiner, K. (2018). Simulation-based Education and Human Factors Training in Postgraduate Medical Education: A Northern Ireland Perspective. The Ulster medical journal, 87(3), 163–167.
Urbina J, Monks SM. Validating Assessment Tools in Simulation. In: StatPearls. StatPearls Publishing, Treasure Island (FL); 2022. PMID: 32809366.
Nevertheless, simulations such as those involving high fidelity mannikins and a team of technicians to choreograph and run are expensive and time-consuming. Therefore, they should be well-designed and validated to support learning objectives. For undergraduate medical education, this is especially critical for program accreditation. The LCME’s competence-based requirements have prompted a re-examination of simulation as an integral part of the curriculum.
At the Carle College of Medicine, we have developed a pipeline to evaluate and validate large scale high-fidelity simulations that allow us to specifically map learning objectives to LCME competency requirements. This was accomplished by implementing pre- and post-simulation questionnaires.
Incorporating pre and post surveys in medical simulation is a valuable tool for enhancing the learning experience, validating knowledge transfer, improving patient safety, and fostering a culture of continuous improvement in healthcare education. It allows educators to tailor their training programs and curricula to better address the specific needs and challenges faced by healthcare professionals in real-world clinical practice.
Here is how you can use pre and post surveys in medical simulation for the purposes:
• Pre-Simulation Survey: Administer a pre-simulation survey to assess learners' existing knowledge, skills, and attitudes related to the specific medical scenario or procedure you plan to simulate. This provides a baseline against which you can measure improvements.
• Human Factors Assessment: Include questions that evaluate learners' understanding of human factors and situation awareness concepts. Ask about their familiarity with the principles of communication, teamwork, situational awareness, and stress management.
• Simulation Scenario: Design a realistic simulation scenario that closely resembles the clinical environment. This should include challenging situations where human factors and situation awareness play a critical role in patient safety.
• Structured Learning: Ensure that the simulation experience is structured to include teamwork, effective communication, and critical thinking to address human factors and situation awareness issues.
• Post-Simulation Survey: Administer a post-simulation survey immediately after the simulation to evaluate the knowledge and skills acquired during the session. This can include specific questions related to the simulation scenario and its associated learning objectives. Ask learners about their awareness of human factors and their ability to manage them during the simulation. Encourage them to reflect on teamwork, communication, and situational awareness aspects of the scenario.
• Feedback and Debriefing: After the simulation, conduct a thorough debriefing session. Discuss the survey results and compare them to the pre-simulation data. Identify areas of improvement and encourage learners to reflect on their performance regarding human factors and situation awareness.
• Iterative improvement: Use the survey results and debriefing feedback to make improvements in subsequent simulations. Tailor future scenarios to address specific knowledge gaps or areas of weakness identified in the surveys.
• Data Analysis: Analyze the survey data to identify trends, common areas of improvement, and individual performance variations. Use this information to inform curriculum development and continuous quality improvement.
• Documentation and Reporting: Maintain records of pre- and post-simulation surveys, debriefing sessions, and other relevant data to demonstrate the effectiveness of your medical simulation program to stakeholders, accrediting bodies, and regulatory agencies.
Applying human factors approach to assessing and improving medical simulation scenarios, we can understand how clinicians interact with patients, technology, and their environment to optimize safety and performance.
Background:
Even though simulation training is not a required component of a medical undergraduate curriculum, simulation training is a cornerstone of the undergraduate education program Carle Illinois College of Medicine (CIMED). Simulation spans the spectrum of standardized patients, part-task trainers, virtual reality and augmented reality simulators, and high-fidelity full-scale simulations. The Jump Simulation Center Urbana in CIMED provides students with an opportunity to transfer theory to practice in an integrated learning environment and serves as an efficient way to practice skills, applying knowledge gained through lectures and/or reading assessments. (Alanazi et al., 2017)
Facilitators and simulation staff designed successful targeted simulation scenarios by breaking down complex medical procedures or disease pathologies into their constituent steps and cognitive processes. Incorporating human factors principles into the simulation scenario design and evaluation, effectiveness of the training would ultimately contribute to better healthcare outcomes and improved learner confidence and competence.
Knowing our learners would be participating within a multidisciplinary team, collaborating with subject matter experts on the simulation scenario design, this enabled us to identify requirements and specific skills that needed to be reinforced through the simulation scenario. It was also crucial that the acquisition of relevant clinical skills knowledge matched didactic teaching within the curriculum. Aligning the learning objectives ensured transfer of knowledge from the classroom to the bedside.
Measuring outcomes with clear performance metrics and assessment criteria based upon the learning objectives validated and supported the effectiveness and educational benefit of the simulation experience and center. There are many aspects to an evaluation in healthcare simulation that range from educating new learners and training current professionals, to a systematic review of programs to improve outcomes. Validation of these assessment tools is essential to ensure that they are valid and reliable. Validity refers to whether any measuring instrument measures what it is intended to measure. Additionally, reliability is part of the validity assessment and refers to the consistent or reproducible results of an assessment tool (Urbina., 2020). Using pre and post assessments within a medical simulation offers benefits for both the learner and educator. Pre-assessments evaluate a baseline measurement of the learners' knowledge, skills, and confidence before engaging within the simulation scenario. The pre-assessment also establishes clear learning objectives allowing the learner to mentally set specific goals for their participation. Post-assessments serve as a valuable tool for assessing competencies gained or improved, evidence of learning, progress and growth that was achieved from the simulation scenario. This is valuable data for educators, institutions, and accreditations bodies. Data from pre and post assessment can be used to evaluate effectiveness of the curriculum and the specific simulation scenarios enable continuous improvement in medical education.
Jump Simulation- Urbana's data clearly validated evidence of learning, progress and growth and competency gained after the simulation scenario.
Pre- Assessment Survey results: 61 participants/ responses. Likert scale 1-5. Overall score of level of confidence and competence 2.82
Post- Assessment Survey results: 58 participants/ responses. Likert scale 1-5. Overall score of level of confidence and competence 3.75
Takeaway Points:
• Using these scores will allow staff and faculty to tailor future simulations to address specific gaps in knowledge, identify trends or common areas of improvement and develop a continuous improvement quality plan.
• Applying the principles of human factors into medical simulation enhances the effectiveness of medical education, leads to better retention of knowledge and skill development to significantly enhance patient safety and healthcare quality.
Alanazi, A., Nicholson, N., & Thomas, S. (2017). The use of simulation training to improve knowledge, skills, and confidence among healthcare students: A systematic review. Internet Journal of Allied Health Sciences and Practice. https://doi.org/10.46743/1540-580x/2017.1666
Lawson, S., Reid, J., Morrow, M., & Gardiner, K. (2018). Simulation-based Education and Human Factors Training in Postgraduate Medical Education: A Northern Ireland Perspective. The Ulster medical journal, 87(3), 163–167.
Urbina J, Monks SM. Validating Assessment Tools in Simulation. In: StatPearls. StatPearls Publishing, Treasure Island (FL); 2022. PMID: 32809366.
Event Type
Oral Presentations
TimeMonday, March 252:15pm - 2:37pm CDT
LocationSalon A-4
Simulation and Education