Presentation
Building the Connection Between Human Factors Analysis and Practical Application: A Case Study On The Complexities Anesthesia Scheduling
DescriptionDeveloping an effective surgical team is critical as poorly formed teams can lead to higher healthcare utilization (Linder et al., 2023, Sawaf et al., 2023, Stucky & De Jong, 2021). Past studies have characterized the complexities of developing an effective team in the medical setting, but few have designed and developed solutions to aid in the scheduling process. In a study by Schoenfelder and Pfefferlen (2018), the researchers developed a model to generate schedules which accounted for all the organizational constraints in schedule development. While this study represents a positive step forward, by only accounting for administrative scheduling constraints, Schoenfelder and Pfefferlen are not accounting for the complex factors that contribute to an effective team including level of experience, role, and familiarity. The most effective scheduling aid would account not only for the administrative constraints of the organization, but also account for the complex interpersonal factors that can improve or impair a team. Therefore, the primary goals for this project are both to investigate the process of composing intraoperative anesthesia teams with task analysis and use the results of the task analysis to form parameters for intervention development.
The study contained three different parts: (1) gathering interview data from practicing schedulers, (2) synthesizing the data into a single model of the decision-making process, as well as establishing the factors that go into making scheduling decisions, and (3) taking the body of evidence and applying it to intervention development. The reported parts of this study were judged to be exempt as quality improvement from the appropriate Institutional Review Board (IRB00371684).
Based on preliminary discussions with stakeholders, we decided to use a two-pronged approach to task understanding. Schedule development follows an iterative and variable workflow, but it also hinges critically on decision criteria which are used to determine the staffing decisions. We decided to use the functional analysis resonance method (FRAM: Hollnagel, 2012) to model the process of creating staffing assignments and interview content analysis to determine the decision criteria. The FRAM is ideal for this task because it lends well to workflow mapping. We used the FRAM to find the places in the workflow where interventions would be most logistically beneficial.
We gathered interview data based on a convenience sample within a Mid-Atlantic teaching hospital. We then conducted individual interviews using a think aloud protocol, where we asked participants to walk us through their scheduling process. At the session start, the participants were told to “begin the process where you perceive it to start.” Throughout the interview, we asked probing questions to elucidate both the different components of the FRAM model and the decision criteria that drove the final scheduling decisions.
We interviewed every member (n=4) of the scheduling team in the pediatric anesthesia department of a Mid-Atlantic Research Hospital. Despite the small sample size, by the final interview we had reached data saturation. After gathering the interview data, we developed both the FRAM model and the complete list of decision factors. We used the information from both of these outputs to inform the development of a novel software scheduling aid.
Based on the FRAM model, we found that while all schedulers approach the scheduling process slightly differently, they all follow the same general workflow. The three primary steps are information acquisition (staffing availability, case load, and patient sickness), information organization (sorting cases by location and people by experience), and iterative assignment (relying on decision factors to determine optimization). The workflow itself is highly iterative with the two largest time burdens being finding the information and making the final assignments. Seeing as no two schedulers followed the exact same workflow, there is no single gap which a technological innovation can fill. Rather, the most beneficial solution would be one that truncates the iterative process by presenting the scheduler with a “starting schedule” which can be built off of. To achieve this, we can leverage the outcomes of the interview content analysis to build a framework of a scheduling aid software.
In order to optimize a schedule, we first compiled the optimization parameters and determined weights for them. From the interview analysis we identified 20 decision factors that play a role in the development of schedules, and they can be broken down into three types of optimization factors: simple, complex, and required. Required optimization factors are administrative rules that must be followed when creating a schedule. Simple optimization factors are factors that are clear and simple to optimize such as “rooms with high turnover need more attention.” Required and simple optimization factors are easily implemented by a linear programming optimization method where only schedules that meet requirements are presented and where the goal is to maximize the extent to which the schedule meets the simple optimization factors.
The only complex optimization factor (and arguably the most critical optimization factor) is that difficult cases should be staffed by more experienced clinicians. The question remains: what constitutes a “difficult” case? Due to a vague definition of “difficulty,” case difficulty is not something that can be linearly optimized. Instead experience recommendations from case difficulty should be made based on past successful cases, and that is where we can leverage machine learning.
We can combine three widely accepted measures for case difficulty, ASA score, patient age, and case type, to predict clinician expertise (under the assumption that more experienced clinicians have been assigned to more difficult cases in the past). We will take a decision tree approach to our machine learning program as we will be using non-linear regression to make these predictions. After generating predicted necessary experience, we can use it as a parameter in our linear optimization function.
In determining the weights of our linear optimizers, we can leverage the frequency with which they were mentioned in interviews. Based on the interview data, it is also clear that clinician experience is the most important optimizer, and it will take twice the optimization weight of the highest weighted simple optimizer.
This research project has important implications for multiple reasons. First, it characterizes the complexity of anesthesia scheduling showing the difficulty of team development and consistency. Schedulers, who are often also clinicians with a full caseload, are heavily burdened by the complex requirements and highly intricate scheduling workflow. Our initial goal for this work was to understand how interpersonal factors contribute to schedule development and create a medical intervention which accounted for interpersonal factors in conjunction with administrative constraints. However, based on our interview content analysis, it has become clear that case success predictors (e.g. anesthesiologist experience, patient health, etc.) are often deemed the most important factors in schedule development. Our new objective with this work is to develop a scheduling aid which accounts for all of the many factors which contribute to schedule development.
The second key takeaway of this research is that it is a great example of a collaborative work of human factors analysis and software engineering. Healthcare human factors engineering projects are often aimed at characterizing the problems with the work system, and medical engineering is often aimed at technology development without accounting for the complexities of the work system. As medical technology is advancing, it has become increasingly important to unify the human factors analysis and medical engineering processes to ensure the development of technologies which will appropriately fit into the work system and address issues from a more environmentally holistic perspective.
The study contained three different parts: (1) gathering interview data from practicing schedulers, (2) synthesizing the data into a single model of the decision-making process, as well as establishing the factors that go into making scheduling decisions, and (3) taking the body of evidence and applying it to intervention development. The reported parts of this study were judged to be exempt as quality improvement from the appropriate Institutional Review Board (IRB00371684).
Based on preliminary discussions with stakeholders, we decided to use a two-pronged approach to task understanding. Schedule development follows an iterative and variable workflow, but it also hinges critically on decision criteria which are used to determine the staffing decisions. We decided to use the functional analysis resonance method (FRAM: Hollnagel, 2012) to model the process of creating staffing assignments and interview content analysis to determine the decision criteria. The FRAM is ideal for this task because it lends well to workflow mapping. We used the FRAM to find the places in the workflow where interventions would be most logistically beneficial.
We gathered interview data based on a convenience sample within a Mid-Atlantic teaching hospital. We then conducted individual interviews using a think aloud protocol, where we asked participants to walk us through their scheduling process. At the session start, the participants were told to “begin the process where you perceive it to start.” Throughout the interview, we asked probing questions to elucidate both the different components of the FRAM model and the decision criteria that drove the final scheduling decisions.
We interviewed every member (n=4) of the scheduling team in the pediatric anesthesia department of a Mid-Atlantic Research Hospital. Despite the small sample size, by the final interview we had reached data saturation. After gathering the interview data, we developed both the FRAM model and the complete list of decision factors. We used the information from both of these outputs to inform the development of a novel software scheduling aid.
Based on the FRAM model, we found that while all schedulers approach the scheduling process slightly differently, they all follow the same general workflow. The three primary steps are information acquisition (staffing availability, case load, and patient sickness), information organization (sorting cases by location and people by experience), and iterative assignment (relying on decision factors to determine optimization). The workflow itself is highly iterative with the two largest time burdens being finding the information and making the final assignments. Seeing as no two schedulers followed the exact same workflow, there is no single gap which a technological innovation can fill. Rather, the most beneficial solution would be one that truncates the iterative process by presenting the scheduler with a “starting schedule” which can be built off of. To achieve this, we can leverage the outcomes of the interview content analysis to build a framework of a scheduling aid software.
In order to optimize a schedule, we first compiled the optimization parameters and determined weights for them. From the interview analysis we identified 20 decision factors that play a role in the development of schedules, and they can be broken down into three types of optimization factors: simple, complex, and required. Required optimization factors are administrative rules that must be followed when creating a schedule. Simple optimization factors are factors that are clear and simple to optimize such as “rooms with high turnover need more attention.” Required and simple optimization factors are easily implemented by a linear programming optimization method where only schedules that meet requirements are presented and where the goal is to maximize the extent to which the schedule meets the simple optimization factors.
The only complex optimization factor (and arguably the most critical optimization factor) is that difficult cases should be staffed by more experienced clinicians. The question remains: what constitutes a “difficult” case? Due to a vague definition of “difficulty,” case difficulty is not something that can be linearly optimized. Instead experience recommendations from case difficulty should be made based on past successful cases, and that is where we can leverage machine learning.
We can combine three widely accepted measures for case difficulty, ASA score, patient age, and case type, to predict clinician expertise (under the assumption that more experienced clinicians have been assigned to more difficult cases in the past). We will take a decision tree approach to our machine learning program as we will be using non-linear regression to make these predictions. After generating predicted necessary experience, we can use it as a parameter in our linear optimization function.
In determining the weights of our linear optimizers, we can leverage the frequency with which they were mentioned in interviews. Based on the interview data, it is also clear that clinician experience is the most important optimizer, and it will take twice the optimization weight of the highest weighted simple optimizer.
This research project has important implications for multiple reasons. First, it characterizes the complexity of anesthesia scheduling showing the difficulty of team development and consistency. Schedulers, who are often also clinicians with a full caseload, are heavily burdened by the complex requirements and highly intricate scheduling workflow. Our initial goal for this work was to understand how interpersonal factors contribute to schedule development and create a medical intervention which accounted for interpersonal factors in conjunction with administrative constraints. However, based on our interview content analysis, it has become clear that case success predictors (e.g. anesthesiologist experience, patient health, etc.) are often deemed the most important factors in schedule development. Our new objective with this work is to develop a scheduling aid which accounts for all of the many factors which contribute to schedule development.
The second key takeaway of this research is that it is a great example of a collaborative work of human factors analysis and software engineering. Healthcare human factors engineering projects are often aimed at characterizing the problems with the work system, and medical engineering is often aimed at technology development without accounting for the complexities of the work system. As medical technology is advancing, it has become increasingly important to unify the human factors analysis and medical engineering processes to ensure the development of technologies which will appropriately fit into the work system and address issues from a more environmentally holistic perspective.
Event Type
Discussion Panel
Oral Presentations
TimeWednesday, March 2710:45am - 11:00am CDT
LocationSalon A-1
Hospital Environments