Presentation
MDD1 - Three Stops en Route to the Root Cause: A Three-Step Approach to Effective Root Cause Analysis
DescriptionFormative and summative usability testing has become standard in the medical device development process. A universal goal of these studies is to understand if a device’s user interface (UI) is sufficient to support safe use by its intended users in its intended use environment. When use problems arise, it’s important to understand why they occur and how their associated risk can be effectively mitigated. The root cause analysis (RCA) process is intended to identify areas of the device UI that may be deficient, leading to use errors, close calls, and difficulties, and to provide insights into potential risk mitigations.
For human factors practitioners new to the field, the gap between the observed use problem and the potential root cause may seem difficult to bridge, particularly when we know that users are not the best at understanding why they do the things they do (Nielsen, 2001). To support practitioners in identifying root causes of use problems and thus deficiencies in the device UI, we offer a three-step process along with potential prompts for conducting a UI-focused root cause analysis.
Step 1: Focus on the error
When the time comes to perform a root cause interview, start broadly. Broad, open-ended questions allow the researcher to begin to clarify and focus what the participant thinks happened during the task. Starting broadly may include prompts such as:
● Walk me through…
● How did you decide to…?
● What was going through your mind when…?
If you’re lucky, the participant will know exactly what was done incorrectly and why. However, it is not uncommon that the participant will not realize a use problem occurred. At this point, it is helpful to let the participant know what you observed and find out why they think it happened. You may want to use prompts such as:
● I noticed that you… Tell me about that.
● I noticed that you… What was happening at that time?
If the participant does recognize a use problem occurred, you will next want to determine if they understand the action the manufacturer intended for them to take. This may mean asking questions such as:
● What do you think the manufacturer intends you to do when…?
● How do you think the manufacturer intends you to…?
Step 2: Determine what the participant relied on
If not evident after asking broad questions during Step 1, follow up with questions to determine what the participant relied on to complete the task. Questions such as “how do you know?” or “what did you use today to help you complete the task?” will help the researcher identify which feature(s) of the device’s UI contributed to the participant’s understanding of how the device should work. This is key for determining a device-focused root cause.
At this point, if the participant has not identified a feature of the UI that led them to commit the observed use error, the focus should shift to the features of the device’s UI that are designed to support correct usage. This will help determine why these components were not successful.
Step 3: Narrow down the root cause with the PCA model
Using the Perception, Cognition, Action (PCA) model for the following line of questioning will help the researcher understand what about the UI led to the incorrect action.
Perception (P):
First, find out if the participant noticed the element that should have led them to the correct action (i.e., perception). This may include questions such as:
● What were you paying attention to when [performing action]?
● Did you happen to notice [part of the device/section of the IFU/etc.]?
If the participant did not notice the element, then you likely have a perception-related root cause.
Cognition (C):
If the participant did notice the element, next find out if they understood what it meant (i.e., cognition). This may include questions such as:
● What do you think this means?
● What do you think this is meant to communicate?
If the participant noticed the element but did not understand its meaning, you likely have a cognition-related root cause.
Action (A):
Finally, if the participant did notice the element and also understood it but was unable to take the correct action, then you likely have an action-related root cause. To confirm, questions should focus on what happened when attempting to perform the intended action.
● What happened when you [attempted to perform the intended action]?
The recommended steps mentioned here do not necessarily occur in a specific order. Although they can, it is more ideal for the practitioner to carry out these steps simultaneously as they gain familiarity and confidence in performing root cause analysis. Therefore, these steps serve as a framework to guide the gathering of pertinent information during the root cause analysis process.
Following this three-step approach to root cause analysis can help to streamline what could otherwise feel like an overwhelming challenge and facilitate identifying deficiencies in the user interface. This approach averts focus from user-related root causes and enables easier determination of whether the user interface adequately supports safe usage by its intended users in the designated environment of use.
References
Nielsen, J. (2001, August 4). First Rule of Usability? Don't Listen to Users. Nielsen Norman Group. https://www.nngroup.com/articles/first-rule-of-usability-dont-listen-to-users/
For human factors practitioners new to the field, the gap between the observed use problem and the potential root cause may seem difficult to bridge, particularly when we know that users are not the best at understanding why they do the things they do (Nielsen, 2001). To support practitioners in identifying root causes of use problems and thus deficiencies in the device UI, we offer a three-step process along with potential prompts for conducting a UI-focused root cause analysis.
Step 1: Focus on the error
When the time comes to perform a root cause interview, start broadly. Broad, open-ended questions allow the researcher to begin to clarify and focus what the participant thinks happened during the task. Starting broadly may include prompts such as:
● Walk me through…
● How did you decide to…?
● What was going through your mind when…?
If you’re lucky, the participant will know exactly what was done incorrectly and why. However, it is not uncommon that the participant will not realize a use problem occurred. At this point, it is helpful to let the participant know what you observed and find out why they think it happened. You may want to use prompts such as:
● I noticed that you… Tell me about that.
● I noticed that you… What was happening at that time?
If the participant does recognize a use problem occurred, you will next want to determine if they understand the action the manufacturer intended for them to take. This may mean asking questions such as:
● What do you think the manufacturer intends you to do when…?
● How do you think the manufacturer intends you to…?
Step 2: Determine what the participant relied on
If not evident after asking broad questions during Step 1, follow up with questions to determine what the participant relied on to complete the task. Questions such as “how do you know?” or “what did you use today to help you complete the task?” will help the researcher identify which feature(s) of the device’s UI contributed to the participant’s understanding of how the device should work. This is key for determining a device-focused root cause.
At this point, if the participant has not identified a feature of the UI that led them to commit the observed use error, the focus should shift to the features of the device’s UI that are designed to support correct usage. This will help determine why these components were not successful.
Step 3: Narrow down the root cause with the PCA model
Using the Perception, Cognition, Action (PCA) model for the following line of questioning will help the researcher understand what about the UI led to the incorrect action.
Perception (P):
First, find out if the participant noticed the element that should have led them to the correct action (i.e., perception). This may include questions such as:
● What were you paying attention to when [performing action]?
● Did you happen to notice [part of the device/section of the IFU/etc.]?
If the participant did not notice the element, then you likely have a perception-related root cause.
Cognition (C):
If the participant did notice the element, next find out if they understood what it meant (i.e., cognition). This may include questions such as:
● What do you think this means?
● What do you think this is meant to communicate?
If the participant noticed the element but did not understand its meaning, you likely have a cognition-related root cause.
Action (A):
Finally, if the participant did notice the element and also understood it but was unable to take the correct action, then you likely have an action-related root cause. To confirm, questions should focus on what happened when attempting to perform the intended action.
● What happened when you [attempted to perform the intended action]?
The recommended steps mentioned here do not necessarily occur in a specific order. Although they can, it is more ideal for the practitioner to carry out these steps simultaneously as they gain familiarity and confidence in performing root cause analysis. Therefore, these steps serve as a framework to guide the gathering of pertinent information during the root cause analysis process.
Following this three-step approach to root cause analysis can help to streamline what could otherwise feel like an overwhelming challenge and facilitate identifying deficiencies in the user interface. This approach averts focus from user-related root causes and enables easier determination of whether the user interface adequately supports safe usage by its intended users in the designated environment of use.
References
Nielsen, J. (2001, August 4). First Rule of Usability? Don't Listen to Users. Nielsen Norman Group. https://www.nngroup.com/articles/first-rule-of-usability-dont-listen-to-users/
Event Type
Poster Presentation
TimeTuesday, March 264:45pm - 6:15pm CDT
LocationSalon C
Digital Health
Simulation and Education
Hospital Environments
Medical and Drug Delivery Devices
Patient Safety Research and Initiatives