Presentation
MDD15 - Development and Pilot Testing of a Health Literacy Participant Screening Measure
DescriptionApproximately 40% of Americans aged 16 to 65 years are classified as having low literacy skills (Organization for Economic Cooperation and Development, 2013). Low literacy is a known barrier to receiving proper medical care and navigating healthcare systems (Baker et al., 1996). Patients with lower levels of literacy are more likely to incorrectly interpret warning labels on prescription medication (Davis et al., 2006) and have poorer knowledge and management of health conditions like asthma, diabetes, and HIV (Kalichman & Rompa, 2000, Williams et al., 1998). In addition, FDA guidance (e.g., Applying Human Factors and Usability Engineering to Medical Devices: February 3, 2016) and international standards (e.g., AAMI/IEC/TIR 62366-2: 2016- Medical Devices- Part 2: Guidance on the application of usability engineering to medical devices) recommend consideration of literacy throughout human factors engineering programs. Together, this underscores the need to include participants who are representative of these various literacy levels in the development and testing phases of medical and drug delivery devices to ensure inclusive and accessible design.
Despite the critical importance of representing individuals with low health literacy in human factors research and medical product development, identifying these individuals for inclusion presents many challenges. Patients with low literacy are considered “hard to reach” populations and have been shown to express less interest in research participation (Kriaplani et. al., 2021). Evidence-based strategies for recruiting “hard to reach” populations are scarce in the current literature.
Existing measures of health literacy are lengthy or designed to be administered in-person, making identification of individuals with low health literacy difficult at the participant recruitment phase of human factors research, which is not in-person and can be a brief interaction. Some of the most widely-used measures of health literacy include the Rapid Estimate of Adult Literacy in Medicine (REALM; Davis, et al., 1991), a test of reading ability and pronunciation that presents a list of 66 medical words and instructs respondents to read them aloud; the REALM-Short Form (Arozullah et al., 2007), a brief 7-item version of the REALM; the Test of Functional Health Literacy in Adults (TOFHLA; Parker et al., 1995), a 67-item test of reading comprehension and numerical ability; and the Newest Vital Sign (NVS; Weiss, et al., 2005), a short clinical screening tool that uses an ice cream label to assess reading comprehension and numeracy. Although these measures are well-validated and useful for confirming health literacy during in-person research sessions, their long length or formatting that requires participants to look at a list or document, make them poor candidates for assessing participant health literacy during participant recruiting phases.
Simpler proxy indicators of literacy, such as an individual’s self-reported educational attainment, often are used in an attempt to identify individuals with low health literacy; however, these indicators are often not accurate or reliable predictors (Davis et al., 1998).
Self-reported instruments have also been developed as screening measures of health literacy. These include Chew et al.'s (2004) three brief validated screening questions and the Single Item Literacy Screener (SILS). Although these screeners have shown promising results with identifying individuals with lower levels of health literacy, these screeners have predominantly been validated using in-person methods, rather than phone administration (Chew et al., 2004; Wallace et al., 2006). A study by Ylitalo et al. (2018) examined the screening question confidence completing medical forms when administered via phone to in-person performance on the NVS; however, this question was completed during a phone call that took place after the oral, in-person NVS portion of the study. In addition, although results indicated that individuals who reported less confidence completing medical forms were more likely to have inadequate health literacy, this perceived confidence demonstrated low sensitivity, and other study factors such as a small sample size and use of a convenience sample complicate generalizability of study findings.
To address challenges with quickly and accurately identifying individuals with low health literacy, we developed a novel tool designed to be administered verbally during the study participant recruitment phase of human factors research. Called the “Word Cue,” this tool is an eight-item word recognition tool that incorporates a selection of real health-related words alongside fictitious words designed to sound like plausible health words (“non-words”). While the tool does not require participants to self-report any measure of their perceived comfort with health-related words, it is designed to account for guesswork or false claims. Administration of the tool would include reading the words aloud and asking potential participants to indicate whether or not they know the meaning of the word. Respondents are given one point for correctly identifying a word or non-word (i.e., indicating they know the meaning of a word or indicating they do not know the meaning of a non-word) and are given a negative point for incorrectly identifying a word or non-word (i.e., indicating they do not know the meaning of a word or indicating they know the meaning of a non-word).
Preliminary results showed a significant, positive, moderate correlation between Word Cue when administered via the telephone and the REALM when administered in-person during a follow-up visit (n=81). Word Cue also showed medium test-retest reliability when administered over the phone versus in-person to the same participant at a later visit (n=81).
These initial results are promising and suggest that Word Cue may help improve identification of people with low literacy for inclusion in human factors research.
Despite the critical importance of representing individuals with low health literacy in human factors research and medical product development, identifying these individuals for inclusion presents many challenges. Patients with low literacy are considered “hard to reach” populations and have been shown to express less interest in research participation (Kriaplani et. al., 2021). Evidence-based strategies for recruiting “hard to reach” populations are scarce in the current literature.
Existing measures of health literacy are lengthy or designed to be administered in-person, making identification of individuals with low health literacy difficult at the participant recruitment phase of human factors research, which is not in-person and can be a brief interaction. Some of the most widely-used measures of health literacy include the Rapid Estimate of Adult Literacy in Medicine (REALM; Davis, et al., 1991), a test of reading ability and pronunciation that presents a list of 66 medical words and instructs respondents to read them aloud; the REALM-Short Form (Arozullah et al., 2007), a brief 7-item version of the REALM; the Test of Functional Health Literacy in Adults (TOFHLA; Parker et al., 1995), a 67-item test of reading comprehension and numerical ability; and the Newest Vital Sign (NVS; Weiss, et al., 2005), a short clinical screening tool that uses an ice cream label to assess reading comprehension and numeracy. Although these measures are well-validated and useful for confirming health literacy during in-person research sessions, their long length or formatting that requires participants to look at a list or document, make them poor candidates for assessing participant health literacy during participant recruiting phases.
Simpler proxy indicators of literacy, such as an individual’s self-reported educational attainment, often are used in an attempt to identify individuals with low health literacy; however, these indicators are often not accurate or reliable predictors (Davis et al., 1998).
Self-reported instruments have also been developed as screening measures of health literacy. These include Chew et al.'s (2004) three brief validated screening questions and the Single Item Literacy Screener (SILS). Although these screeners have shown promising results with identifying individuals with lower levels of health literacy, these screeners have predominantly been validated using in-person methods, rather than phone administration (Chew et al., 2004; Wallace et al., 2006). A study by Ylitalo et al. (2018) examined the screening question confidence completing medical forms when administered via phone to in-person performance on the NVS; however, this question was completed during a phone call that took place after the oral, in-person NVS portion of the study. In addition, although results indicated that individuals who reported less confidence completing medical forms were more likely to have inadequate health literacy, this perceived confidence demonstrated low sensitivity, and other study factors such as a small sample size and use of a convenience sample complicate generalizability of study findings.
To address challenges with quickly and accurately identifying individuals with low health literacy, we developed a novel tool designed to be administered verbally during the study participant recruitment phase of human factors research. Called the “Word Cue,” this tool is an eight-item word recognition tool that incorporates a selection of real health-related words alongside fictitious words designed to sound like plausible health words (“non-words”). While the tool does not require participants to self-report any measure of their perceived comfort with health-related words, it is designed to account for guesswork or false claims. Administration of the tool would include reading the words aloud and asking potential participants to indicate whether or not they know the meaning of the word. Respondents are given one point for correctly identifying a word or non-word (i.e., indicating they know the meaning of a word or indicating they do not know the meaning of a non-word) and are given a negative point for incorrectly identifying a word or non-word (i.e., indicating they do not know the meaning of a word or indicating they know the meaning of a non-word).
Preliminary results showed a significant, positive, moderate correlation between Word Cue when administered via the telephone and the REALM when administered in-person during a follow-up visit (n=81). Word Cue also showed medium test-retest reliability when administered over the phone versus in-person to the same participant at a later visit (n=81).
These initial results are promising and suggest that Word Cue may help improve identification of people with low literacy for inclusion in human factors research.
Event Type
Poster Presentation
TimeTuesday, March 264:45pm - 6:15pm CDT
LocationSalon C
Digital Health
Simulation and Education
Hospital Environments
Medical and Drug Delivery Devices
Patient Safety Research and Initiatives