Mayank Sakhuja1
Brooks Yelton1
Michelle A. Arent1Samuel Noblet2Mark M. Macauda1,3Delores Fedrick4
Daniela B. Friedman1*- 1Department of Health Promotion, Education, and Behavior, University of South Carolina, Columbia, SC, United States
- 2Prevention Research Center, University of South Carolina, Columbia, SC, United States
- 3Core for Applied Research and Evaluation, University of South Carolina, Columbia, SC, United States
- 4Chester County Literacy Council, Chester, SC, United States
Objective: To examine content and readability of COVID-19 information on outpatient clinic websites in South Carolina.
Participants: Thirty-three outpatient clinic websites.
Methods: Using a multi-step search strategy, we located three COVID-19 information content sections from each website. Descriptive statistics were calculated for content section characteristics (focus, information source, target population/race, presence of graphics, mobilizing information). Flesch Reading Ease (FRE), Flesch Kincaid Grade Level (FKGL), and Simple Measure of Gobbledygook (SMOG) were used to calculate reading levels. Mann Whitney U and Kruskal Wallis tests were performed to examine readability levels by clinic type (primary care vs. specialty) and content section characteristics.
Results: Twenty-six clinics offered COVID-19 information; 70 content sections across all 26 clinics were analyzed. Sections focused on COVID-19 clinic policies (48.4%), prevention (22.6%), testing (19.4%), and symptoms (9.7%). 93.5% lacked target population, 41.9% provided no information source, 38.7% had no graphics, and none mentioned racial/ethnic groups. MFRE = 54.3, MFKGL = 9.9, MSMOG = 9.5.
Conclusion: COVID-19 information focused mainly on clinic policy and was written at a ninth-grade skill level. Findings suggest there is opportunity for clinics to update their online content to convey more plain language and sourced information, especially for high-risk groups.
Introduction
Health literacy is “the degree to which individuals have the capacity to obtain, process, and understand basic health information needed to make appropriate health decisions” (Ratzan et al., 2000). Health literacy is predictive of individual health status, and lower levels of health literacy are associated with increased hospitalization rates (MacLeod et al., 2017). The need to understand and assimilate health information through written or verbal communications is heightened during times of crisis when actions and behaviors may be required to change rapidly as new information emerges (Paakkari and Okan, 2020). Health literacy is crucial to understanding information on COVID-19 disease prevention, symptom presentation, testing, local community relief, and access to care. In fact, according to the World Health Organization (WHO), “the best way to prevent and slow down transmission is to be well informed about the COVID-19 virus, the disease it causes, and how it spreads” (World Health Organization, 2021). However, only 12% of adults in the United States have proficient health literacy (Centers for Disease Control and Prevention, 2021a). Further impacting health literacy is racial, ethnic, and cultural bias in language accessibility, word choice, contextual references, and imagery (Arcia et al., 2016). Information source and formatting preferences may also differ by race and ethnicity. Taking these preferences into account is critical for effective communication about health issues (Friedman et al., 2009; Friedman et al., 2012; Best et al., 2016).
During the COVID-19 pandemic, many health care settings have discontinued or delayed in-person services, shifted care to telehealth platforms, and decreased the number and frequency of patients seen, thus limiting access to verbal information, treatment, and preventive services for a variety of health conditions (Bhaskar et al., 2020). With the majority of Americans having access to the Internet, these changes have increased online searches for health information (Worrall et al., 2020).
The harms of misinformation and information uncertainty regarding COVID-19 are numerous and ultimately affect the health of both the individual and the community (Lin et al., 2020). Knowledge of or experience with historical medical abuses and biases in the healthcare setting can also foster mistrust, which may influence whether and how people use and apply health information (Jaiswal et al., 2020; Tai et al., 2020; Yelton et al., 2020). Familiar, local providers, however, may be perceived as reliable sources of health information, especially for older adults and some racial/ethnic populations (Friedman et al., 2009; Friedman et al., 2012; Best et al., 2016). In light of the overlap in heightened COVID-19 risk in those of older age and those with medical comorbidities with limited health literacy or limited health information access, it is important that local providers present clear, accurate, and timely prevention, testing, and treatment messaging.
The US Department of Health and Human Services, American Medical Association, and National Institutes of Health recommend that online health information be written at or below a sixth-grade reading level (Office of Disease Prevention and Health Promotion and United States Department of Health and Human Services, 2016). Studies have found that the majority of online health information does not meet these standards and thus causes difficulty for the general public (Friedman et al., 2014; Kulkarni et al., 2018). Similar assessments have been done regarding health information specific to COVID-19. Two separate studies using several general search terms for COVID-19 health information via popular search engines found that the majority of sites were difficult for the general public to understand (Szmuda et al., 2020; Treanor and Radonjic, 2020). A multinational search of 240 websites including governmental and public health organization pages, social media pages, scientific and educational institution pages, and those categorized as “other” also found readability too difficult, noting that governmental/public health sources were the most readable for the general public (Mishra and Dexter, 2020).
The purpose of this current study was to examine the content and readability of COVID-19 information posted on outpatient clinic websites in South Carolina, a state that has the 13th highest rate of functional illiteracy in the country (The Literacy Center, 2021). Websites assessed belong to clinics with whom we are currently engaging on a separate health literacy initiative focused on improving patient-provider communication.
Methods
Website Search Process
As part of a larger health literacy initiative aimed at improving patient-provider communication in outpatient clinics in South Carolina, approximately 67 clinics were invited via the state hospital association partner (Friedman et al., 2020) to complete a readiness assessment before implementation of an evidence-informed intervention. Of the 67 clinics, 34 completed the readiness assessment and indicated interest in participating in the intervention. Of those 34 clinics, 33 had clinic websites. Between September 14 and 15, 2020, we searched the websites of the participating 33 outpatient clinics for COVID-19 information. This purposive sampling approach has been used for other print and web-based content and/or readability analyses (Friedman et al., 2014; Choi et al., 2018; Olscamp et al., 2019). To standardize our data, we aimed to locate three sections of relevant content per clinic website, as has been done elsewhere in web-based content analysis research (Tanner and Friedman, 2011). For purposes of readability testing, we defined content sections as paragraphs or lists focused on COVID-19 information with a minimum of 10 sentences of text, in accordance with requirements for the validated tools used (Flesch, 1948; Mc Laughlin, 1969; Kincaid et al., 1975; Friedman and Hoffman-Goetz, 2006a). We employed the following multi-step search strategy to locate COVID-19 content on each of the clinic websites:
First, we visited the clinic homepage. If the clinic homepage had a COVID-19 notification banner at the top of the page, we clicked on this link. We scrolled through the resulting webpage to identify up to three separate content sections for coding and analysis. If no content sections were identified via scrolling, we went to the top of the page and clicked on the first COVID-19 relevant link (hypertext or button) located within the body of the webpage; some links directed to clinic-specific COVID-19 pages, while others directed to COVID-19 pages central to a health system. On the resulting page, we moved systematically from left to right across the row, and from top to bottom of the page, to attempt to identify three separate content sections. If all links had been selected from the dedicated COVID-19 information page and we were still not able to identify three separate content sections, then we returned to the clinic homepage. Once back on the clinic homepage, we scanned the page to identify content sections without using any additional clicks. At that point, if we still had fewer than three total content sections between the dedicated COVID-19 information page search and the clinic homepage search, we proceeded to use the clinic website’s search bar, if available, to locate information utilizing the keywords “coronavirus” and “COVID-19.” Duplicate links resulting from using both terms, found on the first page of search results, were examined. If there were no duplicate content sections, we selected content sections from search results from the term “coronavirus” first, since this has been the most commonly searched term about the pandemic in South Carolina according to Google Trends (Google LLC, 2021). If we still did not have three total content sections at that point, we then used search results from the term “COVID-19.” For one clinic in our sample, we could not identify any content sections using the aforementioned search strategies. Modification of our search strategy to include utilization of the search bar located on the link within the dedicated COVID-19 page (which was central to a health system) enabled us to find three content sections appropriate for analysis.
Variables Coded
A comprehensive codebook was developed based on previously conducted content analysis studies (Tanner and Friedman, 2011; Kulkarni et al., 2018). Each clinic was coded for the type of clinic (primary or specialty) and where and how content sections were located (via COVID-19 bar or banner, clinic homepage, internal search engine, search bar on dedicated COVID-19 page). For our analysis, safety-net healthcare organizations were considered primary care clinics. Safety-net healthcare organizations are defined as organizations for lower socio-economic status individuals (Hadley and Cunningham, 2004), including federally qualified health centers, free medical clinics, and rural health clinics. Each content section was coded for multiple variables, including content section source (federal, state, clinic, other, not specified, and multiple sources), and content section author (name, qualification, position/job title). We also coded for focus of the content sections (COVID-19 testing information, facility policies during COVID-19, COVID-19 symptoms, COVID-19 treatment, and COVID-19 prevention) and for specific COVID-19 prevention strategies (handwashing, mask wearing, social distancing), physical activity, mental health, and nutrition. Additionally, content sections were coded for race/ethnicity (White, Black, Latino, other, and not specified), and population groups represented (children, adolescents, adults, older adults, pregnant patients, parents, and not specified). The webpages containing the selected content sections were coded for videos and images present on the page. Further, we coded for the presence or absence of mobilizing information (information that encourages patients and website visitors to seek out additional information) (Tanner and Friedman, 2011), and type of mobilizing information included (website link, phone number, email address, multiple options). Finally, we checked each website to determine if there was a language translation tool available for translating content sections in English to other languages, such as Spanish.
To examine the reading level of the clinic websites’ COVID-19 information, we downloaded each content section and copied and pasted information (as plain text) directly into Microsoft Word. Content information was then copied and pasted into an online readability tool (https://www.webpagefx.com/tools/read-able/). Three published and validated readability instruments—Simple Measure of Gobbledygook (SMOG) (Mc Laughlin, 1969), Flesch-Kincaid Grade Level (FKGL) (Kincaid et al., 1975), and Flesch Reading Ease (FRE) (Flesch, 1948)—were used to calculate reading level using the online tool. SMOG and FKGL report results as a grade level, while FRE is a scale from 0–100, with 0 being most difficult to read, and 100 being easiest to read (Friedman and Hoffman-Goetz, 2006b). Coding was done using Microsoft Excel™. To ensure 100% consistency in coding, two authors (MS, SN) searched for and coded all content sections separately and compared results.
Analysis Plan
Data from Microsoft Excel were imported into IBM SPSS version 27 for analysis. Descriptive statistics (percentages, frequencies, means) were conducted on content section characteristics. Non-parametric tests, including Kruskal Wallis and Mann-Whitney U tests, were performed to examine readability level by clinic website and content section characteristics (significance at p < 0.05). We examined readability by clinic characteristics such as clinic type, location of content sections, focus of content sections, and presence of mobilizing information. Due to the small sample size, safety-net healthcare organizations were not analyzed independently of primary care clinics. To evaluate the impact and reach of the information as it would be available to users, analyses for readability and content location were conducted from a clinic perspective and included all content sections identified, even if that meant content sections were included more than once (such as if several clinics linked to the same health care system site). In order to present information about the specific content sections, we filtered out any repeat sections (sections with the exact same language) for analyses (source, focus, author, mobilizing information, population, graphics) and only included each unique content section once.
Results
Seven clinics did not provide COVID-19 related information on their websites - these were primary care and safety-net health care organizations. Thus, 26 out of 33 clinics offered COVID-19 related information for analysis. Out of those 26 clinics, 15 were primary care clinics (5 of which were also safety-net health care organizations), and 11 offered specialty care. Three separate content sections were found for 22 of the 26 clinics; four primary care clinics (3 of which were also safety-net healthcare organizations) had only one available content section, resulting in a total of 70 content sections collected and analyzed. The majority of the total content sections (n = 63/70, 90%) were found by clicking on the COVID-19 notification banner (present at the top of the clinic homepage), which redirected to an internal or external dedicated COVID-19 information page. Two (2.9%) content sections were found on the clinics’ main homepage, 2 (2.9%) were found using a search engine provided on one clinic’s main homepage, and the remaining 3 (4.2%) were found via search bar available on a linked COVID-19 information page central to a health system.
The content sections specifically associated with specialty care clinics (n = 33) focused mainly on facility policy (n = 20/33, 60.6%) and testing (n = 10, 30.3%) with only 2 (6.0%) content sections geared towards prevention and 1 (3.0%) focused on symptoms. Content sections specifically associated with primary care clinics (n = 37) focused equally on facility policy (n = 11, 29.7%), testing (n = 11, 29.7%), and prevention (n = 12, 32.4%), with a smaller contribution from information regarding COVID-19 symptoms (n = 3, 8.1%). Several clinic sites, however, shared content sections: nine clinics (1 primary care, 8 specialty) were associated with one healthcare system, and shared three content sections; five additional clinics (4 primary care, 1 specialty) were associated with another healthcare system, and also shared three content sections; two additional primary care (safety-net) clinics shared three content sections. Thus, only 31 of the 70 content sections were unique.
Analysis of 31 Unique Content Sections
Thirteen (n = 13/31, 41.9%) of the unique content sections did not specify any source for their information. Eight (25.8%) of the content sections cited federal sources [Centers for Disease Control and Prevention (CDC)] in the information they provided, while five (16.1%) cited both federal (CDC) and state [Department of Health and Environmental Control (DHEC)] sources, and two (6.5%) cited both federal (CDC) and health system sources. The remaining three content sections (9.7%) indicated state sources (DHEC) only.
Most (n = 30, 96.8%) unique content sections did not specify the content author details, such as author’s name, qualification, or title; one content section was authored by the CEO and CMO of the clinic or hospital system. Almost half (n = 15, 48.4%) of the content sections focused on changes in clinic policies due to COVID-19, while seven (22.6%) provided information about how COVID-19 could be prevented, six (19.4%) provided COVID-19 related testing information, and three (9.7%) focused on COVID-19 related symptoms. None of the content sections shared information on physical activity, mental health, or nutrition. The majority (n = 29/31, 93.5%) of content sections did not specify an intended audience, while none (n = 0, 0.0%) of the content sections mentioned high-risk or racial/ethnic groups as their intended audience. One (3.2%) content section mentioned children, adults, and pregnant patients and outlined facility policy. One (3.2%) content section provided information for parents and focused on prevention, testing, and symptoms. Twelve (38.7%) content sections contained only text, eleven (35.5%) contained text, images, and videos, and eight (25.8%) contained a combination of text and images. Videos included information pertaining to hand washing, education for patients/visitors on maintaining social distancing inside the facility and the new protocols and structure of patient-provider interactions. Images depicted health care teams, doctors, COVID-19 testing sites, and health care facilities. A few images provided educational information pertaining to COVID-19 symptoms, and how to wash face coverings. Additionally, 23 (74.2%) content sections contained mobilizing information while 8 (25.8%) did not. Of those 23 content sections, twelve (38.7%) included a website link for additional information, seven (22.6%) included a website link and a phone number, two (6.5%) included only a phone number, and two (6.5%) included both a phone number and an email address. Lastly, only four (12.9%) content sections could be translated into a language other than English. Findings from the content analysis are presented in Table 1.
TABLE 1. Clinic website characteristics [total content sections (n = 70) vs. unique content sections (n = 31)].
Content section readability (mean ± SD) was generally difficult with an overall mean FRE score of 53.4 ± 11.3 for all 70 content sections. The overall mean SMOG and FKGL for the 70 content sections were 9.5 ± 2.1 and 9.9 ± 2.6, respectively. Readability scores differed by type of clinic, with content sections of specialty clinics (MSMOG-SC = 10.3 ± 1.5; MFKGL-SC = 10.8 ± 1.8; MFRE-SC = 49.1 ± 8.6) written at a significantly higher grade level as compared to the content sections of primary care clinics (MSMOG-PC = 8.9 ± 1.4; MFKGL-PC = 9.2 ± 1.8; MFRE-PC = 57.4 ± 8.3). Readability scores also differed by the focus of content sections and availability of mobilizing information. Content sections which focused on change in clinics’ policies or operations, because of COVID-19, were written at a significantly higher reading level (MSMOG-POLICY = 11.4 ± 1.5; MFKGL-POLICY = 12.1 ± 1.7; MFRE-POLICY = 44.1 ± 9.0) than the content sections focused on other areas. The content sections which included mobilizing information were found to have a higher reading grade level than those without mobilizing information (MSMOG-MI = 9.8 ± 2.0 vs. MSMOG-NOMI8.2 ± 2.3; MFKGL-MI = 10.3 ± 2.4 vs. MFKGL-NOMI8.3 ± 2.7; MFRE-MI = 52.1 ± 10.8 vs. MFRE-NOMI59.6 ± 12.0). There were no significant differences in readability scores based on where content sections were found (COVID-19 notification banner, clinic homepage, search bar, COVID-19 page search bar). Readability scores are presented in Table 2.
TABLE 2. Reading level by clinic website characteristics.
Discussion
Understanding risk factors, prevention measures, and treatment options for COVID-19 can be lifesaving; however, adequate access to, and understanding of such information is essential and is impacted by the availability and complexity of written and verbal communications (Arcia et al., 2016). Materials developed at or below a sixth-grade reading level are deemed appropriate for medical information communication (Office of Disease Prevention and Health Promotion and United States Department of Health and Human Services, 2016), and provider organizations should aim to develop or share existing credible COVID-19 information that is at this level. Our investigation of clinic websites found that not all offered COVID-19 specific information and those that did not were primary care clinics serving as safety-net healthcare organizations. These safety-net clinics may not have the necessary financial or personnel resources available to maintain and update website information. As a whole, content section focus heavily on information about facility policy over information on COVID-19 testing, prevention, or symptoms, with specialty clinics sharing more information on facility policy and testing while primary care clinics offered a balance of information on COVID-19-related facility policies, as well as COVID-19 prevention and treatment. As primary care clinics typically provide a larger range of services, patients may look to them for sources of general COVID-19 information rather than to specialty care clinics. This may influence type of information that clinics offered.
The difficulty level of the COVID-19 information offered by clinics will be quite challenging for most people to read and understand. Differences by clinic type were noted, as information from primary care clinics was written at a slightly lower grade level and shared a broader range of information than that offered by specialty clinics, however, the reading level of all information provided by the clinics in our sample is still much higher than recommended. These findings are similar to those from Treanor and Radonjic (2020), Szmuda et al. (2020), and Mishra and Dexter (2020), and highlight the need for COVID-19 specific information and materials written at a more appropriate reading level. Further, as readability of facility policy information was higher than that of other content section types, the favoring of facility policy information by specialty clinics may have impacted the overall readability scores for those clinics. Additionally, there was a missed opportunity to communicate health information through visuals such as videos and graphics. Only 11 content sections included videos along with text. Despite higher overall readability levels, inclusion of visuals could improve people’s understanding of health information. Prior research demonstrates that visual animations in combination with spoken text significantly increased health information recall among people with lower health literacy levels (Meppelink et al., 2015).
Increased dissemination of COVID-19 information via social media and other platforms has contributed to an “infodemic” (Jaiswal et al., 2020; Lin et al., 2020; Parmet and Paul, 2020; Sentell et al., 2020). In addition, rapid academic and scientific publication, contradictory prevention messaging, and an evolving treatment narrative have complicated understanding of COVID-19 health information, prompting the WHO to identify the need to “communicate critical information to all communities and prevent the spread of misinformation” as a key strategic objective regarding COVID-19 management (World Health Organization, 2020). Around 80% of American adults polled in April 2020 reported that they trust both the CDC and Director of the National Institute of Allergies and Infectious Diseases, Dr. Anthony Fauci, for reliable COVID-19 information; similarly the majority of respondents report trust in the information from state governmental officials (74%) and WHO information (70%) (Kirzinger et al., 2020). Health Information National Trends Survey in the US found that local healthcare providers have remained the most trusted source of health information among American adults (Jackson et al., 2019). While more than half of content sections analyzed in this study indicated inclusion of information from federal, state, or health system organizations, the remaining content sections had no specified source for information given. Additionally, most of the content sections did not provide details about the author which could potentially be local healthcare providers. Use of trusted, credible sources of information, such as local healthcare providers, in communicating health information could improve patients’ receptiveness to the information and their uptake of the recommended health message (Jackson et al., 2019). Lack of an identifiable source may result in failure to heed recommendations and/or suggestions and reduce people’s perceptions of the credibility of the information. Opportunities exist for increased clarity of information source and potentially increased recommendation adherence. Further, very few content sections included any form of mobilizing information (either internal or external links) that would link users to additional information or sources of information. Linkage to credible, reliable federal and state sources that are frequently updated (CDC, DHEC, etc.) may allow clinics to provide up-to-date scientific recommendations without the added financial or personnel burden of website maintenance.
COVID-19 disease burden is significant among underrepresented racial/ethnic minority groups. Nationally, African Americans (AA), American Indians, and Alaska Natives demonstrate a hospitalization rate over five times that of non-Hispanic Whites, while Hispanic or Latinx persons experience a rate four times greater (National Center for Immunization and Respiratory Diseases and Division of Viral Diseases, 2020). In South Carolina, AA hospitalization (40.3%) and death rates (31.9%) are disproportionately high compared to respective rates of other races (South Carolina Department of Health and Environmental Control, 2021) and SC population data (27% AA) (United States Census Bureau, 2021). Findings on content focus and intended audience in this study demonstrated that facility policy was largely represented, no indication of relevance to older age populations and race/ethnicity was provided, and few offered the ability to access information in a language other than English. Thus, clinics missed an important opportunity to provide educational information for high-risk groups. This is especially important at the local level, where patient-provider trust may increase receipt of information. In addition, while health is determined by a combination of social, economic, environmental, and biological factors, there are many behavioral factors (e.g., physical activity and healthy eating) that can improve physical health, mental health, and prevention of disease (Office of Disease Prevention and Health Promotion, 2020a). In this current analysis, no information on physical activity, nutrition, or mental health were found. With the large contribution of lifestyle and environmental factors to overall health (Office of Disease Prevention and Health Promotion, 2020a; Office of Disease Prevention and Health Promotion, 2020b), it is recommended that information and resources dedicated to modifiable lifestyle and environmental factors be included in online COVID-19 health information.
In general, clinics have an opportunity to provide accurate, appropriate, and easily understandable online COVID-19 information from reliable and trustworthy sources. In the current environment of misinformation and disinformation, clinics may also serve to clarify and reinforce the prevailing scientific opinion, dispelling any confusion or disinformation regarding COVID-19 and a potential vaccine. Opportunity exists for clinics to better serve their communities in this manner. With the paucity of easily readable science-based COVID-19 information, it appears that clinics may not have taken full advantage of this opportunity.
Limitations
This study had limitations. While the purposive sample of clinics used in this study functioned well to better inform authors regarding readability and content of website information from clinics already engaged in another health literacy initiative, this sample of clinics may not be representative of all outpatient primary and specialty clinics in South Carolina or in other states. We acknowledge that there are limits to the readability measures employed, and thus aimed to mitigate errors by triangulating readability scores with three validated tools. These formulas calculate readability based on an estimate rather than the actual number of syllables included, and produce different grade-level scores depending on the selected content sections and the criterion of comprehension employed (Friedman and Hoffman-Goetz, 2006b; Friedman and Kao, 2008). Given the scope of this study, the three readability tools used were the most appropriate for calculating reading levels. However, future work focused on assessing communication effectiveness of health information and website usability could employ using tools such as CDC Clear Communication Index (Centers for Disease Control and Prevention, 2021b). While we analyzed the inclusion of images and videos with text, readability measures do not account for the influences of graphics, content format, and readers’ prior knowledge. Further, analysis was performed at one point in time and for a maximum of three content sections per clinic. As web content is constantly changing, a single snapshot of data collected may not be representative of all information from any (clinic) website over time. Additionally, the time elapsed from the first COVID-19 appearance in the US to when the clinic websites were searched may have impacted the focus of information found thus explaining the few content sections dedicated to COVID-19 symptoms. For these reasons, future work should include analyses from multiple timepoints, and for a greater number of content sections from a larger, randomized sample of outpatient clinics.
Conclusion and Implications
Content source, readability, and cultural sensitivity influence trust in and receipt of health messaging. As local providers are often seen as trusted sources of information, clinics have a duty to ensure content is accurate, relevant, and written in plain language. Inclusion of content source and mobilizing information to verified, state and national information sources (e.g., DHEC, CDC), facilitates access to accurate, real-time information in nested layers of complexity (e.g., plain language content, infographics, full reports, etc.). Trust in health messaging relies on a variety of factors including cultural perceptions, e-literacy, and heuristics, and trust in official/governmental entities differs among races/ethnicities and in relation to specific concerns (Choi, 2020; McFadden et al., 2020; Peterson et al., 2020; Thompkins et al., 2020). Thus, information sources should be considered when tailoring information for specific patient populations, and inclusion of accurate information from or endorsed by a variety of sources (e.g., faith-based leaders, local community organizations, federal/state health authorities) may improve receipt of general information without a specific consumer focus. As financial and personnel resources differ greatly among clinic type/affiliation, mobilizing information to trusted entities reduces the resource burden of content creation and management. For increased effectiveness and usability, clinics should also consider ease of website navigation to help users quickly locate both important, urgent information, as well as specific content tailored to a variety of patient populations of different ages, races/ethnicities, and/or concerns. Additionally, information targeted to clinical staff can be separated from plain-language information through consumer-specific navigational links to content (e.g., “for providers”, “for patients”). Patient-focused information should also include links to content in languages other than English (Rodriguez et al., 2019). The Department of Health and Human Services (HHS) has developed a number of suggestions to improve health information website readability and usability for a variety of users including those with disabilities, providing free online access through the HHS usability. gov website and the Office of Disease Prevention and Promotion Health Literacy Online website (Office of Disease Prevention and Health Promotion and United States Department of Health and Human Services, 2016; United States Department of Health and Human Services, U.S. General Services Administration Technology Transformation Services, 2020). Clinics may benefit from utilizing or mobilizing users to evidence-based health information portals, similar to the approach used by The McMaster Optimal Aging Portal, for effective, tailored, and accurate health messaging for specific patient concerns (Barbara et al., 2016). Clinical-academic partnerships can also aid development of effective clinic health messaging through collaborative needs assessments and assistance with health communication/research translation and cultural sensitivity (Gubbins et al., 2014; Friedman et al., 2020).
Data Availability Statement
The raw data supporting the conclusion of this article will be made available by the authors, without undue reservation.
Author Contributions
MS, MM, and DF conceived the idea for the research study with support from the full team. MS, SN, BY, and MA developed the coding protocol with input from other authors. MS and SN completed the data extraction. MM guided the data analysis. MS, BY, and MA wrote the manuscript with input from the full team. All authors contributed to the refinement of the study and approved the final submitted manuscript.
Funding
This work was supported by The Duke Endowment Grant Number 6816-SP.
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s Note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
Arcia, A., Suero-Tejeda, N., Bales, M. E., Merrill, J. A., Yoon, S., Woollen, J., et al. (2016). Sometimes More Is More: Iterative Participatory Design of Infographics for Engagement of Community Members with Varying Levels of Health Literacy. J. Am. Med. Inform. Assoc. 23 (1), 174–183. doi:10.1093/jamia/ocv079
Barbara, A. M., Dobbins, M., Haynes, R. B., Iorio, A., Lavis, J. N., Raina, P., et al. (2016). The McMaster Optimal Aging Portal: Usability Evaluation of a Unique Evidence-Based Health Information Website. JMIR Hum. Factors 3 (1), e14. doi:10.2196/humanfactors.4800
Best, A. L., Spencer, S. M., Friedman, D. B., Hall, I. J., and Billings, D. (2016). The Influence of Spiritual Framing on African American Women's Mammography Intentions: A Randomized Trial. J. Health Commun. 21 (6), 620–628. doi:10.1080/10810730.2015.1114055
Bhaskar, S., Bradley, S., Chattu, V. K., Adisesh, A., Nurtazina, A., Kyrykbayeva, S., et al. (2020). Telemedicine across the globe-position Paper from the COVID-19 Pandemic Health System Resilience PROGRAM (REPROGRAM) International Consortium (Part 1). Front. Public Health 8, 556720. doi:10.3389/fpubh.2020.556720
Centers for Disease Control and Prevention (2021a). Health literacy: Understanding literacy & numeracy. Available from: https://www.cdc.gov/healthliteracy/learn/UnderstandingLiteracy.html.
Centers for Disease Control and Prevention (2021b). The CDC Clear Communication Index. Available from: https://www.cdc.gov/ccindex/index.html.
Choi, S. K., Seel, J. S., Yelton, B., Steck, S. E., McCormick, D. P., Payne, J., et al. (2018). Prostate Cancer Information Available in Health-Care Provider Offices: An Analysis of Content, Readability, and Cultural Sensitivity. Am. J. Mens Health 12 (4), 1160–1167. doi:10.1177/1557988318768599
Choi, W. (2020). Older Adultsʼ Credibility Assessment of Online Health Information: An Exploratory Study Using an Extended Typology of Web Credibility. J. Assoc. Inf. Sci. Technol. 71 (11), 1295–1307. doi:10.1002/asi.24341
Flesch, R. (1948). A New Readability Yardstick. J. Appl. Psychol. 32 (3), 221–233. doi:10.1037/h0057532
Friedman, D. B., Arent, M. A., Yelton, B., Sakhuja, M., Haynes, V. E., Noblet, S., et al. (2020). Development of a Clinical-Academic-Community Collaboration to Improve Health Literacy. J. Prim. Care Community Health 11, 2150132720957440. doi:10.1177/2150132720957440
Friedman, D. B., and Kao, E. K. (2008). A Comprehensive Assessment of the Difficulty Level and Cultural Sensitivity of Online Cancer Prevention Resources for Older Minority Men. Prev. Chronic Dis. 5 (1), A07.
Friedman, D. B., Laditka, J. N., Hunter, R., Ivey, S. L., Wu, B., Laditka, S. B., et al. (2009). Getting the Message Out about Cognitive Health: A Cross-Cultural Comparison of Older Adults' media Awareness and Communication Needs on How to Maintain a Healthy Brain. Gerontologist 49 (Suppl. 1), S50–S60. doi:10.1093/geront/gnp080
Friedman, D. B., and Hoffman-Goetz, L. (2006). A Systematic Review of Readability and Comprehension Instruments Used for Print and Web-Based Cancer Information. Health Educ. Behav. 33 (3), 352–373. doi:10.1177/1090198105277329
Friedman, D. B., and Hoffman-Goetz, L. (2006). Assessment of Cultural Sensitivity of Cancer Information in Ethnic Print media. J. Health Commun. 11 (4), 425–447. doi:10.1080/10810730600671920
Friedman, D. B., Kim, S.-H., Tanner, A., Bergeron, C. D., Foster, C., and General, K. (2014). How Are We Communicating about Clinical Trials? Contemp. Clin. Trials 38 (2), 275–283. doi:10.1016/j.cct.2014.05.004
Friedman, D. B., Thomas, T. L., Owens, O. L., and Hébert, J. R. (2012). It Takes Two to Talk about Prostate Cancer. Am. J. Mens Health 6 (6), 472–484. doi:10.1177/1557988312453478
Google LLC (2021). Google Trends. Available from: https://trends.google.com/trends/explore?geo=US-SC&q=coronavirus.
Gubbins, P. O., Micek, S. T., Badowski, M., Cheng, J., Gallagher, J., Johnson, S. G., et al. (2014). Innovation in Clinical Pharmacy Practice and Opportunities for Academic-Practice Partnership. Pharmacotherapy 34 (5), e45–54. doi:10.1002/phar.1427
Hadley, J., and Cunningham, P. (2004). Availability of Safety Net Providers and Access to Care of Uninsured Persons. Health Serv. Res. 39 (5), 1527–1546. doi:10.1111/j.1475-6773.2004.00302.x
Jackson, D. N., Peterson, E. B., Blake, K. D., Coa, K., and Chou, W.-Y. S. (2019). Americans' Trust in Health Information Sources: Trends and Sociodemographic Predictors. Am. J. Health Promot. 33 (8), 1187–1193. doi:10.1177/0890117119861280
Jaiswal, J., LoSchiavo, C., and Perlman, D. C. (2020). Disinformation, Misinformation and Inequality-Driven Mistrust in the Time of COVID-19: Lessons Unlearned from AIDS Denialism. AIDS Behav. 24 (10), 2776–2780. doi:10.1007/s10461-020-02925-y
Kincaid, J. P., Fishburne, R. P., Rogers, R. L., and Chissom, B. S. (1975). Derivation of New Readability Formulas (Automated Readability index, Fog Count and Flesch reading Ease Formula) for Navy Enlisted Personnel. Millington, TN: Naval Technical Training Command Millington TN Research Branch.
Kirzinger, A., Kearney, A., Hamel, L., and Brodie, M. (2020). KFF Health Tracking Poll - Early April 2020: The Impact of Coronavirus on Life in America. San Francisco, CA: Kaiser Family Foundation. Available from: https://www.kff.org/coronavirus-covid-19/report/kff-health-tracking-poll-early-april-2020/.
Kulkarni, S., Lewis, K., Arp Adams, S., Brandt, H. M., Lead, J. R., Ureda, J. R., et al. (2018). A Comprehensive Analysis of How Environmental Risks of Breast Cancer Are Portrayed on the Internet. Am. J. Health Edu. 49 (4), 222–233. doi:10.1080/19325037.2018.1473182
Lin, D., Friedman, D. B., Qiao, S., Tam, C. C., Li, X., and Li, X. (2020). Information Uncertainty: A Correlate for Acute Stress Disorder during the COVID-19 Outbreak in China. BMC Public Health 20, 1867. doi:10.1186/s12889-020-09952-3
MacLeod, S., Musich, S., Gulyas, S., Cheng, Y., Tkatch, R., Cempellin, D., et al. (2017). The Impact of Inadequate Health Literacy on Patient Satisfaction, Healthcare Utilization, and Expenditures Among Older Adults. Geriatr. Nurs. 38 (4), 334–341. doi:10.1016/j.gerinurse.2016.12.003
McFadden, S. M., Malik, A. A., Aguolu, O. G., Willebrand, K. S., and Omer, S. B. (2020). Perceptions of the Adult US Population Regarding the Novel Coronavirus Outbreak. PLoS One 15 (4), e0231808–e. doi:10.1371/journal.pone.0231808
Meppelink, C. S., van Weert, J. C., Haven, C. J., and Smit, E. G. (2015). The Effectiveness of Health Animations in Audiences with Different Health Literacy Levels: an Experimental Study. J. Med. Internet Res. 17 (1), e11. doi:10.2196/jmir.3979
Mishra, V., and Dexter, J. P. (2020). Comparison of Readability of Official Public Health Information about COVID-19 on Websites of International Agencies and the Governments of 15 Countries. JAMA Netw. Open 3 (8), e2018033. doi:10.1001/jamanetworkopen.2020.18033
National Center for Immunization and Respiratory Diseases, Division of Viral Diseases (2020). Health Equity Considerations and Racial and Ethnic Minority Groups. Centers for Disease Control and Prevention. Available from: https://www.cdc.gov/coronavirus/2019-ncov/need-extra-precautions/racial-ethnic-minorities.html.
Office of Disease Prevention and Health Promotion (2020a). United States Department of Health and Human Services. Soc. Determinants Health. Available from: https://www.healthypeople.gov/2020/topics-objectives/topic/social-determinants-of-health.
Office of Disease Prevention and Health Promotion (2020b). United States Department of Health and Human Services. Nutr. Phys. Activity, Obes. Available from: https://www.healthypeople.gov/2020/leading-health-indicators/2020-lhi-topics/Nutrition-Physical-Activity-and-Obesity.
Office of Disease Prevention and Health Promotion, United States Department of Health and Human Services (2016). Health Literacy Online: A Guide for Simplifying the User Experience. Available from: https://health.gov/healthliteracyonline.
Olscamp, K., Tang, W., Choi, S. K., Kannaley, K., Wilcox, S., and Friedman, D. B. (2019). Physical Activity and Brain Health: An Analysis of Messages from Organizations and Caregiver Perceptions. Gerontol. Geriatr. Med. 5, 2333721419855794. doi:10.1177/2333721419855794
Paakkari, L., and Okan, O. (2020). COVID-19: Health Literacy Is an Underestimated Problem. The Lancet Public Health 5 (5), e249–e250. doi:10.1016/s2468-2667(20)30086-4
Parmet, W. E., and Paul, J. (2020). COVID-19: The First Posttruth Pandemic. Am. J. Public Health 110 (7), 945–946. doi:10.2105/ajph.2020.305721
Peterson, E. B., Chou, W.-Y. S., Kelley, D. E., and Hesse, B. (2020). Trust in National Health Information Sources in the United States: Comparing Predictors and Levels of Trust across Three Health Domains. Translational Behav. Med. 10 (4), 978–988. doi:10.1093/tbm/ibz066
Ratzan, S. C. P. R. (2000). “Introduction,” in National Library of Medicine Current Bibliographies in Medicine: Health Literacy. Editors C. R. Z. M. Selden, S. C. Ratzan, and R. M. Parker (Bethesda, MD: National Institutes of Health).
Rodriguez, J. A., Davis, R. B., and Percac-Lima, S. (2019). Non-English Language Availability of Community Health Center Websites. Med. Care 57 (Suppl. 6 Suppl 2), S121–s6. doi:10.1097/MLR.0000000000001027
Sentell, T., Vamos, S., and Okan, O. (2020). Interdisciplinary Perspectives on Health Literacy Research Around the World: More Important Than Ever in a Time of COVID-19. Int. J. Environ. Res. Public Health 17 (9). doi:10.3390/ijerph17093010
South Carolina Department of Health and Environmental Control (2021). South Carolina County-Level Data for COVID-19. Available from: https://scdhec.gov/covid19/south-carolina-county-level-data-covid-19.
Szmuda, T., Özdemir, C., Ali, S., Singh, A., Syed, M. T., and Słoniewski, P. (2020). Readability of Online Patient Education Material for the Novel Coronavirus Disease (COVID-19): A Cross-Sectional Health Literacy Study. Public Health 185, 21–25. doi:10.1016/j.puhe.2020.05.041
Tai, D. B. G., Shah, A., Doubeni, C. A., Sia, I. G., and Wieland, M. L. (2020). The Disproportionate Impact of COVID-19 on Racial and Ethnic Minorities in the United States. Clin. Infect. Dis.. doi:10.1093/cid/ciaa815
Tanner, A., and Friedman, D. B. (2011). Health on the Web: An Examination of Health Content and Mobilising Information on Local Television Websites. Inform. Health Soc. Care 36 (1), 50–61. doi:10.3109/17538157.2010.505311
The Literacy Center (2021). Fast Facts. Available from: https://theliteracycenter.org/about/fast-facts/.
Thompkins, F., Goldblum, P., Lai, T., Hansell, T., Barclay, A., and Brown, L. M. (2020). A Culturally Specific Mental Health and Spirituality Approach for African Americans Facing the COVID-19 Pandemic. Psychol. Trauma Theor. Res. Pract. Pol. 12 (5), 455–456. doi:10.1037/tra0000841
Treanor, L., and Radonjic, A. (2020). Assessing the reading Level of Online Resources on COVID-19. Can. J. Public Health 111 (4), 484–485. doi:10.17269/s41997-020-00363-w
United States Census Bureau (2021). QuickFacts: South Carolina. Available from: https://www.census.gov/quickfacts/SC.
United States Department of Health and Human Services, U.S. General Services Administration Technology Transformation Services (2020). Usability.gov: Improving the User Experience. Available from: https://www.usability.gov.
World Health Organization (2021). Coronavirus. Available from: https://www.who.int/health-topics/coronavirus.
Worrall, A. P., Connolly, M. J., O’Neill, A., O’Doherty, M., Thornton, K. P., McNally, C., et al. (2020). Readability of Online COVID-19 Health Information: A Comparison between Four English Speaking Countries. BMC Public Health 20 (1), 1635. doi:10.1186/s12889-020-09710-5
Yelton, B., Brandt, H. M., Adams, S. A., Ureda, J. R., Lead, J. R., Fedrick, D., et al. (2020). Talk about Cancer and Build Healthy Communities": How Visuals Are Starting the Conversation about Breast Cancer within African-American Communities. Int. Q. Community Health Educ. 272684X20942076. doi:10.1177/0272684x20942076
Keywords: COVID-19, coronavirus, content analysis, reading level, website information
Citation: Sakhuja M, Yelton B, Arent MA, Noblet S, Macauda MM, Fedrick D and Friedman DB (2021) Words Matter: An Analysis of the Content and Readability of COVID-19 Information on Clinic Websites. Front. Commun. 6:738467. doi: 10.3389/fcomm.2021.738467
Received: 30 July 2021; Accepted: 31 August 2021;
Published: 15 September 2021.
Edited by:
Seow Ting Lee, University of Colorado Boulder, United StatesReviewed by:
Angela Fraser, Clemson University, United StatesBryan Abendschein, Western Michigan University, United States
Copyright © 2021 Sakhuja, Yelton, Arent, Noblet, Macauda, Fedrick and Friedman. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Daniela B. Friedman, dbfriedman@sc.edu