Comprehensiveness of online sources for patient education on hereditary hearing impairment

Introduction: The present study aimed at investigating the readability of online sources on hereditary hearing impairment (HHI).

Methods: In August 2022, the search terms “hereditary hearing impairment”, “genetic deafness”, hereditary hearing loss”, and “sensorineural hearing loss of genetic origin” were entered into the Google search engine and educational materials were determined. The first 50 websites were determined for each search. The double hits were removed and websites with only graphics or tables were excluded. Websites were categorized into either a professional society, a clinical practice or a general health information website. The readability tests to evaluate the websites included: Flesch Reading Ease, Flesch–Kincaid grade level, Gunning–Fog Index, Simple Measure of Gobbledygook, Coleman–Liau Index, Automated Readability Index.

Results: Twentynine websites were included and categorized as from 4 professional societies, 11 from clinical practices and 14 providing general information. All analyzed websites required higher reading levels than sixth grade. On average 12–16 years of education is required to read and understand the websites focused on HHI. Although general health information websites have better readability, the difference was not statistically significant.

Discussion: The readability scores of every type of online educational materials on HHI are above the recommended level indicating that not all patients and parents can comprehend the information they seek for on these websites.

1. Introduction

One to three out of thousand children born with a severe to profound sensorineural hearing loss and at least 50% is attributed to a genetic cause (1, 2). There are many research centers focusing on identifying new deafness-causing genes and therefore there is a relative increase in the percentage of hearing losses attributed to genetics (3). As neonatal care improves, there is also a relative decrease in acquired deafness due to neonatal complications (4). Untreated severe (≥90 dB HL) sensorineural hearing loss (SNHL) will endanger timely speech and language development. Early diagnosis and intervention are of vital importance and providing the correct information to patients and parents is crucial (5).

The use of the internet as an information source for health-related issues is gradually increasing (6). In order to achieve positive health outcomes, individuals must be aware of how to take preventive measures against diseases and use available health care resources effectively (7) Patients and parents often consult the internet for information about their health status and treatment (8). It is the second source of health information after the clinician (9). The rate of admission to the internet is increasing particularly in stigmatized illnesses (10). Unfortunately hearing impairment (HI) may be typically accepted as a stigmatized and chronic disease although its often treatable (11). Therefore, the likelihood of parents of children with HI searching the internet is high. Also, the ongoing digitalization and improving access to internet sources will pave the way for more common disease descriptions on the web. The covid-19 pandemic has accelerated this trend and more physicians use internet sources to inform their patients, to collect information with questionnaires prior to the consultation and for teleconsultation (12). All healthcare workers in the field of SNHL know that a cochlear implant is necessary for hearing rehabilitation in patients who do not benefit from conventional hearing devices. Consequently, the educational materials on the internet related to early childhood SNHL become more important. It is hoped that high quality information on the internet accessed by parents of children with SNHL will result in earlier diagnosis and early intervention, leading to more successful rehabilitation. The cause seems very evident, but the quality of the means determines the success.

What defines the quality of an internet source for a patient and for parent education? Readability of the provided text is probably the most determining factor for quality. Thus, the literacy rates of the target population should be considered. Different approaches and methods have been suggested in the literature to determine readability of a text or a website. In the American academic system, the readability of a text is the numerical value corresponding to the academic score required to understand the text. An average American adult reads at an 8th-grade level. The American Medical Association (AMA) and the National Institute of Health recommend writing educational materials at the 6th-grade level (13–15). Insufficient knowledge about the disease or treatment negatively influences health and treatment outcomes (16). However, the websites which parents consult about the medical problems of their children are usually at a reading level that exceeds their reading skills (17).

A set of readability assessment tools are available, each with their own advantages and shortcomings. The literature indicates that 6 tools are popular in otorhinolaryngology to asses readability (18–24). The Flesch Reading Ease (FRE) was created in 1948 and is still widely used (25). It is typically used for the evaluation of medical reading materials written for adults (26, 27). The Flesch–Kincaid Grade Level (FKGL) is used to evaluate academic materials and may be readily translated to the grade of reading level in the United States. It is still used for document standardization by the US military and many institutions (28). The Gunning–Fog Index (GFI) uses the polysyllabic word count and average sentence length in a selected 100-word quotation to determine the readability level. The same variables are used by The Simple Measure of Gobbledygook (SMOG), but in this method the whole text is evaluated. SMOG was advised by the National Cancer Institute for the evaluation of health information (29). SMOG was developed in 1969, and validation tests have demonstrated that it has a significant association with needed text reading levels (27). The Coleman–Liau Index (CLI) formula determines the score level of a written document based on sentence length and number of characters (30). The Automatic Readability Index (ARI) is calculated by using the number of characters, the number of words, and the number of sentences (28).

Readability studies in otology have been published in topics like cochlear implants, tinnitus, and vestibular disorders. However, the area of hereditary hearing impairment (HHI) has not been addressed hitherto which is perhaps even re importance for families with a genetic trait. Moreover, there is a significant underserved population indicated for hearing aids and cochlear implants. As many as 30% of elderly people who could benefit from hearing aids do not have access to them. Similarly, fewer than 10% of profoundly deaf adults have access to a cochlear implant (31, 32). Therefore, it seems evident that parents who need to decide sometimes for a surgical intervention for the placement of a cochlear implant should be informed correctly and adequately, also on the topic of inheritance risks.

The number of children with childhood deafness as a result of hereditary hearing loss are increasing. The internet is a key source of health care information. The readability of health care materials related to HHI and related treatments should be payed attention to. Unfortunately websites providing information on HHI too often use medical jargon. Since there are many complicated issues to be addressed such as heritability, degree of hearing loss and highly technological treatment options such as hearing aids and cochlear implants, most parents at a reading level of AMA 8th grade may not be able to comprehend the information on these websites. In the worst case they may even refrain from seeking help therefore speech and language delay may follow. In the worst case they therefore may even refrain from seeking help, not knowing about the potential speech and language delay that may follow.

Our aims were therefore to study the readability of online sources on HHI: either professional clinical websites or general public websites.

2. Materials and methods

In August 2022, Patient Educational Materials (PEM) about “hereditary hearing impairment” were determined by searching in Google Search. The search terms used were “hereditary hearing impairment”, “genetic deafness”, “hereditary hearing loss”, and “sensorineural hearing loss of genetic origin”. In advanced search, only “full sentence” and “English language” were selected. The first 50 websites for each term research were included. A total of 200 websites were evaluated. The double hits, academic journals, videos, and websites that include only graphics or tables were excluded. Websites were divided into 3 categories: (1) professional society, (2) clinical practices, and (3) general health information websites. “Professional society” was defined as an organization that wants to promote a certain job or interest. “Clinical practice” was defined as a health institution, genetic laboratory, imaging center, hospital, or private practice that provides medical or surgical treatment mainly to outpatients. “General health information website” was defined as the websites from non-clinical institutions that provide general public health information.

Every meaningful text regarding HHI was copied to a separate Microsoft Word (version 2010; Microsoft, Redmond, WA) document. Texts that were not related to education such as webpage navigations, copyright notices, disclaimers, writer information, feedback questionnaires, links, website URLs, references, figures, tables, footnotes, addresses, and telephone numbers were deleted in order not to influence readability scores.

Readability scores were automatically calculated by transferring the texts to https://www.webfx.com/tools/read-able/ (33). Mean, standard deviation, median, minimum, maximum value, frequency, and percentage were used for descriptive statistics. The distribution of variables was checked with the Kolmogorov-Smirnov test. The Kruskal-Wallis was used for the comparison of quantitative data. SPSS 28.0 was used for statistical analyses.

3. Results

After applying our inclusion and exclusion criteria, and deleting copies, we investigated 29 websites. We included 4 PEM manuscripts from professional societies, 11 PEM manuscripts from clinical practice websites, and 14 PEM manuscripts from general information websites. Titles and hyperlinks for the 29 included websites are listed in Table 1.

TABLE 1

Table 1. Titles and hyperlinks for the 29 included professional society, clinical practices and general health information websites websites.

The readability scores are shown in Table 2, which indicate that on average 12–16 years of education is required to read and understand the websites focused on HHI. All of the websites surpassed the recommended reading level of sixth grade.

TABLE 2

Table 2. Overall readability scores for the websites.

Reading difficulty was categorized by US Department of Health and Human Services as 6 grade and below, easy; 7th grade–9th grades were classified as average difficulty and those over 9th grade were classified as difficult (34). There were no manuscripts written at the 6th-grade level or below. While two clinical practice websites were at the 8th-grade readability level according to FRE scoring, only one out of eight general health information websites were at the 8th-grade readability level. In the FKGL scoring, two clinical practice websites were at the 8th-grade readability level. Of the general health information websites, two were at the 8th-grade level, and one was at the 7th-grade level. In the GFI scoring, all manuscripts were above the 8th-grade level. In the SMOG scoring, three clinical practice websites were at the 8th-grade level, one general health information website was at the 7th-grade level, and two were at the 8th-grade level. In the CLI scoring, all manuscripts were above the 8th-grade level. In the ARI scoring, two clinical practice websites were at the 8th-grade level, while one general health information website was at the 7th-grade level, two were at the 8th-grade level. All professional websites were more difficult than the 8th-grade level in all scorings. Although general health information websites had higher easy-to-read scores, the difference was not statistically significant (Table 3).

TABLE 3

Table 3. Comparison of readability scores between professional, general and clinical practice websites.

4. Discussion

It is very important to customize the readability of PEM to a level that is understandable for patient and parents. Just like the use of specific language by clinicians, these texts should be modified so that they are easy for parents to understand. The AMA has stated that to reach that level, manuscripts addressing medical education for patients should be written at a 6th grade reading level.

Almost half of the parents of children with ear-nose-throat problems search for the medical needs of their children on the internet (35). Most of these parents are not familiar with scientific search engines like PubMed and will probably perform the search using the most accessible and most used search engines, such as Google. Therefore, we assessed the readability of the most useful sources for patient education that could be found with Google (36). Manuscripts were analyzed in three groups: professional society, clinical practice and general health information. Most of the related websites were above the 6th-grade level that is recommended by the AMA (37).

Readability scores are not absolute indicators of reader comprehension. They are only predictors. Using fewer words and shorter phrases in a document does not guarantee that the patient will recall or understand more of the information. A patient's comprehension of the content on a website is impacted by a greater number of factors, including, grammar, layout, diagrams, syntax, audio, and videos. Our study did not take these factors into account since they could not be assessed by the readability tools selected. No gold standard metric is available for evaluating a health-related website. In a study on the optimal development of health-related websites, 33 factors were rated as very important. These include, the design of the website and the use of color, an easy login procedure, availability, and the provision of graphical representations of cartoons, pictures, and related information (38). None of these factors were assessed in this study. Marketing strategies involve web design and colors that have been studied and correlated to how long people surf at certain websites. We should keep in mind most evaluations are in English. Readability is difficult to achieve when educational material is translated in from medical jargon to plain English. If, in addition to this, the English is then translated into other languages readability is likely to be further reduced.

The present study has some other limitations. First, the medical terms, “hereditary hearing impairment”, “genetic deafness”, “hereditary hearing loss”, and “sensorineural hearing loss of genetic origin” were used in the initial search. The prevalence of this polysyllabic word in the selected text may have skewed readability scores by overestimating the difficulty. Second, without considering the quality, the first 50 websites, falling within our inclusion criteria, for each term were included. Third, the Google search that was performed for this research may not be representative of the experience of each individual patient. The internet provides access to a great number of search engines. We went with Google since it is by far the most popular search engine and is responsible for around two-thirds of all searches conducted on the internet (39). Fourth, only the sources written in the English language were evaluated. Future studies may be useful to evaluate the readability of online sources written in other languages.

The quality of PEM is also important; however, it is a challenging concept to quantify. The DISCERN and CRAAP tests were used to evaluate the quality of websites (40). But tools used to evaluate quality are more subjective. The potency of readability evaluation tools is their being objective.

In the existing literature, Joo et al. (2021) have shown that 128 PEM on the American Academy of Otolaryngology-Head and Neck Surgery, the Canadian Society of Otolaryngology-Head and Neck Surgery, and Ear, Nose, and Throat United Kingdom websites were also above the readability level recommended in the FRE, FKGL, and SMOG readability tools (18). Another study by Svider et al. (2013) showed that all materials evaluated were written at levels above the recommended guidelines (19). Kevin et al. (2017) assessed 126 online information resources published by the American Academy of Otolaryngology—Head and Neck Surgery Foundation and determined that the average readability level was at 10th grade or higher (20). If we look at other disciplines, we see this same trend. Ashley and Amanda (2017) showed that documents on swallowing disorders were written at the 11th-grade level (21). PEM on parathyroid surgery were analyzed by Patel and colleagues and they showed that documents were written above recommended level (22). Elysia et al. (2021) analyzed 85 internet-based PEM about nasal septoplasty and concluded that the readability was at the 10th-grade level (23). Benjamin and Winslo (2022) evaluated 26 internet-based educational sources about the branchial cleft cyst and found that only 3.8% were written at or below the sixth-grade reading level (24). Similar results were shown in many fields of medicine besides ear-nose-throat. The studies in the field of orthopedics, plastic surgery, ophthalmology, urology, and radiology reported that the analyzed sources did not meet the criteria of the current readability guidelines (41–45).

In our study, the information relating to HHI provided by professional society, clinical practices, and general healthcare websites were above the recommended readability level. As professional websites utilize medical terminology to increase reading rates, this is not surprising. The high level of readability of clinical practice websites may be reasonable due to the medical terms used, although they still provide patient information. However, general websites also have higher readability scores than recommended, similar to professional society and clinical practice websites. All of the FRE, FKGL, GFI, SMOG, CLI, and ARI measures demonstrated similar results. Even the best SMOG scoring of general websites indicates a 10th-grade readability level. Similarly, although the best level for professional society websites was found in the SMOG tool, this level is even above the 11th-grade level. The clinical practice also has the best score in the SMOG tool, and it is at the 11th-grade level. All these scores are far above the recommended level, the 6th-grade level.

The readability scores cannot evaluate how accurate the website is in terms of science. In some studies, authors used the control lists developed by them to evaluate the scientific accuracy of adenotonsillectomy and ear surgery videos on YouTube (46, 47). However, self-developed control lists may not be validated or reliable. Besides, these control lists may be needed to be developed separately for each subject and this may be time-consuming.

5. Conclusion

The treatment of HHI depends on early detection and intervention. Therefore, it is of the utmost importance to provide readable and understandable patient and parent education material. Here we conclude that the available web-based sources of educational material need to improve their readability to serve this goal.

Main points

• No websites provide information for patient and parents on HHI at the recommended level of AMA 6th grade in our study.

• All professional websites were more difficult than the 8th-grade level in all scorings.

• In the CLI scoring, all manuscripts were above the 8th-grade level.

• In the GFI index, all manuscripts were above the 8th-grade level.

• Only, 5 out 29 searched websites were at an 8th grade level.

Data availability statement

The original contributions presented in the study are included in the article, further inquiries can be directed to the corresponding author.

Author contributions

YD and VT contributed to the conception and design of the study. YD and AT wrote the first draft of the manuscript. BE, SS and AC co-designed concept and helped with data collection. SS and BE contributed to the data analysis. PD and VT were involved in inclusion and postoperative evaluation of participants. All authors contributed to the article and approved the submitted version.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. Morton CC, Nance WE. Newborn hearing screening — a silent revolution. N Engl J Med. (2006) 354(20):2151–64. doi: 10.1056/NEJMra050700

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Bitner-Glindzicz M. Hereditary deafness and phenotyping in humans. Br Med Bull. (2002) 63:73–94. doi: 10.1093/bmb/63.1.73

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Tekin AM, Bahşi İ, Bayazit YA, Topsakal V. Global research on hereditary hearing impairment over the last 40 years: a bibliometric study. J Int Adv Otol. (2021) 17(6):482–91. doi: 10.5152/iao.2021.21276

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Topsakal V, Van Camp G, Van De Heyning P. Genetic testing for hearing impairment. B-ENT. (2005) 1(3):125–35.16255497

PubMed Abstract | Google Scholar

5. Yoshinaga-Itano C, Sedey AL, Wiggin M, Chung W. Early hearing detection and vocabulary of children with hearing loss. Pediatrics. (2017) 140(2):e20162964. doi: 10.1542/peds.2016-2964

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Powell J, Inglis N, Ronnie J, Large S. The characteristics and motivations of online health information seekers: cross-sectional survey and qualitative interview study. J Med Internet Res. (2011) 13(1):1–11. doi: 10.2196/jmir.1600

CrossRef Full Text | Google Scholar

7. The Lancet. Why is health literacy failing so many? Lancet. (2022) 400(10364):1655. doi: 10.1016/S0140-6736(22)02301-7

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Bouche G, Migeot V. Parental use of the internet to seek health information and primary care utilisation for their child: a cross-sectional study. BMC Public Health. (2008) 8:1–9. doi: 10.1186/1471-2458-8-300

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Couper MP, Singer E, Levin CA, Fowler FJ, Fagerlin A, Zikmund-Fisher BJ. Use of the internet and ratings of information sources for medical decisions: results from the DECISIONS survey. Med Decis Mak. (2010) 30(5):106S–14S. doi: 10.1177/0272989X10377661

CrossRef Full Text | Google Scholar

10. Berger M, Wagner TH, Baker LC. Internet use and stigmatized illness. Soc Sci Med. (2005) 61(8):1821–7. doi: 10.1016/j.socscimed.2005.03.025

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Wallhagen MI. The stigma of hearing loss. Gerontologist. (2010) 50(1):66–75. doi: 10.1093/geront/gnp107

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Fernández Coves A, Yeung KHT, van der Putten IM, Nelson EAS. Teleconsultation adoption since COVID-19: comparison of barriers and facilitators in primary care settings in Hong Kong and The Netherlands. Health Policy (New York). (2022) 126(10):933–44. doi: 10.1016/j.healthpol.2022.07.012

CrossRef Full Text | Google Scholar

13. Affairs A complete list of the members of the AHC on HL and the members and staff of the C on S. Health literacy report of the council on scientific affairs. Ad hoc committee on health literacy for the council on scientific affairs, American medical association. JAMA. (1999) 281(6):552–7. doi: 10.1001/jama.281.6.552

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Weiss BD. Health Literacy A Manual for Clinicians. (2003).

15. The National Library of Medicine (MedlinePlus). How to Write Easy-to-Read Health Materials. (2022). p. 1–6. Available at: https://medlineplus.gov/pdf/health-education-materials-assessment-tool.pdf

16. Asnani MR, Barton-Gooden A, Grindley M, Knight-Madden J. Disease knowledge, illness perceptions, and quality of life in adolescents with sickle cell disease: is there a link? Glob Pediatr Heal. (2017) 4:1–10. doi: 10.1177/2333794X17739194

CrossRef Full Text | Google Scholar

17. Williams MV. Recognizing and overcoming inadequate health literacy, a barrier to care. Cleve Clin J Med. (2002) 69(5):415–8. doi: 10.3949/ccjm.69.5.415

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Kim JH, Grose E, Philteos J, Forner D, Noel CW, Wu V, et al. Readability of the American, Canadian, and British otolaryngology–head and neck surgery societies’ patient materials. Otolaryngol Head Neck Surg. (2021) 166(5):862–8. doi: 10.1177/01945998211033254

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Svider PF, Agarwal N, Choudhry OJ, Hajart AF, Baredes S, Liu JK, et al. Readability assessment of online patient education materials from academic otolaryngology-head and neck surgery departments. Am J Otolaryngol Head Neck Med Surg. (2013) 34(1):31–5. doi: 10.1016/j.amjoto.2012.08.001.

CrossRef Full Text | Google Scholar

20. Wong K, Levi JR. Readability trends of online information by the American academy of otolaryngology—head and neck surgery foundation. Otolaryngol Head Neck Surg. (2017) 156(1):96–102. doi: 10.1177/0194599816674711

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Ferster APOC, Hu A. Evaluating the quality and readability of internet information sources regarding the treatment of swallowing disorders. Ear, Nose Throat J. (2017) 96(3):128–38. doi: 10.1177/014556131709600312

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Patel CR, Sanghvi S, Cherla DV, Baredes S, Eloy JA. Readability assessment of internet-based patient education materials related to parathyroid surgery. Ann Otol Rhinol Laryngol. (2015) 124(7):523–7. doi: 10.1177/0003489414567938

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Grose EM, Holmes CP, Aravinthan KA, Wu V, Lee JM. Readability and quality assessment of internet-based patient education materials related to nasal septoplasty. J Otolaryngol Head Neck Surg. (2021) 50(16):2–8. doi: 10.1097/MBC.0000000000001051

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Daines BS, Idicula W. Quality and readability of online patient education materials related to branchial cleft cysts. Cureus. (2022) 14(4):10–5. doi: 10.7759/cureus.24287

CrossRef Full Text | Google Scholar

25. Flesch R. A new readability yardstick. J Appl Psychol. (1948) 32(3):221–33. doi: 10.1037/h0057532

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Paasche-Orlow MK, Taylor HA, Brancati FL. Readability standards for informed-consent forms as compared with actual readability. N Engl J Med. (2003) 348(8):721–6. doi: 10.1056/NEJMsa021212

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Eloy JA, Li S, Kasabwala K, Agarwa N, Hansberry DR, Baredes S, et al. Readability assessment of patient education materials on major otolaryngology association websites. Otolaryngol Head Neck Surg. (2012) 147(5):848–54. doi: 10.1177/0194599812456152

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Kincaid JP, Fishburne RP Jr, Rogers RL, Chissom BS. Derivation of new readability formulas (automated readability index, fog count and flesch Reading ease formula) for navy enlisted personnel. Millington: Naval Technical Training Command Millington TN Research Branch (1975). Available at: http://oai.dtic.mil/oai/oai?verb=getRecord&metadataPrefix=html&identifier=ADA006655

29. Kong K, Hu A. Readability assessment of online tracheostomy care resources. Otolaryngol Head Neck Surg. (2015) 152(2):272–8. doi: 10.1177/0194599814560338

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Kasabwala K, Agarwal N, Hansberry DR, Baredes S, Eloy JA. Readability assessment of patient education materials from the American academy of otolaryngology - head and neck surgery foundation. Otolaryngol Head Neck Surg. (2012) 147(3):466–71. doi: 10.1177/0194599812442783

PubMed Abstract | CrossRef Full Text | Google Scholar

31. D’Haese PSC, Van Rompaey V, De Bodt M, Van de Heyning P. Severe hearing loss in the aging population poses a global public health challenge. How can we better realize the benefits of cochlear implantation to mitigate this crisis? Front Public Heal. (2019) 7(August):4–6. doi: 10.3389/fpubh.2019.00227

CrossRef Full Text | Google Scholar

32. D’haese PSC, De Bodt M, Van Rompaey V, Van de Heyning P. Awareness of hearing loss in older adults: results of a survey conducted in 500 subjects across 5 European countries as a basis for an online awareness campaign. Inquiry. (2018) 55:1–9. doi: 10.1177/0046958018759421

CrossRef Full Text | Google Scholar

33. Dowdle TS, Nguyen JM, Steadman JN, Layher H, Sturgeon ALE, Akin R. Online readability analysis: mohs micrographic surgery postsurgical wound care. Adv Ski Wound Care. (2022) 35(4):213–8. doi: 10.1097/01.ASW.0000816960.79821.e3

CrossRef Full Text | Google Scholar

34. Cherla DV, Sanghvi S, Choudhry OJ, Liu JK, Eloy JA. Readability assessment of internet-based patient education materials related to endoscopic sinus surgery. Laryngoscope. (2012) 122(8):1649–54. doi: 10.1002/lary.23309

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Boston MM, Ruwe E, Duggins A, Willging JP. Internet use by parents of children undergoing outpatient otolaryngology procedures. Arch Otolaryngol—Head Neck Surg. (2005) 131(8):719–22. doi: 10.1001/archotol.131.8.719

CrossRef Full Text | Google Scholar

36. Pehora C, Gajaria N, Stoute M, Fracassa S, Serebale-O’Sullivan R, Matava CT. Are parents getting it right? A survey of parents’ internet use for children’s health care information. Interact J Med Res. (2015) 4(2):e12. doi: 10.2196/ijmr.3790

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Weiss BD. Health literacy and patient safety: help patients understand (manual for clinicians, second edition). Am Med Assoc Found Am Med Assoc. (2007) 62:1–62.

Google Scholar

38. Schneider F, Van Osch L, De Vries H. Identifying factors for optimal development of health-related websites: a delphi study among experts and potential future users. J Med Internet Res. (2012) 14(1):1–14. doi: 10.2196/jmir.1863

CrossRef Full Text | Google Scholar

39. Sterling G. Data: google monthly search volume dwarfs rivals because of mobile advantage—search engine land. Available at: https://searchengineland.com/data-google- monthly-search-volume-dwarfs-rivals-mobile-advantage-269120

40. Portillo IA, Johnson CV, Johnson SY. Quality evaluation of consumer health information websites found on google using DISCERN, CRAAP, and HONcode. Med Ref Serv Q. (2021) 40(4):396–407. doi: 10.1080/02763869.2021.1987799

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Ó Doinn T, Broderick JM, Clarke R, Hogan N. Readability of patient educational materials in sports medicine. Orthop J Sport Med. (2022) 10(5):1–8. doi: 10.1177/23259671221092356

CrossRef Full Text | Google Scholar

42. Chang IA, Wells MW, Zheng DX, Mulligan KM, Wong C, Scott JF, et al. A multimetric readability analysis of online patient educational materials for submental fat reduction. Aesthetic Plast Surg. (2022) 46(2):712–8. doi: 10.1007/s00266-021-02675-9

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Huang G, Fang CH, Agarwal N, Bhagat N, Eloy JA, Langer PD. Assessment of online patient education materials from major ophthalmologic associations. JAMA Ophthalmol. (2015) 133(4):449–54. doi: 10.1001/jamaophthalmol.2014.6104

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Pattenden TA, Raleigh RA, Pattenden ER, Thangasamy IA. Quality and readability of online patient information on treatment for erectile dysfunction. BJUI Compass. (2021) 2(6):412–8. doi: 10.1002/bco2.87

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Delaney FT, Doinn T, Broderick JM, Stanley E. Readability of patient education materials related to radiation safety: what are the implications for patient-centred radiology care? Insights Imaging. (2021) 12(1):1–9. doi: 10.1186/s13244-021-01094-3

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Strychowsky JE, Nayan S, Farrokhyar F, MacLean J. YouTube: a good source of information on pediatric tonsillectomy? Int J Pediatr Otorhinolaryngol. (2013) 77(6):972–5. doi: 10.1016/j.ijporl.2013.03.023

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Sorensen JA, Pusz MD, Brietzke SE. YouTube as an information source for pediatric adenotonsillectomy and ear tube surgery. Int J Pediatr Otorhinolaryngol. (2014) 78(1):65–70. doi: 10.1016/j.ijporl.2013.10.045

PubMed Abstract | CrossRef Full Text | Google Scholar