94% of researchers rate our articles as excellent or good
Learn more about the work of our research integrity team to safeguard the quality of each article we publish.
Find out more
Lukas Lange-Drenth1*
Hellena Willemer1
Mirjam Banse2Anke Ernst2Anne Daubmann3Anja Holz3
Christiane Bleich1Susanne Weg-Remers2,†
Holger Schulz1,†
- 1Department of Medical Psychology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
- 2Division Cancer Information Service, German Cancer Research Center (DKFZ), Heidelberg, Germany
- 3Institute of Medical Biometry and Epidemiology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
Background: The Internet allows cancer patients to access information about their disease at any time. However, the quality of online information varies widely and is often inaccurate or does not provide all the details patients need to make informed decisions. Additionally, patients’ often limited ability to find and evaluate cancer-related online information can lead to misinformation.
Objective: An interactive e-learning environment to promote digital health literacy will be developed and evaluated for effectiveness.
Primary hypothesis: Cancer patients who use the e-learning environment (IG1.1–IG1.3) or the content of the environment as a non-interactive PDF file (IG2) will show greater improvement in their digital health literacy from baseline to 8 weeks after baseline compared to patients who receive no such intervention, but are referred to a standard information brochure.
Methods: The hypothesis will be tested in a stratified randomized controlled superiority trial with five parallel groups and the primary endpoint of digital health literacy. In an e-learning environment, patients will learn strategies to use when searching for reliable cancer-related online information. During development, a prototype will be refined through focus groups and tested for usability by experts and patients. 660 cancer patients will be recruited using convenience sampling and randomly assigned in a 3:1:1 ratio to IG1.1-IG1.3 (three variants of the environment), IG2, or the control group. Two thirds of the 660 participants will be recruited through the German Cancer Information Service (CIS) and one third through non-CIS routes. Allocation will follow stratified randomization, accounting for recruitment route (CIS vs. non-CIS) and cancer type (breast cancer vs. other cancers), with variable block length. The primary outcome, digital health literacy, will be measured at baseline, 2 weeks, and 8 weeks after baseline.
Conclusion: If the results support the primary hypothesis, then the e-learning environment could empower patients to retrieve more reliable information about their disease. Concerns about the generalizability of the results, since a disproportionate number of inquiries to the CIS come from breast cancer patients, are addressed by a proportionally stratified randomization strategy and diversified recruitment routes.
1 Introduction
1.1 Background
The need for information is one of the most common unmet support needs of cancer patients during the course of the disease (1). Most cancer patients want to receive all available information about their disease and treatment (2, 3) and the proportion of cancer patients using the internet to search for cancer-related information is high (>70%) (4–6) and will continue to increase in the future (7). In Germany in 2019, even in the 65+ age group, 68% had searched for health-related information on the internet in the previous 3 months (8). Approximately 20% of cancer patients use health-related discussion forums or blogs (6) and about 35% reported using social media (6, 9).
Searching for health-related information on the internet is positively associated with patient-reported outcome measures (PROMs) and socioeconomic characteristics of patients. Cross-sectional studies indicate that cancer patients who search the internet for cancer-related information are more involved in medical decision-making (10, 11), feel better informed about their disease (5), have higher levels of self-reported health (12) and quality of life (QoL) (13), are more likely to have a partner (6), are younger and more educated (4, 6, 13) than patients who do not search the internet. Depending on whether patients discuss health-related online information with their physician, searching for health information online can also improve the doctor-patient relationship for patients with acute or chronic conditions (14).
The quality of cancer-related information available on information websites, social media, and from artificial intelligence (AI) chatbots varies. Information on websites, as measured by the DISCERN instrument developed to help consumers evaluate the quality of health-related information about treatment options for a health problem, is often interest-driven, incomplete, or out-of-date. Additionally, such information may not provide all the details necessary for cancer patients to properly assess its relevance in the context of their individual disease and make informed decisions with their healthcare provider based on the information received (15–18). More research is needed on the quality of information posted on social networking sites such as Facebook or Instagram and on discussion forums and blogs, where informal advice and support is given by layman without quality control. Low to moderate quality and high rates of misinformation for prostate and bladder cancer measured by the DISCERN instrument have been found on several social networks, including TikTok, Instagram, and YouTube (19, 20). Encouragingly, studies results show that the dissemination of evidence-based information and social support through X-messaging is associated with an increase in self-perceived cancer-related knowledge (21), that participation in social media support groups can improve breast cancer patients’ understanding of their disease and its management (21), and that frequently shared (retweeted) X-messages are more likely to contain medically accurate information than randomly selected X-messages (22). Initial studies concerning AI chatbots suggest that for the top cancer-related searches for prostate, bladder, kidney, and testicular cancer, AI chatbots produce information that may be partially accurate and of moderately high quality as measured by the DISCERN instrument. However, their responses are quite difficult to read, moderately difficult to understand, and lack clear instructions for users to follow (23, 24). Deviating or even incorrect information, for example on treatment recommendations, were found in a certain percentage of responses, using the National Comprehensive Cancer Network (NCCN) (25) guidelines as a benchmark (26). Assessment tools for health-related (online) information that patients can use freely, such as the DISCERN criteria, have shortcomings (1): low interrater reliability (27), (2) subscales with only one item, i.e., with only limited reliability (28) and (3) the ratings are highly subjective (28).
Due to the varying quality of health-related online information, patients therefore depend on their own ability to find and critically evaluate online cancer information. These skills are part of the concept of eHealth literacy or digital health literacy. A systematic review shows that eHealth literacy levels among older adults are positive associated with behavioral and cognitive outcomes, such as health-promoting behaviors, decision-making, and health knowledge. However, evidence for physical and psychological outcomes is inconsistent, highlighting the need for high-quality studies to better understand these relationships (29). The overall eHealth literacy level among cancer survivors is not high, and influenced by factors such as age, gender, education and social support (30). An early definition of eHealth literacy described it as “the ability to seek, find, understand, and appraise health information from electronic sources and apply the knowledge gained to addressing or solving a health problem” (31). This definition served as the foundation for developing the eHealth Literacy Scale (eHEALS) (32), which has become the most widely used and translated questionnaire for assessing eHealth literacy (33). However, although the authors of the eHEALS conceptualized eHealth literacy as encompassing six core skills, the scale itself is unidimensional, which does not align with their multidimensional framework. Performance test analyses of the digital health literacy of patients with cancer and rheumatoid arthritis identified five types of skills that are essential for successfully searching for cancer-related information on websites (34, 35): (1) operational skills, to use the digital device and Internet browser, (2) navigation skills, to navigate back and forth between websites and orientate on the world wide web, (3) information searching skills, to use correct search strategies, i.e., to formulate appropriate search terms, (4) evaluating reliability of online information, i.e., review the source of information (5) evaluating relevance of online information (34, 35). In addition, for interactive technologies, such as forums, e-consults, patient portals, or digital treatment applications, patients need the two skills of (6) adding self-generated content to the Internet and (7) considering their own and others’ privacy, i.e., knowing who can read what they have posted on the Internet (35). These seven skills form the foundation of digital health literacy measured with the Digital Health Literacy Instrument (DHLI) (35), a validated tool designed to measure digital health literacy comprehensively. The DHLI has been translated into multiple languages and adapted for various populations, ensuring its applicability across diverse settings (33). Results of the performance based studies show that the level of information and evaluation skills was significantly lower than the level of operating and navigation skills (34, 36). Many participants had problems with formulating task-related search terms (90%) and selecting a task-related search result (52%) (34). These findings are consistent with the results of a survey in a representative sample of the German population (37) where three quarters (76%) of respondents had low digital health literacy (37). Potential confounders of the digital health literacy or eHealth literacy in healthy individuals and cancer patients are age, education and self-reported health status. Higher digital health literacy or eHealth literacy can be found in young people with higher education (30, 34–38), and higher self-reported health (35). Limitations of self-reported questionnaires, such as the DHLI and eHEALS, might also be that self-reported skills are sometimes only weakly associated with the ability to perform eHealth tasks effectively (39). Additionally, study results suggest that individuals tend to overestimate their computer skills (40, 41) as well as their ability to locate and evaluate web-based information (42).
In summary, cancer patients are a vulnerable group who need support in finding and evaluating online information due to their high information needs, the variable quality of cancer-related online information, and the fact that the majority do not have the necessary information search and credibility assessment skills to handle cancer-related information online correctly. An e-learning environment to improve online health literacy is an effective way to reach as many patients as possible and participants can learn and gain information easily from anywhere. We interpret e-learning in its broadest sense as “instructions delivered on a digital device that is intended to support learning” (43). This encompasses a diverse array of digital training tools, including web-based learning modules and simulation training conducted in virtual environments. A key strength of e-learning is its support for autonomous learning, enabling individuals to set goals, choose resources, and self-assess progress. It also allows for repeated review of materials and seamless content updates to maintain relevance and accuracy (44, 45). While e-learning is commonly used for knowledge transfer, it also serves as an effective platform for skills-based training (46). An initial study using an e-learning environment to improve participants’ digital health literacy indicates short-term effectiveness of the program. 134 healthy Japanese participated in interactive e-learning environments (47), which included information on: (1) reliability of information on the Internet, (2) scientific research methods, and (3) cautions regarding health information posted on social networking websites. After 2 weeks participants in the intervention group improved their self-perceived digital health literacy significantly more (Cohen's d = 0.25) than participants in the control group who received no intervention (47).
Interactive and persuasive elements to increase participants’ motivation and engagement are important components in an interactive e-learning environment designed to increase participants’ digital health literacy. Research suggests that online courses with high levels of interactivity lead to higher levels of students’ motivation, improved learning outcomes, and satisfaction than less interactive learning environments (48–51). Simple digital interactions, such as choosing when to start each segment of the presentation and answering questions about the content, have shown to positively impact learning outcomes, motivation and satisfaction with the course compared to passive learning approaches (50–52). Motivation and engagement are factors that are correlated and to some extent overlapping. Motivation can mediate the success of an e-learning intervention (53). High levels of intrinsic motivation for e-learning environments are associated with higher levels of engagement (54). Higher levels of engagement are in turn associated with higher intervention effectiveness (55). Engagement is a complex construct composed of behavioral (e.g., system usage data), cognitive (e.g., intervention credibility and motivation), and affective (e.g., satisfaction with the intervention, acceptability, and feasibility) dimensions and is therefore difficult to measure (56).
One way to improve user motivation and engagement is the use of various persuasive elements (57, 58). Persuasive systems, designed to influence users’ attitudes or behaviors, follow a structured design process, such as in the Persuasive Systems Design (PSD) model (59). This model suggests the implementation of various persuasive elements in four categories of support. In modern e-learning environments, the following persuasive elements may be used (1): the primary task support of e- learning environments is to support the user in acquiring new knowledge. Possible elements of primary task support are: (a) “reduction”: reducing complex behaviors to simple tasks; (b) “rehearsal”: providing the means to practice or repeat a behavior; and (c) “tunneling”: guiding users through a process or experience (58). (2) The credibility of e-learning environments is largely based on the user's trust in the information provider. Health-related websites operated by university institutions generally enjoy a high level of trust among users (60). Possible elements in the credibility support of e-learning environments are (a) “authority”: responsible persons should be named in the system; and (b) “verifiability” content in the system should be easily verifiable through listed external sources (58). (3) Dialog support and social support only take place to a limited extent in e-learning environments. Possible elements are: (a) “Praise”: giving positive feedback via words, images, symbols or sounds; and (b) “Rewards”: reward for performing a target behavior (58). A systematic review found a correlation between the number of persuasive elements and the effectiveness of web-based interventions (61). However, this finding should not be interpreted to mean that implementing more principles always leads to better outcomes. Rather, persuasive elements should be tested with potential end-users in the pilot phase.
E-learning environments are part of the overarching concept of eHealth (62). Although there is growing evidence in systematic reviews of the positive impact of eHealth interventions on the health of patients with chronic conditions (63, 64) the actual implementation of such programs in everyday clinical practice has proven to be a challenge (65). The distinction between the effectiveness of the implementation and the basic experimental effectiveness of the intervention (efficacy) is of crucial importance for the transfer of the interventions from the study situation to real-life care (66). Proctor (66) proposed eight dimensions to evaluate the implementation of interventions in health care: Acceptability, Adoption, Appropriateness, Feasibility, Fidelity, Cost, Penetration, and Sustainability. The implementation of interventions should be evaluated against these eight dimensions in order to obtain accurate information about the implementation process and identify potential barriers.
In summary, cancer patients often lack the digital health literacy needed to effectively find, evaluate, and use online health information. Despite their high information needs, there are no targeted interventions for this group, even as national and international health strategies prioritize improving health literacy. Addressing this gap through tailored e-learning interventions could significantly enhance cancer patients’ health outcomes and quality of life.
1.2 Objectives
We will investigate the following primary research question: (1) can cancer patients improve their digital health literacy from baseline (T0) to 8 weeks after baseline (T2) by using an interactive e-learning environment (IG1.1–IG1.3) or a non-interactive PDF file (IG2) that provides the same content as the interactive learning environment, compared to a control group? In addition, we will investigate whether (2) the primary question also applies to the course from baseline to 2 weeks after baseline (T1); (3) whether the primary question also applies with the total score of the performance-based items of the DHLI as the dependent variable; (4) whether and - if so - which elements of the primary task support, i.e., reduction and tunneling, influence the effectiveness of the intervention.; (5) whether participants’ self-reported motivation mediates the relationship between the intervention group and the change in digital health literacy from T0 to T2; (6) whether cancer patients perceive the interactive, digital e-learning environment as user-friendly, appropriate and feasible, and (7) whether cancer patients perceive the e-learning environment as more user-friendly, appropriate and feasible than the non-interactive PDF file.
1.2.1 Hypotheses
The primary hypothesis is: Cancer patients who use a digital interactive e-learning environment (IG1.1–IG1.3) or the content of the e-learning environment as a non-interactive PDF file (IG2) will show a greater improvement in their digital health literacy from T0 to T2 as compared to cancer patients who receive no intervention. The secondary hypothesis is: cancer patients who use the full e-learning environment (IG1.1) will show a greater improvement in their digital health literacy from T0 to T2 compared to cancer patients who use the e-learning environment without tunneling elements (IG1.2) or the e-learning environment without rehearsal elements (IG1.3).
2 Methods
2.1 Study design
This trial is designed as a stratified randomized controlled superiority trial with five parallel groups and the primary endpoint of digital health literacy. Participants in intervention groups one, two, and three (IG1.1–IG1.3) will receive access to an e-learning environment. Participants in intervention group four (IG2) will receive the content in a non-interactive PDF format. Participants in the control group will not receive any intervention but will be provided with a standard informational brochure. The study also involves the development and pre-testing of the intervention. During development, two focus groups will explore cancer patients’ preferences for intervention topics and design elements to improve digital health literacy. A prototype e-learning environment will undergo usability testing with patient and expert cognitive walkthroughs. The randomization will be performed as a proportionally stratified randomization with variable block length and a 1:1:1:1:1 allocation. The randomization sequence is computer-generated and securely stored within the REDCap system, which ensures that neither participants nor recruiting researchers have access to it. Both, participants and those assessing the outcomes, are unaware of the treatment allocation (blinding). PROMS will be administered at T0, T1 and T2 (Figure 1). The reporting of this study protocol followed the recommendations of the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) (67) (Supplementary Table 1).
Figure 1
Figure 1. Participant timeline (spirit figure) of enrolment, interventions, and assessments.
2.2 Eligibility criteria
Patients are eligible for the trial if they are:
- 18 years of age or older
- self-report having been diagnosed with cancer of any type
- have sufficient knowledge of the German, as all study content and questionnaires will be in German only
- should be able to use a digital device (smartphone, tablet, PC, laptop, etc.) with an Internet connection, as all study content and questionnaires will only be accessible online
- must have their own email address, as all reminders for follow-up questionnaires will be sent via email.
Patients will be excluded, if they are:
- severely cognitively impaired due to their cancer or other illnesses and/or are unable to operate a digital device cannot participate in the study.
2.3 Setting and recruitment
The enrollment period for the main study will extend over 6 months. Two thirds of the planned 660 participants will be recruited through online recruitment routes of the German Cancer Information Service (CIS), such as email, the CIS website (7.7 million visitors per year), the CIS Facebook page or the CIS Instagram page. One third of the participants will be recruited primarily in Hamburg through in a physical setting through recruitment routes not associated with the CIS, such as oncological outpatient clinics of the University Cancer Center Hamburg (UCCH) or patient support groups. Recruitment via the CIS is organized in cooperation between researchers at the University Medical Center Hamburg-Eppendorf (UKE) and the CIS's Internet Editorial Team. Patients who email the CIS for cancer-related advice will receive an invitation to participate in the study and a link to the contact website (https://www.uke.de/orientiertinformiert) in the reply from the CIS, where patients can enter their contact information (last name, first name, and email address). The contact website provides an overview of the study procedure, outlines the study's goals, specifies the inclusion and exclusion criteria, and allows patients to access the contact information of the researchers. An invitation to participate in the study and a link to the contact website will be posted on the CIS website and on the CIS Facebook and Instagram pages (currently approximately 5,800 and 4,300 followers, respectively). Patients in the waiting room of the UCCH outpatient clinic will be approached by one researcher of the study team who will hand them a card resembling a business card. Patients will be addressed without prior identification, acknowledging the possibility that the individual may not be a patient. This card will include a QR code and URL that patients can use to access the study's contact website. Additionally, the researcher will give the patients a brief description of the study, explain its goals, and address any questions they might have. An invitation to participate in the study and the link to the contact website will also be emailed to support group representatives, who will forward the email to their members. Broad recruitment through the various access channels is used to minimize sample bias by attempting to include patients of lower socioeconomic status, and thus potentially lower digital health literacy, in the study. No incentives will be used to recruit participants. Contact information provided on the contact page will be sent to the project email address. Patients will receive individualized access to the electronic data capture software REDCap (68) via the project email address.
In REDCap, patients will be given access to the informed consent form and, after signing, to the baseline questionnaire. In the consent form, patients will receive the following information about participation in the study and the processing of their personal data: (1) Participants are informed that participation in the study is voluntary and that consent to participate can be withdrawn at any time prior to anonymization without giving reasons; (2) the aim and the purpose of the study are communicated; (3) the procedure and scope of the study (time required) (4) which personal data (age, gender, education, type of cancer, self-assessed health) will be collected (5) the person responsible for data processing and their contact details will be specified; (6) the rights of participants and (7) the date for deletion of personal data (10 years after publication).
2.4 Setting and recruitment for pilot studies
The process of recruiting participants, including patients for focus groups, cognitive walkthroughs of the e-learning environment, and pre-testing of questionnaires, will involve collaboration with patient support group representatives, Instagram stories from the CIS and Facebook posts from the CIS. Invitations to participate in the study, along with a link to the contact website, will be emailed to patient support group representatives, who will then email them to their members. CIS will post Instagram stories and Facebook posts with the invitation to participate in the study and a web link to the contact website. Patients interested in participating will provide their contact information (including last name, first name, and email address) through the contact website. The research assistant will schedule a meeting with the patients via E-Mail to conduct the pretests. Patients will be asked to sign an informed consent form before the pre-test begins. In the first focus group, only cancer patients with low digital health literacy will participate. Two items of the HLS-GER 2 (item 1: “Evaluate how trustworthy the information you found is”; item 22: “Use the right words or search terms to find the information you need online”) (37) will be used. Patients will participate in the first focus group if they answer one of the two questions with “very difficult” (about 30% of the population) (37). By including patients with low digital health literacy in the early design process, we hope to increase the uptake of the intervention in this group. The participants in the second focus group and the patient-based cognitive walkthrough should differ on as many relevant factors as possible, especially on digital health literacy (based on the two HLS-GER 2 items) as well as on potential confounders (age and education) of digital health literacy (purposive sampling).
2.5 Intervention
2.5.1 Intervention content
The final e-learning environment will include text, audio instruction (with the text read aloud to support participants with varying literacy levels), images, and short videos. These materials were created using Articulate Rise 360, an easy-to-use e-learning authoring tool. The content of the e-learning environment was based on extensive research and guidelines (34, 69–77), designed to improve the different skills of digital health literacy:
(1) Evaluating reliability skills: Participants will get to know criteria and indicators for evaluating the content of the information they find and for assessing the trustworthiness of website providers. This includes, for example, to check for compliance with the transparency criteria for “good health information” (e.g., identification of the operator of the website and the cooperation partners, funding, and thus potential conflicts of interest, qualifications of authors, methodology of information production, and sources on which the information is based) (78). Moreover, participants are taught the following indicators for checking the quality of content and trustworthiness of providers, among others: Checking information against other websites. Checking whether evidence-based data sources are mentioned and whether the information is up-to-date and how often is it updated. Balance and completeness of the information provided, i.e., are the modes of action, benefits and risks of treatment procedures, and consequences of non-treatment described? What is the purpose of the website, who is the target group (79, 80)? Participants will be informed about warning signs of unreliable content: simplified messages; emotional and/or fear-mongering, unbalanced information, extreme adjectives; the report/article relies heavily on subjective information or personal experience.
Information in social media can be evaluated using the following criteria: Who follows the account? Are there contact details? Is there a reference to respectful interaction? Are advertising and editorial content clearly separated? Are there references to the topic, goal or target group? Evaluate content: Can the accuracy of the information be confirmed by other sources? Can you find similar content elsewhere? If so, is evidence-based data mentioned? (81).
(2) Improving information skills: Participants will learn simple rules on how to optimize their Google searches: use more keywords since research shows that cancer patients often use only single-word searches (34); use search operators such as OR; delete irrelevant search terms; introduction to advanced search. They will also learn simple rules to better identify which pages might be relevant in Google search results and which are not: how to recognize Google ads; view URL; view Google “snippet”.
(3) Improving navigation skills: Participants will learn simple rules for navigation; using the back button; using more than one tab; drop-down lists and anchor links are introduced. The environment will cover tips for navigating on different devices like computers, tablets, and smartphones, as these can work differently.
(4) Additional goals are to raise awareness about various aspects related to online searches, including the exploration of alternative and complementary therapies, and the importance of data protection. Finally, a summarizing checklist with the most important criteria to take into consideration when searching for reliable information and links to reliable websites are provided.
Several persuasive and interactive elements will be integrated into the learning environment to increase user motivation and engagement (57, 59). The following primary task support elements are planned for the e-learning environment (82, 83): (1) The complex task of searching for cancer-related information online will be broken down into smaller steps (e.g., how do I check if the information is up-to-date?) (reduction). (2) The participant is guided step-by-step through the separate chapters of the educational content (tunneling) and (3) then repeated or tested in performance tests or quizzes (rehearsal).
The responsible organization and the developers will be named at various points within the learning platform, which should improve the credibility of the learning platform (authority). At the same time, sources of information are provided within the application that enable participants to verify the content they have learned or to acquire additional knowledge (verifiability). Users will also receive praise in words after completing each chapter (praise).
There will be three different versions of the e-learning environment, tailored to examine different instructional features. Participants in the IG1.1 will receive the e-learning environment with all primary task support elements described. Participants in IG1.2 will receive the same content but will not be guided through the content step by step in a predetermined sequence (tunneling), but can determine the order of the content themselves. For instance, participants could skip content or navigate the e-learning material in reverse order, promoting self-directed learning. Participants in IG1.3 will also receive the same content, but all rehearsal elements such as quizzes, tests or the summary of key points at the end of each chapter will be removed from the e-learning environment. Participants in IG2 will receive the same content as participants in the IG1.1, but not within the interactive e-learning environment, but in a non-interactive PDF format. The PDF format will also not include the audio instructions. Participants in the control group will not receive any intervention and will only be referred to the CIS brochure “Your journey through cancer” (84). The brochure, while emphasizing informed decision-making and offering practical advice on medical, psychological, and social aspects of cancer care, does not include guidance on how to search for cancer-related information online.
2.5.2 Development and pre-testing of the intervention
The development of the intervention is based on the innovation and design framework of the Center for eHealth Research and Disease Management's comprehensive roadmap to guide the design and development of an evidence-based eHealth intervention (85).
The pre-testing is divided into three phases. In the first phase, two focus groups of 5–8 cancer patients each will be conducted, and then evaluated. The aim of the focus groups is to answer the following questions: (1) what topics would cancer patients like to be included in the intervention to improve their digital health literacy? (2) What design elements can help patients to improve their digital health literacy? (3) What suggestions do patients have for improving the prototype (86, 87)? During the focus groups, participants will be given a 10-minute PowerPoint presentation of the current content of the intervention to comment on. They will also be presented with different intervention design models to choose from or comment on. The focus groups will be audio recorded and transcribed verbatim. To identify topics that should be included in the intervention, or to identify design elements or suggestions for improvement, the first and second authors will follow an inductive coding process (88). Statements will first be coded and then grouped into categories and subcategories, which will then be named. The thematic analysis will be performed using MAXQDA 2022 (VERBI GmbH). Possible adjustments to the e-learning environment will be discussed by the group of authors.
The second phase is the expert-based cognitive walkthrough (89). Aim of this evaluation is to identify usability issues, including navigation challenges and interface inconsistencies. In addition, the clarity, relevance, and effectiveness of the educational content will be assessed while identifying opportunities for improvement. Five selected internet editors from the CIS, who are very experienced users and have extensive experience in the field of usability testing, will evaluate the e-learning environment (90). The expert-based cognitive walkthrough will be audio recorded to document the experts’ feedback, insights, and suggestions for improvement for further analysis.
In the third phase, a sample of eight cancer patients will participate in a patient-based cognitive walkthrough of the e-learning environment (90, 91). The goal is to gather feedback and understand the user experience by capturing their interaction with the environment, immediate reactions, difficulties, and decision-making processes (90–93). The participants will first complete a short questionnaire that includes HLS-GER 2 screening items (37), age, gender, and primary cancer diagnosis. Participants are instructed to complete the e-learning environment on their own and to think aloud. Thinking aloud will be practiced with participants prior to the start of the study. Patients’ screen activity will be videotaped, their thinking aloud will be audio recorded, and patients will then be interviewed. The audio and video data will be supplemented by handwritten observations by the research assistant. The video and audio recordings will be analyzed independently by two of the authors and coded into themes representing usability issues, content issues, and suggestions for improvement (90). These themes will be discussed by the group of authors and adjustments will be made if necessary.
2.6 Outcomes of the main study (RCT)
2.6.1 Primary outcome
The primary outcome is the patient-reported digital health literacy based on the German version of the Digital health literacy Instrument (DHLI) (Supplementary Data Sheet 1) (94, 95), resulting from the mean of the 21 items included. All 21 items are answered on a four-point scale (1 = “very difficult” to 4 = “very easy” and from 1 = “never” to 4 = “often”). A higher mean score indicates higher digital health literacy. The internal consistency of the instrument (Cronbach alpha = .87) was acceptable (35). The questionnaire will be administered at T0, T1 and T2.
2.6.2 Secondary outcomes
In addition to the 21 items of the DHLI, the digital health literacy of the participants will also be measured with six performance-based items of the DHLI (35) - one item each for the sub-skills: operational skills, navigation skills, evaluating reliability, information searching, determining relevance and protecting privacy (Supplementary Data Sheet 2). In the original Dutch version, the performance-based items proved difficult to validate as they did not form a single construct (Cronbach's alpha = .47) (35). No item was created for the subskill of “adding self-generated content” for two reasons. (1) In the original Dutch version, this skill was operationalized as a task in which participants were asked to write a short message to their doctor asking about vaccinations for a trip to Morocco. However, there were no clear standards to evaluate whether the written message was correct or incorrect, leading to potential subjectivity and inconsistency in assessment. (2) The content of our intervention does not directly focus on improving this specific subskill, and therefore, we do not expect meaningful changes in it from baseline to follow-up. Participants are asked to apply the six skills mentioned above (navigation skills, etc.) in a fictitious situation. For example, the performance-based item for measuring navigation skills shows a picture of a screen background (=print screen) with an open Internet browser. Participants are offered five options (multiple choice) to open a new tab in the internet browser. The answers are coded as correct (score = 1) or incorrect (score = 0) and then the scores of all six items are added together to form a total score. The items will differ from the original questionnaire, as these items in the original questionnaire are formulated in Dutch and the questions were designed for patients with rheumatoid arthritis. In addition, there will be different items for navigation and operating skills, depending on which digital device (PC/laptop vs. tablet/smartphone) and operating system (Android vs. iOS and windows vs. macOS) the participants prefer for Internet access. For the piloting of the performance-based items, we created a total of eight items. We created two items each for operational skills and information skills. After piloting, the number of items will be reduced to six. The questionnaire will be administered at T0, T1 and T2.
The German translation of the Reduced Instructional Materials Motivation (RIMMS) will be used to measure participants’ motivation to use the e-learning environment. The RIMMS is a 4-dimensional (attention, relevance, confidence, satisfaction), 12-item short version of the “Instructional Materials Motivation Survey”. The items are rated on a 5-point scale from 1 = “strongly disagree” to 5 = “strongly agree” (96). We will use the resulting mean value across all 12 items for the overall assessment of motivation. The questionnaire was translated from English into German according to a standardized procedure described by the European Organization for Research and Treatment of Cancer (EORTC) Quality of Life Group (97). The RIMMS will be piloted with the performance-based items of the DHLI and will be administered at T1.
E-learning usage data, such as time spent in the e-learning environment and which learning blocks were clicked on, is collected anonymously to measure learner engagement with the e-learning environment in cross-section.
There questionnaires will be used to measure acceptance [Acceptability of Intervention Measure (AIM) questionnaire], appropriateness [Intervention Appropriateness Measure (IAM) questionnaire], and feasibility [Feasibility of Intervention Measure (FIM) questionnaire (98)] of the e-learning environment from the participants’ perspective. The AIM, IAM and FIM questionnaires were translated from English into German according to the standardized procedures of the EORTC Quality of Life Group (97). All three questionnaires contain 4 items that are rated on a 5-point scale from 1 = “disagree” to 5 = “agree” (98). Acceptability, appropriateness and feasibility are presented as the mean value of the four items of each questionnaire, with a higher score indicating greater acceptance, appropriateness and feasibility. The three questionnaires AIM, FIM and IAM will be administered at T1.
Other outcomes used to determine the acceptability of the e-learning environment are the number of discontinuations of the e-learning environment or deviations from the treatment protocol. Participants in the intervention groups will be asked at T1 (1) if they completed all chapters of the e-learning, (2) which chapters they completed, if they did not complete all chapters, and (3) the reasons why they stopped the e-learning environment early (open-ended question).
2.6.3 Participants’ sociodemographic, recruitment and medical characteristics and participants’ internet use
To describe the sample, the following sociodemographic and medical data will be collected at T0: age, sex, marital status, cancer type, self-reported health status, time since cancer diagnosis, educational level, and recruitment route (CIS recruitment route vs. non-CIS recruitment route) into the study.
Participants’ Internet use will be measured at T0 using items on the following topics. These are the frequency of Internet use, preferred digital device used to access the Internet, the preferred mobile phone (IOS, Android or other operating system) or computer or laptop operating system (Windows, macOS or other operating system), and the use of various information services.
2.6.4 Safety endpoints
To rule out protocol deviations, participants in the intervention groups will be asked at T1 whether the content of the intervention was completed by themselves or by someone close to them (e.g., spouse or child). Participants in the control group will be asked at T2 whether they had access to the content of the intervention groups (e.g., through friends/acquaintances in a patient support group).
2.6.5 Piloting the questionnaires
The DHLI, the 7 performance-based items of the DHLI, and the RIMMS will be piloted with 8 patients before the start of the study by means of a three-step test interview (TSTI) (99, 100). It combines observation and questioning techniques to determine how items are interpreted and whether problems occur when completing the questionnaire. In the first phase of the TSTI, the participant will complete the questionnaire while thinking aloud while the interviewer will focus and on observing the participant and taking notes (99). In the second phase, respondents will be interviewed regarding their response behavior, augmenting the data gathered in the first step. In the third phase, patients can share their considerations and opinions about the questionnaire (99, 100). The test is audio-recorded and then transcribed to facilitate evaluation. Analysis will be performed by hand, on item level, except for comments made in step 3 which concern the questionnaire as a whole. Comments and problems will be labelled and subsequently grouped into categories. Coding will be performed independent by two of the authors. For the sub-skills of the performance-based DHLI for which more than one item was created, only one of the two items will be selected for the main study questionnaire. It will be evaluated which item is easier for participants to understand and which item seems to discriminate better between people with low and high digital health literacy.
2.7 Sample size calculation
The sample size calculation for the primary hypothesis was calculated using PASS 2008 for a one-way ANOVA. We set the two-sided significance level at 5% and the power at 80%. For the sample size calculation, we expected that at T2 there will be a greater improvement in digital health literacy in the e-learning groups (IG1.1–IG1.3) and IG2 (non-interactive PDF group) compared to the control group. The study by Mitsuhashi (32) found an effect size of Cohen's d = 0.25 at 2 weeks. We assume that we will achieve a slightly higher effect size: First, because we involve patients in the design process of the digital learning offerings, and second, because the content of our e-learning environment is presented in a more interactive, persuasive and engaging manner. As mentioned in the introduction, research suggests that online courses with high levels of interactivity lead to higher levels of students’ motivation, improved learning outcomes, and satisfaction than less interactive learning environments (48–51). Furthermore, a systematic review identified a positive correlation between the number of persuasive elements and the effectiveness of web-based interventions (61). In the case number design, we assume an effect size between the e-learning environment (IG1.1, IG1.2 and IG1.3) and the control group and between IG2 and the control group of Cohen's d = 0.35. These assumptions correspond to an effect size of f = 0.165. The sample size calculation results in 297 participants for the three e-learning environment groups (IG1.1, IG1.2, and IG1.3) and 99 each for IG2 and the control group, for a total of 495 participants (Figure 2).
Figure 2
Figure 2. Expected flow chart for participation in the study.
In the above-mentioned study by Mitsuhashi (47), 6% of participants dropped out in the 2 weeks between baseline and follow-up. In other RCTs involving the promotion of digital health literacy in older adults, dropout rates ranged from 13% to 26% in the period between 2 weeks and 9 months (101–104). With a period of 8 weeks between T0 and T2, we assume a dropout rate of 25%. To compensate for this dropout rate, a total of 660 subjects must be enrolled. 396 people will be equally assigned to the groups IG1.1, IG1.2 and IG1.3 and 132 each for IG2 and the control group.
2.8 Allocation and blinding
After completing the online baseline questionnaire, the participants are assigned to one of the following groups according to the randomization sequence stored in REDCap. Participants in each treatment groups will receive an email containing a link tailored to their assigned group: a link to the appropriate e-learning environment (IG1.1–IG1.3), a PDF file (IG2), or a link to the CIS brochure “Your Journey through Cancer” (control group). The randomization sequence was created using the R package “blockrand” (version 1.5) (105) and follows a 1:1:1:1:1 ratio across five groups: IG1.1 (e-learning environment), IG1.2 (e-learning environment without “tunneling” elements), IG1.3 (e-learning environment without “rehearsal” elements), IG2 (non-interactive PDF file), and the control group. Allocation will be conducted using a stratified randomization with variable block lengths. The strata consider both the recruitment route (CIS vs. non-CIS) and the cancer type (breast cancer vs. other cancers). Given that a disproportionate number of inquiries to the CIS come from women (64%) and from patients with breast cancer (39%) (106), the stratification ensures a proportionally representative allocation. To achieve this, randomization is conducted separately for each stratum, ensuring that the distribution of cancer types within the groups reflects the cancer incidence in 2020 (107) in the German population. In 2020, 14.5% of all new cancer diagnoses were breast cancer diagnoses. Two thirds of the planned 660 cancer patients will be recruited through recruitment routes associated with the CIS and one third through recruitment routes such as the outpatient clinics of the UCCH and affiliated patient support groups. The randomization sequence is securely stored and managed within REDCap's automated allocation module, ensuring that neither participants nor recruiting researchers have access to it, and assignments are made only after enrollment to maintain allocation concealment. Blinding will be maintained throughout the duration of the study to increase the objectivity of the results and minimize the influence of participants’ expectations on the study results. Participants in the control group will not receive any intervention and will be referred to the CIS brochure “Your journey through cancer”, which will not include any information on how to search for cancer-related information online. In the data set and in REDCap, the 5 randomization groups are referred to as groups A, B, C, D and E, so that it is not clear to the research staff evaluating the data which intervention the patients received (blinding).
2.9 Data collection and management
All data will be collected using the electronic data collection software REDCap (68). The research staff who will create the questionnaires in REDCap and who will export the participants’ data to the statistics software R (version 4.2.1) have been trained in the use of REDCap and will be supported by the REDCap support team of the UKE in case of problems. Data entry in REDCap will be simulated for 10 subjects prior to the start of the study to identify possible problems. All REDCap servers are located in Germany. The data in REDCap is pseudonymized. The list linking the pseudonym to the real person is stored separately on UKE servers and is subject to technical and organizational measures to ensure that the personal data cannot be assigned to an identified or identifiable natural person. The REDCap data are downloaded every 2 weeks to create backup data sets and are also stored on the servers of the UKE. The data will be available only to authors affiliated with the UKE. Due to the characteristics of the study, no data monitoring committee will be included.
Participants will receive an invitation to the 2 and 8 week follow-up via email from the project email address. If they do not complete the questionnaire after the first invitation, they will receive a reminder after 48 h and a second reminder 48 h after the first reminder.
2.10 Data analysis
We will use the statistics software R for the statistical analyses. To describe participants’ sociodemographic and medical characteristics, participants’ Internet use, and primary and secondary outcomes at T0, T1, and T2, descriptive statistics such as mean, standard deviation, median, and interquartile range will be calculated for continuous variables. Absolute and relative frequencies will be reported for categorical variables. Descriptive statistics will be reported for the total sample as well as separated by randomization groups.
The evaluation of the primary hypothesis is based on the closed testing principle (108): In the first step, a global test is carried out to determine whether the change in the digital health literacy value from T0 to T2 differs significantly between at least two of the three randomization groups (digital e-learning (IG1.1–IG1.3), non-interactive PDF group, control group). The two-sided significance level is set to 5%. A linear mixed model is calculated to analyze this primary hypothesis. The R package “nlme” will be used for this purpose (109). The dependent variable is the difference between the DHLI value after intervention at T2 and T1, respectively, and the baseline measurement at T0. The randomization group, the follow-up time points after intervention at T1 and T2, the recruitment route (CIS vs. Non-CIS), the cancer type (breast cancer vs. other cancer types) and the interaction between the follow-up time points and the randomization group will be included as fixed factors. To control for differences in baseline DHLI scores, regression to the mean, and to reduce the error variance in the dependent variable (110, 111), and the baseline DHLI value will be included as a covariate. In addition, a random intercept will be estimated for the patients. The global hypothesis shows a significant result if the p-value of the global comparison between the three randomization groups at time T2 is smaller than the two-sided significance level.
Only if the result is significant, the pairwise comparisons between the three randomization groups at time T2 will be calculated in the second step using a priori contrasts (112) using the R package “emmeans” (113). A pairwise comparison is significant, if the associated p-value is smaller than the two-sided significance level. In this way, we keep the overall two-sided type I error at a maximum of 5%. The primary null hypothesis is rejected if either the difference between IG1.1–IG1.3 and the control group and/or between IG2 and the control group is significant. To determine which of the persuasive elements (tunneling, rehearsal) of the e-learning environment influence effectiveness (hypothesis 2), we conduct further pairwise comparisons, again using the R package “emmeans”.
The evaluation of the primary and secondary hypotheses is based on the full analysis set. It is as complete as possible and as similar as possible to the intention-to-treat population. The intention-to-treat population includes all randomized patients belonging to the group to which they were originally randomized, regardless of whether protocol violations are known. As sensitivity analyses, the primary evaluation is repeated in the per-protocol (PP) population. The PP population includes patients without serious protocol violations. We define a serious protocol violation as: (1) Less than 50% of the content of the digital learning offers is processed by participants (is checked with the item on the number of chapters processed at T1); (2) The content is not processed by the participants of IG1.1, IG1.2, IG1.3 and IG2 themselves, but by a close person (e.g., spouse or child) (checked by an item at T1); (3) Participants in the control group are given access to the learning opportunities of the intervention groups, e.g., via patient friends in self-help groups (checked by item at T2); (4) Participants who report that they do not have cancer. In addition, protocol deviations reported by the participants in the REDCap data set are listed and forwarded to the UKE research assistant, who blinds them (he knows neither the group affiliation nor the outcome values of the participants) and classifies them into severe and less severe protocol deviations.
In the full analysis set, we will not replace missing data. However, as a further sensitivity analysis, the primary analysis is performed with a multiply imported data set. For the imputation, we will use the “Amelia II” package (version 1.8.1 in R (114)) which allows us to generate 100 imputations of missing values using a bootstrapping-based algorithm. We will use the Amelia package to impute data for missing values of the DHLI at T2; there should be no missing values at T0 in a FAS population. The DHLI values at T2 will be defined as continuous variables and we will define logical boundaries (min = 1, max = 4). We will pool the results of the linear mixed model for the 100 imputed datasets based on Rubin's rules (115).
The third research question is evaluated analogously to the primary hypothesis using linear mixed models. The dependent variable is changed from the DHLI value to the total score of the performance based items of the DHLI.
A mediation model is used to analyze whether motivation mediates the association between randomization groups and the change in digital health literacy from T0 to T2 (research question 5). The mediation model is calculated with patient motivation (mean RIMMS score) as the mediator; the intervention group as the independent variable. The dependent variable is the difference in digital health literacy from T0 to T2. The mediation analysis is performed based on the model of Hayes (116).
The results of the RIMMS, AIM, IAM and FIM will be presented and interpreted descriptively to investigate user-friendliness, appropriateness and feasibility from the patient perspective (research question 6). To evaluate whether the e-learning environment is perceived as more user-friendly, appropriate and feasible that the non-interactive PDF file (research question 7), we will use Analysis of Variance (ANOVA) tests.
A statistical analysis plan for all intended analyses will be finalized before unblinding.
3 Discussion
The primary objective of this study is to investigate whether cancer patients can improve their digital health literacy over an 8-week period using either an interactive e-learning environment or a non-interactive PDF file compared to a control group. This study is motivated by the recognition of the increasing importance of digital health literacy in empowering patients to navigate complex healthcare systems and make informed decisions about their health. In particular, cancer patients with less common cancers, such as small bowel or anal cancers, can benefit significantly from improved digital health literacy due to the lack of online resources tailored to their specific conditions. Digital health literacy is part of health literacy, which has been found to be insufficient in the populations of many countries (37, 117–119). Poor digital health literacy can negatively influence health by limiting individuals’ ability to access, understand, and use online health information effectively. This can lead to misinformed health decisions, non-adherence to medical instructions, and delayed or inappropriate treatment-seeking behaviors, ultimately resulting in poorer health outcomes (120–122). Promotion of health literacy has become an integral part of European health strategies, of global activities of the United Nations, as well of political strategies or national plans of many countries (123). In contrast to the long-term goals of these national plans, this intervention study aims to improve the digital health literacy of cancer patients, who have a high unmet need for information (1), in a timely manner.
If the study results show that the e-learning environment has a positive impact on cancer patients’ digital health literacy compared to a control group, it would be made freely available to all cancer patients through the CIS website. Caregivers, who play a crucial role in supporting cancer patients, would also have access to the e-learning environment, allowing them to enhance their own understanding and better assist the patients in their care. The wide reach of the CIS website, with approximately 7.7 m visitors in 2023, means that many cancer patients can be reached. The e-learning environment could empower patients to be more involved in their medical decision-making (10, 11), be better informed about their disease (5), and may even positively influence their health-related QoL (124). If adapted accordingly, patients with other chronic diseases or healthy people could also benefit from the e-learning environment. Since cancer patients, patients with other chronic diseases such as rheumatoid arthritis, and healthy people seem to have the same problems when searching for health information on the Internet (34, 36, 125), the latter two groups could also potentially improve their digital health literacy by using the e-learning environment.
There are two main concerns regarding the generalizability of the study results. Two-thirds of the participants are recruited through CIS-related recruitment routes, and a disproportionate number of inquiries to the CIS come from women (64%), and from patients with breast cancer (39%) (106). Although there is no evidence that cancer type has an impact on the primary outcome of digital health literacy (30), we addressed this potential bias by implementing a proportionally stratified randomization strategy in relation to the cancer incidence in 2020 (107) in the German population. Another concern relates to the level of digital health literacy of the participants in the study, especially those who will be recruited through the CIS recruitment route. Given that patients using the CIS platform have a higher level of education compared to the average German cancer patient population, there may be a risk of sampling bias towards individuals with higher digital health literacy. To minimize this bias, a complementary recruitment strategy will be employed in which one-third of participants will be recruited offline from outpatient clinics. By diversifying recruitment routes, we aim to obtain a more representative sample of cancer patients with varying levels of digital health literacy.
The DHLI - the primary outcome - has been validated in different languages and populations (126–132) within the last 5 years and has been translated and used in German-speaking populations (71, 94, 95), but it has not been validated in German, which is a limitation of our study. Nevertheless, we prefer the DHLI to the more commonly used eHealth literacy scale (eHEALS) (33), because we assume that searching for cancer-related information requires different skills that are better measured by the multidimensional DHLI than by the one-dimensional eHEALS. In addition, study results also show that the DHLI has stronger positive correlations with certain factors, such as Internet use for health information and higher educational attainment, than the eHEALS (133). Furthermore, we will pilot test the DHLI with the TSTI using 8 cancer patients to determine how the items are interpreted and if there are any problems in completing the questionnaire. In addition, a validation of the questionnaire using the data from the RCT is planned.
The second limitation of this study is the exclusive use of the German language. This limitation is particularly significant in the context of the diverse population in Germany, where approximately 15% of households primarily speak a language other than German, with Turkish being the most commonly spoken language among this group (134). This language barrier may limit the accessibility and applicability of the study to non-German speaking patients, potentially excluding a portion of the population that could benefit from the intervention. Nevertheless, we should not overlook the fact that the use of multiple languages in the initial study phase could introduce confounding factors related to translation differences, which may complicate the interpretation of results. By focusing on a single language, we aim to maintain the internal validity of the study, which is critical for drawing accurate and reliable conclusions.
The third limitation is the absence of an explicit behavioral theory as a foundation for the intervention. Although the intervention incorporates persuasive design element of the PSD model, it does not explicitly incorporate additional behavioral theories during its development. Behavioral theories could have provided a stronger theoretical foundation to guide the intervention's design and implementation. This represents a limitation in the conceptual framework of the study. Related studies should consider explicitly integrating behavioral theories to enhance the rationale and effectiveness of the approach.
Data availability statement
The original contributions presented in the study are included in the article/Supplementary Material. Further inquiries can be directed to the corresponding author. The study data will be made available to other scientists via the online open access repository Figshare.
Ethics statement
The studies involving humans were approved by Ethics Committee of the Medical Faculty Heidelberg (No. S-158/2024). The research will be carried out in accordance with the current version of the Declaration of Helsinki. The study will be registered on clinicaltrials.gov. Any amendments to the protocol will be communicated and re-approved by the ethics committee.
Author contributions
LL-D: Conceptualization, Funding acquisition, Methodology, Project administration, Writing – original draft, Writing – review & editing, Investigation, Visualization. HW: Conceptualization, Writing – original draft, Writing – review & editing, Investigation. MB: Conceptualization, Data curation, Software, Writing – original draft, Writing – review & editing, Visualization, Investigation. AE: Conceptualization, Supervision, Writing – original draft, Writing – review & editing. AD: Conceptualization, Methodology, Supervision, Writing – original draft, Writing – review & editing, Funding acquisition. AH: Conceptualization, Methodology, Supervision, Writing – original draft, Writing – review & editing. CB: Conceptualization, Funding acquisition, Supervision, Writing – original draft, Writing – review & editing. SW-R: Conceptualization, Funding acquisition, Supervision, Writing – original draft, Writing – review & editing. HS: Conceptualization, Funding acquisition, Supervision, Writing – original draft, Writing – review & editing.
Funding
The author(s) declare financial support was received for the research, authorship, and/or publication of this article. The study was funded by the German Research Foundation (Project number: 525146391).
Acknowledgments
We would like to thank the reviewers who took the time to review our manuscript.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher's note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Supplementary material
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fdgth.2025.1455143/full#supplementary-material
References
1. Harrison JD, Young JM, Price MA, Butow PN, Solomon MJ. What are the unmet supportive care needs of people with cancer? A systematic review. Support Care Cancer. (2009) 17(8):1117–28. doi: 10.1007/s00520-009-0615-5
2. Davies NJ, Kinman G, Thomas RJ, Bailey T. Information satisfaction in breast and prostate cancer patients: implications for quality of life. Psychooncology. (2008) 17(10):1048–52. doi: 10.1002/pon.1305
3. Jenkins V, Fallowfield L, Saul J. Information needs of patients with cancer: results from a large study in UK cancer centres. Br J Cancer. (2001) 84(1):48–51. doi: 10.1054/bjoc.2000.1573
4. Castleton K, Fong T, Wang-Gillam A, Waqar MA, Jeffe DB, Kehlenbrink L, et al. A survey of internet utilization among patients with cancer. Support Care Cancer. (2011) 19(8):1183–90. doi: 10.1007/s00520-010-0935-5
5. van de Poll-Franse LV, van Eenbergen MC. Internet use by cancer survivors: current use and future wishes. Support Care Cancer. (2008) 16(10):1189–95. doi: 10.1007/s00520-008-0419-z
6. Mattsson S, Olsson EMG, Johansson B, Carlsson M. Health-related internet use in people with cancer: results from a cross-sectional study in two outpatient clinics in Sweden. J Med Internet Res. (2017) 19(5):e163. doi: 10.2196/jmir.6830
7. Poushter J, Bishop C, Chwe H. Social media use continues to rise in developing countries but plateaus across developed ones: Pew Research Center (2018). Available online at: https://assets.pewresearch.org/wp-content/uploads/sites/2/2018/06/15135408/Pew-Research-Center_Global-Tech-Social-Media-Use_2018.06.19.pdf (cited 2020- 04-14).
8. Statistisches Bundesamt. Private Haushalte in Der Informationsgesellschaft –Nutzung Von Informations- Und Kommunikations-Technologien (2020). Available online at: https://www.destatis.de/DE/Themen/Gesellschaft-Umwelt/Einkommen-Konsum-Lebensbedingungen/IT-Nutzung/Publikationen/Downloads-IT-Nutzung/private-haushalte-ikt-2150400197004.pdf?blob=publicationFile (cited 2020 -09-10).
9. Atarere JO, Onyeaka HK, Chido-Amajuoyi OG, Adewunmi C, Nwaneki C, Dosunmu GT, et al. Social media use and health promotion among cancer survivors. Psychooncology. (2024) 33(1):e6299. doi: 10.1002/pon.6299
10. Lee CJ, Gray SW, Lewis N. Internet use leads cancer patients to be active health care consumers. Patient Educ Couns. (2010) 81(Suppl):S63–9. doi: 10.1016/j.pec.2010.09.004
11. Heiman H, Keinki C, Huebner J. Ehealth literacy in patients with cancer and their usage of web-based information. J Cancer Res Clin Oncol. (2018) 144(9):1843–50. doi: 10.1007/s00432-018-2703-8
12. Chou WY, Liu B, Post S, Hesse B. Health-related internet use among cancer survivors: data from the health information national trends survey, 2003-2008. J Cancer Surviv. (2011) 5(3):263–70. doi: 10.1007/s11764-011-0179-5
13. Shahrokni A, Mahmoudzadeh S, Lu BT. In whom do cancer survivors trust online and offline? Asian Pac J Cancer Prev. (2014) 15(15):6171–6. doi: 10.7314/APJCP.2014.15.15.6171
14. Tan SS, Goonawardene N. Internet health information seeking and the patient-physician relationship: a systematic review. J Med Internet Res. (2017) 19(1):e9. doi: 10.2196/jmir.5729
15. Borgmann H, Mager R, Salem J, Brundl J, Kunath F, Thomas C, et al. Robotic prostatectomy on the web: a cross-sectional qualitative assessment. Clin Genitourin Cancer. (2016) 14(4):E355. doi: 10.1016/j.clgc.2015.12.020
16. Hargrave DR, Hargrave UA, Bouffet E. Quality of health information on the internet in pediatric neuro-oncology. Neuro Oncol. (2006) 8(2):175–82. doi: 10.1215/15228517-2005-008
17. Liebl P, Seilacher E, Koester MJ, Stellamanns J, Zell J, Hubner J. What cancer patients find in the internet: the visibility of evidence-based patient information - analysis of information on German websites. Oncol Res Treat. (2015) 38(5):212–8. doi: 10.1159/000381739
18. Wasserman M, Baxter NN, Rosen B, Burnstein M, Halverson AL. Systematic review of internet patient information on colorectal cancer surgery. Dis Colon Rectum. (2014) 57(1):64–9. doi: 10.1097/DCR.0000000000000011
19. Xu AJ, Myrie A, Taylor JI, Matulewicz R, Gao T, Perez-Rosas V, et al. Instagram and prostate cancer: using validated instruments to assess the quality of information on social media. Prostate Cancer Prostatic Dis. (2022) 25(4):791–3. doi: 10.1038/s41391-021-00473-7
20. García-Cano-Fernández AM, Szczesniewski-Dudzik JJ, García-Tello A, Diego-García V, Boronat-Catalá J, Llanes-González L. Quality of bladder cancer information on YouTube. Cent European J Urol. (2022) 75(3):248–51. doi: 10.5173/ceju.2022.151
21. Attai DJ, Cowher MS, Al-Hamadani M, Schoger JM, Staley AC, Landercasper J. Twitter social media is an effective tool for breast cancer patient education and support: patient-reported outcomes by survey. J Med Internet Res. (2015) 17(7):e188. doi: 10.2196/jmir.4721
22. Park S, Oh HK, Park G, Suh B, Bae WK, Kim JW, et al. The source and credibility of colorectal cancer information on twitter. Medicine. (2016) 95(7):e2775. doi: 10.1097/MD.0000000000002775
23. Pan A, Musheyev D, Bockelman D, Loeb S, Kabarriti AE. Assessment of artificial intelligence chatbot responses to top searched queries about cancer. JAMA Oncol. (2023) 9(10):1437–40. doi: 10.1001/jamaoncol.2023.2947
24. Musheyev D, Pan A, Loeb S, Kabarriti AE. How well do artificial intelligence chatbots respond to the top search queries about urological malignancies? Eur Urol. (2024) 85(1):13–6. doi: 10.1016/j.eururo.2023.07.004
25. National Comprehensive Cancer Network. NCCN guidelines. Available online at: https://www.nccn.org/guidelines/category_1 (cited 2024 11-06).
26. Chen S, Kann BH, Foote MB, Aerts H, Savova GK, Mak RH, et al. Use of artificial intelligence chatbots for cancer treatment information. JAMA Oncol. (2023) 9(10):1459–62. doi: 10.1001/jamaoncol.2023.2954
27. Duenas-Garcia OF, Kandadai P, Flynn MK, Patterson D, Saini J, O'Dell K. Patient-focused websites related to stress urinary incontinence and pelvic organ prolapse: a discern quality analysis. Int Urogynecol J. (2015) 26(6):875–80. doi: 10.1007/s00192-014-2615-3
28. Beaunoyer E, Arsenault M, Lomanowska AM, Guitton MJ. Understanding online health information: evaluation, tools, and strategies. Patient Educ Couns. (2017) 100(2):183–9. doi: 10.1016/j.pec.2016.08.028
29. Xie L, Zhang S, Xin M, Zhu M, Lu W, Mo PK. Electronic health literacy and health-related outcomes among older adults: a systematic review. Prev Med. (2022) 157:106997. doi: 10.1016/j.ypmed.2022.106997
30. Zhang Y, Xu P, Sun Q, Baral S, Xi L, Wang D. Factors influencing the E-health literacy in cancer patients: a systematic review. J Cancer Surviv. (2023) 17(2):425–40. doi: 10.1007/s11764-022-01260-6
31. Norman CD, Skinner HA. eHealth literacy: essential skills for consumer health in a networked world. J Med Internet Res. (2006) 8(2):e9. doi: 10.2196/jmir.8.2.e9
32. Norman CD, Skinner HA. eHEALS: the eHealth literacy scale. J Med Internet Res. (2006) 8(4):e27. doi: 10.2196/jmir.8.4.e27
33. Lee J, Lee EH, Chae D. eHealth literacy instruments: systematic review of measurement properties. J Med Internet Res. (2021) 23(11):e30644. doi: 10.2196/30644
34. Lange-Drenth L, Schulz H, Endsin G, Bleich C. Patients with cancer searching for cancer- or health-specific web-based information: performance test analysis. J Med Internet Res. (2021) 23(8):e23367. doi: 10.2196/23367
35. van der Vaart R, Drossaert C. Development of the digital health literacy instrument: measuring a broad spectrum of health 1.0 and health 2.0 skills. J Med Internet Res. (2017) 19(1):e27. doi: 10.2196/jmir.6709
36. van der Vaart R, Drossaert CH, de Heus M, Taal E, van de Laar MA. Measuring actual eHealth literacy among patients with rheumatic diseases: a qualitative analysis of problems encountered using health 1.0 and health 2.0 applications. J Med Internet Res. (2013) 15(2):e27. doi: 10.2196/jmir.2428
37. Schaeffer D, Berens E, Gille S, Griese L, Klinger J, de Sombre S, et al. Gesundheitskompetenz Der Bevölkerung in Deutschland Vor Und Während Der Corona Pandemie: Ergebnisse Des Hls-Ger 2. Bielefeld: Universität Bielefeld, Interdisziplinäres Zentrum für Gesundheitskompetenzforschung (2021).
38. van Deursen AJ, van Dijk J, Peters O. Rethinking internet skills: the contribution of gender, age, education, internet experience, and hours online to medium- and content-related internet skills. Poetics. (2011) 39(2):125–44. doi: 10.1016/j.poetic.2011.02.001
39. van der Vaart R, van Deursen AJ, Drossaert CH, Taal E, van Dijk JA, van de Laar MA. Does the eHealth literacy scale (eHEALS) measure what it intends to measure? Validation of a Dutch version of the eHEALS in two adult populations. J Med Internet Res. (2011) 13(4):e86. doi: 10.2196/jmir.1840
40. Merritt K, Smith D, Di Renzo J. An investigation of self-reported computer literacy: is it reliable? Issues Inf Syst. (2005) 6:289–95. doi: 10.48009/1_iis_2005_289-295
41. Palczyńska M, Rynko M. Ict skills measurement in social surveys: can wetrustself-reports? Qual Quant. (2020) 55(3):917–43. doi: 10.1007/s11135-020-01031-4
42. Mahmood K. Do people overestimate their information literacy skills? A systematic review of empirical evidence on the Dunning-Kruger effect. Comminfolit. (2016) 10(2):199–213. doi: 10.15760/comminfolit.2016.10.2.24
43. Clarke R, Mayer R. Introduction: getting the most from this resource. In: Clark RC, Mayer RE, editors. E-Learning and the Science of Instruction. Hoboken, NJ: John Wiley & Sons, Inc. (2016). p. 1–6.
44. Khan RMI, Ali A, Alourani A. Investigating learners’ experience of autonomous learning in E-learning context. Int J Emerg Technol Learn. (2022) 17(08):4–17. doi: 10.3991/ijet.v17i08.29885
45. Akhter S, Javed MK, Qasim S, Javaid A. Highlighting the advantages and disadvantages of E-learning. (2021).
46. Liu C, Scott KM, Lim RL, Taylor S, Calvo RA. Eqclinic: a platform for learning communication skills in clinical consultations. Med Educ Online. (2016) 21:31801. doi: 10.3402/meo.v21.31801
47. Mitsuhashi T. Effects of two-week E-learning on eHealth literacy: a randomized controlled trial of Japanese internet users. PeerJ. (2018) 6:e5251. doi: 10.7717/peerj.5251
48. Espasa A, Meneses J. Analysing feedback processes in an online teaching and learning environment: an exploratory study. High Educ. (2009) 59(3):277–92. doi: 10.1007/s10734-009-9247-4
49. Leem B-H. Impact of interactivity on learning outcome in online learning settings: ordinal logit model. Int J Eng Bus Manag. (2023) 15:1–10. doi: 10.1177/18479790231203107
50. Schwan S, Riempp R. The cognitive benefits of interactive videos: learning to tie nautical knots. Learn Instr. (2004) 14(3):293–305. doi: 10.1016/j.learninstruc.2004.06.005
51. Tung F-W, Deng Y-S. Designing social presence in E-learning environments: testing the effect of interactivity on children. Interact Learn Environ. (2006) 14(3):251–64. doi: 10.1080/10494820600924750
52. Chang L, Wu T-T, Su C-Y. The influence of interactive and non-interactive E-book on the learning effectiveness of high and low achievement nursing students. Innovative Technologies and Learning: Second International Conference, ICITL 2019, December 2–5, 2019, Tromsø, Norway: Springer International Publishing (2019). p. 14–23.
53. Yahiaoui F, Aichouche R, Chergui K, Brika SKM, Almezher M, Musa AA, et al. The impact of E-learning systems on motivating students and enhancing their outcomes during COVID-19: a mixed-method approach. Front Psychol. (2022) 13:874181. doi: 10.3389/fpsyg.2022.874181
54. Priego RG, Peralta AG. Engagement factors and motivation in E-learning and blended-learning projects. Proceedings of the First International Conference on Technological Ecosystem for Enhancing Multiculturality - TEEM '13 (2013). p. 453–60.
55. Gan DZQ, McGillivray L, Han J, Christensen H, Torok M. Effect of engagement with digital interventions on mental health outcomes: a systematic review and meta-analysis. Front Digit Health. (2021) 3:1–14. doi: 10.3389/fdgth.2021.764079
56. Kelders SM, van Zyl LE, Ludden GDS. The concept and components of engagement in different domains applied to ehealth: a systematic scoping review. Front Psychol. (2020) 11:926. doi: 10.3389/fpsyg.2020.00926
57. Sittig S, McGowan A, Iyengar S. Extensive review of persuasive system design categories and principles: behavioral obesity interventions. J Med Syst. (2020) 44(7):128. doi: 10.1007/s10916-020-01591-w
58. Cheong C, Filippou J, Cheong F, Pirker J, Guetl C. Using persuasive system design principles to evaluate two next generation digital learning environments. Interactive Collaborative Learning: Proceedings of the 19th ICL Conference-Volume 2. Springer International Publishing (2017).p. 255–68.
59. Oinas-Kukkonen H, Harjumaa M. Persuasive systems design: key issues, process model, and system features. Commun Assoc Inf Syst. (2009) 24:485–500. doi: 10.17705/1cais.02428
60. Kim Y. Trust in health information websites: a systematic literature review on the antecedents of trust. Health Informatics J. (2016) 22(2):355–69. doi: 10.1177/1460458214559432
61. Wildeboer G, Kelders SM, van Gemert-Pijnen JE. The relationship between persuasive technology principles, adherence and effect of web-based interventions for mental health: a meta-analysis. Int J Med Inform. (2016) 96:71–85. doi: 10.1016/j.ijmedinf.2016.04.005
63. Whitehead L, Seaton P. The effectiveness of self-management mobile phone and tablet apps in long-term condition management: a systematic review. J Med Internet Res. (2016) 18(5):e97. doi: 10.2196/jmir.4883
64. Kruse CS, Beane A. Health information technology continues to show positive effect on medical outcomes: systematic review. J Med Internet Res. (2018) 20(2):e41. doi: 10.2196/jmir.8793
65. Buntin MB, Burke MF, Hoaglin MC, Blumenthal D. The benefits of health information technology: a review of the recent literature shows predominantly positive results. Health Aff. (2011) 30(3):464–71. doi: 10.1377/hlthaff.2011.0178
66. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. (2011) 38(2):65–76. doi: 10.1007/s10488-010-0319-7
67. Chan A-W, Tetzlaff JM, Gøtzsche PC, Altman DG, Mann H, Berlin JA, et al. Spirit 2013 explanation and elaboration: guidance for protocols of clinical trials. BMJ Br Med J. (2013) 346:e7586. doi: 10.1136/bmj.e7586
68. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (redcap)–a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. (2009) 42(2):377–81. doi: 10.1016/j.jbi.2008.08.010
69. Aktionsforum Gesundheitsinformationssystem. Afgis-Transparenzkriterien (2021). Available online at: https://www.afgis.de/zertifizierung/transparenzkriterien/ (cited 2024 12/04).
70. Ärztliches Zentrum für Qualität in der Medizin (ÄZQ). Gesundheits-Apps: Worauf Sollte Ich Achten? (2020). Available online at: https://www.aezq.de/medien/pdf/themen/gesundheitsapps/gesundheitsapps-patienteninformation-1aufl-vers1.pdf (cited 2024 12-04).
71. Kolpatzik K, Mohrmann M, Zeeb H. Digitale Gesundheitskompetenz in Deutschland. Berlin: KomPart Verlagsgesellschaft mbH & Co. KG (2020).
72. Allgemeine Ortskrankenkasse. Suchen Und Finden: Gesundheitsinformationen Im Internet (2024). Available online at: https://www.aok.de/pk/gesundheitswissen/gesundheitsinformationen-suchen-und-finden/ (cited 2024 12-04).
73. Bundesministerium für Gesundheit. Gesundheitskompetenz Stärken (2022). Available online at: https://gesund.bund.de/gesundheitskompetenz-digital-staerken (cited 2024 04-12).
74. Deutsches Netzwerk Evidenzbasierte Medizin. Gute Praxis Gesundheitsinformation. (2015). Available online at: https://www.ebm-netzwerk.de/de/medien/pdf/gpgi_2_20160721.pdf (cited 2024 12-04).
75. Lühnen J, Albrecht M, Mühlhauser I, Steckelberg A. Leitlinienreport Zur „Leitlinie Evidenzbasierte Gesundheitsinformation. (2017). Available online at: http://www.leitlinie-gesundheitsinformation.de (cited 2024 12-04).
76. Schaeffer D, Hurrelmann K, Bauer U, Kolpatzik K. Nationaler Aktionsplangesundheitskompetenz. Die Gesundheitskompetenz in Deutschland Stärken. Berlin: KomPart (2018).
77. Schmidt-Kaehler S, Vogt D, Berens E, Horn A, Schaeffer D. Gesundheitskompetenz: Verständlich Informieren Und Beraten. Material- Und Methodensammlung Zur Verbraucher- Und Patientenberatung Für Zielgruppen Mit Geringer Gesundheitskompetenz. Bielefeld: University Bielefeld (2017).
78. Arbeitsgruppe Gpgi. Gute praxis gesundheitsinformation. Z Evid Fortbild Qual Gesundhwes. (2016) 110:85–92. doi: 10.1016/j.zefq.2015.11.005
79. National Institutes of Health. How to evaluate health information on the internet: questions and answers: national institutes of health (2022). Available online at: https://ods.od.nih.gov/HealthInformation/How_To_Evaluate_Health_Information_on_the_Internet_Questions_and_Answers.aspx (cited 2022 -10-10).
80. National Library of Medicine. Evaluating health information (2022). Available online at: https://medlineplus.gov/evaluatinghealthinformation.html?utm_source=newsletter&utm_id=may4 (cited 2022 -10-10).
81. John Hopkins Seridan Librarys. Evaluating solcial medi: John Hopkins Seridan Librarys (2021). Available online at: https://guides.library.jhu.edu/evaluate/social-media (cited 2022 -10-05).
82. Cheong C, Filippou J, Cheong F, Pirker J, Guetl C. Using persuasive system design principles to evaluate two next generation digital learning environments. Interactive Collaborative Learning: Proceedings of the 19th ICL Conference-Volume 2. Springer International Publishing (2016).
83. Beerlage-de Jong N, Eikelenboom-Boskamp A, Voss A, Sanderman R, Gemert-Pijnen J. Combining user-centered design with the persuasive systems design model: the development process of a web-based registration and monitoring system for healthcare-associated infections in nursing homes. Int J Adv Life Sci. (2014) 6:262–71.
84. Hiller B. Ihr Weg Durch Die Krebserkrankung: Krebsinformationsdienst - Deutsches Krebsforschungszentrum (DKFZ) (2022). Available online at: https://www.krebsinformationsdienst.de/fileadmin/pdf-dateien/broschueren/broschuere-krebs-ihr-weg.pdf (cited 2024 12-02).
85. van Gemert-Pijnen JE, Nijland N, van Limburg M, Ossebaard HC, Kelders SM, Eysenbach G, et al. A holistic framework to improve the uptake and impact of ehealth technologies. J Med Internet Res. (2011) 13(4):e111. doi: 10.2196/jmir.1672
86. Burns C. Human-centred design. In: van Gemert-Pijnen L, Kelders SM, Kip H, Sanderman R, editors. Ehealth Research, Theory and Development. New York: Routledge (2018). p. 207–27.
87. Asbjornsen RA, Wentzel J, Smedsrod ML, Hjelmesaeth J, Clark MM, Solberg Nes L, et al. Identifying persuasive design principles and behavior change techniques supporting End user values and needs in eHealth interventions for long-term weight loss maintenance: qualitative study. J Med Internet Res. (2020) 22(11):e22598. doi: 10.2196/22598
88. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. (2006) 3(2):77–101. doi: 10.1191/1478088706qp063oa
89. Wharton C, Rieman J, Lewis C, Polson P. The cognitive walkthrough method: a practitioner’s guide. In: Nielsen J, Mack RL, editors. Usability Inspection Methods. New York, NY: John Wiley & Sons, Inc (1994). p. 105–40.
90. Jaspers MW. A comparison of usability methods for testing interactive health technologies: methodological aspects and empirical evidence. Int J Med Inform. (2009) 78(5):340–53. doi: 10.1016/j.ijmedinf.2008.10.002
91. Cotton D, Gresty K. Reflecting on the think-aloud method for evaluating E-learning. Br J Educ Technol. (2006) 37(1):45–54. doi: 10.1111/j.1467-8535.2005.00521.x
92. Jaspers MW, Steen T, van den Bos C, Geenen M. The think aloud method: a guide to user interface design. Int J Med Inform. (2004) 73(11-12):781–95. doi: 10.1016/j.ijmedinf.2004.08.003
93. Boren T, Ramey J. Thinking aloud: reconciling theory and practice. IEEE Trans Prof Commun. (2000) 43(3):261–78. doi: 10.1109/47.867942
94. Dadaczynski K, Rathmann K, Schricker J, Bilz L, Sudeck G, Fischer SM, et al. Digital health literacy of pupils. Level and associations with physical activity and dietary behavior. Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz. (2022) 65(7-8):784–94. doi: 10.1007/s00103-022-03548-5
95. Dadaczynski K, Okan O, Messer M, Leung AYM, Rosario R, Darlington E, et al. Digital health literacy and web-based information-seeking behaviors of university students in Germany during the COVID-19 pandemic: cross-sectional survey study. J Med Internet Res. (2021) 23(1):e24097. doi: 10.2196/24097
96. Loorbach N, Peters O, Karreman J, Steehouder M. Validation of the instructional materials motivation survey (IMMS) in a self-directed instructional setting aimed at working with technology. Br J Educ Technol. (2014) 46:204–18. doi: 10.1111/bjet.12138
98. Weiner BJ, Lewis CC, Stanick C, Powell BJ, Dorsey CN, Clary AS, et al. Psychometric assessment of three newly developed implementation outcome measures. Implement Sci. (2017) 12(1):108. doi: 10.1186/s13012-017-0635-3
99. Hak T, Kvd V, Jansen H. The three-step test-interview (TSTI): an observation-based method for pretesting self-completion questionnaires. Surv Res Methods. (2008) 2(3):143–50. doi: 10.18148/srm/2008.v2i3.1669
100. Hak T, van der Veer K, Ommundsen R. An application of the three-step test-interview (TSTI): a validation study of the Dutch and Norwegian versions of the ‘illegal aliens scale’. Int J Soc Res Methodol. (2006) 9(3):215–27. doi: 10.1080/13645570600656264
101. Chu A, Huber J, Mastel-Smith B, Cesario S. “Partnering with seniors for better health”: computer use and internet health information retrieval among older adults in a low socioeconomic community. J Med Libr Assoc. (2009) 97(1):12–20. doi: 10.3163/1536-5050.97.1.003
102. Woodward AT, Freddolino PP, Blaschke-Thompson CM, Wishart DJ, Bakk L, Kobayashi R, et al. Technology and aging project: training outcomes and efficacy from a randomized field trial. Ageing Int. (2010) 36(1):46–65. doi: 10.1007/s12126-010-9074-z
103. Xie B. Effects of an eHealth literacy intervention for older adults. J Med Internet Res. (2011) 13(4):e90. doi: 10.2196/jmir.1880
104. Nahm ES, Zhu S, Bellantoni M, Keldsen L, Russomanno V, Rietschel M, et al. The effects of a theory-based patient portal E-learning program for older adults with chronic illnesses. Telemed J E Health. (2019) 25(10):940–51. doi: 10.1089/tmj.2018.0184
105. Snow G. Blockrand: randomization for block random clinical trials (2020). Available online at: https://CRAN.R-project.org/package=blockrand (Accessed August 19, 2024).
106. Rosset M, Reifegerste D, Baumann E, Kludt E, Weg-Remers S. Trends in cancer information services over 25 years: an analysis of inquiries from patients and relatives made to the cancer information service of the German cancer research center from 1992 to 2016. Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz. (2019) 62(9):1120–8. doi: 10.1007/s00103-019-02996-w
107. Ronckers C, Spix C, Trübenbach C, Katalinic A, Christ M, Cicero A, et al. Krebs in Deutschland Für 2019/2020. Berlin: Robert Koch-Institut (2023).
108. Ruth M, Peritz E, Gabriel KR. On closed testing procedures with special reference to ordered analysis of variance. Biometrika. (1976) 63(3):655–60. doi: 10.1093/biomet/63.3.655
109. Pinheiro J, Bates D. Nlme: linear and nonlinear mixed effects models. r package version 3.1-163 (2023). Available online at: https://CRAN.R-project.org/package=nlme (cited 2023 16-11).
110. Glymour MM, Weuve J, Berkman LF, Kawachi I, Robins JM. When is baseline adjustment useful in analyses of change? An example with education and cognitive change. Am J Epidemiol. (2005) 162(3):267–78. doi: 10.1093/aje/kwi187
111. Allison PD. Change scores as dependent variables in regression analysis. Sociol Methodol. (1990) 20:93–114. doi: 10.2307/271083
112. Schad DJ, Vasishth S, Hohenstein S, Kliegl R. How to capitalize on a priori contrasts in linear (mixed) models: a tutorial. J Mem Lang. (2020) 110:104038. doi: 10.1016/j.jml.2019.104038
113. Lenth R. Emmeans: estimated marginal means, aka least-squares means. R package version 1.10.4 (2024). Available online at: https://cran.r-project.org/web/packages/emmeans/emmeans.pdf (cited 2024 28-08).
114. Honaker J, King G, Blackwell M. Amelia ii: a program for missing data. J Stat Softw. (2011) 45(7):1–47. doi: 10.18637/jss.v045.i07
115. Rubin DB. Multiple Imputation for Nonresponse in Surveys. New York: John Wiley & Sons Inc (1987).
116. Hayes AF. Introduction to Mediation, Moderation, and Conditional Process Analysis: A Regression-Based Approach. New York, NY, US: Guilford Press (2013).
117. Murray TS, Hagey J, Willms D, Shillington R, Desjardins R, Learning C. Health Literacy in Canada: A Healthy Understanding. Ottawa, ON: Canadian Council on Learning (2008).
118. Me K, Greenberg E, Jin Y, Paulsen C. The health literacy of America’s adults: results from the 2003 national assessment of adult literacy (NCES 2006–483) (2006).
119. Sorensen K, Pelikan JM, Rothlin F, Ganahl K, Slonska Z, Doyle G, et al. Health literacy in Europe: comparative results of the European health literacy survey (HLS-EU). Eur J Public Health. (2015) 25(6):1053–8. doi: 10.1093/eurpub/ckv043
120. Berkman ND, Sheridan SL, Donahue KE, Halpern DJ, Viera A, Crotty K, et al. Health literacy interventions and outcomes: an updated systematic review. Evid Rep Technol Assess. (2011) 199:1–941.
121. Liu YB, Liu L, Li YF, Chen YL. Relationship between health literacy, health-related behaviors and health status: a survey of elderly Chinese. Int J Environ Res Public Health. (2015) 12(8):9714–25. doi: 10.3390/ijerph120809714
122. Svendsen MT, Bak CK, Sorensen K, Pelikan J, Riddersholm SJ, Skals RK, et al. Associations of health literacy with socioeconomic position, health risk behavior, and health status: a large national population-based survey among Danish adults. BMC Public Health. (2020) 20(1):565. doi: 10.1186/s12889-020-08498-8
123. Rowlands G, Russell S, O'Donnell A, Kaner E. What is the Evidence on Existing Policies and Linked Activities and Their Effectiveness for Improving Health Literacy at National, Regional and Organizational Levels in the Who European Region? Copenhagen, Denmark: World Health Organization (WHO) Regional Office for Europe (2018).
124. Halverson JL, Martinez-Donate AP, Palta M, Leal T, Lubner S, Walsh MC, et al. Health literacy and health-related quality of life among a population-based sample of cancer patients. J Health Commun. (2015) 20(11):1320–9. doi: 10.1080/10810730.2015.1018638
125. van Deursen AJ, van Dijk J. Internet skills performance tests: are people ready for eHealth? J Med Internet Res. (2011) 13(2):e35. doi: 10.2196/jmir.1581
126. Agormedah EK, Quansah F, Ankomah F, Hagan JE Jr., Srem-Sai M, Abieraba RSK, et al. Assessing the validity of digital health literacy instrument for secondary school students in Ghana: the polychoric factor analytic approach. Front Digit Health. (2022) 4:968806. doi: 10.3389/fdgth.2022.968806
127. Cetin M, Gumus R. Research into the relationship between digital health literacy and healthy lifestyle behaviors: an intergenerational comparison. Front Public Health. (2023) 11:1259412. doi: 10.3389/fpubh.2023.1259412
128. Kim H, Yang E, Ryu H, Kim HJ, Jang SJ, Chang SJ. Psychometric comparisons of measures of eHealth literacy using a sample of Korean older adults. Int J Older People Nurs. (2021) 16(3):e12369. doi: 10.1111/opn.12369
129. Martins S, Augusto C, Martins MRO, Jose Silva M, Okan O, Dadaczynski K, et al. Adaptation and validation of the digital health literacy instrument for Portuguese university students. Health Promot J Austr. (2022) 33(Suppl 1):390–8. doi: 10.1002/hpja.580
130. Nie L, Zhao J, Pan L, Pang M, Wang J, Zhou Y, et al. Validation of the digital health literacy assessment among the university students in China. Front Public Health. (2023) 11:1294183. doi: 10.3389/fpubh.2023.1294183
131. Park E, Kwon M. Testing the digital health literacy instrument for adolescents: cognitive interviews. J Med Internet Res. (2021) 23(3):e17856. doi: 10.2196/17856
132. Rivadeneira MF, Miranda-Velasco MJ, Arroyo HV, Caicedo-Gallardo JD, Salvador-Pinos C. Digital health literacy related to COVID-19: validation and implementation of a questionnaire in hispanic university students. Int J Environ Res Public Health. (2022) 19(7):1–14. doi: 10.3390/ijerph19074092
133. Xie L, Mo PKH. Comparison of eHealth literacy scale (eHEALS) and digital health literacy instrument (DHLI) in assessing electronic health literacy in Chinese older adults: a mixed-methods approach. Int J Environ Res Public Health. (2023) 20(4):1–13. doi: 10.3390/ijerph20043293
134. Statistisches Bundesamt. 80% Der Bevölkerung Sprechen Zu Hause Ausschließlich Deutsch (2023). Available online at: https://www.destatis.de/DE/Presse/Pressemitteilungen/Zahl-der-Woche/2023/PD23_08_p002.html (Accessed August 29, 2024).
Keywords: digital health, eHealth, health literacy, eHealth literacy, health information, e-learning, medical education, digital health literacy
Citation: Lange-Drenth L, Willemer H, Banse M, Ernst A, Daubmann A, Holz A, Bleich C, Weg-Remers S and Schulz H (2025) Development and effectiveness evaluation of an interactive e-learning environment to enhance digital health literacy in cancer patients: study protocol for a randomized controlled trial. Front. Digit. Health 7:1455143. doi: 10.3389/fdgth.2025.1455143
Received: 26 June 2024; Accepted: 10 January 2025;
Published: 24 January 2025.
Edited by:
Steven Hoffman, Brigham Young University, United StatesReviewed by:
Ozden Gokdemir, İzmir University of Economics, TürkiyeDurga R, Enfield and North East London, United Kingdom
Samar J. Melhem, The University of Jordan, Jordan
Copyright: © 2025 Lange-Drenth, Willemer, Banse, Ernst, Daubmann, Holz, Bleich, Weg-Remers and Schulz. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Lukas Lange-Drenth, bHUubGFuZ2VAdWtlLmRl
†These authors have contributed equally to this work and share senior authorship