Utility and usability evaluation of an information diary tool to measure health information access and exposure among patients with high cardiovascular risk

Background: Online health misinformation about statins potentially affects health decision-making on statin use and adherence. We developed an information diary platform (IDP) to measure topic-specific health information exposure where participants record what information they encounter. We evaluated the utility and usability of the smartphone diary from the participants' perspective.

Methods: We used a mixed-method design to evaluate how participants used the smartphone diary tool and their perspectives on usability. Participants were high cardiovascular-risk patients recruited from a primary care clinic and used the tool for a week. We measured usability with the System Usability Scale (SUS) questionnaire and interviewed participants to explore utility and usability issues.

Results: The information diary was available in three languages and tested with 24 participants. The mean SUS score was 69.8 ± 12.9. Five themes related to utility were: IDP functions as a health information diary; supporting discussion of health information with doctors; wanting a feedback function about credible information; increasing awareness of the need to appraise information; and wanting to compare levels of trust with other participants or experts. Four themes related to usability were: ease of learning and use; confusion about selecting the category of information source; capturing offline information by uploading photos; and recording their level of trust.

Conclusion: We found that the smartphone diary can be used as a research instrument to record relevant examples of information exposure. It potentially modifies how people seek and appraise topic-specific health information.

1. Introduction

The increase in the number of sources and conduits for health information means that there are now many more ways to access health information, both credible and evidence-based, as well as low-quality or misleading (1). The health information people are exposed to has a complex influence on their self-care and decision-making. Good health information increases patients' understanding of their diseases, participation in shared decision-making and adherence to treatment (2). People who seek health information have a higher intention to adopt healthy lifestyles (3). However, exposure to low-quality health information can negatively affect health behaviors (4). Frequent online health information-seeking has been shown to associate with lower medication adherence (5). During the COVID-19 pandemic, exposure to online misinformation is correlated to vaccination hesitancy and refusal (6). The World Health Organization (WHO) recently issued a call to action in the area of infodemic management, including recommendations related to finding new ways to use digital technologies to promote access to good information and to measure the burden of the infodemic on behaviors (7).

People can obtain health information either actively or passively (8, 9). Active health information-seeking is a goal-orientated activity where people seek specific health-related information, and includes searching online or a consultation with a health professional (10). Passive health information exposure occurs when people see, read, or hear health-related information while doing other activities, including via traditional media advertising or while browsing social media online (11, 12). Health behavior research has been attempting to measure health information-seeking and exposure, and relate it to health behaviors. However, the relationship remains variable; for example, a recent systematic review published by the authors found no association between health-information-seeking and medication adherence, and raised concern regarding the validity of the research instruments in capturing the health information-seeking behavior (13).

Traditionally, information access and exposure can be measured by asking participants to recall sources as part of surveys conducted at a single point (2, 14). An alternative tool is a media use diary, where participants record what they see as they encounter it, which can enable prospective studies and studies that better examine information exposure over time. Media use diaries, while less likely to lead to recall bias, require efforts from the participants in writing and logging their information diaries (15). More recently, there has been a range of studies that use data from social media at scale to estimate information exposure. However, these studies do not recruit participants, and hence remain disconnected from their health behaviors (16, 17). For studies that recruit participants, there is a lack of standardized approaches and inconsistency in how information access and exposure are measured (2, 13). The wide variations in how research captured health information-seeking behavior make data synthesis and comparison across studies challenging.

Statin use has been extensively debated in mainstream media and social media since 2013 when an article was published stating that statin side effects are much higher than reported in clinical trials (18). In the online platforms, statins received a predominantly negative portrayal that was disproportionately biased toward adverse effects of treatment, questioning the reliability of research evidence with potential pharmaceutical industry influence (19). As a result, several cardiovascular health studies reported that statin-related health misinformation is associated with statin non-adherence (20, 21). Online health misinformation about statins potentially affects health decision-making on statin use for cardiovascular disease prevention (22, 23).

To address these challenges, we developed the Information Diary Platform (IDP) which is a web-based application that includes a smartphone-based diary tool and a researcher dashboard. The smartphone-based diary tool was designed to be simple to use and allowed study participants to record their access and exposure to topic-specific health information. Research participants capture the information sources and rate how much they trust each source as they enter a new record. The research dashboard allows researchers to tailor the diary tool according to the health topic and sources of information they wish to study. In this study, using statin adherence as an example, we evaluated the utility and usability of the IDP diary tool for recording topic-specific health information access and exposure among patients with high cardiovascular risks.

2. Methods

2.1. Information diary platform

The IDP is a web-based research support platform used to design and administer studies that measure health information access and exposure. The diary tool can be used on smartphones or any other device with a web browser to capture health information access and exposure. When participants encounter a piece of health information, they log in to the diary tool and record the health information as a new entry (Figure 1). The IDP includes a researcher dashboard to support the delivery of the smartphone diary tool. The researcher dashboard is a web-based platform with templates for researchers to customize the smartphone diary tool to match their topic and study design. Study investigators can enter their study details into the researcher dashboard and access the data when authenticated on the platform.

FIGURE 1

Figure 1. This storyboard illustrates the process of capturing health information using an information diary platform (IDP) from a user perspective.

The IDP captured category and source of health information, as well as the participant's perception of how much they trust the information (Figure 2). Firstly, the participant was given three categories of health information sources to choose: (a) “Online search”: when they were actively searching for topic-specific health information via a search engine; (b) “Online browsing”: when they encountered relevant health information while browsing social media, online news, and other webpages; and (c) “Offline”: when they encountered information offline, such as printed advertisements or in conversations with health professionals. Secondly, the participant selected a sub-category, which included some predefined examples of sources such as YouTube, Facebook, newspapers and friends. Participants could also add a new source under the sub-category. The participant was encouraged to write a short description of the health information or copy a URL of information from the Internet. For offline information, participants could attach an image of the health information. Finally, the participant rated their trust level for each health information they recorded. Participants were able to access a history page showing the details of their previous entries.

FIGURE 2

Figure 2. Screenshots of the information diary tool including (from left to right): (A) home page of the diary tool; (B) choosing the information category; (C) selecting the information source; (D) determining the trust level of information; (E) history page.

2.2. Study design

We used a convergent mixed-methods study design to evaluate the utility and usability of the diary tool from the perspective of the participants enrolled in this study. A mixed-methods study design supports a comprehensive view of the usability challenges in different domains, including an in-depth understanding of usability problems that can be used to revise and improve the diary tool. Our quantitative component was an evaluation of the usability of the information diary tool using a System Usability Scale (SUS) questionnaire and measured how users interacted with the information diary tool via metadata stored by the IDP during the pilot study. Our qualitative component was a set of interviews using a think-aloud method. Both quantitative and qualitative data were collected at the same time, and data were analyzed and integrated to extend the breadth of our understanding of the usability issues.

2.3. Study setting and study population

This study was approved by the University of Malaya Medical Center Medical Research Ethics Committee (MECID No: 2021324-9981). Written informed consent was obtained from all individual participants included in the study. We conducted this study at an urban primary care clinic in the University Malaya Medical Center (UMMC) in Kuala Lumpur, Malaysia. Kuala Lumpur is the capital city in Malaysia and the main languages used are Bahasa Malaysia, English, and Mandarin.

We recruited patients with high cardiovascular risk attending the primary care clinic during the recruitment period. The inclusion criteria were: (a) aged ≥ 18 years; (b) patients who had high cardiovascular risk where statin use was indicated as per guideline (10), including those with pre-existing cardiovascular disease (CVD), diabetes mellitus, chronic kidney disease ≥ stage 3, and Framingham General CVD risk score ≥ 20%; and (c) patients who owned and used a smartphone. The exclusion criteria were: (a) patients who were unable to read English, Malay, or Mandarin; and (b) patients who were too ill or cognitively impaired to participate.

We purposively sampled participants from different age groups, gender, ethnicity, language, and educational levels to achieve maximum variation. The sample size of a usability study is typically small (around 5–10 participants) (24); as the information diary tool was made available in three languages, we aimed to recruit 5–10 participants per language. We stopped recruiting participants when data saturation was reached after 24 interviews where no new themes emerged from data analysis and field notes.

2.4. Data collection

Researchers briefed the participants about the study objectives and method using a participant information sheet. Researchers encouraged participants to ask questions and informed the participants that their participation was voluntary and that they could leave the study at any time. Once participants agreed to participate, they were asked to sign a consent form and complete a form that captured their demographic characteristics.

A video and infographic user guides were developed to teach the participants how to use the information diary tool with guidance from research assistants. Participants were asked to capture any relevant health information that they encountered for a week. At the end of the week, participants were invited to the clinic for a face-to-face interview. If participants were unable to attend the clinic physically, they were interviewed over video conferencing.

We used the System Usability Scale (SUS) questionnaire to assess the usability of the diary tool from the user perspective (25, 26). Before the start of the interview, participants were asked to answer the SUS questionnaire. Participants were then asked in an interview about their overall opinions of the tool and detailed questions about its functionality and ease of use. We developed a semi-structured interview guide (Appendix 1) based on Nielsen's model of usability (27). For the think-aloud method, we asked the participants to show us how they used the diary tool to capture information and think out loud about how they performed the tasks. We asked the participants to comment on how they captured examples of relevant information and discussed any problems they encountered. We also asked the participants for suggestions on how to improve the usability of the diary tool. The interviews were conducted by researchers (HML and WXL) trained in qualitative research and digital health research. All interviews were audio-recorded.

2.5. Evaluation measures and data analysis

The SUS questionnaire was used to assess usability after a week of testing by the participants. SUS (10 items rated on a 5-point Likert scale) was selected as it has been validated and widely used in digital health applications (25, 26). We reported the SUS results as a total score using mean and standard deviation, as well as with descriptive data for each item. An SUS score of above 68/100 is considered as good usability (25). To evaluate how users recorded entries using the diary tool, we extracted their submission data from the IDP researcher dashboard, which contains data on the number of entries for each participant, categories of information source, and levels of trust for each record.

After each interview, researchers listed the issues and comments on the field notes. All audio recordings were transcribed verbatim and checked independently by a researcher (HML). We used a thematic approach to analyze the data (28). For the first two transcripts, two researchers (HML, WXL) familiarized themselves with the data by reading and re-reading the transcripts and field notes and generating initial codes independently. Both researchers met to discuss and compare the codes and coding frame, together with other researchers (AGD and CJN). Any differences between the codes were solved through consensus. One researcher (HML) then coded the remaining transcripts. The research team met regularly to discuss new codes and emerging themes. The qualitative data analysis was performed using NVivo version 10.

3. Results

3.1. Study participants and IDP usage

A total of 134 eligible patients were approached and invited to participate in this study (Figure 3). Thirty-three participants agreed to participate. The main reasons why eligible patients chose not to participate were lack of interest in participating in a research, time constraints, and lack of confidence with using a smartphone. Of the 33 participants who agreed, 24 completed a week of testing and participated in the interviews. Most participants were over 50 years old and had an educational level above secondary school (Table 1).

FIGURE 3

Figure 3. From 134 patients approached to participate in the study, 24 completed the evaluation.

TABLE 1

Table 1. Characteristics of participants (n = 24).

3.2. Utility of IDP

Five themes emerged from the qualitative analysis related to the utility of the tool (Table 2, includes participant quotes). These are: (1) IDP functions as a health information diary; (2) use of the history function in supporting discussion of health information with doctors; (3) participants wanting a feedback function about credible information; (4) the diary tool increasing awareness of the need to appraise information; and (5) participants wanting to compare levels of trust with other participants or experts.

TABLE 2

Table 2. Themes and subthemes related to the functionality of the information diary tool.

3.2.1. The IDP functions as a health information diary

In terms of its function as a health information diary, participants liked the idea of a specific application they can use to record and retrieve the health information that they have encountered. Participants expressed that they often forget the source of past information they have encountered, especially from social media and the diary tool helps them remember. Participants felt the tool is useful to compare information sources. Participants suggested an additional function to let them categorize or filter their “History” according to diseases, body systems, or topics rather than by source. Participants also suggested a search function or the ability to label entries so they could find them more easily later.

3.2.2. Supporting discussion of health information with doctors

In terms of its use to support discussion with doctors, participants stated that they could bring the health information recorded in the diary tool to show and discuss with their doctors during clinic consultations. They were keen to consult their doctors about the information that they have encountered, especially where they were unsure about its credibility. They stated that the diary tool would save time during consultations, and that it would be useful for healthcare providers to be aware of the types of information they encounter.

3.2.3. Feedback function about credible information

In terms of feedback about credible information, participants were interested in receiving feedback from the diary tool about the credibility of what they see, beyond just recording their own perceptions of trust. They wished that the diary tool would help assess the information they have encountered or provide credible alternative health information. Participants hoped to get feedback from healthcare professionals to verify the credibility of the information they accessed.

3.2.4. Increasing the awareness of the need to appraise information

While recording the health information that they encountered, some participants revealed that they became more conscious of the need to critically appraise the credibility of health information when they entered their trust level into the tool. Participants wished to have more guidance on how to assess the credibility of information sources they encounter.

3.2.5. Views about comparing their results with others

In terms of the views about the value of comparative analytics, participants liked the idea of having a report comparing their information diary with others. Most participants perceived that the information sources they accessed were limited, and they would like to know other information sources on a similar topic. A participant expressed that a comparison of the trust levels of others would help them to reflect on their own trust.

3.3. Usability of IDP

3.3.1. System usability scale

All 24 participants completed the SUS questionnaire after testing the diary tool for a week. The mean score was 69.8, which indicated good usability (Figure 4). Of the 24 participants, 22 agreed that the tool was easy to use, 19 were confident using it, and 18 would use the tool frequently.

FIGURE 4

Figure 4. Results of the system usability scale (SUS) items for 24 participants.

3.3.2. Summary of IDP submissions

There was a total of 452 submissions of health information by the participants (Appendix 2). One participant did not submit any information because he did not understand the function of the diary tool. He was unclear about the function and instructions to use the diary tool. One participant had recorded 230 items because he often encountered health information when surfing through social media. The “online browsing” category had the most submissions (306 submissions, 67.7%), followed by the “online search” category (107 submissions, 23.7%) and then the “offline” category (39 submissions, 8.6%).

Four themes emerged from the interviews related to usability (Table 3, includes participant quotes). These were: (1) ease of learning and using the diary tool; (2) confusion about selecting the category of information source; (3) capturing offline information by uploading photos; and (4) recording their level of trust.

TABLE 3

Table 3. Themes and subthemes related to the usability of the information diary tool.

3.3.3. Ease of learning and using the diary tool

The participants agreed it was easy to learn and use the diary tool. They performed the tasks of submitting their information diary independently without assistance, including those participants who were older and less familiar with mobile technology. Participants found that the short video and visual guide explaining the step-by-step guide were helpful. One participant had difficulty grasping the concept and objectives of the diary tool and suggested a written pamphlet to explain the functions.

3.3.4. Confusion about selecting the category of information source

Participants reported being confused about the selection of categories and subcategories. For example, uncertainty occurred when participants were actively searching for health information from Google, but the Google search results eventually led them to YouTube or other social media applications. Sometimes, participants were being “pushed” with information from Google without actively searching for it, and they were confused whether to choose “online search” or “online browsing” because “Google” is under the category of “online search”. Due to the confusion of the main categories, some participants chose the category based on the sub-category listed. Some participants complained about having too many subcategories listed. Participants suggested changing the main categories to be more aligned with active searching or passive exposure. One of the participants did not upload their information diary from WeChat because the option was not listed, and they were unsure how to add a new source.

3.3.5. Capturing offline information by uploading photos

When participants encountered offline information such as newspapers, billboards, or paper-based written materials, they were encouraged to write a short description of the information or attach a photo showing a snippet of information. Participants faced challenges when attempting to upload photos using the tool because uploading could be slow with slow internet speeds. In addition, some participants could not use the camera function embedded in the diary tool, requiring them to take photos using their phone camera and upload the photo from their photo gallery. Participants rarely wrote the details of offline information in the description box provided.

3.3.6. Recording their level of trust

When deciding on how much they trusted the information sources, some participants were unsure about how to answer. Some participants estimated their trust level relative to their own knowledge, the author of the information, and the source type. Some of the older participants had problems dragging the sliding scale of trust to a specific percentage and took several attempts to move it to the percentage they wanted. One participant preferred the trust scale initially be placed at 0% instead of 50% because they wanted to move the slider to around 50–70%.

4. Discussion

Our findings showed that the diary tool built as a component of the IDP is a usable and useful way to capture topic-specific information access and exposure from study participants, which could then be directly connected to outcome measures such as medication adherence. We found that participants often wanted to use the diary tool in ways that were not intended, so there is a need to maintain the core purpose of the tool for observing behaviors while also ensuring that participants want to continue to use the diary tool appropriately throughout a study observation period.

The diary tool can provide more detailed information about the health information that people access than is possible with traditional one-off administered questionnaires about media use (29). Traditional forms of media use diaries have been used in health research to collect both intentional searching and unintentional exposure to health information (30, 31). The diary tool made it easier for participants to record topic-specific information compared to traditional media use diaries by reducing the effort required. However, since the diary tool requires that participants consciously consider and record relevant information as they encounter it, it also has the potential to influence how participants access and appraise health information during a study observation period. For example, it can make them more aware of what they access, how much information they access, and how they appraise the information they encounter (29).

We discovered that participants using the information diary tool wanted to use it in ways that were not anticipated as part of the design of the diary tool for observational study designs. For example, users reported wanting to use it to discuss health information sources with their doctor, to access credible information, and as a guide to appraise health information. Marien et al. (32) reported a similar finding when tailoring digital interventions to meet the needs of users while also meeting the primary needs of researchers. As intrinsic motivation is a critical factor driving participation in research (33), participants users are likely to be more motivated to engage in research when they perceive the value of the study in addressing their health needs and interest (34). For observational studies, it is important to find a balance between making the diary tool useful for participants so they want to continue to use it throughout a study, while minimizing the potential impact it has on their behaviors.

In our study, study participants have become more aware of the need to appraise health information when they recorded their trust level in the diary tool. This phenomenon was supported by French and Sutton (35), who showed that study participants who are part of the activity of measurement can become more aware of their thinking and feelings. Our study suggested a potential usefulness of the diary tool to remind users to think again the credibility of the health information by just asking them to decide the trust level. Trust in health information depends on how a user perceives credibility and reliability (36), which is in turn influenced by health literacy and their affect toward the information (37). Since trust is an important factor that is likely to be related to how people make sense of information and use it to support their decision making (38), we thought that it was important to keep trust as a component of each record.

This study highlights that an appropriate design for information diaries requires a careful balance between providing features that users want and constraints that minimize the potential impact of the tools on behavior during a study (39). For example, additional features that support critical appraisal of information sources might be valuable for users but may limit the potential for the tool to be used to identify risk signals associated with potential harms such as non-adherence to medication. Our approach to the changes is to limit the number of “behavior-modifying” features that users have access to during a period of observation, and then unlock them to support appraisal and access to evidence-based health information after completion of the data collection.

From the usability evaluation, we learned that users had difficulty with how we described the main categories. It is important to be able to capture the difference between users actively seeking information (both online and offline) and inadvertently being exposed to health information while performing unrelated activities. To effectively capture the differences, future versions of the diary tool would choose the source of the information first (online sources such as a post on a Facebook page, or an advertisement on a webpage, or offline sources such as a consultation with a doctor or discussing with a family member), and then flag whether they were searching for health information or unintentionally encountered it while doing something else.

A key strength of the study was that participants were relevant to topic of study, with high cardiovascular risk and mostly from older age groups. A second strength of the study included the location of Malaysia, a multiracial country with multiple languages, which makes it a useful setting for evaluating the localization of the diary tool and how we made it available in multiple languages.

One limitation of the study was the possibility of selection bias, given that a high proportion of patients declined to participate in this study. Majority of those who participated in this study had at least a degree and, hence, they may have a higher digital literacy, more familiar with using a smartphone and be more inclined to seek high-quality sources of information. A second limitation was related to analyzing whether users were capturing all instances of topic-relevant information they encountered. We had no way of triangulating what they recorded against any form of ground truth observation.

5. Conclusion

The diary tool developed as part of the IDP serves its function as a way of capturing the topic-specific information study participants encounter. Our study showed that use of the diary tool might influence health information-seeking and information appraisal behavior. The diary tool met requirements for usability and can provide a more detailed view of the health information that people encounter compared to traditional methods such as questionnaires. Future development of information diary tools should consider the balance between providing features that users want and the requirements of observational studies that seek to capture information access and exposure.

Data availability statement

The original contributions presented in the study are included in the article/Supplementary material, further inquiries can be directed to the corresponding author.

Ethics statement

The studies involving human participants were reviewed and approved by University of Malaya Medical Center Medical Research Ethics Committee (MECID No: 2021324-9981). The patients/participants provided their written informed consent to participate in this study.

Author contributions

HL, AD, CN, and AA contributed to the conception and design of the study. HL, AD, CN, AA, and JD contributed to developing research tools. HL, WL, and KL were involved in data collection and interviews. HL, WL, AD, and CN performed the data analysis. HL wrote the drafts of the manuscript. All authors contributed to reviewing and editing the submitted version. All authors contributed to the article and approved the submitted version.

Funding

This work was supported by the University of Malaya Specialist Center (UMSC) CARE Fund Grant (Grant No. PV047-2021).

Acknowledgments

The authors thank the staff from the Department of Primary Care Medicine for their support during the data collection.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpubh.2023.1132397/full#supplementary-material

References

1. Cinelli M, Quattrociocchi W, Galeazzi A, Valensise CM, Brugnoli E, Schmidt AL, et al. The COVID-19 social media infodemic. Sci Rep. (2020) 10:16598. doi: 10.1038/s41598-020-73510-5

CrossRef Full Text | Google Scholar

2. Thapa DK, Visentin DC, Kornhaber R, West S, Cleary M. The influence of online health information on health decisions: a systematic review. Patient Educ Counsel. (2021) 104:770–84. doi: 10.1016/j.pec.2020.11.016

CrossRef Full Text | Google Scholar

3. Bujnowska-Fedak MM, Wegierek P. The impact of online health information on patient health behaviours and making decisions concerning health. Int J Environ Res Public Health. (2020) 17:880. doi: 10.3390/ijerph17030880

CrossRef Full Text | Google Scholar

4. Swire-Thompson B, Lazer D. Public health and online misinformation: challenges and recommendations. Annu Rev Public Health. (2020) 41:433–51. doi: 10.1146/annurev-publhealth-040119-094127

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Im H, Huh J. Does health information in mass media help or hurt patients? Investigation of potential negative influence of mass media health information on patients' beliefs and medication regimen adherence. J Health Commun. (2017) 22:214–22. doi: 10.1080/10810730.2016.1261970

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Pierri F, Perry BL, DeVerna MR, Yang K-C, Flammini A, Menczer F, et al. Online misinformation is linked to early COVID-19 vaccination hesitancy and refusal. Sci Rep. (2022) 12:5966. doi: 10.1038/s41598-022-10070-w

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Call for Action: Managing the Infodemic. World Health Organization. Available online at: https://www.who.int/news/item/11-12-2020-call-for-action-managing-the-infodemic (accessed January 4, 2021).

Google Scholar

8. Lambert SD, Loiselle CG. Health information—seeking behavior. Qual Health Res. (2007) 17:1006–19. doi: 10.1177/1049732307305199

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Longo DR, Schubert SL, Wright BA, LeMaster J, Williams CD, Clore JN. Health information seeking, receipt, and use in diabetes self-management. Ann Fam Med. (2010) 8:334–40. doi: 10.1370/afm.1115

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Niederdeppe J, Hornik RC, Kelly BJ, Frosch DL, Romantan A, Stevens RS, et al. Examining the dimensions of cancer-related information seeking and scanning behavior. Health Commun. (2007) 22:153–67. doi: 10.1080/10410230701454189

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Abdullah A, Liew SM, Ng CJ, Ambigapathy SV, Paranthaman PV. Health literacy experiences of multi-ethnic patients and their health-care providers in the management of type 2 diabetes in Malaysia: a qualitative study. Health Expect. (2020) 23:1166–76. doi: 10.1111/hex.13095

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Anker AE, Reinhart AM, Feeley TH. Health information seeking: a review of measures and methods. Patient Educ Couns. (2011) 82:346–54. doi: 10.1016/j.pec.2010.12.008

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Lim HM, Dunn AG, Lim JR, Abdullah A, Ng CJ. Association between online health information-seeking and medication adherence: a systematic review and meta-analysis. Digit Health. (2022) 8:20552076221097784. doi: 10.1177/20552076221097784

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Tan SS, Goonawardene N. Internet health information seeking and the patient-physician relationship: a systematic review. J Med Internet Res. (2017) 19:e9. doi: 10.2196/jmir.5729

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Sillence E, Briggs P, Harris PR, Fishwick L. How do patients evaluate and make use of online health information? Soc Sci Med. (2007) 64:1853–62. doi: 10.1016/j.socscimed.2007.01.012

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Dunn AG, Mandl KD, Coiera E. Social media interventions for precision public health: promises and risks. npj Digital Medicine. (2018) 1:47. doi: 10.1038/s41746-018-0054-0

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Dunn AG, Steffens M, Dyda A, Mandl KD. Knowing when to act: a call for an open misinformation library to guide actionable surveillance. Big Data Soc. (2021) 8:20539517211018788. doi: 10.1177/20539517211018788

CrossRef Full Text | Google Scholar

18. Abramson JD, Rosenberg HG, Jewell N, Wright JM. Should people at low risk of cardiovascular disease take a statin? BMJ. (2013) 347:f6123. doi: 10.1136/bmj.f6123

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Chisnell J, Marshall T, Hyde C, Zhelev Z, Fleming LE, A. content analysis of the representation of statins in the British newsprint media. BMJ Open. (2017) 7:e012613. doi: 10.1136/bmjopen-2016-012613

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Nielsen SF, Nordestgaard BG. Negative statin-related news stories decrease statin persistence and increase myocardial infarction and cardiovascular mortality: a nationwide prospective cohort study. Eur Heart J. (2016) 37:908–16. doi: 10.1093/eurheartj/ehv641

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Matthews A, Herrett E, Gasparrini A, Van Staa T, Goldacre B, Smeeth L, et al. Impact of statin related media coverage on use of statins: interrupted time series analysis with UK primary care data. BMJ. (2016) 353:i3283. doi: 10.1136/bmj.i3283

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Agarwala A, Kohli P, Virani SS. Popular media and cardiovascular medicine: “with great power there must also come great responsibility”. Curr Atheroscl Rep. (2019) 21:43. doi: 10.1007/s11883-019-0807-5

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Navar AM. Fear-based medical misinformation and disease prevention: from vaccines to statins. JAMA Cardiol. (2019) 4:723–4. doi: 10.1001/jamacardio.2019.1972

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Faulkner L. Beyond the five-user assumption: benefits of increased sample sizes in usability testing. Behav Res Methods Instrum Comput. (2003) 35:379–83. doi: 10.3758/BF03195514

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Brooke J. SUS: A “Quick and Dirty” usability scale. In:Jordan PW, Thomas B, Weerdmeester BA, McClelland, , editors. Usability Evaluation in Industry., London: Taylor & Francis (1996). p. 189–94.

Google Scholar

26. Mohamad Marzuki MF, Yaacob NA, Yaacob NM. Translation, cross-cultural adaptation, and validation of the Malay version of the system usability scale questionnaire for the assessment of mobile apps. JMIR Hum Factors. (2018) 5:e10308. doi: 10.2196/10308

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Nielsen Norman Group,. Usability 101: Introduction to Usability. (2003). Available online at: http://www.nngroup.com/articles/usability-101-introduction-to-usability/ (accessed February 22, 2022).

Google Scholar

28. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. (2006) 3:77–101. doi: 10.1191/1478088706qp063oa

CrossRef Full Text | Google Scholar

29. Broekhuis M, van Velsen L, De Franco D, Pease A, Hermens H. Contextual health information behavior in the daily lives of people with type 2 diabetes: a diary study in Scotland. Health Commun. (2022) 37:272–84. doi: 10.1080/10410236.2020.1837426

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Lim HM, Wong SS, Yip KC, Chang FWS, Chin AJZ, Teo CH, et al. Online health information-seeking behaviour of patients attending a primary care clinic in Malaysia: a cross-sectional study. Fam Pract. (2022) 39:38–45. doi: 10.1093/fampra/cmab099

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Wong DK-K, Cheung M-K. Online health information seeking and eHealth literacy among patients attending a primary care clinic in Hong Kong: a cross-sectional survey. J Med Internet Res. (2019) 21:e10831. doi: 10.2196/10831

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Marien S, Legrand D, Ramdoyal R, Nsenga J, Ospina G, Ramon V, et al. A web application to involve patients in the medication reconciliation process: a user-centered usability and usefulness study. J Am Med Inform Assoc. (2018) 25:1488–500. doi: 10.1093/jamia/ocy107

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Tiziana CC, Louise M, Janet S, Gill W, Nikki H, Julie W. Exploring participation needs and motivational requirements when engaging older adults in an emerging living lab. Technol Innov Manag Rev. (2019) 9:1223. doi: 10.22215/timreview/1223

CrossRef Full Text | Google Scholar

34. Lievens B, Baccarne B, Veeckman C, Logghe S, Schuurman D. Drivers for End-users' Collaboration in Participatory Innovation Development and Living Lab Processes, The 17th ACM Conference on Computer Supported Cooperative Work (CSCW), Baltimore, MA, USA (2014).

Google Scholar

35. French DP, Sutton S. Reactivity of measurement in health psychology: how much of a problem is it? What can be done about it? Br J Health Psychol. (2010) 15:453–68. doi: 10.1348/135910710X492341

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Allen Catellier JR, Yang ZJ. Trust and affect: how do they impact risk information seeking in a health context? J Risk Res. (2012) 15:897–911. doi: 10.1080/13669877.2012.686048

CrossRef Full Text | Google Scholar

37. Chen X, Hay JL, Waters EA, Kiviniemi MT, Biddle C, Schofield E, et al. Health literacy and use and trust in health information. J Health Commun. (2018) 23:724–34. doi: 10.1080/10810730.2018.1511658

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Huh J, Shin W. Trust in prescription drug brand websites: website trust cues, attitude toward the website, and behavioral intentions. J Health Commun. (2014) 19:170–91. doi: 10.1080/10810730.2013.798386

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Janssens KAM, Bos EH, Rosmalen JGM, Wichers MC, Riese H. A qualitative approach to guide choices for designing a diary study. BMC Med Res Methodol. (2018) 18:140. doi: 10.1186/s12874-018-0579-6

PubMed Abstract | CrossRef Full Text | Google Scholar