Preferences in the Intention to Download a COVID Tracing App: A Discrete Choice Experiment Study in the Netherlands and Turkey

Introduction: High levels of adoption and usage for the COVID Tracing Apps (CTA) among the population is a stipulated prerequisite for success of the implementation of these apps, aiming to mitigate the pandemic and track spreading of the virus more efficient and effectively. In the current study, the main objective was to investigate individuals' preferences in the intention to download a COVID-19 tracing app in a pilot-study in both the Netherlands and Turkey.

Methods: We conducted a discrete choice experimental study through an online survey in two countries (the Netherlands [N = 62] and Turkey [N = 83]), with four different attributes: (1) data protection (data protection vs. no information), (2) manufacturer (government vs. company), (3) reward (no reward vs. voucher as a reward) and (4) gaming (no gaming elements vs. gaming elements). Participants were recruited among a student population.

Results: The results showed that data protection is one of the most important factors that significantly increases the probability to adopt a CTA. In general, the manufacturer, reward or gaming affected the probability to download the CTA less.

Discussions: Health authorities worldwide have generally released high quality CTA, although scientific studies assessing the most important factors that describe and predict the intention to download is limited. Sensitive personal data is collected through these apps, and may potentially threaten privacy, equality and fairness, which are important attributes to take into account when developing or launching a CTA, following the results of this study.

Introduction

After 2 years of suffering due to the COVID-19 pandemic, considering the negative effects on human health, and on social and economic life, it is important to investigate the mitigation strategies that have been implemented by societies worldwide. Investigating mass acceptance of the requirements of tracing apps is crucial in order to provide contextualized insights and provide policy makers by providing design implications (Anglemyer et al., 2020; Trang et al., 2020). Strategies that have been considered effective are technological ones, for example the use of mobile apps (Bassi et al., 2020; Morley et al., 2020). Early scientific evidence has shown that COVID-19 is highly contagious, often without the carrier noticing any symptoms, thereby subconsciously spreading the virus to others and infecting several close contacts, mostly even before actually testing positive. Considering the rapidly increasing numbers of infected people in a majority of the countries worldwide during some periods over the last two years, manually collecting the information from the population needs a considerable workforce of health officials trained to hold the interviews effectively, which is a highly costly exercise and has shown to be ineffective in times of high percentages of infections.

COVID-19 Tracing Apps (CTAs) have been considered as an (cost-)effective mitigation strategy, by informing people as soon as possible when they have been in close contact with an infected person (Ming et al., 2020; Lee et al., 2021). Nonetheless, the actual adoption and usage of mobile health apps is rather low and lagging behind its promises (Walrave et al., 2020). Recent studies have started to better understand how CTA's can address a wide range of conflicting elements, such as preserving citizens' privacy while gleaning close contacts and epidemiological information as possible (Tsvyatkova et al., 2022), but there is still limited evidence supporting adoption or willingness to download preferences for CTA's. It is therefore needed to better understand which attributes affect people's willingness to download a CTA (Anastasiadou et al., 2018a,b).

As mobile technologies are rapidly developing as a method for delivering behavior change interventions, health communication researchers need to improve the understanding of how to advance current theories such that we can leverage and maximize the potential of the ubiquity, adaptability, and affordability of mobile health apps in changing health related behaviors (Ng et al., 2012; Zakharchenko et al., 2022). A first step in this endeavor is to understand the impact of various mobile health apps attributes in explaining people's willingness to adopt these apps, in particular the CTAs that have limited usefulness, namely only during a pandemic.

Considering the limited understanding of the general cognitive motivators that trigger people's adoption of health apps, especially in times of crisis, it is important to examine which attributes of health apps are preferred in the intention to download of a health app. Intrinsic motivation to use such a health app is considered a strong predictor of the intention to download and usage (Ng et al., 2012). Without proper comprehension of the cognitive motivators that explain CTAs intention to download and use, it will be very difficult to establish the effectiveness of mobile health apps and fully understand individuals' use of such apps. From the technical development point of view, this kind of understanding will be very valuable when designing and implementing new mobile health apps, or improving the existing ones with novel functions.

Preferences for CTA's

Several factors play a role in downloading a CTA. According to Klar and Lanzerath (2020), technological problems and the risks of privacy and equity are important factors that predict which CTAs will be downloaded or not. As many governments have communicated during the implementation of the CTAs, one of the pre-requisites of an ethical deployment of CTAs is voluntariness to download and use the app. In addition, Peschke et al. (2021) discussed voluntariness of CTA deployment in different countries in their systematic review of good practices of CTAs. In their review they analyzed more than 35 CTAs worldwide (Peschke et al., 2022), but how important data protection and privacy issues are regarded by potential users is not sufficiently understood yet.

Furthermore, the manufacturer of a CTA can be considered a heuristic in consumers' intention to download and use the CTA, affecting the preference to download the app (Gallego et al., 2021). There is reason to assume that this may be the case, as the industry has struggled with its public image over the past few decades. Companies need to negotiate a tension between, on the one hand, striving for optimal health care and, on the other hand, striving for profit (Bauchner and Fontanarosa, 2013). In the eyes of the public, it is not always clear that the industry has patients' interests at heart (Olsen and Whalen, 2009). Therefore, if a CTA is manufactured by the government, people probably will prefer this more than when the app is manufactured by a company. In addition, the Edelman Trust Barometer (2021) showed that business is the only institution that is understood as competent and ethical by the public in 18 of 27 countries evaluated. The public is more likely to trust business by being the guardian of information quality, embracing sustainable practices and providing robust COVID-19 healthy and safety response (Edelman Trust Barometer, 2021, p. 36). Bostan et al. (2020) conclude that the public trust in the authorities and the accuracy of the decisions taken by the state is of great importance in order to effectively combat the pandemic and being able to apply societal mitigation strategies. However, Tanca et al. (2020) perceived ambiguity and capriciousness of the public's reactions as a result of the uncertainty of both the government and the national economy. Therefore it is interesting to test whether the manufacturer of a CTA influences the preference of participants to download the CTA.

At the center of these concerns about maintenance of privacy, there also lies the need for an efficient understanding of attitudes toward technology. A recent survey has demonstrated that governments and private companies should take into account the general attitudes toward technology in order to increase willingness and overcome challenges faced by the public in the use of such apps (Jansen-Kosterink et al., 2021). As valuable technological tools, CTAs can be largely benefited in the expansion of preventive measures against COVID-19. Consequently, it should be noted that promoting willingness to maintain digital preventive behavior on behalf of the public partly depends on the perceptions of technological tools in general.

Current Study

In the current study, we conducted a discrete choice experiment (DCE) in two different countries (the Netherlands and Turkey) and assessed likelihood to adopt the CTA. We aim to assess a comprehensive assessment of the preferences of people for the intention to download of a health app. To our knowledge, no previous study has attempted to disentangle preferences the intention to download of a CTA using a DCE. Insights into attributes that could influence whether individuals are more likely to adopt a contact tracing app should therefore be very helpful in informing product development and pathway redesign for future implementation of technologies during pandemic situations. The main objective of the current study is to investigate individuals' preferences in the intention to download a COVID-19 tracing app in a pilot-study in both the Netherlands and Turkey.

Methods

Design

In the current study we used a DCE in an online questionnaire. In comparison with other preference elicitation methods, a DCE can clearly quantify the relative importance of different attributes that characterize a new or existing product and/or service, identifying which attribute(s) people prioritize or accept, and which they may be willing to exchange in order to maximize their utility (Lancsar and Louviere, 2008). A DCE requires participants to choose between competing scenarios that are considered to be relevant in a current context, described in terms of particular attribute (e.g., data protection) and a range of levels (e.g., protected European legislation vs. no information) and to compare these against an alternative scenario that elicits specific preferences among participants. DCE studies are relevant because they allow a direct cognitive assessment of relative preferences for various existing and hypothetical new service configurations or treatment approaches and is a common methodology in consumer and marketing research, and implemented in other research areas more and more.

The online questionnaire that we used in the current study adopted a main effects design, using all attributes included in the study. In a main effects design, all possible levels are included and each pair of levels occurs equally often. A full factorial design was ruled out in favor of a fractional factorial design because a full factorial design would have contained too many possible alternatives, which would have been unmanageable in practice for individuals to complete or for a blocked questionnaire format to handle.

The use of an online questionnaire provided completion time data to support the internal validity checks and enabled an accurate record of time taken to complete the surveys. Two rounds of cognitive testing (n = 12) were undertaken in the Netherlands to check participants' comprehension of information when making choices. These pre-tests confirmed that a study based on the questionnaire was acceptable and understandable for participants, after some minor revisions in the explanation of the task.

The four attributes (data protection, manufacturer, reward, and gaming) and levels selected for inclusion in the DCE are shown in Table 1. The factors have been based on a mix between a literature review (Tsvyatkova et al., 2022) and elements (reward [no reward vs. a voucher as a reward] and gaming [no gaming elements vs. gaming elements]) that were considered relevant for next steps in the Horizon2020 project PandeVITA.¹

TABLE 1

Table 1. The attributes and levels selected for the DCE.

Procedure

All survey participants were informed about the overall study goals and procedures. Only those who agreed to participate in the study gained access to the online survey. Approval of the Ethical Committee of the Tilburg School of Humanities and Digital Sciences to conduct the experiment was obtained (REDC 2021.73). Through signing informed consent, participants were ensured that their data would remain confidential and they were told that they could cease participation at any moment.

First, participants were asked to provide sociodemographic information, including their age, gender, education, and employment status. Subsequently, an introduction to the DCE questionnaire provided an explanation of the four attributes and levels. A generic pairwise choice was selected for the questionnaire design. Participants were presented with a series of choice sets for which there were two responses: “Option A” and “Option B,” varying between the levels of the attributes we selected. A sample choice set is illustrated in Table 2. All participants had to provide their choice in order to go to the next decision, where the same attributes and different levels were presented (randomly) for Option A and Option B.

TABLE 2

Table 2. One example of the discrete choice task participants had to complete.

Participants

For this study, we used data collected from two different countries (Turkey and the Netherlands)². The data in the Netherlands (N = 62) and Turkey (N = 83) were collected through an online survey administered through Qualtrics. Participant characteristics are shown in Table 3.

TABLE 3

Table 3. Descriptive information about the participant per country.

Statistical Analyses

We use a conditional logit model (CLM) to analyze the choice experiment data. We estimate the model first for all participant and then subsequently separately for the two countries. Each app profile is characterized by four attributes with two possible levels (see also Table 1), resulting in 16 possible alternatives among which respondents had to choose in 8 separate choice tasks. The dataset resulting from this experimental design contains one row for each alternative presented to each respondent. Hence, we obtain 2,320 observations, 992 for the Netherlands and 1,328 observations for Turkey.

We specify the deterministic component of the utility function as follows:

$\begin{array}{l}\begin{array}{ccc}{V}_{ij}& =& {\beta }_{1}Developer+{\beta }_{2}Reward+{\beta }_{3 }Gaming\end{array}\\ \begin{array}{c}+ {\beta }_{4}Data Protecion+i.Individual\end{array}& (1)\end{array}$

Where each regressor represents an indicator variable for whether that feature was present in the app. The last term represents the individual fixed-effects, which allow to control for observable and unobservable characteristics constant within participants. The model fits a conditional maximum likelihood for each respondent. When odds ratios (ORs) are lower than 1 it means that the level is less preferred than the reference category. A negative sign in the estimates and probabilities also implies that the feature is less preferred.

Results

Table 4 shows the relative odds ratio of the conditional logit model for the two countries separately and for the full sample. We start with the analysis of the results for the full sample. Unsurprisingly, respondents prefer apps that ensure data protection, with the magnitude of the coefficient being the highest among the four features (OR = 7.675, β = 2.06, 95% CI [6.53, 9.40], P < 0.01). In addition, participants were more likely to choose apps produced by the government, with an OR of 1.863 (β =0.60 95% CI [1.52, 2.20], P < 0.01). The coefficient associated with the presence of a reward mechanism is statistically significant at the 1% level and shows that vouchers increase the attractiveness of tracing apps (OR = 1.173, β = 0.18, 95% CI [1, 1.44], P < 0.01). The odds ratio for the gaming feature is close to 1, suggesting that the gaming component does not influence respondents' preferences toward tracing apps.

TABLE 4

Table 4. Regression results.

The models estimated on the two samples independently portrait a similar picture. Again, respondents prefer apps that ensure data protection, with the magnitude of the coefficient being the highest among the four features in both countries. Interestingly, Turkish people valued data protection more than Dutch respondents (OR = 9.391, β =2.29, 95% CI [7.73, 12.60], P < 0.01 in the Turkish sample; OR = 6.097, β = 1.80, 95% CI [4.58, 7.98], P < 0.01 in the Dutch sample). Apps produced by the government, compared to apps developed by companies, were also positively valued by participants in the two samples, with an OR of 2.607 in the Netherlands (β = 0.97, 95% CI [2.01, 3.53], P < 0.01) and an OR of 1.410 in Turkey (β = 0.29, 95% CI [1.04, 1.73], P = 0.02). The coefficients associated to the reward scheme are small in magnitude in both samples (OR = 1.170, β = 0.14, 95% CI [0.87, 1.51], P < 0.01 in the Dutch sample and OR = 1.25, β = 0.23, 95% CI [0.98, 1.62], P = 0.01 in the Turkish sample). Last, we find that participants in the Netherlands negatively value the presence of a gaming element (OR = 0.934, β = −0.29, 95% CI [0.57, 0.98], P < 0.01). In contrast, the OR associated to the gaming feature is larger than 1 and statistically significant in the Turkish sample (OR = 1.227, β = 0.13, 95% CI [0.89, 1.45], P < 0.01).

Discussion

High levels of adoption and usage of the CTA's among populations worldwide is a stipulated prerequisite for the success of the implementation of these apps to mitigate the pandemic and track spreading of the virus more efficiently and (cost-)effectively (Wienert and Zeeb, 2021). In the current study, we investigated several individuals' preferences in the intention to download of a CTA in a pilot study in both the Netherlands and Turkey. The results clearly showed that data protection is one of the most important factors that significantly increase the probability to adopt a CTA. Accordingly, we can assume that the voluntary intention to download of CTAs is highly dependent on the clear and transparent information and communication strategies about how data are processed during the contact tracing, where and how long the data are stored, and the way of anonymization.

Furthermore, the respondents prefer in both countries CTAs produced by the government rather than by industries. As mentioned above there are obviously two competing concerns of the public. On the one hand, governments are more trusted in the context of patients' interests, while it is suspected that industries' engagement is more driven by profit interest (Olsen and Whalen, 2009; Bauchner and Fontanarosa, 2013). On the other hand, the public is more likely to trust businesses by being the guardian of information quality (Edelman Trust Barometer, 2021). The result of our experiment reveals the respondents' priority for patients' interests which confirms our expectation mentioned above. In general, reward or gaming affected the probability to download the COVID-19 tracing app less, which is also found in other studies (Peng et al., 2016).

CTAs are highly promising to reduce the spread of the virus and make it easier to open up society faster, especially because they can be used quickly and share information rapidly. Health authorities worldwide have generally released high-quality CTAs, although scientific studies assessing the most important factors that describe and predict the intention to download is limited. Sensitive personal data is collected through these apps, and may potentially threaten privacy, equality, and fairness, which are important attributes to take into account when implementing a CTA as a mitigation strategy.

As mounting evidence has shown, COVID-19 has significant negative effects on human health, as well as on social and economic life, making it extremely important to investigate the effectiveness and efficiency of mitigation strategies that have been implemented by societies worldwide (Codagnone et al., 2020; Bogliacino et al., 2021; Clemente-Suárez et al., 2021; Codagnone et al., 2021). As we now have seen after 2 years of the start of the pandemic, during some periods there are rapidly increasing numbers of infected people in the majority of the countries worldwide making manually collecting the information from the population needs a considerable workforce of health officials trained to hold the interviews effectively, which is a highly costly exercise and has shown to be ineffective in times of high percentages of infections. Considering the discussions raised in several countries when the CTAs were implemented, that 2 years later are still as strong as they were in the beginning, it is important to assess how the population can be motivated to download and use the CTAs to maximize the benefit for society (Anglemyer et al., 2020; Ming et al., 2020; Lee et al., 2021).

One of the limitations of the study is that we have used a convenience sampling, whereby most of the participants have been students. This might influence the external validity of the outcomes of the study. Furthermore, we only conducted the study in two countries, whereas we might find different cultural and contextual differences if we would include more countries in our study. Considering the current study is part of a larger study where we will develop an actual CTA, which we will test in five different countries (Spain, the Netherlands, Finland, Germany and Turkey), we will test in this study for different outcomes between countries. Final, gamification elements might have been judged differently if they were actually displayed or could be tried out in the DCE, which unfortunately was not possible. In the next phase of the study we will test gaming elements in more detail.

Conclusion

Health authorities worldwide have generally released high quality CTA, although scientific studies assessing the most important factors that describe and predict the intention to download is limited. Therefore, a systematic contact tracing is urgently needed to be able to inform citizens about contacts at risk. In addition, the learnings of our study on the intention to use CTA's can also be applied to health apps in general, specially health apps such as Electronic Health Record apps or other digital tools that are used for improving and digitalization of healthcare (Anastasiadou et al., 2018b; Piera-Jiménez et al., 2020a,b, 2021). Sensitive personal data is collected through these apps, and may potentially threaten privacy, equality and fairness, which are important attributes to take into account based on this study. Following ethical definitions, contact tracing should be executed anonymized and voluntary with the highest standard of data and privacy protection. Nonetheless, the intention to download and usage of mobile health apps are rather low. This requires a deeper understanding of decision processes and reasons for rejections among citizens aiming to increase the number of voluntary users of CTAs. As we expected, the respondents in this study prefer apps that ensure data protection. Accordingly, clear communication of data policies is vital for the voluntary usage of CTAs. Additionally, governmental institutions were preferred as publishers of CTAs. This fits the high priority given to data protection issues. Finally, the DCE revealed relatively low importance of reward mechanisms and gaming strategies as motivational factors for CTA the intention to download. Based on the DCE results, qualitative interviews based on personal construct approaches will deliver further details about motivational strategies and voluntariness in the context of CTA usage.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The studies involving human participants were reviewed and approved by Ethical Committee of the Tilburg School of Humanities and Digital Sciences. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

FF, LP, YG, KH, and GS contributed to conception and design of the study. FF, LP, and KH organized the database. GS performed the statistical analysis. FF and LP wrote the first draft of the manuscript. AR-U, SG, SS, AG, EG, GF, AK, and FL wrote sections of the manuscript. All authors contributed to manuscript revision, read, and approved the submitted version.

Funding

This paper is based on research that is executed within the scope of the project- PandeVITA – Pandemic Virus Trace Application for the Effective Knowledge Transfer Between Science and Society Inside the Quadruple Helix Collaboration. The project has received funding from the European Union's Horizon 2020 research and innovation program under grant agreement No. 101006316.

Conflict of Interest

AK is employed by VTT Technical Research Centre of Finland Ltd.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnotes

1. ^See for more information: https://www.pandevita.eu/.

2. ^This study was part of a larger project, whereby multiple experiments were conducted. After participants finished the discrete choice experiment they also participated in a separate experiment that is reported in a separate article.

References

Anastasiadou, D., Folkvord, F., and Lupiañez-Villanueva, F. (2018a). A systematic review of mHealth interventions for the support of eating disorders. Eur. Eat. Disord. Rev. 26, 394–416. doi: 10.1002/erv.2609

PubMed Abstract | CrossRef Full Text | Google Scholar

Anastasiadou, D., Folkvord, F., Serrano-Troncoso, E., and Lupiañez-Villanueva, F. (2018b). ’mHealth adoption in mental health: User experience of a mHealth application for patients with an eating disorder. JMIR mHealth uHealth 7. doi: 10.2196/preprints.12920

CrossRef Full Text | Google Scholar

Anglemyer, A., Moore, T. H., Parker, L., Chambers, T., Grady, A., Chiu, K., et al. (2020). Digital contact tracing technologies in epidemics: a rapid review. Cochrane Database Syst. Rev. 8, CD013699 doi: 10.1002/14651858.CD013699

PubMed Abstract | CrossRef Full Text | Google Scholar

Bassi, A., Arfin, S., John, O., and Jha, V. (2020). An overview of mobile applications (apps) to support the coronavirus disease 2019 response in India. Indian J. Med. Res. 151, 468. doi: 10.4103/ijmr.IJMR_1200_20

PubMed Abstract | CrossRef Full Text | Google Scholar

Bauchner, H., and Fontanarosa, P. B. (2013). Restoring confidence in the pharmaceutical industry. JAMA 309, 607–609. doi: 10.1001/jama.2013.58

PubMed Abstract | CrossRef Full Text | Google Scholar

Bogliacino, F., Codagnone, C., Montealegre, F., Folkvord, F., Gómez, C., Charris, R., et al. (2021). Negative shocks predict change in cognitive function and preferences: assessing the negative affect and stress hypothesis. Sci. Rep. 11, 1–10. doi: 10.1038/s41598-021-83089-0

PubMed Abstract | CrossRef Full Text | Google Scholar

Bostan, S., Erdem, R., Öztürk, Y. E., Kiliç, T., and Yilmaz, A. (2020). The Effect of COVID-19 Pandemic on the Turkish Society. Electron. J. Gen. Med. 17. doi: 10.29333/ejgm/7944

CrossRef Full Text | Google Scholar

Clemente-Suárez, V. J., Navarro-Jiménez, E., Moreno-Luna, L., Saavedra-Serrano, M. C., Jimenez, M., Simón, J. A., et al. (2021). The impact of the COVID-19 pandemic on social, health, and economy. Sustainability. 13, 6314. doi: 10.3390/su13116314

CrossRef Full Text | Google Scholar

Codagnone, C., Bogliacino, F., Gómez, C., Charris, R., Montealegre, F., Liva, G., et al. (2020). Assessing concerns for the economic consequence of the COVID-19 response and mental health problems associated with economic vulnerability and negative economic shock in Italy, Spain, and the United Kingdom. PloS ONE 15, e0240876. doi: 10.1371/journal.pone.0240876

PubMed Abstract | CrossRef Full Text | Google Scholar

Codagnone, C., Bogliacino, F., Gómez, C., Folkvord, F., Liva, G., Charris, R., et al. (2021). Restarting “normal” life after COVID-19 and the lockdown: Evidence from Spain, the United Kingdom, and Italy. Soc. Indic. Res. 158, 241–265. doi: 10.1007/s11205-021-02697-5

PubMed Abstract | CrossRef Full Text | Google Scholar

Edelman Trust Barometer. (2021). Available online at: https://www.edelman.com/sites/g/files/aatuss191/files/2021-03/2021%20Edelman%20Trust%20Barometer.pdf (accessed February 20, 2022).

Gallego, A., Gaeta, E., Karinsalo, A., Ollikainen, V., Koskela, P., Peschke, L., et al. (2021). “Human computer interaction challenges in designing pandemic trace application for the effective knowledge transfer between science and society inside the quadruple helix collaboration,” in International Conference on Human-Computer Interaction. (Cham: Springer), 390–401.

Google Scholar

Jansen-Kosterink, S., Hurmuz, M., den Ouden, M., and van Velsen, L. (2021). Predictors to use mobile apps for monitoring COVID-19 symptoms and contact tracing: survey among dutch citizens. JMIR Form. Res. 5, e28416. doi: 10.2196/28416

PubMed Abstract | CrossRef Full Text | Google Scholar

Klar, R., and Lanzerath, D. (2020). The ethics of COVID-19 tracking apps–challenges and voluntariness. Res. Ethics 16, 1–9. doi: 10.1177/1747016120943622

CrossRef Full Text | Google Scholar

Lancsar, E., and Louviere, J. (2008). Conducting discrete choice experiments to inform healthcare decision making. Pharmacoeconomics 26, 661–677. doi: 10.2165/00019053-200826080-00004

PubMed Abstract | CrossRef Full Text | Google Scholar

Lee, B., Ibrahim, S. A., and Zhang, T. (2021). Mobile apps leveraged in the COVID-19 pandemic in east and south-east asia: review and content analysis. JMIR mHealth uHealth 9, e32093. doi: 10.2196/32093

PubMed Abstract | CrossRef Full Text | Google Scholar

Ming, L. C., Untong, N., Aliudin, N. A., Osili, N., Kifli, N., Tan, C. S., et al. (2020). Mobile health apps on COVID-19 launched in the early days of the pandemic: content analysis and review. JMIR mHealth uHealth 8, e19796. doi: 10.2196/19796

PubMed Abstract | CrossRef Full Text | Google Scholar

Morley, J., Cowls, J., Taddeo, M., and Floridi, L. (2020). Ethical guidelines for COVID-19 tracing apps. Nature 582, 29–31. doi: 10.1038/d41586-020-01578-0

PubMed Abstract | CrossRef Full Text | Google Scholar

Ng, J. Y., Ntoumanis, N., Thøgersen-Ntoumani, C., Deci, E. L., Ryan, R. M., Duda, J. L., et al. (2012). Self-determination theory applied to health contexts: a meta-analysis. Perspect. Psychol. Sci. 7, 325–340. doi: 10.1177/1745691612447309

PubMed Abstract | CrossRef Full Text | Google Scholar

Olsen, A. K., and Whalen, M. D. (2009). Public perceptions of the pharmaceutical industry and drug safety. Drug Saf. 32, 805–810. doi: 10.2165/11316620-000000000-00000

PubMed Abstract | CrossRef Full Text | Google Scholar

Peng, W., Kanthawala, S., Yuan, S., and Hussain, S. A. (2016). A qualitative study of user perceptions of mobile health apps. BMC Public Health 16, 1–11. doi: 10.1186/s12889-016-3808-0

PubMed Abstract | CrossRef Full Text | Google Scholar

Peschke, L., Çataloğlu, B., Ceylan, N., Dionio, P., Dündar, I., Folkvord, F., et al. (2021). Review about good practices of quadruple helix collaboration processes and developments of COVID-19 tracking apps and platforms. Available online at: https://www.pandevita.eu/images/D3-1_GoodPractices-CTAs.pdf

Peschke, L., Peschke, S. G., Ağca, Y. G., Seyfafjehi, S., Dündar, I., and Aydoğdu, Y. (2022). Reward mechanisms in COVID-19 tracking apps and its impact on the voluntary participation of the public in sustainable innovation processes. Turkish Rev. Commun. Stud. 39, 54–72. doi: 10.17829/turcom.1019006

CrossRef Full Text | Google Scholar

Piera-Jiménez, J., Daugbjerg, S., Stafylas, P., Meyer, I., Müller, S., Lewis, L., et al. (2020b). BeyondSilos, a telehealth-enhanced integrated care model in the domiciliary setting for older patients: observational prospective cohort study for effectiveness and cost-effectiveness assessments. JMIR Med. Inf. 8, e20938. doi: 10.2196/20938

PubMed Abstract | CrossRef Full Text | Google Scholar

Piera-Jiménez, J., Etzelmueller, A., Kolovos, S., Folkvord, F., and Lupiáñez-Villanueva, F. (2021). Guided internet-based cognitive behavioral therapy for depression: implementation cost-effectiveness study. J. Med. Intern. Res. 23, e27410. doi: 10.2196/27410

PubMed Abstract | CrossRef Full Text | Google Scholar

Piera-Jiménez, J., Winters, M., Broers, E., Valero-Bover, D., Habibovic, M., Widdershoven, J. W., et al. (2020a). Changing the health behavior of patients with cardiovascular disease through an electronic health intervention in three different countries: cost-effectiveness study in the Do Cardiac Health: Advanced New Generation Ecosystem (Do CHANGE) 2 randomized controlled trial. J. Med. Intern. Res. 22, e17351. doi: 10.2196/17351

PubMed Abstract | CrossRef Full Text | Google Scholar

Tanca, D., Aydo,g, E., Murphy, A., and Zinciroglu, Ö. (2020). The Political and Economic Impact of the Coronavirus Pandemic in Turkey. Istanbul: Edam.

Google Scholar

Trang, S., Trenz, M., Weiger, W. H., Tarafdar, M., and Cheung, C. M. (2020). One app to trace them all? Examining app specifications for mass acceptance of contact-tracing apps. Eur. J. Inf. Syst. 29, 415–428. doi: 10.1080/0960085X.2020.1784046

CrossRef Full Text | Google Scholar

Tsvyatkova, D., Buckley, J., Beecham, S., Chochlov, M., O'Keeffe, I. R., Razzaq, A., and COVIGILANT Group. (2022). Digital contact tracing apps for COVID-19: development of a citizen-centered evaluation framework. JMIR mHealth uHealth 10, e30691. doi: 10.2196/30691

PubMed Abstract | CrossRef Full Text | Google Scholar

Walrave, M., Waeterloos, C., and Ponnet, K. (2020). Adoption of a contact tracing app for containing COVID-19: A health belief model approach. JMIR Public Health surveill. 6, e20572. doi: 10.2196/20572

PubMed Abstract | CrossRef Full Text | Google Scholar

Wienert, J., and Zeeb, H. (2021). Implementing health apps for digital public health–an implementation science approach adopting the consolidated framework for implementation research. Front. Public Health 9, 347. doi: 10.3389/fpubh.2021.610237

PubMed Abstract | CrossRef Full Text | Google Scholar

Zakharchenko, O., Avramenko, R., Zakharchenko, A., Korobchuk, A., Fedushko, S., Syerov, Y., et al. (2022). Multifaceted nature of social media content propagating COVID-19 vaccine hesitancy: ukrainian case. Procedia Comput. Sci. 198, 682–687. doi: 10.1016/j.procs.2021.12.306

PubMed Abstract | CrossRef Full Text | Google Scholar