Alberto E. Tozzi1
Francesco Gesualdo1*
Angelo D’Ambrosio1
Elisabetta Pandolfi1
Eleonora Agricola1
Pierluigi Lopalco2†
- 1Unit of Telemedicine, IRCCS, Bambino Gesù Children’s Hospital, Rome, Italy
- 2European Centre for Disease Prevention and Control, Stockholm, Sweden
In order to successfully control and eliminate vaccine-preventable infectious diseases, an appropriate vaccine coverage has to be achieved and maintained. This task requires a high level of effort as it may be compromised by a number of barriers. Public health agencies have issued specific recommendations to address these barriers and therefore improve immunization programs. In the present review, we characterize issues and challenges of immunization programs for which digital tools are a potential solution. In particular, we explore previously published research on the use of digital tools in the following vaccine-related areas: immunization registries, dose tracking, and decision support systems; vaccine-preventable diseases surveillance; surveillance of adverse events following immunizations; vaccine confidence monitoring; and delivery of information on vaccines to the public. Subsequently, we analyze the limits of the use of digital tools in such contexts and envision future possibilities and challenges.
Introduction
Maintaining a high performance of an immunization program is one of the most challenging public health objectives. The history of immunization has shown several cases of success at a global level, including the eradication of smallpox and the control or elimination of several other vaccine-preventable diseases (1–3). Significant efforts and resources are constantly dedicated to supporting, maintaining, and improving immunization strategies, in order to achieve the goals set by national and international health agencies (2).
Several studies have been conducted to identify actions associated with an improvement of the vaccination coverage (4). In this regard, a collection of evidence-based recommendations has been issued by The Community Preventive Services Task Force (5).
Nevertheless, at a global level, a number of issues have become real threats for maintaining high immunization coverage, running an effective surveillance, and allowing immunization programs to timely react to new issues.
A large number of these issues could benefit from the adoption of digital tools (see Table 1). The objective of this review is to explore the role of potential digital solutions to problems of vaccination programs.
Table 1. Potential uses of digital tools in immunization programs.
Data Collection and Management
In most countries, especially in the developing world, the logistics of vaccination systems are paper-based, thus limiting timely update and information accessibility. Limited accessibility to vaccination information has a crucial impact on vaccination strategies, which cannot be timely and comprehensively informed by data. Monitoring vaccine safety and effectiveness can also be affected by lack of data accessibility. Moreover, in several countries, data on vaccinations are stored at the local level; therefore, citizens may have difficulties in accessing their vaccination record.
Regarding vaccination logistics, paper-based dose accountability has clear limits regarding safety and timely administration of doses according to schedules.
Digital solutions for data collection and management may streamline vaccination activities and provide important information to tailor immunization programs.
Immunization Registries
Immunization information systems (IISs) are confidential, population-based, computerized databases designed to record all immunization doses administered to a population, providing health operators with tools for maintaining a high vaccination coverage (6). IIS programs should provide solutions for (a) identification of at-risk individuals and groups; (b) management, storage, and integration of immunization data; (c) data protection; (d) facilitation in the engagement of families and individuals for timely vaccination receipt; (e) clinical decision support for health providers; and (f) framework for data sharing among health providers, at regional, national, and international levels (6).
Though several efforts have been made in some countries for adopting IIS, their use is far from being universal.
In Europe, only Denmark, Iceland, Malta, the Netherlands, and Norway have a national, fully implemented digital IIS, while six other countries have subnational IISs (7). Among the systems adopted by these countries, there is a high variability in methods, frequency of data acquisition, geographic coverage, data storage, and distribution (7, 8).
Initial efforts are being made to build digital registries at the international level, in order to facilitate global surveillance of vaccination programs and sharing of good practices. A system with such characteristics has been implemented by WHO and UNICEF: the Centralized Information System for Infectious Diseases (8).
Since 1998, Canada has had a network of regional IIS registries, which have been integrated in a national, intercommunicating network since the SARS pandemic in 2004 (9).
In the US, the CDC is working closely with electronic health record (EHR) providers to define the set of data and functionalities EHRs should have in order to be useful in the immunization setting, and to adopt a communication standard in order to allow consistent data intercommunication between health-care points and the CDC itself (10). Moreover, the American Academy of Pediatrics (AAP) is developing guidelines for tailoring immunization records to children, including parent refusal, information interchange with IISs, and clinical decision support functionalities (11).
Dose Tracking
Vaccine dose accountability may also benefit from digital solutions. The CDC vaccine tracking system (VTrckS) allows web-based ordering and tracking of publicly funded vaccines (12). The use of two-dimensional barcodes to store vaccine information (vaccine product identification, expiration date, and lot number) has been allowed by the FDA in 2011 (13, 14), is being experimented in a pilot study by the AAP and CDC (15, 16), and is already applied in Canada (17) and in Spain (13).
Integrating barcode scanning technology in EHRs has clear advantages in reducing errors and increasing safety (18, 19). In EHRs, an automatic link to vaccine information could also allow to easily tracking vaccine lots in case of adverse events following immunization (AEFI) (20).
Decision Support Systems
Clinical decision support systems help health professionals to correctly manage immunizations through the following functionalities: they suggest the appropriate immunization schedule for children based on birth date and vaccine history (21), automatically integrate changes in regulations (22, 23), proactively remind physicians of vaccinations for their patients (24), automatically recognize AEFIs (11), and suggest tailored immunizations in at-risk groups (25).
Personal health records (PHR) managed by patients and families can also improve adherence to immunization schedules through automatic notifications (26). A study performed on an adult population showed that the use of immunization PHRs correlates to a higher chance of receiving influenza immunization (27).
Developing Countries
Due to the lack of infrastructures and to high costs, adopting IISs in developing countries is a challenging task. Nevertheless, information systems enabling digital recording and transmission of immunization data are being implemented in Guatemala (28) and South Africa (29).
On the other hand, based on the observation that mobile phones are widely used in developing countries, mobile-based approaches may be promising in such contexts. For example, mobile technologies and advanced algorithms are being used to digitalize old paper-based immunization registries in low resource settings, e.g., Mozambique (30).
In Haiti, a cholera vaccination campaign has been carried out through house-by-house visits by operators equipped with wireless tablets. Children’s immunization status was assessed and recorded using a family-specific bar code; data were geolocalized and sent to a central system, which provided the program staff with a real-time map of vaccination coverage (31). A similar approach has been used in China, with a mobile app for facilitating immunization data recording, tracking unimmunized children, appointment booking (32) and in Thailand, with an app for recording data during antenatal and immunization visits (33).
Tools for Vaccine-Preventable Diseases Surveillance
Traditionally, surveillance is defined as “the systematic collection, consolidation, analysis, and dissemination of data on specific diseases” (34). Classically, the main outputs of traditional surveillance systems have been indicators focused on individuals (34). More recently, surveillance activities have been aiming at rapidly capturing information about events that may represent a threat for public health and are referred to as event-based surveillance. Epidemiological surveillance is therefore extending from individual-based to event-based data (34, 35).
Traditional surveillance systems have a number of limitations:
- information is collected through health-care providers, not directly from individuals; therefore, traditional surveillance systems fail to catch signals from sick people who do not go to the doctor;
- traditional systems are based on case definitions, and therefore may miss emerging diseases with unexpected combinations of symptoms;
- there is a consistent time lag between signals of disease and production and dissemination of aggregated incidence figures.
In the context of event-based surveillance, information can be collected from news, reports, or other sources transmitted both through institutional and informal channels. Web-based data have been used to support public health in Canada since the 1990s, with the Global Public Health Intelligence Network (36), a service that automatically retrieves information about potential public health emergencies from news feed aggregators and distributes this information to public health agencies, including the WHO Global Outbreak Alert and Response Network (37).
Several studies explored the use and interpretation of spontaneous digital traces left by Internet users as a convenient and timely strategy to detect signals of trend variations in diseases and immunizations. The idea of using digital traces for syndromic surveillance was proposed by Eysenbach, who tracked demand for health information on the Internet using keyword-triggered ads for influenza (38). Subsequently, on the basis of the hypothesis that the interest of web users may correlate to disease incidence, a number of studies have focused on measuring the occurrence of specific health-related and disease-related search keywords.
Search volumes on web search engines may represent a surrogate of frequency of specific health events. A correlation between search volumes and disease trends has been shown (39). Search queries were useful to track dengue activity (40), and one study showed a correlation between search terms and laboratory confirmed cases of rotavirus infections (41). In 2008, a Google service (Google Flu Trends) has been developed to estimate and predict influenza activity by aggregating Google search query volumes (42). One study investigated the possibility of applying this approach to vaccinations, showing that search activity for HPV and H1N1 correlated to immunization coverage (43).
This “demand based” approach for surveillance has been subsequently integrated with the “supply based” approach, investigating communication contents and patterns in discussion groups, blogs, and microblogs (38), thus focusing on what users say, rather than on what users search for.
In particular, Twitter, a social network based on the sharing of short messages (up to 140 characters), which are available to the public without restrictions, has often been investigated as a source of information for infectious disease surveillance. Twitter posts are rich in data, allowing to follow disease trends both temporally and geographically. Many studies have explored whether monitoring information flow and networks on Twitter could help following the emergence of health conditions, their evolution, and the public’s interest around them. Influenza surveillance has been one of the main topics in Twitter research (44), with international (45) and local scale studies (46, 47). This approach has been recently used to study the incidence trends of other infectious diseases, namely, pertussis (47), dengue (48), and cholera (49).
On the other hand, vaccine-preventable disease surveillance may benefit from systems based on an active input of information by users. Such methods, grouped under the definition of “participatory surveillance,” are based on platforms (both web- and smartphone-based) that allow users to directly provide information about their health status. Typical examples of this kind of surveillance are platforms dedicated to crowdsourced influenza surveillance (e.g., Flu Near You or Influnet) (50–52), providing a powerful and precise tool for epidemic assessment.
Complex biosurveillance systems aggregate data from a variety sources: news sites, social media, crowdsourcing platforms, official resources (e.g., WHO), audio, and video sources. Information acquired by such platforms may provide geographical details both at the local and at the international level (53). One of the best examples of Internet biosurveillance systems is represented by Healthmap, which provides information about emerging and re-emerging public health threats by aggregating information from various structured and non-structured data sources (54).
GeoChat (55) is another example of an open source platform that can enable the easy deployment of crowdsourced interactive mapping applications for surveillance with web forms/e-mail, short message service (SMS), and Twitter support. This application has been used in Cambodia for disease reporting and outbreak alerts (56).
Epidemic intelligence activities can also integrate traditional surveillance systems for the detection of vaccine failure and lack of effectiveness. Detection of outbreak clusters in vaccinated populations may trigger a signal of potential vaccine failure. Such signals must be verified and properly assessed using “traditional” epidemiological techniques in order to get estimates of vaccine effectiveness and either confirm or reject the signal (57).
Tools for Surveillance of Adverse Events Following Immunizations
Assessment of vaccine safety is a priority for public health and may have a powerful impact on the success of an immunization program. AEFI may be studied before marketing authorization, in the context of phase I-III clinical trials, which, though, may not have a sufficient magnitude for adequately detecting rare AEFIs. Subsequently to licensure, AEFIs are monitored through ad hoc, formal studies, or, more frequently, through passive surveillance by health-care workers. Limits of such systems are under-reporting and biased reporting (8).
Innovative ways to monitor AEFIs and capture signals of vaccine safety have been explored extensively (58).
In particular, large databases of EHRs have been used to set up systems able to capture signals in a semi-automated way. The US Vaccine Safety Datalink (VSD) is the oldest of such infrastructures, established by the CDC in 1990 for population-based, post-marketing monitoring of vaccine safety (59). The VSD project includes a population of nearly 9 million individuals, including 2.1 million children, attending various participating organizations. In addition, the FDA has recently established the Post-Licensure Rapid Immunization Safety Monitoring (PRISM) system, built on the VSD model, linking information from different health insurance databases (60). Such systems are extremely efficient in addressing specific questions related to vaccine safety in a timely manner. On the other hand, they are not specifically designed to detect safety signals.
Clusters of mass psychogenic illness following pandemic influenza vaccination were detected and investigated in Taiwan in 2009 using a combination of institutional data sources (the passive AEFI surveillance system and the web-based Emergency Medical Management System) (61). Association between narcolepsy and H1N1 pandemic vaccine Pandemrix in Finland was initially flagged up by an informal digital network of neurologists and was afterward confirmed by epidemiological studies (62). The same signal would have been hardly detected through the routine AEFI passive reporting system, being an unexpected and rare event.
Surveillance of AEFIs through a participatory approach has been a matter of debate. In many AEFI surveillance systems, patients do report health events. Post-marketing surveillance of AEFI has been implemented through mobile devices (63) and SMS (64).
Monitoring Vaccine Confidence
The media resonate with uncontrolled, scaring information on vaccine safety, and the general public is exposed to conflicting information on the balance between risks and benefits of immunizations. As a matter of fact, vaccine hesitancy, due to lack of confidence in vaccinations, may have a crucial impact on vaccine uptake (65, 66).
Vaccine confidence is traditionally measured through classic surveys or interviews (67, 68).
As information regarding vaccinations is largely acquired on the Internet, web data mining may enlighten various aspects of vaccine confidence. Parents seeking vaccine information on the Internet, compared to those using other information sources, are less likely to agree with accepted principles of vaccine science and less likely to recognize the benefits of vaccinations (69). Side effects, ingredients, and immunization policies are the most searched and discussed topics on the web (70–72).
Web monitoring of vaccine confidence may allow to timely intercept negative trends and, therefore, to set up immediate actions. This objective may be achieved by collection and interpretation of heterogeneous data on vaccines derived from various web sources (73), including social networks (74).
Delivering Information on Immunizations to the Public
Vaccine information campaigns are mostly based on paper material and delivered through traditional media channels. This approach has several limitations. First, a large time lag may exist between the detection of information needs and information delivery. Second, the tailoring potential of this approach is limited, as information campaigns usually target the general population.
Digital tools and new media can be exploited as means for accurately identifying information needs and effectively delivering vaccination campaigns. A number of studies have investigated the presence of vaccine information on the web, often starting from an analysis of Google search results obtained through the use of specific keywords (72, 75–78). This approach can be useful for addressing information trends and gaps on available web pages.
Information on vaccinations published on the web, as well as most common information gaps and misconceptions, have often been assessed through an analysis of vaccine-related websites. A number of studies have analyzed both websites promoting vaccinations and anti-vaccination websites (79–81). To this regard, the World Health Organization conducts the Vaccine Safety Network, an initiative aimed at crediting web sites meeting certain quality criteria, which are periodically reviewed (82).
For a complete assessment of information trends on vaccinations, social networks cannot be neglected, as they are a fertile medium for anti-vaccine sentiments (83). Blogs commonly support variable positions on vaccinations (84); nevertheless, negative messages may have a greater influence on the decision to vaccinate, compared to pro-vaccine messages (85). Twitter has been studied as a source of information on vaccines (86), with pro-vaccine contents being more prevalent on Twitter compared to other social media (87). Moreover, the use of social media and, specifically, of Facebook, seems promising for the implementation of communication strategies targeting adolescents, who are particularly engaged in this kind of media. In this regard, a number of studies investigated patients’ enrollment and HPV vaccination promotion on Facebook (88–91).
Recently, some attempts have been conducted to pilot tailored communication toward vaccine-hesitant parents (92, 93). A successful tailored education on HPV vaccine has been experienced by Chinese health professionals who collaborated with popular websites for women to gear up vaccination uptake (94).
A large part of the literature on digital tools aimed at improving immunization uptake focuses on electronic reminders. Reminders can be included in more complex interventions to improve immunization coverage (95). Several authors have documented the impact of text messaging and computerized reminders in increasing adherence to schedules, especially if integrated with educational interventions (27, 96–99). On the other hand, the evidence supporting these interventions, as for others digital tools for public health, seems insufficient (24).
Comments and Discussion
Only small progresses have been made to integrate digital tools into immunization programs. Moreover, research aimed at assessing the effectiveness of digital technologies as potential responses to problems of immunization programs is scarce.
Electronic data management is not fully applied to traditional components of immunization programs yet. Electronic data on vaccinations should be integrated at the international level, in order to facilitate cross collaboration and coordination among health services and research institutes. Moreover, easy access to large amounts of immunization records may give a robust contribution to vaccine research, facilitating the design of timely and inexpensive epidemiological studies to assess vaccine efficacy and safety (9).
With regard to surveillance, crowdsourcing, mobile phones, personal computing devices, and geolocalization of information promise to become stable pillars of public health strategies. Combining these resources with other tools, such as SMS and social networks, may generate innovative instruments to support surveillance of health events and other public health activities (56). This observation extends to monitoring vaccine coverage and AEFI occurrence, where social media can enhance traditional information sources.
The analysis of search queries and social media information is a powerful approach, although it may suffer from information biases, as interest in specific diseases may be amplified by media attention, independently from the disease incidence (100, 101). Nevertheless, refined algorithms and the use of text mining techniques for sense disambiguation, topic filtering, and mood analysis (102, 103) are allowing to isolate actual signals of disease from information noise with increasing precision. The use of natural language processing techniques and of algorithms allowing an automatic or semi-automatic classification of contents may also allow to overcome the manual evaluation of contents, which is an expensive and energy-consuming task. One step toward a more accurate surveillance may be represented by participatory surveillance. A recent review showed a large potential benefit of participatory surveillance, although specificity may be limited and participating bias can affect its performance (104).
Parallel to surveillance of medical events, the use of digital tools for monitoring vaccine confidence and information needs may greatly enhance the performance of immunization programs. Vaccine hesitancy and vaccine opposition are increasingly worrying phenomena. The detection of a drop in vaccine confidence may anticipate a more severe event, such as a decrease in vaccine coverage, allowing the immunization program to react. Monitoring information needs may allow to identify issues which are potentially linked with vaccine hesitancy or opposition. Moreover, data gathering and analysis of information found on the web has the advantage to shortcut traditional channels and to allow continuous monitoring and interpretation of data (73). An interesting approach for investigating information needs may be represented by the analysis of search behaviors through the study of search terms and queries, which are commonly used by Internet users (105, 106), using specific platforms such as Google Adwords.
On the other hand, monitoring vaccine confidence and information needs on the Internet does not allow to consider segments of the public that do not have access to the Internet or that do not leave digital traces on the web. Therefore, information obtained from the Internet is not sufficient to describe the entire population and should be used to integrate other traditional sources of information. Nevertheless, web-derived information may allow to finely profile specific populations of Internet users, which may become the target of tailored immunization campaigns delivered through multiple web channels.
Business marketing strategies are based on rules for translating information delivery into monetary return of investments. The same approach may be used to rapidly spread vaccine information to different segments of the public to maintain vaccine confidence and increase immunization coverage. Information strategies should exploit more comprehensively the speed and pervasiveness of digital tools. The web community will likely become increasingly demanding, switching from a passive acceptance of static, limited contents to an active request of detailed information. Vaccine information delivery should therefore move from classical, static web sites to real interactions with the users, especially through the use of social networks. The future scenarios of immunization policies will possibly be characterized by a direct participation of the public in designing appropriate information strategies and even efficacious approaches to maintaining immunization programs.
All the potentials hitherto described can be greatly enhanced when the tool used for interaction is a mobile phone. Indeed, people who access the Internet through a smartphone rather than a computer are more likely to interact with the technology than simply consuming information (107). Apps dedicated to vaccination might be highly impacted in this population group. On the other hand, one of the issues in using electronic reminders based on cellular phone may be that the global penetration of these devices is not universal (108).
At present, a number of barriers limit the adoption of digital tools in immunization programs. First, ability to use digital tools may be hampered by cultural background and infrastructure availability. This implies an urgent need for educational activities aimed at empowering health professionals and patients for the use of such tools. Institutional stakeholders should drive political decisions toward the use of digital tools both in research and clinical activities. Second, the adoption of digital tools requires a relatively large, initial investment in human and financial resources. This may represent a limit, in particular, in developing countries or in countries suffering from financial crisis. Nevertheless, the return of such investments is high in terms of increased quality of immunization programs and subsequent cost savings. Third, the use of Internet for managing health data is subject to security and privacy issues. Strategies to maintain anonymity and preserve confidentiality are difficult to implement. Moreover, relying on proprietary resources, such as those offered by Google, may be problematic, since algorithms used by this company are not explicit (109).
Research studies assessing the impact of digital tools in immunization programs are still rare and do not follow the rapid pace of development of technology and digital tools. This discrepancy is common to other domains and should be rapidly filled up to improve immunization programs.
In conclusion, despite digital tools may greatly enhance the effectiveness of immunization programs, only few examples of implementation are available at present. The use of digital tools can favor the intersection of three crucial dimensions of immunization programs: immunization registries, surveillance of vaccine-preventable diseases, and surveillance of AEFI. The fourth dimension, represented by monitoring confidence in immunization programs, can be easily integrated through the use of digital instruments, which would allow implementation of data-driven vaccine information strategies. An expanded use of digital tools is expected to ultimately increase immunization coverage, reduce vaccine-preventable disease incidence, reduce AEFIs, and increase the active participation of the public to immunization strategies through informed decisions.
Author Contributions
AET, FG and AED conceived the project and wrote the article. EP and EA performed the literature review. PL critically revised the article and gave the final approval for publication.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
The Reviewer SW-C and handling Editor PA declared their shared affiliation, and the handling Editor states that the process nevertheless met the standards of a fair and objective review.
References
1. World Health Organization. The Global Eradication of Smallpox: Final Report of the Global Commission for the Certification of Smallpox Eradication, Geneva, December 1979. Geneva: World Health Organization (1980).
2. World Health Organization. Global Vaccine Action Plan 2011–2020. Geneva: World Health Organization (2013).
3. Global polio eradication initiative: 10th meeting of the Independent Monitoring Board. Wkly Epidemiol Rec (2014) 89:361–7.
4. Briss PA, Rodewald LE, Hinman AR, Shefer AM, Strikas RA, Bernier RR, et al. Reviews of evidence regarding interventions to improve vaccination coverage in children, adolescents, and adults. The Task Force on Community Preventive Services. Am J Prev Med (2000) 18:97–140. doi: 10.1016/S0749-3797(99)00118-X
5. Guide to Community Preventive Services. Increasing Appropriate Vaccination (2015). Available from: www.thecommunityguide.org/vaccines/index.html
6. Centers for Disease Control and Prevention (CDC). Increasing Appropriate Vaccination (2012). Available from: http://www.cdc.gov/vaccines/programs/iis/about.html
7. VENICE II Consortium. Vaccination Coverage Assessment in EU/EEA, 2011 (2012). Available from: http://venice.cineca.org/Final_Vaccination_Coverage_Assesment_Survey_2011_1.pdf
8. Lopalco PL, Carrillo Santisteve P. Actual immunization coverage throughout Europe: are existing data sufficient? Clin Microbiol Infect (2014) 20(Suppl 5):7–11. doi:10.1111/1469-0691.12453
9. Writing team for the Public Health Agency of Canada/Canadian Institutes of Health Research Influenza Research Network Vaccine Coverage Theme Group. Why collect individual-level vaccination data? CMAJ (2010) 182:273–5. doi:10.1503/cmaj.091515
10. Centers for Disease Control and Prevention (CDC). Progress in immunization information systems – United States, 2012. MMWR Morb Mortal Wkly Rep (2013) 62:1005–8.
11. Spooner SA, Council on Clinical Information Technology, American Academy of Pediatrics. Special requirements of electronic health record systems in pediatrics. Pediatrics (2007) 119:631–7. doi:10.1542/peds.2006-3527
12. Centers for Disease Control and Prevention (CDC). Progress in immunization information systems – United States, 2010. MMWR Morb Mortal Wkly Rep (2012) 61:464–7.
13. Pebody R. Vaccine registers – experiences from Europe and elsewhere. Euro Surveill (2012) 17:20159.
14. O’Connor AC, Kennedy ED, Loomis RJ, Haque SN, Layton CM, Williams WW, et al. Prospective cost-benefit analysis of a two-dimensional barcode for vaccine production, clinical documentation, and public health reporting and tracking. Vaccine (2013) 31:3179–86. doi:10.1016/j.vaccine.2013.04.073
15. American Academy of Pediatrics. Vaccine Barcoding (2013). Available from: http://www2.aap.org/immunization/pediatricians/barcoding.html
16. Centers for Disease Control and Prevention (CDC). Two-Dimensional (2D) Vaccine Barcodes (2015). Available from: http://www.cdc.gov/vaccines/programs/iis/2d-vaccine-barcodes/
17. Pereira JA, Quach S, Hamid JS, Quan SD, Diniz AJ, Van Exan R, et al. The integration of barcode scanning technology into Canadian public health immunization settings. Vaccine (2014) 32:2748–55. doi:10.1016/j.vaccine.2013.11.015
18. Kaushal R, Barker KN, Bates DW. How can information technology improve patient safety and reduce medication errors in children’s health care? Arch Pediatr Adolesc Med (2001) 155:1002–7. doi:10.1001/archpedi.155.9.1002
19. Larrabee S, Brown MM. Recognizing the institutional benefits of bar-code point-of-care technology. Jt Comm J Qual Saf (2003) 29:345–53.
20. Supply and safety issues surrounding an H1N1 vaccine. Lancet (2009) 374:358. doi:10.1016/S0140-6736(09)61395-7
21. Zhu VJ, Grannis SJ, Tu W, Rosenman MB, Downs SM. Evaluation of a clinical decision support algorithm for patient-specific childhood immunization. Artif Intell Med (2012) 56:51–7. doi:10.1016/j.artmed.2012.04.004
22. Miller PL, Frawley SJ, Brandt C, Sayward FG. Tools for immunization guideline knowledge maintenance. II. Automated Web-based generation of user-customized test cases. Comput Biomed Res (1998) 31:190–208. doi:10.1006/cbmr.1998.1471
23. Miller PL, Frawley SJ, Sayward FG. Issues in accommodating national changes and local variation in a computer-based guideline for childhood immunization and in related knowledge maintenance tools. Proc AMIA Symp (1998):563–7.
24. Stockwell MS, Fiks AG. Utilizing health information technology to improve vaccine communication and coverage. Hum Vaccin Immunother (2013) 9:1802–11. doi:10.4161/hv.25031
25. Fiks AG, Hunter KF, Localio AR, Grundmeier RW, Bryant-Stephens T, Luberti AA, et al. Impact of electronic health record-based alerts on influenza vaccination for children with asthma. Pediatrics (2009) 124:159–69. doi:10.1542/peds.2008-2823
26. Tom JO, Mangione-Smith R, Solomon C, Grossman DC. Integrated personal health record use: association with parent-reported care experiences. Pediatrics (2012) 130:e183–90. doi:10.1542/peds.2011-1786
27. Lau AY, Sintchenko V, Crimmins J, Magrabi F, Gallego B, Coiera E. Impact of a web-based personally controlled health management system on influenza vaccination and health services utilization rates: a randomized controlled trial. J Am Med Inform Assoc (2012) 19:719–27. doi:10.1136/amiajnl-2011-000433
28. World Health Organization. An Integrated Approach to Health Information Systems in Guatemala (2015). Available from: http://www.who.int/immunization/programmes_systems/supply_chain/optimize/guatemala/en/
29. Integrated Health Information Systems for Vaccination in Developing Countries (2015). Available from: https://www.idi.ntnu.no/~ericm/gavi.pdf
30. Dell N, Breit N, Chaluco T, Crawford J, Borriello G. Digitizing paper forms with mobile imaging technologies. Proceedings of the 2nd ACM Symposium on Computing for Development. New York (2012).
31. Teng JE, Thomson DR, Lascher JS, Raymond M, Ivers LC. Using mobile health (mHealth) and geospatial mapping technology in a mass campaign for reactive oral cholera vaccination in rural Haiti. PLoS Negl Trop Dis (2014) 8:e3050. doi:10.1371/journal.pntd.0003050
32. Chen L, Wang W, Du X, Rao X, van Velthoven MH, Yang R, et al. Effectiveness of a smart phone app on improving immunization of children in rural Sichuan Province, China: study protocol for a paired cluster randomized controlled trial. BMC Public Health (2014) 14:262. doi:10.1186/1471-2458-14-262
33. Kaewkungwal J, Singhasivanon P, Khamsiriwatchara A, Sawang S, Meankaew P, Wechsart A. Application of smart phone in “Better Border Healthcare Program”: a module for mother and child care. BMC Med Inform Decis Mak (2010) 10:69. doi:10.1186/1472-6947-10-69
34. Langmuir AD. The surveillance of communicable diseases of national importance. N Engl J Med (1963) 268:182–92. doi:10.1056/NEJM196301242680405
35. World Health Organization. A Guide to Establishing Event-Based Surveillance. Manila: WHO Regional Office for the Western Pacific (2008).
36. Mykhalovskiy E, Weir L. The Global Public Health Intelligence Network and early warning outbreak detection: a Canadian contribution to global public health. Can J Public Health (2006) 97:42–4.
37. Heymann DL, Rodier GR, Operational Support WHO. Team to the Global Outbreak Alert and Response Network: hot spots in a wired world: WHO surveillance of emerging and re-emerging infectious diseases. Lancet Infect Dis (2001) 1:345–53. doi:10.1016/S1473-3099(01)00148-7
38. Eysenbach G. Infodemiology and infoveillance: framework for an emerging set of public health informatics methods to analyze search, communication and publication behavior on the Internet. J Med Internet Res (2009) 11:e11. doi:10.2196/jmir.1157
39. Polgreen PM, Chen Y, Pennock DM, Nelson FD. Using internet searches for influenza surveillance. Clin Infect Dis (2008) 47:1443–8. doi:10.1086/593098
40. Althouse BM, Ng YY, Cummings DA. Prediction of dengue incidence using search query surveillance. PLoS Negl Trop Dis (2011) 5:e1258. doi:10.1371/journal.pntd.0001258
41. Desai R, Lopman BA, Shimshoni Y, Harris JP, Patel MM, Parashar UD. Use of Internet search data to monitor impact of rotavirus vaccination in the United States. Clin Infect Dis (2012) 54:e115–8. doi:10.1093/cid/cis121
42. Ginsberg J, Mohebbi MH, Patel RS, Brammer L, Smolinski MS, Brilliant L. Detecting influenza epidemics using search engine query data. Nature (2009) 457:1012–4. doi:10.1038/nature07634
43. Kalichman SC, Kegler C. Vaccine-related internet search activity predicts H1N1 and HPV vaccine coverage: implications for vaccine acceptance. J Health Commun (2015) 20(3):259–65. doi:10.1080/10810730.2013.852274
44. Gesualdo F, Stilo G, Agricola E, Gonfiantini MV, Pandolfi E, Velardi P, et al. Influenza-like illness surveillance on Twitter through automated learning of naive language. PLoS One (2013) 8:e82489. doi:10.1371/journal.pone.0082489
45. Chew C, Eysenbach G. Pandemics in the age of Twitter: content analysis of Tweets during the 2009 H1N1 outbreak. PLoS One (2010) 5:e14118. doi:10.1371/journal.pone.0014118
46. Broniatowski DA, Paul MJ, Dredze M. National and local influenza surveillance through Twitter: an analysis of the 2012-2013 influenza epidemic. PLoS One (2013) 8:e83672. doi:10.1371/journal.pone.0083672
47. Nagar R, Yuan Q, Freifeld CC, Santillana M, Nojima A, Chunara R, et al. A case study of the New York City 2012-2013 influenza season with daily geocoded Twitter data from temporal and spatiotemporal perspectives. J Med Internet Res (2014) 16:e236. doi:10.2196/jmir.3416
48. Gomide J, Veloso A, Meira W Jr, Almeida V, Benevenuto F, Ferraz F, et al. Dengue surveillance based on a computational model of spatio-temporal locality of Twitter. Proceedings of the 3rd International Web Science Conference. New York (2011).
49. Chunara R, Andrews JR, Brownstein JS. Social and news media enable estimation of epidemiological patterns early in the 2010 Haitian cholera outbreak. Am J Trop Med Hyg (2012) 86:39–45. doi:10.4269/ajtmh.2012.11-0597
50. Paolotti D, Carnahan A, Colizza V, Eames K, Edmunds J, Gomes G, et al. Web-based participatory surveillance of infectious diseases: the Influenzanet participatory surveillance experience. Clin Microbiol Infect (2014) 20:17–21. doi:10.1111/1469-0691.12477
51. Cantarelli P, Debin M, Turbelin C, Poletto C, Blanchon T, Falchi A, et al. The representativeness of a European multi-center network for influenza-like-illness participatory surveillance. BMC Public Health (2014) 14:984. doi:10.1186/1471-2458-14-984
52. Flu Near You (2015). Available from: https://flunearyou.org/
53. Hartley DM, Nelson NP, Arthur RR, Barboza P, Collier N, Lightfoot N, et al. An overview of internet biosurveillance. Clin Microbiol Infect (2013) 19:1006–13. doi:10.1111/1469-0691.12273
54. Schwind JS, Wolking DJ, Brownstein JS, PREDICT Consortium1, Mazet JA, Smith WA. Evaluation of local media surveillance for improved disease recognition and monitoring in global hotspot regions. PLoS One (2014) 9:e110236. doi:10.1371/journal.pone.0110236
55. Innovative Support to Emegencies Diseases and Disasters (InSTEDD). Geochat (2015). Available from: http://instedd.org/technologies/geochat/
56. Kamel Boulos MN, Resch B, Crowley DN, Breslin JG, Sohn G, Burtner R, et al. Crowdsourcing, citizen sensing and sensor web technologies for public and environmental health surveillance and crisis management: trends, OGC standards and application examples. Int J Health Geogr (2011) 10:67. doi:10.1186/1476-072x-10-67
57. Yen C, Figueroa JR, Uribe ES, Carmen-Hernandez LD, Tate JE, Parashar UD, et al. Monovalent rotavirus vaccine provides protection against an emerging fully heterotypic G9P[4] rotavirus strain in Mexico. J Infect Dis (2011) 204:783–6. doi:10.1093/infdis/jir390
58. Lopalco PL, DeStefano F. The complementary roles of phase 3 trials and post-licensure surveillance in the evaluation of new vaccines. Vaccine (2014) 33:1541–8. doi:10.1016/j.vaccine.2014.10.047
59. Baggs J, Gee J, Lewis E, Fowler G, Benson P, Lieu T, et al. The Vaccine Safety Datalink: a model for monitoring immunization safety. Pediatrics (2011) 127(Suppl 1):S45–53. doi:10.1542/peds.2010-1722H
60. Yih WK, Lee GM, Lieu TA, Ball R, Kulldorff M, Rett M, et al. Surveillance for adverse events following receipt of pandemic 2009 H1N1 vaccine in the Post-Licensure Rapid Immunization Safety Monitoring (PRISM) System, 2009-2010. Am J Epidemiol (2012) 175:1120–8. doi:10.1093/aje/kws197
61. Huang WT, Hsu CC, Lee PI, Chuang JH. Mass psychogenic illness in nationwide in-school vaccination for pandemic influenza A(H1N1) 2009, Taiwan, November 2009-January 2010. Euro Surveill (2010) 15:19575.
62. Partinen M, Saarenpaa-Heikkila O, Ilveskoski I, Hublin C, Linna M, Olsen P, et al. Increased incidence and clinical picture of childhood narcolepsy following the 2009 H1N1 pandemic vaccination campaign in Finland. PLoS One (2012) 7:e33723. doi:10.1371/journal.pone.0033723
63. Cashman P, Moberley S, Dalton C, Stephenson J, Elvidge E, Butler M, et al. Vaxtracker: active on-line surveillance for adverse events following inactivated influenza vaccine in children. Vaccine (2014) 32:5503–8. doi:10.1016/j.vaccine.2014.07.061
64. Baron S, Goutard F, Nguon K, Tarantola A. Use of a text message-based pharmacovigilance tool in Cambodia: pilot study. J Med Internet Res (2013) 15:e68. doi:10.2196/jmir.2477
65. Atwell JE, Salmon DA. Pertussis resurgence and vaccine uptake: implications for reducing vaccine hesitancy. Pediatrics (2014) 134:602–4. doi:10.1542/peds.2014-1883
66. Roberts JR, Thompson D, Rogacki B, Hale JJ, Jacobson RM, Opel DJ, et al. Vaccine hesitancy among parents of adolescents and its association with vaccine uptake. Vaccine (2015) 33:1748–55. doi:10.1016/j.vaccine.2015.01.068
67. Kennedy A, Lavail K, Nowak G, Basket M, Landry S. Confidence about vaccines in the United States: understanding parents’ perceptions. Health Aff (Millwood) (2011) 30:1151–9. doi:10.1377/hlthaff.2011.0396
68. Opel DJ, Mangione-Smith R, Taylor JA, Korfiatis C, Wiese C, Catz S, et al. Development of a survey to identify vaccine-hesitant parents: the parent attitudes about childhood vaccines survey. Hum Vaccin (2011) 7:419–25. doi:10.4161/hv.7.4.14120
69. Jones AM, Omer SB, Bednarczyk RA, Halsey NA, Moulton LH, Salmon DA. Parents’ source of vaccine information and impact on vaccine attitudes, beliefs, and nonmedical exemptions. Adv Prev Med (2012) 2012:932741. doi:10.1155/2012/932741
70. Witteman HO, Zikmund-Fisher BJ. The defining characteristics of web 2.0 and their potential influence in the online vaccination debate. Vaccine (2012) 30:3734–40. doi:10.1016/j.vaccine.2011.12.039
71. Bianco A, Zucco R, Nobile CG, Pileggi C, Pavia M. Parents seeking health-related information on the internet: cross-sectional study. J Med Internet Res (2013) 15:e204. doi:10.2196/jmir.2752
72. Ruiz JB, Bell RA. Understanding vaccination resistance: vaccine search term selection bias and the valence of retrieved information. Vaccine (2014) 32:5776–80. doi:10.1016/j.vaccine.2014.08.042
73. Larson HJ, Smith DM, Paterson P, Cumming M, Eckersberger E, Freifeld CC, et al. Measuring vaccine confidence: analysis of data obtained by a media surveillance system used to analyse public concerns about vaccines. Lancet Infect Dis (2013) 13:606–13. doi:10.1016/S1473-3099(13)70108-7
74. Salathe M, Khandelwal S. Assessing vaccination sentiments with online social media: implications for infectious disease dynamics and control. PLoS Comput Biol (2011) 7:e1002199. doi:10.1371/journal.pcbi.1002199
75. Madden K, Nan X, Briones R, Waks L. Sorting through search results: a content analysis of HPV vaccine information online. Vaccine (2012) 30:3741–6. doi:10.1016/j.vaccine.2011.10.025
76. Harmsen IA, Doorman GG, Mollema L, Ruiter RA, Kok G, de Melker HE. Parental information-seeking behaviour in childhood vaccinations. BMC Public Health (2013) 13:1219. doi:10.1186/1471-2458-13-1219
77. Oncel S, Alvur M. How reliable is the internet for caregivers on their decision to vaccinate their child against influenza? Results from googling in two languages. Eur J Pediatr (2013) 172:401–4. doi:10.1007/s00431-012-1889-z
78. Pias-Peleteiro L, Cortes-Bordoy J, Martinon-Torres F. Dr. Google: what about the human papillomavirus vaccine? Hum Vaccin Immunother (2013) 9:1712–9. doi:10.4161/hv.25057
79. Nundy S, Surati M, Nwadei I, Singal G, Peek ME. A web-based patient tool for preventive health: preliminary report. J Prim Care Community Health (2012) 3:289–94. doi:10.1177/2150131911436011
80. Shropshire AM, Brent-Hotchkiss R, Andrews UK. Mass media campaign impacts influenza vaccine obtainment of university students. J Am Coll Health (2013) 61:435–43. doi:10.1080/07448481.2013.830619
81. Starling R, Nodulman JA, Kong AS, Wheeler CM, Buller DB, Woodall WG. Beta-test results for an HPV information web site: GoHealthyGirls.org – increasing HPV vaccine uptake in the United States. J Consum Health Internet (2014) 18:226–37. doi:10.1080/15398285.2014.931771
82. World Health Organization. Vaccine Safety Net (2015). Available from: http://www.who.int/vaccine_safety/initiative/communication/network/vaccine_safety_websites/en/
83. Wilson K, Keelan J. Social media and the empowering of opponents of medical technologies: the case of anti-vaccinationism. J Med Internet Res (2013) 15:e103. doi:10.2196/jmir.2409
84. Keelan J, Pavri V, Balakrishnan R, Wilson K. An analysis of the human papilloma virus vaccine debate on MySpace blogs. Vaccine (2010) 28:1535–40. doi:10.1016/j.vaccine.2009.11.060
85. Nan X, Madden K. HPV vaccine information in the blogosphere: how positive and negative blogs influence vaccine-related risk perceptions, attitudes, and behavioral intentions. Health Commun (2012) 27:829–36. doi:10.1080/10410236.2012.661348
86. Love B, Himelboim I, Holton A, Stewart K. Twitter as a source of vaccination information: content drivers and what they are saying. Am J Infect Control (2013) 41:568–70. doi:10.1016/j.ajic.2012.10.016
87. Keelan J, Pavri-Garcia V, Tomlinson G, Wilson K. YouTube as a source of information on immunization: a content analysis. JAMA (2007) 298:2482–4. doi:10.1001/jama.298.21.2482
88. Raviotta JM, Nowalk MP, Lin CJ, Huang HH, Zimmerman RK. Using Facebook to recruit college-age men for a human papillomavirus vaccine trial. Am J Mens Health (2016) 10(2):110–9. doi:10.1177/1557988314557563
89. Remschmidt C, Walter D, Schmich P, Wetzstein M, Delere Y, Wichmann O. Knowledge, attitude, and uptake related to human papillomavirus vaccination among young women in Germany recruited via a social media site. Hum Vaccin Immunother (2014) 10:2527–35. doi:10.4161/21645515.2014.970920
90. Stratton SL, Spencer HJ, Greenfield WW, Low G, Hitt WC, Quick CM, et al. A novel use of a statewide telecolposcopy network for recruitment of participants in a phase I clinical trial of a human papillomavirus therapeutic vaccine. Clin Trials (2015) 12(3):199–204. doi:10.1177/1740774514566333
91. Gunasekaran B, Jayasinghe Y, Brotherton JM, Fenner Y, Moore EE, Wark JD, et al. Asking about human papillomavirus vaccination and the usefulness of registry validation: a study of young women recruited using Facebook. Vaccine (2015) 33:826–31. doi:10.1016/j.vaccine.2014.11.002
92. Gowda C, Schaffer SE, Kopec K, Markel A, Dempsey AF. A pilot study on the effects of individually tailored education for MMR vaccine-hesitant parents on MMR vaccination intention. Hum Vaccin Immunother (2013) 9:437–45. doi:10.4161/hv.22821
93. Williams SE, Rothman RL, Offit PA, Schaffner W, Sullivan M, Edwards KM. A randomized trial to increase acceptance of childhood vaccines by vaccine-hesitant parents: a pilot study. Acad Pediatr (2013) 13:475–80. doi:10.1016/j.acap.2013.03.011
94. Zhang C, Gotsis M, Jordan-Marsh M. Social media microblogs as an HPV vaccination forum. Hum Vaccin Immunother (2013) 9:2483–9. doi:10.4161/hv.25599
95. Community Preventive Services Task Force. Guide to Community Preventive Services. Increasing Appropriate Vaccination: Client Reminder and Recall Systems (2015). Available from: www.thecommunityguide.org/vaccines/universally/clientreminder.html
96. Peck JL, Stanton M, Reynolds GE. Smartphone preventive health care: parental use of an immunization reminder system. J Pediatr Health Care (2014) 28:35–42. doi:10.1016/j.pedhc.2012.09.005
97. Free C, Phillips G, Felix L, Galli L, Patel V, Edwards P. The effectiveness of M-health technologies for improving health and health services: a systematic review protocol. BMC Res Notes (2010) 3:250. doi:10.1186/1756-0500-3-250
98. Hartzler A, Wetter T. Engaging patients through mobile phones: demonstrator services, success factors, and future opportunities in low and middle-income countries. Yearb Med Inform (2014) 9:182–94. doi:10.15265/IY-2014-0022
99. Odone A, Ferrari A, Spagnoli F, Visciarelli S, Shefer A, Pasquarella C, et al. Effectiveness of interventions that apply new media to improve vaccine uptake and vaccine coverage. Hum Vaccin Immunother (2015) 11:72–82. doi:10.4161/hv.34313
100. Batler D. When Google Got Flu Wrong. US Outbreaks Foxes a Leading Web-Based Method for Tracking Seasonal Flu (2013). Available from: http://www.nature.com/news/when-google-got-flu-wrong-1.12413
101. Eberth JM, Kline KN, Moskowitz DA, Montealegre JR, Scheurer ME. The role of media and the Internet on vaccine adverse event reporting: a case study of human papillomavirus vaccination. J Adolesc Health (2014) 54:289–95. doi:10.1016/j.jadohealth.2013.09.005
102. Cohen AM, Hersh WR. A survey of current work in biomedical text mining. Brief Bioinform (2005) 6:57–71. doi:10.1093/bib/6.1.57
103. Berendt B. Text mining for news and blogs analysis. In: Sammut C, Webb GI, editors. Encycolpedia of Machine Learning. New York: Springer (2010).
104. Wojcik OP, Brownstein JS, Chunara R, Johansson MA. Public health for the people: participatory infectious disease surveillance in the digital age. Emerg Themes Epidemiol (2014) 11:7. doi:10.1186/1742-7622-11-7
105. Baazeem M, Abenhaim H. Google and women’s health-related issues: what does the search engine data reveal? Online J Public Health Inform (2014) 6:e187. doi:10.5210/ojphi.v6i2.5470
106. Lewis SP, Mahdy JC, Michal NJ, Arbuthnott AE. Googling self-injury: the state of health information obtained through online searches for self-injury. JAMA Pediatr (2014) 168:443–9. doi:10.1001/jamapediatrics.2014.187
107. Fox S. Peer-to-Peer Healthcare. Washington, DC: Pew Research Center’s Internet & American Life Project (2011).
108. Clark SJ, Butchart A, Kennedy A, Dombkowski KJ. Parents’ experiences with and preferences for immunization reminder/recall technologies. Pediatrics (2011) 128:e1100–5. doi:10.1542/peds.2011-0270
Keywords: vaccination strategies, vaccination, immunization, immunization programs, registries, communication, vaccine confidence, information dissemination
Citation: Tozzi AE, Gesualdo F, D’Ambrosio A, Pandolfi E, Agricola E and Lopalco P (2016) Can Digital Tools Be Used for Improving Immunization Programs? Front. Public Health 4:36. doi: 10.3389/fpubh.2016.00036
Received: 31 March 2015; Accepted: 19 February 2016;
Published: 08 March 2016
Edited by:
Philip AbdelMalik, Public Health Agency of Canada, CanadaReviewed by:
Domenico Martinelli, University of Foggia, ItalySamantha Dean Wilson-Clark, Public Health Agency of Canada, Canada
Copyright: © 2016 Tozzi, Gesualdo, D’Ambrosio, Pandolfi, Agricola and Lopalco. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Francesco Gesualdo, Zi5nZXN1YWxkbyYjeDAwMDQwO2dtYWlsLmNvbQ==
†Present address: Pierluigi Lopalco, Department of Biomedical Science and Human Oncology, University of Bari, Bari, Italy