A Scoping Review of the Quality and the Design of Evaluations of Mobile Health, Telehealth, Smart Pump and Monitoring Technologies Performed in a Pharmacy-Related Setting

Background: There is currently a need for high quality evaluations of new mobile health, telehealth, smart pump and monitoring technologies undertaken in a pharmacy-related setting. We aim to evaluate the use of these monitoring technologies performed in this setting.

Methods: A systematic searching of English articles that examined the quality and the design of technologies conducted in pharmacy-related facilities was performed using the following databases: MEDLINE and Cumulative index to Nursing and Allied Health Literature (CINAHL) to identify original studies examining the quality and the design of technologies and published in peer-reviewed journals. Extraction of articles and quality assessment of included articles were performed independently by two authors. Quality scores over 75% are classed as being acceptable using a “relatively conservative” quality benchmark. Scores over 55% are included using a “relatively liberal” cut-off point.

Results: Screening resulted in the selection of 40 formal evaluations. A substantial number of studies (32, 80.00%) were performed in the United States, quantitative in approach (33, 82.50%) and retrospective cohort (24, 60.00%) in study design. The most common pharmacy-related settings were: 22 primary care (55.00%); 10 hospital pharmacy (25.00%); 7 community pharmacy (17.50%); one primary care and hospital pharmacy (2.50%). The majority of the evaluations (33, 82.50%) reported clinical outcomes, six (15.00%) measured clinical and economic outcomes, and one (2.50%) economic only. Twelve (30.00%) quantitative studies and no qualitative study met objective criteria for “relatively conservative” quality. Using a lower “relatively liberal” benchmark, 27 quantitative (81.82%) and four qualitative (57.41%) studies met the lower quality criterion.

Conclusion: Worldwide, few evaluations of mobile health, telehealth, smart pump and monitoring technologies in pharmacy-related setting have been published.Their quality is often below the standard necessary for inclusion in a systematic review mainly due to inadequate study design.

Introduction

Pharmacy is an evidence-based profession (Toklu, 2015). Pharmacists are generally unwilling to recommend unproven health technologies, especially when their adoption could have negative unanticipated consequences (Sun and Qu, 2015). Evidence of effectiveness and cost-effectiveness is vitally important for decision-makers (Dawoud and Baines, 2017). New technologies often face adoption problems due to a scarcity of suitable evidence (Van Gemert-Pijnen et al., 2011). A deficiency of scientific studies is a widespread problem in all areas of healthcare provision, including pharmacy (Chaudhry et al., 2006; Atienza et al., 2010; Black et al., 2011; FIP/WHO, 2011). There is a growing gap between the rate of technological advance and the production of supporting evaluations. For instance, there are now over 165,000 health-related apps available on the main smartphone operating systems (The Economist, 2016). Few are launched with accompanying high-quality, peer-reviewed evidence demonstrating their effectiveness and/or cost-effectiveness (Steinhubl et al., 2015).

Technological innovation is important in all industries (Burns and Stalker, 1961). For a pharmacy workforce that has been relatively static for many decades, disruptive technological change (a change that creates a new system and eventually disrupts an existing system) could create a “technology shock” (Spinks et al., 2016). Disrupting technologies can enhance adaptability and flexibility, helping pharmacists to identify and to solve unfamiliar problems in unfamiliar situations (Woods et al., 2015). For instance, the profession could use smart packaging to extend its surveillance of medicines taking (Darkins et al., 2008; Sparks, 2011). As the widespread adoption of robotics and bar-code technologies may soon reduce the number of pharmacists involved in dispensing, pharmaceutical care may be the way ahead (Hepler and Strand, 1990; Baines, 2015). In response, we label (non-dispensing) technologies that enable pharmacists to improve patient health and quality of life with the name of “pharmaceutical care technologies.” We suggest that mobile health (general term for the use of mobile phones and other wireless technology to educate consumers about preventive health care services), telehealth (the distribution of health-related services and information via electronic information and telecommunication technologies), smart pump (programmable infusion device for controlling and administering intravenous medicines) and monitoring technologies (e.g., microchip-containing tablet blister for adherence monitoring) are amongst this group of innovations.

With a deficit of available evidence, pharmacists do not always have the information required to make rational technology adoption decisions (Gregorio et al., 2013). Even when evaluation studies are published, many are poorly designed and their low quality can misguide decision-makers (Shuren and Califf, 2016). To improve this situation, the constant production of well-designed, high-quality evaluations is required. The first step in improving the current situation is to evaluate the current state of the technology appraisal literature that focuses on pharmacy practice. This paper is a scoping review of the quality and the design of evaluations of mobile health, telehealth, smart pump and monitoring technologies performed in a pharmacy-related setting. To our knowledge, no review has yet analyzed the quality and the design of peer-reviewed evaluations of these technologies in this setting. The rationale of this review is to help fill that gap.

Methods

Scope of Review: Eligibility Criteria

A scoping review was undertaken to identify evaluations of mobile health, telehealth, smart pumps and monitoring technologies based upon the classification of technologies identified by Goundrey-Smith (2013, 2014) The review assesses the quality and the design of the identified studies. We limited our search to studies written in English and did not restrict them according to country. Studies had to measure either clinical or economic outcomes, or both. The studies had to fall into our defined technology categories. All study designs (including trials, protocols) were included in the search process. However, only evaluation studies were included in the final analysis.

This scoping review examined the quality and the design of studies published in peer-reviewed journals that report scientific evaluations of mobile health, telehealth, smart pump and monitoring technologies tested in a pharmacy-related setting. This paper is written from the point of view of pharmacy practice research. The quality of the evaluations identified is judged in terms of a composite quality score developed by Kmet et al. (2004) which assesses study design in terms of structure, methodology and outcomes measured. The main objectives of our review are to: (i) identify all relevant studies published in a peer-review journal that evaluate pharmaceutical care technologies; (ii) evaluate the quality of this work; (iii) identify the study designs and the clinical/economic outcomes used; (iv) examine the link between study design and evaluation quality; (v) outline the limitations of our review. To achieve these objectives, we perform the following interlinked tasks. First, we outline our review methods, including inclusion criteria and our approach to quality scoring. Second, our results are presented, including data on quality and design. Third, we discuss the main findings of the review. Finally, we outline the limitations of our work.

Definitions

Hepler and Strand (1990) defined “pharmaceutical care” as the “responsible provision of drug therapy for the purpose of achieving definite outcomes that improve a patient's quality of life.” Subsequently, the board of the Pharmaceutical Care Network Europe (Allemann et al., 2014) suggested that “pharmaceutical care” may be defined as the “pharmacist's contribution to the care of individuals to optimize medicines use and improve health outcomes.” Combining both definitions, we define “pharmaceutical care technologies” as “technologies that enable pharmacists to optimize medicines use and to enhance patient health and quality of life.”

Information Sources

The following databases were searched: MEDLINE and Cumulative index to Nursing and Allied Health Literature (CINAHL) to identify studies published in peer-reviewed journals between January 1, 2012 and December 31, 2016. Reference lists of articles identified in the search were included and were subject to the same eligibility evaluation.

Searching and Screening

We searched the health technology literature using the key terms suggested by Goundrey-Smith. All technology studies selected included the secondary search terms: “pharmacy,” “pharmacist,” or “pharmacies.” The search strategy for each database included the concepts: pharmaceutical care technologies; pharmacy. The former included the following terms derived from Goundrey-Smith: “mobile health,” “monitoring technologies,” “smart pump,” “smart packaging” and “telechealth.” Other keywords associated with telehealth (suggested on EBSCO) were included: “telemonitoring,” “teleconsultation,” “telecare,” and “telemedicine.” Goundrey-Smith also discusses the potential of mobile phone apps having an impact on pharmacy practice. A range of search terms (suggested by EBSCO) associated with mobile technology were included: “mHealth,” “m-health,” “mobile health,” and “medical apps.” “Health technology,” and “health information technology” were specified as umbrella terms for other technologies not included in our list. For our secondary search concept, “pharmacy” was defined as “pharmacy,” “pharmacies,” or “pharmacist.” Titles and abstracts were screened to remove studies that were clearly irrelevant to the aim of the review (by IG and DB). The full texts of the remaining original studies were then examined to determine eligibility.

Study Selection

Article titles and abstracts were screened independently by two different authors, IG and DB against the inclusion and exclusion criteria. Search results were imported into a Microsoft Excel and manually screened by these authors to remove any duplicates. Article titles were shortlisted for a detailed review of their abstracts. Abstracts that seemed appropriate were then screened for a full-text review. A quality assessment was performed on the articles eligible for the review. Throughout this process, we focused on articles published in English that were evaluations in the categories of mobile health, telehealth, smart pumps and monitoring technologies (the “comparisons”) of pharmaceutical care technologies (the “interventions”) tested on patients and staff (the “participants”) in a pharmacy-related setting. Study protocols and reviews were not included.

Data Collection Process

A data extraction table was created to collect relevant information for each included study. The principal investigator extracted data using a data extraction form (table format) that was verified by the second reviewer. The extracted data included the name of the author, year of publication, journal, place, methodology, sample size, study design, intended market, primary outcome and study type.

Quality Analysis

We analyzed the quality (the “outcome”) of this work and reported the evaluation methods used (the “study design”). To assess the quality of each paper, scoring was undertaken using the criteria for evaluating primary research papers developed by Kmet et al. For quantitative studies, a series of 14 questions are used to assess quality. These include questions related to study design, methods of subject selection, random allocation procedures, blinding of investigators, blinding of subjects, outcome measures, sample size, estimates of variance, confounding, reporting of results and the evidence base for the conclusion. Each question is scored using the following options: Yes (2 points), Partial (1 point), No (0 points), and N/A. The equation for estimating the quality scores is: 28 minus (number of N/A × 2). The figure of 28 is the maximum score possible for the 14 questions, which can be given a maximum score of 2. We calculated a quality score for 33 quantitative papers using this method and report the results below.

As they will often use different methods to their quantitative counterparts, Kmet et al. (2004) recommend a different means of evaluating the quality of qualitative studies. Instead of 14, only 10 questions are used to assess the qualitative papers. The questions assess study design, context of the study, sampling strategy, data collection and methods of data analysis. Each question is scored using the following options: Yes (2), Partial (1), and No (0). This method includes a N/A option. The equation calculates the summary score for qualitative study as follows: Total score/20. The figure of 20 is the maximum score possible for 10 questions, which can be given a maximum score of 2. We calculated a quality score for the seven qualitative studies sampled.

The quality score system was developed to define a minimum threshold for the inclusion of studies in systematic reviews. Kmet et al. (2004) suggest two cut-points for article inclusion. Scores over 75% are classed as being acceptable using a “relatively conservative” quality benchmark. Scores over 55% are included using a “relatively liberal” cut-off point. In this paper, we report the number of quantitative and qualitative studies that meet the conservative and liberal criteria. However, we do not exclude papers from our analysis on this basis. Because we do not intend to analyse the effectiveness of individual interventions at a patient-level, the usual forms of bias do not apply to our review (Higgins and Green, 2008).

Results

Study Selection and Characteristics

As Figure 1 shows, our initial search led to a total of 4,377 results (3,710 MEDLINE and 667 CINAHL). Following an analysis of publication titles, 345 papers were manually screened for abstract analysis. After reading the abstracts, 82 publications were selected for full-text assessment. The full reading resulted in 42 papers being deemed irrelevant to our review and 40 formal evaluations were finally selected.

FIGURE 1

Figure 1. Flow diagram of scoping review search.

Table 1 shows that the studies identified had the following characteristics. First, 32 out of 40 (80.00%) of the evaluations reviewed were performed in the United States, followed by three (7.50%) in the Netherlands. Second, 33 studies (82.50%) were quantitative and seven (17.50%) qualitative. Third, amongst the papers retrieved, the most common study designs employed were (with % of frequency in brackets) retrospective cohort study (24, 60.00%) and randomized controlled trials (RCTs) (9, 22.50%). Fourth, the intended setting for the technologies evaluated were: 22 primary care (55.00%); 10 hospital pharmacy (25.00%); 7 community pharmacy (17.50%); one primary care and hospital pharmacy (2.50%). Finally, out of the 40 evaluations, 33 (82.50%) reported clinical outcomes, six (15.00%) measured clinical and economic outcomes, and one (2.50%) economic only.

TABLE 1

Table 1. Summary characteristics of articles included in the review (n = 40).

Results by Technology Category

This review found that only a handful of evaluations were published in each of the technology categories shown in Table 2. First in the ranking is “telehealth” with twenty studies (50.00%), followed by “mobile health” with nine (22.50%). Then, “monitoring technologies” produced seven (17.50%) studies. Finally, four papers (10.00%) evaluated “smart pumps.”

TABLE 2

Table 2. Pharmaceutical care technologies.

Telehealth

Table 2 shows that 20 studies focused on telehealth. They include evaluations of telehealth, telemedicine and telemonitoring. Seventeen of the 20 studies sampled were American, with one study each from Italy, Canada and the Netherlands. Three were qualitative and 17 were quantitative. Nineteen sampled patients to test their technologies; one sampled staff. Retrospective cohort study was the most commonly used study design. Six out of eight studies published were conducted in a primary care setting, one took place in community pharmacy and the other in hospital pharmacy. All studies specified a primary outcome. Six undertook a clinical evaluation, one conducted a clinical and economic evaluation and one performed an economic evaluation only. All six of the clinical evaluations produced positive results. The joint clinical and economic study was positive in both dimensions and the stand alone economic study was negative.

Mobile Health

Table 2 shows that nine evaluations focused on mobile health technologies. However, eight out of the nine studies were conducted in the USA and one in the UK. Of the studies published, three were qualitative and six were quantitative. Six out of nine studies recruited patients to test their technologies. Two studies used providers and another used patient medical records. Three studies employed a retrospective cohort design, five employed a formative design and one was a RCT. Five studies were undertaken in a primary care setting, three in community pharmacy and one in hospital pharmacy. All studies specified a primary outcome and performed a clinical evaluation. No economic or costing work was undertaken. Four of the nine studies had negative results.

Monitoring Technologies

Table 2 shows that seven studies focused on monitoring technologies, which includes smart packaging. All seven were quantitative and used patients to test their technologies. Out of these, three were formative, two were RCTs, one was a prospective observational study and one was a retrospective cohort study. Four studies were conducted in primary care alone, one in community pharmacy, one in hospital pharmacy and one jointly in primary care and hospital pharmacy. All the studies specified a primary outcome and performed a clinical evaluation of their chosen technology. No economic or costing studies were performed. Five of the seven studies produced positive results.

Smart Pumps

Table 2 shows that four studies evaluated smart pumps. One study was qualitative and the remaining three were quantitative. Two studies sampled patients to test their technologies. One evaluation focused on providers and the other performed a system evaluation. A retrospective cohort design was used for two studies. One was a formative evaluation and another was a prospective observational study. All studies were conducted in a hospital pharmacy setting and stated a primary outcome. Two undertook clinical evaluations. Two conducted an economic evaluation alongside clinical studies. All evaluations produced positive findings.

Quality Assessment

Table 3 shows the quality scores for both quantitative and qualitative studies. In terms of quality, 12 (30.00%) quantitative studies and no qualitative study met the Kmet et al. (22) criteria for “relatively conservative” quality. Using the lower “relatively liberal” benchmark, 27 (67.50%) quantitative studies and four (10.00%) qualitative study passed. The table also shows the quality score for each of the 33 quantitative studies, which was calculated using the Kmet et al. framework. The quantitative summary scores vary from 39 to 88. Applying the conservative benchmark of 75%, 21 (63.64%) out of the 33 quantitative studies were of insufficient quality. If the standard is dropped, using the 55% cut-off only six (18.18%) studies fail to meet the required benchmark. Therefore, whether 15 (45.45%) studies have acceptable quality depends upon which of the two cut-offs is used.

TABLE 3

Table 3. Quality analysis table for quantitative and qualitative studies.

Quality scores were also calculated for the seven qualitative papers identified, which produced a range of 20–70. As Table 3 shows, using the conservative criteria only, no study was of sufficient quality. Under the liberal criteria, four (57.14%) papers managed to reach the required standard. Therefore, a switch of criteria results in a difference of four (57.14%) papers being acceptable in quality terms.

Study Quality and Design

Table 4 shows average quality scores by study design and technology categories. The following quality scores were obtained for quantitative studies: cross-sectional (77.00%); economic evaluation (61.00%); formative (66.00%); non-randomized controlled trial (86.00%); prospective observational (74.67%); RCT (86.13%); retrospective cohort (70.11%). The qualitative scores are: formative (52.50%); prospective observational (60.00%). The sub-totals show that the average quality score for quantitative studies is 64.97% and for qualitative is 53.57%. The results also show quality scores by technology: mobile health (67.89%); monitoring technologies (69.86%); smart pumps (49.50%); telehealth (65.35%).

TABLE 4

Table 4. Average percentage quality score by study design and technology.

Discussion

Without sufficient evidence, novel technologies may fail to be adopted in a pharmacy-related setting (Siska and Tribble, 2011). The review presented here suggests that, worldwide, few evaluations of pharmaceutical care technologies are currently being undertaken in such a place. With only 40 relevant studies being identified over a 5-year period, the first finding of our review is that pharmacy-related evaluations of new technologies are relatively rare. This number is small, given the number of new technologies that appear each year. However, in interpreting this data, we must add the caveat that many of the health technologies launched each year may be evaluated for general use in any health setting, which covers pharmacy by default. For instance, Dayer et al. (2013) reviews the potential benefits of Smartphone medication adherence apps to patients and providers, but does not undertake this work solely with pharmacist input or in a pharmacy-related setting. Therefore, their paper did not appear in our search. As this suggests, pharmacy specific evaluations may not be needed for many new health technologies. Our search is, consequently, narrow in its focus. Even if these caveats are considered, the numbers of evaluations identified in this review are still relatively low.

Whilst it is useful to speculate about the relative number of papers published in this area, the main focus of this review is study quality and design. Using the Kmet et al. (2004) scoring system, we found a wide dispersion in the quality of papers published. Only 30.00% of quantitative studies met the higher, conservative benchmark. In comparison, no qualitative studies achieved this quality standard. In a sector familiar with high-quality evidence accompanying the launch of new pharmaceutical products, new technologies with low quality evidence may struggle to be adopted. Therefore, our third finding is that the quality of most of evaluations we reviewed is lacking when compared to objective criteria. As the Kmet et al. (2004) scoring system is primarily based upon the quality of study design, we suggest that inadequate study design is the cause of these lower quality scores. This view is support by the data presented in Table 4.

We classify pharmaceutical care technologies as including mobile health, telehealth, smart pumps and monitoring technologies. These technologies are pharmacy-enabling because they extend the knowledge and the capabilities of pharmacists, enabling them to create better outcomes for patients. This category suggests that pharmaceutical care activities may be enhanced by new health technologies, as well as an extension of professional skills and roles (Lapão et al., 2013). Of the 40 papers we identified, 33 were quantitative and 7 were qualitative. The average quality score in the former group is (2144/33) = 64.97% and (375/7) = 53.57% in the latter. Both these average scores are below the Kmet et al. relatively conservative quality benchmark. We discuss the evaluations in these classes below.

Telehealth is an enabling-technology for patients (Dewsbury and Ballard, 2012). We found 20 studies relevant to pharmacy. Margolis evaluates telemonitoring for home blood pressure in hypertension care (Margolis et al., 2012). Similarly, Wang performs an economic evaluation of telephone self-management for blood pressure control (Wang et al., 2012). Margolis examines the use of telepharmacy to improve inhaler adherence in veterans with COPD (Margolis et al., 2013). Martinez studies the implementation of a pharmacist-led heart titration clinic that employs telemonitoring for the daily monitoring of patient body weight (Martinez et al., 2013). Philip analyses the expansion of clinical pharmacy through increased use of outpatient pharmacists for anticoagulation services (Philip et al., 2015). Verbosky assesses the implementation of diabetes management via telehealth, whilst Owsley evaluates telemedicine for diabetic eye screening (Owsley et al., 2015; Verbosky et al., 2016). Singh studies the implementation and benefits of a pharmacist-managed telehealth anticoagulation clinic (Singh et al., 2015). Aberger evaluates a telehealth system that incorporates home blood pressure monitoring (Aberger et al., 2014). Brunetti describes the CAPITAL study, which focuses on cardiovascular prevention with telecardiology (Brunetti et al., 2016). Cole examines inpatient telepharmacy consultation and medication errors (Cole et al., 2012). Desko evaluates a clinical video telehealth pain management clinic (Desko and Nazario, 2014). Gordon investigates a telepharmacy initiative for cancer patients (Gordon et al., 2012). Fortney evaluates telemedicine collaborative care for depression (Fortney et al., 2015). Fortney analyses telemedicine-based collaborative care for post-traumatic stress disorder (Fortney et al., 2013). Kooy examines patient satisfaction with telephone counseling by pharmacists and medication refill adherence (Kooy et al., 2015). McFarland examines the use of home telehealth monitoring by clinical pharmacists with type 2 diabetic patients (McFarland et al., 2012). Schneider evaluates the impact of telepharmacy through medication errors (Schneider, 2013). Shane-McWhorter analyses a pharmacist-provided diabetes management telemonitoring program (Shane-McWhorter et al., 2015). Young investigates patient and pharmacist telephone encounters with asthma patients (Young et al., 2012).

The above studies suggest that telehealth technologies employed in a pharmacy context focuses on communication between pharmacists and patients. This is very different to telecare as monitoring devices (such as pendant alarms) located in patients' own homes (Steventon et al., 2013; Roulstone, 2016). Therefore, greater clarity on the types and the benefits of telehealth, telecare and telepharmacy would be useful for guiding future pharmacy research and practice, particularly as the use of this category of technology raises practical and ethical issues (Mitka, 2013; Mort et al., 2013). Given the personalized nature of this technology, all of the 20 studies identified recruited patients to test their innovative technologies using a range of study designs. The average quality scores for the studies were: formative evaluations (51%); RCTs (75%); retrospective cohort studies (58%); cross-sectional study (77%); economic evaluation (61%); prospective observational study (72%); non-randomized controlled trial (86%). These results imply that the quality standards achieved in this category of evaluation were variable, with implications for the trustworthiness of the evidence generated.

Our review identified nine papers evaluating mobile health technologies (Steinhubl et al., 2015). Gustafson examines the effects of a patient support system that includes monthly phone calls carried out by pharmacists and other health care professionals (Gustafson et al., 2012). Foreman assesses the impact of text messaging to improve patient medication adherence, whilst Snuggs analyses the use of texting to prevent relapses in smoking (Foreman et al., 2012; Snuggs et al., 2012). Burk examines a web-based app for medication use in a veteran's healthcare system (Burk et al., 2013). Andrus assesses the accuracy of formulary information in both EHR and the Epocrates mobile drug database application, used for making medication selection decisions (Andrus et al., 2015). Wilcox designed a personal health record application called “myNYP inpatient” and assesses its usefulness for patients (Wilcox et al., 2016). DiDonato evaluates patient perceptions of a pharmacy mobile app for patient adherence (DiDonato et al., 2015). Gatwood studies the impact of tailored text messages on medication usage amongst diabetic patients (Gatwood et al., 2016). Sarzynski uses beta-testing on a smartphone application for medication adherence (Sarzynski et al., 2017).

Compared to the large number of health apps launched annually, the handful of studies identified suggests that very few mobile health technologies are reviewed formally in a pharmacy setting. The absence of such scientific evaluations may be evidence of a “valley of death” between product availability and use in pharmacy practice (Wessner, 2005; Páez-Avilés et al., 2015). For a profession keen on creating new roles and opportunities, the failure to adopt new mobile technologies as part of the pharmacist's tool-kit may limit continued professional growth (Miranda et al., 2014; Ventola, 2014). Of the studies published, three studies employ a retrospective cohort design, five employ a formative design and one is a RCT. Their average quality scores are 66, 69, and 71%, retrospectively. These scores suggest that improved study designs could generate higher quality evidence in this category.

Seven studies were identified that evaluate monitoring technologies in a pharmacy context (Speedie et al., 2008; Fox et al., 2011). Van Onzenoort studies smart blister-pack technology for checking patient adherence (Van Onzenoort et al., 2012). Vasbinder assesses the relationship between ethnicity and electronically measured adherence to inhaled corticosteroids amongst children (Vasbinder et al., 2013). Zullig pilots a medication calendar that incorporates reminders using Meducation^® technology (Zullig et al., 2014). Chan evaluates the effects of using electronic monitoring devices for adherence to inhaled corticosteroids, whilst Migliozzi looks at controlling blood pressure using home monitoring (Chan et al., 2015; Migliozzi et al., 2015). Sayner compares self-reported adherence data with information collected form a medication event monitoring system (Sayner et al., 2015). Finally, Bender evaluated the use of a novel technology to improve the adherence of pediatric asthmatic patients (Bender et al., 2015). Combined, these studies suggest that monitoring technologies have an important role to play in enabling pharmacists to achieve better outcomes for patients, particularly in relation to medication adherence (Bosworth et al., 2011). However, the volume of evidence generated is low given the potential benefits offered by innovations in this category. Although all seven evaluations are quantitative and involve patients, study designs are not uniform: three are formative (50, 58, 65%); two are RCTs (83, 86%); one is a prospective observational study (86%); one is a retrospective cohort study (61%). Their quality scores are shown in the proceeding brackets.

Smart pumps are electronic infusion devices that have the potential to reduce intravenous drug administration errors (Franklin, 2017). Our review found four published studies. Harding examines the use of smart pump drug libraries where continuing education is provided for nurses and pharmacists (Harding, 2012). Kennerly studies smart pumps for epidural infusions, whilst Manrique-Rodriguez evaluates the impact of introducing smart pumps into a pediatric intensive care unit (Kennerly et al., 2012; Manrique-Rodríguez et al., 2013). Finally, Gerhart combines the safety features of infusion pumps with software that shows live infusion data (Gerhart et al., 2013). Smart pumps are an important enabling technology for pharmacy practice. However, our review creates a mixed message of whether more studies are required. In our search terms, we specified that evaluations must be performed in a pharmacy-related setting. However, this category of technology may also be identified as medicines or guidelines related, particularly in the case of medication libraries containing dosing guidelines, concentrations, clinical advice and the like. We conclude that smart pumps are a vital technology in pharmacy practice, but we have identified few evaluations in this category using our key terms. In terms of study design, two evaluations employ a retrospective cohort design (39%, 44%), one is a formative evaluation (60%) and another is a prospective observational study (55%). Relevant quality scores are in the brackets shown.

First, health technology is a diverse field, which covers many thousands of new innovations launched each year. We identified a new class of studies, which we call “pharmaceutical care technologies.” This division is innovative. More research is, therefore, required to ascertain whether this novel categorization is useful for pharmacy practice and research. Second, our review focuses on technologies that enable pharmacists and pharmacy practice. However, many technologies are designed to enable patients (not their practitioners). Our approach may be orthogonal to the intentions of many device manufacturers, particularly when considering personalized health apps. Therefore, further research is required to examine the differences between practitioner- and patient-enabling technologies in a pharmacy-related setting. Finally, we focus on technologies suggested by Goundrey-Smith. We have not searched for all possible health technologies that could be adopted in a pharmacy-related setting. Notably, “wearables” and the “internet of things” were not included in our review. However, an initial search identified no relevant studies in the time-period we examined. As these new technologies grow in influence, more comprehensive reviews of the literature will be required.

Implications of the Review for Pharmacy Practice

The technological innovations discussed have the potential to improve pharmacy practice related tasks and functions. In addition, evidence from included studies suggested that the systems could be improved through innovation involving technology. Pharmacists should not assume that all new health technologies are automatically good for patients (Draper and Sorell, 2013). Therefore, this review suggests that the process of evaluating new health technologies should be continuous and be of the highest research quality possible (Catwell and Sheikh, 2009; Wapner, 2016). Not only will high quality evidence support the rational adoption of new pharmacy-enabling technologies, but better evidence may help avoid some of the unintended problems associated with the use of novel innovations (Patterson et al., 2002; Koppel et al., 2008). To secure the future of the profession, as technology evolves, pharmacy practice must evolve, too (Baines, 2008).

Conclusions

This paper reviews the quality and the design of studies published in peer-reviewed journals that report scientific evaluations of new health technologies tested in a pharmacy-related setting. The results suggest that, worldwide, few evaluations of pharmaceutical care technologies are currently being undertaken. With only 40 relevant studies being identified, the first finding of our review is that pharmacy-based evaluations of new technologies are relatively rare (not many have had a pharmacy focus). Our second finding is that the quality of most of the evaluations we reviewed was lacking when compared to objective criteria. As the Kmet et al. scoring system is primarily based upon the quality of study design, we can conclude that inadequate study design may be a significant cause of these lower quality scores. Despite the improvements in technology, there is limited evidence on how this translates to real settings and to consumer satisfaction. Most technology driven systems required significant funding and support, particularly those involving latest technology. Rigorous comparative studies are needed to evaluate the effectiveness of different technologies.

Author Contributions

IG and DB conceptualized the idea. IG developed the search strategy, conducted the searches, screened, selected and performed a quality scoring of the studies. DB was also involved in the title screening as the second reviewer. AH and AK were involved in drafting the manuscript and in the quality scoring of the studies. ZB, SH, and PS were also involved in revising the manuscript, checking the study selection process, quality scoring of the studies and tables. All authors contributed, read and approved the final version of the manuscript.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors. IG was funded by Coventry University's Ph.D Studentship programme.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

References

Aberger, E. W., Migliozzi, D., Follick, M. J., Malick, T., and Ahern, D. K. (2014). Enhancing patient engagement and blood pressure management for renal transplant recipients via home electronic monitoring and web-enabled collaborative care. Telemed. J. E. Health 20, 850–854. doi: 10.1089/tmj.2013.0317

PubMed Abstract | CrossRef Full Text | Google Scholar

Allemann, S. S., van Mil, J. F., Botermann, L., Berger, K., Griese, N., and Hersberger, K. E. (2014). Pharmaceutical care: the PCNE definition 2013. Int. J. Clin. Pharm. 363, 544–555. doi: 10.1007/s11096-014-9933-x

CrossRef Full Text | Google Scholar

Andrus, M. R., Forrester, J. B., Germain, K. E., and Eiland, L. S. (2015). Accuracy of pharmacy benefit manager medication formularies in an electronic health record system and the epocrates mobile application. J. Manag. Care Specialty Pharm. 21, 281–286. doi: 10.18553/jmcp.2015.21.4.281

PubMed Abstract | CrossRef Full Text | Google Scholar

Atienza, A. A., Hesse, B. W., Gustafson, D. H., and Croyle, R. T. (2010). E-health research and patient-centered care examining theory, methods, and application. Am. J. Prev. Med. 38, 85–88. doi: 10.1016/j.amepre.2009.10.027

PubMed Abstract | CrossRef Full Text | Google Scholar

Baines, D. (2008). Has pharmacy run out of time? Pharm. J. 281:21.

Baines, D. (2015). Community pharmacies deserve a renaissance. Int. Pharm. J. 33, 46–47.

Bender, B. G., Cvietusa, P. J., Goodrich, G. K., Lowe, R., Nuanes, H. A., Rand, C., et al. (2015). Pragmatic trial of health care technologies to improve adherence to pediatric asthma treatment: a randomized clinical trial. JAMA Pediatr. 169, 317–323. doi: 10.1001/jamapediatrics.2014.3280

PubMed Abstract | CrossRef Full Text | Google Scholar

Black, A. D., Car, J., Pagliari, C., Anandan, C., Cresswell, K., Bokun, T., et al. (2011). The impact of eHealth on the quality and safety of health care: a systematic overview. PLoS Med. 8:e1000387. doi: 10.1371/journal.pmed.1000387

PubMed Abstract | CrossRef Full Text | Google Scholar

Bosworth, H. B., Granger, B. B., Mendys, P., Brindis, R., Burkholder, R., Czajkowski, S. M., et al. (2011). Medication adherence: a call for action. Am. Heart J. 162, 412–424. doi: 10.1016/j.ahj.2011.06.007

PubMed Abstract | CrossRef Full Text | Google Scholar

Brunetti, N. D., Lanzone, S., Dellegrottaglie, G., Di Giuseppe, G., De Gennaro, L., Novielli, V., et al. (2016). The CAPITAL study (CArdiovascular Prevention wIth Telecardiology in ApuLia): preliminary results. J. Cardiovasc. Med. 17, 455–461. doi: 10.2459/JCM.0000000000000286

PubMed Abstract | CrossRef Full Text | Google Scholar

Burk, M., Moore, V., Glassman, P., Good, C. B., Emmendorder, T., Leadholm, T. C., et al. (2013). Medication-use evaluation with a web application. Am. J. Health Syst. Pharm. 70, 2226–2234. doi: 10.2146/ajhp130252

PubMed Abstract | CrossRef Full Text | Google Scholar

Burns, T. E., and Stalker, G. M. (1961). The Management of Innovation. New York, NY: University of Illinois at Urbana-Champaign's Academy for Entrepreneurial Leadership Historical Research Reference in Entrepreneurship.

Catwell, L., and Sheikh, A. (2009). Evaluating eHealth interventions: the need for continuous systemic evaluation. PLoS Med. 6:e1000126. doi: 10.1371/journal.pmed.1000126

PubMed Abstract | CrossRef Full Text | Google Scholar

Chan, A. H., Stewart, A. W., Harrison, J., Camargo, C. A., Black, P. N., and Mitchell, E. A. (2015). The effect of an electronic monitoring device with audiovisual reminder function on adherence to inhaled corticosteroids and school attendance in children with asthma: a randomised controlled trial. Lancet Respir. Med. 3, 210–219. doi: 10.1016/S2213-2600(15)00008-9

PubMed Abstract | CrossRef Full Text | Google Scholar

Chaudhry, B., Wang, J., Wu, S., Maglione, M., Mojica, W., Roth, E., et al. (2006). Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Ann. Intern. Med. 144, 742–752. doi: 10.7326/0003-4819-144-10-200605160-00125

PubMed Abstract | CrossRef Full Text | Google Scholar

Cole, S. L., Grubbs, J. H., Din, C., and Nesbitt, T. S. (2012). Rural inpatient telepharmacy consultation demonstration for after-hours medication review. Telemed. J. E Health 18, 530–537. doi: 10.1089/tmj.2011.0222

PubMed Abstract | CrossRef Full Text | Google Scholar

Darkins, A., Ryan, P., Kobb, R., Foster, L., Edmonson, E., Wakefield, B., et al. (2008). Care coordination/home telehealth: the systematic implementation of health informatics, home telehealth, and disease management to support the care of veteran patients with chronic conditions. Telemed. J. E. Health 14, 1118–1126. doi: 10.1089/tmj.2008.0021

PubMed Abstract | CrossRef Full Text | Google Scholar

Dawoud, D., and Baines, D. (2017). “Economic evaluation and its types,” in Economic Evaluation of Pharmacy Services, ed Z. Babar (Academic Press), 99–119.

Google Scholar

Dayer, L., Heldenbrand, S., Anderson, P., Gubbins, P. O., and Martin, B. C. (2013). Smartphone medication adherence apps: potential benefits to patients and providers. JAMA 53, 172–181. doi: 10.1331/JAPhA.2013.12202

CrossRef Full Text | Google Scholar

Desko, L., and Nazario, M. (2014). Evaluation of a clinical video telehealth pain management clinic. J. Pain Palliat. Care Pharmacother. 28, 359–366. doi: 10.3109/15360288.2014.969875

PubMed Abstract | CrossRef Full Text | Google Scholar

Dewsbury, G., and Ballard, D. (2012). Telecare: supporting independence at home. Br. J. Healthc. Assist. 6:71. doi: 10.12968/bjha.2012.6.2.71

CrossRef Full Text | Google Scholar

DiDonato, K. L., Liu, Y., Lindsey, C. C., Hartwig, D. M., and Stoner, S. C. (2015). Community pharmacy patient perceptions of a pharmacy-initiated mobile technology app to improve adherence. Int. J. Pharm. Pract. 23, 309–319. doi: 10.1111/ijpp.12168

PubMed Abstract | CrossRef Full Text | Google Scholar

Draper, H., and Sorell, T. (2013). Telecare, remote monitoring and care. Bioethics 27, 365–372. doi: 10.1111/j.1467-8519.2012.01961.x

PubMed Abstract | CrossRef Full Text | Google Scholar

FIP/WHO (2011). Joint FIP/WHO Guidelines on Good Pharmacy Practice: Standards for Quality of Pharmacy Services. WHO Technical Report Series. No.96. Annex8:14.

Foreman, K. F., Stockl, K. M., Le, L. B., Fisk, E., Shah, S. M., Lew, H. C., et al. (2012). Impact of a text messaging pilot program on patient medication adherence. Clin. Ther. 34, 1084–1091. doi: 10.1016/j.clinthera.2012.04.007

PubMed Abstract | CrossRef Full Text | Google Scholar

Fortney, J. C., Pyne, J. M., Kimbrell, T. A., Hudson, T. J., Robinson, D. E., Schneider, R., et al. (2015). Telemedicine-based collaborative care for posttraumatic stress disorder: a randomized clinical trial. JAMA Psychiatry 72, 58–67. doi: 10.1001/jamapsychiatry.2014.1575

PubMed Abstract | CrossRef Full Text | Google Scholar

Fortney, J. C., Pyne, J. M., Mouden, S. B., Mittal, D., Hudson, T. J., Schroeder, G. W., et al. (2013). Practice-based versus telemedicine-based collaborative care for depression in rural federally qualified health centers: a pragmatic randomized comparative effectiveness trial. Am. J. Psychiatry 170, 414–425. doi: 10.1176/appi.ajp.2012.12050696

PubMed Abstract | CrossRef Full Text | Google Scholar

Fox, B. I., Flynn, A. J., Fortier, C. R., and Clauson, K. A. (2011). Knowledge, skills, and resources for pharmacy informatics education. Am. J. Pharm. Educ. 75:93. doi: 10.5688/ajpe75593

PubMed Abstract | CrossRef Full Text | Google Scholar

Franklin, B. D. (2017). Smart'intravenous pumps: how smart are they? BMJ Qual. Saf. 26, 93–94. doi: 10.1136/bmjqs-2016-005302

PubMed Abstract | CrossRef Full Text | Google Scholar

Gatwood, J., Balkrishnan, R., Erickson, S. R., An, L. C., Piette, J. D., and Farris, K. B. (2016). The impact of tailored text messages on health beliefs and medication adherence in adults with diabetes: a randomized pilot study. Res. Soc. Adm. Pharm. 12, 130–140. doi: 10.1016/j.sapharm.2015.04.007

PubMed Abstract | CrossRef Full Text | Google Scholar

Gerhart, D. Jr., O'Shea, K., and Muller, S. (2013). Advancing medication infusion safety through the clinical integration of technology. Hosp. Pract. 41, 7–14. doi: 10.3810/hp.2013.10.1075

PubMed Abstract | CrossRef Full Text | Google Scholar

Gordon, H. L., Hoeber, M., and Schneider, A. (2012). Telepharmacy in a rural Alberta community cancer network. J. Oncol. Pharm. Pract. 18, 366–376. doi: 10.1177/1078155211431858

PubMed Abstract | CrossRef Full Text | Google Scholar

Goundrey-Smith, S. (2013). Information Technology in Pharmacy. 1st Edn, London: Springer.

Goundrey-Smith, S. (2014). Examining the role of new technology in pharmacy: now and in the future. Pharm. J. 292:11. doi: 10.1211/PJ.2014.11134174

CrossRef Full Text

Gregorio, J., Ferreira, T. L., Cavaco, A., da Silva, M. M., Lovis, C., and Lapão, L. V. (2013). “Community pharmacies and ehealth services: barriers and opportunities for real primary healthcare integration,” in IEEE 26th International Symposium on Computer-Based Medical Systems (CBMS) (Porto), 393–396.

Google Scholar

Gustafson, D., Wise, M., Bhattacharya, A., Pulvermacher, A., Shanovich, K., Phillips, B., et al. (2012). The effects of combining Web-based eHealth with telephone nurse case management for pediatric asthma control: a randomized controlled trial. J. Med. Internet Res. 14:e101. doi: 10.2196/jmir.1964

PubMed Abstract | CrossRef Full Text | Google Scholar

Harding, A. D. (2012). Increasing the use of “smart” pump drug libraries by nurses: a continuous quality improvement project. Am. J. Nurs. 112, 26–35. doi: 10.1097/01.NAJ.0000410360.20567.55

PubMed Abstract | CrossRef Full Text | Google Scholar

Hepler, C. D., and Strand, L. M. (1990). Opportunities and responsibilities in pharmaceutical care. Am. J. Hosp. Pharm. 47, 533–543.

PubMed Abstract | Google Scholar

Higgins, J. P., and Green, S. (2008). Cochrane Handbook for Systematic Reviews of Interventions. Vol 5, Chichester: Wiley-Blackwell.

Google Scholar

Kennerly, J., Jenkins, A., Lewis, A. N., and Eckel, S. F. (2012). Implementing smart pumps for epidural infusions in an academic medical center. Am. J. Health Syst. Pharm. 69, 607–611. doi: 10.2146/ajhp110477

PubMed Abstract | CrossRef Full Text | Google Scholar

Kmet, L. M., Lee, R. C., and Cook, L. S. (2004). Standard Quality Assessment Criteria for Evaluating Primary Research Papers From a Variety of Fields. Available online at: https://era.library.ualberta.ca/items/48b9b989-c221-4df6-9e35-af782082280e/view/885da76e-055a-4ee2-8073-87319b6d2abe/standard_quality_assessment_criteria_for_evaluating_primary_research_papers_from_a_variety_of_fields.pdf (Accessed August 03, 17).

Kooy, M. J., Van Geffen, E. C., Heerdink, E. R., Van Dijk, L., and Bouvy, M. L. (2015). Patients' general satisfaction with telephone counseling by pharmacists and effects on satisfaction with information and beliefs about medicines: results from a cluster randomized trial. Patient Educ. Couns. 98, 797–804. doi: 10.1016/j.pec.2015.02.020

PubMed Abstract | CrossRef Full Text | Google Scholar

Koppel, R., Metlay, J. P., Cohen, A., Abaluck, B., Localio, A. R., Kimmel, S. E., et al. (2008). Role of computerized physician order entry systems in facilitating medication errors. JAMA 293, 1197–1203. doi: 10.1001/jama.293.10.1197

PubMed Abstract | CrossRef Full Text | Google Scholar

Lapão, L. V., Gregorio, J., Ferreira, T., Cavaco, A., Lovis, C., Russo, G., et al. (2013). “eHealth services for enhanced pharmaceutical care provision: from counseling to patient education,” in IEEE 2nd International Conference on Serious Games and Applications for Health (SeGAH) (Vilamoura), 1–7.

Manrique-Rodríguez, S., Sánchez-Galindo, A. C., López-Herce, J., Calleja-Hernández, M. Á., Martínez-Martínez, F., Iglesias-Peinado, I., et al. (2013). Impact of implementing smart infusion pumps in a pediatric intensive care unit. Am. J. Health Syst. Pharm. 70, 1897–1906. doi: 10.2146/ajhp120767

PubMed Abstract | CrossRef Full Text | Google Scholar

Margolis, A., Young, H., Lis, J., Schuna, A., and Sorkness, C. A. (2013). A telepharmacy intervention to improve inhaler adherence in veterans with chronic obstructive pulmonary disease. Am. J. Health Syst. Pharm. 70, 1875–1876. doi: 10.2146/ajhp120241

PubMed Abstract | CrossRef Full Text | Google Scholar

Margolis, K. L., Kerby, T. J., Asche, S. E., Bergdall, A. R., Maciosek, M. V., O'Connor, P. J., et al. (2012). Design and rationale for home blood pressure telemonitoring and case management to control hypertension (HyperLink): a cluster randomized trial. Contemp. Clin. Trials 33, 794–803. doi: 10.1016/j.cct.2012.03.014

PubMed Abstract | CrossRef Full Text | Google Scholar

Martinez, A. S., Saef, J., Paszczuk, A., and Bhatt-Chugani, H. (2013). Implementation of a pharmacist-managed heart failure medication titration clinic. Am. J. Health Syst. Pharm. 70, 1070–1076. doi: 10.2146/ajhp120267

PubMed Abstract | CrossRef Full Text | Google Scholar

McFarland, M., Davis, K., Wallace, J., Wan, J., Cassidy, R., Morgan, T., et al. (2012). Use of home telehealth monitoring with active medication therapy management by clinical pharmacists in veterans with poorly controlled type 2 diabetes mellitus. Pharmacother. J. Hum. Pharmacol. Drug Ther. 32, 420–426. doi: 10.1002/j.1875-9114.2011.01038.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Migliozzi, D. R., Zullo, A. R., Collins, C., and Elsaid, K. A. (2015). Achieving blood pressure control among renal transplant recipients by integrating electronic health technology and clinical pharmacy services. Am. J. Health Syst. Pharm.. 72, 1987–1992. doi: 10.2146/ajhp140810

PubMed Abstract | CrossRef Full Text | Google Scholar

Miranda, A. C., Serag-Bolos, E. S., Aungst, T. D., and Chowdhury, R. (2014). A mobile health technology workshop to evaluate available technologies and their potential use in pharmacy practice. BMJ Simul. Technol. Enhanced Learn. 2, 23–26. doi: 10.1136/bmjstel-2015-000067

CrossRef Full Text | Google Scholar

Mitka, M. (2013). Joint commission warns of alarm fatigue: multitude of alarms from monitoring devices problematic. JAMA 309, 2315–2316. doi: 10.1001/jama.2013.6032

PubMed Abstract | CrossRef Full Text | Google Scholar

Mort, M., Roberts, C., and Callén, B. (2013). Ageing with telecare: care or coercion in austerity? Sociol. Health Illn. 35, 799–812. doi: 10.1111/j.1467-9566.2012.01530.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Owsley, C., McGwin, G., Lee, D. J., Lam, B. L., Friedman, D. S., Gower, E. W., et al. (2015). Diabetes eye screening in urban settings serving minority populations: detection of diabetic retinopathy and other ocular findings using telemedicine. JAMA Opthalmol. 133, 174–181. doi: 10.1001/jamaophthalmol.2014.4652

PubMed Abstract | CrossRef Full Text | Google Scholar

Páez-Avilés, C., Juanola-Feliu, E., Tahirbegi, I. B., Mir, M., and Samitier, J. (2015). “CoPs in innovation and technology transfer of medical devices,” in ISPIM Conference Proceedings, The International Society for Professional Innovation Management (ISPIM) (Budapest), 1.

Patterson, E. S., Cook, R. I., and Render, M. L. (2002). Improving patient safety by identifying side effects from introducing bar coding in medication administration. J. Am. Med. Inform. Assoc. 9, 540–553. doi: 10.1197/jamia.M1061

PubMed Abstract | CrossRef Full Text | Google Scholar

Philip, A., Green, M., Hoffman, T., Gautreaux, S., Wallace, D., Roux, R., et al. (2015). Expansion of clinical pharmacy through increased use of outpatient pharmacists for anticoagulation services. Am. J. Health Syst. Pharm. 72, 568–572. doi: 10.2146/ajhp140404

PubMed Abstract | CrossRef Full Text | Google Scholar

Roulstone, A. (2016). “Disability, ageing and technology: they think that throwing a pendant alarm at you equals independence,” in Disability and Technology, ed A. Roulstone (London: Palgrave Macmillan), 53–179.

Google Scholar

Sarzynski, E., Decker, B., Thul, A., Weismantel, D., Melaragni, R., Cholakis, E., et al. (2017). Beta testing a novel smartphone application to improve medication adherence. Telemed. J. E. Health 23, 339–348. doi: 10.1089/tmj.2016.0100

PubMed Abstract | CrossRef Full Text | Google Scholar

Sayner, R., Carpenter, D. M., Blalock, S. J., Robin, A. L., Muir, K. W., Hartnett, M. E., et al. (2015). Accuracy of patient-reported adherence to glaucoma medications on a visual analog scale compared with electronic monitors. Clin. Ther. 37, 1975–1985. doi: 10.1016/j.clinthera.2015.06.008

PubMed Abstract | CrossRef Full Text | Google Scholar

Schneider, P. J. (2013). Evaluating the impact of telepharmacy. Am. J. Health Syst. Pharm. 70, 2130–2135. doi: 10.2146/ajhp130138

PubMed Abstract | CrossRef Full Text | Google Scholar

Shane-McWhorter, L., McAdam-Marx, C., Lenert, L., Petersen, M., Woolsey, S., Coursey, J. M., et al. (2015). Pharmacist-provided diabetes management and education via a telemonitoring program. JAMA 55, 516–526. doi: 10.1331/JAPhA.2015.14285

PubMed Abstract | CrossRef Full Text | Google Scholar

Shuren, J., and Califf, R. M. (2016). Need for a national evaluation system for health technology. JAMA 316, 1153–1154 doi: 10.1001/jama.2016.8708

PubMed Abstract | CrossRef Full Text | Google Scholar

Singh, L. G., Accursi, M., and Black, K. K. (2015). Implementation and outcomes of a pharmacist-managed clinical video telehealth anticoagulation clinic. Am. J. Health Syst. Pharm. 72, 70–73. doi: 10.2146/ajhp130750

PubMed Abstract | CrossRef Full Text | Google Scholar

Siska, M. H., and Tribble, D. A. (2011). Opportunities and challenges related to technology in supporting optimal pharmacy practice models in hospitals and health systems. Am. J. Health Syst. Pharm. 68, 1116–1126. doi: 10.2146/ajhp110059

PubMed Abstract | CrossRef Full Text | Google Scholar

Snuggs, S., McRobbie, H., Myers, K., Schmocker, F., Goddard, J., and Hajek, P. (2012). Using text messaging to prevent relapse to smoking: intervention development, practicability and client reactions. Addiction 107, 39–44. doi: 10.1111/j.1360-0443.2012.04084.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Sparks, K. B. (2011). Medication adherence technology: medicine of the future, emerging privacy concern. J. Contemp. Health Law Policy 28:324.

PubMed Abstract | Google Scholar

Speedie, S. M., Ferguson, A. S., Sanders, J., and Doarn, C. R. (2008). Telehealth: the promise of new care delivery models. Telemed. J. E Health 14, 964–967. doi: 10.1089/tmj.2008.0114

PubMed Abstract | CrossRef Full Text | Google Scholar

Spinks, J., Jackson, J., Kirkpatrick, C. M., and Wheeler, A. J. (2016). Disruptive innovation in community pharmacy–Impact of automation on the pharmacist workforce. Res. Soc. Adm. Pharm. 13, 394–397. doi: 10.1016/j.sapharm.2016.04.009

PubMed Abstract | CrossRef Full Text | Google Scholar

Steinhubl, S. R., Muse, E. D., and Topol, E. J. (2015). The emerging field of mobile health. Sci. Transl. Med. 7:283rv3. doi: 10.1126/scitranslmed.aaa3487

PubMed Abstract | CrossRef Full Text | Google Scholar

Steventon, A., Bardsley, M., Billings, J., Dixon, J., Doll, H., Beynon, M., et al. (2013). Effect of telecare on use of health and social care services: findings from the Whole Systems Demonstrator cluster randomised trial. Age Ageing 42, 501–508. doi: 10.1093/ageing/aft008

PubMed Abstract | CrossRef Full Text | Google Scholar

Sun, J., and Qu, Z. (2015). Understanding health information technology adoption: a synthesis of literature from an activity perspective. Inform. Syst. Front. 17:1177–1190. doi: 10.1007/s10796-014-9497-2

CrossRef Full Text | Google Scholar

The Economist (2016). Things are Looking App: Mobile Health Apps are Becoming More Capable and Potentially Rather Useful. Available online at: https://www.economist.com/news/business/21694523-mobile-health-apps-are-becoming-more-capable-and-potentially-rather-useful-things-are-looking (Accessed August 03, 17).

Toklu, H. Z. (2015). Promoting evidence-based practice in pharmacies. Integr. Pharm. Res. Pract. 4, 127–131. doi: 10.2147/IPRP.S70406

PubMed Abstract | CrossRef Full Text | Google Scholar

Van Gemert-Pijnen, J. E., Nijland, N., Van Limburg, M., Ossebaard, H. C., Kelders, S. M., Eysenbach, G., et al. (2011). A holistic framework to improve the uptake and impact of eHealth technologies. J. Med. Internet. Res. 13:e111. doi: 10.2196/jmir.1672

PubMed Abstract | CrossRef Full Text | Google Scholar

Van Onzenoort, H. A., Neef, C., Verberk, W. W., Van Iperen, H. P., De Leeuw, P. W., and van der Kuy, P. H. (2012). Determining the feasibility of objective adherence measurement with blister packaging smart technology. Am. J. Health Syst. Pharm. 69, 872–879. doi: 10.2146/ajhp100592

PubMed Abstract | CrossRef Full Text | Google Scholar

Vasbinder, E. C., Dahhan, N., Wolf, B., Zoer, J., Blankman, E., Bosman, D., et al. (2013). The association of ethnicity with electronically measured adherence to inhaled corticosteroids in children. Eur. J. Clin. Pharmacol. 69, 683–690. doi: 10.1007/s00228-012-1380-9

PubMed Abstract | CrossRef Full Text | Google Scholar

Ventola, C. L. (2014). Mobile devices and apps for health care professionals: uses and benefits. Pharm Ther. 39, 356–364.

PubMed Abstract | Google Scholar

Verbosky, N., Beckey, C., and Lutfi, N. (2016). Implementation and evaluation of diabetes management via clinical video telehealth. Diabetes Care 39, 1–2. doi: 10.2337/dc15-1391

PubMed Abstract | CrossRef Full Text | Google Scholar

Wang, V., Smith, V. A., Bosworth, H. B., Oddone, E. Z., Olsen, M. K., McCant, F., et al. (2012). Economic evaluation of telephone self-management interventions for blood pressure control. Am. Heart J. 163, 980–986. doi: 10.1016/j.ahj.2012.03.016

PubMed Abstract | CrossRef Full Text | Google Scholar

Wapner, J. (2016). Technology transfer: the leap to industry. Nature 533, S13–S15. doi: 10.1038/533S13a

PubMed Abstract | CrossRef Full Text | Google Scholar

Wessner, C. W. (2005). Driving innovations across the valley of death. Res. Technol. Manage 48, 9–12. doi: 10.1080/08956308.2005.11657289

CrossRef Full Text | Google Scholar

Wilcox, L., Woollen, J., Prey, J., Restaino, S., Bakken, S., Feiner, S., et al. (2016). Interactive tools for inpatient medication tracking: a multi-phase study with cardiothoracic surgery patients. J. Am. Med. Inform. Assoc. 23, 144–158. doi: 10.1093/jamia/ocv160

PubMed Abstract | CrossRef Full Text | Google Scholar

Woods, P., Gapp, R., and King, M. A. (2015). Researching pharmacist managerial capability: philosophical perspectives and paradigms of inquiry. Res. Soc. Adm. Pharm. 11, 265–279. doi: 10.1016/j.sapharm.2014.07.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Young, H. N., Havican, S. N., Griesbach, S., Thorpe, J. M., Chewning, B. A., and Sorkness, C. A. (2012). Patient and phaRmacist telephonic encounters (PARTE) in an underserved rural patient population with asthma: results of a pilot study. Telemed. J. E Health 18, 427–433. doi: 10.1089/tmj.2011.0194

PubMed Abstract | CrossRef Full Text | Google Scholar

Zullig, L. L., McCant, F., Melnyk, S. D., Danus, S., and Bosworth, H. B. (2014). A health literacy pilot intervention to improve medication adherence using Meducation^® technology. Patient Educ. Couns. 95, 288–291. doi: 10.1016/j.pec.2014.02.004

PubMed Abstract | CrossRef Full Text | Google Scholar