Drug-drug interactions with oral anticoagulants: information consistency assessment of three commonly used online drug interactions databases in Switzerland

Background: Toxicity or treatment failure related to drug-drug interactions (DDIs) are known to significantly affect morbidity and hospitalization rates. Despite the availability of numerous databases for DDIs identification and management, their information often differs. Oral anticoagulants are deemed at risk of DDIs and a leading cause of adverse drug events, most of which being preventable. Although many databases include DDIs involving anticoagulants, none are specialized in them.

Aim and method: This study aims to compare the DDIs information content of four direct oral anticoagulants and two vitamin K antagonists in three major DDI databases used in Switzerland: Lexi-Interact, Pharmavista, and MediQ. It evaluates the consistency of DDIs information in terms of differences in severity rating systems, mechanism of interaction, extraction and documentation processes and transparency.

Results: This study revealed 2’496 DDIs for the six anticoagulants, with discrepant risk classifications. Only 13.2% of DDIs were common to all three databases. Overall concordance in risk classification (high, moderate, and low risk) was slight (Fleiss’ kappa = 0.131), while high-risk DDIs demonstrated a fair agreement (Fleiss’ kappa = 0.398). The nature and the mechanism of the DDIs were more consistent across databases. Qualitative assessments highlighted differences in the documentation process and transparency, and similarities for availability of risk classification and references.

Discussion: This study highlights the discrepancies between three commonly used DDI databases and the inconsistency in how terminology is standardised and incorporated when classifying these DDIs. It also highlights the need for the creation of specialised tools for anticoagulant-related interactions.

1 Introduction

Drug-drug interactions (DDIs) are of significant concern in clinical practice, especially for drug classes at particular risk (McDonnell and Jacobs, 2002; Becker et al., 2007). Oral anticoagulants are particularly noteworthy, with several studies identifying them as a leading cause of adverse drug events (ADEs), resulting in increased morbidity, mortality and hospitalization (Budnitz et al., 2011; Jolivot et al., 2014; Shehab et al., 2016; Oscanoa et al., 2017; Ayalew et al., 2019). It has been reported that at least half of ADEs associated with oral anticoagulants, including DDIs, can be prevented (Sennesael et al., 2018; Spada et al., 2020). Vitamins K antagonists (VKAs) were the first oral anticoagulants introduced on the market and are still widely prescribed despite their narrow therapeutic range that requires constant monitoring. A large body of quality evidence shows that VKAs are subject to numerous drug and food interactions (Penning-van Beest et al., 2001; Holbrook et al., 2005; Violi et al., 2016). The introduction of direct oral anticoagulants (DOACs) offered an interesting alternative to VKAs. DOACs exhibit a more predictable dose-concentration-response relationship, have a broader therapeutic index, and do not require routine monitoring for dosage adjustment (Ageno et al., 2012; Cabral, 2013; Nutescu et al., 2016). While this therapeutic class was initially reported to be less prone to DDIs (Di Minno et al., 2017), recent findings showed that interactions involving DOACs are associated with significant risks of ADEs (Lee et al., 2020; Li et al., 2020; Terrier et al., 2021). The lack of direct monitoring complicates DDI management.

To address DDIs in daily clinical practice, numerous computer decisions support tools and online databases have been developed (Clauson et al., 2007; Roblek et al., 2015). A primary concern with these electronic DDIs databases is the overwhelming number of DDIs leading to alerts, many of which have minimal clinical relevance. This leads to an “over-alerting” effect on prescribers, diminishing their vigilance and obscuring the true significance of the DDIs (Glassman et al., 2002; Paterno et al., 2009; Smithburger et al., 2011; Roblek et al., 2015). Furthermore, the information across these databases often lacks consistency. They frequently present conflicting advice, varying severity classifications, and significant differences in sensitivity and specificity for detecting DDIs (Abarca et al., 2006; Clauson et al., 2007; Vonbach et al., 2008; Sweidan et al., 2009; Reis and Cassiani, 2010; Kongsholm et al., 2015; Roblek et al., 2015). Understanding the risk associated with DDIs involving oral anticoagulants is of major importance to minimize the risk of hemorrhages or thromboses (Ufer, 2005; Vazquez, 2018; Hanigan et al., 2020; Li et al., 2020; Gronich et al., 2021; Holm et al., 2021; Grymonprez et al., 2023). While several databases list DDIs associated with oral anticoagulants, none specialize exclusively in this category. The aim of this cross-sectional study was to compare the DDIs information content of four direct oral anticoagulants (DOACs) apixaban, rivaroxaban, edoxaban and dabigatran and two vitamin K antagonists (VKAs) acenocoumarol and phenprocoumon in the three predominant online databases used in clinical practice in Switzerland: Lexi-Interact, Pharmavista and MediQ. The specific aims were to assess the consistency of the DDI information provided and to evaluate the information content, transparency of ownership, sources of funding, level of documentation and staff qualifications associated with each of these three databases.

2 Materials and methods

2.1 Study design, databases and drugs selection

We conducted a cross-sectional study with the aim of comparing oral anticoagulant DDIs among three well-established databases commonly used in Switzerland, namely Lexi-Interact, Pharmavista and MediQ. Pharmavista is known to be widely used in community pharmacies, and both Lexi-Interact and MediQ are favored in hospital settings in the French (Vonbach et al., 2008) and German parts (Frölich et al., 2011; Guzek et al., 2011; Haueis et al., 2011; Zorina et al., 2012; Zorina et al., 2013), respectively. The study design involved the assessment of various parameters, including the number and nature of DDIs, risk classification, interaction type and mechanism, as well as information available on each database’s website. Our study focused on the four DOACs available in Switzerland: edoxaban, rivaroxaban, apixaban, and dabigatran. Additionally, we examined the two VKAs marketed in Switzerland: acenocoumarol which is the VKA of choice in French-speaking Switzerland, and phenprocoumon, which is widely prescribed in the German-speaking areas (Niederer et al., 2001; Bochatay et al., 2016).

2.2 Data extraction

Data extraction covered the period from July 2021 to January 2022. All available interacting compounds were considered, including food, herbal products, dietary supplements, tobacco, and ethanol. The interaction lists obtained from the three DDI databases were compiled in a descriptive table summarizing the following items: (1) the international non-proprietary name (INN), (2) the therapeutic group according to the ATC (Anatomical Therapeutic Chemical) classification), (3) the type of interaction [i.e. pharmacokinetic (PK) and/or pharmacodynamic (PD)], and (4) details of the interaction mechanism. In parallel, a qualitative assessment of the quality and transparency of ownership, funding, information, classifications, staff training and underlying documentation of the DDIs database was performed by retrieving the following information from the websites: formal ownership and funding, information on potential conflict of interest, presence of a DDI interaction classification, availability of a procedure for categorizing the severity of DDIs, definition and procedure for categorizing the level of evidence for DDIs, availability of scientific references, staff qualifications, and information on database updates. Secondly, we sought to recover the missing information by contacting the DDIs database managers.

2.3 Data analysis

The sources consulted were compared in terms of similarities in the number and in the risk classification of DDIs detected by each database. The number of DDIs in each database, the number of similar DDIs in two and in the three databases, and the proportion of DDIs considering the total number of retrieved DDIs in all three databases were assessed. Information on the nature of the DDIs (PK and PD) and the interaction mechanism described in each database were also compared. A further analysis was performed by comparing the number of DDIs classified as “high-risk” by one, two, or all three databases, together with a description of the molecule names classified similarly or divergently among them. Hypotheses underlying the potential differences in classification were also made for the discrepant DDIs. Quantitative results were reported as means and proportions as appropriate.

Given that each DDI database employs a different rating system for assessing the severity of interactions (four risk categories for MediQ, five for Lexi-Interact, and six for Pharmavista), we constructed an equivalent risk-level classification, which is provided in Table 1.

Table 1

Table 1. Drug-drug interaction severity rating systems and equivalence of risk levels between Lexi-Interact, Pharmavista, and MediQ databases.

The overall concordance in the equivalent risk-level classification provided by the three DDI databases was evaluated using the Fleiss’ kappa and Gwet’s AC1 coefficient, while Cohen’s kappa coefficient was used to determine the pairwise classification concordance among two lists. The following degrees of agreement for kappa coefficients suggested by Landis and Koch (Landis and Koch, 1977) were used: a kappa value > 0.80 was considered “almost perfect,” 0.61–0.80, “substantial,” 0.41–0.60, “moderate,” 0.21–0.40, “fair,” 0.00–0.20, “slight” and < 0.00, “poor” (Landis and Koch, 1977). A p-value associated with the kappa coefficient was also calculated, with a p-value <0.05 indicating statistically significant agreement. Data management and analyses were performed with STATA (StataCorp. 2021. Stata Statistical Software: Release 17. College Station, TX: StataCorp LLC). Inter-rater agreement for nominal categorial variables (Cohen’s kappa, Fleiss’ kappa, and Gwet’s AC1 with 95% CI) was calculated with a STATA specific plug-in.

2.4 Qualitative assessment of available database website information

We conducted a qualitative analysis of the information available on each database’s website using a questionnaire derived from the systematic review by Kongholm et al. (Kongsholm et al., 2015). We then assessed whether specific information was present or absent across the three DDI databases.

3 Results

3.1 Overall drug interactions

In total, 2′496 DDIs were retrieved from the three databases for the six oral anticoagulants studied: 335 DDIs for apixaban, 316 for dabigatran, 355 for rivaroxaban, 287 for edoxaban, 592 for acenocoumarol and 613 for phenprocoumon. Lexi-Interact, Pharmavista and MediQ provided, respectively 76.6% (n = 1′912), 41.3% (n = 1′031) and 31.4% (n = 783) of total DDIs. Overall, only 13.2% (n = 330) of DDIs were common to all three databases, whereas 23.0% (n = 571) were common to two databases and 63.8% (n = 1′594) were present in only one source. Table 2 presents the number and proportion of DDIs for each anticoagulant.

Table 2

Table 2. Comparison of total and high, moderate and low-risk drug-drug interactions (DDIs) identified by one or more DDIs databases.

The classification of the DDI risk level differed between the three databases. Lexi-Interact classified DDIs as moderate in a larger proportion of cases than Pharmavista, whereas the latter showed the highest proportion of high-risk DDI. On the other hand, MediQ showed the highest rate of low risk DDI (Figure 1). Similar proportions were observed between DOACs and VKAs. The description of DDI classification detected by the 3 databases, each using its own level of classification is presented in Supplementary Table SA.

Figure 1

Figure 1. Proportion of drug-drug interaction according to severity in each database: DOAC (A), VKA (B).

Our analysis indicates that 55.4% (i.e., 63.2% for DOACs and 47.1% for VKAs) of the discrepancies are related to the classification of DDIs by a single DDI database. For DDIs classified by at least two of the three DDI databases, the majority of DDI discrepancies are congruent on the nature of the DDI, namely PK, PD, or unknown mechanisms, with only 16 discrepancies (4 for DOAC and 12 for VKA) in at least two of the three databases. For tipranavir (with apixaban or acenocoumarol), azithromycine (with apixaban), carvedilol (with rivaroxaban), levetiracetam (with rivaroxaban) and fusidic acid (with phenprocoumon) (n = 5), the mismatch concerned the PK or PD etiology of the interaction mechanism; for ceritinib, chloramphenicol and ifosfamide with phenprocoumon (n = 3), the discrepancy concerned the cytochrome linked to PK interaction, and the remaining 8 discrepancies were due to unidentified interaction mechanism in one database, while others suggested PK or PD causes. These discrepancies pertain to the following drugs: celecoxib, disulfiram, doxycycline, tamoxifen (n = 2), quinidine, eplerenone, and oxacillin in interaction with VKA. The detailed description of the divergent DDIs mechanism of the 16 drugs across the three databases is summarized in Supplementary Table SB.

3.2 High-risk DDIs classification comparison

A total of 207 and 68 high-risk DDIs for DOAC and VKA were retrieved, respectively (Table 2). Molecules that were classified as having a high-risk of DDI by the three databases are other anticoagulants with an additive effect on hemostasis (PD interaction) and strong inhibitors or inducers of the cytochromes and/or Pgp involved in the metabolism of the DOAC and the VKA (i.e., phenytoin, ketoconazole, itraconazole, rifampicin, St John’s wort (Hyperium Perforatum), resulting in a change in drug exposure and, as a consequence, a lack of efficacy or toxicity. Figure 2 presents the molecules classified as high-risk in two or three databases. A listing of high-risk DDIs with varying classifications across the three databases is presented in Figure 3.

Figure 2

Figure 2. Description of molecule names reported as high-risk drug-drug interaction by either two or all three databases.

Figure 3

Figure 3. List of high-risk drug-drug interactions (DDIs) according to at least one database, with their classifications across the other two databases. CYP, Cytochrome P450; Pgp, P-glycoprotein.

3.3 Agreement between risk classification among the DDI databases

The Fleiss’ kappa and the Gwet’s AC1 values for comparing the three databases was 0.131 [0.080, 0.183, p < 0.001], and 0.324 [0.276, 0.372, p < 0.001] which corresponds to a slight and a fair level of agreement, respectively, according to Landis and Koch (Landis and Koch, 1977). The classification comparison for high-risk DDIs showed a higher Fleiss’ kappa value of 0.398 (p < 0.05), which represents a fair level of agreement (Landis and Koch, 1977). Table 3 summarizes the agreement for low, moderate, and high-risk of DDI.

Table 3

Table 3. Agreement among three drug-drug interaction (DDI) databases.

The Cohen’s kappa values showed a slight classification agreement between all databases pairs. A more consistent agreement was found between Pharmavista and MediQ with a Cohen’s kappa coefficient of 0.277 [95% CI 0.198, 0.356, p < 0.05] which corresponds to a fair level of agreement (Landis and Koch, 1977). Cohen’s kappa coefficient between Lexi-Interact and Pharmavista and between Lexi-Interact and MediQ was 0.177 [95% CI 0.137, 0.218, p < 0.05] and 0.140 [95% CI 0.079, 0.201, p < 0.05] respectively, which corresponds to a slight level of agreement.

3.4 Qualitative assessment of available database website information

Table 4 summarizes results of the qualitative assessment of the DDI databases. Information details concerning the three drug-drug interaction databases is available in Supplementary Table SC. All three databases offered severity classification, but a standard operating procedure for the categorization of DDI severity was only provided by Lexi-Interact. Additionally, only Lexi-Interact defined a level of evidence, also including information on the procedure used for categorizing the level of evidence. References were available in all databases.

Table 4

Table 4. Qualitative assessment of the information available on each database website.

Upon request, Lexi-Interact provided a comprehensive overview of the data extraction process and supplied detailed information, including author names, their respective professional affiliations, and collaborating faculties for each medical specialty. Updates of its content are made by a continual review of new drug approvals, revisions to product labeling, and surveillance of published literature. Pharmavista outlined the qualifications of the staff responsible for reviewing DDIs. It did not provide the procedures of classification of the DDI severity to the reader, but these processes seemed available for their staff in their “working instructions.” The classification was based on PK and PD considerations, with reference to professional information, published literature, with regular updates performed by an experienced team of physicians and pharmacists. We did not have any further information by MediQ.

4 Discussion

Our research highlighted notable inconsistencies among three frequently used DDI databases concerning oral anticoagulants. Only a minor proportion of DDIs were consistently captured and categorized at the same risk level, raising particular concerns for the management of DDIs in clinical practice.

Our findings indicate a poor concordance rate of 23% between two and 13% among all three DDI databases, which are in line with previous reported studies examining other databases, drug classes or patient population (Vitry, 2007; Roblek et al., 2015; Shariff et al., 2021; Hecker et al., 2022; Kontsioti et al., 2022). For instance, a study conducted by Vonbach et al. reported a concordance of only 11% in DDIs involving the 100 most frequently used drugs at the cantonal Hospital of Baden among five databases, including Pharmavista and Lexi-Interact (Vonbach et al., 2008). Our results show Lexi-Interact provides the highest number of DDIs, which is consistent with a systematic review assessing the accuracy, completeness and user-friendliness of eight databases including Lexi-Interact, Pharmavista, and MediQ, that indeed reported that Lexi-Interact outperformed other databases in terms of completeness (Pauly et al., 2015).

Discrepancies between databases were mostly related to differences in the number of DDIs and their risk classification, rather than in the underlying mechanism of these interactions. The largest proportion of Lexi-Interact DDIs were classified as “moderate-risk,” whereas it was “low-risk” for MediQ, and Pharmavista listed approximately three times as many high-risk DDIs compared to the other databases. Lexi-Interact designates a DDI as “high-risk” only if it poses a life-threatening risk to the patient, providing a good sensitivity for drugs to absolutely avoid. Pharmavista seems more conservative in its way of classifying “high-risk” DDIs than Lexi-interact, and of the three databases, MediQ appears less concerned about DDIs, specially related to DOACs and to VKAs to a slightly lesser extent.

The Gwet’s AC1 coefficient was computed due to the imbalanced distribution among the observed categories. Similar results were obtained between Fleiss’ Kappa and Gwet’s AC1. Therefore, Fleiss’ Kappa was used to interpret the findings. The agreement levels between two or three DDI databases were generally low (i.e. 0.277 between Pharmavista and MediQ and 0.131 for the three databases), echoing earlier findings by Amkreutz et al., with a concordance coefficient of 0.23 [95% CI 0.11, 0.35] between Pharmavista and MediQ for DDIs focusing on immunosuppressants (Amkreutz et al., 2017). Similarly, Sheriff et al. reported a low level of consistency (0.174 [95% CI 0.145, 0.203]) across eight interaction databases, including Lexi-Interact, for the most commonly used drug pairs in the United Arab Emirates (Shariff et al., 2021). To our knowledge, no studies have been published that specifically compare DDI databases for oral anticoagulants.

Regarding high-risk DDIs, only 5.3% and 1.5% were common among the three databases for DOAC and VKA, respectively, which agrees with the literature (ThomasFulda et al., 2000; Abarca et al., 2004; Wong et al., 2008; Wang et al., 2010). A critical appraisal of the discrepancies showed that for PK DDIs, mostly inconsistencies arose on one hand from the uncertainty in the mechanism of the DDI due to the lack of informative PK data (e.g., chloral hydrate, orlistat) and on the other hand from the lack of robust information in the literature of the inhibitory or inducing potency of the interactor involved. According to the references used, a drug can indeed be classified as a strong, moderate or low CYP inhibitor, which influences as a consequence the risk classification. As an example, dronedarone is classified as high-risk by Pharmavista, moderate by Lexi-Interact and low risk by MediQ. Furthermore, in some cases, the risk associated with the molecule was not directly attributable to its own pharmacological properties, but rather to its combination or concomitant use with other molecules. This could explain some differences in classification for drugs associated by default to the same risk level of others of molecules of the same therapeutic class (e.g., antifungal agents) or for those always co-administered with pharmacokinetic boosters, but without intrinsic interaction (lopinavir w/ritonavir). For high-risk pharmacodynamic DDIs, unclear mechanisms of action contributed to varied classifications among the databases. Pharmavista, opting for a precautionary approach, tended to categorize uncertain mechanisms as severe due to limited information, as observed for example with the interaction between protein kinase inhibitors and a DOAC. Another point of divergence was the consideration of adding two anticoagulants together. While co-administration of two VKAs or two DOACs is clinically contraindicated, MediQ included such interactions in its database, while Pharmavista chose to omit them, considering it clinically implausible. Lexi-Interact regarded it as a common strategy during a VKA switch, though typically, switches occur between a DOAC and a VKA, not between two VKAs. Additionally, the underlying mechanism of varied for some drugs; For example, Lexi-Interact attributes the tipranavir-VKA interaction to an additional inhibition of platelet aggregation, whereas Pharmavista suggests probable Pgp inhibition.

Globally, there are several reasons explaining the discrepancies among the DDI databases. Firstly, each database adopts different criteria for the inclusion of DDI; some encompass all molecules within a therapeutic class, while others are more selective. Secondly, the references used to guide the classification of the severity scores do not seem to be homogeneous between the databases (Kongsholm et al., 2015), as a consequence of a mismatch between the information provided by the review of primary and secondary literature, unpublished data released by pharmaceutical companies, product labels, and reports collected by post-marketing surveillance reports. Thirdly, the lack of in vitro or in vivo data amplifies the heterogeneity of the knowledge of the nature, mechanism and potency of DDIs. Of particular concern is the lack of agreement between sources in the potency of CYP-mediated interactions. Such discrepancies can have significant clinical implications, as accurate knowledge of CYP-mediated interactions is very important for adjusting drug dosages and avoiding potential adverse reactions. Furthermore, the various classification levels of each database complexify the comparison and the interpretation of the provided information (Scheife et al., 2015). These discrepancies could be problematic in clinical practice, affecting decision-making (Paterno et al., 2009) and patient safety. This can lead to a loss of confidence in clinical decision support tools that incorporate DDI databases, and lead to different treatment decisions, depending on which database the healthcare professional relies on. It is important to recognize that DDIs constitute only one risk factor among several that may influence the PK and PD of oral anticoagulants. Additional risk factors include co-morbidities that can lead to DDIs with oral anticoagulants, such as hepatic and renal impairment (Chang et al., 2021; Canonico et al., 2022).

Our study highlighted a lack of transparency in several areas: the data extraction process, the methods used to categorize the severity of interactions, and potential conflicts of interest. Information was presented inconsistently across the DDI databases. Lexi-Interact was the only platform that provided its standard operating protocol for DDI severity categorization and defined a level of evidence for the information used to classify DDIs, setting a higher standard of transparency in this regard. Pharmavista was the only database to explain its data extraction process on its website, as it exercises control over the production of the expert information provided by manufacturers, further enhancing the transparency of its operations and providing insight into its data collection procedures. In a unique position, MediQ explicitly mentioned staff qualifications on its website, providing a clearer picture of the expertise involved in database management. However, despite these positive aspects, none of the databases provided information on potential conflicts of interest, highlighting an area where further transparency and disclosure could be beneficial in ensuring the integrity of the data and information provided to healthcare professionals and researchers. Our findings are corroborated by results from a systematic review conducted by Kongsholm et al. which assessed the online transparency regarding ownership, funding, information provision, classifications, staff training, and foundational documentation for eight databases (including Lexi-Interact). This review also highlighted the variations in content between databases and emphasized the importance of transparency in the processes of extraction, classification and standardization of the terminology used to classify DDIs (Kongsholm et al., 2015).This lack of transparency causes a black box effect and may prevent the prescriber from positioning himself correctly (Kongsholm et al., 2015), although suggestions have been made to standardize the assessment of evidence and clinical consequences (Far et al., 2012; Hines et al., 2012; Payne et al., 2015; Scheife et al., 2015).

These findings highlight the necessity for the clinical practitioner to cross-reference several sources of information. However, in clinical practice, time constraints often challenge the feasibility of such comprehensive cross-referencing. Several recommendations and strategies can be considered to improve the consistency and accuracy of drug interaction information (DDI) in online databases: the elaboration of a unique risk classification scale in order to help healthcare professionals and researchers to interpret the severity of DDIs in a consistent manner (Smithburger et al., 2011), an individual drug-based classification to minimize the risk of including in a risk classification drugs that have different class effects, and a collaborative data integration by combining data from multiple databases, involving experts groups and clinical experience to improve the congruence of DDI severity assessments through expert consensus classification (Smithburger et al., 2010; Smithburger et al., 2011). Our results suggest that a more specific DDI tool for oral anticoagulants should be developed, transparent in its approach for extraction and classification of the DDIs to enhance safe drug prescription of these high-risk drugs. Strategies have now been developed to help clinicians make decisions about DDIs with this therapeutic class. For example, consider switching to a VKA when a CYP3A4 or Pgp inhibitor leads to a doubling of the exposure (AUC) of a DOAC, or when an inducer leads to a reduction of at least 20% in its AUC (Steffel et al., 2021; Terrier et al., 2021). It is essential to target more accurately the DDIs that require action, to minimize the “over-alerting” effect commonly encountered with tools embedded in Clinical Decision Support Systems (CDSS).

Some limitations of our study should be addressed. To begin, the absence of a gold-standard limited the evaluation of each database’s risk categorization adequacy. In addition, the merge of the proposed risk levels into three equivalent risk level (high, moderate and low) might have increased the number of mismatches between the three databases. The use of uniform severity rating scales, as previously suggested, should be more widely accepted to simplify the comparisons (Amkreutz et al., 2017; Shariff et al., 2021; Kontsioti et al., 2022). We focused on the three databases mostly used in Switzerland, which could be considered as a limitation. However, our results are congruent with previous studies and the main message of a lack of concordance in the risk classification among software appears to be generalizable. Also, the absence of warfarin information that is not available on the Swiss market is a limitation. Yet, its potential for DDI is close to that of phenprocoumone, which limits this bias. Antithrombotics are at the forefront of the risk of DDIs. DOACs often show in real-world settings significant variations in dose-concentration response due to DDIs (Vazquez, 2018; Foerster et al., 2020). The clinical impact of inconsistent DDI information for these drugs as revealed by our analysis can be major, especially considering that antithrombotics are among the drugs with the highest risk of adverse events (Lee et al., 2020). Our research explored the complexity of these antithrombotic DDIs and the challenges of transparency in DDI databases. We found notable inconsistencies and gaps in transparency across the major drug databases, creating a “black box effect.” While some databases, such as Lexi-Interact, Pharmavista and MediQ, offer some degree of transparency, there is a glaring lack of consistency. This lack of consistency, particularly for oral anticoagulants, suggests an urgent need for a refined, transparent DDI tool tailored to these drugs. Our findings, novel in the comparison of DDI involving oral anticoagulants across three widely used clinical practice DDI databases, underscore the importance of providing clinicians with clear, reliable DDI information to ensure safe prescribing and patient health.

5 Conclusion

Preventing DDIs related to oral anticoagulants is of major relevance for ensuring patient safety. This study highlights the discrepancies between three commonly used DDI databases and the inconsistency in how terminology is standardized and incorporated when classifying these DDIs. The elaboration of a unique risk classification scale, an individual drug-based classification as well as a collaborative data integration combining data from multiple databases with the involvement of expert groups and clinical experience can improve consistency between all DDI database. Our findings emphasize the necessity of creating specialized tools for anticoagulant-related interactions. Such tools could improve the accuracy and clinical relevance of DDI alerts, thereby enhancing patient safety.

Data availability statement

The raw data supporting the conclusion of this article will be made available by the authors, without undue reservation.

Author contributions

CCo: Conceptualization, Data curation, Formal Analysis, Investigation, Writing–original draft, Writing–review and editing, Conceptualization, Data curation, Formal Analysis, Investigation, Writing–original draft, Writing–review and editing. FG: Writing–review and editing, Writing–review and editing. JT: Writing–review and editing, Writing–review and editing. AS-L: Writing–review and editing, Writing–review and editing. ML: Writing–review and editing, Writing–review and editing. M-AL: Writing–review and editing, Writing–review and editing. CCs: Funding acquisition, Supervision, Writing–review and editing, Funding acquisition, Supervision, Writing–review and editing.

Funding

The author(s) declare that financial support was received for the research, authorship, and/or publication of this article. This research was financed by the Swiss National Science Foundation, project no. 167381.

Acknowledgments

We kindly thank to all the authors who have contributed to the development of this study and the redaction of the article.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fphar.2024.1332147/full#supplementary-material

References

Abarca, J., Colon, L. R., Wang, V. S., Malone, D. C., Murphy, J. E., and Armstrong, E. P. (2006). Evaluation of the performance of drug-drug interaction screening software in community and hospital pharmacies. J. Manag. Care Pharm. 12 (5), 383–389. doi:10.18553/jmcp.2006.12.5.383

PubMed Abstract | CrossRef Full Text | Google Scholar

Abarca, J., Malone, D. C., Armstrong, E. P., Grizzle, A. J., Hansten, P. D., Van Bergen, R. C., et al. (2004). Concordance of severity ratings provided in four drug interaction compendia. J. Am. Pharm. Assoc. JAPhA 44 (2), 136–141. doi:10.1331/154434504773062582

PubMed Abstract | CrossRef Full Text | Google Scholar

Ageno, W., Gallus, A. S., Wittkowsky, A., Crowther, M., Hylek, E. M., and Palareti, G. (2012). Oral anticoagulant therapy: antithrombotic therapy and prevention of thrombosis, 9th ed: American college of chest physicians evidence-based clinical practice guidelines. Chest 141 (2), e44S–e88S. doi:10.1378/chest.11-2292

PubMed Abstract | CrossRef Full Text | Google Scholar

Amkreutz, J., Koch, A., Buendgens, L., Trautwein, C., and Eisert, A. (2017). Clinical decision support systems differ in their ability to identify clinically relevant drug interactions of immunosuppressants in kidney transplant patients. J. Clin. Pharm. Ther. 42 (3), 276–285. doi:10.1111/jcpt.12508

PubMed Abstract | CrossRef Full Text | Google Scholar

Ayalew, M. B., Tegegn, H. G., and Abdela, O. A. (2019). Drug related hospital admissions; A systematic review of the recent literature. Bull. Emerg. Trauma 7 (4), 339–346. doi:10.29252/beat-070401

PubMed Abstract | CrossRef Full Text | Google Scholar

Becker, M. L., Kallewaard, M., Caspers, P. W., Visser, L. E., Leufkens, H. G., and Stricker, B. H. (2007). Hospitalisations and emergency department visits due to drug-drug interactions: a literature review. Pharmacoepidemiol Drug Saf. 16 (6), 641–651. doi:10.1002/pds.1351

PubMed Abstract | CrossRef Full Text | Google Scholar

Bochatay, L., Beney, J., Jordan-von Gunten, V., Petignat, P. A., and Roulet, L. (2016). Appropriateness of the prescriptions of conventional versus new oral anticoagulants at discharge from a department of internal medicine. La Rev. Med. interne 37 (9), 579–586. doi:10.1016/j.revmed.2015.10.355

PubMed Abstract | CrossRef Full Text | Google Scholar

Budnitz, D. S., Lovegrove, M. C., Shehab, N., and Richards, C. L. (2011). Emergency hospitalizations for adverse drug events in older Americans. N. Engl. J. Med. 365 (21), 2002–2012. doi:10.1056/NEJMsa1103053

PubMed Abstract | CrossRef Full Text | Google Scholar

Cabral, K. P. (2013). Pharmacology of the new target-specific oral anticoagulants. J. Thromb. Thrombolysis 36 (2), 133–140. doi:10.1007/s11239-013-0929-5

PubMed Abstract | CrossRef Full Text | Google Scholar

Canonico, M. E., Sanna, G. D., Siciliano, R., Scudiero, F., Esposito, G., and Parodi, G. (2022). Drug-drug interactions between antithrombotics and direct-acting antivirals in hepatitis C virus (HCV) patients: a brief, updated report. Front. Pharmacol. 13, 916361. doi:10.3389/fphar.2022.916361

PubMed Abstract | CrossRef Full Text | Google Scholar

Chang, W. H., Mueller, S. H., Tan, Y. Y., and Lai, A. G. (2021). Antithrombotic therapy in patients with liver disease: population-based insights on variations in prescribing trends, adherence, persistence and impact on stroke and bleeding. Lancet Reg. Health Eur. 10, 100222. doi:10.1016/j.lanepe.2021.100222

PubMed Abstract | CrossRef Full Text | Google Scholar

Clauson, K. A., Marsh, W. A., Polen, H. H., Seamon, J. M., and Ortiz, B. (2007). Clinical decision support tools: analysis of online drug information databases. Med. Inf. Decis. Mak. 7, 7. doi:10.1186/1472-6947-7-7

PubMed Abstract | CrossRef Full Text | Google Scholar

Di Minno, A., Frigerio, B., Spadarella, G., Ravani, A., Sansaro, D., Amato, M., et al. (2017). Old and new oral anticoagulants: food, herbal medicines and drug interactions. Blood Rev. 31 (4), 193–203. doi:10.1016/j.blre.2017.02.001

PubMed Abstract | CrossRef Full Text | Google Scholar

Far, E., Curkovic, I., Byrne, K., Roos, M., Egloff, I., Dietrich, M., et al. (2012). Validation of a transparent decision model to rate drug interactions. BMC Pharmacol. Toxicol. 13, 7. doi:10.1186/2050-6511-13-7

PubMed Abstract | CrossRef Full Text | Google Scholar

Foerster, K. I., Hermann, S., Mikus, G., and Haefeli, W. E. (2020). Drug-drug interactions with direct oral anticoagulants. Clin. Pharmacokinet. 59 (8), 967–980. doi:10.1007/s40262-020-00879-x

PubMed Abstract | CrossRef Full Text | Google Scholar

Frölich, T., Zorina, O., Fontana, A. O., Kullak-Ublick, G. A., Vollenweider, A., and Russmann, S. (2011). Evaluation of medication safety in the discharge medication of 509 surgical inpatients using electronic prescription support software and an extended operational interaction classification. Eur. J. Clin. Pharmacol. 67 (12), 1273–1282. doi:10.1007/s00228-011-1081-9

PubMed Abstract | CrossRef Full Text | Google Scholar

Glassman, P. A., Simon, B., Belperio, P., and Lanto, A. (2002). Improving recognition of drug interactions: benefits and barriers to using automated drug alerts. Med. Care 40 (12), 1161–1171. doi:10.1097/00005650-200212000-00004

PubMed Abstract | CrossRef Full Text | Google Scholar

Gronich, N., Stein, N., and Muszkat, M. (2021). Association between use of pharmacokinetic-interacting drugs and effectiveness and safety of direct acting oral anticoagulants: nested case-control study. Clin. Pharmacol. Ther. 110 (6), 1526–1536. doi:10.1002/cpt.2369

PubMed Abstract | CrossRef Full Text | Google Scholar

Grymonprez, M., Vanspranghe, K., Steurbaut, S., De Backer, T. L., and Lahousse, L. (2023). Non-vitamin K antagonist oral anticoagulants (NOACs) versus warfarin in patients with atrial fibrillation using P-gp and/or CYP450-interacting drugs: a systematic review and meta-analysis. Cardiovasc Drugs Ther. 37 (4), 781–791. doi:10.1007/s10557-021-07279-8

PubMed Abstract | CrossRef Full Text | Google Scholar

Guzek, M., Zorina, O. I., Semmler, A., Gonzenbach, R. R., Huber, M., Kullak-Ublick, G. A., et al. (2011). Evaluation of drug interactions and dosing in 484 neurological inpatients using clinical decision support software and an extended operational interaction classification system (Zurich Interaction System). Pharmacoepidemiol Drug Saf. 20 (9), 930–938. doi:10.1002/pds.2197

PubMed Abstract | CrossRef Full Text | Google Scholar

Hanigan, S., Das, J., Pogue, K., Barnes, G. D., and Dorsch, M. P. (2020). The real world use of combined P-glycoprotein and moderate CYP3A4 inhibitors with rivaroxaban or apixaban increases bleeding. J. Thromb. Thrombolysis 49 (4), 636–643. doi:10.1007/s11239-020-02037-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Haueis, P., Greil, W., Huber, M., Grohmann, R., Kullak-Ublick, G. A., and Russmann, S. (2011). Evaluation of drug interactions in a large sample of psychiatric inpatients: a data interface for mass analysis with clinical decision support software. Clin. Pharmacol. Ther. 90 (4), 588–596. doi:10.1038/clpt.2011.150

PubMed Abstract | CrossRef Full Text | Google Scholar

Hecker, M., Frahm, N., Bachmann, P., Debus, J. L., Haker, M. C., Mashhadiakbar, P., et al. (2022). Screening for severe drug-drug interactions in patients with multiple sclerosis: a comparison of three drug interaction databases. Front. Pharmacol. 13, 946351. doi:10.3389/fphar.2022.946351

PubMed Abstract | CrossRef Full Text | Google Scholar

Hines, L. E., Malone, D. C., and Murphy, J. E. (2012). Recommendations for generating, evaluating, and implementing drug-drug interaction evidence. Pharmacotherapy 32 (4), 304–313. doi:10.1002/j.1875-9114.2012.01024.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Holbrook, A. M., Pereira, J. A., Labiris, R., McDonald, H., Douketis, J. D., Crowther, M., et al. (2005). Systematic overview of warfarin and its drug and food interactions. Archives Intern. Med. 165 (10), 1095–1106. doi:10.1001/archinte.165.10.1095

PubMed Abstract | CrossRef Full Text | Google Scholar

Holm, J., Mannheimer, B., Malmström, R. E., Eliasson, E., and Lindh, J. D. (2021). Bleeding and thromboembolism due to drug-drug interactions with non-vitamin K antagonist oral anticoagulants-a Swedish, register-based cohort study in atrial fibrillation outpatients. Eur. J. Clin. Pharmacol. 77 (3), 409–419. doi:10.1007/s00228-020-03015-7

PubMed Abstract | CrossRef Full Text | Google Scholar

Jolivot, P. A., Hindlet, P., Pichereau, C., Fernandez, C., Maury, E., Guidet, B., et al. (2014). A systematic review of adult admissions to ICUs related to adverse drug events. Crit. Care 18 (6), 643. doi:10.1186/s13054-014-0643-5

PubMed Abstract | CrossRef Full Text | Google Scholar

Kongsholm, G. G., Nielsen, A. K., and Damkier, P. (2015). Drug interaction databases in medical literature: transparency of ownership, funding, classification algorithms, level of documentation, and staff qualifications. A systematic review. Eur. J. Clin. Pharmacol. 71 (11), 1397–1402. doi:10.1007/s00228-015-1943-7

PubMed Abstract | CrossRef Full Text | Google Scholar

Kontsioti, E., Maskell, S., Bensalem, A., Dutta, B., and Pirmohamed, M. (2022). Similarity and consistency assessment of three major online drug-drug interaction resources. Br. J. Clin. Pharmacol. 88 (9), 4067–4079. doi:10.1111/bcp.15341

PubMed Abstract | CrossRef Full Text | Google Scholar

Landis, J. R., and Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics 33 (1), 159–174. doi:10.2307/2529310

PubMed Abstract | CrossRef Full Text | Google Scholar

Lee, J. Y., Oh, I. Y., Lee, J. H., Kim, S. Y., Kwon, S. S., Yang, H. J., et al. (2020). The increased risk of bleeding due to drug-drug interactions in patients administered direct oral anticoagulants. Thromb. Res. 195, 243–249. doi:10.1016/j.thromres.2020.07.054

PubMed Abstract | CrossRef Full Text | Google Scholar

Li, A., Li, M. K., Crowther, M., and Vazquez, S. R. (2020). Drug-drug interactions with direct oral anticoagulants associated with adverse events in the real world: a systematic review. Thromb. Res. 194, 240–245. doi:10.1016/j.thromres.2020.08.016

PubMed Abstract | CrossRef Full Text | Google Scholar

McDonnell, P. J., and Jacobs, M. R. (2002). Hospital admissions resulting from preventable adverse drug reactions. Ann. Pharmacother. 36 (9), 1331–1336. doi:10.1345/aph.1A333

PubMed Abstract | CrossRef Full Text | Google Scholar

Niederer, A., Wuillemin, W. A., and Mde Moerloose, P. (2001). Anticoagulation orale: attitudes pratiques. Forum Med. Suisse 17, 425–430. doi:10.4414/fms.2001.04125

CrossRef Full Text | Google Scholar

Nutescu, E. A., Burnett, A., Fanikos, J., Spinler, S., and Wittkowsky, A. (2016). Pharmacology of anticoagulants used in the treatment of venous thromboembolism. J. Thromb. Thrombolysis 41 (1), 15–31. doi:10.1007/s11239-015-1314-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Oscanoa, T. J., Lizaraso, F., and Carvajal, A. (2017). Hospital admissions due to adverse drug reactions in the elderly. A meta-analysis. Eur. J. Clin. Pharmacol. 73 (6), 759–770. doi:10.1007/s00228-017-2225-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Paterno, M. D., Maviglia, S. M., Gorman, P. N., Seger, D. L., Yoshida, E., Seger, A. C., et al. (2009). Tiering drug-drug interaction alerts by severity increases compliance rates. J. Am. Med. Inf. Assoc. JAMIA 16 (1), 40–46. doi:10.1197/jamia.M2808

PubMed Abstract | CrossRef Full Text | Google Scholar

Pauly, A., Wolf, C., Busse, M., Strauß, A.-C., Krebs, S., Dörje, F., et al. (2015). Evaluation of eight drug interaction databases commonly used in the German healthcare system. Eur. J. Hosp. Pharm. 22 (3), 165–170. doi:10.1136/ejhpharm-2014-000561

CrossRef Full Text | Google Scholar

Payne, T. H., Hines, L. E., Chan, R. C., Hartman, S., Kapusnik-Uner, J., Russ, A. L., et al. (2015). Recommendations to improve the usability of drug-drug interaction clinical decision support alerts. J. Am. Med. Inf. Assoc. JAMIA. 22 (6), 1243–1250. doi:10.1093/jamia/ocv011

PubMed Abstract | CrossRef Full Text | Google Scholar

Penning-van Beest, F. J., van Meegen, E., Rosendaal, F. R., and Stricker, B. H. (2001). Drug interactions as a cause of overanticoagulation on phenprocoumon or acenocoumarol predominantly concern antibacterial drugs. Clin. Pharmacol. Ther. 69 (6), 451–457. doi:10.1067/mcp.2001.115723

PubMed Abstract | CrossRef Full Text | Google Scholar

Reis, A. M., and Cassiani, S. H. (2010). Evaluation of three brands of drug interaction software for use in intensive care units. Pharm. world & Sci. PWS 32 (6), 822–828. doi:10.1007/s11096-010-9445-2

PubMed Abstract | CrossRef Full Text | Google Scholar

Roblek, T., Vaupotic, T., Mrhar, A., and Lainscak, M. (2015). Drug-drug interaction software in clinical practice: a systematic review. Eur. J. Clin. Pharmacol. 71 (2), 131–142. doi:10.1007/s00228-014-1786-7

PubMed Abstract | CrossRef Full Text | Google Scholar

Scheife, R. T., Hines, L. E., Boyce, R. D., Chung, S. P., Momper, J. D., Sommer, C. D., et al. (2015). Consensus recommendations for systematic evaluation of drug-drug interaction evidence for clinical decision support. Drug Saf. 38 (2), 197–206. doi:10.1007/s40264-014-0262-8

PubMed Abstract | CrossRef Full Text | Google Scholar

Sennesael, A. L., Larock, A. S., Devalet, B., Mathieux, V., Verschuren, F., Muschart, X., et al. (2018). Preventability of serious thromboembolic and bleeding events related to the use of oral anticoagulants: a prospective study. Br. J. Clin. Pharmacol. 84 (7), 1544–1556. doi:10.1111/bcp.13580

PubMed Abstract | CrossRef Full Text | Google Scholar

Shariff, A., Belagodu, S. S., Abdullah Basha, N. F., Bin, T. A. S. S. H., and Ahmed, A. A. (2021). Assessing consistency of drug-drug interaction-related information across various drug information resources. Cureus 13 (3), e13766. doi:10.7759/cureus.13766

PubMed Abstract | CrossRef Full Text | Google Scholar

Shehab, N., Lovegrove, M. C., Geller, A. I., Rose, K. O., Weidle, N. J., and Budnitz, D. S. (2016). US emergency department visits for outpatient adverse drug events, 2013-2014. Jama 316 (20), 2115–2125. doi:10.1001/jama.2016.16201

PubMed Abstract | CrossRef Full Text | Google Scholar

Smithburger, P. L., Buckley, M. S., Bejian, S., Burenheide, K., and Kane-Gill, S. L. (2011). A critical evaluation of clinical decision support for the detection of drug-drug interactions. Expert Opin. Drug Saf. 10 (6), 871–882. doi:10.1517/14740338.2011.583916

PubMed Abstract | CrossRef Full Text | Google Scholar

Smithburger, P. L., Kane-Gill, S. L., Benedict, N. J., Falcione, B. A., and Seybert, A. L. (2010). Grading the severity of drug-drug interactions in the intensive care unit: a comparison between clinician assessment and proprietary database severity rankings. Ann. Pharmacother. 44 (11), 1718–1724. doi:10.1345/aph.1P377

PubMed Abstract | CrossRef Full Text | Google Scholar

Spada, G., Vighi, G. V., Pagani, S., Vighi, G. D., Venegoni, M., and Ruocco, M. (2020). What are the characteristics of patients experiencing adverse drug reactions to oral anticogulants and how can such reactions be prevented? Curr. Drug Saf. 15 (1), 38–44. doi:10.2174/1574886314666191003162104

PubMed Abstract | CrossRef Full Text | Google Scholar

Steffel, J., Collins, R., Antz, M., Cornu, P., Desteghe, L., Haeusler, K. G., et al. (2021). 2021 European heart rhythm association practical guide on the use of non-vitamin K antagonist oral anticoagulants in patients with atrial fibrillation. Europace 23 (10), 1612–1676. doi:10.1093/europace/euab065

PubMed Abstract | CrossRef Full Text | Google Scholar

Sweidan, M., Reeve, J. F., Brien, J. A., Jayasuriya, P., Martin, J. H., and Vernon, G. M. (2009). Quality of drug interaction alerts in prescribing and dispensing software. Med. J. Aust. 190 (5), 251–254. doi:10.5694/j.1326-5377.2009.tb02832.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Terrier, J., Gaspar, F., Fontana, P., Youssef, D., Reny, J. L., Csajka, C., et al. (2021). Drug-drug interactions with direct oral anticoagulants: practical recommendations for clinicians. Am. J. Med. 134 (8), 939–942. doi:10.1016/j.amjmed.2021.04.003

PubMed Abstract | CrossRef Full Text | Google Scholar

ThomasFulda, RRJV, Zanden, J. V., Parker, S., and Byrns, P. J. (2000). Disagreement among drug compendia on inclusion and ratings of drug-drug interactions. Curr. Ther. Res. 61 (8), 540–548. doi:10.1016/s0011-393x(00)80036-3

CrossRef Full Text | Google Scholar

Ufer, M. (2005). Comparative pharmacokinetics of vitamin K antagonists: warfarin, phenprocoumon and acenocoumarol. Clin. Pharmacokinet. 44 (12), 1227–1246. doi:10.2165/00003088-200544120-00003

PubMed Abstract | CrossRef Full Text | Google Scholar

Vazquez, S. R. (2018). Drug-drug interactions in an era of multiple anticoagulants: a focus on clinically relevant drug interactions. Blood 132 (21), 2230–2239. doi:10.1182/blood-2018-06-848747

PubMed Abstract | CrossRef Full Text | Google Scholar

Violi, F., Lip, G. Y., Pignatelli, P., and Pastori, D. (2016). Interaction between dietary vitamin K intake and anticoagulation by vitamin K antagonists: is it really true? a systematic review. Med. Baltim. 95 (10), e2895. doi:10.1097/MD.0000000000002895

CrossRef Full Text | Google Scholar

Vitry, A. I. (2007). Comparative assessment of four drug interaction compendia. Br. J. Clin. Pharmacol. 63 (6), 709–714. doi:10.1111/j.1365-2125.2006.02809.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Vonbach, P., Dubied, A., Krähenbühl, S., and Beer, J. H. (2008). Evaluation of frequently used drug interaction screening programs. Pharm. world & Sci. PWS. 30 (4), 367–374. doi:10.1007/s11096-008-9191-x

PubMed Abstract | CrossRef Full Text | Google Scholar

Wang, L. M., Wong, M., Lightwood, J. M., and Cheng, C. M. (2010). Black box warning contraindicated comedications: concordance among three major drug interaction screening programs. Ann. Pharmacother. 44 (1), 28–34. doi:10.1345/aph.1M475

PubMed Abstract | CrossRef Full Text | Google Scholar

Wong, C. M., Ko, Y., and Chan, A. (2008). Clinically significant drug-drug interactions between oral anticancer agents and nonanticancer agents: profiling and comparison of two drug compendia. Ann. Pharmacother. 42 (12), 1737–1748. doi:10.1345/aph.1L255

PubMed Abstract | CrossRef Full Text | Google Scholar

Zorina, O. I., Haueis, P., Greil, W., Grohmann, R., Kullak-Ublick, G. A., and Russmann, S. (2013). Comparative performance of two drug interaction screening programmes analysing a cross-sectional prescription dataset of 84,625 psychiatric inpatients. Drug Saf. 36 (4), 247–258. doi:10.1007/s40264-013-0027-9

PubMed Abstract | CrossRef Full Text | Google Scholar

Zorina, O. I., Haueis, P., Semmler, A., Marti, I., Gonzenbach, R. R., Guzek, M., et al. (2012). Comparative evaluation of the drug interaction screening programs MediQ and ID PHARMA CHECK in neurological inpatients. Pharmacoepidemiol Drug Saf. 21 (8), 872–880. doi:10.1002/pds.3279

PubMed Abstract | CrossRef Full Text | Google Scholar