Usefulness of Bnet, a Simple Linear Metric in Discerning Torsades De Pointes Risks in 28 CiPA Drugs

The Comprehensive in vitro Proarrhythmia Assay (CiPA) project suggested the torsade metric score (TMS) which requires substantial computing resources as a useful biomarker to predict proarrhythmic risk from human ether-à-go-go–related gene (hERG) and a few other ion channel block data. The TMS was useful to predict low TdP risks of drugs blocking Na⁺ (ranolazine) and Ca²⁺ (verapamil) channels as well as the hERG channel. However, Mistry asserted that the simple linear metric, Bnet reflecting net blockade of a few influential ion channels has similar predictive power. Here we compared the predictability of Bnet and TMS for the 12 training and 16 validation CiPA drugs which were pre-classified into three categories according to the known TdP risks (low, intermediate, and high risk) by CiPA. Bnet at 5×C_max (Bnet_5×Cmax) was calculated using the ion-channel IC50 and Hill coefficients of CiPA drugs collected from previous reports by the CiPA team and others. The receiver operating characteristic curve area under curve (ROC AUC) values for TMS and Bnet_5×Cmax as performance metrics in discerning low versus intermediate/high risk categories for the 28 CiPA drugs were similar. However, Bnet_5×Cmax was much inferior to TMS at discerning between intermediate- and high-risk drugs. Dynamic Bnet, which used in silico hERG dynamic parameters unlike conventional Bnet, improved the misspecification. Thus, we propose that Bnet_5×Cmax is used for quick screening of TdP risks of drug candidates and if the “intermediate/high” risk is predicted by Bnet_5×Cmax, in silico approaches, such as dynamic Bnet or TMS, may be further considered.

Introduction

The International Council on Harmonization (ICH) established the guidelines, S7B for non-clinical evaluation and E14 for clinical evaluation of the proarrhythmic risk of drugs. As recommended by the guidelines, the conventional practice to evaluate the Torsades de Pointes (TdP) risks has been focused on the QTc interval from blockade of human ether-à-go-go–related gene (hERG) channel (Shah, 2005) that is associated with rapidly activating delayed rectifier potassium current I_Kr. (Sager et al., 2014) Although ICH S7B and E14 regulatory guidelines have been successful in screening TdP risks of new drugs, there are several low TdP risk drugs with the prolonged action potential duration (APD) and QTc interval. Thus, the current practice according to ICH guidelines is sensitive but not specific enough to evaluate proarrhythmic (TdP) risks.

One of the major objectives of the Comprehensive in vitro Proarrhythmia Assay (CiPA) initiative was to improve the current ICH guidelines to avoid the misclassification of TdP risks by evaluating mechanistically based in vitro assays and in silico reconstruction of the cardiac action potential. The CiPA ion channel working group and in silico modeling group suggested qNet and the torsade metric score (TMS) as conclusive markers via the CiPAORdv1.0, the mechanistic in silico model (Li et al., 2019b) based on a series of modification of O’Hara-Rudy (ORd) human ventricular myocyte model (O’hara et al., 2011).

However, a few groups have raised questions on the superiority in accuracies of model-driven in silico approaches. (Mistry et al., 2015; Mistry, 2017; Parikh et al., 2017; Mistry, 2018; Parikh et al., 2019; Mistry, 2019a) Especially, Mistry asserted that Bnet, a simple linear metric using the net difference between inward and outward ion channel blocking, has predictive power similar to that of TMS. (Mistry, 2019a) Mistry questioned the usefulness of the complicated in silico approaches proposed by CiPA if the performance to assess the proarrhythmic risk is similar, although the CiPA researchers asserted the superiority of in silico approaches that consider the trapping of the hERG and other channels through rigorous validation of the model (Li et al., 2019b).

In this report, we compared the performance of Bnet and TMS in discerning TdP risks of the whole 28 CiPA drugs (12 training and 16 validation) to gain insight into potentials and limitations of in silico approaches by CiPA.

Methods

Channel Block Data to Calculate Bnet

The CiPA have chosen 12 training and 16 validation drugs which have been classified by a team of clinical cardiologists and electrophysiologists into three categories according to the known TdP risks (high, intermediate, and low risk) (Colatsky et al., 2016).

To compare the relationship between TMS and Bnet of the 28 drugs, we first collected the ion channel block data (IC50 and Hill coefficients by the drugs) that were used to estimate the TMS (Li et al., 2019b). They were used to calculate Bnet values. Because the CiPA aimed to automate the assays by using high-throughput patch-clamp systems (HTS) (Sager et al., 2014), hybrid patch-clamp data collected using both manual (for hERG channel) and automated (other channels) methods were compared with data from the manual method for all channels. The performance of the hybrid and manual methods seemed equivalent (Li et al., 2019b). However, we picked the TMS values obtained from the manual method that has long been used as a standard in patch clamp studies.

In the many ion channels, only the four channels that were finally chosen by CiPA as significantly influencing the qNet and TMS: rapidly activating delayed rectifier potassium current (I_Kr), late sodium current (INaL), L-type calcium current (ICaL), and peak sodium current (INa) (Li et al., 2019b). Thus, for the calculation of Bnet, we used the IC50 and Hill coefficients for the four channels identical to those used to estimate qNet and TMS by CiPA. Those for the 12 training drugs were retrieved from the report by Crumb et al. (2016) that was utilized by Li et al. (2019b) In the case of the 16 validation drugs, the CiPA researchers did not use published data but have performed patch-clamp studies on their own. (Li et al., 2019b) Thus, we retrieved the IC50 and Hill coefficients for INaL, ICaL, and INa channels from the report by Li et al. However, the CiPA researchers did not use simple channel block, but employed a channel-trapping model in the case of the hERG channel and the IC50 or Hill coefficients for hERG channel for the 16 validation drugs were not mentioned in their report at all. (Li et al., 2019b). Thus, we had to search other published data (Table 1) to replace those for I_Kr (hERG) of the 16 validation drugs.

TABLE 1

Table 1 IC50 and Hill coefficient values of the hERG channel retrieved from the literature to calculate Bnet_5×Cmax for the 16 validation drugs.

Calculation of Percentage Block and Bnet_5×Cmax

The percentage block (%block) against a repolarization or depolarization ion-channel inputted into the Bnet_5×Cmax model was calculated using the mean maximal concentration observed (C_max) corrected for plasma protein binding (thus, unbound concentration)(Mistry, 2018).

$$\%\text{block} = \frac{100 \times {(5\times C_{\mathit{max}})}^{\mathit{Hill}}}{{(5\times C_{\mathit{max}})}^{\mathit{Hill}} + \mathit{IC}{50}^{\mathit{Hill}}}$$

Bnet_5×Cmax was defined as the net difference in %block of the four most influential channels on the AP shape (%block of hERG channel − sum of %blocks of the other channels) at 5× C_max (Supplementary Table 1).

$$\mathit{Bne}{t}_{5\times C_{\max }} (\%)={\displaystyle \sum _{i=0}^n{R}_i}-{\displaystyle \sum _{j=0}^m{D}_j}$$

where Ri and Dj represent the %block against repolarization (I_Kr) and depolarization (INaL, ICaL, and INa) ion-channels, respectively for a specific drug.

There are three major differences between the original Bnet (Mistry, 2018) and Bnet_5×Cmax. First, compared to the original Bnet proposed by Mistry, %block in our study (Bnet_5×Cmax) took Hill coefficient into consideration. Second, the original Bnet proposed by Mistry did not include INa, we included it because the four channels have been selected to calculate TMS by CiPA researchers. Third, the original Bnet used values at 1× C_max but Bnet_5×Cmax used values at 5× C_max.

Calculation of Dynamic Bnet

We compared TMS and Bnet_5×Cmax with “dynamic Bnet” (Mistry, 2019a), which reflects hERG dynamics as TMS used. We utilized the publicly available data set that Mistry provided (Mistry, 2019b) and in the data set, hERG dynamics was included into Bnet by replacing the static hERG block with the dynamic hERG blocking using IC50 and maximal inhibition at the 1× C_max.

Torsade Metric Score

The TMS, mean of qNet values at 1×, 2×, 3×, or 4× C_max derived from the CiPAORdv1.0 model was digitized from a report by Li et al. (2019b). As mentioned in the previous section, only the TMS values from manually measured data were collected for comparison.

Ranking Performance Measures

Statistical analysis was performed using R Statistical Software version 3.6.0 (R Core Team, 2019). The ROC AUC (receiver operating characteristic curve area under the curve) (Zou et al., 2007) for TMS and Bnet_5×Cmax was calculated based on the known risk classifier. A logistic regression analysis using maximum likelihood estimation of the metric and the torsadogenic risk categories was performed by the rms R package (Harrell, 2019).

Results

Risk Misspecification by TMS and Bnet_5×Cmax in the Validation Drug Data Set

The TMS and Bnet_5×Cmax of all 12 training drugs tested with Crumb’s data (Crumb et al., 2016) were accordant to the risk categories (low vs. intermediate/high) (Figure 1). In the case of the 16 validation drugs, there were a few mismatches in categories both in the TMS (Figure 1A) and Bnet_5×Cmax methods (Figure 1B): tamoxifen and metoprolol (low-risk drugs) were located in the intermediate-risk cluster in the TMS and clarithromycin, domperidone, and risperidone (intermediate-risk drugs) were located in the low-risk cluster in the Bnet_5×Cmax. Dynamic Bnet using hERG dynamics decreased the misspecification but a drug of the validation data set, risperidone, still was misclassified (Figure 1C).

FIGURE 1

Figure 1 (A) median TMS (torsade metric score), (B) Bnet_5×Cmax, and (C) Bnet using hERG dynamics at 1× Cmax for the 28 CiPA drugs by data sets (training or validation drugs) and risk categories (high, intermediate, or low risk). The 28 drugs are sorted according to the median TMS or Bnet values in each panel. The two vertical lines indicate borderlines dividing low- and intermediate-risk clusters (yellow) and intermediate and high-risk clusters (blue).

Strong Correlation Between TMS and Bnet_5×Cmax in the Training Drug Data Set

Bnet_5×Cmax has shown performance similar to that of TMS as they are correlated with each other (r² = 0.663) (Figure 2). The correlation between Bnet_5×Cmax and TMS was stronger in the training drugs (r² = 0.867) than in the validation drugs (r² = 0.597), suggesting that the training drugs may possess better in vitro (patch-clamp study) data quality.

FIGURE 2

Figure 2 Correlation between Bnet_5×Cmax and median TMS (torsade metric score, the average of qNet at 1–4× C_max) of (A) 12 training drugs, (B) 16 validation drugs, and (C) all 28 CiPA drugs. C_max, peak plasma concentration.

Performance Comparison: TMS and Bnet_5×Cmax

Both TMS and Bnet_5×Cmax seemed to discriminate low proarrhythmic risk drugs from intermediate/high-risk drugs quite well because the TMS and Bnet_5×Cmax values of low-risk drugs were significantly different (t-test, p < 0.005, and p < 0.001, respectively) from those of intermediate- and high-risk drugs (Supplementary Figure 1). The ranking performance measure for TMS and Bnet_5×Cmax was evaluated using ROC AUC (low vs. intermediate/high risk), and the values were 0.956 and 0.959, respectively (Table 2, Supplementary Figure 2).

TABLE 2

Table 2 Prediction performance for 28 CiPA drugs of TMS, Bnet_5×Cmax, and Dynamic Bnet.

The ROC AUC (Low/Intermediate Vs. High Risk) and χ² Statistic Derived From Univariable Logistic Regression Analysis of TMS Were Higher Than Those of Bnet_5×Cmax Suggesting That TMS Outperforms Bnet_5×Cmax in Discriminating Intermediate- and High-Risk Drugs (Table 2).

Discussion

This is the first study to examine the performance of Bnet_5×Cmax, a simple metric calculated as the gap in blocking four representative channels by 28 CiPA drugs. We showed that Bnet_5×Cmax provided predictability comparable to the large-scale mechanistic model.

The therapeutic C_max value directly affects the TMS and the Bnet metric. The TMS is calculated by averaging qNet values at 1×, 2×, 3×, or 4× C_max and Bnet_5×Cmax is calculated based on %block at 5× C_max. We have first screened Bnet for 1×, 5×, and 10× C_max and the Bnet_5×Cmax showing the best performance was further used in our study (Supplementary Table 2).

Although the TMS and Bnet_5×Cmax of all 12 training drugs tested with Crumb’s data (Crumb et al., 2016) were in the exact order of risk categories pre-defined by previous reports (Shah, 2005), the data based on the 16 validation drugs showed a few incorrect predictions both in TMS and Bnet_5×Cmax. This misspecification seems to have been caused by the patch-clamp experiments on the validation drugs not as qualified as in the 12 training drugs.

The reliability of patch-clamp experiment data is known to be highly variable by laboratories and skillfulness of the experimenter. Thus, measurement of IC50 and Hill coefficients using the patch-clamp method performed by well-trained personnel appears critical for the appropriate assessment of both TMS and Bnet, regardless of using the silico method. The CiPA’s attempt to estimate TMS with combined in vitro and in silico approaches is worthwhile in that the variability in multiple channel blocking is rigorously validated. Nonetheless, the performance of TMS is also dependent on the quality of patch-clamp experiment data for the ion channels that are input into the in silico simulation step.

The major limitation of Bnet is the inability to discriminate drugs with atypical binding kinetics (Li et al., 2019a). Because the hERG trapping observed in some drugs is not measured by the conventional ion channel blocking assay, CiPA has used the dynamic-hERG binding model for the data obtained using the Milnes protocol (Li et al., 2017). When the conventional IC50 is used to calculate Bnet for drugs that are significantly trapped in the hERG channel (e.g., dofetilide, bepridil, and terfenadine exemplified by Li et al. (2017)), their Bnet would be underestimated. However, the three drugs have shown Bnet values high enough to fall in the “intermediate/high” cluster in our study (Figure 1).

Recently, Mistry calculated “dynamic Bnet” (Mistry, 2019a) using hERG dynamic IC50 which may partly reflect binding kinetics and showed the higher correlation of dynamic Bnet with TMS of the 28 CiPA drugs (r² = 0.86) than the conventional Bnet_5×Cmax (r² = 0.66) presented in Figure 2 in this report. The ROC AUC values of low- versus intermediate/high-risk for TMS, Bnet_5×Cmax, and dynamic Bnet were 0.956, 0.959, and 0.994, respectively. Although dynamic Bnet showed the best performance, it also requires the additional in silico approach and the time and resources spent to acquire the metric in the discovery or preclinical stage may still be substantial. The Bnet_5×Cmax can be a straightforward, accessible, and simple screening tool to discern the low-risk drugs.

The highest prediction performance of low/intermediate-risk versus high-risk drugs was observed in the TMS (ROC AUC = 0.99, Table 2). However, in the actual early development process, drug candidates with an intermediate risk often cannot survive to the next development step, and we believe that this limitation of poor discerning between intermediate and high risks may not affect go/no-go decision at the early stage in almost of therapeutic areas except for antiarrhythmics.

The Bnet_5×Cmax metric may be used as a simple screening biomarker in drug discovery and early development. We demonstrated that the Bnet_5×Cmax (or Bnet at concentrations regarded high enough when no Cmax data are available) provides initial information whether a candidate is at low proarrhythmic risk or not. For a candidate worthy of further development even with intermediate/high risk according to the Bnet_5×Cmax metric, in silico approaches proposed by CiPA or dynamic Bnet may be helpful.

Author’s Note

The abstract was submitted to Safety Pharmacology Society Anual Meeting 2019, held in Barcelona, Spain from September 23rd to 26th, 2019.

Data Availability Statement

Publicly available datasets were analyzed in this study. This data can be found here: https://ascpt.onlinelibrary.wiley.com/doi/abs/10.1002/cpt.1184 and https://github.com/HiteshBMistry/Re-analysis-of-CiPA/blob/master/dynamic_qnet_versus_dynamic_bnet.csv.

Author Contributions

SeH, K-SK, H-AL, and D-SY contributed to the acquisition of data and provided experimental data guidance. SuH, SeH, and D-SY contributed to designing the work and carried out simulations. SuH, SeH, and D-SY contributed to the analysis and interpretation of the data and writing of the manuscript.

Funding

This research was supported by a grant (18182MFDS406) from Ministry of Food and Drug Safety in 2018.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fphar.2019.01419/full#supplementary-material

References

Abbott, G. W., Sesti, F., Splawski, I., Buck, M. E., Lehmann, M. H., Timothy, K. W., et al. (1999). MiRP1 forms IKr potassium channels with HERG and is associated with cardiac arrhythmia. Cell 97, 175–187. doi: 10.1016/S0092-8674(00)80728-X

PubMed Abstract | CrossRef Full Text | Google Scholar

Chae, Y. J., Lee, K. J., Lee, H. J., Sung, K. W., Choi, J. S., Lee, E. H., et al. (2015). Endoxifen, the active metabolite of tamoxifen, inhibits cloned hERG potassium channels. Eur. J. Pharmacol. 752, 1–7. doi: 10.1016/j.ejphar.2015.01.048

PubMed Abstract | CrossRef Full Text | Google Scholar

Claassen, S., Zünkler, B. J. (2005). Comparison of the effects of metoclopramide and domperidone on HERG channels. Pharmacology 74, 31–36. doi: 10.1159/000083234

PubMed Abstract | CrossRef Full Text | Google Scholar

Colatsky, T., Fermini, B., Gintant, G., Pierson, J. B., Sager, P., Sekino, Y., et al. (2016). The Comprehensive in Vitro Proarrhythmia Assay (CiPA) initiative - Update on progress. J. Pharmacol. Toxicol. Methods 81, 15–20. doi: 10.1016/j.vascn.2016.06.002

PubMed Abstract | CrossRef Full Text | Google Scholar

Crumb, W. J., Vicente, J., Johannesen, L., Strauss, D. G. (2016). An evaluation of 30 clinical drugs against the comprehensive in vitro proarrhythmia assay (CiPA) proposed ion channel panel. J. Pharmacol. Toxicol. Methods 81, 251–262. doi: 10.1016/j.vascn.2016.03.009

PubMed Abstract | CrossRef Full Text | Google Scholar

Crumb, W. J. Jr. (2000). Loratadine blockade of K(+) channels in human heart: comparison with terfenadine under physiological conditions. J. Pharmacol. Exp. Ther. 292, 261–264.

PubMed Abstract | Google Scholar

Drolet, B., Zhang, S., Deschenes, D., Rail, J., Nadeau, S., Zhou, Z., et al. (1999). Droperidol lengthens cardiac repolarization due to block of the rapid component of the delayed rectifier potassium current. J. Cardiovasc. Electrophysiol. 10, 1597–1604. doi: 10.1111/j.1540-8167.1999.tb00224.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Harrell, F. E. (2019). “rms: Regression Modeling Strategies”.

Google Scholar

Kawakami, K., Nagatomo, T., Abe, H., Kikuchi, K., Takemasa, H., Anson, B. D., et al. (2006). Comparison of HERG channel blocking effects of various beta-blockers– implication for clinical strategy. Br. J. Pharmacol. 147, 642–652. doi: 10.1038/sj.bjp.0706508

PubMed Abstract | CrossRef Full Text | Google Scholar

Kirsch, G. E., Trepakova, E. S., Brimecombe, J. C., Sidach, S. S., Erickson, H. D., Kochan, M. C., et al. (2004). Variability in the measurement of hERG potassium channel inhibition: effects of temperature and stimulus pattern. J. Pharmacol. Toxicol. Methods 50, 93–101. doi: 10.1016/j.vascn.2004.06.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Kongsamut, S., Kang, J., Chen, X. L., Roehr, J., Rampe, D. (2002). A comparison of the receptor binding and HERG channel affinities for a series of antipsychotic drugs. Eur. J. Pharmacol. 450, 37–41. doi: 10.1016/S0014-2999(02)02074-5

PubMed Abstract | CrossRef Full Text | Google Scholar

Lee, S. Y., Kim, Y. J., Kim, K. T., Choe, H., Jo, S. H. (2006). Blockade of HERG human K+ channels and IKr of guinea-pig cardiomyocytes by the antipsychotic drug clozapine. Br. J. Pharmacol. 148, 499–509. doi: 10.1038/sj.bjp.0706744

PubMed Abstract | CrossRef Full Text | Google Scholar

Lee, H. A., Hyun, S. A., Byun, B., Chae, J. H., Kim, K. S. (2018). Electrophysiological mechanisms of vandetanib-induced cardiotoxicity: comparison of action potentials in rabbit Purkinje fibers and pluripotent stem cell-derived cardiomyocytes. PloS One 13, e0195577. doi: 10.1371/journal.pone.0195577

PubMed Abstract | CrossRef Full Text | Google Scholar

Li, Z., Dutta, S., Sheng, J., Tran, P. N., Wu, W., Chang, K., et al. (2017). Improving the in silico assessment of proarrhythmia risk by combining hERG (Human Ether-a-go-go-Related Gene) channel-drug binding kinetics and multichannel pharmacology. Circ. Arrhythm Electrophysiol. 10, e004628. doi: 10.1161/CIRCEP.116.004628

PubMed Abstract | CrossRef Full Text | Google Scholar

Li, Z., Mirams, G. R., Strauss, D. G. (2019a). Response to “CiPA’s Complexity Bias”. Clin. Pharmacol. Ther. 105, 1325. doi: 10.1002/cpt.1399

PubMed Abstract | CrossRef Full Text | Google Scholar

Li, Z., Ridder, B. J., Han, X., Wu, W. W., Sheng, J., Tran, P. N., et al. (2019b). Assessment of an in silico mechanistic model for proarrhythmia risk prediction under the CiPA initiative. Clin. Pharmacol. Ther. 105, 466–475. doi: 10.1002/cpt.1184

PubMed Abstract | CrossRef Full Text | Google Scholar

Lin, C., Ke, X., Ranade, V., Somberg, J. (2008). Extracellular acidification and hyperkalemia induce changes in HERG inhibition by ibutilide. Cardiology 110, 209–216. doi: 10.1159/000111932

PubMed Abstract | CrossRef Full Text | Google Scholar

Mistry, H. B., Davies, M. R., Di Veroli, G. Y. (2015). A new classifier-based strategy for in-silico ion-channel cardiac drug safety assessment. Front. In Pharmacol. 6, 59. doi: 10.3389/fphar.2015.00059

CrossRef Full Text | Google Scholar

Mistry, H. (2017). Complexity vs. Simplicity: The Winner Is?. Clin. Pharmacol. Ther. 101, 326–326. doi: 10.1002/cpt.503

PubMed Abstract | CrossRef Full Text | Google Scholar

Mistry, H. B. (2018). Complex versus simple models: ion-channel cardiac toxicity prediction. PeerJ 6, e4352. doi: 10.7717/peerj.4352

PubMed Abstract | CrossRef Full Text | Google Scholar

Mistry, H. B. (2019a). Comprehensive in vitro proarrhythmic assay complexity bias. Clin. Pharmacol. Ther. 105, 1323–1324. doi: 10.1002/cpt.1400

PubMed Abstract | CrossRef Full Text | Google Scholar

Mistry, H. B. (2019b). Re-analysis-of-CIPA [Online]. Available: https://github.com/HiteshBMistry/Re-analysis-of-CiPA/blob/master/dynamic_qnet_versus_dynamic_bnet.csv [Accessed September 28, 2019].

Google Scholar

O’hara, T., Virág, L., Varró, A., Rudy, Y. (2011). Simulation of the undiseased human cardiac ventricular action potential: model formulation and experimental validation. PloS Comput. Biol. 7, e1002061. doi: 10.1371/journal.pcbi.1002061

PubMed Abstract | CrossRef Full Text | Google Scholar

Parikh, J., Gurev, V., Rice, J. J. (2017). Novel two-step classifier for torsades de pointes risk stratification from direct features. Front. Pharmacol. 8, 816. doi: 10.3389/fphar.2017.00816

PubMed Abstract | CrossRef Full Text | Google Scholar

Parikh, J., Di Achielle, P., Kozloski, J., Gurev, V. (2019). Intrinsic structure of model-derived metrics for in silico proarrhytmic risk assessment identified by global sensitivity analysis. bioRxiv. doi: 10.1101/543926

CrossRef Full Text | Google Scholar

Paul, A. A., Witchel, H. J., Hancox, J. C. (2001). Inhibition of HERG potassium channel current by the class 1a antiarrhythmic agent disopyramide. Biochem. Biophys. Res. Commun. 280, 1243–1250. doi: 10.1006/bbrc.2001.4269

PubMed Abstract | CrossRef Full Text | Google Scholar

R Core Team (2019). R: A Language and Environment for Statistical Computing (Vienna, Austria: R Foundation for Statistical Computing).

Google Scholar

Redfern, W. S., Carlsson, L., Davis, A. S., Lynch, W. G., Mackenzie, I., Palethorpe, S., et al. (2003). Relationships between preclinical cardiac electrophysiology, clinical QT interval prolongation and torsade de pointes for a broad range of drugs: evidence for a provisional safety margin in drug development. Cardiovasc. Res. 58, 32–45. doi: 10.1016/S0008-6363(02)00846-5

PubMed Abstract | CrossRef Full Text | Google Scholar

Sager, P. T., Gintant, G., Turner, J. R., Pettit, S., Stockbridge, N. (2014). Rechanneling the cardiac proarrhythmia safety paradigm: a meeting report from the Cardiac Safety Research Consortium. Am. Heart J. 167, 292–300. doi: 10.1016/j.ahj.2013.11.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Shah, R. R. (2005). Drugs, QTc interval prolongation and final ICH E14 guideline: an important milestone with challenges ahead. Drug Saf. 28, 1009–1028. doi: 10.2165/00002018-200528110-00003

PubMed Abstract | CrossRef Full Text | Google Scholar

Tarantino, P., Appleton, N., Lansdell, K. (2005). Effect of trazodone on hERG channel current and QT-interval. Eur. J. Pharmacol. 510, 75–85. doi: 10.1016/j.ejphar.2005.01.009

PubMed Abstract | CrossRef Full Text | Google Scholar

Walker, B. D., Singleton, C. B., Tie, H., Bursill, J. A., Wyse, K. R., Valenzuela, S. M., et al. (2000). Comparative effects of azimilide and ambasilide on the human ether-a-go-go-related gene (HERG) potassium channel. Cardiovasc. Res. 48, 44–58. doi: 10.1016/S0008-6363(00)00155-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Zhang, J. H., Chung, T. D., Oldenburg, K. R. (1999). A simple statistical parameter for use in evaluation and validation of high throughput screening assays. J. Biomol. Screen 4, 67–73. doi: 10.1177/108705719900400206

PubMed Abstract | CrossRef Full Text | Google Scholar

Zou, K. H., O’malley, A. J., Mauri, L. (2007). Receiver-operating characteristic analysis for evaluating diagnostic tests and predictive models. Circulation 115, 654–657. doi: 10.1161/CIRCULATIONAHA.105.594929

PubMed Abstract | CrossRef Full Text | Google Scholar