Improving the hERG model fitting using a deep learning-based method

The hERG channel is one of the essential ion channels composing the cardiac action potential and the toxicity assay for new drug. Recently, the comprehensive in vitro proarrhythmia assay (CiPA) was adopted for cardiac toxicity evaluation. One of the hurdles for this protocol is identifying the kinetic effect of the new drug on the hERG channel. This procedure included the model-based parameter identification from the experiments. There are many mathematical methods to infer the parameters; however, there are two main difficulties in fitting parameters. The first is that, depending on the data and model, parametric inference can be highly time-consuming. The second is that the fitting can fail due to local minima problems. The simplest and most effective way to solve these issues is to provide an appropriate initial value. In this study, we propose a deep learning-based method for improving model fitting by providing appropriate initial values, even the right answer. We generated the dataset by changing the model parameters and trained our deep learning-based model. To improve the accuracy, we used the spectrogram with time, frequency, and amplitude. We obtained the experimental dataset from https://github.com/CardiacModelling/hERGRapidCharacterisation. Then, we trained the deep-learning model using the data generated with the hERG model and tested the validity of the deep-learning model with the experimental data. We successfully identified the initial value, significantly improved the fitting speed, and avoided fitting failure. This method is useful when the model is fixed and reflects the real data, and it can be applied to any in silico model for various purposes, such as new drug development, toxicity identification, environmental effect, etc. This method will significantly reduce the time and effort to analyze the data.

1 Introduction

It is a well-known fact that it is crucial to evaluate the effects of pharmaceuticals on heart rhythm because an unstable heart rhythm causes significant problems, including death. Additionally, cardiotoxicity has resulted in the withdrawal of some previously marketed drugs and restrictions on some clinically useful drugs (Lasser et al., 2002). Therefore, there have been many discussions on the mechanisms, prevention methods, and management of such toxicity by drugs (Kelleni and Abdelbasset, 2018). In particular, screening for the human Ether-a`-go-go-Related Gene (hERG) is critical. The hERG is a gene that forms part of the rapid delayed rectifier potassium current of the heart, I_Kr, and plays an important role in causing repolarization of the cardiac action potential. Many drugs that cause cardiotoxicity are known to block the hERG channel. Blockade by drugs leads to a decrease in I_Kr, which can prolong ventricular action potential (Jurkiewicz and Sanguinetti, 1993). This is also associated with an increase in the QT interval (QT) in the electrocardiogram (ECG) (Sanguinetti and Tristani-Firouzi, 2006), which is likely related to Torsade de Pointes (Malik and Camm, 2001). Therefore, in 2005, the International Council for Harmonization included the following in its guidelines for non-clinical evaluation: “Preclinical Evaluation of the Possibility of Delayed Ventricular Repolarization (QT-Interval Prolongation) by Human Medicines (S7B)” (Food and Drug Administration, 2005; Friedrichs et al., 2005).

Advances in mathematical modeling and computational simulations of ion channels have made cell reactions and electrophysiological phenomena understandable and predictable, meaning they can help predict drug-induced changes. The mathematical modeling of hERG has also been continuously developed by (Zeng et al., 1995), (Beattie et al., 2018), and (ten Tusscher et al., 2004). These mathematical models are completed by fitting them to experimental data and finding the parameters. The parameters are important since they provide physiological and biophysical significance (Pathmanathan et al., 2015). However, the fitting process is by no means easy. To obtain more accurate model parameters, many mathematical and statistical methods, such as the least-squares optimization (Grisetti et al., 2020), the gradient descent (Ruder, 2016), and the Covariance Matrix Adaptation Evolution Strategy (CMA-ES) (Hansen, 2006; Khan, 2018) have been proposed and studied. Furthermore, the development of parallel computing technology and hardware has significantly aided in problem-solving by further improving the performance of these methods (Khan, 2018). However, neither method is easy to completely avoid the local minima problem, and it is time-consuming depending on the data and model. Mathematically, the best way to solve these problems is to suggest initial values close to the true values. Initial values are usually sampled from a particular distribution associated with the characteristic of the problem or given based on past experience, but none are perfect solutions.

Deep learning-based artificial intelligence (AI) has recently made tremendous progress. Great achievements have been made not only in regression and classification problems but also in the creative field. For example, in the field of vision and image processing, convolutional neural network (CNN)-based models (O’Shea and Nash, 2015), such as ResNet (He et al., 2016), EfficientNet (Tan and Le, 2019; 2021), and RegNet (Radosavovic et al., 2020), have shown better performance than humans, and in the field of time-series like natural language processing, recurrent neural network (Sherstinsky, 2020), long short-term memory (LSTM) (Hochreiter and Schmidhuber, 1997; Sherstinsky, 2020), and transformer-based models (Vaswani et al., 2017; Lin et al., 2022) are showing remarkable results.

Medicine and biotechnology also have numerous images and time-series data. Therefore, various problems can be solved through deep learning-based AI, and several studies on deep learning-based analysis are already being conducted. (Alhusseini et al., 2020; Sevakula et al., 2020; Aghasafari et al., 2021; Jeong and Lim, 2021; Rogers et al., 2021).

In this paper, we introduce a method to predict the approximate value of the hERG ion channel model parameters using a neural network and improve the fitting operation using the predicted parameters. We confirmed the performance of improved parameter fitting using the experimental data released by https://github.com/CardiacModelling/hERGRapidCharacterisation in (Lei et al., 2019).

2 Methods

Our method is as follows. First, the simulation generates $I_{\mathrm{K}\mathrm{r}}$ current data. The generated current data was then converted into a spectrogram. Next, our parametric prediction network was trained from the generated simulation data. The experimental data were used only for the validation and testing of the network. Using the trained prediction network, nine parameters were predicted based on the current generated by the patch-clamp experiment. Figure 1 depicts the overall overview.

FIGURE 1

FIGURE 1. An overview of our method. A total of 500,000 were generated to train hERGNet, and each data consisted of ($I_{\mathrm{K}\mathrm{r}}^{sim}$, $\mathit{\theta }$). The trained hERGNet predicts ${\mathit{\theta }}^{*}$ through $I_{\mathrm{K}\mathrm{r}}^{exp}$ obtained by a patch-clamp experiment. Parameter inference is performed using ${\mathit{\theta }}^{*}$ as the initial value.

2.1 hERG model

In this study, we used the experimental data published in (Lei et al., 2019, https://github.com/CardiacModelling/hERGRapidCharacterisation). Therefore, the hERG model and basic settings we use are the same as those of (Lei et al., 2019). For ease of training and evaluation of our deep learning model, we excluded 11 of the 211 cells, which seemed to have a large difference between the experimental results and the results produced by the hERG model. The currents were recorded for the “staircase protocol” (Figure 2). As shown in the top image in Figure 2, each step is 500 ms, long enough to see the characteristics of $I_{\mathrm{K}\mathrm{r}}$. Thus, it is possible to observe the dynamics at different voltage values. Lei et al. showed that their protocol provided enough information to infer true parameters through a synthetic data study (Lei et al., 2019).

FIGURE 2

FIGURE 2. The figure above shows the “staircase protocol” introduced in (Lei et al., 2019) and the $I_{\mathrm{K}\mathrm{r}}$ currents for cells “A01,” “A06,” “B10,” and “K14.”

This hERG model with Hodgkin and Huxley style structure used in this experiment is Beattie’s model (Beattie et al., 2018) slightly improved by ten Tusscher et al. (ten Tusscher et al., 2004), and I_Kr is the same as Eq. (1).

${\mathit{I}}_{\mathrm{K}\mathrm{r}}={\mathit{g}}_{\mathit{K}\mathrm{r}}\mathit{a}\mathit{r}\left(\mathit{V}-{\mathit{E}}_{\mathit{K}}\right)(1)$

where $g_{Kr}$ is the maximum conductance, and $a$ and $r$ denote a Hodgkin and Huxley activation gate and an inactivation gate, respectively. $V$ is the transmembrane voltage. $E_K$ is called Nernst potential or the reversal potential and obtained by Eq. (2).

${\mathit{E}}_{\mathit{K}}=\frac{\mathit{R}\mathit{T}}{\mathit{z}\mathit{F}}\ln \left(\frac{{\left[{\mathrm{K}}^{+}\right]}_{\mathrm{o}}}{{\left[{\mathrm{K}}^{+}\right]}_{\mathrm{i}}}\right)(2)$

where

$\mathit{R}\left(\mathrm{i}\mathrm{d}\mathrm{e}\mathrm{a}\mathrm{l}\mathrm{g}\mathrm{a}\mathrm{s}\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{s}\mathrm{t}\mathrm{a}\mathrm{n}\mathrm{t}\right):\mathrm{8.314472}\left[\frac{\mathrm{J}}{\mathrm{K}\bullet \mathrm{m}\mathrm{o}\mathrm{l}}\right]$

$\mathit{T}\left(\mathrm{a}\mathrm{b}\mathrm{s}\mathrm{o}\mathrm{l}\mathrm{u}\mathrm{t}\mathrm{e}\mathrm{t}\mathrm{e}\mathrm{m}\mathrm{p}\mathrm{e}\mathrm{r}\mathrm{a}\mathrm{t}\mathrm{u}\mathrm{r}\mathrm{e}\right):\mathrm{298.15}\left[\mathrm{K}\right]$

$\mathit{F}\left({\mathrm{F}\mathrm{a}\mathrm{r}\mathrm{a}\mathrm{d}\mathrm{a}\mathrm{y}}^{\prime }\mathrm{s}\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{s}\mathrm{t}\mathrm{a}\mathrm{n}\mathrm{t}\right):\mathrm{96485.3415}\left[\frac{\mathrm{C}}{\mathrm{m}\mathrm{o}\mathrm{l}}\right]$

$\mathit{z}\left(\mathrm{v}\mathrm{a}\mathrm{l}\mathrm{e}\mathrm{n}\mathrm{c}\mathrm{y}\mathrm{o}\mathrm{f}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{i}\mathrm{o}\mathrm{n}\mathrm{s}\right):+\mathrm{1} \mathrm{f}\mathrm{o}\mathrm{r} {\mathrm{K}}^{+}$

${\left[{\mathrm{K}}^{+}\right]}_{\mathrm{o}}\left(\mathrm{e}\mathrm{x}\mathrm{t}\mathrm{r}\mathrm{a}\mathrm{c}\mathrm{e}\mathrm{l}\mathrm{l}\mathrm{u}\mathrm{l}\mathrm{a}\mathrm{r}\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{c}\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{r}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}\right):\mathrm{4} \left[\mathrm{m}\mathrm{M}\right]$

${\left[{\mathrm{K}}^{+}\right]}_{\mathrm{i}}\left(\mathrm{i}\mathrm{n}\mathrm{t}\mathrm{r}\mathrm{a}\mathrm{c}\mathrm{e}\mathrm{l}\mathrm{l}\mathrm{u}\mathrm{l}\mathrm{a}\mathrm{r}\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{c}\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{r}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}\right):\mathrm{110}\left[\mathrm{m}\mathrm{M}\right]$

The model has nine parameters $\mathit{\theta }=\left\{g_{Kr},p_1,p_2,p_3,p_4,p_5,p_6,p_7,p_8\right\}$, and Figure 3 shows its structure, where

$\begin{array}{cc}\frac{\mathit{d}\mathit{a}}{\mathit{d}\mathit{t}}=\frac{{\mathit{a}}_{\mathrm{\infty }}-\mathit{a}}{{\mathit{\tau }}_{\mathit{a}}}& ,\frac{\mathit{d}\mathit{r}}{\mathit{d}\mathit{t}}=\frac{{\mathit{r}}_{\mathrm{\infty }}-\mathit{r}}{{\mathit{\tau }}_{\mathit{r}}}\\ {\mathit{a}}_{\mathrm{\infty }}=\frac{{\mathit{k}}_{\mathrm{1}}}{{\mathit{k}}_{\mathrm{1}}+{\mathit{k}}_{\mathrm{2}}}& ,{\mathit{r}}_{\mathrm{\infty }}=\frac{{\mathit{k}}_{\mathrm{4}}}{{\mathit{k}}_{\mathrm{3}}+{\mathit{k}}_{\mathrm{4}}}\\ {\mathit{\tau }}_{\mathit{a}}=\frac{\mathrm{1}}{{\mathit{k}}_{\mathrm{1}}+{\mathit{k}}_{\mathrm{2}}}& ,{\mathit{\tau }}_{\mathit{r}}=\frac{\mathrm{1}}{{\mathit{k}}_{\mathrm{3}}+{\mathit{k}}_{\mathrm{4}}}\end{array}$

$\begin{array}{c}{\mathit{k}}_{\mathrm{1}}={\mathit{p}}_{\mathrm{1}}\exp \left({\mathit{p}}_{\mathrm{2}}\mathit{V}\right)\\ {\mathit{k}}_{\mathrm{2}}={\mathit{p}}_{\mathrm{3}}\exp \left(-{\mathit{p}}_{\mathrm{4}}\mathit{V}\right)\\ {\mathit{k}}_{\mathrm{3}}={\mathit{p}}_{\mathrm{5}}\exp \left({\mathit{p}}_{\mathrm{6}}\mathit{V}\right)\\ {\mathit{k}}_{\mathrm{4}}={\mathit{p}}_{\mathrm{7}}\exp \left(-{\mathit{p}}_{\mathrm{8}}\mathit{V}\right)\end{array}$

where $k_1$ is an activation rate, $k_2$ is a deactivation rate, $k_3$ is an inactivation rate, and $k_4$ is a recovery rate.

FIGURE 3

FIGURE 3. The Hodgkin–Huxley model structure. The probabilities for states CI, I, O, and C are $\left(1-r\right)\left(1-a\right)$, $a\left(1-r\right)$, $ar$ and $r\left(1-a\right)$.

2.2 Dataset generation

A lot of data is required to train. The best condition for good performance is the presence of a large amount of experimental data. However, there are only 200, and it is very insufficient for learning and testing with them. Even if a large amount of data exists, fitting work for labeling requires a lot of time and resources, which is contrary to the purpose of our study. So, we generated a large amount of data using simulations of the hERG channel to compensate for the lack of data. The dataset consists of I_Kr as input data and nine parameters $\mathit{\theta }$ of the hERG model as the target data. Five hundred thousand data generated by simulation were used for training. For 200 experimental data, after sorting by name, odd-numbered data were configured as validation dataset and even-numbered data were configured as test dataset.

First, parameters for the hERG channel were generated under the following two conditions. The first condition is that each parameter follows a uniform distribution within a specific range, as in Eq. (3).

${\mathit{g}}_{\mathrm{K}\mathrm{r}}\sim \mathit{U}\left({\mathit{g}}_{\mathit{min}},{\mathit{g}}_{\mathit{max}}\right)$

${\mathit{p}}_{\mathrm{1}},{\mathit{p}}_{\mathrm{2}},{\mathit{p}}_{\mathrm{3}},{\mathit{p}}_{\mathrm{4}}\sim \mathit{U}\left(\mathit{a},\mathit{b}\right)(3)$

${\mathit{p}}_{\mathrm{5}},{\mathit{p}}_{\mathrm{6}},{\mathit{p}}_{\mathrm{7}},{\mathit{p}}_{\mathrm{8}}\sim \mathit{U}\left(\mathit{c},\mathit{d}\right)$

where $U\left(\bullet \right)$ represents a uniform distribution. In this study, for the conductance $g_{Kr}$, $g_{\mathit{min}}=100$ [pA/V] and $g_{\mathit{min}}=500000$ [pA/V]. For ${\mathrm{p}}_1,{\mathrm{p}}_3,{\mathrm{p}}_5,\mathrm{and}{\mathrm{p}}_7$, $a$ and $b$ are 0.0001 [s⁻¹] and 10⁶ [s⁻¹], respectively. For ${\mathrm{p}}_2,{\mathrm{p}}_4,{\mathrm{p}}_6,\mathrm{and}{\mathrm{p}}_8$, $c$ and $d$ are 0.0001 [V⁻¹] and 400 [V⁻¹], respectively. The second condition is that each parameter must satisfy the inequalities of Eq. (4):

$\mathrm{0.0167}<{\mathit{p}}_{\mathrm{1}}\exp \left({\mathit{p}}_{\mathrm{2}}*{\mathit{V}}_{\mathit{max}}\right)<{\mathrm{10}}^{\mathrm{6}}$

$\mathrm{0.0167}<{\mathit{p}}_{\mathrm{3}}\exp \left(-{\mathit{p}}_{\mathrm{4}}*{\mathit{V}}_{\mathit{min}}\right)<{\mathrm{10}}^{\mathrm{6}}(4)$

$\mathrm{0.0167}<{\mathit{p}}_{\mathrm{5}}\exp \left({\mathit{p}}_{\mathrm{6}}*{\mathit{V}}_{\mathit{max}}\right)<{\mathrm{10}}^{\mathrm{6}}$

$\mathrm{0.0167}<{\mathit{p}}_{\mathrm{7}}\exp \left(-{\mathit{p}}_{\mathrm{8}}*{\mathit{V}}_{\mathit{min}}\right)<{\mathrm{10}}^{\mathrm{6}}$

where $V_{\mathit{min}}=-0.12$ and $V_{\mathit{max}}=0.06$. The lower and upper bounds of the above conditions were determined by the constraints of physical and physiological phenomena (Beattie et al., 2018). Figure 4A shows the distribution of each parameter for 100,000 data generated by the above conditions.

FIGURE 4

FIGURE 4. (A) Distribution of the nine parameters generated by the two conditions of Eqs 3, 4. The distributions of $p_1$; $p_3$; $p_5$; $p_7$ are very clustered in a specific range. (B) Data distributions after applying log scale to $p_1$, $p_3$, $p_5$ and $p_7$.

Second, we used simulations to generate the current $I_{\mathrm{K}\mathrm{r}}$ corresponding to the parameters sampled above. Myokit (Clerx et al., 2016) with CVODE solver (Hindmarsh et al., 2005) was used for the simulation. The tolerance settings for CVODE were abs_tol = 10^–8 and rel_tol = 10^–10, as in the condition in (Lei et al., 2019). The length of the experimental data was 15.4 s with a sampling rate of 5 kHz. In this study, we reduced the data number to 1/50 with sampling one point every 100 because there seemed to be no problem reflecting the trend. Because we only used the data generated by simulations for training, the results of this study depend on the similarity between the simulation data and experimental data. Therefore, noises, $\alpha$, were added to the simulation data. The noises were extracted from a normal distribution $\alpha \sim N\left(0,{\sigma }^2\right)$. σ is 10.84, which was measured at the steady-state current in the experimental data.

${\mathit{I}}_{\mathrm{K}\mathrm{r}}^{\mathit{exp}}\approx {\mathit{I}}_{\mathrm{K}\mathrm{r}}^{\mathit{s}\mathit{i}\mathit{m}}+\mathit{\alpha }(5)$

where $I_{\mathrm{K}\mathrm{r}}^{exp}$ and $I_{\mathrm{K}\mathrm{r}}^{sim}$ represent the experimental and simulated current, respectively.

2.3 Preprocessing and hERGNet

Our method involves several simple preprocessing processes on the data for learning. In Figure 4A, The distributions of $p_1$, $p_3$, $p_5$ and $p_7$ are very clustered in a specific range. To make them as uniform as possible, the log scale was applied. The min–max normalization was then used to transform all parameter ranges between 0 and 1, as shown in Figure 4B.

Recurrent neural network (Sherstinsky, 2020) and LSTM (Hochreiter and Schmidhuber, 1997; Sherstinsky, 2020) series models have been widely used to analyze time-series data, such as the current data that we want to analyze, and recently, transformer-based models (Vaswani et al., 2017; Lin et al., 2022) are leading this field. Since CNN is designed for the purpose of extracting information between adjacent values of data, it obtains spatial information well in the local domain (Krizhevsky et al., 2017). Transformer calculates the relationship between all elements of input data through attention, so it understands overall features better than CNN, but is weaker than CNN in extracting local information, and requires a very large size dataset for this purpose (Dosovitskiy et al., 2020). We thought our goal was closer to finding changes in the characteristics and patterns at specific times than predicting the current change over time. Therefore, we adopted a CNN-based model rather than a transformer-based model, and among them, an EfficientNet (Tan and Le, 2019) type model. We called our model hERGNet. Our hERGNet is very simple, as shown in Figure 5. It consists of a CNN-encoder network and a pretrained EfficientNetV2-M (Tan and Le, 2021). The CNN-encoder network extracts the features of the spectrogram and increases the number of channels to three to obtain an image-like shape, thereby making it possible to use the pretrained EfficientNetV2-M. EfficientNetV2-M is responsible for finding parameters by extracting features from encoded data.

FIGURE 5

FIGURE 5. The structure of hERGNet. The spectrogram of current is converted into 2D data with three channels like the image shape through the CNN encoder, making it possible to utilize the pretrained EfficientNet.

We converted the current into a spectrogram with a frequency perspective, as shown in Figure 6, so that hERGNet can better learn the characteristics of the current data. By adding frequency features to current data consisting of only time and intensity, it has the advantage of increasing information about data and transforming it into a two-dimensional form like an image, making it easier for a 2D CNN-based model to learn. In this study, the parameters “n_fft,” “hop_length,” and “win_length” of Short-Time Fourier Transform (Owens and Murphy, 1988) for spectrogram transformation were set to 256, 12, and 48, respectively.

FIGURE 6

FIGURE 6. Spectrograms transformed from current data for 4 cells.

2.4 Training

Mean Squared Error (MSE) was used as the loss function, so the cost function J(θ) is the same as Eq. (6)

$\begin{array}{r}\mathit{J}\left(\mathit{\theta }\right)=\frac{\mathrm{1}}{\mathit{N}}{\displaystyle \sum }_{\mathit{i}=\mathrm{1}}^{\mathit{N}}\left({\mathit{f}}_{\mathit{\theta }}\left({\mathit{x}}_{\mathit{i}}\right)-{\mathit{y}}_{\mathit{i}}\right)\end{array}(6)$

where $x_i$ is the spectrogram of the current $I_{\mathrm{K}\mathrm{r}}$, $f_{\theta }$ is the neural network, and $y_i$ is the true parameters of the ion channel.

In general, it is known that the higher the resolution of the input, the higher the performance. However, the higher the resolution, the more memory is required, which is time-consuming to train due to the small batch size. Also, if you train all the generated 500,000 data from the beginning, the time will increase even more. Therefore, we first trained the hERGNet with a small resolution and a small number of data and then proceeded with transfer learning by increasing the resolution and the number of data. First, learning was performed on 300,000 spectrograms with a 97 × 97 resolution to 200 epochs. Then, we increased the resolution to 129 × 129 and performed transfer learning to 140 epochs using all 500,000 data.

2.5 Parameter inference

All fitting operations were the same as those in (Lei et al., 2019), and their open-source library PINTS (Clerx et al., 2019) was used. Furthermore, the CMA-ES algorithm (Hansen, 2006; Khan, 2018) was used as a global optimization algorithm to fit the model to the experimental data, and Markov Chain Monte Carlo (Jasra et al., 2007) with adaptive Metropolis (Haario et al., 2001) was used to explore the posterior probability distribution. Parameter inference was performed three times based on the initial value for each cell. The first was when the initial value was given as a parameter predicted by our hERGNet, the second was a prior value, and the last was given as a value randomly extracted from the previously described parameter distribution.

3 Results

When hERGNet was trained on 300,000 data with a 97 × 97 resolution until 200 epochs, the best MSE for 100 experimental data (validation data) was 0.00781. After 140 additional learnings on 500,000 data with an increased resolution to 129 × 129, we identified MSEs lowered to 0.003035. Then, we tested hERGNet on 100 experimental data not used for learning. The MSE was recorded as 0.002688, which is better than the results for the validation dataset. Figure 7 shows the prediction results for 100 experimental data. The fitting test was performed on 50 experimental data out of the 100 test data.

FIGURE 7

FIGURE 7. Prediction results for 100 cells. The parameters for most cells are clustered in a specific range. $p_6$ had the best predictions, while $p_2$ had the worst predictions.

First, when we generated current data with the predicted parameters, we compared how different it was from the experimental data as shown in Figure 8. As shown in the two figures above in Figure 8, the prediction was accurate enough that no fitting work was required for a significant number of cells. However, as shown in the two figures below in Figure 8, the prediction was not perfect, necessitating a fitting operation.

FIGURE 8

FIGURE 8. Black is experimental data, and red is simulated current data with parameters predicted by hERGNet. There are some very close predictions, such as A16 and A22, and results showing differences, such as A19 and G13. However, the flow and shape of the current are somewhat predictable.

Next, as shown in Table 1, we compared the results when the initial values were given as parameters predicted by hERGNet, prior parameters, and random parameters. The results confirmed that our method significantly improved the fitting operation. The initial value of hERGNet did not result in a single failure in the fitting operation for 50 cells. However, the prior parameter caused one local minima problem, and in the random parameter, 16 failures occurred out of 50 fittings, and 11 local minima problems occurred. We compared the fitting rates for cells that succeeded in parameter inference. As shown in Table 1, the average iteration was 341.8 in the predicted parameter, 546.4 in the prior parameter, and 601.3 in the random parameter. The average time was 396.3 s for the predicted parameters, 630.9 s for the prior parameter, and 686.0 s for the random parameters. Of the 50 cells, all but two, D17 and G13, showed faster fitting rates when using the predicted parameters.

TABLE 1

TABLE 1. When the predicted value by hERGNet was used, there was a great improvement in the fitting speed.

4 Discussion and conclusion

Parameter inference is an important part of the toxicity evaluation of drugs because it is possible to understand and predict physiological changes in cells caused by drugs by predicting the parameters of ion channels. However, the difficulty of the fitting and the time-consuming problem make us hesitant to use in silico. In this study, we propose a method for improving the fitting operation for the hERG channel model by setting the parameters predicted by hERGNet as initial values. The test results showed a clear improvement in the hERG model fitting. There was no fitting failure, and the time-consuming problem was also improved. Depending on the range of parameters, training the neural network required a lot of data generation and was time-consuming. However, if experiments are conducted with the same voltage protocol for other cells in the future, our method could be very useful for inferring the parameters of ion channels.

Our method still has a lot to improve. The first is to improve the fitting method rather than simply presenting initial values. This is because stochastic methods, such as CMA-ES, may not immediately find optimal parameters due to the characteristics of the method, even if parameters close to the correct answer are presented. The second is to increase the similarity between experimental and simulation data, which is, after all, the most important factor for AI to predict parameters. We trained our hERGNet only with simulation data. If the similarity between experimental and simulation data can be increased through noise removal, etc., the predicted parameters will be closer to the correct answer.

In fact, our ultimate goal is to predict parameters that are very close to the correct answer, eliminating the need for model fitting. If this is possible, a new paradigm will be presented in drug development or drug toxicity assessment. To this end, we will first conduct a study on parameter prediction in multiple ion channels. Parametric prediction for multiple ion channels may aid in greatly reducing the amount and cost of experiments performed in the non-clinical stage.

Data availability statement

The original contributions presented in the study are included in the article, further inquiries can be directed to the corresponding author.

Author contributions

Design and conceptualization of the study: JS and CL; Data analysis: JS, YK; Software development: JS; Writing and editing: JS and CL; All authors contributed to the revision of the manuscript, read, and approved the submitted version.

Funding

This research was supported by a grant (22213MFDS392) from the Ministry of Food and Drug Safety.

Acknowledgments

We thank the Ministry of Food and Drug Safety for the financial support and support of our research.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Aghasafari P., Yang P. C., Kernik D. C., Sakamoto K., Kanda Y., Kurokawa J., et al. (2021). A deep learning algorithm to translate and classify cardiac electrophysiology. Elife 10, e68335. doi:10.7554/ELIFE.68335

PubMed Abstract | CrossRef Full Text | Google Scholar

Alhusseini M. I., Abuzaid F., Rogers A. J., Zaman J. A. B., Baykaner T., Clopton P., et al. (2020). Machine learning to classify intracardiac electrical patterns during atrial fibrillation: Machine learning of atrial fibrillation. Circ. Arrhythm. Electrophysiol. 13, e008160. doi:10.1161/CIRCEP.119.008160

PubMed Abstract | CrossRef Full Text | Google Scholar

Beattie K. A., Hill A. P., Bardenet R., Cui Y., Vandenberg J. I., Gavaghan D. J., et al. (2018). Sinusoidal voltage protocols for rapid characterisation of ion channel kinetics. J. Physiol. 596, 1813–1828. doi:10.1113/JP275733

PubMed Abstract | CrossRef Full Text | Google Scholar

Clerx M., Collins P., de Lange E., Volders P. G. A. (2016). Myokit: A simple interface to cardiac cellular electrophysiology. Prog. Biophys. Mol. Biol. 120, 100–114. doi:10.1016/j.pbiomolbio.2015.12.008

PubMed Abstract | CrossRef Full Text | Google Scholar

Clerx M., Robinson M., Lambert B., Lei C. L., Ghosh S., Mirams G. R., et al. (2019). Probabilistic inference on noisy time series (PINTS). J. Open Res. Softw. 7, 23. doi:10.5334/jors.252

CrossRef Full Text | Google Scholar

Dosovitskiy A., Beyer L., Kolesnikov A., Weissenborn D., Zhai X., Unterthiner T., et al. (2020). An image is worth 16x16 words: Transformers for image recognition at scale. Available At https://arxiv.org/abs/2010.11929.doi:10.48550/arXiv.2010.11929

CrossRef Full Text | Google Scholar

Food and Drug Administration (2005). International conference on harmonisation; guidance on S7B nonclinical evaluation of the potential for delayed ventricular repolarization (QT interval prolongation) by human pharmaceuticals; availability. Notice. Fed. Regist. 70, 61133–61134.

PubMed Abstract | Google Scholar

Friedrichs G. S., Patmore L., Bass A. (2005). Non-clinical evaluation of ventricular repolarization (ICH S7B): Results of an interim survey of international pharmaceutical companies. J. Pharmacol. Toxicol. Methods 52, 6–11. doi:10.1016/j.vascn.2005.05.001

PubMed Abstract | CrossRef Full Text | Google Scholar

Grisetti G., Guadagnino T., Aloise I., Colosi M., della Corte B., Schlegel D. (2020). Least squares optimization: From theory to practice. Robotics 9, 51. doi:10.3390/robotics9030051

CrossRef Full Text | Google Scholar

Haario H., Saksman E., Tamminen J. (2001). An adaptive Metropolis algorithm. Bernoulli 7, 223–242. doi:10.2307/3318737

CrossRef Full Text | Google Scholar

Hansen N. (2006). “The cma evolution strategy: A comparing review,” in Towards a new evolutionary computation (Berlin, Heidelberg: Springer Berlin Heidelberg), 75–102. doi:10.1007/3-540-32494-1_4

CrossRef Full Text | Google Scholar

He K., Zhang X., Ren S., Sun J. (2016). “Deep residual learning for image recognition,” in Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, June 2016 (IEEE), 770–778. doi:10.1109/CVPR.2016.90

CrossRef Full Text | Google Scholar

Hindmarsh A. C., Brown P. N., Grant K. E., Lee S. L., Serban R., Shumaker D. E., et al. (2005). Sundials. ACM Trans. Math. Softw. 31, 363–396. doi:10.1145/1089014.1089020

CrossRef Full Text | Google Scholar

Hochreiter S., Schmidhuber J. (1997). Long short-term memory. Neural comput. 9, 1735–1780. doi:10.1162/neco.1997.9.8.1735

PubMed Abstract | CrossRef Full Text | Google Scholar

Jasra A., Stephens D. A., Holmes C. C. (2007). On population-based simulation for static inference. Stat. Comput. 17, 263–279. doi:10.1007/s11222-007-9028-9

CrossRef Full Text | Google Scholar

Jeong D. U., Lim K. M. (2021). Convolutional neural network for classification of eight types of arrhythmia using 2D time–frequency feature map from standard 12-lead electrocardiogram. Sci. Rep. 11, 20396. doi:10.1038/s41598-021-99975-6

PubMed Abstract | CrossRef Full Text | Google Scholar

Jurkiewicz N. K., Sanguinetti M. C. (1993). Rate-dependent prolongation of cardiac action potentials by a methanesulfonanilide class III antiarrhythmic agent. Specific block of rapidly activating delayed rectifier K⁺ current by dofetilide. Circ. Res. 72, 75–83. doi:10.1161/01.RES.72.1.75

PubMed Abstract | CrossRef Full Text | Google Scholar

Kelleni M. T., Abdelbasset M. (2018). “Drug induced cardiotoxicity: Mechanism, prevention and management,” in Cardiotoxicity (London, UK: InTech). doi:10.5772/intechopen.79611

CrossRef Full Text | Google Scholar

Khan N. (2018). A parallel implementation of the covariance matrix adaptation evolution strategy. Available At https://arxiv.org/abs/1805.11201.doi:10.48550/arXiv.1805.11201

CrossRef Full Text | Google Scholar

Krizhevsky A., Sutskever I., Hinton G. E. (2017). ImageNet classification with deep convolutional neural networks. Commun. ACM 60, 84–90. doi:10.1145/3065386

CrossRef Full Text | Google Scholar

Lasser K. E., Allen P. D., Woolhandler S. J., Himmelstein D. U., Wolfe S. M., Bor D. H. (2002). Timing of new Black box warnings and withdrawals for prescription medications. JAMA 287, 2215–2220. doi:10.1001/jama.287.17.2215

PubMed Abstract | CrossRef Full Text | Google Scholar

Lei C. L., Clerx M., Gavaghan D. J., Polonchuk L., Mirams G. R., Wang K. (2019). Rapid characterization of hERG channel kinetics I: Using an automated high-throughput system. Biophys. J. 117, 2438–2454. doi:10.1016/j.bpj.2019.07.029

PubMed Abstract | CrossRef Full Text | Google Scholar

Lin T., Wang Y., Liu X., Qiu X. (2022). A survey of transformers. AI Open 3, 111–132. doi:10.1016/j.aiopen.2022.10.001

CrossRef Full Text | Google Scholar

Malik M., Camm A. J. (2001). Evaluation of drug-induced QT interval prolongation: Implications for drug approval and labelling. Drug Saf. 24, 323–351. doi:10.2165/00002018-200124050-00001

PubMed Abstract | CrossRef Full Text | Google Scholar

O’Shea K., Nash R. (2015). An introduction to convolutional neural networks. Available At https://arxiv.org/abs/1511.08458.doi:10.48550/arXiv.1511.08458

CrossRef Full Text | Google Scholar

Owens F. J., Murphy M. S. (1988). A short-time Fourier transform. Signal Process. 14, 3–10. doi:10.1016/0165-1684(88)90040-0

CrossRef Full Text | Google Scholar

Pathmanathan P., Shotwell M. S., Gavaghan D. J., Cordeiro J. M., Gray R. A. (2015). Uncertainty quantification of fast sodium current steady-state inactivation for multi-scale models of cardiac electrophysiology. Prog. Biophys. Mol. Biol. 117, 4–18. doi:10.1016/j.pbiomolbio.2015.01.008

PubMed Abstract | CrossRef Full Text | Google Scholar

Radosavovic I., Kosaraju R. P., Girshick R., He K., Dollár P. (2020). “Designing network design spaces,” in Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, June 2020, 10428–10436. doi:10.48550/arXiv.2003.13678

CrossRef Full Text | Google Scholar

Rogers A. J., Selvalingam A., Alhusseini M. I., Krummen D. E., Corrado C., Abuzaid F., et al. (2021). Machine learned cellular phenotypes in cardiomyopathy predict sudden death. Circ. Res. 128, 172–184. doi:10.1161/CIRCRESAHA.120.317345

PubMed Abstract | CrossRef Full Text | Google Scholar

Ruder S. (2016). An overview of gradient descent optimization algorithms. Available At https://arxiv.org/abs/1609.04747.doi:10.48550/arXiv.1609.04747

CrossRef Full Text | Google Scholar

Sanguinetti M. C., Tristani-Firouzi M. (2006). hERG potassium channels and cardiac arrhythmia. Nature 440, 463–469. doi:10.1038/nature04710

PubMed Abstract | CrossRef Full Text | Google Scholar

Sevakula R. K., Au-Yeung W. M., Singh J. P., Heist E. K., Isselbacher E. M., Armoundas A. A. (2020). State-of-the-Art machine learning techniques aiming to improve patient outcomes pertaining to the cardiovascular system. J. Am. Heart Assoc. 9, e013924. doi:10.1161/JAHA.119.013924

PubMed Abstract | CrossRef Full Text | Google Scholar

Sherstinsky A. (2020). Fundamentals of recurrent neural network (RNN) and long short-term memory (LSTM) network. Phys. D. 404, 132306. doi:10.1016/j.physd.2019.132306

CrossRef Full Text | Google Scholar

Tan M., Le Q. v. (2019). “EfficientNet: Rethinking model scaling for convolutional neural networks,” in International conference on machine learning (PMLR), 6105–6114. doi:10.48550/arXiv.1905.11946

CrossRef Full Text | Google Scholar

Tan M., Le Q. v. (2021). EfficientNetV2: Smaller models and faster training. Proc. Mach. Learn. Res. (PMLR) 139, 10096–10106. doi:10.48550/arXiv.2104.00298

CrossRef Full Text | Google Scholar

ten Tusscher K. H. W. J., Noble D., Noble P. J., Panfilov A. v. (2004). A model for human ventricular tissue. Am. J. Physiology-Heart Circulatory Physiology 286, H1573–H1589. doi:10.1152/ajpheart.00794.2003

CrossRef Full Text | Google Scholar

Vaswani A., Brain G., Shazeer N., Parmar N., Uszkoreit J., Jones L., et al. (2017). “Attention is all you need,” in Proceedings of the Neural Information Processing Systems (NIPS), Long Beach, CA, USA, December 2017. doi:10.48550/arXiv.1706.03762

CrossRef Full Text | Google Scholar

Zeng J., Laurita K. R., Rosenbaum D. S., Rudy Y. (1995). Two components of the delayed rectifier K⁺ current in ventricular myocytes of the Guinea pig type. Circ. Res. 77, 140–152. doi:10.1161/01.RES.77.1.140

PubMed Abstract | CrossRef Full Text | Google Scholar