Multivariate generalized mixed-effects models for screening multiple adverse drug reactions in spontaneous reporting systems

Introduction: For assessing drug safety using spontaneous reporting system databases, quantitative measurements, such as proportional reporting rate (PRR) and reporting odds ratio (ROR), are widely employed to assess the relationship between a drug and a suspected adverse drug reaction (ADR). The databases contain numerous ADRs, and the quantitative measurements need to be calculated by performing the analysis multiple times for each ADR. We proposed a novel, simple, and easy-to-implement method to estimate the PRR and ROR of multiple ADRs in a single analysis using a generalized mixed-effects model for signal detection.

Methods: The proposed method simultaneously analyzed the association between any drug and numerous ADRs, as well as estimated the PRR and ROR for a specific combination of drugs and suspected ADRs. Furthermore, the proposed method was applied to detect drug-drug interactions associated with the concurrent use of two or more drugs.

Results and discussion: In our simulation studies, the false-positive rate and sensitivity of the proposed method were similar to those of the traditional PRR and ROR. The proposed method detected known ADRs when applied to the Food and Drug Administration Adverse Event Reporting System database. As an important advantage, the proposed method allowed the simultaneous evaluation of several ADRs using multiple drugs.

1 Introduction

During the clinical development of new drugs, collecting sufficient information on drug safety poses a considerable challenge. Hence, spontaneous reporting systems are crucial sources for post-marketing drug safety surveillance. Importantly, these systems are commonly used to detect suspected adverse drug reactions (ADRs) and generate potential ADRs in real-world settings. Since the 1960s, regulatory authorities such as the US Food and Drug Administration (FDA) have established databases for spontaneous reporting.

When assessing drug safety using spontaneous reporting system databases, quantitative signal detection methods can be valuable for identifying the relationship between a drug and suspected ADR, given the considerable amount of data obtained. This data mining approach is crucial for the early detection of safety signals and for generating hypotheses regarding new ADRs. Several methods, including the proportional reporting rate (PRR) (Evans et al., 2001), reporting odds ratio (ROR) (Rothman et al., 2004), Bayesian confidence propagation neural network (BCPNN) (Bate et al., 1998), and multi-item gamma Poisson shrinker (MGPS) (DuMouchel, 1999), have been proposed and employed by regulatory authorities for signal detection. These methods typically assess disproportionality in the observed and expected numbers of counts for specific combinations of a drug and suspected ADRs. Thus, if the ratio of the observed count to the expected count (henceforth, the O/E ratio) estimated using these methods is far from 1, it is considered a signal. Although the performance of these methods has been extensively evaluated and compared (van Puijenbroek et al., 2002; Kubota et al., 2004; Almenoff et al., 2006; Matsushita et al., 2007; Hochberg et al., 2009; Ahmed et al., 2010; Bunchuailua et al., 2010; Chen et al., 2015), no gold standard method has been established worldwide.

Unlike the BCPNN and MGPS, the PRR and ROR are easy to calculate and interpret. The PRR is a simple risk ratio (or relative risk), while the ROR is a simple odds ratio derived from a 2 × 2 contingency table (Table 1), with both measurements closely related to statistical models occasionally used for signal detection. Considering Poisson regression models, the parameter estimates in the model yield the PRR, which is the reporting ratio of drug use to non-use. Likewise, the ROR can be estimated using a logistic regression model. In particular, these models help assess drug-drug interactions (DDIs) during the concurrent administration of two or more drugs (Thakrar et al., 2007). By including a statistical interaction term in the model, the presence of DDIs can be evaluated using a spontaneous reporting system (van Puijenbroek et al., 1999; van Puijenbroek et al., 2000). Importantly, these modeling approaches can detect only one ADR, and multiple models need to be constructed for each ADR to estimate the PRR and ROR of various ADRs. For example, to evaluate 100 types of ADRs, 100 regression models must be constructed with each ADR as a response variable.

TABLE 1

TABLE 1. Two-by-two contingency table for summarizing the specific ADR reported in the target drug.

As another approach for detecting DDIs, Norén et al. (2008) proposed a criterion using the O/E ratio of the number of reports for the ADR for a combination of two drugs. Gosho et al. (2017) also proposed a criterion based on chi-square test statistics to measure the discrepancy between the observed and expected number of reports. Although these methods have been effectively reviewed and compared (Noguchi et al., 2019; Noguchi et al., 2020), the detection of DDIs between three or more drugs is not possible. Moreover, similar to the analysis using regression models, the methods can detect only one ADR, and multiple analyses are required to assess each ADR.

In the present study, we propose a novel, simple, and easy-to-implement method using Poisson and logistic mixed-effect models for signal detection. The proposed method could simultaneously analyze the relationship between any drug and numerous ADRs and estimate the PRR and ROR for a specific combination of drugs and suspected ADRs. Furthermore, the proposed method could be applied to detect DDIs during the concurrent administration of two or more drugs. We also provide a sample SAS code for implementing the proposed method.

2 Methodology

2.1 Notation

Spontaneous reporting systems include multiple drugs and ADRs in each report. This information can be summarized in a 2 × 2 contingency table, as shown in Table 1. We used $I$ drugs and $J$ ADRs. Here, $n_{ij}$ is the number of events reported for the $i$ th drug ($\left.i=1,\dots ,I\right)$ and $j$ th ADR $\left(j=1,\dots ,J\right)$; $n_{i\cdot }$ is the total number of events reported with the target drug $i$; $n_{\cdot j}$ is the total number of events reported with specific ADR $j$; $n$ indicates the total number of ADRs reported with any drug; $p_{ij}$ is the incidence probability for the $j$ th ADR with the $i$ th drug. Let $i^{-}$ denote all other drugs except the target drug $i$.

2.2 Standard strategy and signal detection methods

Typically, signal detection is used to assess disproportionality in the observed number of counts, $n_{ij}$, and the expected number of counts, $E_{ij}$, for a specific combination of drug $i$ and ADR $j$. $E_{ij}=n_i{n}_j/n$ is defined as the expected number of counts under the null hypothesis, with no association between the $i$ th drug and the $j$ th ADR. O/E ratios were evaluated using several methods. The direct estimator of the O/E ratio is the relative reporting ratio, defined as:

$\frac{n_{ij}/n_i}{n_j/n}=\frac{n_{ij}}{E_{ij}}$

The PRR was calculated as the ratio of the proportion of the ADR $j$ reported with drug $i$ to the proportion of the same ADR reported with all other drugs combined:

$\frac{n_{ij}/n_i}{\left(n_j-n_{ij}\right)/\left(n-n_i\right)}(1)$

The PRR can be interpreted as a measure of the reporting rate, with and without target drug $i$. In addition, PRR is considered an approximation of the relative reporting ratio, given that $n_{ij}\ll n_j$ and $n_i\ll n$ in almost cases. If PRR = 1, the absence of an association between the $i$ th drug and the $j$ th ADR can be assumed.

The ROR was calculated as the ratio of the odds for ADR $j$ reported with drug $i$ to the odds that the same ADR was reported with all other drugs combined, as follows:

$\frac{n_{ij}/\left(n_i-n_{ij}\right)}{\left(n_j-n_{ij}\right)/\left(n-n_i-n_j+n_{ij}\right)}(2)$

If the lower limit of the 95% confidence interval (CI) for PRR is greater than 1, the relationship between the target drug and specific ADR was detected as a signal; the same was applied to ROR in Eq. 2.

2.3 Poisson mixed-effect model and PRR

We assumed that the random variable $n_{ij}$ follows a Poisson distribution, expressed as $n_{ij}\sim \text{\hspace{0.17em}Poisson}\left(n_i{p}_{ij}\right)$, where $p_{ij}$ is the incidence probability of ADR $j$ when drug $i$ is used (Table 1). The probability function is expressed as

$\mathrm{Pr}\left(n_{ij}\left|\right.{\lambda }_{ij}\right)=\frac{\exp \left(-{\lambda }_{ij}\right){{\lambda }_{ij}}^{n_{ij}}}{n_{ij}!},n_{ij}=0,1,2,\dots$

Here, ${\lambda }_{ij}$ was defined as the mean (expected) value of $n_{ij}$. Accordingly, ${\lambda }_{ij}=n_i{p}_{ij}$. The relationship between mean value ${\lambda }_{ij}$ and covariate $x_i$ is generally modeled using a natural log link function, as follows:

$\ln \left({\lambda }_{ij}\right)={\beta }_0+{\beta }_i{x}_i(3)$

where ${\beta }_i$ is the unknown regression parameter for drug $i$. We aimed to evaluate all ADRs ($\left.j=1,\dots ,J\right)$ when drug $i$ is used. Considering that $x_i$ denotes the binary indicator for drug $i$, $x_i=1$ if the use of drug $i$ is reported, and $x_i=0$ if the use of other drugs (excluding the $i$ th drug) is reported. In this model, ${\beta }_i$ can be interpreted as the marginal effect of all ADRs using drug $i$ compared to the use of other drugs. Thus, ${\beta }_i$ is a common effect that does not specify ADRs. Furthermore, $\exp \left({\beta }_i\right)$ is the PRR for drug $i$ that is not ADR specific. However, this interpretation of ${\beta }_i$ must be oversimplified and cannot detect the signal of a specific ADR.

Next, to assess a specific ADR, we included a random effect in Eq. 3, with the linear predictor ${\eta }_{ij}$ expressed as follows:

${\eta }_{ij}=\ln \left({\lambda }_{ij}\right)={\beta }_0+b_0+\left({\beta }_i+b_j\right)x_i(4)$

where $b_0$ and $b_j$ are random effects for the intercept and $j$ th ADR, respectively, assumed to follow normal distributions, $b_0\sim N\left(0,{\gamma }_0\right)$ and $\left.b_j\sim N\left(0,\right.{\gamma }_j\right)$. Here, ${\gamma }_0$ and ${\gamma }_j$ are the variances of the random effects. Eq. 4, known as the Poisson mixed-effect model, was used to estimate the mean value ${\lambda }_{ij}$ for each drug and each ADR.

Based on Eq. 4, the linear predictor for each $x_i$ was

$\ln \left({\lambda }_{ij}\right)=\left\{\begin{array}{cc}{\beta }_0+b_0+{\beta }_i+b_j,& x_i=1\\ {\beta }_0+b_0,& x_i=0\end{array}\right.$

Thus, the PRR of ADR $j$ for drug $i$ is

${\text{PRR}}_{ij}=\exp \left({\beta }_i+b_j\right).$

Using Eq. 4, we simultaneously estimated the PRRs of all ADRs (i.e., any $j$) for drug $i$.

Extending Eq. 4 allowed the simultaneous evaluation effects mediated by multiple drugs; for example, consider a DDI in which two drugs ($i$ and $i^{\prime }$) are administered simultaneously. In this case, the linear predictor in Eq. 4 is expressed as follows:

$\ln \left({\lambda }_{i{i}^{\prime }j}\right)={\beta }_0+b_0+{\beta }_i{x}_i+{\beta }_{i^{\prime }}{x}_{i^{\prime }}+{\beta }_{i{i}^{\prime }}{x}_i{x}_{i^{\prime }}+b_{ij}{x}_i+b_{i^{\prime }j}{x}_{i^{\prime }}+b_{i{i}^{\prime }j}{x}_i{x}_{i^{\prime }}$

where $x_i=1$ if drug $i$ was used, $x_i=0$ otherwise, $x_{i^{\prime }}=1$ if drug $i^{\prime }$ was used, and $x_{i^{\prime }}=0$ otherwise. ${\beta }_i$, ${\beta }_{i^{\prime }}$, and ${\beta }_{i{i}^{\prime }}$ are unknown regression parameters for $x_i$, $x_{i^{\prime }}$, and $x_i{x}_{i^{\prime }}$, respectively; $b_0$, $b_{ij}$, $b_{i^{\prime }j}$, and $b_{i{i}^{\prime }j}$ are random effects for intercepts, $x_i$, $x_{i^{\prime }}$, and $x_i{x}_{i^{\prime }}$, respectively. Assuming that $b_0\sim N\left(0,{\gamma }_0\right)$, $b_{ij}\sim N\left(0,{\gamma }_i\right)$, $b_{i^{\prime }j}\sim N\left(0,{\gamma }_{i^{\prime }}\right)$, and $b_{i{i}^{\prime }j}\sim N\left(0,{\gamma }_{i{i}^{\prime }}\right)$, each linear predictor can be calculated as follows:

$\ln \left({\lambda }_{i{i}^{\prime }j}\right)=\left\{\begin{array}{cc}{\beta }_0+b_0,& x_i=0\text{\hspace{0.17em}and\hspace{0.17em}}{x}_{i^{\prime }}=0\\ {\beta }_0+b_0+{\beta }_i+b_{ij},& x_i=1\text{\hspace{0.17em}and\hspace{0.17em}}{x}_{i^{\prime }}=0\\ {\beta }_0+b_0+{\beta }_{i^{\prime }}+b_{i^{\prime }j},& x_i=0\text{\hspace{0.17em}and\hspace{0.17em}}{x}_{i^{\prime }}=1\\ {\beta }_0+b_0+{\beta }_i+{\beta }_{i^{\prime }}+{\beta }_{i{i}^{\prime }}+b_{ij}+b_{i^{\prime }j}+b_{i{i}^{\prime }j},& x_i=1\text{\hspace{0.17em}and\hspace{0.17em}}{x}_{i^{\prime }}=1\end{array}\right.$

Thus, the PRRs of ADR $j$ for drugs $i$ and $i^{\prime }$ and the combined use of drugs $i$ and $i^{\prime }$ are as follows:

${\text{PRR}}_{ij}=\exp \left({\beta }_i+b_{ij}\right),{\text{PRR}}_{i^{\prime }j}=\exp \left({\beta }_{i^{\prime }}+b_{i^{\prime }j}\right),\text{and\hspace{0.17em}\hspace{0.17em}}{\text{PRR}}_{i{i}^{\prime }j}=\exp \left({\beta }_{i{i}^{\prime }}+b_{i{i}^{\prime }j}\right),$

Here, ${\text{PRR}}_{ij}$ is the PRR of ADR $j$ for drug $i$, ${\text{PRR}}_{i^{\prime }j}$ is the PRR of ADR $j$ for drug $i^{\prime }$, and ${\text{PRR}}_{i{i}^{\prime }j}$ is the PRR of ADR $j$ under the concomitant use of drugs $i$ and $i^{\prime }$. The proposed method could allow the detection of DDIs during the concurrent use of two or more drugs, as it allows for flexible modeling by including a statistical interaction term. The proposed method is based on a multiplicative model for DDI, whereas the criteria for detecting DDIs established by Norén et al. (2008) and Gosho et al. (2017) are based on an additive model for DDI (Thakrar et al., 2007).

The fixed and random effects in Eqs 3, 4 were estimated using the restricted pseudo-likelihood method (Wolfinger and O’Connell, 1993). The PRR and its 95% CI were estimated using the estimation of ${\beta }_i$ and $b_{ij}$, ${\widehat{\beta }}_i$ and ${\widehat{b}}_{ij}$ and their variance estimates via pseudo-likelihood theory. Stroup (2013) provides a more detailed explanation regarding the theory of generalized mixed-effect models, such as Poisson and logistic mixed-effect models. If the lower limit of the 95% CI for the PRR was >1, the relationship between the target drug and the specific event was detected as a signal.

The simple PRR in Eq. 1 cannot be applied for signal detection when $n_{ij}=0$, given that the 95% CI for PRR in Eq. 1 cannot be estimated when $n_{ij}=0$. However, the proposed method could provide a 95% CI for the PRR estimated using Eq. 4 even when $n_{ij}=0$.

2.4 Logistic mixed-effect model and ROR

The modeling strategies described in Section 2.3 can be easily applied to logistic regression analysis. We assumed that the random variable $n_{ij}$ follows the binomial distribution $n_{ij}\sim \text{\hspace{0.17em}Bin}\left(n_i,p_{ij}\right)$. As described in Section 2.3., the logistic mixed-effects model can be expressed as follows:

${\eta }_{ij}=\ln \frac{p_{ij}}{1-p_{ij}}={\beta }_0+b_0+\left({\beta }_i+b_j\right)x_i(5)$

Using Eq. 5, the linear predictor for each $x_i$ can be calculated as follows:

$\ln \frac{p_{ij}}{1-p_{ij}}=\left\{\begin{array}{cc}{\beta }_0+b_0+{\beta }_i+b_j,& x_i=1\\ {\beta }_0+b_0,& x_i=0\end{array}\right.$

Thus, the ROR of ADR $j$ for drug $i$ is

${\text{ROR}}_{ij}=\exp \left({\beta }_i+b_j\right).$

Based on Eq. 5, we could simultaneously estimate the RORs of all ADRs (i.e., any $j$) for drug $i$.

Using the method described in Section 2.3, Eq. 5 was used to simultaneously evaluate the effects of multiple drugs. For example, consider a DDI in which two drugs are administered simultaneously. The linear predictor in Eq. 5 is expressed as follows:

$\ln \frac{p_{i{i}^{\prime }j}}{1-p_{i{i}^{\prime }j}}={\beta }_0+b_0+{\beta }_i{x}_i+{\beta }_{i^{\prime }}{x}_{i^{\prime }}+{\beta }_{i{i}^{\prime }}{x}_i{x}_{i^{\prime }}+b_{ij}{x}_i+b_{i^{\prime }j}{x}_{i^{\prime }}+b_{i{i}^{\prime }j}{x}_i{x}_{i^{\prime }}$

Thus, the RORs of ADR $j$ for drugs $i$ and $i^{\prime }$ and the combined use of drugs $i$ and $i^{\prime }$ are as follows:

${\text{ROR}}_{ij}=\exp \left({\beta }_i+b_{ij}\right),{\text{ROR}}_{i^{\prime }j}=\exp \left({\beta }_{i^{\prime }}+b_{i^{\prime }j}\right),\text{and\hspace{0.17em}\hspace{0.17em}}{\text{ROR}}_{i{i}^{\prime }j}=\exp \left({\beta }_{i{i}^{\prime }}+b_{i{i}^{\prime }j}\right).$

${\text{ROR}}_{ij}$ is the ROR of ADR $j$ for drug $i$, ${\text{ROR}}_{i^{\prime }j}$ is the ROR of ADR $j$ for drug $i^{\prime }$, and ${\text{ROR}}_{i{i}^{\prime }j}$ is the ROR of ADR $j$ during the concomitant use of drugs $i$ and $i^{\prime }$. If the lower limit of the 95% CI for the PRR was >1, the relationship between the target drug and the specific event was detected as a signal.

If $p_{ij}$ is small, the ROR well-approximated the PRR. Given that $p_{ij}$ is usually small in signal detection analyses, ROR and PRR did not differ significantly in almost all cases.

3 Application

We analyzed the FDA Adverse Event Reporting System (FAERS), a well-known database comprising adverse event reports designed to support the FDA’s post-marketing drug safety surveillance program. FAERS includes seven data files: demographics (e.g., sex and age), drugs (e.g., drug name and route of drug administration), reaction (e.g., terms of an adverse event), outcome (patient outcome), report source (code for the source of the report), therapy date (e.g., the date on which the therapy was started and stopped), and indications for use. Adverse events are determined using the Medical Dictionary for Regulatory Activities (MedDRA) as the preferred term.

Recently, sodium glucose-linked transporter 2 (SGLT2) inhibitors, a class of oral antidiabetic drugs, have been widely used to treat type 2 diabetes. The FDA approved canagliflozin as the first SGLT2 inhibitor for treating type 2 diabetes in 2013 (Mosley et al., 2015). Since then, six SGLT2 inhibitors have been approved in the US and Japan. The proposed logistic mixed-effect and Poisson mixed-effect models were applied to the signal detection analysis of these SGLT2 inhibitors for potential ADRs in two scenarios: 1) signal detection for one drug and 2) DDIs following the concomitant use of two drugs, as well as a simulation study.

Data files were downloaded from the FDA website (https://fis.fda.gov/extensions/FPD-QDE-FAERS/FPD-QDE-FAERS.html) and analyzed between 2014 Q1 and 2022 Q4 after the launch of SGLT2 inhibitors. The analyses included records describing 13,344,838 patient characteristics, 54,869,999 drug properties, and 43,029,283 reactions/events. All analyses were performed using SAS software version 9.4 (SAS Institute, Cary, NC). The SAS code is provided in the Supplementary Material.

3.1 Scenario 1 (single drug)

We applied the two proposed models to the FAERS database to screen for ADRs when seven SGLT2 inhibitors (canagliflozin, empagliflozin, ipragliflozin, dapagliflozin, tofogliflozin, luseogliflozin, and ertugliflozin) were used. As a reference, we also applied the traditional ROR and PRR to the database.

A list of ADRs determined as signals using the proposed methods is presented in Supplementary Table S1. The total run time of the analysis was 30 min (2.2 GHz Intel Xeon processor with 64 GB memory). When the lower limit of the 95% CI for the proposed ROR and PRR was greater than 1, the ADR was considered detected. The results of the proposed ROR were similar to those of the proposed PRR owing to the low reporting rate. In addition, the ADRs detected using the proposed methods were similar to those detected using traditional methods. As numerous ADRs were detected (Supplementary Table S1), we summarized the ADRs detected with three or more SGLT2 inhibitors (Table 2). We only presented ROR results because there was no significant difference between PRR and ROR. In addition, owing to space limitation, only ADRs classified as “metabolism and nutrition disorders metabolism” in the system organ class (SOC) of MedDRA or recognized in the package insert of SGLT2 inhibitors are listed in Table 2.

TABLE 2

TABLE 2. ADRs detected with three or more kinds of SGLT2 inhibitors, and the lower limit of 95% CI for ROR calculated using the proposed model in Scenario 1 (single drug).

In the current analysis, hypoglycemia, ketoacidosis, and several infarctions, all well-known ADRs of SGLT2 inhibitors, were detected with almost all SGLT2 inhibitors. Euglycemic diabetic ketoacidosis, ketoacidosis, and pyelonephritis were detected with all seven SGLT2 inhibitors. The detection results of the proposed methods were similar to those observed with the traditional ROR and PRR (Supplementary Table S1).

3.2 Scenario 2 (DDI)

Patients with diabetes frequently coadminister SGLT2 inhibitors with glimepiride, a sulfonylurea that stimulates pancreatic β cells to release insulin. Accordingly, the proposed models were applied to assess DDIs between seven SGLT2 inhibitors and glimepiride.

A list of ADRs determined as signals using the proposed methods is presented in Supplementary Table S2. The number of drugs reported is also shown in Supplementary Table S3. The total run time of the analysis was 232 min (2.2 GHz Intel Xeon processor with 64 GB memory). For the proposed PRR and ROR, an ADR was considered to be detected when the lower limits of 95% CI were >1. The results of the proposed ROR were similar to those of the proposed PRR. As observed in Scenario 1, Table 3 presents a list of ADRs detected with two or more types of SGLT2 inhibitors using the proposed methods. Only ROR results are presented, given the absence of any significant difference between the PRR and ROR. Owing to space limitations, only ADRs classified as “metabolism and nutrition disorders metabolism,” “cardiac disorders,” “nervous system disorders,” “renal and urinary disorders,” “reproductive system and breast disorders,” “infections and infestations,” and “surgical and medical procedures” in SOC of MedDRA are listed in Table 3. These SOC classes include ADRs that are likely to occur with the use of SGLT2 inhibitors.

TABLE 3

TABLE 3. ADRs detected with two or more kinds of SGLT2 inhibitors with glimepiride and the lower limit of 95% CI for ROR calculated using the proposed model in Scenario 2 (DDIs).

In addition, ketosis-related ADRs were frequently detected following the concomitant use of glimepiride and several SGLT2 inhibitors. Coronary artery stenosis, acute myocardial infarction, cardiac failure, cerebral infarction, renal impairment, and acute pyelonephritis were detected in patients treated with four SGLT2 inhibitors (Table 3).

4 Simulation study

We examined the performance of the proposed method using a simulation study. We calculated the ROR and PRR using the logistic mixed-effect and Poisson mixed-effect models, respectively, as defined in Section 2.

The performance was evaluated in terms of sensitivity and false-positive rates. Sensitivity is the proportion of correctly identified signals, whereas the false-positive rate is the proportion of falsely detected signals. In this section, we considered two simulations: 1) signal detection for one drug and 2) DDI for the concomitant use of two drugs.

4.1 Data generation

Data generation was repeated 1,000 times for each setting. The number of ADR types was set as $J=100,500$.

4.1.1 Simulation 1 (single drug)

Signal detection was considered when only one drug was administered. The number of ADRs reported for each drug was set to $n=10,000,000$. For drug 1 ($i=1$), the number of prescriptions with drug 1 was set to $n_{1\cdot }=10,000,50,000$. In Scenario 1, we investigated the false-positive rate of the proposed method, setting the incidence probabilities of ADR $j$ to $p_{1j}=p_{1^{-}j}=0.05,0.1,0.2$ (%). Here, $p_{1^{-}j}$ is the incidence probability of ADR $j$ when drug 1 is not used (see Table 1). In Scenario 2, we examined the sensitivity by setting $\left(p_{1^{-}j},p_{1j})=\left(0.05,0.075\right),\left(0.05,0.1\right),\left(0.05,0.125\right),\left(0.1,0.15\right),\left(0.1,0.2\right),(0.1,0.25\right)$ (%). In both scenarios, the values of $n_{1j}$ and $n_j-n_{1j}$ were independently generated by the binomial distributions Bin($\left.n_{1\cdot },p_{1j}\right)$ and Bin($\left.n-n_{1\cdot },p_{1^{-}j}\right)$, respectively.

4.1.2 Simulation 2 (DDI)

We evaluated DDIs with the concomitant use of drugs 1 (D₁) and 2 (D₂) by assuming the number of prescriptions in the absence of both drugs, the presence of either D₁ or D₂, and the presence of both drugs to be 10,000,000, 100,000, and 10,000, respectively (Table 4). The incidence probabilities in Table 4 vary depending on the simulation scenario. In Scenario 1, we determined the false-positive rate of the proposed method. In this case, no DDIs were observed. We then set (1–1) $p_{00}=p_{10}=p_{01}=p_{11}$; (1–2) $p_{00}=p_{10}<p_{01}=p_{11}$; and (1–3) $p_{00}<p_{10}=p_{01}<p_{11}$ (Figure 1). An additional effect was observed between drugs 1 and 2 in (1–2) and (1–3), although no interaction was observed under the multiplicative assumption because $p_{00}{p}_{11}/\left(p_{10}{p}_{01}\right)=1$. In Scenario 2, we investigated the sensitivity and detected a positive DDI because $p_{00}{p}_{11}/\left(p_{10}{p}_{01}\right)>1$. Under this assumption, we set (2–1) $p_{00}=p_{10}=p_{01}<p_{11}$, (2–2) $p_{00}=p_{10}<p_{01}<p_{11}$, and (2–3) $p_{00}<p_{10}=p_{01}<p_{11}$ (Figure 1). The details of these settings are shown in Figure 1 and described in the Results section. In both scenarios, the values of $n_{00}$, $n_{10}$, $n_{01}$, and $n_{11}$ were independently generated using binomial distributions. As a competitor (henceforth, the existing method), we calculated the PRR and ROR using simple Poisson and logistic models, including two factors D₁ and D₂, and the interaction term for each ADR, respectively (van Puijenbroek et al., 1999).

TABLE 4

TABLE 4. Four-by-two contingency table summarizing the specific ADR reported with the target drugs for evaluating DDIs.

FIGURE 1

FIGURE 1. Simulation scenarios and settings. The height of bars correspond to the incidence probability of ADR A (i) in the absence of both D₁ and D₂; (ii) with D₁ but not D₂; (iii) with D₂ but not D₁; (iv) with D₁ and D₂. The shades correspond to the marginal relative probability of ADR A (lightest), the increased frequency attributable to D₁, the increased frequency attributable to D₂, and the increased incidence probability attributable to an interaction between D₁ and D₂ (darkest). The darkest bars indicate the DDI effect. ADR, adverse drug reaction; DDI, drug-drug interaction.

4.2 Results

4.2.1 Simulation 1 (single drug)

Table 5 presents the false-positive rate in Scenario 1 and the sensitivity in Scenario 2 when the traditional and proposed PRR and ROR are applied under $J=100$. The simulation results under $J=500$ are presented in Supplementary Table S4. At the top of Table 5, the false-positive rates of the proposed PRR and ROR were similar to those of the traditional PRR and ROR across all simulation settings. Additionally, false-positive rates of the PRR and ROR differed minimally. The false-positive rate for the proposed method was not dependent on the incidence probability or the number of ADR types. As shown in Table 5 (bottom), the sensitivities of the proposed PRR and ROR were similar to those of the traditional PRR and ROR for all simulation settings. Furthermore, the false-positive rate of PRR and ROR differed minimally.

TABLE 5

TABLE 5. Simulation results in Scenarios 1 and 2 with single-dose settings (the number of ADR types = 100).

4.2.2 Simulation 2 (DDI)

Table 6 presents the false-positive rate in Scenario 1 and the sensitivity in Scenario 2 when the proposed PRR and ROR were applied under $J=100$. The simulation results under $J=500$ are presented in Supplementary Table S5. At the top of Table 6, the false-positive rate for DDIs using the proposed PRR and ROR was generally controlled at a nominal significance level of 5%. The false-positive rates of the proposed PRR and ROR were similar to those of the PRR and ROR derived using existing methods across all simulation settings. In addition, the false-positive rates of the PRR and ROR differed minimally. The false-positive rate for the proposed method was not dependent on the incidence probability or the number of ADR types. As shown in Table 6 (bottom), there was minimal difference in the sensitivity between PRR and ROR as a false-positive rate. The sensitivities of the proposed PRR and ROR were also similar to those of the PRR and ROR from the existing method across all simulation settings. Although the sensitivity of PRR and ROR increased as the incidence probability increased, the sensitivity was not dependent on the number of ADR types.

TABLE 6

TABLE 6. Simulation results in Scenarios 1 and 2 with DDI settings (the number of ADR types = 100).

5 Discussion

Herein, we proposed a new signal detection method within the framework of a generalized mixed-effect model. The proposed models can directly estimate the PRR and ROR, which are used worldwide to detect signals in spontaneous reporting systems. In terms of the advantages, the proposed method can allow the simultaneous evaluation of several ADRs using multiple drugs. The proposed method is suitable for signal detection because ADRs should be comprehensively and efficiently screened in post-marketing drug safety surveillance. Our study also found that the PRR and ROR calculated using the proposed model were almost identical to the traditional PRR and ROR. While the traditional PRR and ROR can only be calculated in the presence of one drug, the proposed method can be applied to multiple drugs and is a more generalized and convenient method.

For screening ADRs in spontaneous reporting systems, the Medicines and Healthcare Products Regulatory Agency adopts the traditional PRR, and the Netherlands Pharmacovigilance Center and the Pharmaceutical and Medical Devices Agency in Japan employ the traditional ROR (Noguchi et al., 2021). The proposed method also provides the PRR and ORR, and it can be interpreted similarly to the results of traditional methods routinely used by the regulatory authorities. Although the criterion established by Norén et al. (Norén et al., 2008) would be the most widely used for detecting DDIs, the method proposed in the current study is more convenient for practical applications, given that ADRs from “single use of a drug” and “concomitant use of drugs” can be uniformly evaluated using one methodology. Thus, we anticipate that the proposed method will become one of the most useful applications in drug safety surveillance in the future.

However, some ingenuity is required to construct a generalized mixed-effects model. For example, the model may lead to convergence problems in numerical optimization when many drugs are included in the model as factors. Specifically, we cannot obtain PRR and ROR estimates when the constructed model is markedly complicated. In this case, the model is simplified. In addition, the calculation to obtain parameter estimates may be prolonged in the presence of numerous ADRs and drug types.

Several limitations need to be cautiously considered when undertaking signal detection analyses. For example, only observed ADRs are registered in spontaneous reporting systems databases, resulting in underreporting bias (Noguchi et al., 2021). Furthermore, the incidence rate for ADRs cannot be calculated because databases collect only patient information with the ADR (Tada and Gosho, 2022). Moreover, even if the patients are actually taking multiple drugs, some drug information might be missing. Therefore, the measures for detecting DDI tend to be underestimated (Norén et al., 2008). These limitations are inherent to databases and cannot be overcome even when using the proposed method. Although signal detection analysis fails to establish definite conclusions regarding the association between ADRs and target drugs due to the limitations, the analysis results generate hypotheses about the association.

Data availability statement

Publicly available datasets were analyzed in this study. This data can be found here: Publicly available datasets were analyzed in this study. This data can be found here: The following publicly available datasets were used in this study: https://fis.fda.gov/extensions/FPD-QDE-FAERS/FPD-QDE-FAERS.html.

Author contributions

MG: Conceptualization, Formal Analysis, Funding acquisition, Methodology, Project administration, Software, Supervision, Visualization, Writing–original draft, Writing–review and editing. RI: Writing–review and editing, Methodology. TO: Writing–review and editing. KM: Writing–review and editing.

Funding

The author(s) declare financial support was received for the research, authorship, and/or publication of this article. This study was supported by JSPS KAKENHI (Grant number: 22K19682).

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fphar.2024.1312803/full#supplementary-material

Abbreviations

ADR, adverse drug reaction; BCPNN, Bayesian confidence propagation neural network; CI, confidence interval; DDI, drug-drug interaction; FAERS, FDA adverse event reporting system; FDA, food and drug administration; MedDRA, medical dictionary for regulatory activity; MGPS, multi-item gamma poisson shrinker; PRR, proportional reporting rate; ROR, reporting odds ratio; SGLT2, sodium glucose-linked transporter 2; SOC, system organ class.

References

Ahmed, I., Thiessard, F., Miremont-Salamé, G., Bégaud, B., and Tubert-Bitter, P. (2010). Pharmacovigilance data mining with methods based on false discovery rates: a comparative simulation study. Clin. Pharmacol. Ther. 88, 492–498. doi:10.1038/clpt.2010.111

PubMed Abstract | CrossRef Full Text | Google Scholar

Almenoff, J. S., LaCroix, K. K., Yuen, N. A., Fram, D., and DuMouchel, W. (2006). Comparative performance of two quantitative safety signalling methods: implications for use in a pharmacovigilance department. Drug Saf. 29, 875–887. doi:10.2165/00002018-200629100-00005

PubMed Abstract | CrossRef Full Text | Google Scholar

Bate, A., Lindquist, M., Edwards, I. R., Olsson, S., Orre, R., Lansner, A., et al. (1998). A Bayesian neural network method for adverse drug reaction signal generation. Eur. J. Clin. Pharmacol. 54, 315–321. doi:10.1007/s002280050466

PubMed Abstract | CrossRef Full Text | Google Scholar

Bunchuailua, W., Zuckerman, I. H., Kulsomboon, V., Suwankesawong, W., Singhasivanon, P., and Kaewkungwal, J. (2010). Detection of adverse drug reaction signals in the Thai FDA database: comparison between reporting odds ratio and Bayesian confidence propagation neural network methods. Drug Inf. J. 44, 393–403. doi:10.1177/009286151004400404

CrossRef Full Text | Google Scholar

Chen, M., Zhu, L., Chiruvolu, P., and Jiang, Q. (2015). Evaluation of statistical methods for safety signal detection: a simulation study. Pharm. Stat. 14, 11–19. doi:10.1002/pst.1652

PubMed Abstract | CrossRef Full Text | Google Scholar

DuMouchel, W. (1999). Bayesian data mining in large frequency tables, with an application to the FDA spontaneous reporting system. Am. Stat. 53, 177–190. doi:10.1080/00031305.1999.10474456

CrossRef Full Text | Google Scholar

Evans, S. J. W., Waller, P. C., and Davis, S. (2001). Use of proportional reporting ratios (PRRs) for signal generation from spontaneous adverse drug reaction reports. Pharmacoepidemiol. Drug Saf. 10, 483–486. doi:10.1002/pds.677

PubMed Abstract | CrossRef Full Text | Google Scholar

Gosho, M., Maruo, K., Tada, K., and Hirakawa, A. (2017). Utilization of chi-square statistics for screening adverse drug-drug interactions in spontaneous reporting systems. Eur. J. Clin. Pharmacol. 73, 779–786. doi:10.1007/s00228-017-2233-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Hochberg, A. M., Hauben, M., Pearson, R. K., O’Hara, D. J., Reisinger, S. J., Goldsmith, D. I., et al. (2009). An evaluation of three signal-detection algorithms using a highly inclusive reference event database. Drug Saf. 32, 509–525. doi:10.2165/00002018-200932060-00007

PubMed Abstract | CrossRef Full Text | Google Scholar

Kubota, K., Koide, D., and Hirai, T. (2004). Comparison of data mining methodologies using Japanese spontaneous reports. Pharmacoepidemiol. Drug Saf. 13, 387–394. doi:10.1002/pds.964

PubMed Abstract | CrossRef Full Text | Google Scholar

Matsushita, Y., Kuroda, Y., Niwa, S., Sonehara, S., Hamada, C., and Yoshimura, I. (2007). Criteria revision and performance comparison of three methods of signal detection applied to the spontaneous reporting database of a pharmaceutical manufacturer. Drug Saf. 30, 715–726. doi:10.2165/00002018-200730080-00008

PubMed Abstract | CrossRef Full Text | Google Scholar

Mosley, J. F., Smith, L. S., Everton, E., and Fellner, C. (2015). Sodium-glucose linked transporter 2 (SGLT2) inhibitors in the management of type-2 diabetes: a drug class overview. P T 40, 451–462.

PubMed Abstract | Google Scholar

Noguchi, Y., Tachi, T., and Teramachi, H. (2019). Review of statistical methodologies for detecting drug–drug interactions using spontaneous reporting systems. Front. Pharmacol. 10, 1319. doi:10.3389/fphar.2019.01319

PubMed Abstract | CrossRef Full Text | Google Scholar

Noguchi, Y., Tachi, T., and Teramachi, H. (2020). Comparison of signal detection algorithms based on frequency statistical model for drug-drug interaction using spontaneous reporting systems. Pharm. Res. 37, 86. doi:10.1007/s11095-020-02801-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Noguchi, Y., Tachi, T., and Teramachi, H. (2021). Detection algorithms and attentive points of safety signal using spontaneous reporting systems as a clinical data source. Brief. Bioinform. 22, bbab347. doi:10.1093/bib/bbab347

PubMed Abstract | CrossRef Full Text | Google Scholar

Norén, G. N., Sundberg, R., Bate, A., and Edwards, I. R. (2008). A statistical methodology for drug–drug interaction surveillance. Stat. Med. 27, 3057–3070. doi:10.1002/sim.3247

PubMed Abstract | CrossRef Full Text | Google Scholar

Rothman, K. J., Lanes, S., and Sacks, S. T. (2004). The reporting odds ratio and its advantages over the proportional reporting ratio. Pharmacoepidemiol. Drug Saf. 13, 519–523. doi:10.1002/pds.1001

PubMed Abstract | CrossRef Full Text | Google Scholar

Stroup, W. W. (2013). Generalized linear mixed models. Boca Raton, FL: CRC Press.

Google Scholar

Tada, K., and Gosho, M. (2022). Increased risk of urinary tract infection and pyelonephritis under concomitant use of sodium-dependent glucose cotransporter 2 inhibitors with antidiabetic, antidyslipidemic, and antihypertensive drugs: an observational study. Fundam. Clin. Pharmacol. 36, 1106–1114. doi:10.1111/fcp.12792

PubMed Abstract | CrossRef Full Text | Google Scholar

Thakrar, B. T., Grundschober, S. B., and Doessegger, L. (2007). Detecting signals of drug-drug interactions in a spontaneous reports database. Br. J. Clin. Pharmacol. 64, 489–495. doi:10.1111/j.1365-2125.2007.02900.x

PubMed Abstract | CrossRef Full Text | Google Scholar

van Puijenbroek, E. P., Bate, A., Leufkens, H. G., Lindquist, M., Orre, R., and Egberts, A. C. (2002). A comparison of measures of disproportionality for signal detection in spontaneous reporting systems for adverse drug reactions. Pharmacoepidemiol. Drug Saf. 11, 3–10. doi:10.1002/pds.668

PubMed Abstract | CrossRef Full Text | Google Scholar

van Puijenbroek, E. P., Egberts, A. C., Heerdink, E. R., and Leufkens, H. G. (2000). Detecting drug-drug interactions using a database for spontaneous adverse drug reactions: an example with diuretics and non-steroidal anti-inflammatory drugs. Eur. J. Clin. Pharmacol. 56, 733–738. doi:10.1007/s002280000215

PubMed Abstract | CrossRef Full Text | Google Scholar

van Puijenbroek, E. P., Egberts, A. C., Meyboom, R. H., and Leufkens, H. G. (1999). Signalling possible drug-drug interactions in a spontaneous reporting system: delay of withdrawal bleeding during concomitant use of oral contraceptives and itraconazole. Br. J. Clin. Pharmacol. 47, 689–693. doi:10.1046/j.1365-2125.1999.00957.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Wolfinger, R., and O’Connell, M. (1993). Generalized linear mixed models a pseudo-likelihood approach. J. Stat. Comput. Simul. 48, 233–243. doi:10.1080/00949659308811554

CrossRef Full Text | Google Scholar