An Introduction to Terminology and Methodology of Chemical Synergy—Perspectives from Across Disciplines

The idea of synergistic interactions between drugs and chemicals has been an important issue in the biomedical world for over a century. As complex diseases, especially cancer, are being treated with various drug cocktails, understanding the interactions among these drugs is increasingly vital to ensuring successful treatment regimens. However, the idea of synergy is not limited to only the biomedical realm and these ideas have developed across many different disciplines, as well. In this review, we first discuss the various terminology surrounding the idea of synergy, providing a comprehensive list of terms defined across numerous disciplines. We then review the most common methodology for detection and quantification of synergy, including the two most prominent reference models for describing additive interactions: Loewe Additivity and Bliss Independence. We also discuss advantages and limitations to each method, with a focus on the Chou-Talalay Combination Index method. Finally, we describe how methods development and terminology have developed among disciplines outside of biomedicine and pharmacology, to synthesize the literature for readers.

Introduction

The idea that 1 + 1 = 2 is not novel and virtually agreed upon and understood throughout various academic disciplines and even different cultures around the world. So, it's very counterintuitive to say 1 + 0 = 2, 0 + 0 = 1, or even 1 + 1 = 0. However, that paradigm is a very basic analogy for understanding synergist interactions, or synergy. Synergy is commonly defined as the effect of two or more agents working in combination that is greater than the expected additive effect of said agents (Greco et al., 1996). Now, going back to the 1 + 0 = 2 analogy (Berthoud, 2013; Geary, 2013), it can be asserted that there is a synergistic interaction occurring. Unfortunately, in practice, it is far from that simple to quantify such interactions.

Synergy is most often defined in relation to the realms of pharmacology and medicine. This is because many diseases require treatment that consists of “cocktails,” or mixtures of various drugs taken at once. This is particularly true in cancer treatment. This potentially allows for maximizing the therapeutic effect while minimizing the adverse effect, or side effects, upon being treated with a given drug regimen (Greco et al., 1996; Foucquier and Guedj, 2015). If two drugs act synergistically, lower doses of each drug could potentially be used which could allow for less adverse effects while still provided the desired outcome, such as cancer cell death.

There are a number of examples of clinical synergy in the area of cancer treatment, often established between established chemotherapy agents and agents representing other classes. Herceptin has been shown to synergistically interact with both doxorubicin and paclitaxel, though cardiotoxicity risk is increased in combination therapy (Pegram et al., 2000). Another example of this has been shown to between anticancer drugs edatrexate and cisplatin as well as methotrexate and cisplatin, resulting in the ability to use lower doses of each drug in combination while maintaining the same cell death levels as when using higher doses of these drugs individually (Chou et al., 1993). The addition of a complement-fixing monoclonal antibody, rituximab, to cyclophosphamide, hydroxydaunomycin, Oncovin, and prednisone (CHOP) therapy in non-Hodgkin's lymphoma increases the overall response without increasing toxicities (Weiner, 2010). Finally, the long-established combination of vincristine and prednisone is highly potent blood born tumors, with the addition of the anthracyline crucial for cure, not just remission (Weiner, 2010).

The majority of this review will discuss synergy in relation to the domains of pharmacology and biomedicine, however there are a range of applications of synergy that can be seen among various non-clinical fields. Synergistic combinations have been observed in chemical mixtures more generally (Carpenter et al., 2002), often evaluated in the realm of toxicology and environmental health. For example, Laetz et al. describe the effects of synergistic neurotoxicity in salmon when exposed to various chemical mixture of heavily used insecticides (Laetz et al., 2009). While the overall properties of synergistic interactions are the same across fields, as often happens in the scientific literature, separate methods and corresponding terminology emerge across disciplines. The use of different “jargon” is a challenge for interdisciplinary research, as it limits cross-talk across methods developed in different domain applications. In this review, we will unify the various terms that have developed across disciplines in a comprehensive list, to enable researchers to more readily synthesize the diverse literature. We will also review the statistical and computational approaches used to quantify synergistic interactions, and discuss their relative advantages and disadvantages. Throughout this review, empirical examples of synergy using the described methods and among disciplines can be found in the referenced reviews papers and experimental articles.

Terminology

As models and methods for detecting synergy among interactions have developed over the past century, terminology used across the various fields and within these methods has not always been consistent. This inconsistency can even be seen within the field of biomedicine, one of the most common fields to apply methods for synergy detection. This results in potential confusion when trying to discern which methods best suit a given situation. Here, we offer a comprehensive list of some common terms in defining synergy and their respective equivalencies to help alleviate any potential conflicts and inconsistencies in defining and quantifying synergy.

The first definition necessary when regarding synergy is the concept of additive effects. This has also been referred to as noninteraction, and inertism (Greco et al., 1995). An additive effect is generally considered as the baseline effect for synergy detection methods. It is the effect that is theoretically expected from the combination of multiple drugs when synergy is not present. Although seemingly simple from its name, the idea of simply adding two, or more, effects together do not accurately reflect what happens realistically. A quick, simple example to show this involves two separate drugs A and B that both exhibit 60% cell death at a saturating dose, for example. Would it be possible to simply add these effects together to get a combined effect of 120% cell death; no, that is not realistically possible (Chou, 2010). The problem of mathematically defining additivity has been the center of controversy among leading researchers of this topic for the last century. However, there are two models that have prevailed and will further be described in the Reference Models section. Along with these prevailing models, there are two additional terms, Loewe Additivity and Bliss Independence (Greco et al., 1995), that are essentially synonymous with the basic definition of additive but exist for these specific reference models. Loewe Additivity has also been referenced as dose additivity and Concentration Addition (Cedergreen, 2014). Additionally, Bliss Independence is sometimes referred to as Bliss Additivity (Geary, 2013), Response Multiplication, Response Addition, Effect Addition, and Independent Action (Cedergreen, 2014).

Any (significant) deviation from additivity would be classified as synergy or antagonism. It is often agreed upon that synergy can be defined as a combination effect that is greater than the additive effect expected from good knowledge of the individual drugs. Synergy has also been called superadditivity (Tallarida, 2001), potentiation, augmentation (Berenbaum, 1977), supra-additivity (Geary, 2013). The term coalism is also sometimes used to refer to synergy when neither drug, or none of the drugs in mixtures of more than two chemicals, is effective on its own (Greco et al., 1995). There are also two distinctively termed ideas to describe synergy under the previously mentioned specific models. Those terms are Loewe Synergy and Bliss Synergy; the models from which the terms have been derived will be discussed further in Reference Models.

Antagonism is the opposite of synergy; it occurs when the combined effect of compounds is less than what would be expected. In the biomedical world, it is often considered more of a negative scenario, as many researchers are looking to identify synergistic interactions among compounds for some sort of added therapeutic effect. However, in a toxicological sense, it may be beneficial to have an antagonistic effect in a mixture of chemicals. Antagonism has also been named subadditivity (Tallarida, 2001), infra-additive (Geary, 2013), negative interaction, depotentiation and even negative synergy (Berenbaum, 1977). Synonyms used in discussions of synergy, additivity, and antagonism are summarized in Table 1.

TABLE 1

Table 1. Synonyms used in discussions of synergy.

Central to some of the methods subsequently discussed is the idea of a dose-response relationship modeled as a curve. This curve is referred to as a dose-response curve, dose-effect curve, concentration-effect curve or concentration-response curve (Chou et al., 1993; Schwartz et al., 2002). These terms are sometimes used interchangeably (Aronson, 2007), but often coincide with the definitions of a dose being the amount of agent administered or experimentally used while a concentration is the measurable amount (typically per some volume of substance) in the experimental system (Nielsen and Friberg, 2013). The experimental system could be an intact organism such as a mouse, or a model system such as lymphoblastoid cell lines, the latter of which has been implicated on various occasions as a viable, high-throughput model for assessing individual drug cytotoxicity (Peters et al., 2011; Brown et al., 2012, 2014; Abdo et al., 2015). A common mathematical model when modeling the dose-response relationship, such as the cytotoxic effect of anticancer agents on cell viability, is the Hill model (Konkoli, 2011; Beam and Motsinger-Reif, 2014). This model was first developed by Hill in 1910 as a model for percent of hemoglobin saturated with oxygen and is now widely used in biological sciences to model various processes, specifically in the form of the Michaelis-Menten equation describing the relationship between enzyme and substrates (Goutelle et al., 2008). This model has also been mathematically rewritten, and even directly referred to, as the sigmoid E_max model, or simply the E_max model or sigmoidal model (Goutelle et al., 2008). The general equation for this model is given in Equation (1), where E is the predicted response of the agent on the system, E₀ is the baseline response for a drug concentration of 0, E_max is the maximum response, C is the concentration used for the predicted response E, and EC₅₀ is the concentration for which 50 percent of the maximal response is obtained, and h is the hill coefficient of sigmoidicity, also referred to as the slope parameter, which affects the shape of the curve (Goutelle et al., 2008). Examples of this curve and how the slope parameter can shape it are shown in Figure 1. It is also worth noting that while modeling dose-response curves is often a necessary step in many of the synergy detection methods, it is not always a simple task, especially when the curves are nonlinear. However, there have been various approaches to optimizing this procedure, such as an evolutionary algorithm method (EADRM) developed by Beam and Motsinger-Reif (2011).

$\begin{array}{rc}E=E_0+\frac{\left(E_{max}\times C^h\right)}{\left({{EC}_{50}}^h+C^h\right)}(\text{Hill equation})& (1)\end{array}$

FIGURE 1

Figure 1. Example of curves following the Hill equation with different hill slope parameter values.

Reference Additivity Models

To detect and define synergy, it is first necessary to establish a reference, or “null,” model. This serves as the baseline for quantifying how an interaction between two drugs should occur based on their individual performance, i.e., an interaction that does not exhibit any synergy or antagonism, defined previously as additivity. Deviation from the reference model can then be seen as some sort of synergistic or antagonistic interaction, depending on the direction of the deviation. There have been many attempts at trying to define the best reference model for the general case as well as for specific situations. Upon first glance, it may seem as a rather simple problem, similar to 1 + 1 = 2. However, when considering how drugs interact within the human body, for example, many things are to be considered. Where the drugs are metabolized, the specific target(s), etc., could all contribute to the interaction of the two, or more, drugs used in combination. Unfortunately, a specific reference model cannot take into account every detail of how drugs may interact, and the more general models have prevailed. Throughout the past century, numerous reference models for additive drug interactions have been proposed, however there are two generally accepted models (Greco et al., 1995). Those two models, described as follows, are the Bliss Independence model and the Loewe Additivity model.

Loewe Additivity

The Loewe Additivity model is based on the idea of a “sham mixture,” one where a single drug is mixed with itself. In this type of mixture, there is not expected to be any sort of interaction since a single drug, or various similar drugs, cannot interact with itself, or each other (Loewe, 1926). Thus, the result of a drug combined with itself is called Loewe Additivity. A similar drug would be one with perhaps similar structure and target. This reference model is the basis for many commonly applied methods to detect synergy. As previously stated, it is generally considered one of the two most used reference models, potentially even the most accurate (Greco et al., 1995). However, there are limitations to the model which will be discussed later.

The basic idea of Loewe Additivity assumes a drug cannot interact with itself. Assume two distinct compounds α and β exist and have dose response curves as seen in Figure 2. It is necessary to mention that this model assumes a constant potency ratio, which is the ratio of doses of two individual drugs that give the same effect. Hence, a constant potency ratio is one that maintains some specific ratio $\frac{\mathit{\text{A}}}{\mathit{\text{B}}}$, for example, for all doses (Tallarida, 2012), where A and B are any doses on curves α and β that achieve the same effect level. It should be noted that in some literature, this has been referred to as a fixed ratio (Hennessey et al., 2010). This can be seen graphically by noticing the parallel structure of the dose response curves for the individual compounds α and β, as shown in Figure 2. A given dose or concentration (ex: EC₅₀) that produces a given effect can be measured on either curve. Because the curves have a constant potency ratio, any dose on curve α can be easily translated to its curve β counterpart by taking advantage of this ratio, R = $\frac{A}{B}$. Let us consider measuring some combination of doses, a and b, that produces some effect, F, corresponding to dose A on curve α and B on curve β. To do this, we must first convert the doses to the same curve. As stated, the ratio of doses $\frac{A}{B}$ or $\frac{B}{A}$ permits conversion between curves α and β. Thus, we have:

$\begin{array}{rc}{b}_a=a\times \frac{B}{A}& (2)\end{array}$

$\begin{array}{rc}{a}_b=b\times \frac{A}{B}& (3)\end{array}$

Where b_a is the dose of compound β needed to achieve the same effect as dose a of compound α, similarly for a_b. We can now use to either curve to calculate the effect of

$\begin{array}{rc}F=\beta (b+b_a)=\alpha (a+a_b)& (4)\end{array}$

$\begin{array}{rc}b+b_a=B& (5)\end{array}$

$\begin{array}{rc}b+a\times \frac{B}{A}=B& (6)\end{array}$

$\begin{array}{rc}\frac{a}{A}+\frac{b}{B}=1& (7)\end{array}$

FIGURE 2

Figure 2. Example of curves with a constant potency ratio.

This is the fundamental equation that has come out of the Loewe Additivity reference model. It has been the basis for numerous subsequent models to quantify synergistic interactions. It also has led to the widely used combination index (Loewe, 1953), which is simply the left side of Equation (7). If the combination index is less than 1, synergy is said to occur, greater than 1, antagonism. Additionally, this model can be extended to more than 2 compounds (Berenbaum, 1977; Goldoni and Johansson, 2007), where the equation becomes:

$\begin{array}{rc}\frac{a}{A}+\frac{b}{B}+\frac{c}{C}\dots \frac{n}{N}=1& (8)\end{array}$

As with all models, there are limitations and assumptions made. One previously stated assumption is a constant potency ratio for the two dose response curves. As previously stated, curves with constant relative potency will have parallel log dose response curves (Tallarida, 2012; Geary, 2013; Foucquier and Guedj, 2015). Parallel dose response curves and constant potency ratios are considered by some to rarely be the case or to be more of an exception (Loewe, 1953; Geary, 2013). In fact, according to Geary, miniscule deviations from a constant potent ratio could result in a nonparallel log dose effect curves (Geary, 2013). However, Grabovsky and Tallarida (2004) derived similar formulas for nonparallel log dose response curves to deal with such situations, though they may not be as simple as the general equation for constant potency ratios. Furthermore, Loewe Additivity also requires that the dose response curves be accurately estimated individually for each drug in the combination (Foucquier and Guedj, 2015). This is often not a trivial task. A further consideration when using Loewe Additivity is an indeterminate solution resulting, when the result from the conversion of dose A on curve Alpha to a new dose on curve Beta does not align with the conversion of dose B on curve Beta to a dose on curve Alpha (Geary, 2013). According to Geary, this situation occurs frequently (Geary, 2013) and can be illustrated more in-depth by him as well as Tallarida (Tallarida, 2006, 2007; Geary, 2013). Despite these limitations, Loewe Additivity has still been one of the major reference models used and the foundation for many synergy methods.

Bliss Independence

A commonly used alternative to Loewe Additivity is the Bliss Independence model. The Bliss model is based on the idea of noninteraction, that each drug is acting independently of one another (Greco et al., 1995). However, this does not mean that they do not both potentially contribute to the overall effect, but presumably that they take different routes to achieve said effect. An example originally described by Bliss involves an organism that dies from the effects of two distinct compounds. Under the idea of Bliss Independence, these compounds do not interact and perhaps affect different vital systems within the organism. Both compounds do affect the organism, however (Bliss, 1939). The general form of the equation describing Bliss Independence is simply the product of the two fractional responses (Greco et al., 1995):

$\begin{array}{rc}{F}_{uc}=f_{u1}\times f_{u2}& (9)\end{array}$

Where F_uc is the fraction unaffected by some outcome for the combination of drugs 1 and 2, f_u1 is the fraction unaffected for drug 1 and f_u2 is the fraction unaffected for drug two. This can also be written in terms of the fraction affected by some event, as shown originally by Bliss (1939):

$\begin{array}{rc}{F}_{ac}=f_{a1}+f_{a2}-f_{a1}\times f_{a2}& (10)\end{array}$

Where F_ac is the fraction affected by some outcome for the combination of drugs 1 and 2, f_a1 is the fraction affected for drug 1 and f_a2 is the fraction affected for drug two. The two equations can be related by the following:

$\begin{array}{l}{F}_a=\left(1-F_u\right)\to \left(1-F_{ac}\right)=\left(1-f_{a1}\right)\times \left(1-f_{a2}\right)\\ \to 1-F_{ac}=1-f_{a1}-f_{a2}+f_{a1}\times f_{a2}\to F_{ac}\\ =f_{a1}+f_{a2}-f_{a1}\times f_{a2}\text{Relation}(1)\end{array}$

A more mathematical interpretation of this can be seen from understanding the idea of probabilistic independence (Foucquier and Guedj, 2015). It is often appropriate to consider this problem in terms of probabilities because responses are often measured as fractions of living or killed components, for example fraction of cell death when administering anticancer drugs. Given this, consider two compounds (drugs, chemicals, etc.), A and B, that act independently of each other. From probability theory,

$\begin{array}{rc}P(A+B)=P(A)+P(B)-P(AB)& (11)\end{array}$

And, if A and B are independent events, which is assumed under Bliss Independence,

$\begin{array}{rc}P(AB)=P(A)\times P(B)& (12)\end{array}$

Thus, combining Equations (11) and (12), the common formula for Bliss Independence can be derived,

$\begin{array}{rc}{E}_c=E_a+E_b-E_a\times E_b& (13)\end{array}$

Where E_c is the effect produced by the combination of compounds A and B, at doses a and b, E_a is the effect of compound A at dose a and E_b is the effect of compound B at dose b. The above formula is often used as the reference for how a combination of compounds should act if no synergy or antagonism exists. If the combined effect is greater than what would be expected, as predicted from this formula, synergy is declared, antagonism otherwise. Goldoni shows that this model can be expanded to numerous compounds, though the mathematics become increasingly complex upon using more than 3 compounds (Goldoni and Johansson, 2007).

Though still commonly used as a basic reference model, there has been much criticism over the validity of the Bliss Independence model. The main assumption of this model is that two drugs are acting independently. However, as asserted by Gessner (1974, 1988), for a large proportion of drug interactions, this may not truly be the case. Additionally, for this model to hold true, it must be applicable along the entire dose response curve, something that may not be true in many cases (Gessner, 1988). Advocates of the Loewe Additivity reference model often bring up an additional limitation of the Bliss Independence model. That is that it fails when applied to the “sham mixture” scenario, the basis for the Loewe Additivity model (Berenbaum, 1981; Greco et al., 1995). Many consider the “sham mixture” scenario to be fairly intuitive, so failure to adhere to this could pose a problem when advocating for use of this method.

Methods for Explicit Detection of Synergy

Aside from simply using the reference models as a baseline for additivity and deviations from that as a measure of synergy, specific methods have been developed for enhanced detection of synergy. Here we discuss some of the more common methodologies along with their benefits and disadvantages. We then further discuss more recent, statistical approaches in a similar manner.

Isobologram Method

One of the most prolific methods to come out of the Loewe Additivity reference model is the graphical procedure known as the isobologram method. This type of analysis dates back to the late 1800s with Fraser (Fraser, 1870–1871, 1872) and was continued by Loewe (Loewe, 1926; Greco et al., 1995) but was officially published by Loewe (Loewe, 1953, 1957; Tallarida, 2001). An isobologram is a graphical procedure in which doses of one compound are displayed along one axis and doses of a second compound are displayed along the second axis. The entire plot contains combinations of doses for a specific effect level. This method relates to Loewe Additivity because Equation (5) is used to plot a line of additivity, the isobole (Greco et al., 1995). Dose combinations plotted below this line require a lower dose than expected from the Loewe Additivity line and thus can be classified as synergistic, while those plotted above it are antagonistic. This can be seen in Figure 3.

FIGURE 3

Figure 3. (A) Individual dose-response curves with EC₅₀ values and various points used in combination in (B). (B) Isobologram showing line of additivity. Point W is a combination using doses of W_α and W_β from (A), similarly for points Q and Z. Point W indicates synergy, point Q indicates additivity and point Z indicates antagonism.

This method is very simple to achieve and a graphical, intuitive interpretation of synergy. However, there have been various drawbacks, further detailed in reviews by Greco et al. (1995) and from a more mathematically perspective by Geary (2013). We will briefly summarize these limitations here. One major drawback is that the approach is too simple for a majority of real world applications, as linear isoboles are relatively rare according to Loewe (1953) and Geary (2013). When dose response curves are nonparallel, as discussed under Loewe Additivity, a nonlinear isobole will result, referred to as a curvilinear isobole (Grabovsky and Tallarida, 2004; Geary, 2013). Another major drawback is that this method lacks a formal statistical framework and does not allow for formally quantifying the intensity of a synergistic interaction (Greco et al., 1995). A similar, relatively popular (Greco et al., 1996) method by Steel and Peckham (1979) was developed to take into account a region around the line of additivity, known as an envelope of additivity. This helps alleviate the problem of simple departures from the additivity line and while there are some other benefits to this method, it also has some of its own drawbacks, discussed more by Greco et al. (1995).

Chou-Talalay Method

The Chou-Talalay method is one of the most widely used methods for detecting and quantifying synergistic interactions between two or more drugs (Greco et al., 1996; Boik et al., 2008), having been cited over 7,000 times over the past few decades (Chou and Talalay, 1984; Chou, 2006, 2010). According to Chou, the main equation forming the basis of this method was derived as a unified theory of the Michaelis-Menten, Hill, Henderson-Hasselbalch, and Scatchard equations (Chou and Talalay, 1984; Chou, 2010). They termed this equation the median-effects equation:

$\begin{array}{rc}\frac{f_a}{f_u}={\left(\frac{D}{D_m}\right)}^m(\text{Median-effects equation})& (14)\end{array}$

where f_a is the fraction of cells affected (i.e., killed), f_u is the fraction of cells unaffected (i.e., living), D is the dose of drug given, D_m is the median-effect dose and m is the sigmoidicity of the dose-effect curve. This equation can be simplified into the following linearized version:

$\begin{array}{l}log(\frac{f_a}{f_u})=m\times log(D)-m\times log(D_m)\\ (\text{Log-linearized median-effects equation})& (15)\end{array}$

A linear regression can then be applied for the various doses (D) and responses (f_a/f_u). From this, values for D_m and m can be estimated. These values can then be used to calculate estimates for variables in the following equation giving the (generalized) combination index (CI):

$\begin{array}{rc}CI=\frac{D_1}{E_1}+\frac{D_2}{E_2}& (16)\end{array}$

where D₁ and D₂ are the actual drug doses used in the combinations during dosing experiments and E₁ and E₂ are theoretical individual drug levels that would be expected to be needed to achieve the experimentally measured response. While D₁ and D₂ are known from experimental design, E₁ and E₂ can be calculated using the D_m and m values previously computed. A CI value less than 1 indicates synergism, greater than 1 indicates antagonism, and equal to 1 indicating additivity. This equation can get more complicated depending on the factors such as the exclusivity/independence of the compounds and can be expanded to more than two compounds (Chou and Talalay, 1984).

Though one of the most prolific methods (Greco et al., 1996), there are some considerations to note about this method. Among a number of limitations mentioned by Geary (2013), this method requires drugs have a constant potency ratio (Geary, 2013). Chou has asserted that a constant ratio is not strictly necessary, however a constant potency ratio does result in better accuracy (Chou, 2010). Additionally, in his review, Greco notes three major flaws with the Chou-Talalay Method (Greco et al., 1995). First, he notes calculation of D_m and m, shown in Equation (5), from a median effects plot for mutually nonexclusive drugs can never be correct as dose response curves are primarily nonlinear. Second, Greco shows that the form of one of the equations, for a specific set of parameters, used to calculate the combination index for the mutually nonexclusive case is slightly incorrect. Finally, he mentions that when the median effects plot for the mutually exclusive with interaction case is nonlinear, incorrect conclusions can occur which was shown by Greco as the combination index for part of his data set incorrectly resulting in antagonism instead of synergy due to “incorrect linear extrapolation of the nonlinear median effects curve for the drug combination” (Greco et al., 1995). Even with these considerations, the Chou-Talalay method is by far the most impactful approach to quantifying synergy.

Response Surface

As an alternative to the reductionist approaches based on curve fits discussed above, response surface modeling tries to use higher dimension data to quantify synergistic responses. The response surface approach has been considered by numerous theorists, each developing a method with a slightly different perspective. The general idea of these types of approaches are to create a 3-dimensional surface consisting of predicted (noninteraction) data points from doses of two drugs. The x- and y-axes are labeled with doses of each drug, similar to an isobologram, but the z-axis is the effect or response at a given combination of doses. The experimentally determined responses for various combinations can be then plotted on this 3D surface where points below or above the surface indicate synergy or antagonism, depending on whether the z-axis is a measurement of inhibition, effect, etc. (Greco et al., 1995). Predictions can be calculated using either Loewe Additivity models, as in the method of Berenbaum (1985), or using Bliss Independence and the Fraction Product Method of Webb (1963) as described by Greco et al. (1995). Similar methods, such as the Method of Prichard and Shipman (1990), also graph response surfaces for the raw data, as opposed to simply plotting points on the predicted surface. Subtraction of the predicted surface from the raw surface can then be performed to get a new 3D surface where peaks above zero and valleys below zero indicate synergy or antagonism, again, depending on the measurement of the z-axis (Greco et al., 1995). Figure 4 shows an example of this approach. A further, more in-depth, discussion of response surface approaches is available in Greco et al. (1995).

FIGURE 4

Figure 4. Example response surface showing viability for the combination of two drugs.

Statistical Based Approaches

The methods discussed above have a long history of evaluation and use, but methods development is an active field, and a number of new approaches have recently been developed (Boik et al., 2008; Hennessey et al., 2010). Here we discuss two more recent approaches that address some of the limitations of the earlier methods. These approaches bring a statistical framework to assessing synergy, with both frequentist and Bayesian perspectives.

The first method developed by Boik et al. (2008), named MixLow (Mixed-effects Loewe), uses Loewe Additivity as the reference model. The method consists of three basic parts: estimation of dose response curve parameters, calculation of the Loewe Additivity index (i.e., combination index), and calculation of confidence intervals for the index (Boik, 2010). The method assumes a sigmoidal dose response curve and a nonlinear mixed-effects model is used to estimate curve parameters and then uses those parameters to estimate their Loewe Additivity Index and confidence intervals. This method has been favorably compared to the Chou-Talalay Method in simulation experiments (Boik et al., 2008). The major benefits of the MixLow method is that it avoids the necessary preprocessing of the Chou-Talalay method as well as avoiding the need for log-linearization (Boik, 2010). Limitations of this method may be assumptions of a sigmoidal dose response curve and the necessity for fixed-ratio drug combinations.

The second recent method is based on a Bayesian statistical framework and was developed by Hennessey et al. (2010). The idea of analyzing this problem in a Bayesian framework is intriguing and seems to have been overlooked within the field for the most part. This method involves a Bayesian hierarchical nonlinear regression model for the dose response curves of individual and combined agents. Their 3 stage model accounts for “variability between experiments, variability within experiments, and variability in the observed responses of controls.” A priori knowledge, for instance EC₅₀ values of a particular agent, can also be incorporated into the model. Markov Chain Monte Carlo is used to estimate the posterior distributions of necessary parameters to calculate what they refer to as the Bayesian Effect Interaction Index. A posterior probability is then calculated and synergy can be declared in a more statistical sense, using probabilities to compare the observed interactions to the expected additive effects. Limitations to this method could include its potential complexity for those unfamiliar with Bayesian statistics and, like many other approaches, that it assumes a constant potency ratio.

Synergy Across Disciplines

Thus far, the methods discussed have been developed for the detecting synergy in the biomedical, pharmacological domain. This is likely due to the importance of synergy in the search for more therapeutic treatments for cancer and disease (Greco et al., 1995). There has also been simultaneous development and application of methods for quantifying synergy in other fields, especially in toxicology. Below, we discuss the history of methods development and use in other fields, with a focus on methods developed for domain-specific issues. Although not an exhaustive list by any means, it will hopefully lend a better understanding of different applications of the general idea of synergy.

Toxicology

The idea of synergy is familiar to the field of toxicology and very related to the traditional biomedical definitions previously discussed in this review. In fact, Bliss originally published his famous reference model, Bliss Independence, in a publication titled “The Toxicity of Applied Poisons” (Bliss, 1939). And, according to Geary, Bliss Independence is often used within the realm of toxicology (Geary, 2013). In their review of mathematical methods in the toxicological realm, Goldoni and Johansson note that both Loewe Additivity and Bliss Independence are used, the former in a more general sense and the latter in more specific subfields such as irradiation (Greco et al., 1995; Groten et al., 2001; Goldoni and Johansson, 2007). The Loewe Additivity model, referred to as Concentration Addition, was used in a comprehensive review article of mixture toxicity within environmental toxicology due to the fact that it is often “recommended for risk assessment purposes” (Cedergreen, 2014). A seemingly common trend among various articles (Hertzberg and MacDonell, 2002; Goldoni and Johansson, 2007; Laetz et al., 2009; Cedergreen, 2014) is to simply look for departures from the reference model being used. Problems with analyses similar to this idea have been discussed by (Greco et al., 1995).

There has been some traction among methods development within the realm of toxicology. One of the older methods for assessing interaction effects and potential synergy was developed by Marking (1977). Mumtaz and Durkin (1992) also developed a method that extends the hazard index. The hazard index was originally introduced by the US EPA to assess the hazard of chemical mixtures (Groten et al., 2001) by focusing on chemicals affecting the same target. The hazard index is limited to assessing mixtures with compounds that produce the same toxic effect through the same mode of action (Groten, 2000). The Mumatz and Durkin method extends the hazard index method to take potential interactions into account using a weight of evidence classification in which information about the individual compounds, such as route of exposure or demonstration of toxicity, is used to more precisely predict the joint action of the compounds in the mixture (Mumtaz and Durkin, 1992; Groten, 2000).

A major problem in the current synergy methodology is the ability to detect synergy among mixtures of more than 2 agents as well as detecting which agents in a multi-agent mixture are actually contributing to the synergistic interactions. For pharmacological applications, the number of drugs/chemicals included in mixtures is typically small, but in environmental mixtures, the mixtures can be complex, and not readily measured. However, mixtures represent more realistic exposure scenarios than the single-agents typically applied in standard studies (Scher, 2012). The basic reference methods, as well as some quantification methods such as, for example, the Chou-Talalay method, can be expanded to more than 2 agents (Chou and Talalay, 1984). However, it is much more difficult to identify the agents within the mixture that do contribute to the synergistic effect without proper, often nearly impossible, experimental design (Foucquier and Guedj, 2015). This has been known to be a problem among drug mixtures and biomedicine but also presents a problem within the field of toxicology.

Epidemiology

Synergy has also been observed in epidemiological studies as well, often through the observation of diseases that seem to increase the risk of acquiring other diseases. This can be seen in the study of HIV and sexually transmitted diseases (STDs). Coined “epidemiological synergy” by Wasserheit (1992), it was postulated that HIV could augment or even prolong the infectiousness of STDs in a co-infected individual (Wasserheit, 1992). Synergistic interactions are also discussed among risk factors for leukemia in patients that had family histories of breast cancer (Olshan et al., 2001). Schwartz elaborates on the idea of synergy in epidemiology and genetic interactions, describing it as “deviations from additivity” (Schwartz, 2006), the same concept as the synergy defined in the biomedical field. She details an interaction contrast, essentially a specific linear combination of the various independent variable groups within the specific model, as a measure of synergy in this field, also used in subsequent publications (Li et al., 2005; Shen et al., 2005).

Food Sciences

Synergistic interactions have also been observed in food sciences and nutrition. Liu notes that phytochemicals in different foods, that help with cancer prevention, have potential synergistic interactions and that a complex mixture of them is present in whole foods (Liu, 2004). Phytochemicals in food were also implicated in similar analyses by Jacobs and Steffen (2003). The parallels between these and mixtures described in the toxicological world are clear. The chemicals analyzed in studies dealing with phytochemicals, however, are generally looking for helpful effects as opposed to minimizing deleterious ones as many toxicological experiments hope to achieve. Additionally, Gutierrez et al have shown that combinations of plant essential oils and their interactions with food ingredients shows potential synergy, additivity or antagonism using fractional inhibitory concentration indices (Gutierrez et al., 2008, 2009), a similar concept to Loewe Additivity.

Conclusion

Although the problem of synergy detection and compound interaction is not straightforward, it is of critical importance across multiple academic disciplines, and especially important in cancer treatment. In the current review, we detail the terminology used in quantifying synergy in different disciplines. It can be seen that across disciplines, different nomenclatures have developed and continue to be used today. By unifying these terms in this review, we hope to enhance communication and understanding across disciplines. We also discussed reference methods used to predict the effect of individual drugs under an additive model, as well as benefits and disadvantages to the two most common additive models used. We then detail the methods most commonly used for quantifying synergy, and highlight emerging methods. Finally, we discuss applications of synergy quantification in a range of fields.

Although the pace of methods development has steadily increased since the early twentieth century, there remain challenges with the current, common methodologies for measuring synergy. The majority of currently used methods discussed lack much statistical rigor (Greco et al., 1995; Foucquier and Guedj, 2015), as demonstrated by Greco et al in their 1995 review paper. Additionally, many of the discussed methods assume a constant potency ratio which is rarely biochemically appropriate. And while some methods have standalone software, the Chou-Talalay method, or an R package, MixLow, many still require user implementation or knowledge of particular programming languages to run supplied pieces of code as opposed to having a dedicated tool for running the method. Synergy is something that merits our attention and pursuit (Geary, 2013), and, given its potential importance across many domains, development of new methods and enhancements to existing methods could have far-reaching impacts.

Before concluding, we would also like to highlight one area of importance that is often unmentioned in the current methodological literature: synergistic interactions in sequential therapeutic approaches. This occurs when one or more drugs is administered prior to another drug or mixture of drugs and a synergistic interaction occurs between the first and second administration of drugs. This has been shown to exist among anti-cancer drugs as well as between epigenetic drugs and anti-cancer drugs (Azrak et al., 2004; Vijayaraghavalu et al., 2013), for example. Quantifying this type of synergistic interaction presents its own unique challenges. Challenges such as determining which drugs are interacting if working with mixtures or the significance of specific timing between administration of the drugs needs to be considered. There is an emerging clear gap between the known clinical utility of this type of dosing strategy, and quantitative methods to characterize this that needs to be addressed in future studies.

Author Contributions

All authors listed, have made substantial, direct and intellectual contribution to the work, and approved it for publication.

Funding

This work was supported by NIH NCI RO1CA161608 and NIH P01 CA142538 from the National Cancer Institute.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

References

Abdo, N., Xia, M., Brown, C. C., Kosyk, O., Huang, R., Sakamuru, S., et al. (2015). Population-based in vitro hazard and concentration-response assessment of chemicals: the 1000 genomes high-throughput screening study. Environ. Health Perspect. 123, 458–466. doi: 10.1289/ehp.1408775

PubMed Abstract | CrossRef Full Text | Google Scholar

Aronson, J. K. (2007). Concentration-effect and dose-response relations in clinical pharmacology. Br. J. Clin. Pharmacol. 63, 255–257. doi: 10.1111/j.1365-2125.2007.02871.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Azrak, R. G., Cao, S., Slocum, H. K., Tóth, K., Durrani, F. A., Yin, M.-b., et al. (2004). Therapeutic synergy between irinotecan and 5-fluorouracil against human tumor xenografts. Clin. Cancer Res. 10, 1121–1129. doi: 10.1158/1078-0432.CCR-0913-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Beam, A. L., and Motsinger-Reif, A. A. (2011). Optimization of nonlinear dose- and concentration-response models utilizing evolutionary computation. Dose Response 9, 387–409. doi: 10.2203/dose-response.09-030.Beam

PubMed Abstract | CrossRef Full Text | Google Scholar

Beam, A., and Motsinger-Reif, A. (2014). Beyond IC50s: towards robust statistical methods for in vitro association studies. J. Pharmacogenomics Pharmacoproteomics 5:1000121. doi: 10.4172/2153-0645.1000121

PubMed Abstract | CrossRef Full Text | Google Scholar

Berenbaum, M. C. (1977). Synergy, additivism and antagonism in immunosuppression. A critical review. Clin. Exp. Immunol. 28, 1–18.

PubMed Abstract | Google Scholar

Berenbaum, M. C. (1981). Criteria for analyzing interactions between biologically active agents. Adv. Cancer Res. 35, 269–335. doi: 10.1016/S0065-230X(08)60912-4

PubMed Abstract | CrossRef Full Text | Google Scholar

Berenbaum, M. C. (1985). The expected effect of a combination of agents: the general solution. J. Theor. Biol. 114, 413–431. doi: 10.1016/S0022-5193(85)80176-4

PubMed Abstract | CrossRef Full Text | Google Scholar

Berthoud, H. R. (2013). Synergy: a concept in search of a definition. Endocrinology 154, 3974–3977. doi: 10.1210/en.2013-1420

PubMed Abstract | CrossRef Full Text | Google Scholar

Bliss, C. I. (1939). The toxicity of poisons applied jointly. Ann. Appl. Biol. 26, 585–615. doi: 10.1111/j.1744-7348.1939.tb06990.x

CrossRef Full Text | Google Scholar

Boik, J. C., Newman, R. A., and Boik, R. J. (2008). Quantifying synergism/antagonism using nonlinear mixed-effects modeling: a simulation study. Stat. Med. 27, 1040–1061. doi: 10.1002/sim.3005

PubMed Abstract | CrossRef Full Text | Google Scholar

Boik, J. N. (2010). An R package for assessing drug synergism/antagonism. J. Stat. Softw. 34, 1–18. doi: 10.18637/jss.v034.i06

CrossRef Full Text | Google Scholar

Brown, C. C., Havener, T. M., Medina, M. W., Auman, J. T., Mangravite, L. M., Krauss, R. M., et al. (2012). A genome-wide association analysis of temozolomide response using lymphoblastoid cell lines reveals a clinically relevant association with MGMT. Pharmacogenet. Genomics 22, 796–802. doi: 10.1097/FPC.0b013e3283589c50

PubMed Abstract | CrossRef Full Text | Google Scholar

Brown, C. C., Havener, T. M., Medina, M. W., Jack, J. R., Krauss, R. M., McLeod, H. L., et al. (2014). Genome-wide association and pharmacological profiling of 29 anticancer agents using lymphoblastoid cell lines. Pharmacogenomics 15, 137–146. doi: 10.2217/pgs.13.213

PubMed Abstract | CrossRef Full Text | Google Scholar

Carpenter, D. O., Arcaro, K., and Spink, D. C. (2002). Understanding the human health effects of chemical mixtures. Environ. Health Perspect. 110(Suppl. 1), 25–42. doi: 10.1289/ehp.02110s125

PubMed Abstract | CrossRef Full Text | Google Scholar

Cedergreen, N. (2014). Quantifying synergy: a systematic review of mixture toxicity studies within environmental toxicology. PLoS ONE 9:e96580. doi: 10.1371/journal.pone.0096580

PubMed Abstract | CrossRef Full Text | Google Scholar

Chou, T. C. (2006). Theoretical basis, experimental design, and computerized simulation of synergism and antagonism in drug combination studies. Pharmacol. Rev. 58, 621–681. doi: 10.1124/pr.58.3.10

PubMed Abstract | CrossRef Full Text | Google Scholar

Chou, T. C. (2010). Drug combination studies and their synergy quantification using the Chou-Talalay method. Cancer Res. 70, 440–446. doi: 10.1158/0008-5472.CAN-09-1947

PubMed Abstract | CrossRef Full Text | Google Scholar

Chou, T. C., and Talalay, P. (1984). Quantitative analysis of dose-effect relationships: the combined effects of multiple drugs or enzyme inhibitors. Adv. Enzyme Regul. 22, 27–55. doi: 10.1016/0065-2571(84)90007-4

PubMed Abstract | CrossRef Full Text | Google Scholar

Chou, T. C., Tan, Q. H., and Sirotnak, F. M. (1993). Quantitation of the synergistic interaction of edatrexate and cisplatin in vitro. Cancer Chemother. Pharmacol. 31, 259–264. doi: 10.1007/BF00685668

PubMed Abstract | CrossRef Full Text | Google Scholar

Foucquier, J., and Guedj, M. (2015). Analysis of drug combinations: current methodological landscape. Pharmacol. Res. Perspect. 3:e00149. doi: 10.1002/prp2.149

PubMed Abstract | CrossRef Full Text | Google Scholar

Fraser, T. R. (1870–1871). An experimental research on the antagonism between the actions of physostigma atropia. Proc. R. Soc. Edinb. 7, 506–511. doi: 10.1017/S0370164600042565.

CrossRef Full Text | Google Scholar

Fraser, T. R. (1872). The antagonism between the actions of active substances. Br. Med. J. 2, 485–487. doi: 10.1136/bmj.2.618.485

CrossRef Full Text | Google Scholar

Geary, N. (2013). Understanding synergy. Am. J. Physiol. Endocrinol. Metab. 304, E237–E253. doi: 10.1152/ajpendo.00308.2012

PubMed Abstract | CrossRef Full Text | Google Scholar

Gessner, P. K. (1974). “The isobolographic method applied to drug interactions,” in Drug Interactions, eds P. L. Morselli, S. Garattini, and S. Cohen (New York, NY: Raven Press), 349–362.

Gessner, P. K. (1988). A straightforward method for the study of drug interactions: an isobolographic analysis primer. J. Am. Coll. Toxicol. 7, 987–1012. doi: 10.3109/10915818809014529

CrossRef Full Text | Google Scholar

Goldoni, M., and Johansson, C. (2007). A mathematical approach to study combined effects of toxicants in vitro: evaluation of the Bliss independence criterion and the Loewe additivity model. Toxicol. In Vitro 21, 759–769. doi: 10.1016/j.tiv.2007.03.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Goutelle, S., Maurin, M., Rougier, F., Barbaut, X., Bourguignon, L., Ducher, M., et al. (2008). The Hill equation: a review of its capabilities in pharmacological modelling. Fundam. Clin. Pharmacol. 22, 633–648. doi: 10.1111/j.1472-8206.2008.00633.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Grabovsky, Y., and Tallarida, R. J. (2004). Isobolographic analysis for combinations of a full and partial agonist: curved isoboles. J. Pharmacol. Exp. Ther. 310, 981–986. doi: 10.1124/jpet.104.067264

PubMed Abstract | CrossRef Full Text | Google Scholar

Greco, W. R., Bravo, G., and Parsons, J. C. (1995). The search for synergy: a critical review from a response surface perspective. Pharmacol. Rev. 47, 331–385.

PubMed Abstract | Google Scholar

Greco, W. R., Faessel, H., and Levasseur, L. (1996). The search for cytotoxic synergy between anticancer agents: a case of Dorothy and the ruby slippers? J. Natl. Cancer Inst. 88, 699–700. doi: 10.1093/jnci/88.11.699

PubMed Abstract | CrossRef Full Text

Groten, J. P. (2000). Mixtures and interactions. Food Chem. Toxicol. 38(1 Suppl), S65–S71. doi: 10.1016/S0278-6915(99)00135-0

PubMed Abstract | CrossRef Full Text | Google Scholar

Groten, J. P., Feron, V. J., and Sühnel, J. (2001). Toxicology of simple and complex mixtures. Trends Pharmacol. Sci. 22, 316–322. doi: 10.1016/S0165-6147(00)01720-X

PubMed Abstract | CrossRef Full Text | Google Scholar

Gutierrez, J., Barry-Ryan, C., and Bourke, P. (2008). The antimicrobial efficacy of plant essential oil combinations and interactions with food ingredients. Int. J. Food Microbiol. 124, 91–97. doi: 10.1016/j.ijfoodmicro.2008.02.028

PubMed Abstract | CrossRef Full Text | Google Scholar

Gutierrez, J., Barry-Ryan, C., and Bourke, P. (2009). Antimicrobial activity of plant essential oils using food model media: efficacy, synergistic potential and interactions with food components. Food Microbiol. 26, 142–150. doi: 10.1016/j.fm.2008.10.008

PubMed Abstract | CrossRef Full Text | Google Scholar

Hennessey, V. G., Rosner, G. L., Bast, R. C. Jr., and Chen, M.-Y. (2010). A Bayesian approach to dose-response assessment and synergy and its application to in vitro dose-response studies. Biometrics 66, 1275–1283. doi: 10.1111/j.1541-0420.2010.01403.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Hertzberg, R. C., and MacDonell, M. M. (2002). Synergy and other ineffective mixture risk definitions. Sci. Total Environ. 288, 31–42. doi: 10.1016/S0048-9697(01)01113-5

PubMed Abstract | CrossRef Full Text | Google Scholar

Jacobs, D. R. Jr., and Steffen, L. M. (2003). Nutrients, foods, and dietary patterns as exposures in research: a framework for food synergy. Am. J. Clin. Nutr. 78(3 Suppl), 508S–513S.

PubMed Abstract | Google Scholar

Konkoli, Z. (2011). Safe uses of Hill's model: an exact comparison with the Adair-Klotz model. Theor. Biol. Med. Model. 8:10. doi: 10.1186/1742-4682-8-10

PubMed Abstract | CrossRef Full Text | Google Scholar

Laetz, C. A., Baldwin, D. H., Collier, T. K., Hebert, V., Stark, J. D., and Scholz, N. L. (2009). The synergistic toxicity of pesticide mixtures: implications for risk assessment and the conservation of endangered Pacific salmon. Environ. Health Perspect. 117, 348–353. doi: 10.1289/ehp.0800096

PubMed Abstract | CrossRef Full Text | Google Scholar

Li, Y., Millikan, R. C., Bell, D. A., Cui, L., Tse, C.-K. J., Newman, B., et al. (2005). Polychlorinated biphenyls, cytochrome P450 1A1 (CYP1A1) polymorphisms, and breast cancer risk among African American women and white women in North Carolina: a population-based case-control study. Breast Cancer Res. 7, R12–R18. doi: 10.1186/bcr941

PubMed Abstract | CrossRef Full Text | Google Scholar

Liu, R. H. (2004). Potential synergy of phytochemicals in cancer prevention: mechanism of action. J. Nutr. 134(12 Suppl), 3479S–3485S.

PubMed Abstract | Google Scholar

Loewe, S. (1953). The problem of synergism and antagonism of combined drugs. Arzneimittelforschung 3, 285–290.

PubMed Abstract | Google Scholar

Loewe, S. (1957). Antagonisms and antagonists. Pharmacol. Rev. 9, 237–242.

PubMed Abstract | Google Scholar

Loewe, S. M. (1926). Effect of combinations: mathematical basis of problem. Arch. ExPathol. Pharmakol. 114, 313–326.

Google Scholar

Marking, L. L. (1977). “Method for assessing additive toxicity of chemical mixtures,” in Aquatic Toxicology and Hazard Evaluation. ASTM STP 634, eds F. L. Mayer and J. L. Hamelink (Philadelphia, PA: American Society for Testing and Materials), 99–108. Available online at: https://www.astm.org/DIGITAL_LIBRARY/STP/PAGES/STP32392S.htm

Google Scholar

Mumtaz, M. M., and Durkin, P. R. (1992). A weight-of-evidence approach for assessing interactions in chemical mixtures. Toxicol. Ind. Health 8, 377–406.

PubMed Abstract | Google Scholar

Nielsen, E. I., and Friberg, L. E. (2013). Pharmacokinetic-pharmacodynamic modeling of antibacterial drugs. Pharmacol. Rev. 65, 1053–1090. doi: 10.1124/pr.111.005769

PubMed Abstract | CrossRef Full Text | Google Scholar

Olshan, A. F., Weissler, M. C., Watson, M. A., and Bell, D. A. (2001). Risk of head and neck cancer and the alcohol dehydrogenase 3 genotype. Carcinogenesis 22, 57–61. doi: 10.1093/carcin/22.1.57

PubMed Abstract | CrossRef Full Text | Google Scholar

Pegram, M. D., Lopez, A., Konecny, G., and Slamon, D. J. (2000). Trastuzumab and chemotherapeutics: drug interactions and synergies. Semin. Oncol. 27(6 Suppl. 11), 21–25. discussion: 92–100.

PubMed Abstract | Google Scholar

Peters, E. J., Motsinger-Reif, A., Havener, T. M., Everitt, L., Hardison, N. E., Watson, V. G., et al. (2011). Pharmacogenomic characterization of US FDA-approved cytotoxic drugs. Pharmacogenomics 12, 1407–1415. doi: 10.2217/pgs.11.92

PubMed Abstract | CrossRef Full Text | Google Scholar

Prichard, M. N., and Shipman, C. Jr. (1990). A three-dimensional model to analyze drug-drug interactions. Antiviral Res. 14, 181–205. doi: 10.1016/0166-3542(90)90001-N

PubMed Abstract | CrossRef Full Text | Google Scholar

Scher, S. (2012). Opinion on the Toxicity and Assessment of Chemical Mixtures. Available online at: http://ec.europa.eu/health/scientific_committees/environmental_risks/docs/scher_o_155.pdf

Schwartz, J., Laden, F., and Zanobetti, A. (2002). The concentration-response relation between PM(2.5) and daily deaths. Environ. Health Perspect. 110, 1025–1029. doi: 10.1289/ehp.021101025

PubMed Abstract | CrossRef Full Text | Google Scholar

Schwartz, S. (2006). “Modern epidemiologic approaches to interaction: applications to the study of genetic interactions,” in Genes, Behavior, and the Social Environment: Moving Beyond the Nature/Nurture Debate, eds L. M. Hernandez and D. G. Blazer (Washington, DC: Institute of Medicine (US) Committee on Assessing Interactions Among Social, and Genetic Factors in Health, Editor. National Academies Press).

Google Scholar

Shen, J., Gammon, M. D., Terry, M. B., Wang, L., Wang, Q., Zhang, F., et al. (2005). Polymorphisms in XRCC1 modify the association between polycyclic aromatic hydrocarbon-DNA adducts, cigarette smoking, dietary antioxidants, and breast cancer risk. Cancer Epidemiol. Biomarkers Prev. 14, 336–342. doi: 10.1158/1055-9965.EPI-04-0414

PubMed Abstract | CrossRef Full Text | Google Scholar

Steel, G. G., and Peckham, M. J. (1979). Exploitable mechanisms in combined radiotherapy-chemotherapy: the concept of additivity. Int. J. Radiat. Oncol. Biol. Phys. 5, 85–91. doi: 10.1016/0360-3016(79)90044-0

PubMed Abstract | CrossRef Full Text | Google Scholar

Tallarida, R. J. (2001). Drug synergism: its detection and applications. J. Pharmacol. Exp. Ther. 298, 865–872.

PubMed Abstract | Google Scholar

Tallarida, R. J. (2006). An overview of drug combination analysis with isobolograms. J. Pharmacol. Exp. Ther. 319, 1–7. doi: 10.1124/jpet.106.104117

PubMed Abstract | CrossRef Full Text | Google Scholar

Tallarida, R. J. (2007). Interactions between drugs and occupied receptors. Pharmacol. Ther. 113, 197–209. doi: 10.1016/j.pharmthera.2006.08.002

PubMed Abstract | CrossRef Full Text | Google Scholar

Tallarida, R. J. (2012). Revisiting the isobole and related quantitative methods for assessing drug synergism. J. Pharmacol. Exp. Ther. 342, 2–8. doi: 10.1124/jpet.112.193474

PubMed Abstract | CrossRef Full Text | Google Scholar

Vijayaraghavalu, S., Dermawan, J. K., Cheriyath, V., and Labhasetwar, V. (2013). Highly synergistic effect of sequential treatment with epigenetic and anticancer drugs to overcome drug resistance in breast cancer cells is mediated via activation of p21 gene expression leading to G2/M cycle arrest. Mol. Pharm. 10, 337–352. doi: 10.1021/mp3004622

PubMed Abstract | CrossRef Full Text | Google Scholar

Wasserheit, J. N. (1992). Epidemiological synergy. Interrelationships between human immunodeficiency virus infection and other sexually transmitted diseases. Sex Transm. Dis. 19, 61–77. doi: 10.1097/00007435-199219020-00001

PubMed Abstract | CrossRef Full Text | Google Scholar

Webb, J. L. (1963). Effect of more than one inhibiter. Enzymes Metab. Inhibit. 1, 66–79, 487–512.

Weiner, G. J. (2010). Rituximab: mechanism of action. Semin. Hematol. 47, 115–123. doi: 10.1053/j.seminhematol.2010.01.011

PubMed Abstract | CrossRef Full Text | Google Scholar