Mechanisms of Perceived Treatment Assignment and Subsequent Expectancy Effects in a Double Blind Placebo Controlled RCT of Major Depression

Objective: It has been suggested that patients' perception of treatment assignment might serve to bias results of double blind randomized controlled trials (RCT). Most previous evidence on the effects of patients' perceptions and the mechanisms influencing these perceptions relies on cross-sectional associations. This re-analysis of a double blind, placebo controlled RCT of pharmacological treatment of major depression set out to gather longitudinal evidence on the mechanism and effects of patients' perceived treatment assignment in the pharmacological treatment of major depression.

Methods: One-hundred eighty-nine outpatients with DSM-IV diagnosed major depression were randomized to SAMe 1,600–3,200 mg/d, escitalopram 10–20 mg/days, or placebo for 12 weeks. Data on depressive symptoms (17-item Hamilton Depression Scale; HDRS-17), adverse events and patients' perceived treatment assignment was collected at baseline, week 6, and week 12. The re-analysis focused on N = 166 (out of the originally included 189 participants) with available data on perceived treatment assignment.

Results: As in the parent trial, depressive symptoms (HDRS-17) significantly decreased over the course of 12 weeks and there was no difference between placebo, SAMe or escitalopram. A significant number of patients changed their perceptions about treatment assignment throughout the trial, especially between baseline and week 6. Improvement in depressive symptoms, but not adverse events significantly predicted perceived treatment assignment at week 6. In turn, perceived treatment assignment at week 6, but not actual treatment, predicted further improvement in depressive symptoms at week 12.

Conclusions: The current results provide longitudinal evidence that patients' perception of treatment assignment systematically change despite a double blind procedure and in turn might trigger expectancy effects with the potential to bias the validity of an RCT.

Parent study grant number: R01 AT001638

Parent study ClinicalTrials. gov Identifier: NCT00101452

Introduction

The belief that one is taking a medication can lead to improvement in numerous health conditions regardless of the presence or absence of a pharmacologic agent (1, 2). This expectation effect is specifically pronounced in the pharmacological treatment of depression (3, 4). Double blind randomized controlled trials (RCTs) assume that these expectations are equally balanced across treatment arms. Yet, the effectiveness of blinding in RCTs is rarely assessed or reported, and there are suggestions in the literature that patients are frequently un-blinded (5–7). If patients do learn which treatment arm they are in, expectancy effects due to perceived treatment assignment are no longer controlled for. This introduces a considerable amount of bias, as meta-epidemiological studies typically find un-blinded studies to exaggerate effect size by more than 30% compared to blinded studies (8–10). Moreover, it is important to note that not only an actual un-blinding, but any between-groups imbalance of the perceived treatment assignment can bias the results of a trial (11, 12).

Possible mechanisms that may influence perceived treatment assignment include the physical characteristics of the medication and the placebo, medication side effects, or beneficial effects on the health condition (13). Regarding the former, taste, color, shape, size, route and process of administration (13) might lead to un-blinding, if they differ between drug and placebo. Moreover, medication side effects could inadvertently serve to influence perception of treatment assignment (14, 15). Studies have shown that side effects are associated with patients and independent evaluators guessing treatment assignment (15, 16). The experience of side effects could then increase the treatment effect by enhancing the patient's expectation of benefit. This possibility is supported by both experimental and clinical evidence. Thus, for example, in an experimental pain task (17), participants receiving a placebo that produces side effects (so called, “active placebo”) achieved higher pain thresholds than those who received a non-active placebo. Clinical trials on the pharmacological treatment of pain or depression using active placebos as a control condition have found smaller differences between active medication and the placebo arm compared to similar trials using non-active placebos (18–21). It should be noted that in order for physical characteristics or side effects to influence perceived treatment assignment, the participant needs at least a certain amount of knowledge about these characteristics of the drug. Finally, improvement of symptoms may also indicate participants' perception of treatment assignment. Previous studies among various health conditions have shown an association between clinical improvement and patient perceptions regarding treatment assignment (22–27). However, a major limitation of most of those analyses is that perceived treatment assignment was elicited either before or after treatment, making it impossible to investigate mechanisms of un-blinding and its prospective impact on treatment outcome. If assessed at the beginning of a trial, it cannot be concluded whether mechanisms such as side effects or health improvement have had an influence on the perception of treatment assignment. In contrast, assessment at the end of a study does not indicate whether more side effects or greater improvement in health were due to the perceived treatment assignment, or whether the perception of treatment assignment was due to experienced side effects or improvement in health. Experimental evidence suggests that experienced improvement influences perceived treatment assignment, which in turn influences treatment outcome (28). Whether this is true for clinical trials remains unclear so far. Moreover, when using a single time point assessment, one cannot determine whether participants change their perception of treatment assignment throughout a trial.

To better understand (1) whether the perception of the treatment assignment changes over the course of a study, (2) and whether these changes are influenced by side effects or health improvement, and (3) whether the perceived treatment assignment is prospectively related to the treatment effects, a re-analysis of a three-armed, double blind RCT on the treatment of major depression was conducted. The parent trial (29) examined the effects of escitalopram or S-adenosyl-L-methionine (SAMe) vs. placebo in patients with major depression and assessed perceived treatment assignment at several points throughout the trial. Moreover, in the parent study, no significant differences in improvement in the 17-item Hamilton Depression Rating Scale (HDRS-17) total score or response rates were found between the three treatment arms, making it particularly interesting to investigate expectancy effects due to possible bias in perceived treatment.

Materials and Methods

Study Design and Procedure

This study is a secondary analysis of a two center, three-arm, double blind RCT (29) on the treatment of major depression with escitalopram or SAMe vs. placebo (clinical trials.gov: NCT00101452) conducted at two academic psychiatry centers in the U.S. Detailed methods for the parent trial have been described elsewhere (29). The study was approved by both local Institutional Review Boards.

Participants

One-hundred eighty-nine outpatients, 18–80 years old, who met criteria for current major depressive episode according to Structured Clinical Interview for DSM-IV (30) plus screening and baseline scores of ≥25 on the Inventory of Depressive Symptomatology-Clinician-Rated (31), were recruited from April 2005 to December 2009 through clinician referral and general advertisement (e.g., “Have you lost interest in things you used to enjoy, had appetite or sleep changes? Are you interested in natural remedies? Participate in a research study of a naturally occurring supplement called SAMe in treating Major Depressive Disorder”) in local newspapers, radio, and television. A ≥ 6 week use of SAMe or escitalopram during the concurrent episode as well as severe medical or other primary psychiatric disorder were exclusion criteria [for detailed description of exclusion criteria see (29)].

Procedure

After written informed consent participants were randomized in a 1:1:1 manner (stratified by center) for 12 weeks of double-blind treatment with SAMe (1,600 mg/d), escitalopram (10 mg/d), or placebo. A double-dummy design was used to maintain the blind, since SAMe tablets differed in appearance from escitalopram tablets. Participants were made aware of their odds of receiving any particular one of the three possible treatments. At week 6, for non-responders (<50% HAM-D reduction) escitalopram dose could be increased to 20 mg/d and SAMe to 3,200 mg/d for weeks 7–12. Participants who experienced intolerable side effects at the higher dose were allowed to decrease their dose to the previous level.

Assessment

Assessment relevant for the reanalysis took place during baseline, visit 4 (week 6) and visit 7 (week 12; end of active treatment). Antidepressant efficacy was assessed with the Hamilton 17-item Depression Rating Scale [HDRS-17; (32)]. Side effects were assessed using the Systematic Assessment for Treatment Emergent Events-Systematic Inquiry [SAFTEE (33)]. Side effects documented on the SAFTEE were categorized by severity as 0 (none), 1 (mild), 2 (moderate), and 3 (severe). Scores were calculated based on the number of adverse events reported by each subject that were treatment-emergent, which we defined as any SAFTEE side effect for which severity increased by 1 or more levels from baseline. Besides an overall side effect score, sub-category scores for gastrointestinal and sexual functioning side effects were calculated based on known pharmacologic profiles of the active treatments and side effect patterns reported in the parent study (29). In order to assess perception of treatment assignment patients were asked to guess whether they believed to have received SAMe, escitalopram or placebo.

Data Analysis

This re-analysis focused on the acute treatment phase only (baseline-12 weeks) including N = 166 participants from the intent-to-treat sample, with at least one post-baseline visit and available data on perceived treatment assignment. Patients with missing data on perceived treatment assignment did not differ from those included in the analysis with regard to HRDS-17 scores and side effects at the respective time points.

Frequency distribution, means and standard deviations were assessed for each variable. Non-normally distributed variables were log₁₀-transformed in order to satisfy the statistical assumptions of parametric tests. Baseline differences were tested with analysis of variance (ANOVA) for parametric data, and χ²-tests for categorical data. Change in clinical variables was analyzed with mixed ANOVAs with treatment assignment as the between-participant factor and time as the within-participants factor. Significant main or interaction effects were analyzed by Bonferroni corrected post-hoc tests. Differences in perceived treatment assignment distributions were analyzed using χ²-tests. To assess whether clinical improvement or side effects were prospectively associated with perceived treatment assignment, logistic regression analyses were performed using clinical improvement and side effects as predictors, actual treatment assignment (dummy coded with placebo as the reference condition) and previous perceived treatment assignment (active treatment vs. placebo) as covariates, and whether participants perceived themselves to be on active medication [placebo = 0; active treatment (SAMe or escitalopram) = 1] at the successive time point as the dependent variable. To assess whether perceived active treatment affected subsequent clinical improvement, linear multiple regression analysis were performed with perceived treatment [placebo = 0; active treatment (SAMe or escitalopram) = 1] as predictor, actual treatment assignment (dummy coded with placebo as the reference condition) and pre HDRS-17 score as covariates, and successive HDRS-17 score as dependent variable. To test for treatment arm specific effects of perceived treatment, the interaction term between actual treatment assignment and perceived treatment assignment was added to the multiple regression analysis and additional Bonferroni corrected post-hoc tests were carried out for each treatment arm individually. For all analyses, two-tailed significance was set at p < 0.05. All calculations were performed with SPSS Version 23 (Chicago, Illinois).

Results

Trial Characteristics

Demographic and baseline clinical characteristics for each treatment group are reported in Table 1. No significant differences were observed between the treatment groups.

TABLE 1

Table 1. Demographic and clinical characteristics throughout trial.

Changes in Depressive Symptoms and Side Effects Over Time

Depressive symptoms significantly declined over time (see Table 1). However, there was no significant between-group difference or group × time interaction. For both total side effect score and gastrointestinal side effects (see Table 1), there was no significant change from week 6 to week 12, between-group difference, or group × time interaction. For sexual functioning side effects (see Table 1), there was no significant change from week 6 to week 12, or group × time interaction, but the between-group difference was significant. Bonferroni-corrected post-hoc tests revealed that sexual functioning side effects were significantly higher with escitalopram compared to SAMe (p = 0.002) and to placebo (p = 0.009), but not different between SAMe and placebo (p = 0.714).

Changes in Perceived Treatment Assignment Distributions Over Time

Participants' perceptions of treatment assignment throughout the study can be seen in Figure 1. At baseline (before application of treatment), participants in the SAMe group (χ² = 14.00, df = 2, p = 0.001), the escitalopram group (χ² = 18.88, df = 2, p < 0.001) and the placebo group (χ² = 11.41, df = 2, p = 0.003) were expecting to receive SAMe significantly more often than would be expected, based on an equal assignment probability (1/3). At week 6, only participants in the escitalopram arm perceived themselves to be on SAMe more often than by chance (χ² = 6.37, df = 2, p = 0.041). Participants' perceived treatment assignment in the SAMe (χ² = 2.00, df = 2, p = 0.368) and placebo groups (χ² = 2.53, df = 2, p = 0.282) did not significantly differ from that expected in an equal treatment distribution.

FIGURE 1

Figure 1. Perceived treatment assignment throughout the trial in (A) all treatment arms, (B) the placebo arm, (C) the SAMe arm, and (D) the escitalopram arm.

At week 12, participants' perceived treatment assignment did not significantly differ from an equal distribution in the SAMe arm (χ² = 4.36, df = 2, p = 0.113), the escitalopram arm (χ² = 1.92, df = 2, p = 0.283) or the placebo arm (χ² = 0.54, df = 2, p = 0.764). There was no association between participants' baseline perceived treatment assignment and drop out by week 6 (χ² = 0.04, df = 2, p = 0.982) and week 6 perceived treatment assignment and drop out by week 12 (χ² = 1.90, df = 2, p = 0.387).

Overall, almost twice as many patients changed their perceived treatment assignment between baseline and week 6 (48.9%), compared to between week 6 and week 12 (26.0%; see Table 2 for detailed within-person change patterns). Between baseline and week 6 patients in the placebo group changed their perceived treatment assignment more often (60%) than patients in the SAMe and the escitalopram group (42.4%; 48.6%). Between week 6 and week 12 patients in the SAMe group changed their perception of treatment assignment more frequently (38.8%) than in the escitalopram and the placebo group (both: 20.8%).

TABLE 2

Table 2. Changes in perceived treatment assignment from baseline to week 6 and week 6 to week 12 by actual treatment group.

Factors Associated With Perceived Treatment Assignment

Prospective associations of clinical improvement and side effects with whether participants perceived to be on an active medication (SAMe or escitalopram) or placebo are reported in Table 3. At week 6, participants were significantly more likely to perceive that they were receiving active medication if they experienced clinical improvement. Among participants who experienced a reduction in HDRS-17 score of 13 or greater, all perceived that they were assigned to an active medication group. There was no threshold below which participants would certainly perceive themselves to be on placebo. Side effects and actual treatment assignment were not associated with participants' perception that they were receiving an active medication at week 6. Clinical improvement between week 6 and week 12, side effects, and actual treatment assignment, were not associated with participants' perceived treatment assignment at week 12.

TABLE 3

Table 3. Improvement and adverse effects predicting patients' perception to be on active medication (SAMe or escitalopram) or placebo at week 6 and week 12 using logistic regression analysis.

Prospective Associations of Perceived Treatment Assignment and Subsequent Improvement

Prospective associations of participants' perceived treatment assignment (active vs. placebo) and actual treatment assignment on clinical improvement are shown in Table 4. Controlling for baseline HDRS-17, neither actual nor perceived treatment assignment predicted HDRS-17 scores at week 6. However, controlling for week 6 HDRS-17, patients' perceived treatment assignment at week 6, but not actual treatment assignment predicted week 12 HDRS-17 scores. Participants perceiving they were taking active medication at week 6 showed significantly higher improvement in depressive symptoms at week 12 than participants believing they were taking placebo. Post-hoc analysis revealed no significant interaction between actual treatment assignment and week 6 perceived treatment predicting HDRS-17 improvement at week 12 [R² = 0.02, F_{(2, 76)} = 1.28, p = 0.284]. However, Bonferroni corrected post-hoc sub group analyses of the effect of perceived treatment at week 6 on treatment outcome at week 12 within each treatment group individually (Table 5) indicated that the effect of perceived treatment assignment was significant in the escitalopram treatment arm but not in the placebo and SAMe treatment arm (Figure 2).

TABLE 4

Table 4. Longitudinal associations of patients' perceived treatment assignment and actual drug assignment with subsequent improvement in depression (HDRS-17).

TABLE 5

Table 5. Longitudinal associations of patients' perceived treatment assignment with subsequent improvement in depression (HDRS-17) at week 12 individually by treatment group.

FIGURE 2

Figure 2. Mean week 12 treatment outcome by week 6 perceived treatment and actual treatment assignment.

Discussion

This re-analysis of a three arm, two-center, double blind RCT of SAMe or escitalopram vs. placebo for the treatment of major depression shows that a significant number of patients did change their perception of treatment assignment throughout the trial, corroborating previous analyses on longitudinal changes of perceived treatment assignment in clinical trials (11, 23). The large majority of change in perceived treatment assignment happened throughout the first 6 weeks of treatment and was significantly predicted by the preceding improvement in depressive symptomatology. Side effects did not seem to have influenced perceived treatment assignment. Although side effects are frequently mentioned as a potential mechanism informing perceived treatment assignment, this result is consistent with another re-analysis of an RCT containing two active treatments for opioid dependency. Oviedo-Joekes et al. (34) found that desired drug effects (drug related highs) but not overall side effects were associated with perceived treatment allocation. Possibly, in trials with more than one active treatment, side effects might not be a pivotal mechanism influencing perceived treatment assignment, since it may be more difficult to guess between two active treatments based simply on side effects. Moreover, patients randomized to placebo in the current re-analysis did not differ from the active treatment groups in terms of side-effects indicating a nocebo effect (35, 36). Hence, variance in side effects might not have been heterogeneous enough to be associated with perceived treatment. While a majority of patients expected to receive SAMe before treatment began, there was no imbalance of perceived group allocation in favor of actual group allocation during the active treatment phase. This indicates that patients overall were successfully blinded regarding their specific treatment allocation until the end the treatment phase.

However, patients believing to be on active medication at week 6 showed significantly higher improvement in depressive symptoms at week 12 than patients believing to be on placebo, indicating expectancy effects due to perceived treatment assignment. The expectancy effect did not seem to be influenced by selective drop out, given the lack of association between perceived treatment and subsequent attrition. It remains unclear whether the expectancy effect was of the same or different magnitude among all treatment arms. Post-hoc analysis suggests that this effect might have been more pronounced in the escitalopram treatment arm. However, the omnibus test for an interaction between perceived and actual treatment was not significant. Therefore, this suggestion remains speculative for the trial at hand and should encourage better powered re-analyses to further explore this issue.

Notwithstanding, while most previous studies—due to their cross-sectional analysis—were unable to differentiate whether the perception of receiving active medication enhanced the treatment response via expectancy effects, or whether the improvement at the end of these trials influenced the final perception of treatment assignment, this re-analysis indicates that both might be true successively. The suggested pathway of expectancy effects due to perceived treatment assignment found in experimental research (28)–improvement influences perceived treatment, which in turn influences treatment outcome-appears to be validated within this re-analysis of a clinical trial.

In view of the above, this re-analysis now longitudinally confirms expectancy effects in double blind RCTs, and further ads to research highlighting important limitations for the interpretation of effects found in double blind placebo controlled RCTs. In clinical practice, patients do not have reason to doubt that they are receiving active medication. In placebo controlled RCTs, however, this doubt is justified and potentially induces decreased expectancy, which results in an underestimation of the effectiveness of a drug compared to routine clinical practice (37, 38). More generally, double blind RCTs evaluate the specific treatment effect of a pharmacologic verum by comparing it to the response generated by the unspecific treatment effect in the placebo group. If the verum group's treatment response exceeds that in the placebo group, the drug is considered to have drug specific effects. However, such a judgment is only justifiable under the assumption that the treatment response in the verum group consists of the equivalent unspecific effects observed in the placebo group plus the specific effects of the verum (“additive model”). Such an additive model of drug and placebo effects, however, has frequently been questioned theoretically and empirically (39–41). In fact evidence from clinical trials and both behavioral and neuro-physiological experiments suggest that drug specific and unspecific effects can interact (38, 39) and hence might not be equal between a verum and a placebo arm. Therefore, drug specific effects can be both over and underestimated in double blind RCTs. Although results regarding differential expectancy effects with the treatment arms are inconclusive in the current re-analysis, based on the results of this RCT, it cannot be said with absolute certainty that neither SAMe nor escitalopram do have or do not have drug specific efficacy. There is the possibility that specific characteristics of the SAMe or esctitalopram arm induced expectancy effects, which would have led to an overestimation of drug specific effects. On the other hand, RCTs with more than one active treatment arm have been found to show enhanced expectancy effects since the uncertainty of receiving active treatment is lower (38). As a result, this might have led to ceiling effects masking the drug specific expectancy. To circumvent these pitfalls, new study designs to better evaluate the effects of pharmacological treatment have been proposed; these include the balanced placebo design, the balanced cross over design, balanced open-hidden design or delayed response design [see 39, 40 for more details]. However, until such innovations for testing pharmacological interventions become more established, double blind RCTs should at least assess and test for expectancy effects in a systematic manner.

Concerning the assessment of expectancy effects, the results of this re-analysis suggest that the current practice of measuring perceived treatment assignment only once—either at the beginning or at the end of a trial—is questionable. The disadvantages of end of trial assessment of perceived treatment assignment have already been mentioned above. However, some previous studies used baseline or early assessment of perceived treatment allocation for their analysis of blinding or expectancy effects. Yet, in this re-analysis the perception of treatment assignment at the start of the trial did not at all reflect the perception of treatment assignment throughout the trial, making it a very unreliable measure to operationalize un-blinding or expectancy effects. Future trials should use repeated assessment of patients' perceived treatment assignment to determine un-blinding and evaluate expectancy effects. In addition, it would be even more advisable to assess patients' outcome expectations throughout a trial. First, a patient who believes to be on a specific active medication but does not believe the medication to be effective will most likely not have any expectancy effects (37), a case that would not be differentiated by only assessing perceived treatment assignment. Secondly, expectations can be assessed on parametric (or at least ordinal) scales, giving the advantage of statistical power over the assessment of perceived treatment assignment assessed as nominal data. For further details on the assessment of patients' expectations in medical treatment see (42).

While not the focus of this re-analysis, one additional finding is of interest: a large majority of patients before the start of treatment expected to receive SAMe, despite being informed about the equal assignment probability. A similar pattern in favor of the “new” treatment has been observed in other trials before (11, 34). This might shed interest on the role of drug trial advertisement. For the trial of this re-analysis, advertisements emphasized the treatment of depression with SAMe, because it was thought that this would attract more participants interested in taking a natural product; this could have influenced patients' expectations. While the current re-analysis did not find any indication that these expectations influenced treatment effects, one might see some indication of advertisement or novelty effect (14) in the newest net-work meta-analysis on anti-depressants (43). Cipriani et al. did find that the same anti-depressant had a positive pooled effect size in trials when evaluated as a new agent, and negative effect sizes when evaluated as the “old” comparator agent. Hence, in trials with two or more active comparators it might be worthwhile to investigate whether framing such as “new” vs. “old” toward participants is an additional source of bias. If such a “novelty effect” existed, this would pose further evidence against an additive model.

Some limitations have to be considered when interpreting the results presented. First, the external validity is limited by the fact that the re-analysis was based on a three arm RCT with SAMe and escitalopram as the active treatments. As discussed above expectancy effects are considered to be higher in a study with two active treatments. Generalization to clinical practice is limited since patients usually have no doubt about receiving active medication. However, patients in clinical practice are likely to have expectations regarding the efficacy of the medication or expectations about side effects (42), which might also serve to influence treatment effects. Additionally, both active treatments are reasonably well tolerated. Therefore, it remains unclear, whether results would be different among active treatment with stronger side effect profiles (e.g., tricyclic antidepressants). Moreover, it remains unclear whether comparing a “classical” anti-depressant with a natural supplement anti-depressant might have different mechanisms of perceived treatment than in other trials. Related to that, it is unclear whether trial advertisement attracted individuals with specific interest in natural remedies, who might for example have more negative expectations about “classical” anti-depressants. Further, although there was no difference in participants providing perceived treatment data regarding depressive symptoms and side effects, nor an association with drop-out, one can not completely rule out whether missing guess data might be a source of bias.

In conclusion, this re-analysis showed that patients' perceptions about treatment assignment do change throughout a trial, that these perceptions appeared to be influenced by preceding improvement in depressive symptoms, and that perceptions about treatment assignment predicted further improvement. Building on previous cross-sectional and experimental evidence the current results further highlight issues with the interpretation of effects found in double blind RCTs. Future RCTs should apply multiple assessment of perceived treatment assignment and/or expectations throughout the trial. This would permit testing for possible expectation effects that might bias the comparison for specific efficacy, and provide the opportunity to further explore mechanisms of bias in double blind RCTs. Moreover, new study designs for testing pharmacological interventions should be considered, to get a more concise estimate of the specific effects of a pharmacological treatment.

Ethics Statement

This study was carried out in accordance with the recommendations of the U.S. Food and Drug Administration (FDA) guidelines with written informed consent from all subjects. All subjects gave written informed consent in accordance with the Declaration of Helsinki. The protocol was approved by the Institutional Review Boards of the Massachusetts General Hospital in Boston, MA and the Butler Hospital in Providence, RI.

Data Availability Statement

The datasets for this manuscript are not publicly available yet, because the principal investigators are still in the process of actively analyzing the data and preparing manuscripts for publication, based on the main goals and outcome measures put forth in the original funded grant proposal. Requests to access the datasets should be directed to David Mischoulon, MD, Ph.D., ZG1pc2Nob3Vsb25AbWdoLmhhcnZhcmQuZWR1.

Author Contributions

JL, SV, and DM designed the re-analysis, conducted data analysis, and wrote the current manuscript. DM, LP, LC, AT, and MF designed and conducted the original randomized controlled trial this re-analysis is based on. LB and AJC assisted with the statistical analysis. GP and AC assisted with the writing of the manuscript.

Funding

The original study was supported by the NIH and the National Center for Complementary and Alternative Medicine (NCCAM), R01 grant R01AT001638. SAMe tosylate and matching placebo were supplied by Pharmavite LLC, CA, United States. JL was supported by a fellowship within the Postdoc-Program of the German Academic Exchange Service (DAAD).

Disclaimer

The article reflects the views of the authors and may not reflect the opinions or views of all the study investigators, the NIH, or the NCCAM.

Conflict of Interest Statement

JL was supported by a fellowship within the Postdoc-Program of the German Academic Exchange Service (DAAD).

LP has received research support from Neuronetics, NIH, HRSA, and NeoSync. He has served on advisory panels for Abbott and AstraZeneca. He has served as a consultant to Gerson Lehrman, Wiley, Springer, Qatar National Research Fund, and Abbott.

LC has received research support from Neuronetics, NIH, and NeoSync. She has served on advisory panels or provided consulting services for Abbott, Corcept, Johnson &Johnson, and Takeda-Lundbeck.

AT has received research support from Neuronetics, NeoSync, and NIH.

GP has served as a consultant for Abbott Laboratories, Acadia Pharmaceuticals, Inc*, Alkermes, Inc, AstraZeneca PLC, Avanir Pharmaceuticals, Axsome Therapeutics*, Boston Pharmaceuticals, Inc., Brainsway Ltd, Bristol-Myers Squibb Company, Cephalon Inc., Dey Pharma, L.P., Eli Lilly Co., Genentech, Inc*, Genomind, Inc*, GlaxoSmithKline, Evotec AG, H. Lundbeck A/S, Inflabloc Pharmaceuticals, Janssen Global Services LLC*, Jazz Pharmaceuticals, Johnson & Johnson Companies*, Methylation Sciences Inc, Mylan Inc*, Novartis Pharma AG, One Carbon Therapeutics, Inc*, Osmotica Pharmaceutical Corp.*, Otsuka Pharmaceuticals, PAMLAB LLC, Pfizer Inc., Pierre Fabre Laboratories, Ridge Diagnostics (formerly known as Precision Human Biolaboratories), Shire Pharmaceuticals, Sunovion Pharmaceuticals, Taisho Pharmaceutical Co, Ltd, Takeda Pharmaceutical Company LTD, Theracos, Inc., and Wyeth, Inc.

GP has received honoraria (for lectures or consultancy) from Abbott Laboratories, Acadia Pharmaceuticals Inc, Alkermes Inc, Asopharma America Cntral Y Caribe, Astra Zeneca PLC, Avanir Pharmaceuticals, Bristol-Myers Squibb Company, Brainsway Ltd, Cephalon Inc., Dey Pharma, L.P., Eli Lilly Co., Evotec AG, Forest Pharmaceuticals, GlaxoSmithKline, Inflabloc Pharmaceuticals, Grunbiotics Pty LTD, Jazz Pharmaceuticals, H. Lundbeck A/S, Medichem Pharmaceuticals, Inc, Meiji Seika Pharma Co. Ltd, Novartis Pharma AG, Otsuka Pharmaceuticals, PAMLAB LLC, Pfizer, Pharma Trade SAS, Pierre Fabre Laboratories, Ridge Diagnostics, Shire Pharmaceuticals, Sunovion Pharmaceuticals, Takeda Pharmaceutical Company LTD, Theracos, Inc., Titan Pharmaceuticals, and Wyeth Inc.

GP has received research support (paid to hospital) from AstraZeneca PLC, Bristol-Myers Squibb Company, Forest Pharmaceuticals, the National Institute of Mental Health, Neuralstem, Inc, PAMLAB LLC, Pfizer Inc., Ridge Diagnostics (formerly known as Precision Human Biolaboratories), Sunovion Pharmaceuticals, Tal Medical, and Theracos, Inc.

GP has served (not currently) on the speaker's bureau for BristolMyersSquibb Co and Pfizer, Inc.

*Asterisk denotes activity undertaken on behalf of Massachusetts General Hospital.

MF: All disclosures can be view on line at: http://mghcme.org/faculty/faculty-detail/maurizio_fava

Research Support: Abbott Laboratories; Acadia Pharmaceuticals; Alkermes, Inc.; American Cyanamid;Aspect Medical Systems; AstraZeneca; Avanir Pharmaceuticals; AXSOME Therapeutics; Biohaven; BioResearch; BrainCells Inc.; Bristol-Myers Squibb; CeNeRx BioPharma; Cephalon; Cerecor; Clintara, LLC; Covance; Covidien; Eli Lilly and Company;EnVivo Pharmaceuticals, Inc.; Euthymics Bioscience, Inc.; Forest Pharmaceuticals, Inc.; FORUM Pharmaceuticals; Ganeden Biotech, Inc.; GlaxoSmithKline; Harvard Clinical Research Institute; Hoffman-LaRoche; Icon Clinical Research; i3 Innovus/Ingenix; Janssen R&D, LLC; Jed Foundation; Johnson & Johnson Pharmaceutical Research & Development; Lichtwer Pharma GmbH; Lorex Pharmaceuticals; Lundbeck Inc.; Marinus Pharmaceuticals; MedAvante; Methylation Sciences Inc; National Alliance for Research on Schizophrenia & Depression (NARSAD); National Center for Complementary and Alternative Medicine (NCCAM);National Coordinating Center for Integrated Medicine (NiiCM); National Institute of Drug Abuse (NIDA); National Institute of Mental Health (NIMH); Neuralstem, Inc.; NeuroRx; Novartis AG; Organon Pharmaceuticals; Otsuka Pharmaceutical Development, Inc.; PamLab, LLC.; Pfizer Inc.; Pharmacia-Upjohn; Pharmaceutical Research Associates., Inc.; Pharmavite&174; LLC; PharmoRx Therapeutics; Photothera; Reckitt Benckiser; Roche Pharmaceuticals; RCT Logic, LLC (formerly Clinical Trials Solutions, LLC); Sanofi-Aventis US LLC; Shire; Solvay Pharmaceuticals, Inc.; Stanley Medical Research Institute (SMRI); Synthelabo; Taisho Pharmaceuticals; Takeda Pharmaceuticals; Tal Medical; VistaGen; Wyeth-Ayerst Laboratories

Advisory Board/Consultant: Abbott Laboratories; Acadia; Affectis Pharmaceuticals AG; Alkermes, Inc.; Amarin Pharma Inc.; Aspect Medical Systems; AstraZeneca; Auspex Pharmaceuticals; Avanir Pharmaceuticals; AXSOME Therapeutics; Bayer AG; Best Practice Project Management, Inc.; Biogen; BioMarin Pharmaceuticals, Inc.; Biovail Corporation; BrainCells Inc; Bristol-Myers Squibb; CeNeRx BioPharma; Cephalon, Inc.; Cerecor; CNS Response, Inc.; Compellis Pharmaceuticals; Cypress Pharmaceutical, Inc.; DiagnoSearch Life Sciences (P) Ltd.; Dinippon Sumitomo Pharma Co. Inc.; Dov Pharmaceuticals, Inc.; Edgemont Pharmaceuticals, Inc.; Eisai Inc.; Eli Lilly and Company; EnVivo Pharmaceuticals, Inc.; ePharmaSolutions; EPIX Pharmaceuticals, Inc.; Euthymics Bioscience, Inc.; Fabre-Kramer Pharmaceuticals, Inc.; Forest Pharmaceuticals, Inc.; Forum Pharmaceuticals; GenOmind, LLC; GlaxoSmithKline; Grunenthal GmbH; Indivior; i3 Innovus/Ingenis; Intracellular; Janssen Pharmaceutica; Jazz Pharmaceuticals, Inc.; Johnson & Johnson Pharmaceutical Research & Development, LLC; Knoll Pharmaceuticals Corp.; Labopharm Inc.; Lorex Pharmaceuticals; Lundbeck Inc.; Marinus Pharmaceuticals; MedAvante, Inc.; Merck & Co., Inc.; MSI Methylation Sciences, Inc.; Naurex, Inc.; Navitor Pharmaceuticals, Inc.; Nestle Health Sciences; Neuralstem, Inc.; Neuronetics, Inc.; NextWave Pharmaceuticals; Novartis AG; Nutrition 21; Orexigen Therapeutics, Inc.; Organon Pharmaceuticals; Osmotica; Otsuka Pharmaceuticals; Pamlab, LLC.; Pfizer Inc.; PharmaStar; Pharmavite^® LLC.; PharmoRx Therapeutics; Precision Human Biolaboratory; Prexa Pharmaceuticals, Inc.; PPD; Purdue Pharma; Puretech Ventures; PsychoGenics; Psylin Neurosciences, Inc.; RCT Logic, LLC (formerly Clinical Trials Solutions, LLC); Relmada Therapeutics, Inc.; Rexahn Pharmaceuticals, Inc.; Ridge Diagnostics, Inc.; Roche; Sanofi-Aventis US LLC.; Sepracor Inc.; Servier Laboratories; Schering-Plough Corporation; Shenox Pharmaceuticals; Solvay Pharmaceuticals, Inc.; Somaxon Pharmaceuticals, Inc.; Somerset Pharmaceuticals, Inc.; Sunovion Pharmaceuticals; Supernus Pharmaceuticals, Inc.; Synthelabo; Taisho Pharmaceuticals; Takeda Pharmaceutical Company Limited; Tal Medical, Inc.; Tetragenex; Teva Pharmaceuticals; TransForm Pharmaceuticals, Inc.; Transcept Pharmaceuticals, Inc.; Usona Institute,Inc.; Vanda Pharmaceuticals, Inc.; Versant Venture Management, LLC; VistaGen

Speaking/Publishing: Adamed, Co; Advanced Meeting Partners; American Psychiatric Association; American Society of Clinical Psychopharmacology; AstraZeneca; Belvoir Media Group; Boehringer Ingelheim GmbH; Bristol-Myers Squibb; Cephalon, Inc.; CME Institute/Physicians Postgraduate Press, Inc.; Eli Lilly and Company; Forest Pharmaceuticals, Inc.; GlaxoSmithKline; Imedex, LLC; MGH Psychiatry Academy/Primedia; MGH Psychiatry Academy/Reed Elsevier; Novartis AG; Organon Pharmaceuticals; Pfizer Inc.; PharmaStar; United BioSource, Corp.; Wyeth-Ayerst Laboratories.

Stock/Other Financial Options: Equity Holdings: Compellis; PsyBrain, Inc.

Royalty/patent, other income: Patents for Sequential Parallel Comparison Design (SPCD), licensed by MGH to Pharmaceutical Product Development, LLC (PPD) (US_7840419, US_7647235, US_7983936, US_8145504, US_8145505); and patent application for a combination of Ketamine plus Scopolamine in Major Depressive Disorder (MDD), licensed by MGH to Biohaven. Patents for pharmacogenomics of Depression Treatment with Folate (US_9546401, US_9540691).

Copyright for the MGH Cognitive & Physical Functioning Questionnaire (CPFQ), Sexual Functioning Inventory (SFI), Antidepressant Treatment Response Questionnaire (ATRQ), Discontinuation-Emergent Signs & Symptoms (DESS), Symptoms of Depression Questionnaire (SDQ), and SAFER; Lippincott, Williams & Wilkins; Wolkers Kluwer; World Scientific Publishing Co.Pte.Ltd.

DM has received research support from the Bowman Family Foundation, Nordic Naturals, and PharmoRx. He has provided unpaid consulting for Pharmavite LLC and Gnosis USA, Inc. He has received writing honoraria from Pamlab and Nordic Naturals, and speaking honoraria from Blackmores, and the MGH Psychiatry Academy. He has received royalties from Lippincott Williams & Wilkins, for textbook “Natural Medications for Psychiatric Disorders: Considering the Alternatives” (DM and Jerrold F Rosenbaum, Eds.).

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

References

1. Enck P, Bingel U, Schedlowski M, Rief W. The placebo response in medicine: minimize, maximize or personalize? Nat Rev Drug Discov. (2013)12:191–204. doi: 10.1038/nrd3923

CrossRef Full Text

2. Benedetti F. Mechanisms of placebo and placebo-related effects across diseases and treatments. Annu Rev Pharmacol Toxicol. (2008) 48:33–60. doi: 10.1146/annurev.pharmtox.48.113006.094711

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Kirsch I, Moore TJ, Scoboria A, Nicholls SS. The emperor's new drugs: an analysis of antidepressant medication data submitted to the U.S. food and drug administration. Prev Treat. (2002) 5:23. doi: 10.1037/1522-3736.5.1.523a

CrossRef Full Text

4. Kirsch I, Deacon BJ, Huedo-Medina TB, Scoboria A, Moore TJ, Johnson BT. Initial severity and antidepressant benefits: a meta-analysis of data submitted to the food and drug administration. PLoS Med. (2008) 5:0260–8. doi: 10.1371/journal.pmed.0050045

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Baethge C, Assall OP, Baldessarini RJ. Systematic review of blinding assessment in randomized controlled trials in schizophrenia and affective disorders 2000-2010. Psychother Psychosom. (2013) 82:152–60. doi: 10.1159/000346144

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Hróbjartsson A, Forfang E, Haahr MT, Als-Nielsen B, Brorson S. Blinded trials taken to the test: an analysis of randomized clinical trials that report tests for the success of blinding. Int J Epidemiol. (2007) 36:654–63. doi: 10.1093/ije/dym020

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Bello S, Moustgaard H, Hróbjartsson A. Unreported formal assessment of unblinding occurred in 4 of 10 randomized clinical trials, unreported loss of blinding in 1 of 10 trials. J Clin Epidemiol. (2017) 81:42–50. doi: 10.1016/j.jclinepi.2016.08.002

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Wood L, Egger M, Gluud LL, Schulz KF, Jüni P, Altman DG, et al. Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study. BMJ (2008) 336:601–5. doi: 10.1136/bmj.39465.451748.AD

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Hróbjartsson A, Emanuelsson F, Thomsen ASS, Hilden J, Brorson S. Bias due to lack of patient blinding in clinical trials. a systematic review of trials randomizing patients to blind and nonblind sub-studies. Int J Epidemiol. (2014) 43:1272–83. doi: 10.1093/ije/dyu115

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Hróbjartsson A, Sofia A, Thomsen S. Observer bias in randomised clinical trials with binary outcomes: systematic review of trials with both blinded and non-blinded outcome assessors. BMJ (2012) 1119:1–11. doi: 10.1136/bmj.e1119

CrossRef Full Text

11. Rees JR, Wade TJ, Levy DA, Colford JM, Hilton JF. Changes in beliefs identify unblinding in randomized controlled trials: a method to meet CONSORT guidelines. Contemp Clin Trials. (2005) 26:25–37. doi: 10.1016/j.cct.2004.11.020

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Mathieu E, Herbert RD, McGeechan K, Herbert JJ, Barratt AL. A theoretical analysis showed that blinding cannot eliminate potential for bias associated with beliefs about allocation in randomized clinical trials. J Clin Epidemiol. (2014) 67:667–71. doi: 10.1016/j.jclinepi.2014.02.001

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Desbiens NA. In randomized controlled trials, should subjects in both placebo and drug groups be expected to guess that they are taking drug 50% of the time? Med Hypotheses (2002) 59:227–32. doi: 10.1016/S0306-9877(02)00205-0

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Schulz KF, Grimes DA. Blinding in randomised trials: hiding who got what. Lancet (2002) 359:696–700. doi: 10.1016/S0140-6736(02)07816-9

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Turner JA, Jensen MP, Warms CA, Cardenas DD. Blinding effectiveness and association of pretreatment expectations with pain improvement in a double-blind randomized controlled trial. Pain. (2002) 99:91–9. doi: 10.1016/S0304-3959(02)00060-X

PubMed Abstract | CrossRef Full Text | Google Scholar

16. White K, Kando J, Pharm D, Park T, Waternaux C, Brown WA. Side effects and the “Blindability” of clinical drug trials. Am J Psychiatry (1992) 149:1730–1.

PubMed Abstract | Google Scholar

17. Rief W, Glombiewski JA. The hidden effects of blinded, placebo-controlled randomized trials: An experimental investigation. Pain (2012) 153:2473–7. doi: 10.1016/j.pain.2012.09.007

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Gilron I, Tu D, Holden RR. Sensory and affective pain descriptors respond differentially to pharmacological interventions in neuropathic conditions. Clin J Pain (2013) 29:124–31. doi: 10.1097/AJP.0b013e31824ce65c

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Kinney MAO, Mantilla CB, Carns PE, Passe MA, Brown MJ, Hooten WM, et al. Preoperative gabapentin for acute post-thoracotomy analgesia: a randomized, double-blinded, active placebo-controlled study. Pain Pract. (2012) 12:175–83. doi: 10.1111/j.1533-2500.2011.00480.x

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Simmonds MK, Rashiq S, Sobolev IA, Dick BD, Gray DP, Stewart BJ, et al. The effect of single-dose propofol injection on pain and quality of life in chronic daily headache: a randomized, double-blind, controlled trial. Anesth Analg. (2009) 109:1972–80. doi: 10.1213/ANE.0b013e3181be3f86

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Moncrieff J, Wessely S, Hardy R. Active placebos versus antidepressants for depression. Cochrane Database Syst Rev. (2004):CD003012. doi: 10.1002/14651858.CD003012.pub2

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Dar R, Barrett SP. The effects of beliefs regarding drug assignment in experimental and field studies of nicotine delivery devices: a review. J Psychopharmacol. (2014) 28:1071–9. doi: 10.1177/0269881114548295

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Correa JB, Heckman BW, Marquinez NS, Drobes DJ, Unrod M, Roetzheim RG, et al. Perceived medication assignment during a placebo-controlled laboratory study of varenicline: temporal associations of treatment expectancies with smoking-related outcomes. Psychopharmacology (2014) 231:2559–66. doi: 10.1007/s00213-013-3420-2

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Colagiuri B, Morley K, Boakes R, Haber P. Expectancy in double-blind placebo-controlled trials: An example from alcohol dependence. Psychother Psychosom. (2009) 78:167–71. doi: 10.1159/000206871

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Thomas JL, Guo H, Lynam IM, Powell JN, Okuyemi KS, Bronars CA, et al. The impact of perceived treatment assignment on smoking cessation outcomes among African-American smokers. J Gen Intern Med. (2008) 23:1361–6. doi: 10.1007/s11606-008-0656-3

PubMed Abstract | CrossRef Full Text | Google Scholar

26. McRae C, Cherin E, Yamazaki TG, Diem G, Vo AH, Russell D, et al. Effects of perceived treatment on quality of life and medical outcomes in a double-blind placebo surgery trial. Arch Gen Psychiatry (2004) 61:412–20. doi: 10.1001/archpsyc.61.4.412

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Chen JA, Papakostas GI, Youn SJ, Baer L, Clain AJ, Fava M, et al. Association between patient beliefs regarding assigned treatment and clinical response: reanalysis of data from the Hypericum Depression Trial Study Group. J Clin Psychiatry (2011) 72:1669–76. doi: 10.4088/JCP.10m06453

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Colagiuri B, Boakes R. Perceived treatment, feedback, and placebo effects in double-blind RCTs: an experimental analysis. Psychopharmacology (2010) 208:433–41. doi: 10.1007/s00213-009-1743-9

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Mischoulon D, Price LH, Carpenter LL, Tyrka AR, Papakostas GI, Baer L, et al. A double-blind, randomized, placebo-controlled clinical trial of S-adenosyl-L -methionine (SAMe) vs. escitalopram in major depressive disorder. J Clin Psychiatry (2014) 75:370–6. doi: 10.4088/JCP.13m08591

CrossRef Full Text

30. First MB, Spitzer RL, Gibbon M, Williams JB. Structured Clinical Interview For DSM-IV Axis I Disorders: Clinician Version (SCID-CV). Washington, DC: American Psychiatric Press (1997).

31. Rush AJ, Gullion CM, Basco MR, Jarrett RB, Trivedi MH. The inventory of depressive symptomatology (IDS): psychometric properties. Psychol Med. (1996) 26:477–86. doi: 10.1017/S0033291700035558

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Hamilton M. A rating scale for depression. J Neurol Neurosurg Psychiatry (1960) 23:56–62. doi: 10.1136/jnnp.23.1.56

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Levine J, Schooler NR. SAFTEE: a technique for the systematic assessment of side effects in clinical trials. Psychopharmacol Bull. (1986) 22:343–81.

PubMed Abstract | Google Scholar

34. Oviedo-Joekes E, Marchand K, Palis H, Guh D, Brissette S, Lock K, et al. Predictors of treatment allocation guesses in a randomized controlled trial testing double-blind injectable hydromorphone and diacetylmorphine for severe opioid use disorder. Addict Res Theory (2017) 25:263–72. doi: 10.1080/16066359.2016.1263729

CrossRef Full Text

35. Barsky AJ, Saintfort R, Rogers MP, Borus JF. Nonspecific medication side effects and the nocebo phenomenon. JAMA (2002) 287:622–7. doi: 10.1001/jama.287.5.622

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Colloca L, Finniss D. Nocebo effects, patient-clinician communication, and therapeutic outcomes. JAMA (2012) 307:567–8. doi: 10.1001/jama.2012.115

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Colagiuri B. Participant expectancies in double-blind randomized placebo-controlled trials: potential limitations to trial validity. Clin Trials. (2010) 7:246–55. doi: 10.1177/1740774510367916

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Enck P, Klosterhalfen S, Weimer K, Horing B, Zipfel S. The placebo response in clinical trials: more questions than answers. Philos Trans R Soc B Biol Sci. (2011) 366:1889–95. doi: 10.1098/rstb.2010.0384

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Kube T, Rief W. Are placebo and drug-specific effects additive? questioning basic assumptions of double-blinded randomized clinical trials and presenting novel study designs. Drug Discov Today (2017) 22:729–35. doi: 10.1016/j.drudis.2016.11.022

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Enck P, Klosterhalfen S, Zipfel S. Novel study designs to investigate the placebo response. BMC Med Res Methodol. (2011) 11:90. doi: 10.1186/1471-2288-11-90

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Kirsch I. Are drug and placebo effects in depression additive? Biol Psychiatry (2000) 47:733–5. doi: 10.1016/S0006-3223(00)00832-5

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Laferton JAC, Kube T, Salzmann S, Auer CJ, Shedden-Mora MC. Patients' expectations regarding medical treatment: a critical review of concepts and their assessment. Front Psychol. (2017) 8:233. doi: 10.3389/fpsyg.2017.00233

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Cipriani A, Furukawa TA, Salanti G, Chaimani A, Atkinson LZ, Ogawa Y, et al. Comparative efficacy and acceptability of 21 antidepressant drugs for the acute treatment of adults with major depressive disorder: a systematic review and network meta-analysis. Lancet (2018) 1–10. doi: 10.1016/S0140-6736(17)32802-7

PubMed Abstract | CrossRef Full Text | Google Scholar