Commentary: Cannabis use and resting state functional connectivity in the aging brain

A Commentary on
Cannabis use and resting state functional connectivity in the aging brain

by Watson, K. K., Bryan, A. D., Thayer, R. E., Ellingson, J. M., Skrzynski, C. J., and Hutchison, K. E. (2022). Front. Aging Neurosci. 14:804890. doi: 10.3389/fnagi.2022.804890

Introduction

We would like to share ideas on the publication “Cannabis Use and Remaining Functional Connectivity State in the Aging Brain” that compares behavioral and neuroimaging data across users and non-users. As the primary outcome, Watson et al. (2022) identified greater connectivity of the hippocampus and of the parahippocampal cortex with targets in the anterior lobes of the cerebellum in older cannabis users compared to non-users. Despite the relevant results with promising applications in several clinical settings that enclose degenerative disorders as Alzheimer's or Parkinson's disease, Watson et al. interpreted their data relying on significance tests to make inferences. A P-value that surpasses an arbitrary level of statistical significance is not a guarantee that the effect produced by the intervention has practical implications or clinical relevance (Amrhein et al., 2019). Likewise, to reduce a significance threshold (α) or to have the P-value replaced by other statistics are not the solution either. P-values are just the tip of the iceberg (Leek and Peng, 2015; Amrhein et al., 2019). It is fundamental to go beyond a threshold of statistical significance and discuss the size of the observed effect (by including mean or median values, for instance), as well as the (im)precision of this estimate (by adding confidence intervals). Measures of central tendency and dispersion were not disclosed in the original paper, and we re-analyzed the main findings using the Bayesian method under the assumption that this approach can help interpretating results on a “more likely” or “less likely” basis.

Reanalyses

The essential characteristic of Bayesian methods is their explicit use of probability for quantifying uncertainty in inferences based on statistical data analysis. Although one might assume that P < 0.05 is evidence against the null hypothesis (H₀), this is not necessarily the case since P-values can be highly misleading measures. Being so, although Watson et al. have suggested significantly altered connections between the left hippocampus and the cerebellum (Vermis IV, V and Cerebellum III left) based on P = 0.049, analysis on P-values based on the Minimum Bayes factor [minBF(p)] suggests weak evidence for the refusal of H₀. In other words, by converting P-value into minBF(p) = 0.40 (1/2.5), it could be concluded that the alternative hypothesis (H₁) is only 2.5 times more likely of occurring than H₀, being this level of evidence insufficient to allow assuming that the treatment (cannabis use) is effective to alter connections between targets, since the value lies under a score of 10 which is usually set as standard threshold (Stefan Angelika et al., 2019). Under the assumption of equal prior probabilities [Pr (H₁) = Pr (H₀) = 50%], the re-analysis found H₀ with nonnegligible minimum posterior probability (MPP) of 28.7% (i.e., 1/3.484). If MPP of H₀ is large, it would be appropriate not to reject the null hypothesis.

On the other hand, a second re-analysis on data shown by Watson et al. has taken T-values (3.52 and 3.43) and group size (N₁ = 43 and N₂ = 143) into account. Through this method, the Bayes Factor (BF₁₀) indicates probabilities in favor of H₁ of the order of 48 to 36 greater in relation to H₀, respectively (Figure 1). To examine the robustness of the Bayes factor (BF₁₀), we used a wide range of prior distributions, as follows: user specific prior (r = 1/$\sqrt{2}$), wide prior (r = 1), and ultrawide prior (r = $\sqrt{2}$.) In this perspective, the evidence in favor of H₁ is stable, suggesting that the analysis is robust. Most importantly, the a posteriori probabilities of 97.9% and 97.3% support a differential effect between elderly users and non-users, against the 2.1% and 2.7% chance that cannabis use does not change the connections at stake. In our re-analysis, the best estimate for an effect size was the median of 0.567 (Figure 1), with the credibility interval suggesting that the data is compatible with an effect of no < 0.230 or no more than 0.910 in magnitude.

Figure 1

Figure 1. Bayesian two-sample t-test (Two-sided analysis for H₁ estimation: δ Cauchy). (Left) The probability wheel on top visualizes the evidence in favor of the two rival hypotheses. The two gray dots indicate the prior and posterior density at the test value. The median and the 95% central credible interval of the posterior distribution are shown above the region with greater density. Figures on the (right) indicate the BF robustness plot. The maximum BF₁₀ value is obtained when setting the prior width r ≈ 0.53. Analyses were conducted in Jasp 0.17.1 software.

Given the variability across re-analyses, the relationship between the T critical values and respective significance probabilities (as originally reported) was tested and revealed scores incompatible to each other (for P = 0.049, a T value of 1.98 is expected). For T-values of 3.52 and 3.43, our re-analyses suggest P ≤ 0.001. Under the circumstances described, we believe that the original authors have committed an error in some of their analyses or in the description of the results.

Discussion

In summary, to better assess the value of a new therapeutic proposal, it is important to present sufficient information to answer the questions that motivated the research. One cannot properly conclude on potential benefits from a new therapy without knowing the estimated effect size of the treatment and the degree of statistical evidence. The level of confidence in experiments with “P < 0.05” cannot be the same for every significant result. In view of that, it is more informative to provide statistical conclusions that go beyond a single parameter and regardless of any preference in statistical approaches. In line, it is important that interpretating any probabilistic event should consider a solid theoretical foundation, aiming intervention in the context of clinical practices and the formulation of future actions or public health policies.

Author contributions

CC: substantial contributions to study conception and design. ON: substantial contributions to acquisition of data. All authors contributed to the article and approved the submitted version.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Amrhein, V., Greenland, S., and McShane, B. (2019). Scientists rise up against statistical significance. Nature 567, 305e7. doi: 10.1038/d41586-019-00857-9

PubMed Abstract | Crossref Full Text | Google Scholar

Leek, J. T., and Peng, R. D. (2015). Statistics: P values are just the tip of the iceberg. Nature 520, 612–612. doi: 10.1038/520612a

Crossref Full Text | Google Scholar

Stefan Angelika, M., Gronau Quentin, F., Schonbrodt Felix, D., and Eric-Jan, W. (2019). A tutorial on Bayes factor design analysis using an informed prior. Behav. Res. Meth. 51, 1042–1058. doi: 10.3758/s13428-018-01189-8

PubMed Abstract | Crossref Full Text | Google Scholar

Watson, K. K., Bryan, A. D., Thayer, R. E., Ellingson, J. M., Skrzynski, C. J., and Hutchison, K. E. (2022). Cannabis use and resting state functional connectivity in the aging brain. Front. Aging Neurosci. 14, 804890. doi: 10.3389/fnagi.2022.804890

PubMed Abstract | Crossref Full Text | Google Scholar