Milena Čukić1,2,3*
Victoria López4- 1Institute for Technology of Knowledge, Complutense University, Madrid, Spain
- 23EGA B.V., Amsterdam, Netherlands
- 3General Physiology and Biophysics Department, Belgrade University, Belgrade, Serbia
- 4Quantitative Methods Department, Cunef University, Madrid, Spain
The advent of artificial intelligence (AI) and machine learning (ML) in particular, in medicine, holds many promises. Although the acceptance of any innovation in medicine is chronically slow, psychiatry showed to be especially conservative in this regard (1). There are brilliant examples of ML applications in medicine, such as skin-cancer/sarcoma detection (2), early detection of retinopathies (3), and many more (4). But despite a lot of effort invested in computational psychiatry projects (5) we can see zero clinical applications (6). In addition, recent publications coming from review done by AI experts are showing that ‘medical information is more complex and less available than the web data that many algorithms were trained on, so results can be misleading' (7, 8).
Our group focused mainly on data-driven computational psychiatry research (9–14). We also became aware of so-called unwarranted optimism (15–17) and reported on it (10, 12). The expression' unwarranted optimism' is coined in ML community to signify for unrealistically inflated high accuracies of models due to unresolved Dimensionality of problem, absent external validation, unproportional ratio between number of variables and number of subjects in high-dimensional medical datasets, and existance of unattended blind spots. It also illustrates the phenomenon that the scientific community, in our opinion, lulles itself into thinking that we are developing models that work much better than they actually do (7).
A recent publication demonstrated that ML's purely reliance on patient's medical history, medication, epidemiological data, and scales/questionaries data (18) are simply not capable of providing practically useful results. We also explored the possibilities of this methodology in forecasting mania in bipolar depression disorder-BDD (13, 14, 19, 20). In this research we collected daily self-reports (via mobile phone applications), clinical assessment (standard clinical interviews and scales/questionaries), medical histories (including medication, and other important variables), several sleep variables, smartwatch variables (173 variables per person in total) in attempt to construct an accurate dynamical model of transition between five clinically defined states and in order to forecast mania phase. We used complex pipeline, several feature extraction methods, several feature selection models, and applied four different ML models and network flow model, in order to mathematically describe clinically compiled data (to represent the bidirectional transition between five distinct phases in BDD). The aim was in essence to extract the most relatable variables that have prognostic value in early warning of mania, that resulted in real personalized medicine application. Among all the variables the best predictors of mania were sleep quality (and duration) and irritability (13), and Random Forest scored the best. The classification using only selected variables produced better results than using all available information. Hence, dimensionality reduction of a problem was crucial to this research.
Whelan, Garavan, Gillan, and their colleagues, explained in their publications before 2017, why computational psychiatry projects, even when relying on neuroimaging data are flawed (16, 17), arguing that some basic postulates from Information theory and Statistical learning theory are ignored, despite wide accessibility of many ML models. The consequence is overly optimistic (and misleading) results, that are not leading to clinically useful applications [see also (21–23)]. There are many publications that confirmed (among them the 2021 report from Alan Turing Institute on faulty AI application in Health) the notion that majority of AI applications in Health are simply yielding very poor results, like for example famous IBM's Watson for Oncology that failed catastrophically [(24) report (25)]. See for example (7, 26–28), for review of this particularly inflated expectations of machine learning applications in Health. As phenomenon described in Statistical Learning Theory, a “Curse of dimensionality”, demonstrated to be the central problem in particular with datasets with the large number of features in vast digital health data, shown to be challenging the development of robust AI models (in particular, their generalizability). Whenever you sample from all the possible values, the average interpoint distance between samples is rising as the dimensionality of that data space changes (1D, 2D, 3D, etc.). The increased sparsity in the relevant feature space exponentially increases the volume of blind spots in data (7). Those are contiguous regions of feature space for which we don't have samples. By this the training set becomes biased in an important way, and so fails to include samples from the region (7). A small high-dimensional training sample (characteristic for majority of health applications) is susceptible to dataset blind spots (26). Also, the volume of blind spots scales exponentially with the number of features. If data from the sample is susceptible to blind spots and the data from those blind spots are encountered after deployment, the model can produce incorrect treatment recommendations that are not detected during model development (7).
We argue here that the central thing that can lead to the resolution of this frustration, is an addition of electrophysiological signals analysis, and appropriate characterization of it which yields highly accurate results in detection and prediction of any ML model used (11, 12). The overall accuracies per seven ML models used, depending of the number of principal components included, were between 92 and 95%, showing that the proper non-linear characterization of a resting EEG was the key for practically useful detection. We showed that in this way (characterizing EEG with non-linear measures capable to accurately detect its intrinsic dynamics), it is possible to discern between episode and remission phase in MDD (9), besides accurate detection of depression. Other groups of researchers demonstrated that it is possible to detect who is the responder to transcranial magnetic stimulation (rTMS), since this therapy has repeatedly been shown to be effective even in treatment resistant depression (TRD) (29, 30). Another non-invasive brain stimulation technique (NIBS), transcranial direct current stimulation (tDCS), has shown to be effective in MDD treatment (31, 32). We demonstrated how this modality of stimulation leaves a detectable impact on the brain lasting longer than half an hour after the stimulus was presented (9). In another publication, we explained why NIBS techniques might work in depression treatment, based on the physiological complexity approach (32). By connecting earlier findings coming from fMRI research (33), observed increased complexity in EEG (34, 35) and already mentioned decreased complexity after the therapy (29, 30), we concluded that the feature of successful therapy for depression, must be connected to its ability to decrease said aberrated complexity, that represents the distinct internal dynamic.
The key concept to understand here is the so-called physiological complexity (or complex variability in physiology), an analytical approach to electrophysiological signal analysis stemming from electrical engineering, statistical physics and complex systems dynamics theory (chaos) (36–40). A more familiar name for this approach is fractal and nonlinear analysis (41). Despite the fact that many medical professionals are labeling this approach “novel” it is not novel by any standard; seminal work by Mandelbrot from 70's, Pincus, Hausdorf, Peng, and Goldeberger from 80's and 90's last century made that possible. They all built on early mathematics work of Cantor (Cantor's set, 1893), Peano (1890), Sierpinski (Triangle,1907), Koch (Snowflake, 1909), Lucia (Lucia's set, 1917) and others who could not generalize their findings before the advent of modern computers.
If a researcher in any medical field wants to explore the effect of a certain factor, the most probable way to do it is to calculate the means, standard deviations, p-values, and other measures coming from frequentist statistics that dominates the field. Irregularity statistics, like any entropy-based measure, for example, quantify the changes in physiological systems in a much more accurate and practically meaningful way (42). These two approaches (standard or conventional vs. non-linear) are simply measuring different information contained in the data, but as repeatedly shown physiological signals are far more complicated that we previously thought (43). Knowing that human physiology is not linear (in essence, not additive) and has many fractal dependencies in its control mechanisms, a better approach to analyzing signals from such a complex systems dynamics would be non-linear analysis (38, 39). Wouldn't it be logical to apply analytics that is better suited for non-stationary, non-linear, and noisy signals, than to just focus on how smeared are the data around the means?
From the existing literature, coming mainly from engineering and technical background, it is clear that fractal and non-linear analysis is much better suited for this task (38, 41).
Just to mention some of the facts important in research of depression (and mood disorders in general), to improve the understanding of the above-mentioned research. Cortico-vagal control (CVC) is connected to heart rate variability (HRV), which showed to be a robust marker of depression, anxiety, and several other psychiatric conditions (44–46). Cortico-vagal control (as well as many structural and functional physiological phenomena) is proven to have fractal nature (40, 44). Heart dynamics also has a fractal structure (40). There is much research evidence on the connection between autonomous nervous system (ANS) and heart dynamics in depression, obtained by use of non-linear analysis approach (47–51). There is also evidence that non-linear measures are much more effective in detecting this relationship with a much larger effect size in publications in the last two decades [(52) in review].
What we know now, from the analytical perspective and possible application in clinical practice, is that by the mere addition of that specific non-linear characterization of signal, possible in real-time, one can: detect depression (10, 11, 34, 53, 54), detect the subtypes of depression—melancholic vs. non-melancholic (49), detect comorbidities (48), discern episode and remission phase (9), detect cardiovascular risks early (55), differentiate between unipolar and bipolar depression (56) and even detect existing but unreported suicidal ideation (57). As we already know that small sample sizes jeopardize the overall accuracy of the ML models, the only solution to generalize and effectively arrive at real-life translation to clinical practice of those promising methods of detection/classification is to collect more data. The only way to go, is to organize large collaborative projects with identical protocols of data collection, similar to STAR*D. Like many things in life we should try to keep it simple: base our research on already successful research based on small samples, but increase the size of a sample; add some form of electrophysiological data and non-linear feature extraction; keep dimensionality of a problem as low as possible; always perform external validation and once we deploy the model developed in lab, keep monitoring its performance. In order to make the research reproducible, we might preregister the protocols and methods, and publish our negative results. Collaborative data sharing (anonimizied data are a good practice but time series required here are already GDPR compliant) can also contribute to the solution of this problem.
With todays' technology that made possible Telehealth &IoT (portable monitoring devices, with medical-grade signal quality), as a reliable way of remote monitoring of outpatients, we can support clinicians with objective additional information that might largely improve the effectiveness of therapy for depression. It might be close to previously envisaged personal medicine, increasing the ability of every clinician to better navigate many diagnostic decisions. Revisiting some not-so-well-known mathematical concepts that can thrive with cloud technology, would pay off in improved psychiatric diagnostics and treatment.
Although the citation is coming from the economy, it effectively applies to the adoption of these innovations in psychiatry: The difficulty lies not in the absence of new ideas, but an escape from the old ones (58).
Author Contributions
MČ envisaged and designed the paper and performed a literature search. MČ and VL wrote the paper, reviewed the paper, and corrected the text. All authors contributed to the article and approved the submitted version.
Funding
This work was partially supported by the grant PID2020-113192GB-I00 (Mathematical Visualization: Foundations, Algorithms, and Applications) from the Spanish MICINN.
Conflict of Interest
MČ was employed by 3EGA B.V.
The remaining author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher's Note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
1. Shorter E, Fink M. Endocrine Psychiatry: Solving the Riddle of Melancholia. Oxford: Oxford University Press (2010).
2. Foersch S, Eskstein M, Wagner DC, Gach F, Woerl AC, Roth W, et al. Deep learning for diagnosis and survival prediction in soft tissue sarcoma. Ann Oncol. (2021) 32:P1178–87. doi: 10.1016/j.annonc.2021.06.007
3. Rajalakshmi R, et al. Automated diabetic retinopathy detection in smartphone-based fundus photography using artificial intelligence. Eye. (2018) 32:1138–44. doi: 10.1038/s41433-018-0064-9
4. El-Sappagh S, et al. A multilayer multimodal detection and prediction model based on explainable artificial intelligence for Alzheimer's disease. Nat Sci Rep. (2021) 11:2660. doi: 10.1038/s41598-021-82098-3
5. Yahata N, Kiyoto K, Kawato M. Computational neuroscience approach to biomarkers and treatments for mental disorders. Psychiatry Clin Neurosci. (2017) 71:215–37. doi: 10.1111/pcn.12502
6. Jiang L, et al. Opportunities and challenges of artificial intelligence in the medical field: current application, emerging problems, problem-solving strategies. J Int Med Res. (2021) 49:1–11. doi: 10.1177/03000605211000157
7. Berisha V, Krantsevich C, Hahn PR, Hahn S, Dasarathy G, Turaga P, et al. Digital medicine and the curse of dimensionality. Nature Digit Med. (2021) 4:153. doi: 10.1038/s41746-021-00521-5
8. The Alan Turing Institute Report. Data science, AI, in the Age of COVID-19; Reflections on the Response of the UK's Data Science, AI. Community to the COVID-19 Pandemic (2021).
9. Cukić M, Stokić M, Radenković S, Ljubisavljević M, Pokrajac D. The shift in brain-state induced by tDCS: an EEG study. In: Cukic M, editor. Book Novel Approaches in Treating Major Depressive Disorder. New York, NY: NOVA Scientific Publishers Ltd. (2019). p. 27–48.
10. Čukić M, López V, Pavón J. Classification of depression through resting-state electroencephalogram as a novel practice in psychiatry. J Med Internet Res. (2020) 22:e19548. doi: 10.2196/19548
11. Čukić M, Stokić M, Simić S, Pokrajac D. The successful discrimination of depression from EEG could be attributed to proper feature extraction and not to a particular classification method. Cogn Neurodyn. (2020) 14:443–55. doi: 10.1007/s11571-020-09581-x
12. Čukić M, Pokrajac D, Lopez V. On mistakes we made in prior computational psychiatry data driven approach projects and how they Jeopardize translation of those findings in clinical practice. In: Proceedings of the 2020 Intelligent Systems Conference (IntelliSys). Amsterdam: Springer Nature (2020). doi: 10.1007/978-3-030-55190-2
13. Llamocca P, López V, Santos M, Cukić M. Personalized characterization of emotional states in patients with bipolar disorder. Mathematics. (2021) 9:1174. doi: 10.3390/math9111174
14. Llamocca P, López V, Cukić M. The proposition for future bipolar depression forecasting based on wearables data collection. Front Physiol. (2021) 12:777137. doi: 10.3389/fphys.2021.777137
15. Whelan R, Garavan H. When optimism hurts: inflated predictions in psychiatric neuroimaging. Biol Psychiatry. (2014) 75:746–8. doi: 10.1016/j.biopsych.2013.05.014
16. Gillan CM, Whelan R. What big data can do for treatment in psychiatry. Curr Opin Behav Sci. (2017) 18:34–42. doi: 10.1016/j.cobeha.2017.07.003
17. Gillan CM, Daw ND. Taking psychiatry research online. Neuron. (2016) 91:19–23. doi: 10.1016/j.neuron.2016.06.002
18. Coley RJ, Boggs JM, Beck A, Simon GE. Predicting outcomes of psychotherapy for depression with electronic health record data. J Affect Disord Rep. (2021) 6:1–18. doi: 10.1016/j.jadr.2021.100198
19. Llamocca PH, Junestrand A, Cukic M, Llopez V, Urgelésv D. Data source analysis in mood disorder research. In: XVIII Conference of the Spanish Association of Artificial Intelligence (CAEPIA 2018). Seville (2018). p. 893–99.
20. Llamocca P, Urgelés D, Cukic M, Lopez V. Bip4Cast: some advances in mood disorders data analysis. In: Proceedings of the 1st International Alan Turing Conference on Decision Support and Recommender Systems. London (2019). p. 5–10.
21. Ng AY. Preventing “overfitting” of cross-validation data. In: Presented at the 14th International Conference on Machine Learning (ICML). (1997).
22. Tibshirani R. Regression shrinkage and selection via the lasso. J R Stat Soc Ser B Methodol. (1996) 58:267–88. doi: 10.1111/j.2517-6161.1996.tb02080.x
23. Efron B, Tibshirani RJ. Improvement on cross- validation. the 632+ bootrstrap method. J Am Stat Assoc. (1997) 92:548–60. doi: 10.1080/01621459.1997.10474007
24. Ross C, Swetliz I. IBM's Watson Supercomputer Recommended 'Unsafe Incorrect' Cancer Treatments, Internal Document Show. Stat News (2018). Available online at: https://www.statnews.com/2018/07/25/ibm-watson-recommended-unsafe-incorrect-treatments/
25. Greco A, Messerotti Benvenuti S, Gentili C, Palomba D, Scilingo EP, Valenza G. Assessment of linear and nonlinear/complex heartbeat dynamics in subclinical depression (dysphoria). Physiol Meas. (2017) 39:034004. doi: 10.1088/1361-6579/aaaeac
26. Vabalas A, Gowen E, Poliakoff E, Casson AJ. Machine learning algorithm validation with a limited sample size. PLos ONE. (2019) 7:e0224365. doi: 10.1371/journal.pone.0224365
27. Arbabshirani MR, Plis S, Sui J, Calhoun VD. Single subject of brain disorders in neuroimaging: promises and pitfalls. Neuroimage. (2017) 145:137–65. doi: 10.1016/j.neuroimage.2016.02.079
28. Andaur Navarro CL, et al. Risk of bias in studies on prediction models developed using supervised machine learning techniques: systematic review. BMJ. (2021) 375:n2281. doi: 10.1136/bmj.n2281
29. Jaworska N, et al. Leveraging machine learning approaches for predicting antidepressant treatment response using electroencephalography (EEG) and clinical data. Front Psychiatry. (2019) 9:768. doi: 10.3389/fpsyt.2018.00768
30. Lebiecka K, Zuchowicz U, Wozniak-Kwasniewska A, Szekely D, Olejarczyk E, David O. Complexity analysis of EEG data in persons with depression subjected to transcranial magnetic stimulation. Front Physiol. (2018) 9:1385. doi: 10.3389/fphys.2018.01385
31. Antal A, Alekseichuk I, Bikson M, Brockmöller J, Brunoni AR, Chen R, et al. Low intensity transcranial electric stimulation: safety, ethical, legal regulatory and application guidelines. Clin Neurophysiol. (2017) 128:1774–809. doi: 10.1016/j.clinph.2017.06.001
32. Cukić M. The reason why rTMS and tDCS are efficient in treatments of depression. Front. Psychol. (2020) 10:2923. doi: 10.3389/fpsyg.2019.02923
33. de Kwaasteniet B, Ruhe E, Caan M, Rive M, Olabarriaga S, Groefsema M. Relation between structural and functional connectivity in major depressive disorder. Biol Psychiat. (2013) 74:40–7. doi: 10.1016/j.biopsych.2012.12.024
34. Ahmadlou M, Adeli H, Adeli A. Fractal analysis of frontal brain in major depressive disorder. Int J Psychophysiol. (2012) 8:206–11. doi: 10.1016/j.ijpsycho.2012.05.001
35. Bachmann M, Lass J, Suhhova A, Hinrikus H. Spectral asymmetry and Higuchi's fractal dimension of depression electroencephalogram. Comput Math Methods Med. (2013) 2013:31. doi: 10.1155/2013/251638
36. Eke A, Herman P, Kocsis L, Kozak LR. Fractal characterization of complexity in temporal physiological signals. Physiol Meas. (2002) 23:R1–38. doi: 10.1088/0967-3334/23/1/201
37. Pincus SM, Goldberger AL. Physiological time-series analysis: what does regularity quantify? Am J Physiol Heart Circ Physiol. (1994) 266:H1643–56. doi: 10.1152/ajpheart.1994.266.4.H1643
38. Klonowski W. Chaotic dynamics applied to signal complexity in phase space and in time domain. Chaos Solitons Fractals. (2002) 14:1379. doi: 10.1016/S0960-0779(02)00056-5
39. Klonowski W. From conformons to human brains: an informal overview of nonlinear dynamics and its applications in biomedicine. Nonlinear Biomed Phys. (2007) 1:5. doi: 10.1186/1753-4631-1-5
40. Peng CK, Hausdorff J, Goldberger A. Fractal mechanisms in neural control: Human heartbeat gait dynamics in health disease. In: Walleczek J., editor. Self-Organized Biological Dynamics and Nonlinear Control. Cambridge: Cambridge University Press (1999) 66–97. doi: 10.1017/CBO9780511535338.006
41. Goldberger AL, Peng CK, Lipsitz LA. What is physiologic complexity and how does it change with aging and disease? Neurobiol Aging. (2002) 23:23–6. doi: 10.1016/S0197-4580(01)00266-4
42. Pincus SM. Quantitative assessment strategies and issues for mood and other psychiatric serial study data. Bipolar Disord. (2003) 5:287–94. doi: 10.1034/j.1399-5618.2003.00036.x
43. Mandelbrot BB. The Fractal geometry of Nature. Echo Point Books and Media, Battleboro. Vermont (1977).
44. Rottenberg J. Cardiac vagal control in depression: a critical analysis. Biol Psychol. (2007) 74:200–11. doi: 10.1016/j.biopsycho.2005.08.010
45. Willner P, Scheel-Krüger J, Belzung C. The neurobiology of depression and antidepressant action. Neurosci Biobehav Rev. (2013) 37:2331–71. doi: 10.1016/j.neubiorev.2012.12.007
46. Van der Kolk B. The Body Keeps the Score: Mind, Brain Body in the Transformation of Trauma. New York, NY: Penguin UK (2014).
47. Kemp AH, Quintana DS, Gray MA, Felmingham KL, Brown K, Gatt JM. Impact of depression and antidepressant treatment on heart rate variability: a review and meta-analysis. Biol Psychiatry. (2010) 67:1067–74. doi: 10.1016/j.biopsych.2009.12.012
48. Kemp AH, Quintana DS, Felmingham KL, Matthews S, Jelinek HF. Depression, comorbid anxiety disorders, and heart rate variability in physically healthy, unmedicated patients: implications for cardiovascular risk. PLoS ONE. (2012) 7:e30777. doi: 10.1371/journal.pone.0030777
49. Kemp AH, Quintana DS, Quinn CR, Hopkinson P, Harris AW. Major depressive disorder with melancholia displays robust alterations in resting state heart rate and its variability: implications for future morbidity and mortality. Front Psychol. (2014) 5:1387. doi: 10.3389/fpsyg.2014.01387
50. Jelinek HF, Khandoker AH, Quintana DS, Imam MH, Kemp H. Complex Correlation Measure as a sensitive indicator of risk for sudden cardiac death in patients with depression. Comput Cardiol. (2011) 38:809–12.
51. Kop WJ, Stein PK, Tracy RP, Barzilay JI, Schulz R, Gottdiener JS. Autonomic nervous system dysfunction and inflammation contribute to the increased cardiovascular mortality risk associated with depression. Psychosom Med. (2010) 72:626–35. doi: 10.1097/PSY.0b013e3181eadd2b
52. Cukić M, Savić D, Sidorova J. When heart beats differently in depression: a review of nonlinear HRV measures. Biocybernet Biomed Eng. (2021).
53. Hosseinifard B, Moradi MH, Rostami R. Classifying depression patients and normal subjects using machine learning techniques and nonlinear features from EEG signal. Comput Methods Programs Biomed. (2013) 109:339–345.
54. Bachmann M, Päeske L, Kalev K, Aarma K, Lehtmets A, Ööpik P, et al. Methods for classifying depression in single channel EEG using linear and nonlinear signal analysis. Comput Methods Programs Biomed. (2018) 155:11–17.
55. Khodayari-Rostamabad A, Reillya JP, Hasey GM, de Bruin H, MacCrimmon DJ. A machine learning approach using EEG data to predict response to SSRI treatment for major depressive disorder. Clin Neurophysiol. (2013) 124:1975–85. doi: 10.1016/j.clinph.2013.04.010
56. Chang H-A, Chang C-C, Kuo TBJ, Huang SY. Distinguishing bipolar II depression from unipolar major depressive disorder: differences in heart rate variability. World J Biol Psychiatry. (2015) 16:351–60. doi: 10.3109/15622975.2015.1017606
57. Khandoker AH, Luthra V, Abouallaban Y, Saha S, Ahmed KI, Mostafa R, et al. Predicting depressed patients with suicidal ideation from ECG recordings. Med Biol Eng Comput. (2017) 55:793–805. doi: 10.1007/s11517-016-1557-y
58. Keynes JM. The General Theory of Employment, Interest and Money. (1935). Available online at: https://www.files.ethz.ch/isn/125515/1366_KeynesTheoryofEmployment.pdf
Keywords: detection, monitoring, depression, machine learning, physiological complexity
Citation: Čukić M and López V (2022) Progress in Objective Detection of Depression and Online Monitoring of Patients Based on Physiological Complexity. Front. Psychiatry 13:828773. doi: 10.3389/fpsyt.2022.828773
Received: 03 December 2021; Accepted: 02 March 2022;
Published: 28 March 2022.
Edited by:
Heleen Riper, VU Amsterdam, NetherlandsReviewed by:
Lei Feng, Capital Medical University, ChinaZhaocheng Huang, University of New South Wales, Australia
Copyright © 2022 Čukić and López. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Milena Čukić, bWljdWtpYyYjeDAwMDQwO3VjbS5lcw==