A prospective study of remote delirium screening using the modified K-4AT for COVID-19 inpatients

Background: Delirium is a neuropsychiatric condition strongly associated with poor clinical outcomes such as high mortality and long hospitalization. In the patients with Coronavirus disease 2019 (COVID-19), delirium is common and it is considered as one of the risk factors for mortality. For those admitted to negative-pressure isolation units, a reliable, validated and contact-free delirium screening tool is required.

Materials and methods: We prospectively recruited eligible patients from multiple medical centers in South Korea. Delirium was evaluated using the Confusion Assessment Method (CAM) and 4‘A’s Test (4AT). The attentional component of the 4AT was modified such that respondents are required to count days, rather than months, backward in Korean. Blinded medical staff evaluated all patients and determined whether their symptoms met the delirium criteria of the Diagnostic and Statistical Manual of Mental Disorders 5 (DSM-5). An independent population of COVID-19 patients was used to validate the 4AT as a remote delirium screening tool. We calculated the area under the receiver operating characteristic curve (AUC).

Results: Out of 286 general inpatients, 28 (9.8%) inpatients had delirium. In this population, the patients with delirium were significantly older (p = 0.018) than the patients without delirium, and higher proportion of males were included in the delirium group (p < 0.001). The AUC of the 4AT was 0.992 [95% confidence interval (CI) 0.983–1.000] and the optimal cutoff was at 3. Of the independent COVID-19 patients, 13 of 108 (12.0%) had delirium. Demographically, the COVID-19 patients who had delirium only differed in employment status (p = 0.047) from the COVID-19 patients who did not have delirium. The AUC for remote screening using the 4AT was 0.996 (0.989–1.000). The optimal cutoff of this population was also at 3.

Conclusion: The modified K-4AT had acceptable reliability and validity when used to screen inpatients for delirium. More importantly, the 4AT efficiently screened for delirium during remote evaluations of COVID-19 patients, and the optimal cutoff was 3. The protocol presented herein can be used for remote screening of delirium using the 4AT.

Introduction

During the Coronavirus disease 2019 (COVID-19) pandemic, the prevalence of acute mental change or delirium in COVID-19 patients ranged from 10% to 70% (1–4). Delirium, defined as fluctuating cognitive disturbance, in critically ill patients is strongly associated with poor outcomes, and a recent meta-analysis confirmed that delirium in COVID-19 patients was significantly associated with high mortality (5). As delirium is associated with long-term hospitalization, high medical costs, and high mortality (6), close monitoring of COVID-19 patients for early delirium detection is crucial. Although the prevalence of delirium in COVID-19 patients has been reported in many studies (1–4), some of which highlighted the importance of delirium screening (5), few studies have specifically explored how to assess features of delirium. The limited patient contact, shortage of trained healthcare professionals, and restrictions on family visits were speculated as barriers to routine assessment and early detection of delirium (7). Thus, a detailed method for delirium screening in COVID-19 patients is essential; any such method must consider COVID-19 patient-specific circumstances, such as isolation in negative pressure units.

The 4 ‘A’s test (4AT) is a delirium screening tool (8). Assessment using the 4AT is brief and simple; clinical experience is not required. In contrast, the Confusion Assessment Method (CAM), which has been widely used and validated in many languages (9), is relatively complex and must be administered by a skilled professional. Moreover, as 4AT delirium screening does not require physical contact with the patient, using 4AT as a delirium assessment tool for COVID-19 patients could be useful. However, the 4AT has not yet been validated for any patient groups in South Korea, and the attention component requires modification during the translation from the English to Korean version (K-4AT), because the two task versions differ in terms of difficulty. Thus, in this study, we first validated the K-4AT in general inpatients. Then, we focused on COVID-19 inpatients who required remote screening. Finally, we present a contact-free delirium screening protocol for COVID-19 patients.

Materials and methods

Subjects

In order to evaluate the reliability and validity of the modified K-4AT, adult general inpatients were recruited from the intensive care unit, postoperative unit or progressive cancer unit of seven South Korean medical centers. The recruitment period was from March to December 2021. Patients who were unable to communicate verbally, or who were diagnosed with dementia or cognitive impairment, were excluded. Patients were recruited only when informed consent could be obtained from them or their family members. The Institutional Review Board of each center approved this part of the study. Patients with COVID-19 diagnoses were independently recruited on admission to the Seoul National University Bundang Hospital or Seoul National University Hospital for treatment of COVID-19-related symptoms. Recruitment and screening proceeded via two steps. First, as all patients were isolated in COVID-19 units, verbal consents were obtained via interphones or their cellphones after being informed about the study. Delirium assessments were then performed. The detailed flowchart of remote delirium assessment by 4AT was elaborated in Figure 1. In the next step, patients who had previously given verbal consent were discharged and thus able to meet researchers face-to-face. We obtained written consent at this time. The protocol was approved by the Institutional Review Boards of both centers (joint approval no. 2103-675-302).

FIGURE 1

Figure 1. Protocol of delirium screening by 4‘A’s Test (4AT) for Coronavirus disease 2019 (COVID-19) inpatients. 4AT, 4‘A’s Test; COVID-19, Coronavirus disease 2019.

Assessments

Researchers assessed the clinical features of all recruited general inpatients and COVID-19 patients to determine whether they were eligible for the study. For eligible patients who gave informed consent, age, sex, educational level, marital status, comorbidities including the current diagnosis, and diagnostic history data were recorded. Two independent researchers of each medical center evaluated delirium using the CAM and 4AT, respectively (10, 11). Before the commencement of this study, in order for all researchers and health care professionals to administer CAM, multiple workshops and training sessions for delirium evaluation by CAM and CAM-ICU were held. In workshops and sessions, board certified psychiatrists shared case examples and studied actual evaluations through videotaped records. The CAM is a validated and widely used tool; many versions are available. We employed the CAM-ICU, which includes the Richmond Agitation and Sedation Scale (RASS) and also assesses acute onset, inattention, fluctuating cognition, and disorganized thinking (12). The final decision by CAM is either delirium positive or delirium negative. When acute onset and inattention features are positive and either fluctuating cognition or disorganized thinking is positive, the patient is determined as delirium present. Although the 4AT has similar features with CAM, the 4AT is designed as scoring system. The 4AT includes four features. The first feature evaluates alertness by observation. The second and the third features assess cognition and attention via brief cognitive tasks, which are modules of the Abbreviated Mental Test 4 (AMT4) (8). The last feature evaluate the acute change or fluctuating course of mental status within last 24 h. The total score is calculated by adding up scores of each feature and the score range is minimum 0 to maximum 12. The original English version of 4AT has cutoff at 4, meaning that when the total score is greater than 3, the patient is considered as delirium present. To ensure that the English and Korean versions of the tasks were equally challenging, the attention component was modified such that the participants had to count days backwards, rather than months, as counting months in Korean is the same as reciting numbers from 12 to 1. Apart from these assessments, medical staffs including doctors and nurses blindly evaluated features of delirium on the assessment days. These bedside evaluations involved patient contact and were based on the Diagnostic and Statistical Manual of Mental Disorders 5 (DSM-5) delirium criteria (13). In order to make the diagnosis of delirium based on DSM-5, inpatient physician, nurses, and consultation-liaison psychiatrists of each center obtained information from family members regarding patients’ mental change or fluctuation and also reviewed medical charts. Additionally, the direct cognitive tasks from the Korean version of Mini-Mental State Exam are asked to the patients, such as registration and recall of words, orientation and serial sevens (14).

Statistical analysis

We compared the demographic data of the patients with and without delirium using the Pearson χ2 test and Fisher’s exact test for categorical variables, and the t-test for continuous variables. We calculated the Cronbach’s alpha as a measure of reliability of the modified K-4AT, and generated receiver operating characteristic (ROC) curves to analyze validity. In this analysis, the DSM-5 based diagnoses were set as a true value. Sensitivity, specificity, and accuracy were calculated at various cutoffs; the optimal cutoff could be determined at the point of maximized sensitivity and specificity. For the COVID-19 population, the same analyses were performed to determine the reliability and validity of remote 4AT assessment, and the effectiveness of CAM and 4AT. We used R (pROC ver. 1.17.0; Microsoft R Open Version 3.6.2. Microsoft and R Core Team, Microsoft Corporation, Redmont, WA, United States) for the analyses; p-values < 0.05 were considered significant. Interrater reliability was determined according to the extent of the agreement in delirium diagnoses based on DSM-5 criteria between the two raters. Cohen’s kappa showed that the agreement was substantially consistent [κ = 0.710; 95% confidence interval (CI) = 0.412–1.000; p < 0.001].

Results

Hospitalized general patients and contact delirium screening

Of 286 general inpatients, 28 (9.8%) had delirium diagnosis based on DSM-5 and 258 (90.2%) did not. Demographically, the two groups differed significantly in age and gender only (Table 1). Patients with delirium were significantly older (mean age = 73.21 years, standard deviation = 10.56 years) than those without delirium (p = 0.018), and there was a higher proportion of males in the delirium group (p < 0.001). There were significant group differences according to diagnoses (p < 0.001). The Cronbach α of the 4AT was 0.786, indicating high internal reliability. The optimal cutoff for the K-4AT was 3, at which the sensitivity was 1.000 (95% CI = 1.000-1.000), the specificity was 0.980 (95% CI = 0.960–0.996), and the accuracy was 0.982 (95% CI = 0.964–0.996) (Table 2). The area under the ROC curve (AUC) was 0.992 (95% CI = 0.983–1.000). On the other hand, the sensitivity of CAM was 0.643 (95% CI = 0.464-0.821), the specificity was 0.981 (95% CI = 0.961-0.996), and the accuracy was 0.948 (95% CI = 0.923–0.969). The AUC was 0.812 (95% CI = 0.721–0.903), which was lower than AUC of 4AT.

TABLE 1

Table 1. Demographics between delirious and non-delirious patients diagnosed by the Diagnostic and Statistical Manual of Mental Disorders 5 (DSM-5) in general inpatient population.

TABLE 2

Table 2. Sensitivity, specificity, and accuracy of the 4‘A’s Test (4AT) and the Confusion Assessment Methods (CAM) in general inpatient population (n = 286).

Hospitalized COVID-19 patients and remote delirium screening

Out of 108 COVID-19 inpatients, 13 (12.0%) had delirium diagnosis based on DSM-5 and 95 (88.0%) did not. Demographically, the two groups differed in employment status only (p = 0.047) and there were no significant differences in age, gender, education level, and marital status (Table 3). The Cronbach’s α of the remotely conducted K-4AT was 0.810, indicating high internal consistency. The AUC of remote K-4AT was 0.996 (95% CI = 0.989–1.000) (Figure 2). The optimal cutoff of remote assessment by K-4AT was 3. At this point, the sensitivity was 1.000 (95% CI = 1.000–1.000), the specificity was 0.978 (95% CI = 0.944–1.000), and the accuracy was 0.980 (95% CI = 0.951–1.000). The AUC of the remote CAM was 0.885 (95% CI = 0.765-1.000); the sensitivity was 0.769 (95% CI = 0.539–1.000), the specificity was 1.000 (95% CI = 1.000–1000), and the accuracy was 0.972 (95% CI = 0.944–1000). Thus, for COVID-19 inpatients, remote delirium screening by K-4AT yielded more reliable results than by CAM.

TABLE 3

Table 3. Demographics between delirious and non-delirious patients diagnosed by the Diagnostic and Statistical Manual of Mental Disorders 5 (DSM-5) in Coronavirus disease 2019 (COVID-19) inpatients.

FIGURE 2

Figure 2. Receiver operating characteristics (ROC) analysis was performed to compare the 4‘A’s Test (4AT) with the Confusion Assessment Methods (CAM) in Coronavirus disease 2019 (COVID-19) inpatients. ROC, Receiver operating characteristics; 4AT, 4‘A’s Test; CAM, Confusion Assessment Methods; COVID-19, Coronavirus disease 2019.

Discussion

As the 4AT is a brief assessment (< than 3 min) that can be administered by non-professionals (8), we used the K-4AT as an adequate delirium screening tool for a prospective multi-center study. Our study findings showed that the K-4AT proved to be a valid screening tool. Among the general inpatients, 9.8% had delirium. In agreement with earlier studies (15, 16), we found significant differences in age, sex, and diagnoses between the delirious and non-delirious patients. By including patients with three different principal diagnoses, we confirmed the validity and generality of delirium screening by K-4AT. The original 4AT has a cutoff of 4; however, we found that the optimal cutoff was 3. Similarly, a recent study of the Spanish version of the 4AT indicated that the maximized combination of sensitivity and specificity was at a cutoff of 3 (17). These results implied that the modified and translated version of 4AT may be better at detecting delirium when it has strict cutoff.

Of the COVID-19 inpatients admitted to negative pressure isolation units, 12% developed delirium; this rate is similar to those of previous studies (1–4), which can imply that delirium is not uncommon in COVID-19 patients. The recent international cohort study reported that more than half of the COVID-19 patients from the cohort had delirium and more than 80% of them underwent comma status during hospitalization (18). As these COVID-19 patients with delirium are exposed to risk of poor clinical outcomes (5, 6), early detection and prevention is critical. A recent meta-analysis revealed that symptoms of delirium, especially confusion and agitation, were common among patients in the acute stage of severe acute respiratory syndrome (SARS), Middle East respiratory syndrome (MERS), and COVID-19 (19). When dealing with an infectious disease outbreak, control of the spread is crucial but difficult, depending on whether isolation or quarantine be mandated and how long the asymptomatic period is (20). Isolation and quarantine that are sudden environmental changes for patients may be risk factors for delirium onset. An observational study from the early COVID-19 era reported that the mortality of COVID-19 patients with delirium was 10% higher than that of COVID-19 patients without delirium (21); another study reported that COVID-19 patients with delirium were at great risk of 30-day mortality and readmission after discharge (22). Thus, to avoid exacerbation of the clinical symptoms of infectious diseases, early detection and treatment of delirium are imperative. This implied that delirium screening should be routine in patients with respiratory infectious diseases. However, for isolated patients with highly contagious respiratory infections (SARS, MERS, and COVID-19), delirium evaluation on a routine basis is difficult. Studies regarding delirium screening for these patients are still limited.

Thus, we used the K-4AT and remotely assessed COVID-19 inpatients; as with the face-to-face setting, the test was valid and useful, and the cutoff score was 3. In the course of delirium assessment, we found 4AT highly applicable for a remote delirium screening tool over CAM in that the raters require minimal training and K-4AT includes indirect observation and direct cognitive task. These components of K-4AT helps raters evaluate alertness and acute changes even when patients were in a state that direct cognitive tasks cannot be proceeded. Additionally, K-4AT assesses delirious features of patients within the past 24 h, while CAM only evaluates patients’ delirious symptoms at the point of assessment. This has been already recognized as a major weakness of the CAM.

Previous studies have proposed telemedicine for patients with limited access to mental healthcare (i.e., those living in rural areas, attending large schools, or incarcerated in prison), and interactive devices allow diagnosis, tele-psychotherapy, and tele-counseling to be performed by mental health professionals including psychiatrists, psychologists, and physicians (23). The remote delirium evaluation by K-4AT is analogous to telemedicine. In the COVID-19 era, remote assessments and interventions protected both patients and caregivers (24). Recent studies have used remote physical examinations for both the primary care and cognitive assessment of COVID-19 patients (24–26) and a recent mobile platform assesses depression, suicidal ideation, and anxiety (27). However, very few studies have discussed remote screening of neuropsychiatric conditions that are under-recognized such as delirium. Such conditions exert a major influence on the prognosis of COVID-19 and other infectious diseases, and must be tackled early to reduce the risk of complications. Our protocol was written for medical staff but could be modified for use by COVID-19 caregivers, who typically lack medical training. A few studies emphasized that caregivers can detect delirium, and provided guidelines for both general patients (28, 29) and COVID-19 patients (30). Future studies should extend the remote delirium screening protocol to caregivers of COVID-19 patients receiving remote telehealth therapy.

We only verified the inter-rater reliability of COVID-19 delirium diagnoses made by independent medical staff based on the DSM-5 criteria. Although the assessments were not remote, patient contact was limited as the staff were wearing protective clothing and could thus spend little time with each patient. This could have resulted in biased diagnoses. However, the inter-rater reliability of the DSM-5 diagnoses was excellent (κ = 0.710; 95% CI = 0.412–1.000; p < 0.001). The K-4AT includes direct measures (i.e., cognitive tasks), re-evaluations by two raters might lead to a practice effect. Moreover, the results from DSM-5-based diagnoses and remote K-4AT evaluations were in agreement, verifying the inter-rater reliability of the K-4AT may not be necessary (31).

This study had some limitations. First, we enrolled general inpatients with various diagnoses in three different settings. The aim was to optimize generalizability, but this may have caused some bias. Small sample size was another limitation of our study, particularly that of the COVID-19 population. Additionally, we had to exclude inpatients’ data for final analysis because they were in a critical condition or ineligible for the study for other reasons, which is a major limitation of most prospective studies. The exclusion of patients with dementia or cognitive impairment may also have created bias. However, to the best of our knowledge, this is the first prospective study to validate the 4AT specifically for remote assessment of delirium in COVID-19 inpatients. Future studies with larger samples should assess delirium both in COVID-19 and uninfected patients with various forms of cognitive impairment. Our remote delirium screening protocol will require further development to that end. Finally, the inclusion of patients with hearing disabilities might lead to false-positives.

In summary, we validated the modified version of the K-4AT, which accurately detected delirium in COVID-19 and general inpatients. Specifically, for COVID-19 patients who require remote evaluation, the K-4AT reliably screened for delirium; the best combination of sensitivity and specificity was obtained at 3, suggesting the optimal cutoff of 3. Finally, we provided a protocol for remote screening of delirium in COVID-19 patients using the K-4AT.

Data availability statement

The datasets generated and/or analyzed during the current study are not publicly available because the datasets include personally identifiable medical information of all participants, but are available from the corresponding author on reasonable request.

Ethics statement

The studies involving human participants were reviewed and approved by Institutional Review Board, Seoul National University Bundang Hospital. The patients/participants provided their written informed consent to participate in this study.

Author contributions

HH and YJ contributed to the analysis and interpretation of data, and drafted this manuscript. HoP, YL, DR, BK, and K-HS contributed to the conceptualization and data curation for this research. HuP contributed to the funding acquisition, conceptualization, data curation, and preparation or the original draft. All authors have read and approved the submission of this manuscript.

Funding

This research was supported by a grant of the Korea Health Technology R&D Project through the Korea Health Industry Development Institute (KHIDI) and funded by the Ministry of Health & Welfare, Republic of Korea (grant numbers: HI19C0481 and HC20C0040).

Acknowledgments

We are grateful for support by Korea Health Technology R&D Project through the Korea Health Industry Development Institute (KHIDI).

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. Ticinesi A, Cerundolo N, Parise A, Gold G, Graf CE, Zekry D. Delirium in COVID-19: epidemiology and clinical correlations in a large group of patients admitted to an academic hospital. Aging Clin Exp Res. (2020) 32:2159–66.

Google Scholar

2. Helms J, Kremer S, Merdji H, Clere-Jehl R, Schenck M, Kummerlen C, et al. Neurologic features in severe SARS-COV-2 infection. New Engl J Med. (2020) 382:2268–70. doi: 10.1056/nejmc2008597

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Mendes A, Herrmann FR, Périvier S, Gold G, Graf CE, Zekry D. Delirium in older patients with COVID-19: prevalence, risk factors, and clinical relevance. J Gerontol Ser A. (2021) 76:e142–6. doi: 10.1093/gerona/glab039

PubMed Abstract | CrossRef Full Text | Google Scholar

4. O’Hanlon S, Inouye SK. Delirium: a missing piece in the COVID-19 pandemic puzzle. Age Ageing. (2020) 49:497–8. doi: 10.1093/ageing/afaa094

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Pranata R, Huang I, Lim MA, Yonas E, Vania R, Kuswardhani RA. Delirium and mortality in coronavirus disease 2019 (covid-19) – a systematic review and meta-analysis. Arch Gerontol Geriatr. (2021) 95:104388. doi: 10.1016/j.archger.2021.104388

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Inouye SK, Rushing JT, Foreman MD, Palmer RM, Pompei P. Does delirium contribute to poor hospital outcomes? J Gen Int Med. (1998) 13:234–42. doi: 10.1046/j.1525-1497.1998.00073.x

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Duggan MC, Van J, Ely EW. Delirium assessment in critically ill older adults. Crit Care Clin. (2021) 37:175–90. doi: 10.1016/j.ccc.2020.08.009

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Tieges Z, Maclullich AM, Anand A, Brookes C, Cassarino M, O’connor M, et al. Diagnostic accuracy of the 4at for delirium detection in older adults: systematic review and meta-analysis. Age Ageing. (2020) 50:733–43. doi: 10.1093/ageing/afaa224

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Bellelli G, Morandi A, Davis DH, Mazzola P, Turco R, Gentile S, et al. Validation of the 4AT, a new instrument for rapid delirium screening: a study in 234 hospitalised older people. Age Ageing. (2014) 43:496–502. doi: 10.1093/ageing/afu021

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Inouye SK, van Dyck CH, Alessi CA, Balkin S, Siegal AP, Horwitz RI. Clarifying confusion: the confusion assessment method. new method for detection of delirium. Ann Intern Med. (1990) 113:941–8.

Google Scholar

11. 4AT - RAPID CLINICAL TEST FOR DELIRIUM. 4AT - Rapid Clinical Test For Delirium (2011). Available online at: https://www.the4at.com (accessed July 22, 2022).

Google Scholar

12. Ely EW, Inouye SK, Bernard GR, Gordon S, Francis J, May L, et al. Delirium in mechanically ventilated patients: validity and reliability of the confusion assessment method for the intensive Care Unit (CAM-ICU). JAMA. (2001) 286:2703–10. doi: 10.1001/jama.286.21.2703

PubMed Abstract | CrossRef Full Text | Google Scholar

13. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders: DSM-5. 5th ed. Arlington, VA: American Psychiatric Publishing (2013).

Google Scholar

14. Han C, Jo SA, Jo I, Kim E, Park MH, Kang Y. An adaptation of the Korean mini-mental state examination (K-MMSE) in elderly Koreans: demographic influence and population-based norms (the AGE study). Arch Gerontol Geriatr. (2008) 47:302–10. doi: 10.1016/j.archger.2007.08.012

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Lee HB, Mears SC, Rosenberg PB, Leoutsakos J-MS, Gottschalk A, Sieber FE. Predisposing factors for postoperative delirium after hip fracture repair in individuals with and without dementia. J Am Geriatr Soc. (2011) 59:2306–13. doi: 10.1111/j.1532-5415.2011.03725.x

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Oh ES, Sieber FE, Leoutsakos J-M, Inouye SK, Lee HB. Sex differences in hip fracture surgery: preoperative risk factors for delirium and postoperative outcomes. J Am Geriatr Soc. (2016) 64:1616–21. doi: 10.1111/jgs.14243

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Sepúlveda E, Bermúdez E, González D, Cotino P, Viñuelas E, Palma J, et al. Validation of the delirium diagnostic tool-provisional (DDT-pro) in a skilled nursing facility and comparison to the 4 ‘A’s test (4AT). Gen Hosp Psychiatry. (2021) 70:116–23. doi: 10.1016/j.genhosppsych.2021.03.010

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Pun BT, Badenes R, La Calle GH, Orun OM, Chen W, Raman R, et al. Prevalence and risk factors for delirium in critically ill patients with COVID-19 (COVID-D): a multicentre cohort study. Lancet Respir Med. (2021) 9:239–50. doi: 10.1016/S2213-2600(20)30552-X

CrossRef Full Text | Google Scholar

19. Rogers JP, Chesney E, Oliver D, Pollak TA, McGuire P, Fusar-Poli P, et al. Psychiatric and neuropsychiatric presentations associated with severe coronavirus infections: a systematic review and meta-analysis with comparison to the COVID-19 pandemic. Lancet Psychiatry. (2020) 7:611–27. doi: 10.1016/s2215-0366(20)30203-0

CrossRef Full Text | Google Scholar

20. Day T, Park A, Madras N, Gumel A, Wu J. When is quarantine a useful control strategy for emerging infectious diseases? Am J Epidemiol. (2006) 163:479–85. doi: 10.1093/aje/kwj056

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Khan SH, Lindroth H, Perkins AJ, Jamil Y, Wang S, Roberts S, et al. Delirium incidence, duration, and severity in critically ill patients with coronavirus disease 2019. Crit Care Exp. (2020) 2:e0290. doi: 10.1097/cce.0000000000000290

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Pagali S, Fu S, Lindroth H, Sohn S, Burton MC, Lapid M. Delirium occurrence and association with outcomes in hospitalized COVID-19 patients. Int Psychogeriatr. (2021) 33:1105–9. doi: 10.1017/s104161022100106x

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Grady B, Myers KM, Nelson EL, Belz N, Bennett L, Carnahan L, et al. Evidence-based practice for telemental health. Telemed J E-Health. (2011) 17:131–48.

Google Scholar

24. Geddes MR, O’Connell ME, Fisk JD, Gauthier S, Camicioli R, Ismail Z. Remote cognitive and behavioral assessment: report of the Alzheimer society of canada task force on dementia care best practices for Covid-19. Alzheimer’s Dementia. (2020) 12:e12111. doi: 10.1002/dad2.12111

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Greenhalgh T, Koh GC, Car J. Covid-19: a remote assessment in primary care. BMJ. (2020) 368:m1182. doi: 10.1136/bmj.m1182

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Thompson LI, Harrington KD, Roque N, Strenger J, Correia S, Jones RN, et al. A highly feasible, reliable, and fully remote protocol for mobile app-based Cognitive Assessment in cognitively healthy older adults. Alzheimer’s Dementia. (2022) 14:e12283. doi: 10.1002/dad2.12283

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Pardes A, Lynch W, Miclette M, McGeoch E, Daly BP. Use of a mobile health (mhealth) platform for remote assessment of suicidal ideation, depression, and anxiety: a longitudinal retrospective study. Innov Digital Health Diagn Biomark. (2021) 2:8–15. doi: 10.36401/iddb-21-03

CrossRef Full Text | Google Scholar

28. Davidson JE, Aslakson RA, Long AC, Puntillo KA, Kross EK, Hart J, et al. Guidelines for family-centered care in the neonatal, pediatric, and adult ICU. Crit care Med. (2017) 45:103–28.

Google Scholar

29. Rosgen B, Krewulak K, Demiantschuk D, Ely EW, Davidson JE, Stelfox HT, et al. Validation of caregiver-centered delirium detection tools: a systematic review. J Am Geriatr Soc. (2018) 66:1218–25. doi: 10.1111/jgs.15362

PubMed Abstract | CrossRef Full Text | Google Scholar

30. LaHue SC, James TC, Newman JC, Esmaili AM, Ormseth CH, Ely EW. Collaborative delirium prevention in the age of COVID-19. J Am Geriatr Soc. (2020) 68:947–9. doi: 10.1111/jgs.16480

PubMed Abstract | CrossRef Full Text | Google Scholar

31. De J, Wand APF, Smerdely PI, Hunt GE. Validating the 4A’s test in screening for delirium in a culturally diverse geriatric inpatient population. Int J Geriatr Psychiatry. (2016) 32:1322–9. doi: 10.1002/gps.4615

PubMed Abstract | CrossRef Full Text | Google Scholar