Age-Related Performance in Using a Fully Immersive and Automated Virtual Reality System to Assess Cognitive Function

Introduction: Cognition generally declines gradually over time due to progressive degeneration of the brain, leading to dementia and eventual loss of independent functions. The rate of regression varies among the six cognitive domains (perceptual motor, executive function, complex attention, learning and memory, social cognition and language). Current modality of cognitive assessment using neuropsychological paper-and-pencil screening tools for cognitive impairment such as the Montreal Cognitive Assessment (MoCA) has limitations and is influenced by age. Virtual reality (VR) is considered as a potential alternative tool to assess cognition. A novel, fully immersive automated VR system (Cognitive Assessment using Virtual Reality, CAVIRE) has been developed to assess the six cognitive domains. As cognition is associated with age, VR performance is postulated to vary with age using this system.

Aims: This is a feasibility study to evaluate the VR performance of cognitively healthy adults aged between 35 and 74 years old, based on the performance score and completion time using the CAVIRE system.

Methods: Conducted in a public primary care clinic in Singapore, 25 multi-ethnic Asian adults were recruited in each of the four age groups in years: (1) 35–44; (2) 45–54; (3) 55–64, and (4) 65–74. The eligibility criteria included a MoCA score of 26 or higher to reflect normal cognition and understanding English instructions. They completed common daily activities ranging from brushing teething to shopping, across 13 VR segments. Their performances scores and completion time were automatically computed by the CAVIRE system. These VR performance indices were compared across the four age groups using one-way ANOVA, F-test of the hypothesis, followed by pair-wise comparisons in the event of a significant F-test (p < 0.05).

Results: One participant dropped out from Group 1. The demographic characteristics of 99 participants were similar across the 4 age groups. Overall, younger participants in Groups 1 and 2 attained higher VR performance scores and shorter completion time, compared to those in Groups 3 and 4, in all six cognitive domains (all p < 0.05).

Conclusion: The CAVIRE VR performance scores and completion time significantly differ between the younger and older Asian participants with normal cognition. Enhancements to the system are needed to establish the age-group specific normal performance indices.

Introduction

Dementia is prevalent globally. According to the World Health Organisation, approximately 50 million people worldwide are affected by dementia in 2019, and this figure is projected to triple to 152 million by 2050 (World Health Organisation, 2019).

Being a progressive neurodegenerative disorder, dementia involves gradual cognitive decline over time, leading to eventual loss of independent functions. The decline varies across the six cognitive domains. According to the Diagnostic and Statistical Manual of Mental Disorders (DSM-5), the six domains of cognitive function include: perceptual-motor function, executive function, complex attention, social cognition, learning and memory, and language (American Psychiatric Association, 2013). Therefore, early identification of cognitive impairment in any domain is pivotal to initiate appropriate interventions to retard its further decline (Brasure et al., 2018).

At present, the standard cognitive assessment method involves traditional neuropsychological paper-and-pencil screening tools for cognitive impairment such as the Montreal Cognitive Assessment (MoCA) and the Mini-Mental State Examination (MMSE) (Abd Razak et al., 2019). Neuropsychological tests are influenced by sociodemographic variables such as age, number of years of education, gender and cultural background (Acevedo et al., 2007). Younger age is associated with better performance among cognitively healthy participants in various neuropsychological tests, including the MoCA (Larouche et al., 2016); and the MMSE (Piccinin et al., 2013).

Neuropsychological paper-and-pencil tests have shown to be lacking in ecological validity, where the individual’s performance on the tests do not adequately predict the real-world functioning of the individual (Spooner and Pachana, 2006). On the other hand, studies have shown that virtual reality has the potential for enhanced ecological validity to provide a better assessment of the individual’s cognitive function in the real-world setting (Parsons, 2015).

In recent years, virtual reality (VR) has been deployed to assess cognition. By wearing a head-mounted device, VR enables a person to experience simulated real-life situations. VR allows the evaluation of multiple cognitive domains such as memory and executive functions. It can potentially be self-administered, requires minimal training, provides a pleasant experience and decreases the psychological distress caused by using conventional screening tools (Jin et al., 2020). Studies have demonstrated the promising applications of VR in cognition evaluation. Vallejo et al. (2017) used VR to assess cognitive processes and everyday life, including a virtual cooking scenario. Davison et al. (2017) developed a virtual parking simulator for cognitive assessment.

Age affects both cognition and VR performance. Cognition declines with age, although younger adults can be affected by Alzheimer’s Disease and other forms of dementia. Studies have revealed that generally younger individuals have better VR performance compared to those who are older (Davison et al., 2017; Ouellet et al., 2018; Plechatá et al., 2019). Sakai et al. (2018) reported significant differences in VR performance scores across age bands of 20 years, and the decline of VR performance is more obvious from the fifth decade onward. However, the results of these studies are often based on their participation in limited number of VR tasks. The study by Plechatá et al. (2019) is an example. They assessed participants’ recall of a shopping list to complete their virtual supermarket task. Overall, very little research has been done on using VR to assess cognitive function based on all six cognitive domains. Moreover, evidence is scarce on the implementation of a single VR system to assess the cognitive function of individuals across a wide span of age groups.

Therefore, the need to design a more comprehensive VR system is imperative. A novel VR system that is capable of giving automated audio-visual instructions while individuals perform tasks that cover all six cognitive domains has been developed for Asians living in urban setting (Lim et al., 2021). Known as CAVIRE (Cognitive Assessment using VIrtual REality) system, it is developed to assess the cognition of community dwelling, ambulatory, older multi-ethnic Asians living in densely populated housing estates.

In this paper, we have recruited participants from different age groups, ranging from 35 years old to 74 years old, and assessed their cognitive function using CAVIRE based on the six cognitive domains. By assessing the performance indicators for each of the six cognitive domains, we would like to establish the system-specific normogram of VR performance for adults across different age groups. Such a normogram is crucial to identify adults with abnormal cognition based on their VR performance in the context of their age group.

Cognitively healthy younger adults are postulated to achieve better performance scores and shorter completion time compared to cognitively healthy older people. Understanding the differential VR performance across the different age groups and cognitive domains will enhance the utility potential of CAVIRE to assess cognition in normal aging.

Aims

This is a feasibility study aimed to assess the performance of cognitively healthy Asian adults aged between 35 and 74 years old to complete tasks in a fully immersive and automated VR system. Their performance was evaluated using a score matrix on the correct attempts and the time taken to complete the tasks across thirteen virtual segments.

Materials and Methods

This manuscript presents the results from a sub-analysis of a published study protocol on the CAVIRE system (Lim et al., 2021).

The Cognitive Assessment Using Virtual Reality System

The system includes a pre-assessment tutorial session, thirteen virtual segments to assess cognitive function, and an automated scoring system.

During the tutorial session, participants put on a head-mounted display device (HMD), namely HTC VIVE Pro, and are inducted to the VR environment by using head movements and hand movements respectively. The hand movements are recorded by the Leap Motion controller. It is a small USB peripheral device mounted onto the HMD. Using two monochromatic infra-red (IR) cameras and three IR LEDs, the device observes a roughly hemispherical area, to a distance of about one meter. The LEDs generate pattern-less IR light and the cameras generate almost 200 frames per second of reflected data. This is then sent through a USB cable to the computer, where it is analyzed by the Leap Motion software using complex algorithms. The 3D position data of the hand movements is synthesized by comparing the 2D frames generated by the two cameras.

The Voice Recognition software utilizes built-in microphone from Windows PC for the voice input. The Application Programming Interface (API) is integrated within the system. It will automatically detect the participant’s speech without recording or active speech extraction. However, for the purposes for our study, we use an external USB microphone. The Voice Recognition software is customized to detect the participant’s correct pronunciation of multiple fruit names in English with a Graphic User Interface (GUI) feedback indication. They are integrated and run in parallel with other Unity3D Applications of the CAVIRE software.

Once participants indicate readiness to the researcher, they proceed to begin with the VR segments. Each segment features common day-to-day activities to assess specific domains of cognitive function. These activities are contextualized to the local Singapore setting and would be familiar to both the young and old participants. Participants perform these virtual tasks via hand gestures and head movements which are detected by motion sensors. Their speech is assessed using the Voice Recognition technology embedded in the system. The automated voice and visual instructions in English guide the participants to complete the tasks.

The participants perform the VR tasks across the following thirteen segments:

(1) Brushing and rinsing teeth.

(2) Preparing peanut butter bread for breakfast.

(3) Identifying pictures of important persons in the newspaper.

(4) Watching television, while listening to the weather forecast regarding impending rain on the radio.

(5) Naming the fruits in a shopping list and remembering the fruits.

(6) Choosing the appropriate clothing to go for grocery shopping.

(7) Remembering to pick up the umbrella, before opening and locking the door.

(8) Taking the lift to level 1 in an apartment block by pressing the correct buttons.

(9) Looking to the left and right, and waiting for green pedestrian light, before crossing the street.

(10) Remembering and choosing the stipulated stall, i.e., the one which sells fruits.

(11) Picking the correct fruits based on recall from the shopping list.

(12) Calculating and paying the correct sum of money for all the fruits.

(13) Selecting the appropriate emotion, with regards to scenes of a birthday party and car accident respectively.

With regards to Figure 1, these thirteen segments cover the six cognitive domains, as defined by the 5th edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5) (American Psychiatric Association, 2013). Four segments (6, 8, 11, and 13) assess a single cognitive domain, while the tasks in the remaining nine segments evaluate two or more cognitive domains. For a balanced evaluation framework, each cognitive domain is assessed over four different segments. The system was pilot-tested internally with young and old volunteers and was further enhanced over three revisions. The maximum amount of time needed to complete the VR assessment across the thirteen segments is approximately 12 min.

FIGURE 1

Figure 1. Cognitive domain assessed in each segment.

The CAVIRE system features an automated scoring system to assess each task performance. The scores are computed automatically for each segment and for the entire VR assessment. The scoring matrix comprises (i) the proportion of tasks performed correctly; (ii) number of attempts needed to complete each task and (iii) completion of tasks within the stipulated time. The matrix is illustrated in “Appendix Table A1 – Scoring algorithm of the VR assessment.”

Study Site

A public primary care clinic (polyclinic) in the southern region of Singapore was determined as the study site. This polyclinic provides comprehensive primary healthcare services to an estimated population of 18,960 residents of varying Asian ethnicity in the Outram estate, of which 24.7% were aged 65 years and above in 2019 (Department of Statistics et al., 2019). About 80% of its patient population originated from other estates in Singapore, as it provides accessible healthcare services to employees working in the central business district in the proximity.

Study Population

Participants included those who sought medical consultation at the polyclinic as well as visitors and accompanying persons of patients at the polyclinic if they satisfied the eligibility criteria: (1) aged between 35 and 74 years old, and (2) understood English (the medium of instruction in CAVIRE, and (3) willing to complete the questionnaires and the CAVIRE VR assessment, and (4) MoCA score of 26 or more.

Individuals with any of the following were excluded: pre-existing diagnosis of cognitive impairment or dementia as documented in their electronic medical record; any disability which rendered them incapable of providing written informed consent; neurological deficits that might affect vision, hearing, speech or motor skills; or known motion sickness or epilepsy.

Sample Size

A total of 25 participants were recruited in each 10-year age group: (1) 35–44 years old; (2) 45–54 years old; (3) 55–64 years old; (4) 65–74 years old. As this is a feasibility study, sample size calculation is not necessary, but sample size justifications need to be provided (Billingham et al., 2013). The Modified Wald method was used to calculate the confidence interval of a proportion. Using a one-sided 95% confidence interval, based on a proportion of 23 out of 25 participants in a particular age group, at least 77% of participants in that age group who undertake the CAVIRE assessment in the future would be able to complete it. For the purposes of this feasibility study, a 90% completion rate, which equates to 23 out of 25 participants in each age group, would be considered adequate. Thus, VR performance of 100 cognitively healthy participants, 25 in each of four different age groups, were enrolled.

Recruitment and Procedure During the Study Administration

Prior to the start of the study, the Research Assistant (RA) was trained by physicians and Advanced Practice Nurse on the recruitment procedure and the administration of the MoCA test. During the study implementation, the RA screened the eligibility of potential participants at the waiting area of the clinic, or via internal referral from study investigators. The RA explained the study protocol, obtained written informed consent, and confirmed the participants’ eligibility criteria from their electronic medical records before administering the MoCA.

Cognitively healthy participants, i.e., with a MoCA score of 26 or higher were enrolled. While scoring the MoCA, an additional point correction was accorded to those with ≤10 years of education (Ng et al., 2013). Those who attained a MoCA score of less than 26 were excluded and referred for further clinical assessment at the polyclinic.

The Pre-VR questionnaire gathered the participants’ demographic data (age, gender, ethnicity, number of years of education) and scores from the following validated cognitive and functional status assessments: (1) Abbreviated Mental Test (AMT) and (2) Mini-Mental State Examination (MMSE).

Next, the participants were briefed on the VR procedure and equipment. With the help of the RA, the participants sat on a chair and put on the VR head-mounted device. The participants were then introduced to a tutorial session. This tutorial session allowed all participants, regardless of age, to familiarize themselves and feel comfortable in using their head and hand movements in the VR environment. Once the participants were ready, they proceeded to complete the 13 segments of the VR assessment. The scores and the time taken to complete each segment were automatically computed in the CAVIRE system and were aggregated for their overall VR performance.

Outcome Measurements

The CAVIRE performance-indices were computed according to the matrix for each cognitive domain in the four age groups: (1) 35–44 years old; (2) 45–54 years old; (3) 55–64 years old; (4) 65–74 years old. The aggregated indices are comprised of both the VR performance scores and completion time.

Data Management and Monitoring

The data from the questionnaires were transcribed into Redcap, a secure research database, and audited by a data management officer in the institution for errors. The VR data from the CAVIRE system was exported to the same database and merged with the audited questionnaire data. The anonymized combined data were handed over to data analysts in the study team. Participants who complained of headache, nausea or giddiness during the VR assessment were advised to stop the procedure and considered as dropouts.

Statistical Analysis

Potential confounders were compared among the four age groups. The potential confounders include gender (male, female), ethnicity (Chinese, non-Chinese), education level (up to secondary, post-secondary/tertiary), and housing (public, private) as a surrogate for socio-economic status. As no significant difference among the four age groups for any confounder was found, the VR performance indices (VR performance score and VR completion time) for the six cognitive domains were compared across the age groups. This was done using a one-way analysis of variance (ANOVA) F-test of the hypothesis H₀: μ₁ = μ₂ = μ₃ = μ₄ (all means equal) vs. H₁: μ_i ≠μ_j for at least one i ≠ j (i = j = 1, 2, 3, 4) (at least two means different). For the rejection of H₀, an F-test p-value of p < 0.05 was considered statistically significant. This is followed by pair-wise comparisons among the four age groups for the VR performance score and the VR completion time respectively. In addition, a linear trend across the four age groups was tested using a contrast. Based on linearized Q-Q plots, the normality of residuals was assessed visually and found to be tenable. For this study, all statistical analyses were done by utilizing the software SAS v9.4.

Results

This segment of the study commenced recruitment in October 2020 and was completed by January 2021. Only one participant from the 35 to 44 years age group failed to complete the study due to apprehension during the administration of the MoCA questionnaire, constituting a dropout rate of 1%. No participant experienced any adverse effect while performing the VR tasks.

Demographic characteristics of the remaining 99 participants are presented in Table 1. The characteristics are similar across the four age groups, labeled as Group 1 for 35–44 years old; Group 2 for 45–54 years old; Group 3 for 55–64 years old and Group 4 for 65–74 years old.

TABLE 1

Table 1. Demographic characteristic frequency counts (%) and comparison among age groups.

Figure 2 shows the VR performance scores in the six cognitive domains by the four age groups. The performance score of each cognitive domain was computed based on the aggregated scores of the respective segments stipulated in the matrix (Figure 2).

FIGURE 2

Figure 2. Virtual reality performance scores in each of the six cognitive domains by age group. Age Group (years): 1 = 35–44, 2 = 45–54, 3 = 55–64, 4 = 65–74.

Table 2 summarizes the mean (SD) performance scores for each age group and the inter-age group mean score differences. Overall, younger participants achieved higher scores compared to the older participants in each cognitive domain (all p < 0.05, Table 2). The inter-age group VR performance scores are discriminant in all six cognitive domains between the Groups (1) and (4); and discriminant in all cognitive domains except for “Learning and Memory” between Groups (1) and (3). Performance scores show no significant difference among the older participants in Groups (3) and (4) across the domains except for “Social Cognition.”

TABLE 2

Table 2. Virtual reality mean (SD) performance score by age group with differences and pair-wise comparisons.

For specific cognitive domains (Table 2), significant differences are noted in sequential age groups from (1) to (3) in “Executive Function”. In the “Perceptual Motor” domain, the performance scores differ between the youngest participants in Group (1) and older participants in Groups (3) and (4). In “Complex Attention” and “Social Cognition” domains, differential VR performances were recorded between younger participants in Group 1 and those in Groups (2), (3), and (4). In “Learning and Memory,” the only significant difference is between the extreme age groups (1) and (4). In “Language,” significant differences in performance scores are noted between younger and older participants [i.e., Groups (1) vs. (3), (1) vs. (4), (2) vs. (3), and (2) vs. (4)].

Figure 3 summarizes the time to complete the VR tasks in each cognitive domain. Overall, younger participants took a shorter time to complete the VR tasks compared to those in the older age groups regardless of the cognitive domain (all p < 0.01, Table 3).

FIGURE 3

Figure 3. Time to complete the virtual reality tasks in each cognitive domain across the age groups. Age Group (years): 1 = 35–44, 2 = 45–54, 3 = 55–64, 4 = 65–74.

TABLE 3

Table 3. Virtual reality mean (SD) completion time by age group with differences and pair-wise comparisons.

Table 3 summarizes the mean (SD) completion time for each age group and the inter-age group mean score differences. The completion time is discriminant in all six cognitive domains between the younger participants in Groups (1) and (2) compared to older participants in Group (4) respectively. Significant differences in completion time are noted between Groups (1) and (3) except for “Social Cognition” and between Groups (2) and (3) except for “Learning and Memory”. Completion time is similar between the younger participants in Groups (1) and (2) across all cognitive domains (Table 3).

In this study, all statistical analysis methods were parametric, as well as assuming normal error distributions. The assumption of normality of the ANOVA residuals was evaluated, and subsequently found to be tenable. All comparisons were done using ANOVA F-tests, followed by t-tests for pair-wise comparisons among the four age groups.

Discussion

Virtual reality is increasingly applied in cognitive assessment for neurocognitive disorders, conferring merits and advantages over the paper-and-pencil methods (Jin et al., 2020). The results show differentiation of the CAVIRE-based VR performance indices between the younger and older cognitively healthy adults across six cognitive domains. This study distinguishes itself from most VR studies that assessed cognitive function on older adults. Bottiroli et al. (2017) validated a VR platform using serious games, known as Smart Aging, for assessing cognitive functions in normal aging. Their participants were largely Italians aged 50 years and older, whose memory, executive functions, working memory, visual spatial elaboration, language, and orientation were assessed. As for the current study, it focused on the VR performance of multi-ethnic Asian adults to complete activities of daily living in an urban environment, which could better reflect the impact of cognition on their life.

The findings in this study show that cognitively healthy younger Asian adults tended to attain higher performance scores and completed the VR within shorter time, compared to their older counterparts. The differentiation cuts across all six domains between the age groups of 35–44 and 65–74. Kinesthesia, the conscious awareness of limb position or limb motion, can influence the participant’s VR performance. Fry et al. (2003) reported age-related changes in upper extremity kinesthesia. According to their study, the younger participants demonstrated significantly lower thresholds of kinesthetic acuity and less error on the kinesthetic memory test compared with those who were older. Thus, the time-based VR performance provides another dimension to cognitive assessments, especially those involving executive function based on the brain-upper limbs axis.

Preparing a toast exemplifies such assessment of executive function, which is incorporated in segment 2 of the CAVIRE system. Such perspectives cannot be objectively evaluated via self-reporting in questionnaire-based assessments. The effect of kinesthesia in executive function alludes to the need to establish age-normalized performance indices specific to both the VR software and to the target population.

For executive function, significant declines in performance scores were noted from the 35–44 years to the 55–64 years age groups (Table 2 and Figure 2). A significant increase in the length of time to complete four tasks in this domain was noted between the 45–54 and 55–64 years age groups (Table 3 and Figure 3). Performing tasks such as preparing a toast, bringing along an umbrella in the context of a rainy day, crossing a road with signal from traffic lights and settling payment by selecting the correct currency notes are reflective of day-to-day executive functions. Such activities of daily living are difficult to assess without direct and objective observations in a clinical setting. Executing such simulated tasks requires reasoning and problem solving. This finding is compatible to the decline in executive function performance using VR from the fifth decade onward, as reported by Sakai et al. (2018). The results of this study suggest that while the performance score decreases markedly in participants aged around mid-50s, the VR performance among the older age groups seems to be stable from 55 to 74 years of age (Table 2).

The VR performance scores for social cognition decreased significantly between the younger age groups and between the older age groups (Table 2). However, the VR completion time shows a distinct increase with age, shorter for the younger (up to 54 years) and longer for the older age groups (55 years and older). The selection of clothing, stall and emotions in the appropriate context are the main tasks in this cognitive domain. Broadly, social cognition includes processes such as social interactions, which are deemed appropriate in the local social context. It represents a complex integration of cognitive functions which are organized to deal with complex processing demands. The conventional paper-and-pencil cognitive assessments have limitations in evaluating social cognition. The study results suggest the potential of VR to screen for deficits in social cognition based on the performance score and completion time in people of different age groups.

Complex attention refers to a person’s ability to maintain information in their mind for a short time and to manipulate that information. As the demands on attention increase, performance slows down and information is less able to be retained. Segment 4 is one of the four settings to provide insight on complex attention: a participant watches a television program while listening to the weather forecast on impending rain over the radio. The results show significant decline in performance score between the 35–44 years and 45–54 years age groups, and longer completion time between the 45–54 years and 55–64 years age groups. Complex attention seems to be similar for the older participants aged 55 years and older.

Language is based on the understanding of the English instructions to perform the stipulated task. Naming the fruits is a good example to demonstrate this cognitive domain. Both the performance score and completion time show significant difference between the 45–54 and 55–64 years age groups. Incorporating voice recognition technology in the CAVIRE software provides objective evaluation of a participant’s articulation of the fruits’ names. Such integration of voice recognition and VR technologies to assess cognition is novel and likely synergistic. The combined applications have to be further evaluated when culturally adapted editions of CAVIRE in other languages are developed to cater to the multi-lingual Asian adults.

Virtual reality completion time is shown to distinguish the perceptual motor domain between the 45–54 and 55–64 years age groups. Activating the lift button to get to the correct floor is one of the four segments to assess perpetual motor function. The completion time is a reflection of the processing speed to carry out the task. Younger participants seem to accomplish such activity more rapidly than their older counterparts. Even the completion time varies across the younger participants in the 35–44 and 45–54 years age groups, although the differences are not statistically significant (Figure 3 and Table 3). Zhang et al. (2001) had also revealed that the VR completion time varied in the control group of younger healthy participants in a trial to compare with the VR performance of their counterparts with brain injury. Nevertheless, no difference is noted in their performance scores across the age groups. This calls for a review in the matrix score board to improve its discrimination of perceptual motor function.

In summary, the VR performances of Asian adults varied with age in using the CAVIRE system, with variable discrimination among the different cognitive domains. Nonetheless, users of this system should be cognizant of the differences in the performance scores and completion time between the younger and older participants, with the most significant demarcation around the age of 55 years old. Further refinements to the VR segments and scoring matrix are needed to reduce the wide dispersion of the scores and completion time.

Strengths and Limitations

The study shows the unprecedented ability of VR technology (CAVIRE) to discriminate the six cognitive domains according to the performance score and completion time in adults aged between 35 and 74 years old with healthy cognition (based on the MoCA). The CAVIRE system also shows significant differences in VR performance across the age groups, regardless of the cognitive domains. The voice recognition incorporated in the VR system is novel and allows objective assessment of language. Older participants were receptive of using the headset and completed the VR tasks without experiencing any adverse effects. No participants dropped out from the study due to failure to use the VR equipment. The embedded automated scoring system standardizes the appraisal of the VR tasks and measures the associated completion time. The results of this study complement that of a previous publication on CAVIRE, where the entire completion time for CAVIRE was significantly shorter compared to that of MoCA (Wong et al., 2021). These results show that CAVIRE is feasible to be used for cognitive assessment in the six cognitive domains for individuals across different age groups.

Nevertheless, the study has its limitations. The limited number of participants in each age group restricts the generalizability of this study to the general population. Cognitively healthy participants were identified using only a single MoCA cut-off score of 26. According to a multi-ethnic meta-analysis, a MoCA cut-off of 26 may not be suitable (Carson et al., 2017). However, it is important to note that the meta-analysis was based on different ethnic populations, which also included non-Asians. This current study on Asian individuals is contextualized to Singapore, in which a MoCA cut-off score of 26 is widely used (Ng et al., 2013).

Although no significant difference was observed in the basic demographic characteristics of the participants across the age groups, including education level, it was noted that a higher proportion of the younger participants (35–44 years old) received post-secondary/tertiary education. Nevertheless, it is possible that the effect of education was not accentuated, since the tasks in the CAVIRE assessment consist of common activities of daily living. Other studies on VR have also shown that the effect of education is not significant (Oliveira et al., 2018). In the current study, participants were stratified according to different age groups, but due to the limited sample size, stratification according to education level was not possible. Potential confounders such as intellectual capacity, occupations and intra-age group variations were also not addressed in this study.

The performance scores for each cognitive domain were aggregated based on four different segments to mitigate execution variations and functional assessment in a single segment. The segments will require review and refinement to better differentiate the cognitive functions. An adequately powered randomized controlled trial using an enhanced edition of CAVIRE system will be developed to attest its discrimination in cognitive assessment across the different age groups, compared to conventional paper-and-pencil-based assessments.

Conclusion

The study shows that the VR performance scores and completion time of using the fully immersive and automated CAVIRE system significantly differ between the younger and older Asian adults with healthy cognition. The system has shown potential to assess adults across the six cognitive domains although its degree of discrimination varies between domains. Enhancements to the VR segments and matrix score are needed to establish the age group-specific normal performance indices.

Data Availability Statement

The original contributions presented in the study are included in the article, further inquiries can be directed to the corresponding author.

Ethics Statement

The studies involving human participants were reviewed and approved by SingHealth Centralised Institutional Review Board, Singapore. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

JL, WW, JQ, and NT designed the VR performance tasks and score matrix and the study protocol. RM reviewed the face validity of the 13 segments for cognitive assessment. TT and SL collaborated with FXMedia Internet Pte Ltd, the industrial collaborator to develop the VR system, and including the voice recognition component. WW and NT secured the funding. PM and JL conducted the study and collated and audited the data. JA analyzed the data. NT, JA, JL, RM, WW, and JQ interpreted the results. JL and NT drafted the manuscript. All the authors reviewed, critiqued, and revised the draft, finalized and approved before submitting the manuscript to the journal.

Funding

This study was funded by the Mitsui Sumitomo Insurance Welfare Foundation Research Grant 2019, the SingHealth Polyclinics Seed Funding (SHP-SEED53-2019[4]) and the AM-ETHOS Duke-NUS Medical Student Fellowship Award (AM-ETHOS01/FY2019/34-A34). The software development was funded by the Infocomm Media Development Authority (IMDA) of Singapore. The funds were used to procure the VR equipment, consumables and reimburse the research participants.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

The authors would like to acknowledge IMDA, the efforts of the VR software developers and graphic designers from FX Media Internet Pte Ltd for their help in developing the CAVIRE system. The authors also grateful to the administrative team in the Research Department in SHP who assisted in the ethics review board application and the procurement of the equipment.

References

Abd Razak, M. A., Ahmad, N. A., Chan, Y. Y., Mohamad Kasim, N., Yusof, M., Abdul Ghani, M. K. A., et al. (2019). Validity of screening tools for dementia and mild cognitive impairment among the elderly in primary health care: a systematic review. Public Health 169, 84–92. doi: 10.1016/j.puhe.2019.01.001

CrossRef Full Text | Google Scholar

Acevedo, A., Loewenstein, D. A., Agrón, J., and Duara, R. (2007). Influence of sociodemographic variables on neuropsychological test performance in Spanish-speaking older adults. J. Clin. Exper. Neuropsychol. 29, 530–544. doi: 10.1080/13803390600814740

PubMed Abstract | CrossRef Full Text | Google Scholar

American Psychiatric Association. (2013). Diagnostic and Statistical Manual of Mental Disorders, 5th Edn (DSM-5). Arlington, VA: American Psychiatric Association, doi: 10.1176/appi.books.9780890425596

CrossRef Full Text | Google Scholar

Billingham, S. A., Whitehead, A. L., and Julious, S. A. (2013). An audit of sample sizes for pilot and feasibility trials being undertaken in the United Kingdom registered in the United Kingdom Clinical Research Network database. BMC Med. Res. Methodol. 13:104. doi: 10.1186/1471-2288-13-104

CrossRef Full Text | Google Scholar

Bottiroli, S., Tassorelli, C., Lamonica, M., Zucchella, C., Cavallini, E., Bernini, S., et al. (2017). Smart Aging Platform for Evaluating Cognitive Functions in Aging: A Comparison with the MoCA in a Normal Population. Front. Aging Neurosci. 9:379. doi: 10.3389/fnagi.2017.00379

CrossRef Full Text | Google Scholar

Brasure, M., Desai, P., Davila, H., Nelson, V. A., Calvert, C., Jutkowitz, E., et al. (2018). Physical Activity Interventions in Preventing Cognitive Decline and Alzheimer-Type Dementia: A Systematic Review. Ann. Internal Med. 168, 30–38. doi: 10.7326/M17-1528

PubMed Abstract | CrossRef Full Text | Google Scholar

Carson, N., Leach, L., and Murphy, K. J. (2017). A re-examination of Montreal Cognitive Assessment (MoCA) cutoff scores. Int. J. Geriatric Psychiatry 33, 379–388. doi: 10.1002/gps.4756

CrossRef Full Text | Google Scholar

Davison, S. M. C., Deeprose, C., and Terbeck, S. (2017). A comparison of immersive virtual reality with traditional neuropsychological measures in the assessment of executive functions. Acta Neuropsychiatrica 30, 79–89. doi: 10.1017/neu.2017.14

PubMed Abstract | CrossRef Full Text | Google Scholar

Department of Statistics, Ministry of Trade and Industry, and Republic of Singapore. (2019). Population Trends. Available online at: https://www.singstat.gov.sg/-/media/files/publications/population/population2019.pdf (accessed on Feb 25, 2020)

Google Scholar

Fry, D. K, Campbell, J, Foltz, B, and Macek, R. (2003). Age-related changes in upper extremity kinesthesis. Phys. Occupat. Ther. Geriatrics 20, 137–154. doi: 10.1080/J148v20n03_09

CrossRef Full Text | Google Scholar

Jin, R., Pilozzi, A., and Huang, X. (2020). Current Cognition Tests, Potential Virtual Reality Applications, and Serious Games in Cognitive Assessment and Non-Pharmacological Therapy for Neurocognitive Disorders. J. Clin. Med 2020:3287. doi: 10.3390/jcm9103287

CrossRef Full Text | Google Scholar

Larouche, E., Tremblay, M. P., Potvin, O., Laforest, S., Bergeron, D., Laforce, R., et al. (2016). Normative Data for the Montreal Cognitive Assessment in Middle-Aged and Elderly Quebec-French People. Arch. Clin. Neuropsychol. 31, 819–826. doi: 10.1093/arclin/acw076

PubMed Abstract | CrossRef Full Text | Google Scholar

Lim, J. E., Wong, W. T., Teh, T. A., Lim, S. H., Allen, J. C. Jr., Quah, J. H. M., et al. (2021). “A Fully-Immersive and Automated Virtual Reality System to Assess the Six Domains of Cognition: Protocol for a Feasibility Study. Front. Aging Neurosci. 12:604670. doi: 10.3389/fnagi.2020.604670

CrossRef Full Text | Google Scholar

Ng, A., Chew, I., Narasimhalu, K., and Kandiah, N. (2013). Effectiveness of Montreal Cognitive Assessment for the diagnosis of mild cognitive impairment and mild Alzheimer’s disease in Singapore. Singapore Med. J. 54, 616–619. doi: 10.11622/smedj.2013220

CrossRef Full Text | Google Scholar

Oliveira, C. R., Lopes Filho, B., Esteves, C. S., Rossi, T., Nunes, D. S., Lima, M., et al. (2018). Neuropsychological Assessment of Older Adults With Virtual Reality: Association of Age, Schooling, and General Cognitive Status. Front. Psychol. 9:1085. doi: 10.3389/fpsyg.2018.01085

CrossRef Full Text | Google Scholar

Ouellet, É, Boller, B., Corriveau-Lecavalier, N., Cloutier, S., Belleville, S., et al. (2018). The Virtual Shop: A new immersive virtual reality environment and scenario for the assessment of everyday memory. J. Neurosci. Methods 303, 126–135. doi: 10.1016/j.jneumeth.2018.03.010

PubMed Abstract | CrossRef Full Text | Google Scholar

Parsons, T. D. (2015). “Virtual Reality for Enhanced Ecological Validity and Experimental Control in the Clinical, Affective and Social Neurosciences.”. Front. Hum. Neurosci. 9:660. doi: 10.3389/fnhum.2015.00660

CrossRef Full Text | Google Scholar

Piccinin, A. M., Muniz-Terrera, G., Clouston, S., Reynolds, C. A., Thorvaldsson, V., Deary, I. J., et al. (2013). Coordinated analysis of age, sex, and education effects on change in MMSE scores. J. Gerontol. Ser. B Psychol. Sci. Soc. Sci. 68, 374–390. doi: 10.1093/geronb/gbs077

PubMed Abstract | CrossRef Full Text | Google Scholar

Plechatá, A., Sahula, V., Fayette, D., Fajnerová, I., et al. (2019). Age-Related Differences With Immersive and Non-immersive Virtual Reality in Memory Assessment. Front. Psychol. 10:1330. doi: 10.3389/fpsyg.2019.01330

CrossRef Full Text | Google Scholar

Sakai, H., Nagano, A., Seki, K., Okahashi, S., Kojima, M., Luo, Z., et al. (2018). Development of a cognitive function test using virtual reality technology: examination in healthy participants. Aging Neuropsychol. Cognition 25, 561–575. doi: 10.1080/13825585.2017.1351916

PubMed Abstract | CrossRef Full Text | Google Scholar

Spooner, D. M., and Pachana, N. A. (2006). “Ecological validity in neuropsychological assessment: a case for greater consideration in research with neurologically intact populations.” Arch. Clin. Neuropsychol. 21, 327–337. doi: 10.1016/j.acn.2006.04.004

CrossRef Full Text | Google Scholar

Vallejo, V., Wyss, P., Rampa, L., Mitache, A. V., Müri, R. M., Mosimann, U. P., et al. (2017). Evaluation of a novel Serious Game based assessment tool for patients with Alzheimer’s disease. PLoS One 12:e0175999. doi: 10.1371/journal.pone.0175999

CrossRef Full Text | Google Scholar

Wong, W. T., Tan, N. C., Lim, J. E., Allen, J. C. Jr., Lee, W. S., Quah, J. H. M., et al. (2021). Comparison of Time Taken to Assess Cognitive Function Using a Fully Immersive and Automated Virtual Reality System vs. the Montreal Cognitive Assessment. Front. Aging Neurosci. 13:756891. doi: 10.3389/fnagi.2021.756891

CrossRef Full Text | Google Scholar

World Health Organisation. (2019). Dementia. Available online at :https://www.who.int/news-room/fact-sheets/detail/dementia (accessed May 14, 2019)

Google Scholar

Zhang, L., Abreu, B. C., Masel, B., Scheibel, R. S., Christiansen, C. H., Huddleston, N., et al. (2001). Virtual reality in the Assessment for selected Cognitive Functions after Brain Injury. Am. J. Phys. Med Rehabil. 80, 597–604.

Google Scholar

Appendix

TABLE A1

Table A1. Scoring algorithm of the VR assessment.