Cosimo Tuena1,2*
Elisa Pedroli1,3
Pietro Davide Trimarchi4
Alessia Gallucci4
Mattia Chiappini1Karine Goulene5
Andrea Gaggioli1,2
Giuseppe Riva1,2
Fabrizia Lattanzio6Fabrizio Giunco4
Marco Stramba-Badiale5- 1Applied Technology for Neuro-Psychology, IRCCS Istituto Auxologico Italiano, Milan, Italy
- 2Department of Psychology, Catholic University of the Sacred Hearth, Milan, Italy
- 3Faculty of Psychology, University of eCampus, Novedrate, Italy
- 4IRCCS Fondazione Don Carlo Gnocchi, Milan, Italy
- 5Department of Geriatrics and Cardiovascular Medicine, IRCCS Istituto Auxologico Italiano, Milan, Italy
- 6Scientific Direction, IRCCS INRCA, Ancona, Italy
Aging is a condition that may be characterized by a decline in physical, sensory, and mental capacities, while increased morbidity and multimorbidity may be associated with disability. A wide range of clinical conditions (e.g., frailty, mild cognitive impairment, metabolic syndrome) and age-related diseases (e.g., Alzheimer's and Parkinson's disease, cancer, sarcopenia, cardiovascular and respiratory diseases) affect older people. Virtual reality (VR) is a novel and promising tool for assessment and rehabilitation in older people. Usability is a crucial factor that must be considered when designing virtual systems for medicine. We conducted a systematic review with Preferred Reporting Items for Systematic reviews and Meta-analysis (PRISMA) guidelines concerning the usability of VR clinical systems in aging and provided suggestions to structure usability piloting. Findings show that different populations of older people have been recruited to mainly assess usability of non-immersive VR, with particular attention paid to motor/physical rehabilitation. Mixed approach (qualitative and quantitative tools together) is the preferred methodology; technology acceptance models are the most applied theoretical frameworks, however senior adapted models are the best within this context. Despite minor interaction issues and bugs, virtual systems are rated as usable and feasible. We encourage usability and user experience pilot studies to ameliorate interaction and improve acceptance and use of VR clinical applications in older people with the aid of suggestions (VR-USOP) provided by our analysis.
Introduction
Life expectancy is rapidly increasing and is expected to rise in the years to come, thereby creating an aging population. However, a significant proportion of older people may develop frailty, multi-morbidity, and disability causing a significant impact both on their quality of life and also on health care and social costs (Lutz et al., 2008; World Health Organization, 2015). Aging is associated with physiological changes (e.g., apoptosis, senescence, inflammation) that may lead to systemic alterations (Flatt, 2012). This potential decline may involve sensory, mental, and physical functioning thus leading to-increased morbidity, multi-morbidity, disability, and mortality (World Health Organization, 2015). On the other hand, motor skills, visual, hearing, proprioception, and cognitive abilities (e.g., memory) may be reduced even in healthy older people (Kuehn et al., 2017). In addition, aging hampers psychosocial well-being by adding new developmental tasks or situations (e.g., isolation; Steptoe et al., 2015). In particular, the prevalence of Alzheimer's disease, cancer, chronic obstructive pulmonary disease, maculopathy, osteoarthritis, osteopenia, Parkinson's disease, periodontitis, rheumatoid arthritis, sarcopenia, cardiovascular diseases, and type 2 diabetes increases with age (Tolosa et al., 2006; Dubois et al., 2010; Marengoni et al., 2011; Edwards et al., 2015; Steenman and Lande, 2017; Yakaryilmaz and Öztürk, 2017; Franceschi et al., 2018). Additionally, several clinical conditions may jeopardize the well-being of older people, such as mild cognitive impairment, frailty, or metabolic syndrome (Fried et al., 2001; Petersen, 2004; Portet et al., 2006; Huang, 2009; Xue, 2011; Fedarko, 2012). The main priority of successful management of aging is enabling older people to be healthy, active, and autonomous for as long as possible (World Health Organization, 2002). Accordingly, functional decline is one of the key issues to be managed (World Health Organization, 2015). Among other practices, the use of assistive health technology (AHT; i.e., technologies devoted to maintain or improve functionality, autonomy and well-being) or medical devices (MD; i.e., technologies used for prevention, diagnosis and treatment) may also produce a beneficial effect in older people (Garçon et al., 2016); however, a critical aspect is to ensure accessibility and use of these technologies in the older population (World Health Organization, 2015; Beard et al., 2016).
Virtual reality (VR) is one of the emerging AHT and MD in the field of aging, frailty, and disability (Lange et al., 2010; Bohil et al., 2011). VR is defined as a system based on an interactive computer-simulated 3D environment (Gorini and Riva, 2008), which incorporates mainly auditory and visual feedback, and sometimes also haptic. VR can be divided in non-immersive, semi-immersive, and fully immersive systems (Mujber et al., 2004). The non-immersive system is a desktop-based VR with low interaction (e.g., keyboard, joypad) and immersion (e.g., PC, tablet). The semi-immersive system consists of a large monitor/projector with moderate immersion and interaction (e.g., Kinect, data gloves). The immersive system is characterized by the use of tools such as a head-mounted display (HMD) or the cave automatic virtual environment (CAVE) that enables a high degree of interaction (e.g., trackers) and immersion in the virtual environment (VE). Additionally, VR can be conceptualized as a continuum between reality and virtuality, where some aspects of VE are mixed with the real environment (augmented reality) or vice-versa (augmented virtuality) (Milgram et al., 1995). The sensorimotor channels connected to the VR define the degree of immersion; the psychological consequence of immersion on perception is the sense of presence that felt through being in the VE or, alternatively, the “perceptual illusion of non-mediation” with the VE (Riva, 2008; Bohil et al., 2011). Moreover, mobile applications (e.g., tablet) with tracking systems of the user and/or visors (e.g., Google Cardboard) can be considered mobile VR that allow for different degrees of immersion and interaction with the VE (Pallavicini et al., 2015; Fang et al., 2017).
VR has several requirements for motor and cognitive neurorehabilitation interventions: repetitive practice, feedback about performance, multimodal stimulation, and controlled, secure, and ecologically valid environments (Bohil et al., 2011). It is possible to control and manipulate tailored exercises within meaningful and motivating environments using virtual environments, i.e., transformation of flow (Riva et al., 2006). For these reasons, VR has been utilized for rehabilitation in different fields and, particularly, after stroke. Accordingly, guidelines have recently included the use of VR for both motor and cognitive rehabilitation in patients who suffered a stroke (ISO, 2016b; Winstein et al., 2016). However, access to this kind of technology may be limited by the lack of accessibility in the older population, as compared to other AHT and MD (World Health Organization, 2015). For instance, VR in the context of stroke rehabilitation is facing challenges concerning end-users' interaction, such as feasibility of VR training, lack of functional relevance, patient frustration to feedback, and lack of integration of environmental factors that link to motor performance (Teo et al., 2016).
On the macroscopic level, access to AHT and MD is limited by socio-demographic and economic terms, while on the microscopic level, access is the use itself of a device. Indeed, according to the MOLD-US framework (Wildenbos et al., 2018), the use of technology among older people is hampered by different barriers: (1) cognitive (e.g., reduced working memory, spatial cognition, attention, language, and reasoning) and motivational (e.g., self-efficacy, self-confidence, benefits identification, computer literacy, integration in daily life) that affect the use with errors, efficiency, learnability, memorability and satisfaction; (2) physical (e.g., motor speed, flexibility, hand-eye coordination, strength) and perception (e.g., vision, auditory, haptic) that influence errors and efficiency. According to Nielsen (Nielsen, 2012), usability is defined by learnability (is it easy to accomplish the task?), efficiency (once learned, is the user fast in performing the task?), memorability (is the user able to reestablish proficiency with the design after a period of stop?), errors (how many errors does the user make?) and satisfaction (how pleasant is the design?). Along with usability (i.e., easiness and pleasure), the technology should provide the attributes needed by the user (i.e., utility). Usability can be assessed by a means of a wide range of methods, such as the system usability scale (SUS), heuristic evaluation, cognitive and pluralistic walkthrough, formal usability, pluralistic, consistency, and standard inspections (Brooke, 1986; Nielsen, 1994).
Nevertheless, usability tends to focus more on the task rather than on the experience (Vermeeren et al., 2010). Indeed, researchers investigating user experience (UX) point out a role of factors that go beyond the technology and its usability/usefulness. UX facets embrace emotion and affective reactions toward the technology and experiential, hedonic, holistic, and aesthetic factors. The interaction with a technology is “a subjective, situated, complex, and dynamic encounter” (Hassenzahl and Tractinsky, 2006). If it is true that satisfaction plays a critical role in usability, UX takes into account emotions, motivation, and expectation of human-computer interaction (Vermeeren et al., 2010). For instance, the user experience questionnaire (UEQ) aims at evaluating six factors: attractiveness, perspicuity, efficiency, dependability, stimulation, and novelty (Laugwitz et al., 2008), or the usability metric for user experience (UMUX) taps UX facets of usability (Finstad, 2010). Additionally, 96 UX methods (http://www.allaboutux.org/all-methods) have been identified in the UX research field (Vermeeren et al., 2010). UX methods range from qualitative to quantitative techniques, target technology, period of assessment (e.g., developmental, conceptual), time, information source (e.g., experts, specific users, individual, group), and location (e.g., lab, online, field). Methods range from semantic differential, checklists, heuristics, think-aloud, psychophysiological measures, self-report, questionnaires, in situ observation, and video analysis (Vermeeren et al., 2010). A critical aspect of UX is the prototype development (Novak, 2008), which follows the concept (idea) and pre-production (demo) phases and precedes production & localization (development), Alpha, Beta, and Gold/post-production phases.
A wide range of theories have been proposed to understand and explain user acceptance and use of technology (for a literature review see Taherdoost, 2018). The most inclusive model is the unified theory of acceptance and use of technology model (UTAUTM) (Venkatesh et al., 2003), which includes the technology acceptance model (TAM), theory of reasoned action, theory of planned behavior (TPB), combined TAM and TPB, model of PC utilization, the diffusion of innovation model, motivational model, and social cognitive theory. In this model, the significant factors are: effort, expectancy, performance expectancy, social influence, and facilitating conditions. Interestingly, starting from the TPB (Fishbein and Ajzen, 1975), TAM (Davis et al., 1989), and UTAUTM (Chen and Shou, 2014), developed the senior technology acceptance model (STAM). Controlling age, gender, educational level, and economic status, their model included gerontotechnology self-efficacy and anxiety, facilitating health conditions, cognitive abilities, social relationships, attitude to life and satisfaction, and physical functioning as factors that influenced perceived usefulness, usage behavior, and perceived ease of use, which in turn affects general attitude toward the use. A similar model (senior citizens' acceptance of information systems; SCAIS) was developed by Phang et al. (2006). This model takes into account preference for human contact, self-actualization, resource saving, anxiety, computer support, physiological decline which influences perceived usefulness, ease of use, internet safe perception and in turn, intention. Another theoretical framework used to approach technology use and acceptance is the user-centered design (UCD). UCD enables technology systems to be made more usable and interactive to end-users, but it can also be applied to assess needs, wants, and limitations of general products (Sebe, 2010; ISO, 2016a; Brox et al., 2017). UCD can be investigated using a variety of qualitative and quantitative methods such as field studies, user requirements analyses, iterative design, usability evaluation, task analyses, focus groups, user interviews, participatory design, and prototypes (Vredenburg et al., 2002). UX can be explored with the playability model (i.e., immersion, socialization, emotion, satisfaction, effectiveness) that is crucial when building games for clinical purposes (Sánchez et al., 2012; Valladares-Rodriguez et al., 2019); emotive design for VR should be followed for designing human-computer interaction systems (see Vredenburg et al., 2002).
Lastly, human-computer interfaces are also conceptualized in terms of architecture and layers needed to provide a service (Tsai et al., 2012; Nikitina et al., 2018). For instance, the user remote console (URC) is a framework used for telemedicine systems to define abstract user interface layers, hubs, and devices. If a researcher wishes to consider a VR AHT or MD for healthcare purposes, in addition to the usability and UX aspects, they may want to assess the sense of presence in the VE. According to the Inner Presence theory (Lee, 2004; Riva and Waterworth, 2014), presence is not necessarily related to media characteristics (e.g., graphic realism) but rather to an everyday life flow that controls actions through a constant intentions-perceptions comparison. In this sense, a VR user may experience the system as usable, as they are able to enact actions thanks to an easy-to-learn interface that tracks user's movements, an understandable game/training structure, and engaging storytelling (Triberti and Riva, 2016). These elements are particularly relevant for videogames and serious games used also for therapeutic purposes (Sáenz-de-Urturi et al., 2015). This conceptualization of presence has relevant consequences when taking clinical practice and change into consideration. VR clinical applications should exploit the transformation of flow (transformative and optimal experience allowed by the sense of presence) to discover and use new and unexpected resources to foster clinical change (Riva et al., 2006, 2016) and consider sensorimotor and cognitive impairments in the old population to customize VR for cognitive (Tuena et al., 2019) or physical (Pedroli et al., 2018) rehabilitation.
This paper aims at systematically reviewing the studies that evaluated feasibility, usability, and UX of assessment and treatment VR systems in healthy aging and age-related clinical conditions. In order to provide an overview of the current research status we analyzed characteristics of participants involved, technological apparatus and use, usability/UX assessments, theoretical framework, and primary outcomes. VR use is classified as the task being accomplished and the training sessions and the aims, which include assessment and rehabilitation. Additionally, we outlined suggestions to assess usability of VR applications for older people in clinical and research contexts.
Methods
Preferred Reporting Items for Systematic reviews and Meta-Analysis (PRISMA) guidelines were followed (Moher et al., 2009).
Search Strategy
Three high-profile databases (PubMed, PsycINFO, and Web of Science) were used to perform the computer-based research on 3 September 2019. The string used to carry out the search (Title/Abstract for PubMed, Topic for Web of Science, Abstract for PsycINFO) was as follows: (“aging” OR “frailty” OR “elder*” OR “multimorbidity”) AND (“usability” OR “user experience” OR “UX” OR “user centered design” OR “human centered design” OR “human computer interaction”) AND (“virtual”). The search resulted in 507 articles for Web of Science, 22 for PubMed, and 20 for PsycINFO (total of 529). We made a first selection by reading titles and abstracts after removing duplicates. A total of 66 manuscripts were chosen for full-text screening. This procedure resulted in 25 experimental studies. See the flow diagram (Figure 1) for the paper selection procedure.
Figure 1. PRISMA flow chart.
Selection Criteria
Studies concerning the usability, UX, and feasibility of VR (see introduction for definition) systems for assessment/monitoring and rehabilitation/empowerment in healthy and pathological aging were included. In particular, we focused on the age-related clinical conditions in older people. We excluded articles that did not involve usability of VR clinical systems in non-age-related conditions that do not fall into the context of frailty, multimorbidity, or chronicity in aging and with technologies that do not meet VR definition. Additionally, studies for which the full text was not available or for which the abstract lacked basic information for review were removed. Non-English papers, reviews, meeting abstracts, conference proceedings, notes, case reports, letters to the editor, research protocols, patents, editorials, and other editorial materials were also excluded.
Quality Assessment and Data Abstraction
PRISMA guidelines were strictly followed; search results found by the first author (CT) were shared with the review author (MC) for individual selection of papers in order to reduce the risk of bias, and disagreements were resolved through consensus. The risk of bias for each single study was assessed following the Cochrane guidelines (Higgins et al., 2011) by CT and MC. The research question was formulated according to suggested PICO (Population: older people with age ≥ 65, Intervention: VR for assessment or rehabilitation in age-related conditions and diseases, Comparison: N/A as usability at this time adopt quasi-experimental or pilot study designs (see also risk of bias Supplementary Figure 1), Outcome: measures of usability and acceptance) research question guidelines (Abigail et al., 2014). The Comparison is mainly applied to randomized clinical trials and within our search only one study (Schwenk et al., 2014) satisfied this criterion. Consequently, data extracted from each included study were as follows: reference, year, sample (s), aims, technology, VR training, technology design framework, usability/UX/feasibility assessment tools, primary outcomes, and type (assessment/rehabilitative) of VR system.
Results
Our search identified several usability, user experience (UX), and feasibility studies in healthy aging and age-related clinical conditions. A critical aspect of virtual reality (VR) and new technologies is their interaction with humans and in particular, those whose physical, psychological, or social barriers hamper the use of technological devices. The aim of this systematic review was to analyze the current research in the field of usability of clinical VR systems in older people and to provide an overview on this topic. Findings are shown in Table 1 according to reference, year, sample(s), aims of the study, VR technology, VR training, theoretical framework, usability assessment, primary outcomes, and clinical aims. Figures 2–8 summarize the results as well.
Table 1. Summary of the studies included.
Figure 2. Clinical conditions; OA, older adults.
Figure 3. Clinical applications.
Figure 4. Degree of virtual immersion; VR, virtual reality.
Figure 5. Methodological approach.
Figure 6. Technology usability and acceptance models; SCAIS, senior citizens acceptance of information systems; TAM, technology acceptance model; ToF, transformation of flow; UX, user experience.
Figure 7. Assessment tools by the total of unique instruments; SUS, system usability scale; TAM, technology acceptance model; UX, user experience; VR, virtual reality.
Figure 8. Available mean SUS scores with standard deviations; SUS, system usability scale; G1, group 1; G2 group 2; T0, baseline session; T2, third session.
Which Are the Samples Involved in VR Usability Studies?
The majority of the studies (Kizony et al., 2006; Tsai et al., 2012; Castilla et al., 2013; Corno et al., 2014; Wüest et al., 2014; Im et al., 2015; Morán et al., 2015; Cook and Winkler, 2016; Trombetta et al., 2017; Vanbellingen et al., 2017; Plechatá et al., 2019; Rebsamen et al., 2019) recruited healthy older adults (OA) to assess the usability of clinical VR systems. Two studies collected data from the fifth decade to old age (Kiselev et al., 2015; Money et al., 2019). Nevertheless, in these studies, systems were created for clinical conditions such as stroke (Wüest et al., 2014; Morán et al., 2015; Trombetta et al., 2017; Vanbellingen et al., 2017) or movement disorders (e.g., balance, physical frailty; Pedroli et al., 2018; Money et al., 2019). Indeed, only two studies recruited stroke patients for stroke VR systems (Kizony et al., 2006; Fordell et al., 2011). Sáenz-de-Urturi et al. (2015) and Pedroli et al. (2018) recruited OA and, among these individuals, some had mild or moderate cognitive impairment (O'Bryant et al., 2017). Patients with mixed age-related conditions (e.g., Parkinson's disease, macular degeneration, muscular dystrophy, arthritis, diabetes, hypertension) were recruited in Sáenz-de-Urturi et al. (2015) and Shubert et al. (2015). Frail patients were collected in Nikitina et al. (2018), mixed frail and physical-motor patients in Brox et al. (2017), participants at risk of falling in Schwenk et al. (2014) and Kiselev et al. (2015), mild cognitive impairment (MCI) and Alzheimer's disease (AD) individuals in Valladares-Rodriguez et al. (2019), Parkinson's disease (PD) individuals in van Beek et al. (2019), OA with atrial fibrillation (AF) in Desteghe et al. (2017), and with orthopedics impairments in Epelde et al. (2014). Experts and medical professionals were included in some pilot studies for their opinion on the design or on the VR system (Castilla et al., 2013; Epelde et al., 2014; Morán et al., 2015; Sáenz-de-Urturi et al., 2015; Brox et al., 2017; Desteghe et al., 2017; Valladares-Rodriguez et al., 2019).
Which are the Aims and the Clinical Fields of the Studies?
All of the studies—except one that was principally devoted to the clinical efficacy of the training (Schwenk et al., 2014)—were mainly designed for the usability, UX and feasibility of VR systems in aging (Tsai et al., 2012; Castilla et al., 2013; Corno et al., 2014; Epelde et al., 2014; Wüest et al., 2014; Im et al., 2015; Kiselev et al., 2015; Morán et al., 2015; Sáenz-de-Urturi et al., 2015; Shubert et al., 2015; Cook and Winkler, 2016; Brox et al., 2017; Desteghe et al., 2017; Trombetta et al., 2017; Vanbellingen et al., 2017; Nikitina et al., 2018; Pedroli et al., 2018; Money et al., 2019; Plechatá et al., 2019; Rebsamen et al., 2019; Valladares-Rodriguez et al., 2019; van Beek et al., 2019). Most of the studies concerned the assessment of therapeutic (i.e., rehabilitative or psychological empowerment) VR systems Tsai et al., 2012; Castilla et al., 2013; Corno et al., 2014; Epelde et al., 2014; Wüest et al., 2014; Im et al., 2015; Kiselev et al., 2015; Morán et al., 2015; Sáenz-de-Urturi et al., 2015; Shubert et al., 2015; Cook and Winkler, 2016; Brox et al., 2017; Desteghe et al., 2017; Trombetta et al., 2017; Vanbellingen et al., 2017; Nikitina et al., 2018; Pedroli et al., 2018; Money et al., 2019; Plechatá et al., 2019; Rebsamen et al., 2019; Valladares-Rodriguez et al., 2019; van Beek et al., 2019, whereas only a few were on assessment or monitoring tools (Fordell et al., 2011; Corno et al., 2014; Desteghe et al., 2017; Plechatá et al., 2019; Valladares-Rodriguez et al., 2019). The intervention/assessment of the studies included were physical-motor (e.g., limb physiotherapy, physical activity, hand motricity; Kizony et al., 2006; Epelde et al., 2014; Schwenk et al., 2014; Wüest et al., 2014; Im et al., 2015; Kiselev et al., 2015; Morán et al., 2015; Sáenz-de-Urturi et al., 2015; Shubert et al., 2015; Brox et al., 2017; Trombetta et al., 2017; Vanbellingen et al., 2017; Nikitina et al., 2018; Pedroli et al., 2018; Money et al., 2019; van Beek et al., 2019), neuro/psychological (Fordell et al., 2011; Tsai et al., 2012; Castilla et al., 2013; Corno et al., 2014; Desteghe et al., 2017; Plechatá et al., 2019; Valladares-Rodriguez et al., 2019), cardiovascular fitness (Rebsamen et al., 2019), or non-specific healthcare applications (Cook and Winkler, 2016).
Which Are the VR Technologies Used and the Training?
Non-immersive VR (i.e., desktop-based VR, tablet, and mobile app) were used in most of the studies (Kizony et al., 2006; Castilla et al., 2013; Wüest et al., 2014; Morán et al., 2015; Shubert et al., 2015; Cook and Winkler, 2016; Desteghe et al., 2017; Vanbellingen et al., 2017; Nikitina et al., 2018; Money et al., 2019; Valladares-Rodriguez et al., 2019; van Beek et al., 2019). Application exclusively for tablets were used in Valladares-Rodriguez et al. (2019) and multi-device (i.e., PC, tablet or mobile) apps in Castilla et al. (2013), Desteghe et al. (2017), and Nikitina et al. (2018). Semi-immersive VR (i.e., large TV or projector screens with sensors for interaction) systems were tested in some studies (Tsai et al., 2012; Epelde et al., 2014; Schwenk et al., 2014; Im et al., 2015; Kiselev et al., 2015; Sáenz-de-Urturi et al., 2015; Brox et al., 2017; Rebsamen et al., 2019), whereas full immersive VR (i.e., visors or CAVE with interaction devices) was tested only in Fordell et al. (2011), Corno et al. (2014), and Pedroli et al. (2018). Interestingly, Trombetta et al. (2017) compared a semi-immersive vs. an immersive version of the training to evaluate their usability, whereas Plechatá et al. (2019) tested a VR memory test with non-immersive vs. immersive VR.
Sessions lasted from nine to 90 min (mean time = 30 min ca.), ranging from one to 36 sessions spread over the course of 1 single day to 4 years; indeed, usability along with effectiveness of training was tested for three (Vanbellingen et al., 2017), four (Im et al., 2015; Rebsamen et al., 2019; van Beek et al., 2019), six (Kiselev et al., 2015), eight (Nikitina et al., 2018), and 12 weeks (Wüest et al., 2014; Desteghe et al., 2017), and three (Brox et al., 2017) and 4 years (Schwenk et al., 2014). Exergames (i.e., serious games used for balance and fall risk training) were used in most of the motor training (Kizony et al., 2006; Schwenk et al., 2014; Wüest et al., 2014; Kiselev et al., 2015; Sáenz-de-Urturi et al., 2015; Shubert et al., 2015; Brox et al., 2017; Trombetta et al., 2017; Money et al., 2019; Rebsamen et al., 2019; van Beek et al., 2019), physiotherapy exercise in others (Epelde et al., 2014; Im et al., 2015), cognitive-physical dual-task in Pedroli et al. (2018), neuropsychological testing in three studies (Fordell et al., 2011; Corno et al., 2014; Plechatá et al., 2019; Valladares-Rodriguez et al., 2019), gesture therapy in Morán et al. (2015) and Vanbellingen et al. (2017), and psychosocial support/educational in four studies (Tsai et al., 2012; Castilla et al., 2013; Cook and Winkler, 2016; Desteghe et al., 2017).
Which Are the Theories and Tools Used to Assess VR?
A key component regarding the use and acceptance of technology is understanding elements that facilitate or reduce its use in terms of human factors, not only in terms of technical ones (Wildenbos et al., 2018). An architecture structure model, the user remote console (URC), was used as a theoretical background to design the physical training in Epelde et al. (2014), whereas the majority of the studies used psychological models to develop the VR systems. Technology acceptance model (TAM) and modified versions were used in most of the studies (Tsai et al., 2012; Wüest et al., 2014; Morán et al., 2015; Cook and Winkler, 2016; Money et al., 2019; Rebsamen et al., 2019; Valladares-Rodriguez et al., 2019), transformation of flow (ToF) in Pedroli et al. (2018), UX playability in two studies (Sáenz-de-Urturi et al., 2015; Valladares-Rodriguez et al., 2019), and user-centered design model (UCD) in Kiselev et al. (2015) and Brox et al. (2017). Importantly, some studies adopted technological theoretical frameworks adapted for older people, such as the senior UCD or the senior citizens' acceptance of information systems (SCAIS) (Brox et al., 2017; Nikitina et al., 2018). However, several studies did not report a theoretical model to design their systems (Kizony et al., 2006; Castilla et al., 2013; Corno et al., 2014; Schwenk et al., 2014; Im et al., 2015; Shubert et al., 2015; Desteghe et al., 2017; Trombetta et al., 2017; Vanbellingen et al., 2017; Plechatá et al., 2019; van Beek et al., 2019). Concerning the assessment of usability of VR systems, a wide range of quantitative and qualitative methods have been used (see Table 1 for specific information and Figures 5–7 for models and methods overviews). Concerning quantitative data, system usability scale (SUS) (Corno et al., 2014; Sáenz-de-Urturi et al., 2015; Shubert et al., 2015; Vanbellingen et al., 2017; Nikitina et al., 2018; Pedroli et al., 2018; Money et al., 2019; Rebsamen et al., 2019; van Beek et al., 2019), TAM-based questionnaires (Tsai et al., 2012; Wüest et al., 2014; Morán et al., 2015; Cook and Winkler, 2016; Rebsamen et al., 2019; Valladares-Rodriguez et al., 2019), UX questionnaires (Schwenk et al., 2014; Sáenz-de-Urturi et al., 2015; Brox et al., 2017; Desteghe et al., 2017; Rebsamen et al., 2019), UCD-based questionnaire (Brox et al., 2017), flow of experience scale (Pedroli et al., 2018), other usability questionnaires (Fordell et al., 2011; Epelde et al., 2014; Trombetta et al., 2017; Plechatá et al., 2019; Valladares-Rodriguez et al., 2019), adherence or motivation to training questionnaires (Im et al., 2015; Desteghe et al., 2017; Vanbellingen et al., 2017; Nikitina et al., 2018; Rebsamen et al., 2019; van Beek et al., 2019) or with VR data (Wüest et al., 2014; Kiselev et al., 2015; Desteghe et al., 2017; Vanbellingen et al., 2017; Nikitina et al., 2018; Rebsamen et al., 2019; van Beek et al., 2019), cybersickness assessment (Corno et al., 2014; Im et al., 2015; Plechatá et al., 2019), technology expertise (Corno et al., 2014; Rebsamen et al., 2019; Valladares-Rodriguez et al., 2019), and video analysis (Morán et al., 2015) were used. Regarding qualitative data, think aloud technique (Corno et al., 2014; Wüest et al., 2014; Shubert et al., 2015; Money et al., 2019; Rebsamen et al., 2019), heuristic evaluation or cognitive walkthrough (Castilla et al., 2013; Sáenz-de-Urturi et al., 2015), focus group (Castilla et al., 2013; Epelde et al., 2014; Kiselev et al., 2015; Brox et al., 2017; Desteghe et al., 2017), and semi-structured or structured usability post-experience interviews (Corno et al., 2014; Kiselev et al., 2015; Shubert et al., 2015; Brox et al., 2017; Vanbellingen et al., 2017; Pedroli et al., 2018; Money et al., 2019; van Beek et al., 2019) were used. The sense of presence was assessed only in three studies (Kizony et al., 2006; Nikitina et al., 2018; Pedroli et al., 2018).
Concerning the tools used, a variety of quantitative and qualitative methods are reported. However, it is important to remember that each of these instruments assess different aspects of usability and acceptance; some are more concerned about the task to perform (e.g., SUS) while others tap the emotional/motivational elements of the interaction (e.g., UX questionnaires) or the factors that hamper/facilitate the use of a technology (e.g., TAM-based tools). Qualitative tools are able to grasp different perspectives (individual or group) of the experience or the design by asking experts in the sector or the end-user itself. A multidimensional approach emerged in our search and should be preferred when selecting assessment tools.
Are VR Clinical Systems for the Older People Usable?
In this section we outlined the findings of the included studies, reporting their strengths and weaknesses. Figure 8 shows mean and standard deviation for the available SUS scores, which display moderate to acceptable usability despite some cases of wide variation.
Cook and Winkler (2016) showed that OA find virtual environments (VE) from Second Life (SL) as feasible and applicable for healthcare purposes, especially for improving social interactions. Despite a high number of drop-outs, participants liked the realism and virtual experience (e.g., sports, changing avatar, teleporting, shopping) but bugs frustrated them and they found it hard to control the avatar and to learn SL. According to users, SL might be improved by clear training (i.e., individualized, small group), step-by-step teaching, by enlarging the screen, and facilitating the interaction. The exergame Falls Sensei was rated as engaging and usable for educating OA about risk fall (Money et al., 2019). Falls Sensei was rated as having a good usability (score SUS > 70, Bangor et al., 2009), especially by older users. Unified theory of acceptance and use of technology (UTAUT) thematic analysis on interviews (i.e., performance expectancy, effort, social influence) showed that users rated the training as a useful, positive experience, relevant for specific populations. Similarly, the Positive Bike (Pedroli et al., 2018) was rated as having good usability (mean SUS = 76.88, SD =17). Problems were found concerning the size of items on the screen and low realism or interaction users felt in the VE, but still had a positive experience and found the system useful. Stand Tall (ST) (Shubert et al., 2015) was rated by participants as having a nearly good usability (mean SUS = 65.5, SD = 21.2) and agreed in using ST to improve balance autonomously and accepted the Kinect sensor and the avatar. Senso system (Rebsamen et al., 2019) had high adherence, usability (mean SUS = 93.5, SD = 5.52), enjoyment, usefulness, and acceptability, also confirmed by think aloud technique. Similarly, van Beek et al. (2019) found optimal adherence and motivation toward their VR training. Despite some interaction issues with LMC and difficulty of the exercises, the system had marginal usability (mean SUS = 58.25, SD = 17.9) and was also rated positively at the interviews. Lineage was evaluated with high satisfaction by its users (Sáenz-de-Urturi et al., 2015). Gaming experience was positive, exercise adequate, and participants stated that they would use the game again. SUS improved across the three sessions (first mean SUS = 73.84, SD = 4.72; third mean SUS = 86.25, SD = 3.06). Acceptable usability was reported by OA and stroke patients for the TheraGame (mean SUS = 73.8, SD = 14.5) that also found the VR training adequate and enjoyable (Kizony et al., 2006). Good usability (first session mean SUS = 75.4, SD =13.8) was found by Vanbellingen et al. (2017) in their upper limb video game with a leap motion controller; however, usability did not change across the nine sessions. The training had a compliance of 87.4% and the adherence was rated as very good and remained stable across time. Users expressed that a 30 min session is the best time to not overload arm fatigue. Optimal (100%) adherence and good acceptance (e.g., ease, usefulness, intention to use) were found by Wüest et al. (2014). Nikitina et al. (2018) found that usability of the virtual gym App did not differ between groups with social interaction (mean SUS = 63, SD = 9) or interaction with coach only (mean SUS = 66, SD = 14). Moreover, the participants positively accepted the app, with high co-presence for the interaction group (interactions occurred especially with private messages), but adherence was similar for individual vs. group exercises with social support predicating adherence when social connections are low. Despite Corno et al. (2014) finding that virtual-multitasking test (V-MT) induces cybersickness symptoms, it was rated as usable (mean SUS = 69.17, SD = 8.2), the head-mounted display (HMD) was comfortable, interaction with the wand was difficult, instructions hard to remember, and realism sufficient. Similar results on HMD were found by Plechatá et al. (2019). HMD lead to the worst memory performances compared to non-immersive VR in OA, with users preferring neither desktop-based VR nor immersive VR, whereas young users liked immersive versions of the virtual supermarket shopping task (vSST). However, authors suggest non-immersive scenarios for OA. Fordell et al. assessed VR-DiSTRO, an immersive VR version of “paper and pencil” neglect neuropsychological battery, and showed that stroke patients tolerated and were engaged during the assessment, which was much faster than the classic evaluation (Fordell et al., 2011).
In order to design Game Up exergames and a senior-UCD model (Brox et al., 2017), it is crucial to involve older people and experts to create safe, fun, and usable games. Three-point Likert scale short questionnaires are suggested for end evaluations, whereas in the requirement, design, and implementation phases, interviews, observations, and group discussions are preferred for senior UCD. Similarly, in order to develop the Butler app (Castilla et al., 2013) it is important to gather information from end-users and experts from the first stage of the development and to create prototypes of the app. Graphics and navigations systems must be adequate and understandable for older people in order to reduce mental load. In the same way, the Health Buddies app (Desteghe et al., 2017) was initially designed with the end-users (AF patients and grandchildren). Participants, especially patients, were motivated to use the app but its usage decreased across 90 days. Despite adherence improving only in one patient, the UX with the app was easy to use and educational, and 60% of patients would use the app again. Experts and end-users of a joint rehabilitation virtual therapy were also involved in the evaluation phase in Epelde et al. (2014). Medical professionals and patients positively accepted the virtual therapist and training but patients stated that the avatar was too serious and lacked empathy. A team of experts developed an augmented reality exergame (Im et al., 2015), which did not have any side effects (e.g., cybersickness) and led to high adherence to the training.
The Interactive Trainer (Kiselev et al., 2015), despite some technical problems being reported, was evaluated according to interviews as easy to use, challenging, and motivating. Schwenk et al. (2014) assessed the gaming UX of a exergame with sensors, which was found to be effective, fun, easy to learn thanks to feedback, adequate, and well-designed. Interestingly, Valladares-Rodriguez et al. (2019) aimed at assessing UX and player eXperience (PX) of Panoramix neuropsychological touchscreen battery in OA, mild cognitive impairment (MCI), and Alzheimer's disease individuals. They found that Panoramix perception and acceptance were positive after the pilot study in the groups but was judged as more playable by OA, MCI, and AD in this order; nevertheless, PX improved after the second interaction in all groups. Additionally, administrators also evaluated the battery as playable, usable, useful, and with a good interface. Morán et al. (2015) used a TAM-based questionnaire and video analysis to assess usability. Users rated the VR gesture therapy (GT) as useful, easy, and with high UX and found that even technological expertise did not affect task performance. By analyzing verbal and non-verbal reactions, raters judged the system as more usable and fun for non-expert participants. Conversely, anxiety was low for expert users. Authors defined two approach strategies according to expertise, explore-and-learn and score-and-complete, respectively, for inexperienced and experienced participants that guided behaviors (e.g., anxiety, interaction strategies with the games) and reactions through the experience.
A comparison of semi vs. full immersive versions of Motion Rehab AVE 3D was done by Trombetta et al. (2017). Training was feasible for users and participants evaluated as important for usability feedback, third-person perspective, comfort (semi-immersion version), and immersion (full immersion). Authors suggested that, for post-stroke rehabilitation, semi-immersive systems are more comfortable than full-immersive VR. Tsai et al. (2012) showed that Sharetouch is a well-designed, easy, and usable system, independent of gender or age, and facilitates social interactions in OA. Importantly, significant effects of the rehabilitative training on different motor/physical measures were found in all the studies that tested efficacy and usability (Schwenk et al., 2014; Wüest et al., 2014; Im et al., 2015; Vanbellingen et al., 2017; Rebsamen et al., 2019; van Beek et al., 2019). However, risk of bias (see Supplementary Figure 1) is high for most of the categories (randomization, allocation, blinding, missing data, and reporting bias), as the majority of the research is quasi or non-experimental. Of note, the risk of incomplete data outcome was low.
In general, despite some technical weaknesses (e.g., realism, bugs), interaction constraints and physical/psychological barriers to technology use, the included VR studies showed that with adequate usability design methods, it is possible to develop effective and usable systems for clinical purposes in aging.
Discussion
In the present paper we reviewed the current research on usability, user experience (UX), and feasibility of virtual reality (VR) clinical systems in older people.
Our work can be summarized in the following points: (1) most of the usability pilots involved healthy or heterogeneous diseased older people; (2) usability mainly concerned VR physiotherapy training; (3) most of the studies involved non-immersive scenarios; (4) quantitative (e.g., SUS) and qualitative (e.g., interviews) methods are the most used and suggested approach in usability piloting and technology acceptance model (TAM) is the main theoretical framework; (5) despite some interaction issues, VR systems are rated as having good usability by end-users.
Usability is a critical and complex task when specific end-users with particular needs are involved. Conditions that hamper the interaction with the device (Wildenbos et al., 2018), and also cultural and technology background, should be taken into account (Corno et al., 2014; Nikitina et al., 2018). For instance, Tuena et al. found that executive functions are overloaded by input device use in older people and this leads to worse memory performances (Tuena et al., 2019). Design guidelines should be used to avoid basic sensorimotor and interaction issues (see Phiriyapokanon, 2011; Loureiro and Rodrigues, 2014).
If, on the one hand, the studies included collected data from the target population (e.g., Parkinson's disease patients tested usability for Parkinson's disease rehabilitation), several others assessed usability with healthy older people or mixed-pathologies patients (e.g., Wüest et al., 2014; Sáenz-de-Urturi et al., 2015; Shubert et al., 2015; Trombetta et al., 2017; Vanbellingen et al., 2017); in this sense, diagnostic criteria were not clear or end-users characteristic do not match potential technology barriers of end-users. Future research should use strict inclusion/exclusion criteria according to diagnostic criteria of the diseases or syndromes. Moreover, in the context of healthcare, the end-users are also the medical professionals that use the technology with the patients. Usability should be assessed via questionnaires or interviews in the design and test phases (e.g., Castilla et al., 2013; Valladares-Rodriguez et al., 2019). Finally, despite some studies reporting the number of participants as a limitation (Corno et al., 2014; Desteghe et al., 2017; Vanbellingen et al., 2017; van Beek et al., 2019), a number of 5–10 individuals is sensible enough to identify a minimum of 80% circa of usability issues (Wüest et al., 2014; Brox et al., 2017).
The uses of VR systems in our review were mainly focused on motor rehabilitation. In healthcare, VR is mainly applied for the assessment and rehabilitation of sensorimotor, physical, and psychological deficits via non-immersive to immersive technologies (Lange et al., 2010; Bohil et al., 2011; García-Betances et al., 2015; Muratore et al., 2019; Tuena et al., 2019). We also encourage the use of pilot studies in other domains where VR is used for clinical purposes. For instance, it is important to evaluate usability of assessment tools (e.g., Pedroli et al., 2015; Desteghe et al., 2017). Mean usability session testing lasted 30 min; nevertheless, depending on the aims of the studies (e.g., memorability), longitudinal usability studies can be done as usability might improve after some sessions (Valladares-Rodriguez et al., 2019). Lastly, future research should focus more on immersive technology as technical development will lead to new forms of immersive VR and costs will be reduced. It is important to also assess these systems because they might lead to reduced cybersickness compared to desktop-based VR (Lange et al., 2010; Bohil et al., 2011; Plechatá et al., 2019).
Several studies (see Table 1) did not report a model on which usability and acceptance of a technology can be assumed. TAM-based and UX-based are useful for investigating and understanding psychological factors, whereas architecture design and user remote control (URC) are more useful for technical development. Indeed, usability, and in particular UX, are devoted not only to the ease of use and the technical bugs but also to the psychological domains (e.g., emotions, motivations; Vermeeren et al., 2010). However, as researchers in the context of aging face specific needs and barriers, adapted models with relevant variables should be used as the senior user-centered design (UCD) by Brox et al. (2017) or the senior citizens' acceptance of information systems (SCAIS) by Phang et al. (2006). Surprisingly, none of the authors used the senior technology acceptance model (STAM) by Chen and Shou (2014), which could be more suitable than TAM models not adapted to older people. Interestingly clinical researchers interested in technology usability, sense of presence, and clinical change may want to use the transformation of flow (ToF) theory, as presence and flow experiences might facilitate clinical change by means of VR (Riva et al., 2006).
Usability assessment (see Table 1 and Figure 7) tools should include a mix of quantitative methods (e.g., SUS, TAM-based questionnaires, UX-based questionnaires) and qualitative techniques (e.g., experience interviews, think aloud, heuristic evaluation). The systematic review on telemedicine systems by Klaassen et al. (2016) recommend SUS, TAM2, and PSSUQ and state that questionnaires along with interviews, which are both low-cost and flexible methods, can be used from early to final phases of usability. Indeed, questionnaires give useful quantitative data that, however, still need qualitative information to tap individual sources of variation. Therefore, a mixed approach composed of quantitative and qualitative tools is the preferred way to carry out complete, interpretable, and useful usability studies in older people. Additionally, we encourage a critical adoption of assessment tools according to the aims of the study, thus considering the aspects (e.g., individual, group, task, emotions/motivation, acceptance, adherence) to be engaged during the VR interaction.
Additionally, innovative quantitative techniques could be useful to track unexpected information about psychophysiological (e.g., eye-tracking, heart-rate, galvanic skin response, non-verbal communication) responses of the users to assess their affective and cognitive reactions to the VR system (Morán et al., 2015; Sáenz-de-Urturi et al., 2015). VR can also be used for evaluating usability and adherence (good >80%) by using time spent, number of log-ins, or interaction modality, giving additional quantitative data (Cipresso, 2015; Rebsamen et al., 2019; van Beek et al., 2019). Importantly, when testing immersive VR, cybersickness should always be assessed because it may negatively influence clinical practice and its reduction is a key objective of pilot studies (Kober et al., 2013; Corno et al., 2014; Tuena et al., 2017; Plechatá et al., 2019) and virtual embodiment with questionnaires if avatars are used (Kilteni et al., 2012; Gonzalez-Franco and Tabitha, 2018). Finally, in the early design phases, information from end-users (e.g., patients, medical professionals) could be gathered from group interviews or focus groups, where ideas from experts' opinions and needs can be used to guide VR development (Castilla et al., 2013; Brox et al., 2017). For instance, Brox et al. (2017) developed a senior-UCD with a mixed use of quantitative and qualitative methods to design a semi-immersive exergame for older people, through iteration from the early phases to the prototype. Researchers should be aware that step-by-step UCD (e.g., prototype development) and pretesting are critical for clinical VR settings (Novak, 2008; Im et al., 2015). However, we know that time is a limitation to some research projects and, in some occasions, there is no time for longitudinal and proper VR design. When this is not possible, we strongly encourage the use of qualitative and quantitative evaluation of the VR experience. In the same manner, it would be better to assess usability and acceptance separately from efficacy of a VR system, as quality of patients' healthcare services is intertwined with usability, acceptance, and adherence (Middleton et al., 2013).
Despite some technical and interaction issues (e.g., bugs, interaction difficulties, realism, sensors application), the included studies showed that usability of a wide range of VR clinical systems is good, well-accepted, adequate, effective, and useful. Skepticism of older people and digital divide are walls that could be successfully broken after the use of VR devices (Desteghe et al., 2017) and comfort of immersive VR can be improved by replacing visors with CAVE, although non-HMD systems are considered better for older people (Corno et al., 2014; Pedroli et al., 2018; Plechatá et al., 2019). Nevertheless, a recent study shows that OA positively accept and tolerate HMD VR (Huygelier et al., 2019). Indeed, Fordell et al. showed that stroke patients enjoyed the immersive VR assessment (Fordell et al., 2011). However, a future objective in the field is to make sensors application and use easier for this population, as home-based training, where no professional is present to provide assistance, is rising in popularity in VR clinical practice (Schwenk et al., 2014). Moreover, online assistance could be useful to help patients with set-up and exercises (Im et al., 2015; Nikitina et al., 2018). Morán et al. (2015) provided some guidelines concerning the feedback the VR training should give to older users:
• “Provide timely feedback on successful actions in a simple and salient manner”;
• “Provide feedback on erroneous actions in a simple and salient manner”;
• “Provide simple and salient instructions on how to recover or solve an error”;
• “Provide feedback that fosters or inhibits specific behaviors from the user in a salient and concise manner.”
Additionally, Teo et al. (2016) provide specific suggestions in their review for VR training in individuals with stroke-related impairments, such as flexible activity according to patients' objectives, possibility to adapt online the task by the therapist according to patient's needs, multiplayer services, and automated recording of patient tracking. Moreover, Teo et al. (2016) show that VR can be enriched with neurophysiological tools (e.g., EEG, fNIRS) that the researcher or the clinician can use to adapt the task according to individual effort or needs.
Finally, it is worth mentioning some solutions provided by the Cochrane guidelines (Higgins et al., 2011) that avoid risk of bias in usability experiments. Despite blinding procedures in cognitive/motor rehabilitation trials (VR vs. treatment as usual) being a hard task to fulfill, still randomization, attrition bias, and reporting bias can be improved, respectively, with random number generators, shuffling cards, or throwing dice, with adequate missing data manipulation (e.g., balanced observation, imputation) and via adequate hypotheses and primary/secondary outcomes specification in the introduction and then in the discussion and adequate analyses in the result section.
The present review outlined current VR usability piloting issues and strengths in healthy aging and age-related clinical conditions. In the following paragraph, we will provide suggestions for researchers who wish to run usability testing in the context of clinical application of VR systems for older patients.
VR-usability Suggestions for the Older People (VR-USOP)
In the present paragraph we presented some suggestions we derived from findings of the systematic review. VR-USOP will be mainly focused on human-interaction factors rather than on technical aspects of developing VR clinical systems. Table 2 summarizes some suggestions in four steps to follow if researchers and clinicians wish to design and test their VR clinical apparatus to older end-users.
Table 2. VR-USOP.
The assessment of potential barriers and facilitators of the end-users, which can also include the medical professionals and technology acceptance models, is the first step. In our opinion, this is crucial as it allows the identification and the development of adequate characteristics of VR interaction and task (step 2). The latter aspects will be provided also by adopting architecture design, senior-UCD, and guidelines and prototyping, thus allowing the definition of usability assessment. In addition, we encourage ameliorating the methodology (risk of bias, see Supplementary Figure 1; i.e., randomization, allocation, blinding, manipulation of missing data, and reporting bias) to overcome the limitations of the available studies analyzed in the present review. VR usability and acceptance assessment should be defined and developed in accordance to the aims of the study (step 3). We suggest a mixed-approach with quantitative and qualitative methods (mainly focused on psychological experience of usability) and additional aspects to consider (see Table 2). Lastly, we suggest ensuring usability before clinical testing (step 4).
Conclusions
This systematic review aimed at describing an overview of state of the art VR clinical systems for older people in relation to usability and providing researchers with suggestions based on the results of the review. Despite some limitations concerning the criteria used to recruit the samples, the low number of immersive technologies so far tested, and the high risk of bias of the studies, VR systems show good usability and acceptance among older people. A wide variety of quantitative and qualitative methods can be used to evaluate usability. We suggest adopting mixed-methodology with appropriate tools in order to grasp different aspects of the usability, acceptability, and user experience and to plan sessions according to objectives of usability. Piloting is a critical aspect of clinical studies with VR technology and we encourage future research to test usability of their applications following VR-USOP.
Data Availability Statement
The datasets generated for this study are available on request to the corresponding author.
Author Contributions
CT wrote the first draft of the manuscript. MS-B supervised and wrote the following drafts of the manuscript. MS-B, FG, CT, EP, AGal, and PT defined the methodology and objectives of the manuscript. MC assessed risk of bias and made the second search strategy. AGag gave framework for the VR-USOP. KG provided clinical expertise and support. GR and FL revised the manuscript. All authors contributed to the revision and final approval of the manuscript.
Funding
This work was funded by the Italian Ministry of Health IRCCS Network on Aging Research roadmap on aging and age-related diseases RRC-2018-2365820.
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Acknowledgments
CT wishes to thank all the authors for providing useful ideas and supervision during the conceptualization, writing, and revision of the manuscript.
Supplementary Material
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fnhum.2020.00093/full#supplementary-material
Supplementary Figure 1. Risk of bias assessment.
References
Abigail, M. M., Campbell, S., Chew-Graham, C., McNally, R., and Cheraghi-Sohi, S. (2014). PICO, PICOS and SPIDER: a comparison study of specificity and sensitivity in three search tools for qualitative systematic reviews. BMC Health Serv. Res. 14:579. doi: 10.1186/s12913-014-0579-0
Bangor, A., Kortum, P., and Miller, J. (2009). Determining what individual SUS scores mean: adding an adjective rating scale. J. Usability Stud. 4, 114–123. doi: 10.5555/2835587.2835589
Beard, J. R., Officer, A. M., and Cassels, A. K. (2016). The world report on ageing and health. J. Gerontol. B. Psychol. Sci. Soc. Sci. 56, S163–S166. doi: 10.1093/geront/gnw037
Bohil, C. J., Alicea, B., and Biocca, F. A. (2011). Virtual reality in neuroscience research and therapy. Nat. Rev. Neurosci. 12, 752–762. doi: 10.1038/nrn3122
Brooke, J. (1986). “SUS: A “quick and dirty” usability scale,” in Usability Evaluation in Industry. eds P. W. Jordan, B. Thomas, B. A. Weerdmeester, and A. L. McClelland (London: Taylor & Francis), 189–194.
Brox, E., Konstantinidis, S. T., and Evertsen, G. (2017). User-centered design of serious games for older adults following 3 years of experience with exergames for seniors : a study design. JMIR Serious Game 5:e2. doi: 10.2196/games.6254
Castilla, D., Garcia-Palacios, A., Breto, J., Mari, R., Etchemendy, E., Farfallini, L., et al. (2013). Process of design and usability evaluation of a telepsychology web and virtual reality system for the elderly: Butler. Int. J. Hum. Comput. Stud. 71, 350–362. doi: 10.1016/j.ijhcs.2012.10.017
Chen, K., and Shou, A. H. C. (2014). Gerontechnology acceptance by elderly Hong Kong Chinese : a senior technology acceptance model (STAM). Ergonomics 57, 635–652. doi: 10.1080/00140139.2014.895855
Cipresso, P. (2015). Modeling behavior dynamics using computational psychometrics within virtual worlds. Front. Psychol. 6:1725. doi: 10.3389/fpsyg.2015.01725
Cook, N., and Winkler, S. L. (2016). Acceptance, usability and health applications of virtual worlds by older adults : a feasibility study. JMIR Res. Protoc. 5:e81. doi: 10.2196/resprot.5423
Corno, G., Bouchard, S., and Forget, H. (2014). Usability assessment of the virtual multitasking test (v-mt) for elderly people. Annual Rev. Cyberther. Telemed. 199, 168–172.
Davis, F. D., Bagozzi, R. P., and Warshaw, P. R. (1989). User acceptance of computer technology: a comparison of two theoretical models. Manag. Sci. 35, 982–1003. doi: 10.1287/mnsc.35.8.982
Desteghe, L., Kluts, K., Vijgen, J., Koopman, P., Dilling-Boer, D., Schurmans, J., et al. (2017). The health buddies app as a novel tool to improve adherence and knowledge in atrial Fibrillation patients : a pilot study. JMIR Mhealth Uhealth 5:e98. doi: 10.2196/mhealth.7420
Dubois, B., Feldman, H. H., Jacova, C., Cummings, J. L., DeKosky, S. T., Barberger-Gateau, P., et al. (2010). Revising the definition of Alzheimer's disease: a new lexicon. Lancet Neurol. 9, 1118–1127. doi: 10.1016/S1474-4422(10)70223-4
Edwards, M. H., Dennison, E. M., Sayer, A. A., Fielding, R., and Cooper, C. (2015). Osteoporosis and sarcopenia in older age. Bone 80, 126–130. doi: 10.1016/j.bone.2015.04.016
Epelde, G., Carrasco, E., Rajasekharan, S., Jimenez, M., Vivanco, K., Gomez-Fraga, I., et al. (2014). Universal remote delivery of rehabilitation : validation with seniors ' joint rehabilitation therapy. Cybern. Syst. 45, 109–122. doi: 10.1080/01969722.2014.874807
Fang, W., Zheng, L., Deng, H., and Zhang, H. (2017). Real-time motion tracking for mobile augmented/virtual reality using adaptive visual-inertial fusion. Sensors 17:1037. doi: 10.3390/s17051037
Fedarko, N. S. (2012). The biology of aging and frailty. Clin. Geriat. Med. 27, 27–37. doi: 10.1016/j.cger.2010.08.006
Finstad, K. (2010). The usability metric for user experience. Interact. Comput. 22, 323–327. doi: 10.1016/j.intcom.2010.04.004
Fishbein, M., and Ajzen, I. (1975). Belief, Attitude, Intention, and Behavior: An Introduction to Theory and Research. Reading, MA: Addison-Wesley.
Fordell, H., Bodin, K., Bucht, G., and Malm, J. (2011). A virtual reality test battery for assessment and screening of spatial neglect. Acta Neurol. Scand. 123, 167–174. doi: 10.1111/j.1600-0404.2010.01390.x
Franceschi, C., Garagnani, P., Morsiani, C., Conte, M., Santoro, A., Grignolio, A., et al. (2018). The continuum of aging and age-related diseases :common mechanisms but different rates. Front. Med. 5:61. doi: 10.3389/fmed.2018.00061
Fried, L. P., Tangen, C. M., Walston, J., Newman, A. B., Hirsch, C., Gottdiener, J., et al. (2001). Frailty in older adults : evidence for a phenotype. J. Gerontol. 56, 146–157. doi: 10.1093/gerona/56.3.M146
García-Betances, R. I., Arredondo Waldmeyer, M. T., Fico, G., and Cabrera-Umpiérrez, M. F. (2015). A succinct overview of virtual reality technology use in Alzheimer's disease. Front. Aging Neurosci. 7:80. doi: 10.3389/fnagi.2015.00235
Garçon, L., Khasnabis, C., Walker, L., Nakatani, Y., Lapitan, J., Borg, J., et al. (2016). Medical and assistive health technology : meeting the needs of aging populations. J. Gerontol. A Biol. Soc. Sci. Med 56, S293–S302. doi: 10.1093/geront/gnw005
Gonzalez-Franco, M., and Tabitha, C. P. (2018). Avatar embodiment. Towards a standardized questionnaire. Front. Rob. AI 5:74. doi: 10.3389/frobt.2018.00074
Gorini, A., and Riva, G. (2008). Virtual reality in anxiety disorders: the past and the future. Expert Rev. Neurother. 8, 215–233. doi: 10.1586/14737175.8.2.215
Hassenzahl, M., and Tractinsky, N. (2006). User experience - a research agenda. Behav. Inform. Technol. 25, 91–97. doi: 10.1080/01449290500330331
Higgins, J. P., Altman, D. G., Gøtzsche, P. C., Jüni, P., Moher, D., Oxman, A. D., et al. (2011). The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ 343:d5928. doi: 10.1136/bmj.d5928
Huang, P. L. (2009). A comprehensive definition for metabolic syndrome. Dis. Model. Mech. 2, 231–237. doi: 10.1242/dmm.001180
Huygelier, H., Schraepen, B., van Ee, R., Abeele, V. V., and Gillebert, C. R. (2019). Acceptance of immersive head-mounted virtual reality un older adults. Sci. Rep. 9:4519. doi: 10.1038/s41598-019-41200-6
Im, D. J., Jeunghun, K., Joon, K. Y., Sangwoo, C., Kyung, C. Y., Teo, L., et al. (2015). Utility of a three-dimensional interactive augmented reality program for balance and mobility rehabilitation in the elderly :a feasibility study. Ann. Rehabil. Med. 39, 462–472. doi: 10.5535/arm.2015.39.3.462
ISO (2016a). Ergonomics of Human-System Interaction—Part 210: Human-Centred Design For Interactive Systems.
ISO (2016b). SPREAD: Linee Guida Italiane di Prevenzione e Trattamento. Retrieved from: http://www.iso-stroke.it/?page_id=200 (accessed October 15, 2019).
Kilteni, K., Groten, R., and Slater, M. (2012). The sense of embodiment in virtual reality. Presence Teleo. Virt. Environ. 21, 373–387. doi: 10.1162/PRES_a_00124
Kiselev, J., Haesner, M., Gövercin, M., and Steinhagen-Thiessen, E. (2015). Implementation of a home-based interactive training. J. Gernontol. Nurs. 41, 14–19. doi: 10.3928/00989134-20141201-01
Kizony, R., Weiss, P. L., Shahar, M., and Rand, D. (2006). Theragame: a home based virtual reality rehabilitation system. Int. J. Dis. Hum. Develop. 5, 265–270. doi: 10.1515/IJDHD.2006.5.3.265
Klaassen, B., van Beijnum, B. J. F., and Hermens, H. (2016). Usability in telemedicine systems – a literature survey. Int. J. Med. Inform. 93, 57–69. doi: 10.1016/j.ijmedinf.2016.06.004
Kober, S. E., Wood, G., Hofer, D., Kreuzig, W., Kiefer, M., and Neuper, C. (2013). Virtual reality in neurologic rehabilitation of spatial disorientation. J. Neuroeng. Rehabil. 10:17. doi: 10.1186/1743-0003-10-17
Kuehn, E., Perez-lopez, M. B., Diersch, N., Döhler, J., Wolbers, T., and Riemer, M. (2017). Embodiment in the aging mind. Neurosci. Biobehav. Rev. 86, 207–225. doi: 10.1016/j.neubiorev.2017.11.016
Lange, B. S., Requejo, P., Flynn, S. M., and Rizzo, A. A. (2010). The potential of virtual reality and gaming to assist successful aging with disability. Phys. Med. Rehabil. Clin. 21, 339–356. doi: 10.1016/j.pmr.2009.12.007
Laugwitz, B., Schrepp, M., and Held, T. (2008). “Construction and evaluation of a user experience questionnaire,” in Symposium of the Austrian HCI and Usability Engineering Group (Berlin: Springer), 63–76. doi: 10.1007/978-3-540-89350-9_6
Lee, K. M. (2004). Why presence occurs: evolutionary psychology, media equation, and presence. Presence 13, 494–505. doi: 10.1162/1054746041944830
Loureiro, B., and Rodrigues, R. (2014). “Design guidelines and design recommendations of multi-touch interfaces for elders,” in Proceedings of the ACHI (Barcelona), 41–47.
Lutz, W., Sanderson, W., Scherbov, S., and Asia, S. (2008). The coming acceleration of global population ageing. Nature 451, 716–719. doi: 10.1038/nature06516
Marengoni, A., Angleman, S., Melis, R., Mangialasche, F., Karp, A., Garmen, A., et al. (2011). Aging with multimorbidity : a systematic review of the literature. Ageing Res. Rev. 10, 430–439. doi: 10.1016/j.arr.2011.03.003
Middleton, B., Bloomrosen, M., Dente, M. A., Hashmat, B., Koppel, R., Overhage, J. M., et al. (2013). Enhancing patient safety and quality of care by improving the usability of electronic health record systems: recommendations from AMIA. J. Am. Med. Inform. Assoc. 20, e2–e8. doi: 10.1136/amiajnl-2012-001458
Milgram, P., Takemura, H., Utsumi, A., and Kishino, F. (1995). “Augmented reality: a class of displays on the reality-virtuality continuum,” in SPIE 2351 Telemanipulator and Telepresence Technologies (Boston, MA), 2351. doi: 10.1117/12.197321
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., and Grp, P. (2009). Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement (reprinted from annals of internal medicine). Phys. Ther. 89, 873–880. doi: 10.1093/ptj/89.9.873
Money, A. G., Atwal, A., Boyce, E., Gaber, S., Windeatt, S., and Alexandrou, K. (2019). Falls Sensei :a serious 3D exploration game to enable the detection of extrinsic home fall hazards for older adults. BMC Med. Inform. Decis. Mak. 19:85. doi: 10.1186/s12911-019-0808-x
Morán, A. L., Ramírez-Fernández, C., Meza-Kubo, V., Orihuela-Espina, F., García-Canseco, E., Grimaldo, A. I., et al. (2015). On the effect of previous technological experience on the usability of a virtual rehabilitation tool for the physical activation and cognitive stimulation of elders. J. Med. Syst. 39:104. doi: 10.1007/s10916-015-0297-0
Mujber, T. S., Szecsi, T., and Hashmi, M. S. J. (2004). Virtual reality applications in manufacturing process simulation. J. Mat. Process. Technol. 156, 1834–1838. doi: 10.1016/j.jmatprotec.2004.04.401
Muratore, M., Tuena, C., Pedroli, E., Cipresso, P., and Riva, G. (2019). Virtual reality as a possible tool for the assessment of self-awareness. Front. Behav. Neurosci. 13:62. doi: 10.3389/fnbeh.2019.00062
Nielsen, J. (1994). “Usability inspection methods,” in Usability Inspection Methods, eds J. Nielsen and R. L. Mack (New York, NY: John Wiley & Sons), 413–414. doi: 10.1145/259963.260531
Nikitina, S., Didino, D., Baez, M., and Casati, F. (2018). Feasibility of virtual tablet-based group exercise among older adults in siberia : findings from two pilot. JMIR Mhealth Uhealth 6:e40. doi: 10.2196/mhealth.7531
Novak, J. (2008). Game Development Essentials: An Introduction. 3rd Edn. Clifton Park, NY: Delmar Cengage Learning.
O'Bryant, S. E., Humphreys, J. D., Smith, G. E., Ivnik, R. J., Graff-radford, N. R., Petersen, R. C., et al. (2017). Detecting Dementia with the mini-mental state examination in highly educated individuals. Arch. Neurol. 65, 963–967. doi: 10.1001/archneur.65.7.963
Pallavicini, F., Pedroli, E., Serino, S., Dell'Isola, A., Cipresso, P., Cisari, C., et al. (2015). Assessing unilateral spatial neglect using advanced technologies: the potentiality of mobile virtual reality. Technol. Health Care 23, 795–807. doi: 10.3233/THC-151039
Pedroli, E., Greci, L., Serino, S., Cipresso, P., Arlati, S., Mondellini, M., et al. (2018). Characteristics, usability, and users experience of a system combining cognitive and physical therapy in a virtual environment :positive bike. Sensors 18:2343. doi: 10.3390/s18072343
Pedroli, E., Serino, S., Cipresso, P., Pallavicini, F., Chicchi Giglioli, I., Guastafierro, E., et al. (2015). “Neglect App. Usability of a new application for assessment and rehabilitation of neglect,” in REHAB '15 Proceedings of the 3rd 2015 Workshop on ICTs for improving Patients Rehabilitation Research Techniques (Lisbon), 139–143. doi: 10.1145/2838944.2838978
Petersen, R. C. (2004). Mild cognitive impairment as a diagnostic entity. J. Intern. Med. 256, 183–194. doi: 10.1111/j.1365-2796.2004.01388.x
Phang, C. W., Sutanto, J., Kankanhalli, A., Li, Y., Tan, B. C., and Teo, H. H. (2006). Senior Citizens' acceptance of information systems: a study in the context of e-government services. IEEE Trans. Eng. Manage.53, 555–569. doi: 10.1109/TEM.2006.883710
Phiriyapokanon, T. (2011). Is a Big Button Interface Enough for Elderly Users: Towards user Interface Guidelines for Elderly Users. Kittery, ME: LAP LAMBERT Academic Publishing.
Plechatá, A., Sahula, V., Fayette, D., and Fajnerová, I. (2019). Age-related differences with immersive and non-immersive virtual reality in memory assessment. Front. Psychol. 10:1330. doi: 10.3389/fpsyg.2019.01330
Portet, F., Ousset, P. J., Visser, P. J., Frisoni, G. B., Nobili, F., Scheltens, P., et al. (2006). Report of the MCI working group of the European consortium on Alzheimer's disease. J. Neurol. Neurosur. Psychiatry 77, 714–719. doi: 10.1136/jnnp.2005.085332
Rebsamen, S., Knols, R. H., Pfister, P. B., Jung, M. E., and Elliot, C. A. (2019). Exergame-Driven high-intensity interval training in untrained community dwelling older adults : a formative one group quasi- experimental feasibility trial. Front. Physiol. 10:1019. doi: 10.3389/fphys.2019.01019
Riva, G. (2008). From virtual to real body: virtual reality as embodied technology. J. Cyber Ther. Rehabil. 1, 7–22.
Riva, G., Baños, R. M., Botella, C., Mantovani, F., and Gaggioli, A. (2016). Transforming experience : the potential of augmented reality and virtual reality for enhancing personal and clinical change. Front. Psychiatry 7:164. doi: 10.3389/fpsyt.2016.00164
Riva, G., Castelnuovo, G., and Mantovani, F. (2006). Transformation of flow in rehabilitation: the role of advanced communication technologies. Behav. Res. Methods 38, 237–244. doi: 10.3758/BF03192775
Riva, G., and Waterworth, J. A. (2014). “Being present in a virtual world,” in The Oxford Handbook of Virtuality, ed M. Grimshaw (Oxford: Oxford University Press), 205–221. doi: 10.1093/oxfordhb/9780199826162.013.015
Sáenz-de-Urturi, Z., Zapirain, B. G., and Zorrilla, A. M. (2015). Elderly user experience to improve a Kinect-based game playability. Behav. Inform. Technol. 34, 1040–1051. doi: 10.1080/0144929X.2015.1077889
Sánchez, J. L., Vela, F. L., Simarro, F. M., and Zea, N. P. (2012). Playability: analysing user experience in video games. Behav. Infor. Technol. 31, 1033–1054. doi: 10.1080/0144929X.2012.710648
Schwenk, M., Grewal, G. S., Honarvar, B., Schwenk, S., Mohler, J., Khalsa, D. S., et al. (2014). Interactive balance training integrating sensor-based visual feedback of movement performance : a pilot study in older adults. J. Neuroeng. Rehabil. 11:164. doi: 10.1186/1743-0003-11-164
Sebe, N. (2010). “Human-centered computing,” in Handbook of Ambient Intelligence and Smart Environments, (New York, NY: Springer). 349–370. doi: 10.1007/978-0-387-93808-0_13
Shubert, T. E., Basnett, J., Chokshi, A., Barrett, M., Hill, C., and Carolina, N. (2015). Are virtual rehabilitation technologies feasible models to scale an evidence-based fall prevention program? A pilot study using the kinect camera. JMIR Rehabil. Assist. Technol. 2:e10. doi: 10.2196/rehab.4776
Steenman, M., and Lande, G. (2017). Cardiac aging and heart disease in humans. Biophys. Rev. 9, 131–137. doi: 10.1007/s12551-017-0255-9
Steptoe, A., Deaton, A., and Stone, A. A. (2015). Psychological wellbeing, health and ageing. Lancet 385, 640–648. doi: 10.1016/S0140-6736(13)61489-0
Taherdoost, H. (2018). “A review of technology acceptance and adoption models and theories,” in A Review of Technology Acceptance and Adoption Models and Theories, eds L. Moldovan and A. Gligor (Tirgu Mures: Elsevier B.V.), 960–967. doi: 10.1016/j.promfg.2018.03.137
Teo, W.-P., Muthalib, M., Yamin, S., Hendy, A. M., Bramstedt, K., Kotsopoulos, E., et al. (2016). Does a combination of virtual reality, neuromodulation and neuroimaging provide a comprehensive platform for neurorehabilitation? – a narrative review of the literature. Front. Hum. Neurosci. 10:284. doi: 10.3389/fnhum.2016.00284
Tolosa, E., Wenning, G., and Poewe, W. (2006). The diagnosis of Parkinson's disease. Lancet Neurol. 5, 75–86. doi: 10.1016/S1474-4422(05)70285-4
Triberti, S., and Riva, G. (2016). Being present in action: a theoretical model about the “interlocking” between intentions and environmental affordances. Front. Psychol. 6:2052. doi: 10.3389/fpsyg.2015.02052
Trombetta, M., Paula, P., Henrique, B., Brum, M. R., Colussi, E. L., Carolina, A., et al. (2017). Motion Rehab AVE 3D: a VR-based exergame for post-stroke rehabilitation. Comput. Methods Programs Biomed. 151, 15–21. doi: 10.1016/j.cmpb.2017.08.008
Tsai, T., Chang, H., Chang, Y., and Huang, G. (2012). A system to enrich social network experiences for the elderly. J. Syst. Software 85, 1363–1369. doi: 10.1016/j.jss.2012.01.023
Tuena, C., Serino, S., Dutriaux, L., Riva, G., and Piolino, P. (2019). Virtual enactment effect on memory in young and aged populations :a systematic review. J. Clin. Med. 8:620. doi: 10.3390/jcm8050620
Tuena, C., Serino, S., Gaston-Bellegarde, A., Orriols, E., Makowski, D., Riva, G., et al. (2017). How virtual embodiment affects episodic memory functioning : a proof-of-concept study. Ann. Rev. Cyberther. Telemed. 98–103.
Valladares-Rodriguez, S., Fernández-Iglesias, M. J., Anido-Rifón, L., Facal, D., Rivas-Costa, C., and Pérez-Rodríguez, R. (2019). Touchscreen games to detect cognitive impairment in senior adults. A user- interaction pilot study. Int. J. Med. Inform. 127, 52–62. doi: 10.1016/j.ijmedinf.2019.04.012
van Beek, J. J. W., van Wegen, E. E. H., Bohlhalter, S., and Vanbellingen, T. (2019). Exergaming-based dexterity training in persons with parkinson disease: a pilot feasibility study. J. Neurol. Phys. Ther. 43, 168–174. doi: 10.1097/NPT.0000000000000278
Vanbellingen, T., Filius, S. J., Nyffeler, T., and van Wegen, E. E. (2017). Usability of videogame-based dexterity training in the early rehabilitation phase of stroke patients : a pilot study. Front. Neurol. 8:654. doi: 10.3389/fneur.2017.00654
Venkatesh, V., Morris, M. G., Davis, G. B., and Davis, F. D. (2003). User acceptance of information technology: toward a unified view. MIS Q 425–478. doi: 10.2307/30036540
Vermeeren, A. P. O. S., Law, E. L.-C., Roto, V., Obrist, M., Hoonhout, J., and Väänänen-Vainio-Mattila, K. (2010). “User experience evaluation methods : current state and development needs,” in Proceedings of the 6th Nordic Conference on Human-Computer Interaction, 521–530. doi: 10.1145/1868914.1868973
Vredenburg, K., Mao, J., Smith, P., and Carey, T. (2002). “A survey of user-centered design practice,” in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Edinburgh), 3460–3478. doi: 10.1145/503376.503460
Wildenbos, G. A., Peute, L., and Jaspers, M. (2018). Aging barriers in fl uencing mobile health usability for older adults : a literature based framework (MOLD-US). Int. J. Med. Inform. 114, 66–75. doi: 10.1016/j.ijmedinf.2018.03.012
Winstein, C. J., Stein, J., Arena, R., Bates, B., Cherney, L. R., Cramer, S. C., et al. (2016). Guidelines for adult stroke rehabilitation and recovery: a guideline for healthcare professionals from the American Heart Association/American Stroke Association. Stroke 47, e98–e169. doi: 10.1161/STR.0000000000000098
Wüest, S., Borghese, N. A., Pirovano, M., Mainetti, R., van de Langenberg, R., and de Bruin, E. D. (2014). Usability and effects of an exergame-based balance training program. Games Health J. 3, 106–114. doi: 10.1089/g4h.2013.0093
Xue, Q. (2011). The frailty syndrome: definition and natural history. Clin. Geriatr. Med. 27, 1–15. doi: 10.1016/j.cger.2010.08.009
Keywords: aging, assessment, rehabilitation, usability, user-experience, virtual reality
Citation: Tuena C, Pedroli E, Trimarchi PD, Gallucci A, Chiappini M, Goulene K, Gaggioli A, Riva G, Lattanzio F, Giunco F and Stramba-Badiale M (2020) Usability Issues of Clinical and Research Applications of Virtual Reality in Older People: A Systematic Review. Front. Hum. Neurosci. 14:93. doi: 10.3389/fnhum.2020.00093
Received: 28 October 2019; Accepted: 02 March 2020;
Published: 08 April 2020.
Edited by:
Valerio Rizzo, University of Palermo, ItalyReviewed by:
Hasan Ayaz, Drexel University, United StatesDiogo Morais, Lusophone University of Humanities and Technologies, Portugal
Copyright © 2020 Tuena, Pedroli, Trimarchi, Gallucci, Chiappini, Goulene, Gaggioli, Riva, Lattanzio, Giunco and Stramba-Badiale. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Cosimo Tuena, cosimo.tuena@unicatt.it