Potential and Pitfalls of Digital Voice Assistants in Older Adults With and Without Intellectual Disabilities: Relevance of Participatory Design Elements and Ecologically Valid Field Studies

Introduction

While digital voice assistants (VAs) are increasingly becoming part of everyday life (Auxier, 2019), specific user groups that might benefit from such technology are often not considered in research. This applies to older adults (Stigall et al., 2019), a population with pronounced heterogeneity in terms of cognitive, sensory and functional competencies. Given the rapid expansion of digital tools, it is important to consider the use of VAs as a promising tool able to maintain and enhance social participation, autonomy, and leisure activities for a wide range of older adults. There is still a digital divide between younger and older generations (Anderson, 2019) and older adults might benefit to a smaller extent from the advantages of new technologies such as VAs. Older people with intellectual disabilities are at particular risk of digital exclusion (Ehlers et al., 2020) and digital technologies such as VAs have hardly been evaluated for this target group (e.g., Smith et al., 2020). In this opinion paper, we synthesize current research in the context of VAs for older adults. Building on this, we propose specific research designs to provide better insights into the adoption and use of VAs in advanced age.

Benefits of Voice Assistants

An overview of literature provides six clusters of insights. First, VAs offer technology access for individuals who do not use conventional computing devices. Usability problems caused by small font or buttons are eliminated (Ziman and Walsh, 2018; Kowalski et al., 2019; Corbett et al., 2021), which is especially useful for individuals with limited motor, sensory, or cognitive functions (e.g., Yaghoubzadeh et al., 2013; Wulf et al., 2014). Second, in the social domain, VAs enable contact and communication with others, especially for older people with disabilities like limited vision or impaired hand movement (Kowalski et al., 2019; Scherr et al., 2020; Trajkova and Martin-Hammond, 2020). Additionally, the VA itself can be a social companion to some extent (Scherr et al., 2020; Smith et al., 2020; Corbett et al., 2021). Third, a benefit has been identified in the health domain. VAs can assist with daily well-being activities such as health tracking, medication management, or dietary planning (Tsiourti et al., 2014; Nallam et al., 2020; Trajkova and Martin-Hammond, 2020). Fourth, the use of VAs might contribute to enjoyable leisure time experiences, via entertainment features like music, videos, and jokes (Scherr et al., 2020; Corbett et al., 2021). Fifth, VAs can provide support for independent living including aspects of time-structuring (e.g., setting a timer or reminders) and instrumental activities like access to online information (Nallam et al., 2020; Pradhan et al., 2020; Scherr et al., 2020; Corbett et al., 2021). Sixth, taken together, VAs may have a positive effect on a person's agency because VAs can support older adults with intellectual disabilities in better managing different aspects of their everyday lives (Smith et al., 2020).

Challenges of Voice Assistants and Limitations in Current Research on Voice Assistants in Older Adults

Although it boasts good usability in general, problems interacting with VAs are frequently observed, because the user is required to follow a pre-structured form of dialogue, thus limiting the conversational abilities of VAs (Scherr et al., 2020). Older adults often have problems recalling the specific commands necessary to operate the devices (Wulf et al., 2014; Pradhan et al., 2020). Another limiting factor can be the lack of added value, which may result in a preference for devices already used (Trajkova and Martin-Hammond, 2020). VAs are perceived as time consuming and a lack of compatibility is criticized (Kowalski et al., 2019). Furthermore, a barrier for using VAs can be seen in reported fear of losing one's own competences and autonomy, because the VA may take care of a number of tasks without considering the competencies of the user (Kowalski et al., 2019; Trajkova and Martin-Hammond, 2020). In this way, the benefit of support for independent living and a reduction of dependency on personal assistance may also result in a higher level of dependency on VA assistance. Finally, concerns about privacy and data security are potential barriers to using VAs (Nallam et al., 2020; Trajkova and Martin-Hammond, 2020).

On a more general level, two major limitations in research on VAs for older adults exist. There is limited knowledge on VA use in specific groups of older adults. We identified two studies that focus on benefits and challenges for older adults with cognitive impairment (e.g., dementia; Wargnier et al., 2015; Wolters et al., 2016) and two other studies that included older people with intellectual disabilities (Braun et al., 2020; Smith et al., 2020). We also see limitations at the methodological and design level. In particular, we identified six studies that apply field data collection—collecting data in everyday life settings to analyze the use of VAs among older adults (Tsiourti et al., 2014; Kopp et al., 2018; Oh et al., 2020; Pradhan et al., 2020; Scherr et al., 2020; Corbett et al., 2021). Similarly, user-centered research approaches have been infrequently applied thus far.

Recommendations For Future Research

Based on these insights, we derive recommendations for research on VAs (see Table 1). We put emphasis on subgroups of older adults who have been largely neglected, i.e., older adults with cognitive impairment like dementia-related disorders or intellectual disabilities.

TABLE 1

Table 1. Recommendations for future research on Voice Assistants (VAs) in heterogeneous groups of older adults.

Need for Participatory Design Elements in Research With Older Adults

We recommend a high level of user involvement in research about commercial VAs for older people. A participatory design strategy should identify the best ways to introduce VA use, benefits as well as limits and risks of VAs to different groups of older adults. In particular, this allows to address the identified challenges (see above) and may ensure that older people can use VAs according to their preferences. We argue that the need for instruction is higher among older adults compared to younger people because they did not grow up with digital technologies. As an example, we propose a participatory conception and implementation of user trainings and manuals¹: To realize a participatory design in this domain, older adults should be involved in different co-designing activities such as co-design workshops to develop and discuss own ideas (e.g., Davidson and Jensen, 2013). Thereby, older users will be able to directly and actively influence the design of different kinds of material:

• User trainings: Educational programs and training courses can lower barriers in VA interaction (Czaja et al., 2019). The contents and format should be developed together with the target group and tailored to varying prior technology experiences, disability, and cognitive competencies. Relevant aspects may include the requirements for a successful interaction with VAs (e.g., specific voice commands), the possibilities of using VAs for different purposes (e.g., social domain, health, leisure time), and possible concerns (e.g., privacy issues, losing competencies due to using VAs). We recommend including older users with different skill levels in the training conception to make VAs widely accessible. Due to the different competencies (e.g., in reading and writing, attentional control, executive functions), the presentation and complexity must be adapted individually in each case.

• User manuals: Another aspect of VA learning and adoption are user manuals, an aspect that has so far been insufficiently addressed in research. Well-designed guidelines could help older adults to explore the possibilities of VAs according to their needs and in their own pace. Different versions of user manuals should be discussed with older adults to achieve the best design possible. These group-specific manuals are especially helpful for older adults with cognitive impairments or intellectual disabilities who may have specific needs, like instructions in easy-to-read language (Maaß, 2020) or a visualization of the instructions (Spriggs et al., 2017).

Need for Field Studies in the Everyday Lives of Older Adults

Field studies can contribute to a high level of ecological validity of the research–implying the empirical validity of findings in everyday life settings. Depending on the research questions, researchers should thoroughly consider the appropriate observation period. Existing field studies about VA use report durations from only a few days (Lopatovska et al., 2019), several months (Scherr et al., 2020), up to 1 year (Trajkova and Martin-Hammond, 2020). Our recommendation would be an observation period of at least 4 weeks, which would allow analysis of more routinized interactions with VAs after an initial phase of learning and curiosity.

Despite existing field studies, we claim that the potentials are not yet fully exploited, and should be extended focusing on innovative approaches and heterogeneous user groups:

• Diary studies: In open-ended and closed questions, the users can report their experiences, likes, and dislikes about using a VA. A digital diary format allows to provide additional assistance if necessary. Participants can be reminded of the diary with prompts, and questions can be repeated and adapted to the individual. Participants can be actively encouraged to provide comprehensive feedback on enjoyable, useful, and negative experiences of VA interactions. In addition to this, cultural probes such as self-taken photos, cards with reflection tasks about VA use, and other activities (e.g., creation of relationship or neighborhood maps) can be applied to gain further insights into the everyday lives of the participants and to capture older adults' experiences with the devices in a comprehensive manner (Jarke and Gerhard, 2017).

• Analysis of automatically collected data: Beyond users' self-reports, we see high potential in collecting additional data associated with usage behaviors like back-end data combined with external recordings of audio (Porcheron et al., 2018) or video (Lahoual and Frejus, 2019) data of VA use. These data provide information about which VA functions are used by individuals with different levels of expertise and competences, about used voice commands, and changes of use patterns over time. In this context, researchers should in any case consider ethical concerns of automatically collected data such as a threat of permanent observation or the fear of providing too intimate information. Attention should be paid to the design of the informed consent and continuous support of study participants concerning these aspects should be provided.

• Emotion analysis: State-of-the-art software solutions allow to automatically analyze emotional experiences based on speech and facial expressions (Garcia-Garcia et al., 2017; Dupré et al., 2018). The emotional experience of VA use is another aspect that is important to our understanding of their benefits and challenges for older adults. An analysis of these data allows researchers to study user experiences of VAs in situations when verbal feedback is scarce due to possible cognitive impairments or intellectual disabilities. Still, due to the novelty of the approach, automatic emotion analysis is not always reliable. Regarding older adults with intellectual disabilities, their emotional expression may differ from older adults without disabilities due to more frequent motor impairments (e.g., spasticity), differences in physical appearance (especially regarding genetic syndromes), cognitive deficits (e.g., in perception or appraisal processes), stereotypical behaviors, or earlier aging processes (von Gontard, 2013). Facial expression may be altered so that automatic face recognition cannot detect known patterns. The validity of automatic emotion analysis in this group has not yet been proven (Adams and Oliver, 2011; Martínez-González and Veas, 2019) and should be the focus of future research.

In particular, the triangulation of the different data sources can provide an overarching picture of user experiences of VAs, e.g., by analyzing in which situations the VA is used, how the older person evaluates this interaction, and how it is experienced emotionally.

Conclusions

In this opinion paper, we offer a set of recommendations that may guide future research on VAs for and with older adults. In a nutshell, we posit that future research designs should strongly rely on analyzing VA interactions in everyday ecologies and strictly apply participatory design elements where possible. Data protocols should include a balanced mixture of automatized data including emotional aspects as well as structured and open assessments. From our perspective, considering these recommendations can significantly help to create evidence-based findings able to inform interventions with VAs in heterogeneous groups of older adults. This also contributes to the goal of getting the best out of VA systems to improve quality of life and avoid possible risks.

Author Contributions

All authors provided substantial contributions to this article from conception to final approval.

Funding

This publication was part of the AI-Aging project (AI-based voice assistance for older adults with and without intellectual disabilities), which was funded by the Baden-Württemberg Stiftung within the funding line Responsible Artificial Intelligence. The article processing charge was funded by the Baden-Württemberg Ministry of Science, Research and Culture and the Heidelberg University of Education in the funding programme Open Access Publishing.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Footnotes

1. ^We put emphasis on commercial VAs that are available “off the shelf” but offer lower customization and co-designing options. However, in the case of VA design and development, participatory research and co-designing is equally important to better understand users' needs and their preferences. A co-design of VAs will guarantee that style, content, wording and tasks of the VAs will fit the mental models of older adults (Wolters et al., 2016; Fischer et al., 2020).

References

Adams, D., and Oliver, C. (2011). The expression and assessment of emotions and internal states in individuals with severe or profound intellectual disabilities. Clin. Psychol. Rev. 31, 293–306. doi: 10.1016/j.cpr.2011.01.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Anderson, M. (2019). Mobile Technology and Home Broadband 2019. Available online at: https://www.pewresearch.org/internet/wp-content/uploads/sites/9/2019/06/PI_2019.06.13_Mobile-Technology-and-Home-Broadband_FINAL2.pdf (accessed May 17, 2021).

Auxier, B. (2019). 5 Things to Know About Americans and Their Smart Speakers. Available online at: https://www.pewresearch.org/fact-tank/2019/11/21/5-things-to-know-about-americans-and-their-smart-speakers/ (accessed May 20, 2021).

Braun, M., Wölfel, M., Renner, G., and Menschik, C. (2020). “Accessibility of different natural user interfaces for people with intellectual disabilities,” in 2020 International Conference on Cyberworlds (CW) (Caen: IEEE), 211–218.

Corbett, C. F., Combs, E. M., Wright, P. J., Owens, O. L., Stringfellow, I., Nguyen, T., et al. (2021). Virtual home assistant use and perceptions of usefulness by older adults and support person dyads. Int. J. Environ. Res. Public Health 18, 1–13. doi: 10.3390/ijerph18031113

PubMed Abstract | CrossRef Full Text | Google Scholar

Czaja, S. J., Boot, W., Charness, N., and Rogers, W. A. (2019). Designing for Older Adults: Principles and Creative Human Factors Approaches, 3rd Edn. London: Chapman and Hall/CRC.

Google Scholar

Davidson, J. L., and Jensen, C. (2013). “Participatory design with older adults,” in Proceedings of the 9th ACM Conference on Creativity and Cognition, eds E. Y.-L. Do, S. Dow, J. Ox, S. Smith, K. Nishimoto, C. T. Tan, Y. Nagai, S. Ferguson, and T. Hewett (Sydney, NSW: ACM), 114–123.

Google Scholar

Dupré, D., Andelic, N., Morrison, G., and McKeown, G. (2018). “Accuracy of three commercial automatic emotion recognition systems across different individuals and their facial expressions,” in 2018 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops) (Athens: IEEE), 627–632.

Google Scholar

Ehlers, A., Heß, M., Frewer-Graumann, S., Olbermann, E., and Stiemke, P. (2020). “Digitale Teilhabe und (digitale) Exklusion im Alter: Expertise zum Achten Altersbericht der Bundesregierung,” in Expertisen zum Achten Altersbericht der Bundesregierung, eds C. Hagen, C. Endter, and F. Berner (Berlin: Deutsches Zentrum für Altersfragen).

Fischer, B., Peine, A., and Östlund, B. (2020). The importance of user involvement: a systematic review of involving older users in technology design. Gerontologist 60, e513–e523. doi: 10.1093/geront/gnz163

PubMed Abstract | CrossRef Full Text | Google Scholar

Garcia-Garcia, J. M., Penichet, V. M. R., and Lozano, M. D. (2017). “Emotion detection: technology review,” in Proceedings of the XVIII International Conference on Human Computer Interaction–Interacción '17, ed J. M. González-Calleros (Cancun: ACM Press), 1–8. doi: 10.1145/3123818.3123852

CrossRef Full Text | Google Scholar

Jarke, J., and Gerhard, U. (2017). “Using cultural probes for co-creating a digital neighbourhood guide with and for older adults,” in Mensch und Computer–Workshopband, eds M. Burghardt, R. Wimmer, C. Wolff, and C. Womser-Hacker (Regensburg), 79–85.

Google Scholar

Kopp, S., Brandt, M., Buschmeier, H., Cyra, K., Freigang, F., Krämer, N., et al. (2018). “Conversational assistants for elderly users–the importance of socially cooperative dialogue,” in Proceedings of the AAMAS 2018 Workshop on Intelligent Conversational Agents in Home and Geriatric Care Applications (Stockholm).

Google Scholar

Kowalski, J., Jaskulska, A., Skorupska, K., Abramczuk, K., Biele, C., Kopeć, W., et al. (2019). “Older adults and voice interaction: a pilot study with google home,” in Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, eds S. Brewster, G. Fitzpatrick, A. Cox, and V. Kostakos (Glasgow: ACM), 1–6.

Google Scholar

Lahoual, D., and Frejus, M. (2019). “When users assist the voice assistants: from supervision to failure resolution,” in Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, eds S. Brewster, G. Fitzpatrick, A. Cox, and V. Kostakos (Glasgow: ACM), 1–8.

Google Scholar

Lopatovska, I., Rink, K., Knight, I., Raines, K., Cosenza, K., Williams, H., et al. (2019). Talk to me: exploring user interactions with the Amazon Alexa. J. Librariansh. Inf. Sci. 51, 984–997. doi: 10.1177/0961000618759414

CrossRef Full Text | Google Scholar

Maaß, C. (2020). Easy Language—Plain Language—Easy Language Plus: Balancing Comprehensibility and Acceptability. Berlin: Frank and Timme.

Google Scholar

Martínez-González, A. E., and Veas, A. (2019). Identification of emotions and physiological response in individuals with moderate intellectual disability. Int. J. Dev. Disabil. 1–6. doi: 10.1080/20473869.2019.1651142

CrossRef Full Text | Google Scholar

Nallam, P., Bhandari, S., Sanders, J., and Martin-Hammond, A. (2020). A question of access: exploring the perceived benefits and barriers of intelligent voice assistants for improving access to consumer health resources among low-income older adults. Gerontol. Geriatr. Med. 6:2333721420985975. doi: 10.1177/2333721420985975

PubMed Abstract | CrossRef Full Text | Google Scholar

Oh, Y. H., Chung, K., and Ju, D. Y. (2020). Differences in interactions with a conversational agent. Int. J. Environ. Res. Public Health 17:3189. doi: 10.3390/ijerph17093189

CrossRef Full Text | Google Scholar

Porcheron, M., Fischer, J. E., Reeves, S., and Sharples, S. (2018). “Voice interfaces in everyday life,” in Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems–CHI '18, eds R. Mandryk, M. Hancock, M. Perry, and A. Cox (Montreal, QC: ACM Press), 1–12.

Google Scholar

Pradhan, A., Lazar, A., and Findlater, L. (2020). Use of intelligent voice assistants by older adults with low technology use. ACM Trans. Comput. Hum. Interact. 27, 1–27. doi: 10.1145/3373759

CrossRef Full Text | Google Scholar

Scherr, S. A., Meier, A., and Cihan, S. (2020). “Alexa, tell me more – about new best friends, the advantage of hands-free operation and life-long learning,” in Mensch und Computer 2020–Workshopband, eds C. Hansen, A. Nürnberger, and B. Preim (Bonn: Gesellschaft für Informatik).

Google Scholar

Smith, E., Sumner, P., Hedge, C., and Powell, G. (2020). Smart-speaker technology and intellectual disabilities: agency and wellbeing. Disabil. Rehabil. Assist. Technol. 1–11. doi: 10.1080/17483107.2020.1864670

PubMed Abstract | CrossRef Full Text | Google Scholar

Spriggs, A. D., Mims, P. J., van Dijk, W., and Knight, V. F. (2017). Examination of the evidence base for using visual activity schedules with students with intellectual disability. J. Spec. Educ. 51, 14–26. doi: 10.1177/0022466916658483

CrossRef Full Text | Google Scholar

Stigall, B., Waycott, J., Baker, S., and Caine, K. (2019). “Older adults' perception and use of voice user interfaces: a preliminary review of the computing literature,” in OZCHI'19, eds S. Berkovsky, Y. Hijikata, J. Rekimoto, M. Burnett, M. Billinghurst, and A. Quigley (Fremantle, WA: ACM Press), 55–66.

Google Scholar

Trajkova, M., and Martin-Hammond, A. (2020). ““Alexa is a Toy”: exploring older adults' reasons for using, limiting, and abandoning echo,” in Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, eds R. Bernhaupt, F. Mueller, D. Verweij, J. Andres, J. McGrenere, A. Cockburn, I. Avellino, A. Goguey, P. Bjørn, S. Zhao, B. P. Samson, and R. Kocielnik (Honolulu, HI: ACM), 1–13.

Tsiourti, C., Joly, E., Wings, C., Ben Moussa, M., and Wac, K. (2014). “Virtual assistive companions for older adults: qualitative field study and design implications,” in PervasiveHealth ‘14: proceedings of the 8^th International Conference on Pervasive Computing Technologies for Healthcare, Oldenburg, Germany, May 20-23, 2014, eds A. Hein and S. Boll (Oldenburg: ICST, Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering), 57–64.

von Gontard, A. (2013). “Genetische und biologische Grundlagen.” in Geistige Behinderung: Grundlagen, Erscheinungsformen und klinische Probleme, Behandlung, Rehabilitation und rechtliche Aspekte, eds G. Neuhäuser, H.-C. Steinhausen, F. Häßler, K. Sarimski, A. von Gontard, J. Buchmann, O. Reis, T. Klauß, S. Wachsmuth, K. Fischer, G. Grampp, S. Elstner, M. Buchka, C. Schanze, T. Frühauf, S. Wendt, U. Hellmann, and K. Lachwitz (Stuttgart: Kohlhammer Verlag), 30–43.

Google Scholar

Wargnier, P., Malaise, A., Jacquemot, J., Benveniste, S., Jouvelot, P., Pino, M., et al. (2015). “Towards attention monitoring of older adults with cognitive impairment during interaction with an embodied conversational agent,” in 3rd IEEE VR International Workshop on Virtual and Augmented Assistive Technology (VAAT) (Arles: IEEE), 23–28.

Google Scholar

Wolters, M. K., Kelly, F., and Kilgour, J. (2016). Designing a spoken dialogue interface to an intelligent cognitive assistant for people with dementia. Health Inform. J. 22, 854–866. doi: 10.1177/1460458215593329

PubMed Abstract | CrossRef Full Text | Google Scholar

Wulf, L., Garschall, M., Himmelsbach, J., and Tscheligi, M. (2014). “Hands free–care free: elderly people taking advantage of speech-only interaction,” in Proceedings of the 8^th Nordic Conference on Human-Computer Interaction Fun, Fast, Foundational-NordiCHI'14, eds V. Roto, T. Olsson, K. Väänänen-Vainio-Mattila, J. Häkkilä, E. Hvannberg, and O. Juhlin (Helsinki: ACM Press), 203–206.

Google Scholar

Yaghoubzadeh, R., Kramer, M., Pitsch, K., and Kopp, S. (2013). “Virtual agents as daily assistants for elderly or cognitively impaired people: Studies on acceptance and interaction feasibility,” in Intelligent Virtual Agents. IVA 2013. Lecture Notes in Computer Science, eds R. Aylett, B. Krenn, C. Pelachaud, and H. Shimodaira (Berlin, Heidelberg: Springer). doi: 10.1007/978-3-642-40415-3_7

CrossRef Full Text | Google Scholar

Ziman, R., and Walsh, G. (2018). “Factors affecting seniors' perceptions of voice-enabled user interfaces.” in Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems, eds R. Mandryk, M. Hancock, M. Perry, and A. Cox (Montreal, QC: ACM), 1–6.

Google Scholar