Methods to facilitate co-production in mental health are important for engaging end users. As part of the Technology for Healthy Aging and Wellbeing (THAW) initiative we organized two interactive co-production workshops, to bring together older adults, health and social care professionals, non-governmental organizations, and researchers. In the first workshop, we used two activities: Technology Interaction and Scavenger Hunt, to explore the potential for different stakeholders to discuss late life mental health and existing technology. In the second workshop, we used Vignettes, Scavenger Hunt, and Invention Test to examine how older adults and other stakeholders might co-produce solutions to support mental wellbeing in later life using new and emerging technologies. In this paper, we share the interactive materials and activities and consider their value for co-production. Overall, the interactive methods were successful in engaging stakeholders with a broad range of technologies to support mental health and wellbeing and in co-producing ideas for how they could be leveraged and incorporated into older people’s lives and support services. We offer this example of using interactive methods to facilitate co-production to encourage greater involvement of older adults and other under-represented groups in co-producing mental health technologies and services.

Introduction

Co-production or ‘making things together’ (Social Care Institute for Excellence [SCIE], 2015) is specifically encouraged in health and social care in the United Kingdom and other countries to engage service users. In the United Kingdom co-production was introduced as a key lesson from two public inquiries which identified that “service providers need to develop more equal partnerships with people who use services and carers” (Social Care Institute for Excellence [SCIE], 2015). Co-production is based on five principles: building and maintaining relationships, reciprocity, sharing of power, including all perspectives and skills, and respecting and valuing the knowledge of all (New Economics Foundation [NEF], 2013). Successful co-production requires an environment where everyone’s voice is heard, which is essential for populations not used to speaking in public, or with unequal power dynamics, such as patients speaking with doctors, or frontline staff speaking to managers and commissioners (Astell and Fels, 2021).

Co-production has been used in a variety of healthcare settings to improve services, increase choice, respond to user needs and reduce waste (Batalden et al., 2016). One aspect of co-production that can be difficult to achieve is power sharing (Farr et al., 2021). This is especially so in mental health research where it is a systemic barrier to engaging people with lived experience (Clark, 2015). Despite this obstacle, examples of co-production in mental health exist, including as a method to overcome health inequalities (Lwembe et al., 2017). People with lived experience of mental health problems have also been involved in co-producing programs including self-management of depression (Turner et al., 2015), and youth mental health (Mayer and McKenzie, 2017).

Fewer co-production studies have been conducted around late life mental health, in part due to negative perceptions about the abilities of older adults to participate (Mannheim et al., 2019). However, late life mental health is a growing global concern, with depression peaking in over 55s (World Health Organization [WHO], 2017). Co-production involving older adults in identifying gaps, priorities and developing new solutions is urgently needed. In particular, exploration of the potential of technology as part of services to support people’s mental health and wellbeing is increasingly important.

Fortunately, there is growing evidence that older adults are interested in exploring technology to self-support their mental health and wellbeing (Mannheim et al., 2019). However, there are still barriers to seeking mental health support, including fear of being labeled, difficulty navigating mental health services or mistrust of those working in these services (Reynolds et al., 2020). Older adults are also wary of who will have access to their data, how their data will be used, and of being a burden to others (Andrews et al., 2019). Additionally, older adults, especially those who are socially isolated and on fixed incomes, are more likely to be digitally excluded (Helsper and Reisdorf, 2017). These ‘hard to reach’ lonely and socially isolated individuals also have a significantly increased risk of depression (Lee et al., 2021). Exploring how stakeholders from different sectors, including older adults, view the potential of technologies in the support of mental health and how they can be implemented is therefore timely.

To explore understanding of technology for supporting late life mental health, we organized two co-production workshops using interactive tools as part of the Technology for Healthy Aging and Wellbeing (THAW) Ph.D. network. The THAW projects investigated three topics: (1) Using technology to improve early detection of late life mental health problems; (2) Understanding social isolation in a connected society; and (3) Affective computing to support good mental health in later life. Bringing together older adults, health and social care professionals, non-governmental organizations, and researchers, each workshop was facilitated with specific goals in mind. In designing these co-production workshops, the THAW team were aiming to gather information about how late life mental health and wellbeing were viewed and understood by attendees, attitudes to the role of technology in addressing late life mental health and wellbeing, and stakeholders’ visions for how emerging technologies could be utilized to benefit late life mental health and wellbeing. These facilitated workshops were also designed for attendees to have the opportunity to speak with people they did not normally come into contact with, make new contacts and have the opportunity to try out and provide feedback on existing and emerging technologies. Here, we describe the organization of the two workshops, the methods used and share the outcomes in respect of technology for late life mental health, to illustrate the utility and practicality of our approach for other researchers interested in co-production on this topic.

Workshop One. ‘Mental Wellbeing: Can Technology Help Older People Lead a Healthier and Happier Life?’

Aims and Objectives

The primary aim was to establish the feasibility of co-production for exploring the potential of new and emerging technologies in the support of older adults’ mental health. The secondary aim was to form a network of experts by experience who could contribute to future events and research activities.

Design

This 4-h long workshop was designed to provide an inclusive environment where everyone’s voice could be heard. Two interactive activities – Technology Interaction (Astell et al., 2020) and Scavenger Hunt (Astell et al., 2020) – were combined with student presentations on the three THAW projects. Activity descriptions, materials and aims are set out in Table 1.

TABLE 1

Table 1. Design of THAW interactive workshops.

Attendees

A mapping exercise was undertaken to identify local older people and carer’s groups/forums and organizations who may wish to be involved. Information was put on the Centre for Assistive Technology and Connected Healthcare (CATCH; University of Sheffield) website and flyers were produced to promote the workshop to stakeholders by email and social media. To secure the widest participation, the following stakeholders with an interest in the mental wellbeing of older people were invited: older people and informal carers; local Councils – social care practitioners and commissioners; Health Authorities – mental health practitioners and commissioners; third sector providers; researchers. Thirty-two people registered and 36 turned up on the day.

Materials

Materials for the workshop were as follows: Six tables with six chairs at each for attendees. Box of six small technologies on each table. Five tables around the sides of the room for Scavenger Hunt. Refreshment station, PA system, and digital camera.

Information Sheet 1: Running order of activities/program for the day.

Information Sheet 2: Ground rules to ensure an inclusive, safe, respectful, and positive event

Stalls were set up in an annex for local organizations to promote their activities relating to late life mental health and wellbeing.

Technology Interaction: Feedback Sheet for collecting the following information: ‘What is the technology?,’ ‘How do you feel using it?,’ ‘What worked and/or didn’t work well with the technology?’ e.g., any good and bad points, ‘Who do you feel could use it?’ and ‘Do you think this could support or hinder mental wellbeing and make people healthier or happier?’

Scavenger Hunt: Six technology stations around the room, and Feedback Form with six boxes, one per item, with the following statements: ‘This is easy to use,’ ‘This is something I would use,’ ‘This is something I enjoy,’ with a space to circle ‘yes’ or ‘no.’ Names and descriptions of the specific technologies are detailed in Table 2. These were chosen for their connection to each of the THAW projects (see Table 2).

TABLE 2

Table 2. Technological apps/equipment used for Workshop 1 and Workshop 2 Scavenger Hunts.

Workshop Evaluation Form.

Procedure

Attendees signed in, were assigned to the five tables, and asked to introduce themselves. Information Sheets 1 and 2 were provided to each participant, and they were asked to respect the equipment, not to use it unsafely or put others at risk, let everyone have a go and have a say and to treat other people with respect. After an introduction to the aims of THAW, attendees engaged in Technology Interaction (Table 1 and Supplementary Appendix A). This required pairs of attendees to try to get a piece of technology working in 10 min (Figure 1). A wide range of different technology types were provided to stimulate initial discussions and break the ice.

FIGURE 1

Figure 1. Mood light.

After a networking lunch, attendees had the opportunity to visit various stalls and speak with researchers, technology companies, and providers of mental health support, including a number of national charities. This was intended to provide further opportunities for attendees to find out about and discuss local services and support. The afternoon session resumed with brief presentations on each of the THAW doctoral projects. The presentations outlined the three different research projects and enabled attendees to gain an insight into the different elements being explored. Following these presentations, attendees engaged in the second interactive activity: Scavenger Hunt as described above (Table 1). The workshop ended with further networking opportunities over afternoon tea, where attendees were asked to complete the Workshop Evaluation Form before they left.

Results

The 36 people who attended the workshop were: 13 health or social care staff, 11 older adults, 4 advocates for older adults, 3 researchers and 5 others, comprising 1 commissioning manager of public health, 1 health and social care commissioner, 1 charity project worker, 1 volunteer with older people and 1 activities coordinator. Information about age or sex/gender was not collected from attendees.

Technology Interaction

The aim of Technology Interaction is to empower attendees to feel confident and comfortable discussing technology. It also promotes discussion of what people like and dislike in a range of technologies. Feedback from the five groups (one per table) identified features of the technologies that they liked, and the ones they did not like among the selection of off-the-shelf technologies. These features influence people’s reaction to a device and whether they will persevere with trying to use it. Positive features included items being simple or intuitive to use, items that have clear accessible instructions, and items where the intended use is obvious. For example, a well-received TomTom navigation device was described as “very easy and logical, it could be used by anyone, not just drivers.” Negative feedback highlighted lack of instructions, small text or small buttons, lack of indication what exactly the technology is for or how it should be used. A blood pressure cuff was rejected because it was “okay – comes with a manual but no idea what to expect. Took three attempts to get all readings and mine was not accurate.” During the feedback component, one person commented that it was clear that services could not or should not just give devices to people to work out for themselves.

Scavenger Hunt

Attendees’ answers on the feedback forms to the three use questions about each of the six technologies (Table 2; MantaroBot was presented as two technologies – one as controller and one as receiver), showed clear differences in their perceptions. Of the six technologies the two mood apps (MoodPanda and Mr Mood) and social networking sites were judged most easy to use and the MantaroBot in both roles (as controller and receiver) as least (Table 2). Despite the low rating of its usability, attendees saw potential in the MantaroBot: “Provides digital telepresence for engagement and inclusion” and “I feel the MantaroBot is a useful tool and would be able to support/navigate face to face contact with an individual.”

Of particular interest, while the two mood apps were judged easy to use, the majority of attendees who completed the feedback say they would neither use nor enjoy these apps. Their comments indicated concerns about the age appropriateness of the apps “Simple, childlike”; the functionality “useful to record mood over time but very limited”; and usefulness: “Does it add to a paper diary?” By contrast, attendees found the social networking sites easy to use and the majority said they would enjoy using them: “…Allows access to a wide range of support.” Similarly, Lexulous, an online scrabble game was rated in the middle for ease of use, but more people would use it and enjoy it than any of the other technologies. However, this did seem to depend on whether people were already digital gamers.

Workshop Evaluation

The workshop activities appeared successful in encouraging attendees to work together and share their expertise and experience. As one attendee commented…

“I felt that the group work at the beginning [Technology Interaction] worked really well as it allowed people from different generations to work out how to use bits of technology that they may have never seen before. It allowed people to share ideas, views and opinions on use, ease, and product design.”

Many forms commented on the diversity of professions and representatives among attendees, for example:

“meeting people from a broad range of organizations, community groups, third sector” and “interacting with people from different backgrounds and experience in social and health care”

For one attendee:

“by talking to lots of people” [they got] “a stronger feeling that technology will help people now and, in the future.”

While another was impressed with

“how the event was set up and enabled more effective networking.”

The mix of activities and presentations was also welcomed by attendees with one attendee commenting on the importance of “being able to see the technology and hear about research” in the THAW Ph.D. presentations.

At the end of the day, the majority of attendees agreed that the workshop had helped them gain a better understanding of the needs, barriers, and benefits of technology for mental wellbeing in later life. One attendee was:

“excited about technology and how much it’s changing in relation to older adults and mental health.”

Another felt the workshop was:

“thought-provoking, some very clever ideas that could be adopted to improve the lives of older people.”

One attendee also proposed the interactive aspect of the workshop could have a therapeutic value to those with mental health issues:

“Not just about today but as a former volunteer tutor for the Expert Patients’ Program and with relatives who work in mental health, I would like to stress the vital importance of enabling people to interact and offer mutual support in coping with their problems.”

Workshop 1 Summary and Reflection

Overall, the first workshop was a success in attracting a diverse range of attendees collaborating together to gain insight and interest in mental wellbeing and technology helping older people lead a healthier and happier life. The Technology Interaction activity successfully created a supportive environment for attendees to interact and share their views of technology. In terms of feedback, attendees identified factors that influence their decisions about persevering with or rejecting a technology, highlighting instructions, ease of use and functionality as key features. The Scavenger Hunt revealed the differing impressions and assessments of the six technologies. There was some suggestion that familiarity influenced the judgment about ease of use but did not influence people’s judgments about using and enjoying the technologies.

The findings confirmed that older adults could engage as equals in discussions about new and emerging technologies. They were interested to try the technologies on offer and were able to share their opinions and responses to them. The older adults were also comfortable discussing late life mental health and tools for supporting this. The interactive activities and workshop format were also successful at facilitating participation by frontline staff in health and social care with older adults, researchers, and other stakeholders. Encouraged by these results we organized a second workshop to focus on co-producing new solutions for late-life mental health and wellbeing.

Workshop Two: ‘Healthy Aging Within Your Reach: Shaping New Technologies to Support Mental Wellbeing for Older People’

Aims and Objectives

The second THAW workshop focused on co-production of solutions involving technology for late-life mental health and wellbeing.

Design

Building on the success and lessons learnt from the first workshop, activities were designed to facilitate discussion and thinking about ways of using new technology to support mental wellbeing in later life within services. This second workshop lasted 5.5 h and included three interactive activities: Vignettes (McGrath et al., 2016), Scavenger Hunt (Astell et al., 2020), and Invention Test (McGrath et al., 2016). These are detailed in Table 2.

Attendees

A list of invitees was created from the attendees at the first workshop plus other technology workshops hosted by CATCH who expressed an interest in attending future events. As before, information was put on the CATCH Website and flyers were produced to promote the workshops to stakeholders by email and social media. A total of 48 attendees registered and on the day 50 people attended – demographic data was not collected and not all attendees supplied their affiliation or role.

Materials

Materials for the workshop were as follows: Six tables with eight/nine chairs at each for attendees. Five tables around the sides of the room (for Scavenger Hunt). Refreshment station, PA system, and digital camera.

Information Sheet 1: Running order of activities/program for the day.

Information Sheet 2: Ground rules to ensure an inclusive, safe, respectful, and positive event.

Stalls for local organizations to promote their activities relating to late life mental health and wellbeing.

Vignettes: Six vignettes presenting an individual in a situation that could lead to loneliness, social isolation or have a negative impact on mental wellbeing. The process of creation and content for the vignettes used personal insights and professional experiences of working with older people from all the THAW researchers. They were written to enable participants to reread the scenario and discuss it on their table. Each vignette was different for each table but contained a similar amount of information relating to gender, age, presenting health conditions, family relationships, possible triggers or threats to social isolation, loneliness and mental well-being, and digital technological use. It was felt important that the vignette should resonate with participants (Skilling and Stylianides, 2020) and so the vignettes were written in the first person enabling participants to draw on their personal experiences whilst avoiding personal disclosure, protecting their confidentiality, and allowing discussion of sensitive and complex issues safely. As the range of stakeholders attending was diverse, the terminology used in the vignettes avoided jargon and was kept simple. The vignettes included six different perspectives; the characters involved in the scenario along with characters who may support and shape new services and the use of new technologies to help the mental well-being of older people (Table 1 and full details in Supplementary Appendix A).

Scavenger Hunt Materials: Six technology stations, detailed in Table 2. These were chosen as emerging technologies which may have a role in future health and care services.

Abridged version of the System Usability Scale (SUS; Brooke, 1996) which measures perceived usability – as used in other older adult studies relating to THAW (Bennion et al., 2020). A SUS score above 68 represents an above-average usability (Brooke, 2013).

Invention Test: Flipchart, paper, pens.

Workshop Evaluation Form.

Procedure

As Workshop 1, attendees signed in, were assigned to tables, and asked to introduce themselves. Information sheets 1 and 2 were given to each attendee asking them to respect the equipment, not to use it unsafely or put others at risk, let everyone have a go and have a say and to treat other people with respect. After an introduction to the aims of the day, attendees engaged in activity 1, Vignettes (Table 1). Following a 15-min refreshment break, attendees engaged in the second interactive activity: Scavenger Hunt (Table 1) with six new and in development technologies (Table 2). This included a new version of Eliza, the original therapy chatbot, developed as part of the THAW project studying the potential of chatbots in therapeutic interventions for older adults.

After a networking lunch and visits to stalls, the afternoon session resumed with brief update presentations on each of the THAW doctoral projects (Table 1). Following these presentations, attendees completed the Invention Test (Table 1). A representative from each group gave an overview of their idea to the whole workshop attendance at the end of the day and a prize was awarded to the best invention. The overviews were filmed and analyzed using Content Analysis. The workshop ended with further networking opportunities over afternoon tea, where attendees were asked to complete the Workshop Evaluation Form before they left.

Results

Vignettes

Each group of stakeholders were provided with one written vignette depicting a snapshot of a situation for an older person where mental wellbeing was an issue. The vignettes were used to introduce a range of factors and situations that could negatively impact or threaten late life mental health and wellbeing. The groups were asked to discuss these different perspectives. The vignette activity successfully stimulated discussion within the groups and fostered thinking about the need for and potential of holistic solutions. This discussion also set the scene for the other two activities. Participants engaged with, discussed, and identified the factors that affect mental well-being in later life in their vignettes. At the end of the workshop, the majority of attendees reported that they had a better understanding of the factors that could contribute to good and poor mental health and a better understanding of the roles of different services and organizations that can support mental health in later life. Participants appreciated the contribution of the vignettes to this improved understanding. One person, for example, valued: “Working together as a group to look at how we can deal with our given case study.” While another valued: “Exchanging views and ideas of how best to support the person in our case study”

The vignettes were referred to throughout the day. In particular, the combination of bringing together stakeholders with current and emerging technology, and a vignette to shape the discussions, created an environment where new ideas and thinking could be generated. Overall, 97% of attendees agreed or strongly agreed that the event had generated new thinking and a number of groups worked together and were inspired by the vignette personas when developing their ideas in the last activity of the day, the Invention Test (see below).

Scavenger Hunt

In total, 43 attendees completed the abridged SUS questionnaire. The sample was comprised of 39.5% (17/43) Health/social care staff, 25.6% (11/43) Older adults, 4.7% (2/43) Academic and Research staff, 7.0% (4/43) Policy Makers, 7.0% (4/43) Industry, Advocate for Older Adults or Others and 16.3% (7/43) Undisclosed.

Across all groups only two of the six technologies had a SUS score that was equal to or above the acceptable cut-off point (i.e., >68) (see Table 3): these were ‘MiRo’ (Figure 2) with a mean of 82.96 (SD 16.74) and ‘Pacifica’ with a mean of 68.37 (SD 21.73). Looking just at the older adults only MiRo scored above the SUS cut-off (mean of 81.25 [12.59]; see Table 3), while health and social care staff rated both MiRo (mean of 89.34 [16.01]) and Pacifica (mean of 76.84 [SD 17.83]) as usable.

TABLE 3

Table 3. Mean (SD) for System Usability Scale measure.

FIGURE 2

Figure 2. MiRo.

The SUS scores provided an informal ranking of the usability of the six technologies with MiRo highest and the MantaRobot scored lowest (Table 3). There were statistically significant, weak positive correlation between Pacifica system usability ratings and Eliza system usability ratings (r41 = 0.36, P < 0.005) and moderate positive correlation between Pacifica system usability ratings and MiRo system usability ratings (r41 = 0.44, P < 0.001). There was a statistically significant, weak negative correlation between MantaroBot system usability ratings and MiRo system usability ratings (r41 = −0.38, P < 0.005) and a moderate positive correlation between MantaroBot system usability ratings and VR system usability ratings (r41 = 0.51, P < 0.001).

In their free text notes, attendees considered MiRo to have “potential” and “great appeal,” however, further development was recommended in terms of a “softer feel.” The Pacifica app was “quick and easy with instant feedback” and could be used on a mobile phone “discreetly” although its accessibility for older adults was questioned. Eliza attracted mixed reviews, with some suggesting the conversational agent “lacks empathy” and that it was “hard to see usefulness in current format.” However, others suggested it “has potential” and that it was “good for communication, nice to talk to.”

Attendees envisaged applications of virtual reality to include “visit[ing] special places from childhood” and “for partially sighted people to see the ground or objects out of range.” The augmented reality application was posited as a way of reminding older adults how to complete activities of daily living such as making a cup of tea. However, attendees wondered if people living with dementia would “remember how to use the augmented reality to help them make a cup of tea.” Comments also included that “AR has potential for future use but would need to be integrated into the NHS” although confidentiality and risk were highlighted as potential barriers to use. Finally, the MantaroBot, which attracted the lowest SUS score, was found to be “clunky” and in need of a “better controller.” While some were “doubtful of real benefit,” others saw the MantaroBot as “potentially useful for discharge from hospital.” Again, the activity stimulated attendees’ imaginations, and many of the technologies they had experienced in this activity, some for the first time, were incorporated into their ideas for the final, Invention Test, activity later in the day.

Invention Test

Three of the six multi-perspective groups (1, 4, 5) co-created ideas which targeted the specific needs of the persona that they had considered in the first activity, while others were tangential or more general. The majority of groups also took inspiration from the technologies they had tried using earlier in the day and used these within the solutions they proposed.

Of those that directly addressed the persona in their Vignette (see Supplementary Appendix A), Group 1 (John: recently made redundant, isolated, and looking for things to occupy his time) presented a suite of solutions including use of online education groups and an online train enthusiast group to address social isolation, use of video calling and a Nintendo Wii to increase fitness and engage with family and grandchildren. Group 4 (Ashok: main carer for wife with dementia, social isolation compounded by hearing problem) suggested a person-centered technology care package consisting of technology chosen specifically for the user, and adapted to their specific needs, for example ensuring that the manuals for the technology were provided in the user’s own language. Group 5 (Julie; aging with an intellectual disability, mobility restricted through broken ankle, isolated at home) suggested that the MantaroBot could be used on an allotment, if it could be improved with solar power and be made waterproof. The aim was to support the care recipient to visit her allotment and re-engage with friends on neighboring allotments whilst she is recovering at home.

Group 2 proposed a virtual avatar to help older people who experienced depression, anxiety, or loneliness, to enable them to start new social groups. Group 3 presented a generic system they called ‘I care,’ based on principles of shared care, but in a virtual world. The idea was to provide a local forum for people in a discrete locality, whereby members could assist each other, refer each other to help and support for confidence building and other low risk activities that might replace some of the work of a support worker. The final group addressed older adults’ mental health needs in general by imagining a pair of glasses used as an augmented reality/virtual reality device, which the group titled ‘Virtual Library Live.’ The tool would allow access to downloading books, music, photographs, and videos. It could be used for a future hospital patient to review the journey into hospital and preview the experience of a certain procedure, to reduce anxiety around operations in adults and children. The same device was envisaged to be of benefit in care homes as a form of cognitive stimulation. Virtual gloves connected to the device would permit game-playing, and augmented reality elements would assist people with memory difficulties to complete activities of everyday living. This final proposal was judged the winner of the Invention Test.

Workshop 2 Evaluation

As with the first workshop, attendees said they enjoyed sharing ideas and views and meeting new people from a range of backgrounds and professions as well as hearing the THAW Ph.D. students’ presentations. The addition of Vignettes to the THAW presentations and hands-on Scavenger Hunt in the second workshop was well-received:

“Interactive nature, case study to focus discussions, hearing about emerging research, with a desire to learn from us too… worked because of careful matching/planning by the organizers.”

Additionally, seating different stakeholders together, positively contributed to the attendee experience, with suggestions for future events to engage even more older people from different backgrounds, including Black, Asian and Minority communities.

However, it was the ability to see and use new technology that for many attendees was the most important aspect of the workshop. As one attendee said, it was

“Good to get your hands on technology”

and another, that it was:

“good to see different devices and try!”

This interaction appeared to help attendees realize the potential of using new technology with older people, as one stated it is

“Very useful to see devices that can help older people in everyday life.”

Another commented that they appreciated:

“Applying the information to devising a new product that supports good mental health and wellbeing”

Workshop 2 Summary and Reflection

The second workshop focused on envisaging and co-creating services and products for older adults to address situations that can challenge mental health and wellbeing in later life. The program of activities eased attendees into conversations which may otherwise have been difficult to start, and naturally led to the final activity, which provided space for co-production based on the vignettes and technologies tried on the day.

Having each group work with a different vignette, permitted exploration of a range of situations that can lead older adults to be lonely, isolated or experience poor mental health. Attendees were thereby able to gain some insights into the complexity of people’s lives to develop a realistic proposal for how or where technology could play a role. In addition, many of the solutions generated in the Invention Test centered on providing older adults with ways of connecting with new groups, or even starting their own groups, using technology, demonstrating the potential of technology in this area to be well recognized.

Discussion

As a method, the facilitated, interactive THAW workshops were successful in engaging stakeholders from multiple perspectives in open conversations about late life mental health, social isolation, and technology. The intention was to go beyond known obstacles and barriers (e.g., lack of knowledge, inability to access the necessary technology and services, and barriers presented by mental health difficulties (Greer et al., 2019) to consider how to get technology into people’s hands for long-term benefit. The first workshop confirmed the feasibility of bringing stakeholders together to discuss late life mental health and potential roles for technology. The second session demonstrated how older adults can be involved as equals in co-producing proposals for new technology services in line with the five principles of co-production: building and maintaining relationships; reciprocity; sharing of power; including all perspectives and skills and respecting and valuing the knowledge of all (Walker, 2019).

Working together as equals to develop new solutions, provides opportunities for deeper understanding of the situations and concerns of all groups. Older adults sharing their personal experiences of mental health challenges, as well as attitudes toward and interactions with services, can provide an invaluable reality-check. For example, being able to discuss why people do or do not use a particular device or service (e.g., NHS website and apps, online mindfulness) and hearing how they currently use technology to manage their mental health (e.g., self-reliance, averting loneliness, and improving mood; Andrews et al., 2019).

By using a seating plan to mix up attendees with different professional and personal backgrounds, the workshops provided a safe environment, with equal value placed on the contributions of each group member. One benefit of using vignettes in the second workshop was that they were versatile and enabled different problems to be highlighted and responses from the different groups to be compared and contrasted (Skilling and Stylianides, 2020). Vignettes have been used for a wide range of purposes and to explore a variety of issues such as the learning and development of care home staff (Flick et al., 2013), mental health disparities (Lapatin et al., 2011), educational research (Skilling and Stylianides, 2020) and researching social work values (Wilks, 2004). This study suggest that vignettes can be an important tool in coproduction. Specifically, the vignettes helped elucidate the attendee’s views, feelings, and interpretations of mental health issues during the second workshop. In this respect, the use of vignettes is situated within qualitative research methodologies and interpretative approaches aiming to elicit beliefs and understanding (Miller et al., 1997; Gourlay et al., 2014). However, the vignettes were designed to provoke participant responses which should not be mistaken as reactions to ‘actual events’ (Skilling and Stylianides, 2020). They were beneficial because they helped guide conversations and safely promoted interactions on sensitive or potentially distressing issues, and so enabled the constructive sharing of ideas between participants.

Furthermore, by ‘simulating’ life situations, the vignettes provoked discussion and offered an entry point into the complex issues of mental health. This enabled the views of attendees from different fields to be unpacked and for them to interrogate these issues together (Kandemir and Budd, 2018). The simulated nature of vignettes provides enough emotional ‘distance’ to enable attendees to contribute confidently and safely (Simon and Tierney, 2011). Three of the vignettes directly shaped and structured interactions during the Invention Test, with those groups co-producing solutions targeted at the specific needs of the personas (1, 4, 5). These groups developed person-centered solutions compared to the more generic solutions offered by other groups. Additionally, the attendees used the vignettes to speak openly and honestly about the challenges that may be presented by some of the technologies. They also felt confident to suggest non-typical uses of the technologies on show, for example, for scenario 5 (Julie) suggesting that a telepresence robot (MantaroBot) could be used outside on an allotment.

Frontline staff in health care and social care services – who are usually the ones setting up and supporting technologies for their clients – were also able to speak as equals with managers and commissioners of services. Hearing the voice of staff as experts and recognizing their needs in relation to technology adoption and support are also crucial for successful implementation and rollout of technologies as part of person-centered services (Christie and Marshall, 2019). Sharing expertise from different stakeholders, including social care and health care, meant it was possible to consider how both types of services might make use of technology to support their work with older adults, and to better understand the experiences and views of the older people they support (Astell et al., 2018).

Considering priorities for services and technologies to address the needs of older adults’ mental health and wellbeing, the activities successfully elicited thinking that bridged current gaps between (health and social care) services. Overcoming existing silos is essential for tackling the conditions that negatively impact late life mental health and wellbeing as the COVID-19 pandemic has highlighted (e.g., Steptoe and Di Gessi, 2021). The attendees were also mindful of the barriers that face many people in participating in the digital world, especially those who are socially isolated (Helsper and Reisdorf, 2017). This led to discussion about the need for Internet access, digital literacy and affordable or loanable devices for empowering people to meet their needs for social interaction and mental stimulations.

Results from the Scavenger Hunt also demonstrated that while older adults and other stakeholders foresee potential benefits from new technology, they also recognize potential barriers to their development and roll out, in addition to limitations around the applicability of some technologies requiring physical dexterity or cognitive engagement. For some attendees, this was their first opportunity to try using technologies such as virtual reality and mental health apps, providing them with an experiential learning opportunity.

The interactive methods used here also elicited engaged discussion and debate about the technologies the attendees were able to try out. In relation to late life mental health, the poor usability rating of the MantaroBot at both workshops, is interesting given the focus on telepresence robots as a potential solution for tackling loneliness and social isolation, particularly among older adults (Niemalä et al., 2019). A growing number of telepresence robots are available for purchase, and one in particular – Giraff – has been developed and used in a number of European research projects focused on supporting older adults at home e.g., Coradeschi et al., 2014). However, some attendees did see potential benefits of MantaroBot, particularly for monitoring after hospital discharge, with one team proposing that it could be part of their new co-produced service solution in the Workshop 2 Invention Test. In this activity, the groups also made clear that technology-based solutions should continue to be person-centered and acknowledged the need for support to be in place to ensure older adults can truly benefit.

These findings suggest that such discussion and hands-on interaction is important for co-producing future solutions, for example the potential of using robots. In the second workshop there was a negative perceived usability correlation from the adapted-SUS scores between MantaroBot and MiRo, indicating a weak but visible relationship between those that rated MiRo with high perceived usability and MantaroBot with low perceived usability and vice versa. This suggests that the usability of robotics can be heavily influenced by their implementation. The positive perceived usability correlation between MantaroBot and VR indicates a moderate relationship between people who rated MantaroBot and VR with high perceived usability. The problems found by attendees with the MantaroBot may have been equal to the problems identified with VR and this was reflected in the usability scores.

The successful inclusion of measures such as the adapted SUS (Bennion et al., 2020) in co-production is important, particularly for iterative projects requiring users to evaluate technology solutions over time. In the second workshop, mean SUS scores helped highlight the usability of the technologies within the target demographic, while the individual questions of the scale helped generate discussion about each technology. It was noted that some of the questions of the scale were not answered by all older adult attendees. This may have been due to the language of the scale being problematic for older adults to understand. Use of the more recent simplified SUS scale for cognitively impaired and older adults by Holden (2020) would be beneficial in future older adult co-production workshops.

Overall, both workshops were very successful but there was some learning for organizing future events. Attendees’ keenness to interact with the Scavenger Hunt technology in both workshops, meant that queues formed, and timings overran. The size of the room may be important in this respect. Similarly, in the first workshop, multiple attendees at each table wished to present back on the technologies they tried to get working, leading to this taking longer than anticipated.

In conclusion, interactive facilitated workshops can provide a method for co-producing new thinking about late life mental health and wellbeing. This includes taking a holistic view of the complexities of older adults’ lives that can lead to loneliness, social isolation and poor mental health and wellbeing. Such understanding is vital for considering how technology can be offered and incorporated into people’s lives. The findings suggest that solutions are needed that combine technologies (e.g., tablets, robots, apps) with access to Internet, support for setting up and maintaining technologies and the potential for personalization (e.g., hobbies, language, and cultural history). These outputs indicate that the methods described in this paper constituted important tools for those who wish to shape technology for mental wellbeing in later life.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

Ethical review and approval was not required for the study on human participants in accordance with the local legislation and institutional requirements. The patients/participants provided their written informed consent to participate in this study. Written informed consent was obtained from the individual(s) for the publication of any potentially identifiable images or data included in this article.

Author Contributions

AA drafted the final manuscript. JA drafted sections of the manuscript about late life mental health, summarized the invention test data and provided feedback on the draft. MB summarized the Workshop 2 Scavenger Hunt, conducted the SUS analysis, and provided feedback on the draft. DC drafted sections about the vignettes and results from the evaluation surveys and provided feedback on the draft. All authors contributed to the article and approved the submitted version.

Funding

Funding for THAW was provided through the University of Sheffield, Directors of Research, and Innovation competition (CCDRI) to AA.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

We are grateful to Laura Murray’s administrative support in organizing the workshops and Simon Butler (deceased) for photography at the events, plus everyone else at the Centre for Assistive Technology and Connected Healthcare who helped to make the workshops successful.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyg.2021.745947/full#supplementary-material

References

Andrews, J. A., Brown, L. J., Hawley, M. S., and Astell, A. J. (2019). Older adults’ perspectives on using digital technology to maintain good mental health: interactive group study. J. Med. Int Res. 21:e11694. doi: 10.2196/11694

PubMed Abstract | CrossRef Full Text | Google Scholar

Astell, A. J., Andrews, J., Bennion, M., Clayton, D., Dove, E., Ellis, M. P., et al. (2018). COBALT, TUNGSTEN, and THAW: new tools for engaging older adults as technology experts. Gerontechnology 17:76. doi: 10.4017/gt.2018.17.s.076.00

CrossRef Full Text | Google Scholar

Astell, A. J., Dove, E., Morland, C., and Donovan, S. (2020). “Using the TUNGSTEN approach to co-create DataDay: a self-management app for dementia,” in HCI and Design in the Context of Dementia, eds G. Kenning and R. Brankaert (New York, NY: Springer).

Google Scholar

Astell, A. J., and Fels, D. I. (2021). “Co-production methods in health research,” in Knowledge, Innovation, and Impact in Health - A Handbook for the Engaged Researcher, eds A. Sixsmith, J. Sixsmith, A. Mihailidis, and M. L. Fang (New York, NY: Springer).

Google Scholar

Batalden, M., Batalden, P., Margolis, P., Seid, M., Armstrong, G., Opipari-Arrigan, L., et al. (2016). Coproduction of healthcare service. BMJ Qual. Safety 25, 509–517.

Google Scholar

Bennion, M. R., Hardy, G. E., Moore, R. K., Kellett, S., and Millings, A. (2020). Usability, acceptability, and effectiveness of web-based conversational agents to facilitate problem solving in older adults: controlled study. J. Med. Int. Res. 22:e16794. doi: 10.2196/16794

PubMed Abstract | CrossRef Full Text | Google Scholar

Brooke, J. (1996). SUS: A “Quick and Dirty” Usability Scale. Boca Raton, FL: CRC Press.

Google Scholar

Brooke, J. (2013). SUS: a retrospective. J. Usabil. Stud. 8, 29–40.

Google Scholar

Christie, J., and Marshall, M. (2019). “Technology for organisations supporting people with dementia,” in Using Technology in Dementia Care: A Guide to Technology Solutions for Everyday Living, eds A. Astell, S. Smith, and P. Joddrell (London: Jessica Kingsley Publishers).

Google Scholar

Clark, M. (2015). Co-production in mental health. Mental Health Rev. J. 20, 213–219.

Google Scholar

Coradeschi, S., Cesta, A., Cortellessa, G., Coraci, L., Galindo, C., Gonzalez, J., et al. (2014). GiraffPlus: a system for monitoring activities and physiological parameters and promoting social interaction for elderly. Human-Comp. Systems Interact. Backgrounds Appl. 3, 261–271.

Google Scholar

Farr, M., Davies, P., Andrews, H., Bagnall, D., Brangan, E., and Davies, R. (2021). Co-producing knowledge in health and social care research: reflections on the challenges and ways to enable more equal relationships. Human. Soc. Sci. Commun. 8:105. doi: 10.1057/s41599-021-00782-781

CrossRef Full Text | Google Scholar

Flick, D. M., Kolanowski, A. M., Hill, N. L., Yevchak, A., DiMeglio, B., and Mulhall, P. M. (2013). Using standardized case vignettes to evaluate nursing home staff recognition of delirium and delirium superimposed on dementia. Annals Long-term Care 21, 34–38.

Google Scholar

Gourlay, A., Mshana, G., Birdthistle, I., Bulugu, G., Zaba, B., and Urassa, M. (2014). Using vignettes in qualitative research to explore barriers and facilitating factors to the uptake of prevention of mother-to-child transmission services in rural Tanzania: a critical analysis. BMC Med. Res. Methodol. 14:21. doi: 10.1186/1471-2288-14-21

PubMed Abstract | CrossRef Full Text | Google Scholar

Greer, B., Robotham, D., Simblett, S., Curtis, H., Griffiths, H., and Wykes, T. (2019). Digital exclusion among mental health service users: qualitative investigation. J. Med. Int. Res. 21:e11696.

Google Scholar

Helsper, E. J., and Reisdorf, B. C. (2017). The emergence of a “digital underclass” in Great Britain and Sweden: changing reasons for digital exclusion. New Media Soc. 19, 1253–1270. doi: 10.1177/1461444816634676

CrossRef Full Text | Google Scholar

Holden, R. J. (2020). A Simplified System Usability Scale (SUS) for cognitively impaired and older adults. Proc. Int. Symp. Hum. Fact. Ergon. Health Care 9, 180–182. doi: 10.1177/2327857920091021

CrossRef Full Text | Google Scholar

Kandemir, A., and Budd, R. (2018). Using vignettes to explore reality and values with young people. Forum Qual. Sozialforschung 19, 1–23.

Google Scholar

Lapatin, S., Gonçalves, M., Nillni, A., Chavez, L., Quinn, R. L., Green, A., et al. (2011). Lessons from the use of vignettes in the study of mental health service disparities. Health Serv. Res. 47, 1345–1362.

Google Scholar

Lee, S. L., Pearce, E., Ajnakina, O., Johnson, S., Lewis, G., Mann, F., et al. (2021). The association between loneliness and depressive symptoms among adults aged 50 years and older: a 12-year population-based cohort study. Lancet Psychiatry 8, 48–57. doi: 10.1016/s2215-0366(20)30383-7

CrossRef Full Text | Google Scholar

Lwembe, S., Green, S., Chigwende, J., Ojwang, T., and Dennis, R. (2017). Co-production as an approach to developing stakeholder partnerships to reduce mental health inequalities: an evaluation of a pilot service. Prim. Health Care Res. Dev. 18, 14–23. doi: 10.1017/S1463423616000141

CrossRef Full Text | Google Scholar

Mannheim, I., Schwartz, E., Xi, W., Buttigieg, S. C., McDonnell-Naughton, M., Wouters, E. J. M., et al. (2019). Inclusion of older adults in the research and design of digital technology. Int. J. Environ. Res. Public Health 16:3718. doi: 10.3390/ijerph16193718

PubMed Abstract | CrossRef Full Text | Google Scholar

Mayer, C., and McKenzie, K. (2017). ‘…it shows that there’s no limits: the psychological impact of co-production for experts by experience working in youth mental health. Health Soc. Care Commun. 25, 1181–1189. doi: 10.1111/hsc.12418

PubMed Abstract | CrossRef Full Text | Google Scholar

McGrath, C., Ellis, M., Hwang, F., Williams, E., Archer, L., Harney-Levine, S., et al. (2016). Exploring the use of Participatory Methods to Understand Older Adults’ Experiences of Technology Adoption: Findings from the Challenging Obstacles and Barriers to Assisted Living Technologies (COBALT) Project. Toronto, ON: HCI International.

Google Scholar

Miller, A., Velleman, R., Rigby, K., Orford, J., Tod, A., Copello, A., et al. (1997). “The use of vignettes in the analysis of interview data: relatives of people with drug problems,” in Doing Qualitative Analysis in Psychology, ed. N. Hayes (Hove: Psychology Press)

Google Scholar

New Economics Foundation [NEF] (2013). Co-production in Mental Health. A Literature Review. London: Mind and New Economics Foundation.

Google Scholar

Niemalä, M., van Aerschot, L., Tammela, A., Alltonen, L., and Lammi, H. (2019). Towards ethical guidelines of using telepresence robots in residential care. Int. J. Soc. Robot. 13, 431–439. doi: 10.1007/s12369-019-00529-8

CrossRef Full Text | Google Scholar

Reynolds, K., Medved, M., Mackenzie, C. S., Funk, L. M., and Koven, L. (2020). Older adults’ narratives of seeking mental health treatment: making sense of mental health challenges and “muddling through” to care. Qual. Health Res. 30, 1517–1528. doi: 10.1177/1049732320919094

PubMed Abstract | CrossRef Full Text | Google Scholar

Simon, M., and Tierney, R. (2011). Use of Vignettes in Educational Research on Sensitive Teaching Functions Such as Assessment. Available online at: https://docplayer.net/96698594-Use-of-vignettes-in-educational-research-on-sensitive-teaching-functions-such-as-assessment-marielle-simon-university-of-ottawa-robin-tierney-canada.html (accessed March, 2013)

Google Scholar

Skilling, K., and Stylianides, G. J. (2020). Using vignettes in educational research: a framework for vignette construction. Int. J. Res. Method Educ. 43, 541–556.

Google Scholar

Social Care Institute for Excellence [SCIE] (2015). Co-Production in Social Care: What it is and how to do it – At a Glance. London: SCIE.

Google Scholar

Steptoe, A., and Di Gessi, G. (2021). Mental health and social interactions of older people with physical disabilities in England during the COVID-19 pandemic: a longitudinal cohort study. Lancet 6, E365–E373. doi: 10.1016/S2468-2667(21)00069-4

CrossRef Full Text | Google Scholar

Turner, A., Realpe, A. X., Wallace, L. M., and Kosmala-Anderson, J. (2015). A co-produced self-management programme improves psychosocial outcomes for people living with depression. Mental Health Rev. J. 20, 242–255. doi: 10.1108/MHRJ-05-2014-0017

CrossRef Full Text | Google Scholar

Walker, E. (2019). What Should Co-Production in Health Look Like? Available online at: https://uclpartners.com/blogpost/co-production-health-look-like/ (accessed April 1, 2019).

Google Scholar

Wilks, T. (2004). The use of vignettes in qualitative research into social work values. Qual. Soc. Work 3, 78–87.

Google Scholar

World Health Organization [WHO] (2017). Depression and Other Common Mental Disorders: Global Health Estimates. Geneva: World Health Organization.

Google Scholar