Co-creation of Research and Design During a Coding Club With Autistic Students Using Multimodal Participatory Methods and Analysis

Ward, Verity; Parsons, Sarah; Kovshoff, Hanna; Crump, Ben

doi:10.3389/feduc.2022.864362

ORIGINAL RESEARCH article

Front. Educ., 13 May 2022

Sec. Digital Learning Innovations

Volume 7 - 2022 | https://doi.org/10.3389/feduc.2022.864362

This article is part of the Research Topic Championing Inclusion and Diversity: Inclusive Design Practices and Approaches for Education View all 6 articles

Co-creation of Research and Design During a Coding Club With Autistic Students Using Multimodal Participatory Methods and Analysis

$\r\nVerity Ward,*$ Verity Ward^1,2*

Sarah Parsons^1,2

Hanna Kovshoff^2,3 $Ben Crump\r\n$ Ben Crump⁴

¹Southampton Education School, University of Southampton, Southampton, United Kingdom
²The Autism Community Research Network @ Southampton (ACoRNS), Southampton, United Kingdom
³School of Psychology, University of Southampton, Southampton, United Kingdom
⁴Fairmead School, Yeovil, United Kingdom

Participatory design aims to work with those who are often excluded from design processes so that their interests are better represented in design solutions. Autistic children are often marginalised and excluded from design processes due to concerns about how their social and communication differences may act as barriers to participation, leading to calls for design processes to be more inclusive and examined more closely to understand the value of participation for (autistic) children and young people. This research describes a participatory design project to develop a computer game during a weekly coding club at a special school. Fourteen autistic (neurodivergent) young people, eight staff members, four technology industry representatives and a Doctoral researcher worked together to design, develop, test, and evaluate the game. This article focuses specifically on the views and experiences of two of the students, which are captured primarily through a Digital Story. Digital Stories are short student-centred videos which show educational experiences. We use a social semiotic multimodal approach to analysis which does not prioritise linguistically encoded meaning, instead recognising the importance and validity of the many and varied ways in which students contributed to the project. The findings highlight the valuable opportunities that participatory design processes can provide for students as both learners and as expert knowers. It emphasises the need to allow room for students’ agency in the design process, so that they really can have a say in the outcomes of design and feel ownership over the process and outcomes of their research participation.

Introduction

Participatory design is not easily defined in terms of methods, formulas and rules, however, the many ways in which people interpret participatory design can be characterised by “directly involving people in the co-design of the artefacts, processes and environments that shape their lives” (Robertson and Simonsen, 2013, p. 2). This definition reflects the centrality of users to participatory design, where users are the people who apply or interact with the solutions created through participatory design (Bødker et al., 2011; Robertson and Simonsen, 2013). The involvement of users and other stakeholders throughout the design process acts to capture their knowledge, experience, and interests within designs (Kensing and Blomberg, 1998). In turn, designers facilitate and oversee the progress of participatory design (Bødker et al., 2011); they are typically responsible for both the product that is designed and the process which enables other participants to contribute to designs (Robertson and Simonsen, 2013). Participatory design can therefore be the result of knowledge contributions from users, other stakeholders, and designers in a way which facilitates them learning from the others’ expertise. Learning is consequently a key process in participatory design, and design more generally, as solutions are developed through the ongoing transfer and co-construction of knowledge between those involved (Béguin, 2003).

Much of the research within the field of participatory design has focused on the design and development of technologies for children (Druin, 2002; Benton et al., 2014; Malinverni et al., 2014; Schepers et al., 2018), including how children’s views can be equitably represented in ways that value their expertise while accepting the limitations of their knowledge and experience (Scaife and Rogers, 1999; Druin, 2002; Nesset and Large, 2004; Large et al., 2006). Balancing children’s agency with their perceived or expected limitations in knowledge is particularly an issue within the development of educational technologies, where children are not expected to understand all the concepts intending to be included (Scaife and Rogers, 1999; Good and Robertson, 2006). Furthermore, the perceived barriers which adults anticipate when co-designing with children can appear to be magnified when working with more marginalised groups, such as autistic¹ children or children with learning disabilities (Frauenberger et al., 2013).

Autism is characterised by both the DSM-5 (American Psychiatric Association, 2013) and ICD-11 (World Health Organization, 2018) as including deficits in the domains of social interaction and communication, and restrictive and repetitive behaviours. Within this medicalised conceptualisation of autism, the difficulties an autistic person faces are considered a direct result of their perceived impairments in these domains, rather than relating to the context in which a difficulty is experienced. In contrast, social-developmental accounts of autism emphasise the importance of previous experiences in shaping behaviour, how we are perceived by others, and how others respond to us, which in turn forms part of our experience that influences our subsequent behaviour (Mitchell, 2016). Within these accounts, innate differences between autistic and non-autistic people are not denied, but rather understanding how autistic people are perceived by others is key to our understanding of autism (Mitchell et al., 2021). Indeed, Milton (2012) proposed that autism can be understood in terms of a double empathy problem, in which autistic and non-autistic individuals struggle to understand each other’s intentions, resulting in marginalisation of autistic people who are a minority group within a largely non-autistic society.

Outside the medical model of autism, the neurodiversity movement defines autism as part of the natural variation of human neurology (Singer, 1999) rather than as a disorder. Within this context, it is important to recognise that the differences in neurology highlighted by the neurodiversity movement can lead to unique strengths and perspectives, such as strengths relating to attention to detail, visual perception and creativity (Best et al., 2015; de Schipper et al., 2016; Warren et al., 2021). Moreover, these unique strengths and perspectives are crucial for understanding the importance of embodied experiences in the construction of knowledge (de Jaegher, 2020). For example, differences in sensory processing may mean people experience the same environment and set of events in fundamentally different ways (de Jaegher, 2013).

Due to their perceived difficulties in social communication, autistic children, especially those with co-occurring learning disabilities and/or communication differences, are often marginalised and excluded from technology design processes (Frauenberger et al., 2013). When autistic children are included in design, research often reinforces deficit-focused conceptualisations of autism, for example focusing on developing technologies aimed at reducing social and communication “deficits” (Kientz et al., 2013). Such attitudes reflect out-dated conceptualisations of children as incompetent and vulnerable (Morrow and Richards, 1996; Grover, 2004), and autistic children as defined predominantly by perceived deficits and difficulties (Broderick and Ne’eman, 2008). These assumptions result in limited opportunities for autistic children to be considered experts and recognised for the value of their lived experiences, skills, and interests. Furthermore, the exclusion of autistic children may also mean they miss out on the potential learning opportunities which participatory design allows (Bell and Davis, 2016).

Some authors have argued that where participation is considered to be genuine in participatory design, it enables an educational aspect to the experience, leading to the development of participants’ competence, agency and knowledge, primarily through the process of mutual learning (Chawla and Heft, 2002; Greenbaum and Loi, 2012; Kinnula and Iivari, 2021). Mutual learning is a key goal and outcome of participation within participatory design (Robertson and Simonsen, 2012). For Robertson et al. (2014), mutual learning is limited and structured, focusing on the design problem under investigation. However, this approach does not account for other exchanges of skill and knowledge between participants. For example, participatory design potentially provides an opportunity for the development of skills which support such collaborations and learning more holistically about each other. Within the field of autism research, this idea has been referred to as gaining “interactional expertise” (Milton, 2014, p. 794), i.e. non-autistic people gaining knowledge and understanding about the lived experiences of autism that can only be accrued through sustained and meaningful interactions with autistic people.

Nonetheless, this more holistic interpretation of learning within participatory design is dependent on the same prerequisites as more restricted definitions. For example, participants must be willing to learn, respect each other and recognise the validity of others’ expertise in order for mutual learning to occur (Bratteteig, 1997). This idea of mutual respect is supported by Fowles (2000) who discussed the need to recognise not only the expertise of each group of stakeholders participating in the design process, but also their areas of ignorance through a process which he termed “symmetry of knowledge” (Fowles, 2000, p. 63). By understanding what each group does not know, groups can work together to build each other’s knowledge, and marginalised groups can recognise the value of their own expertise. This recognition of knowledge within themselves, and by other people, can be an empowering process, as it enables people to assume the power associated with the knowledge which results from their lived experiences (Bratteteig and Wagner, 2012). Such opportunities may be particularly valuable for autistic children, who are often excluded from the design process (Frauenberger et al., 2013).

There are some good examples of participatory and inclusive design approaches being applied to successfully gather and integrate the views of autistic children and young people, in ways which incorporate a more strengths-based conceptualisation of autism (Frauenberger et al., 2013, 2019; Benton et al., 2014; Malinverni et al., 2016; Wilson et al., 2019). In doing so, researchers and designers have enabled children and young people not simply to have a voice but also to have a say about the design and development of products and services which are intended for them (van der Velden and Mörtberg, 2014). This important distinction means that children and young people are (to some extent) able to have influence over and take ownership of the process and the outcomes of participatory design. Nevertheless, researchers have also recognised that there is a tension between the extent to which the participation of autistic children and young people is prioritised versus the more fixed objectives that researchers may have in mind regarding the outcomes for the designs themselves (Parsons and Cobb, 2014; Bossavit and Parsons, 2017). Indeed, much participatory design research concerns itself primarily with the outcomes of design (e.g., game, artefact, and prototype), often overlooking the nature and experiences of participation that led to those outcomes. This focus in the research has led to calls from some for the design process to be examined more closely to understand the value of participation for (autistic) children and young people as well as the potential usefulness or acceptability of any outputs produced (Parsons and Cobb, 2014). Similarly, Guha et al. (2010) recognised the importance of studying children’s experiences of participatory design processes to ensure that their involvement does not cause them harm, and to improve understanding for research of the potential benefits of children’s participation. This article aims to contribute to this area by focusing on children’s varied ways of participating in a design process and sharing their knowledge.

Where the effects of participation in design processes on children have been investigated, studies have mostly focused on the children who have had a greater input and influence over designs (Benton and Johnson, 2015). However, data collection methods have remained largely centred around adult accounts rather than facilitating and documenting children’s views and experiences. Where children have participated in the evaluations of design process, their contributions were limited and often not clearly described. For example, Mazzone et al. (2008), Millen et al. (2010), and Zarin and Fallman (2011) all made use of “informal evaluations” (Benton and Johnson, 2015, p. 33–34) to gain insights into the experiences of the children and young people involved but few details are provided about the nature or outcomes of these evaluations. This lack of clarity is problematic in terms of the replicability of such research, and the reliability of insights relating to children and young people’s experiences of participatory design.

Furthermore, where children have produced information about their experiences of participatory technology design which are less bounded by adult expectation, this data is often not used in evaluation. For example, during the final session of Benton et al.’s (2012) participatory design project to produce a maths game with autistic children, the children produced a display of their work which they presented to their headteacher. The children used this display as a memory aid while completing the survey evaluating the final prototype. The display itself was not considered as a source of data, even though it was a creative expression of the children’s experiences. This display was much less closely tied to the concerns and priorities of the researchers conducting the overall evaluation and so was seemingly overlooked by them. Indeed, this reinforces the argument being made here that children’s participation in design processes has remained somewhat tokenistic and largely undocumented and underexplored. Beyond these examples, few have investigated the effects of participatory technology design for children, and so its full potential as a means of learning, skill acquisition, and empowering children to use their voices to affect change is relatively unknown.

A rare exception comes from Spiel et al. (2017) who demonstrated how autistic young children’s contributions could be respected within the evaluation of participatory design processes, by coproducing evaluations which prioritised the perspectives of the children rather than adult researchers and other stakeholders. For example, a Super Mario alarm clock developed in an earlier co-design project (Frauenberger et al., 2019) was evaluated by creating an advert for the alarm clock, which linked to the child’s interest in creating newspapers (Spiel et al., 2017). Spiel et al. (2017) stressed the role of the researcher as a facilitator of these activities who provides scaffolding and support so that children can participate in ways which suit them. Accordingly, they respected the complexity of children’s experiences and the validity of their different ways of knowing, especially those ways of knowing that are communicated without speech or language. The adoption of a participatory research methodology is the main way we prioritise autistic children’s perspectives within our research. However, we also go further than Spiel et al. (2017) by extending participatory evaluation to examine children’s experiences of participatory design more holistically and throughout the process, rather than focusing primarily on participatory evaluation of the objects or outputs resulting from design processes. In order to do this, we have chosen to employ Digital Stories to explore the experiences of the young people involved in our research.

Digital Stories are short films which consist of video footage, images, and narrative slides which depict practices and experiences in educational settings (Parsons et al., 2015, 2020; Guldberg et al., 2017). They were initially used to explore the experiences of autistic children interacting with educational technologies in a research context (Parsons et al., 2015), and have also been used to support the transitions of autistic pre-school children by providing a more holistic view of the child for the adults involved in those transitions (Wood-Downie et al., 2021). In both contexts, Digital Stories were child-centred, but often largely produced by adults; here we extend the methodology such that autistic children and young people take a more active role in the co-production of their own Digital Stories. This process is documented in more detail below in Digital Stories for Documenting the Process of Participation.

Consequently, this research examines autistic children’s experiences of a participatory design process, from their own perspectives, by engaging in participatory research which respects and values their different ways of knowing about the world. This examination takes a holistic approach to their experiences, including a focus on the potential of participatory design as a tool for learning, as this is an area which is yet to be studied specifically with autistic children. We aimed to answer the following research question:

What insights can co-produced research give into the experiences of autistic children and young people involved in participatory technology design?

Materials and Methods

Following Harrison et al. (2011), we adopt an embodied constructivist epistemology which draws upon embodied experience as an important way of knowing about the world. Consistent with the work of Heron and Reason (1997), we see embodied experiences as underlying propositional knowledge (linguistic conceptualisations of knowledge), presentational knowledge (creative expressions of knowledge), and practical knowledge (behavioural manifestations of knowledge). By emphasising experiential knowledge like this, we embrace the situatedness of knowledge, and consequently recognise the importance of understanding the contexts through which different stakeholders’ perspectives are formed, and the different ways in which knowledge can be expressed. Consequently, the differences in experience across different neurotypes may be exaggerated as people’s neurology may also influence the way they express their experiences as propositional, presentational, or practical knowledge. Acknowledging the differences in embodied experiences which accompany neurological differences is particularly important when considering the experience of autistic people, as studies have shown considerable diversity in the neurology of those sharing a diagnosis (Toal et al., 2010; Lenroot and Yeung, 2013). The diversity of autistic experience resulting from these neurological differences is reflected in its status as a spectrum condition (Lai et al., 2013). The spectrum also indicates the different communication needs, preferences, and strengths of the autistic community and the resulting differences in the ways autistic people might experience the world. With respect to participatory design, the centrality of embodied experiences to understanding and interpreting the world, and the resulting recognition of the varied ways knowledge can be constructed and shared, means more flexible data collection methods are necessary to capture the inherently localised negotiation of meaning during knowledge co-construction through designing with autistic people (Gunkel, 2018).

Co-creating Research for and About Design

Following Kagan et al. (2006), we recognise the similarities between action research and participatory design. Like participatory design, action research is explicitly iterative, however the nature of these iterative cycles are different (Hayes, 2011). We conceptualise participatory design to fulfil a similar role to the actions implemented within action research. Specifically, the project follows an action research-type spiral, shown diagrammatically in Figure 1, whereby the iterations of participatory design are analysed and reflected upon before the next iteration begins. Along with the adoption of inclusive (participatory) methods which allow stakeholders to express views as they wish, an iterative approach allows adjustments of representation, ensuring that the views of less powerful stakeholders are equitably captured. As in Hayes (2011), the notion of a spiral rather than a cycle reflects the way each iteration changed our understanding, with the consequence that each iteration starts from a different position.

FIGURE 1

Figure 1. Diagram of iterative research progress.

Research Context and Participants (Co-creators)

The project was conducted at Fairmead School,² a special school which caters for autistic students, and students with moderate learning difficulties aged 4–19 (Fairmead School, 2020).

In total, 14 key stage four³ students engaged in the research as co-creators, we were supported by eight members of school staff, and four technology professionals from outside the school. Students were aged 13–16 years, and included 2 girls and 12 boys. Ethnic and diagnostic information was gathered through opt-in access to the students’ Education Health and Care Plans (EHCPs; UK Government, 2022a). Of those whose parents’ agreed access to their EHCPs, all identified as White British and most were on the autism spectrum or had autistic traits. Many also had co-occurring conditions such as ADHD, dyslexia, global developmental delay, or other learning disabilities. Accordingly, it was paramount that consideration was made with respect to the neurodiversity of the group, and the subsequent need for flexible data collection methods which allowed for communication and expression of different ways of knowing.

As students participated in multiple ways, their named role within the research must reflect this. Within action research, the groups that collaborate with researchers to co-produce research are often termed co-researchers (Martin et al., 2019), whereas within participatory design, those who collaborate with researchers and designers are often called co-designers (Holone and Herstad, 2013). However, as the role children occupied within this research straddles both these activities neither seems to be an adequate description for them and so we use the term co-creator to describe their role. This not only reflects the duality of the children’s roles, but also highlights their contribution to co-creating the knowledge which results through their involvement in research and design.

Co-creating Computer Games in the Coding Club

We used a participatory action research methodology that involved students designing, developing and evaluating a computer game with support from staff where needed. This methodology also included students documenting their own process of participation through the co-creation of Digital Stories, which are described in more detail in The Design Artefact: ‘Birds with Guns’ below. This work was completed during extra-curricular enrichment sessions held during regular school time starting in the 2020/2021 academic year. These sessions were colloquially known in school as the “coding club.” In total, 26 sessions took place between December 2020 and July 2021, an overview of research progression is shown in Figure 2. A brief overview of the activities of each session and the corresponding opportunities for learning, knowledge co-construction and sharing can be found in Supplementary Material.

FIGURE 2

Figure 2. Timeline of research progression.

Sessions were structured with each consisting of a set of activities for the students to work through in small groups; initially these groups were three or four students but restrictions due to COVID-19 in January 2021 meant the groups combined to work online via Google Classroom as a single group. This structured approach was informed by Benton et al. (2014) who suggested that it can improve the predictability of sessions for neurodivergent co-designers, thereby reducing anxiety associated with uncertainty. The structure of each session was presented at the beginning of the session and was included in a schedule on the front page of an accompanying resources booklet (see Figure 3 for an example). The content of activities was based around those with which students were familiar, and supplemented by established participatory research activities such as diamond nine and speech bubble based feedback activities (McCabe and Horsley, 2008). Sessions lasted around 90 min.

FIGURE 3

Figure 3. Example schedule provided for students with resources pack for week 22 of the participatory technology design project.

The design problem to be worked on was co-created by the researcher and school staff supporting the project, and was introduced to student co-creators in the first session. The brief for the project was intentionally very open so as to maximise children’s agency and contributions. Few restrictions were placed on the limits of the co-creators’ contributions, although co-creators were encouraged to think about the feasibility of building the games themselves. Game development was mostly undertaken by the students, with some support from the lead researcher (first author) and the school staff supporting the coding enrichment sessions. For those with less coding experience, worksheets were produced to guide them through creating their game. Student involvement in development aimed to further develop and demonstrate their practical knowledge, which was initially developed during the early stages of the coding club.

In addition to the support from school staff, we aimed to maximise the opportunities for learning by engaging with representatives from the technology industry who are involved in developing software professionally. The aim of these sessions was to promote the mutual learning which has been reported across interdisciplinary teams in other participatory design projects (see Bratteteig, 1997; Robertson et al., 2014 for example). Speakers had skills relevant to the activities for a given session; for example, a software developer attended (virtually) on the week the group worked on their plan for programming the game. Following discussions with collaborators at school concerning the possible stress to students relating to meeting new people, we only planned to have guests on three different weeks of the project.

The Design Artefact: “Birds With Guns”

The game which students created is called “Birds with Guns.” It is a retro-style 2D shooting game, where the player may choose from four bird characters which must shoot zombie-birds to pass through the levels and win the game. The artwork for two of the player’s characters, the zombie-birds, including the effect of being hit, and the backgrounds were created by students involved in the project. Students composed an original soundtrack for each of the levels and recorded a voice-over with instructions on how to play the game. As such, we see the game itself as the result of the practical knowledge which students had and developed in coding, graphics, and music composition, and a reflection of their presentational knowledge about the ideas which were co-created through collaborative discussion and ideation in early stages of the project. Figure 4 shows a selection of screenshots from the game including custom characters, zombie-birds, and backgrounds developed by the student co-creators.

FIGURE 4

Figure 4. Screenshots from the game developed by student co-creators, showing the opening screen, character menu, and levels two and three, respectively (from top left to bottom right).

Digital Stories for Documenting the Process of Participation

The project was documented through a variety of methods. The lead researcher collated design outputs from each session, including students’ drawings, digital images, voice recordings and compositions, and produced written observations and reflections after each session. Additionally, student co-creators took part in interviews, which they chose to do in small groups of two or three alongside the researcher. Interviews were semi-structured and based around an interview schedule which had been co-produced by the lead researcher and the student co-creators during earlier coding club sessions. In some cases, students chose the researcher to act as interviewer, but in others they chose to interview each other, with the researcher providing support where needed.

As well as building understanding via more traditional forms of data collection, we employed the Digital Story methodology to gain deeper insights into the experiences of the student co-creators. Parsons et al. (2015) argued that Digital Stories can be a way for unheard voices to capture their own experiences of educational practices through collaborative filming, editing, and narrative creation. They maintained that through the generation of Digital Stories, those authoring them can communicate their own experiences and knowledge, as their choice of what to film and what footage to include in Digital Stories make explicit their own priorities and sense-making practices (Parsons et al., 2015). Accordingly, the stories also potentially avoid the privileging of researcher perspectives, which more traditional data-collection methods can reinforce.

The process of making the Digital Stories involved four stages: filming, footage preparation, planning, and editing. Student co-creators were introduced to idea of Digital Stories in the first session, along with the protocols for filming. Each week a different student (or sometimes pair of students) took a turn to film for the session. The researcher then prepared the footage for inclusion in the Digital Stories: trimming long videos into shorter clips, labelling them according to who and what they show, and organising them so they could be easily navigated by the students.

Toward the end of the project, student co-creators spent a session reflecting on their involvement, and planning what they wanted to include in their Digital Stories. The lead researcher then collated their ideas so that students could use them to build their Stories. Finally, student co-creators worked individually or in pairs to edit together their Digital Stories by selecting narrative phrases from their collective pool of ideas, combining them with clips from the archive of footage, and using editing software to add music, visual effects, and animations. Though almost all the video footage was made available to student co-creators while editing together the Digital Stories, the influence of the researcher’s interpretation and organisation of this footage, and the scaffolding provided for editing the stories was not neutral, and so while we consider the Digital Stories to be a strong representation of student co-creators experiences and perspectives on their participation, it is the case that they were also co-produced.

Multimodal Theory, Data, and Analysis

Theory and Data

The importance of different ways of knowing and communicating knowledge underpins our adoption of multimodal theory and analysis. Multimodal analysis relates to theories of social semiotics where communication is conceptualised as a series of signs (Kress, 2010; Jewitt, 2013). Multimodal theory, upon which multimodal analysis is based, originates from Halliday’s (1978) linguistic theory of social semiotics, however more modern interpretations of multimodality challenge the primacy of language by extending social semiotics to other forms of communication, i.e., modes (Kress, 2010; Jewitt, 2013). Each mode has its own affordances and limitations which are shaped by both their own nature (materiality) and by the social experiences of the communicator with that given mode (their communicative competence) (Kress, 2010).

The meaning behind any given expression is not assumed to be determined by macro-level social norms, established through largely non-autistic communication, but instead by the previous experience of the communicators. In other words, the nature of the relationship between any given signifier and what it means is non-arbitrary; within social semiotics this is encapsulated by the notion of the motivated sign (Kress, 1993). Through the motivated sign we understand that the use of a particular sign by a sign maker reflects their experiences, and the choice between semiotic resources available to them at the time they create the sign (Kress, 2010). Consequently, our adoption of multimodal analysis highlights the agency of autistic co-creators in creating and communicating meaning through all the modes available to them in any given context (Jewitt and Henriksen, 2016). Furthermore, it highlights the role of the researcher as a communication partner, who has an active role as the interpreter of this meaning (Kress and Bezemer, 2015).

Within this context, learning is seen as the remaking of signs by the learner, based on their previous experiences of the sign (Kress and Bezemer, 2015). The extension of social semiotic multimodality to learning theory emphasises the need for allowing different ways of demonstrating knowledge, outside traditional means of data collection. As the Digital Stories provide a particularly flexible means of representing the experiences of student co-creators that allow students to draw upon a wide variety of modes and semiotic resources, our analysis draws heavily upon them as a starting point for data analysis. Our interpretations are strengthened by drawing upon other sources, including co-produced interviews, design outputs, and researcher reflections mentioned in the previous section. The aim of our social semiotic analysis is to investigate the process of meaning making, looking at both the sign itself (the content of communication) and the sign makers’ interest (how choices reflect the sign makers experiences and the context of sign-making).

Data Sampling and Transcription

The analysis of this multimodal data focused on micro-level details from a range of communicative moments to draw out wider conclusions about student co-creators’ experiences of the project, and how they chose to represent and express these within the co-produced data. Consequently, initial stages of data familiarisation, viewing and sampling are key to the analysis procedure (Jewitt, 2015). In our case, sampling started with the Digital Stories, as they offered a uniquely rich representation of student co-creators’ involvement. Other communicative moments were selected for transcription based on themes which the first author identified within the Digital Stories and other data sources. Once key moments were identified, they were transcribed.

The process of transcription is considered crucial within multimodal data, as researchers must choose which modes to transcribe, and how these transcriptions will be organised (Flewitt et al., 2009; Bezemer and Mavers, 2011). Transcription is a key aspect of analysis for the researcher, in terms of their own understanding of the data (Bezemer and Mavers, 2011). Within the social semiotic approach, transcription can be seen as the remaking of signs (Bezemer and Mavers, 2011), and transduction of meaning from one mode to another. As such, transcription is a learning process (Kress and Bezemer, 2015), through which the researcher gains deeper insight into the original instance of sign-making.

We made two types of transcriptions: micro- and macro-level. Macro-level transcriptions give insight into the relationships between large complex multimodal objects, whereas micro-level transcriptions aim to highlight the relationships between smaller units of multimodal compositions (Flewitt et al., 2009). Here, we made use of micro-level transcriptions of the Digital Stories. For this rich, multi-modal data, the process of transcription allows researchers to produce more detailed descriptions of these complex multimodal artefacts, which supports the initial stages of analysis, focusing on the content and composition of different modes (Jewitt, 2015). We also used macro-level transcriptions which set the interim design artefacts (including images, music, and storyboards) within the context of researcher observations and reflections. This was intended to connect the instances of meaning making with the context in which they were made.

Stages of Analysis

Analysis then proceeded in four stages using NVivo 12:

1. Examination of individual modes including the creation of inventories of modes, semiotic resources, and how they were used and configured (Jewitt, 2015; Jewitt and Henriksen, 2016). In practice, this was realised as a close reading of original data alongside the transcript and coding the transcript for meanings and modes within it. Using this coding, we used a search across the mode-related and meaning-related codes to create an inventory of modes and meanings. We also used memos linked to each of the codes to reflect upon the opportunities and constraints of each of the modes.

2. Consideration of the composition of modes and how interacting modes created or transformed meaning (Jewitt, 2015; Jewitt and Henriksen, 2016), through the comparison of memos and code for each mode, and the creation of further memos reflecting the composition of meaning across modes.

3. Linking the data with the context of meaning making through comparison across sign-makers and contexts (Jewitt, 2015; Jewitt and Henriksen, 2016). This involved comparing data from different communicators and contexts to strengthen conclusions about the interests of the sign-makers.

4. Drawing connections with wider social theory (Jewitt, 2015; Jewitt and Henriksen, 2016). In this case we turned to the concepts of participation, learning, and the different ways of knowing about the world referenced earlier.

Results

To explore the data in more depth and provide context for our analysis we draw upon the contributions of two students: Jay⁴ and Terry, who are good friends. Jay and Terry were involved with the project from its beginning, having chosen to do coding enrichment for the 2020–2021 academic year. However, their involvement was interrupted by the restrictions on face-to-face teaching due to the ongoing COVID-19 pandemic. Consequently, they missed several of the sessions during which the initial ideas were combined to produce a single idea (“Birds with Guns”), which was then developed further. When they re-joined the group, we were working to bring the idea to life, by programming the basic game mechanics. Jay and Terry engaged with the project in several ways: they contributed to initial game ideas, imagined extensions to the game as it developed, implemented some of these ideas, helped with formative testing, and developed a Digital Story together. As described in the section above, their Digital Story is the starting point for this analysis.

Their Digital Story is structured around two types of shot: a series of narrative sentences, presented as (relatively) static slides, interspersed with clips of footage filmed by student co-creators. The process of creating the Digital Stories included selecting clips from the large catalogue of footage filmed over the course of the 20 sessions during which the students designed and built the game. A brief overview of the contents of their Digital Story is shown in Table 1, and further details about the distribution and modes employed across the different types of shot are shown in Table 2.

TABLE 1

Table 1. Overview of Jay and Terry’s Digital Story about making “Birds with Guns”.

TABLE 2

Table 2. Distribution of meaning across shot types in Jay and Terry’s Digital Story.

Modes, Semiotic Resources, and Meaning Potentials

Student co-creators were involved in decision making on multiple levels while creating the Digital Stories. Not only did student co-creators choose what to film during coding club sessions, but also how to film it (choosing the equipment and framing). At the editing stage, they chose what clips they wanted to use, and the narrative sentences to use to frame the clips. For this first stage of analysis, we focused on the surface level meaning and content of the Digital Stories, rather than higher level meanings which might be inferred from how modes are combined in the Digital Story as a multimodal artefact (this is covered in Multimodal Composition and Design), and meanings inferred from the selection of one clip over another (addressed in Sign-Makers and Context) in relation to how this reflected the choices of Jay and Terry as sign-makers.

Within the Digital Stories we identified 11 modes through which the student co-creators could encode meaning. These modes were then coded individually for the meanings which the students chose to embed within each mode. The results of cross-referencing the modes and meanings for Jay and Terry’s Digital Story are shown in Figure 5.

FIGURE 5

Figure 5. Diagram showing use of individual modes in Jay and Terry’s Digital Story about making their game “Birds with Guns”.

Each mode had its own constraints, relating to its materiality and what was available for use as semiotic resources. For example, footage was limited to the location of the sessions, and the additional media overlaid on the footage for the Digital Stories, such as animations, sound effects, and music, were limited to those which were readily available within the editing software used. However, within these limitations, students found ways to express their agency and creativity. Within Jay and Terry’s Digital Story, they included footage of the car park outside the classroom, filmed from within (shot 2), presented interactions concocted for the purpose of the Digital Story (shot 10), as well as exploratory and observational footage of the ongoing session (shot 10). Though the instructions for filming had been intentionally open, these extensions of filming practices beyond simply capturing the activities within the session demonstrate the extent to which student co-creators took control of filming for the Digital Stories in their own ways.

Multimodal Composition and Design

As mentioned above, the composition of the Digital Stories can be roughly divided into two types of shot: the composition of the narrative slides (shown in Figure 6), and the video clips overlaid with music, animations and sound effects (described in Table 1). These two multimodal artefacts were then combined to create the full multimodal artefact which is the Digital Story. The distribution of meaning across different modes in each shot type are shown below in Table 2.

FIGURE 6

Figure 6. Narrative slides used in Jay and Terry’s Digital Story in shots 1, 3, 5, 7, 9, and 11 (respectively from top left to bottom right).

Within the narrative slides, the text contains literal descriptions of students’ participation, whereas the font, colours and music reflect more holistic messages about the tone of the video. Importantly the text captures both habitual actions (i.e., the act of programming- “copying code” in shot 5) and key moments for co-creators, for instance drawing ideas (shot 3). As shown in Figure 6, the narrative sentences describing the activities are accompanied by brightly coloured slides and thick fonts, suggesting a sense of fun and enjoyment. These choices are also complemented by music, which begins with a percussive rattle and continues with an upbeat rhythmic riff, suggesting a positive and energetic tone. These choices are applied consistently to the narrative slides describing the project, despite wording of text not always reflecting the same level of enjoyment. For instance, the text for shot 5 references “copying code,” which is an otherwise neutral description of their coding activity, however the colours and font are consistent with students’ positive attitude toward it.

The last two narrative slides serve different purposes, drawing upon references from popular culture. The penultimate slide structures the Digital Story, marking the end of their descriptions of involvement; the delicate, curly white font and greenish-blue background are reminiscent of a fairy tale. The colours continue into the final slide, but the font is simpler, and much easier to read. The text recognises Jay and Terry as authors of the Digital Story and acknowledges another student and the researcher who helped them. The final text acts as end credits, emphasising the importance of doing the project together.

Within the sections of student footage, co-creators mostly chose to distribute meaning in a similar way. The content of video clips mostly detailed the experience of being involved while the music and overlaid effects enhanced the playful tone of the video. For instance, the video clip for shot 6 featured a student working at a computer and is overlaid with both an explosion and a “live” sign, evoking practices that might be seen in television news coverage. This adds interest to the otherwise fairly static, low-energy shot. In contrast, no additional effects were added to shot 8 which featured more movement of the camera as the student filming explores and observes the classroom. The final video clip, shot 10, does not seem to refer to their experiences of building the game at all.

Sometimes, the descriptive text and accompanying video appeared to work together to form a coherent narrative, but at other times this was not the case. For example, there is a dissonance between the context of the narrative slide in shot 1, which introduces the project “we made a game using fuze,” and the content of the footage shown in shot 2, featuring the explosion effect over video of the car park. In contrast, the footage in shots 4 and 6 featuring students working in the classroom ending with one developing artwork and explaining their ideas, respectively, pertain more closely to the text in the corresponding narrative slides (in shots 3 and 5). The meaning suggested by these mismatches are explored further in the next section.

Sign-Makers and Context

As discussed above, the meaning in the dissonance between narrative slides and the accompanying footage is revealed when considering Jay and Terry as sign makers and the context in which the Digital Stories were made. For instance, the mismatch between the text in the first shot and the footage of the car park in the next can be seen to reflect the students’ sense of humour, and the playful nature with which they approached making their Digital Story. Their intention for humour was reflected in how they named their initial version of the Digital Story which included the first narrative slide and video clip: “Jay and Terry’s funny video.” The playfulness and enjoyment which Jay and Terry experienced through their participation in the project was also evidenced in their interview. Terry revelled in playing the game, describing the enjoyment gained from the effect of holding down the space bar to fire the gun, and the resulting noise and animation of the ammunition being fired. He also spoke about the enjoyment of the more relaxed working environment compared to other lessons, described coding as a “soothing” activity, and spoke about the opportunity to talk to friends and express critical opinions about the game “without being told off.” The focus and significance of communicating fun is something which may have been missed if relying on more adult-led data collection methods that did not centre student co-creators’ voices in the same way. For instance, the researcher’s reflections rarely mention the extent to which individual students enjoyed sessions, instead tending to comment on the activities students chose to do (or not do) and reflections of how she could facilitate their engagement further. This further highlights the need for data collection methods which represent students’ rather than purely researchers’ interests.

The Digital Stories also allowed student co-creators to emphasise the significance of particular moments. In contrast to the other sections of the Digital Story, which describe recurring activities or give general information about their experiences, shots 3 and 4 describe and show Jay explaining his ideas for the game. The realisation of Jay’s idea into a level within the game was an important moment for him. In an interview, he related his answers to multiple questions back to this experience, reinforcing the value he placed on seeing his idea through, and the satisfaction and pride he drew from seeing it realised within the game, especially given that there was insufficient time to implement all his ideas.

“I just came up with the new ideas which, were very good, but some didn’t make it into the game.” –Jay, evaluation interview

The limitation on time to respond to Jay’s ideas was a product of the restrictions on student movement between classes which had been imposed because of the ongoing COVID-19 pandemic. Although Jay and Terry missed sessions during which the other student co-creators developed the initial designs for the game, flexibility in the schedule meant that the researcher was able to integrate his idea into the program of work. This involved developing worksheets to support student co-creators in implementing Jay’s design. The researcher noted Jay’s pleasure at seeing this during the session in which it was implemented.

“I explained that the last part of the coding for the day was implementing Jay’s idea which he had mentioned the last two sessions. He smiled and seemed enthusiastic to get to that part.” –Researcher reflection

However, the inclusion of the moment in which Jay explained his idea to the researcher in their Digital Story stresses the significance for Jay of feeling as if he had some influence over the game, and feeling that the game was, to some extent, his creation.

This sense of influence and ownership over the game appears to be extended to the production of the Digital Story. Despite student co-creators having access to all the footage, Jay and Terry only chose to use their own footage, or footage which they contributed to producing. Only shot 10 was not filmed in a session in which either Jay or Terry had control over the camera. Unlike the rest of the shots chosen for the Digital Story, shot 10 was filmed using a tripod during a session when another student, Levi, had control of the camera. Levi presents facts about computer programming directly to camera, with Jay joining to add his own knowledge about binary code, and the history of computers. The following quote from the researcher’s reflections on the session during which it was filmed gives some context about the way Levi and Jay worked together to create it.

“Though Jay had chosen to do coding, he was very interested in the video camera which Levi was using. In his normal way, Levi was totally absorbed in the process of filming, and also wanted to create material for the Digital Stories, not just film what was already going on. He pulled me over a few times to ask me questions about coding, and I could hear him narrating other people’s work throughout the session. Jay joined him a few times, and I saw him operating the camera while Levi was in front of it, presenting to it.” –Researcher reflection

Unlike other clips which Jay and Terry chose to include in the Digital story, the clip of Levi presenting facts about coding neither describes the students’ involvement or experiences, nor does it particularly add to the sense of fun, instead it appears to be showing the skills and knowledge of the two students featured. Through this clip, Jay displays his knowledge about computing, and the other student shows off his showmanship and skill at engaging with the camera and the imagined audience (he had aspirations to work as an actor or in film). The video therefore acts as a way of capturing the students’ other related knowledge, including actively promoting Jay to the role of expert knower.

Connections to Theories of Learning and Participation in Design

As mentioned in the Introduction, assuming the role of an expert knower is key to the process of mutual learning which is at the heart of participatory design. The other side of this process is in recognising gaps in knowledge and being open to the expertise of others, that is recognising the symmetry of knowledge within the group designing together (Fowles, 2000, p. 63). The inclusion of the final clip, although contrived for the Digital Story, is important in the sense that is appears to show Levi and Jay establishing some kind of symmetry of knowledge. Jay’s interruption of Levi’s monologue about programming results in Levi recognising his own misunderstanding and accepting Jay’s interpretation. However, the distribution of labour across this clip, where Levi presents directly to camera and Jay pops in to provide understanding recognises each of their strengths. Where Jay provides propositional knowledge about the meaning of the word binary, Levi demonstrates his practical knowledge about engaging with a camera and the eventual audience. The inclusion of this clip within Jay and Terry’s Digital Story can therefore be understood not only as Jay actively assuming the role of expert knower, but also recognising Levi as one too, though his knowledge pertains to showmanship and communication with an audience.

Though Jay and Levi do not include all the activities which co-creators engaged in during the process of making the game, their Digital Story shows an understanding of the range of activities which go toward making even a simple game. This learning is also evidenced in interview data, in which Terry reflected upon the contributions of the whole group to making the game and remarked upon how long it had taken to make. He related these experiences back to his own knowledge of Minecraft, which is built by a team of professionals, rather than a relatively small class of school students. Through the experience of making the game, he had come to more fully realise the extent of work involved in creating the types of games he enjoys, despite having missed the sessions with direct input from technology professionals. Across both their Digital Story and interview, Jay and Terry demonstrated their own learning as theorised within social semiotics (Kress and Bezemer, 2015), through the making and remaking of signs which demonstrated their experiential knowledge of designing and developing “Birds with Guns.”

In terms of their participation, the Digital Story suggests that Jay and Terry experienced their engagement with the project as both meaningful and enjoyable. Notably, we see significance assigned to moments where they were enabled to practice their agency to a greater extent, and take ownership of specific ideas, such as Jay’s idea about having birds fall from the sky in one level. In contrast, co-creators recognised elements of the process in which their agency was felt to be limited by the scaffolding provided to them. For example, Jay and Terry included “copying code” as their description of programming as part of their Digital Story, rather than language which might reflect that they developed it for themselves. This highlights the difficulty in providing structure and support that enables students to fully participate in activities without limiting their contributions. As relatively novice programmers, for whom some of the required programming syntax was new, providing examples and templates was necessary to enable the student co-creators to contribute to building the game. Furthermore, the complexity involved in building even a simple game from scratch is considerable, and so the time limitations placed on the project because of changing timetables between school years meant that ensuring the game was completed by the end of the academic year was a necessity. Consequently, the students’ experience was one of “copying code” which was written by the researcher, rather than writing their own. Despite this, the activity of programming the game was a positive experience, and this was reflected in both the Digital Story and the interview discussed above.

Overall, the Digital Story communicates a sense of the fun which Jay and Terry took from the project, the activities which they enjoyed, the significance of seeing their ideas realised, and being seen as experts. Additionally, the Digital Story methodology appears to have provided a way for student co-creators to effectively communicate their understanding and perspectives on designing and building “Birds with Guns,” including providing them with ways to demonstrate their different forms of knowledge and ways of knowing.

Discussion

In this article, we have drawn upon Digital Stories and other co-produced data to explore the views and experiences of neurodiverse students who designed and developed a computer game during a coding club. In particular, we have focused on the insights gained from a detailed multimodal analysis of the Digital Story produced by two students, Jay and Terry. Based on these findings, students appeared to enjoy and value their experiences of co-creating their game “Birds with Guns.” Through the Digital Story, interview data and researcher reflections presented in this article, we see the significance students placed on their enjoyment of the process, which might have otherwise been missed or recognised less fully if solely relying on adult-led or generated data. The Digital Story methodology that documented student’s involvement in the project provided an opportunity to capture different forms of knowledge with the student co-creators. Filming allowed individual student co-creators to approximate experiential knowledge through the capturing of audio-visual data which was literally from their perspective when they held the camera. Filming also captured the behavioural manifestations of practical knowledge displayed by other co-creators, such as Levi’s knowledge of addressing an audience.

The process of selecting and editing footage allowed co-creators to construct their own multimodal narrative about their involvement in the project and choose how to communicate it to others, using their own footage and any other resources which they had available, such as music, animations, and sound effects. Furthermore, the freedom and space which the Digital Stories allowed seems to have furthered how meaningful the project was, with students creating opportunities for their own sense of playfulness within the methods. In particular, students made the most of available music and special effects within editing software to set a fun, playful tone. We therefore see the adoption of the Digital Story methodology as a successful response to calls from other participatory design researchers such as Spiel et al. (2017) for the adoption of more flexible data collection methods. Our findings show the value of Digital Stories as a flexible data collection method which move beyond adults’ concerns and perspectives.

The nature of this research as school-based, especially within the context of the COVID-19 pandemic raises several issues related to research participation. Most notably, the move to online learning and the subsequent restriction of student movement within school meant that some students were not able to consistently engage in the project throughout. This meant the extent to which students such as Jay and Terry could influence early stages of design was considerably reduced. However, opportunities for them to build upon the design as it was being developed were seized, and Jay drew considerable value from this aspect of his engagement with the project. Consequently, it appears that the allowance for Jay’s contributions was sufficient to avoid the frustration which has been reported by other participatory design researchers trying to negotiate and facilitate the contributions of different stakeholders over the course of a project (Bossen et al., 2012).

Though the extent to which students were able to guide the process of game development was somewhat more limited due the perceived need for scaffolding to enable continual progress within the time available for the project, the opportunities to integrate their own ideas and decisions was extremely important to students. The conception, development and integration of Jay’s idea which formed the third level of the game was prominent across the data he contributed to, and a defining feature of his experiences of the project. This highlights the need for allowing students to have a say rather than merely have a voice (van der Velden and Mörtberg, 2014). However, we must also recognise the important role of researchers and designers as active listeners who must respond to students’ voices and this responsibility comes with inevitable power over the participatory design process. Even where students are building the outcomes of designs themselves, they will likely require scaffolding to do so, and as providers of that scaffolding it is up to us to adapt to students’ contributions to design. It is therefore a requirement that there is enough flexibility within project plans to respond to the students’ emerging contributions. In our case, a more rigid schedule of work with more clearly defined boundaries between design and development phases might not have allowed for Jay’s contributions to be integrated, meaning the value he gained from seeing his idea realised would have been lost.

With respect to students’ learning, Jay and Terry developed a much greater understanding of game-making through their engagement with the project, as shown through the variety of activities included in their description of participation in the Digital Story. Terry also reflected upon his own learning about game development in his interview. Through both the Digital Story and the interview, students developed skills relating to collaboration, and working together in ways that drew upon people’s strengths. In his interview, Jay recognised his own strength in generating ideas, and there was recognition of teamwork through the end credits of their Digital Story. Once again, this illustrates that the flexibility and agency afforded to students to choose how they engaged in participatory design was key. Following Bratteteig and Wagner (2012), we see this recognition of individual strengths and knowledge as an empowering process for student co-creators which is further strengthened by the agency they showed in adopting and adapting the resources available through the creation of the Digital Stories for their own means.

These findings highlight the crucial insights which Digital Stories can provide into the experiences of autistic children and young people within participatory design, and the value which they place on such experiences. As such this research demonstrates some of the potential benefits to autistic children and young people of being included in participatory design processes, where there is scope for them to do so in a meaningful way. However, further research is needed to understand how this might occur in projects with more fixed objectives, where there is more potential for conflict between ensuring meaningful participation for autistic children and young people, and achieving research or design aims, as in Parsons and Cobb (2014). Such research may choose to draw on the Digital Stories methodology as a means for accessing the views and experiences of any participants involved, as this article has shown it to be an insightful way of gathering their views. This article further demonstrated the extent to which autistic young people can take agency and responsibility over Digital Story co-production, ensuring that they are a stronger reflection of students’ experiences and priorities. In doing so, this article has also shown the potential for the use of co-produced Digital Stories as a means for schools to capture student’s views on other educational experiences.

Conclusion

The Digital Story methodology allowed for less constrained description of experiences, enabling expression outside what was necessarily intended or expected by researchers. The nature of Digital Stories as a multimodal artefact was key to this, where the wider choice of modes allowed for expression of knowledge which may not readily be expressed through spoken or written language. Furthermore, this research has suggested that even within the limitations of school-based research, it is possible to facilitate participatory design in a meaningful way that allows students to feel they have influence over outcomes. Within this context, participatory design can be an empowering process through which a developing sense of symmetry of knowledge allows students to recognise both their own strengths and the strengths of others.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The studies involving human participants were reviewed and approved by the University of Southampton Education School’s Research Ethics Committee. Adult participants provided written informed consent to participate in this study. Written informed consent to participate was provided by the student co-creators’ legal guardian/next of kin. Student co-creators also provided written assent to participate.

Author Contributions

VW was responsible for the research design, data collection, and writing this manuscript. SP and HK provided supervision, critical review, and support in preparing the manuscript. BC helped to plan and coordinated the research at Fairmead School. All authors have reviewed, read, and approved this manuscript for publication.

Funding

The research for this article was funded by the Economic and Social Research Council South Coast Doctoral Training Partnership (Grant Number ES/P000673/1).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

We would like to thank Fairmead School for their ongoing support and collaboration.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/feduc.2022.864362/full#supplementary-material

Footnotes

^ We use identity-first language (i.e., autistic person, person on the autism spectrum) in line with the preferences of the majority of autistic adults in Kenny et al.’s (2016) study on the use of language, and self-advocates who use it to reflect the nature of autism as part of their identity (Brown, 2011; Sinclair, 2013).
^ This is the school’s real name. At their request, and given the invaluable contributions to the research which the school has made, we feel it would be unethical to hide their identity through the use of a pseudonym.
^ Key stage four is the final stage of secondary education in England during which students work toward national qualifications such as GCSEs (UK Government, 2022b).
^ To protect the students’ anonymity, these are pseudonyms. All names given for students within this article are pseudonyms.

References

American Psychiatric Association (2013). Diagnostic and Statistical Manual of Mental Disorders (DSM-5^®). Washington, DC: American Psychiatric Pub.

Google Scholar

Béguin, P. (2003). Design as a mutual learning process between users and designers. Int. Comput. 15, 709–730. doi: 10.1016/S0953-5438(03)00060-2