Uncovering inclusivity gaps in design pedagogy through the digital design marginalization framework

Designers play a key role in the design of inclusive and socially conscious interfaces. Thus, it is imperative for designers to be thoughtful of the ethical and social implications of design. However, gaps in the foundational training that designers receive (e.g., as university students) can negatively impact their ability to consider the social implications of their design practice. This can result in consequences such as digital marginalization, which, as defined by the Digital Design Marginalization (DDM) framework, is the “pushing away”, whether intentional or not, of a defined group of users from a digital or online service or system, where the exclusion has additional, indirect, and long-lasting social consequences on that particular user group. Designers can contribute, even unintentionally, to digital marginalization through their design practices and the design choices they make. We argue that our role as educators includes ensuring not only that our design pedagogy is inclusive, but that the designers we train now are prepared to conduct their future design practice in a manner that is inclusive to all users. As such, we propose to use the Digital Design Marginalization as a lens to guide a reflection-based approach to identify gaps in our pedagogy that may lead to designers becoming ill-equipped to identify how their designs may lead to digital marginalization. Through seven case studies from our own teaching practice, we demonstrate the use of the DDM framework to guide marginalization-focused introspective reflections of curricula. These reflections through the DDM lens revealed gaps in our pedagogy with respect to providing future designers with training that enables them to consider the broader societal and individual implications of the design choices they will make in future practice. Based on our experience using the DDM framework, we then discuss in greater depth how reflection of social consequences of design pedagogy can be operationalized within institutions to reduce educational gaps that may be associated with design-mediated digital marginalization. Finally, we comment on avenues for further development of pedagogical reflection using DDM.

Introduction

Essential services, such as personal finance, healthcare, social connectivity, and retail, are becoming increasingly–even exclusively–digital. Transitions to digital services risk leaving some users of these services behind. It is now more important than ever for interfaces and digital services to be designed and deployed in ways that factor their broader societal impact. Designers play a key role in users' access to and willingness to adopt digital services. This makes it imperative for designers to be thoughtful of the ethical and social implications of design and the impact of their design decisions, especially on users in vulnerable or marginalized communities. However, it is still not known how well designers' training prepares them to examine such far-reaching considerations and what gaps may exist in the ability of design pedagogy to help designers-in-training to consider the social consequences of their future designs.

While addressing these gaps requires multiple modes of inquiry, one starting point may be to reflect on current design education curricula and pedagogy. Self-reflections have been used and have been valuable in the past as illustrations of situational approaches to ethical issues in HCI work, e.g., in Munteanu et al. (2015). Additionally, inclusive design frameworks such as design justice also practice self-reflections for (e.g., Spitzberg et al., 2020), a study by members of the Design Justice Network Principles at Work Working Group).

To identify design pedagogical gaps that may result in far-reaching negative social consequences, we turned to the Digital Design Marginalization (DDM) framework (Sin et al., 2021). The DDM framework has been proposed to help expose the ways in which our designs can lead to users being pushed away and thus marginalized from both the actual design and from other aspects that may be connected to that design (e.g., services that have been transitioned from in-person to online). On the other hand, such designs are also the result of the foundational training that designers received. Many designers acquire this knowledge through university or college programs. In turn, our role as educators includes ensuring not only that our pedagogy is inclusive, but that the designers we train now will be equipped to conduct their future design practice in a manner inclusive to all users.

As such, in this paper we present the application of the DDM framework to help uncover pedagogical gaps and guide reflection of curricula in ways that better train future designers in considering broader societal and individual implications of the choices they will make in their future practice. We consider the DDM framework to be an additional tool to existing practices and approaches adopted around ethics and inclusivity (e.g., Microsoft's Inclusive Design Toolkit) by institutions when training designers. Through this paper, we aim to support, as examples, the increasing call for reflexivity in action research (Hayes, 2011) and the need to take care of risks exacerbating peoples' vulnerability, as supported by Sensitive HCI (Waycott et al., 2015). Our reflection is performed through our experience and expertise as educators and active researchers. Our academic background is grounded in (user experience) design education, as well as in the research and design of technology for populations that are often at risk of being marginalized, such as older adults, the homeless, indigenous people, disabled people, and people who are deaf or hard of hearing.

Through our reflections, which we present in this paper as case studies, we illustrate the types of gaps that can be identified by using the DDM as a reflection tool – namely, the shortcomings of our current curricula and pedagogical practices with respect to preparing designers to address long-term social consequences of their work. We further hope that readers of this paper can use our case study reflections as inspiration for similar introspection of their own curricula or consider further steps on how to practice pedagogy-oriented reflection through the DDM lens. Our work will also highlight the applicability of sociotechnical theories, such as digital marginalization and the DDM framework, in understanding the ways in which current design pedagogy can better train designers in avoiding practice that leads to the marginalization of certain user groups.

Designers-in-training will move on to create designs that can have great societal impact. In turn, as educators of design practice, we believe that analyzing our own pedagogical practices through the lens of DDM can help us better equip these designers to make more societally sound choices and adopt practices that minimize social harm to users. Through our case studies, we show that the DDM framework reveals gaps in what and how design methods are taught, how design courses are administered, and opportunities to enhance students' knowledge of DDM. Grounded in this introspection guided by the DDM framework, we then discuss in greater depth how the reflection of social consequences of design pedagogy can be operationalized within institutions, in order to reduce downstream design-mediated digital marginalization. Finally, we comment on avenues for further development of pedagogical reflection through DDM.

Background

Design education and inclusion

Inclusive design aims to design in a manner that accounts for the full range of human diversity through consideration of diversity and uniqueness, inclusive processes and tools, and awareness of the broader impact of design (Inclusive Design Research Centre, n.d.). Not much is known about how to best teach inclusive design to students (Oleson et al., 2018). However, evidence suggests that education within higher education institutions about the importance of inclusive design is valuable for motivating designers to use such techniques in their own practice (Zitkus et al., 2013; Lazem, 2021). Furthermore, research into the pedagogy of inclusive design approaches such as GenderMag (Burnett et al., 2016a,b), a set of personas to evaluate a design's gender inclusiveness, suggests that strong knowledge of how to teach inclusive design (in addition to strong knowledge about the topic of inclusive design itself) is important to positive student learning outcomes on this topic (Oleson et al., 2018).

Inclusive design is an area of interest in the design education community (Hanson, 2007; Koepfler et al., 2014; St-Cyr et al., 2020; Gray et al., 2021; Pillai et al., 2021) and in higher education institutions (Grosz et al., 2019), such as through Harvard's Embedded EthiCS program (Grosz et al., 2019). Such interest and programs are created in part in reaction to students' interest in ethical issues yet lack the skill set (even upon graduation) to tackle these topics (Grosz et al., 2019). However, challenges remain in fully incorporating inclusive design into design education such as the need for strong administrative support and the lack of teaching resources (Putnam et al., 2019).

In all, inclusive design is a current and relevant topic to design and HCI education. Additionally, evidence suggests value in incorporating inclusive design in designers' training. Yet, challenges exist in fully incorporating inclusive design into design education. This provides an opportunity for the use of the Digital Design Marginalization framework to serve as an additional teaching resource and motivate administrative support.

Efforts to reflect on design pedagogy

Reflection is a tool often used to discuss and improve upon design and Human-Computer Interaction education. For example, reflections have been conducted to understand challenges and start dialogues with the HCI community on teaching HCI to computer science undergraduates (Larsen-Ledet et al., 2019). Furthermore, reflexive self-studies have been presented in papers before to discuss perspectives and experiences on successes, challenges, and obstacles in the remote teaching of inclusive design in HCI courses (Byers et al., 2021).

Reflection is also an important tool for evaluating existing design practice as well. An example exists with “empathy”, which refers to the design practice and taught as an initial design phase of human-centered design processes to gain a better understanding of user needs and desires (Doorley et al., 2018). Empathy as a process is popular in design (Battarbee et al., 2015; Doorley et al., 2018), business (Deszca et al., 1999), and HCI (Mattelmäki, 2006; Wright and McCarthy, 2008; Dong et al., 2018). Empathy maps are taught in design education as a way for designers to synthesize and understand users' thoughts, feelings, and behaviors (Gray, 2018). However, empathy as a design technique has some shortcomings including potentially encouraging a “design savior” attitude (Irani and Silberman, 2016; Bennett and Rosner, 2019), promote feelings of fear, apprehension, or pity toward people with disabilities (Nario-Redmond et al., 2017), and fail to account for people's coping mechanism and capabilities (Nario-Redmond et al., 2017; Abreu, 2018) (Abreu, 2018; Bennett and Rosner, 2019). Concerns like these raise the need for designers to be reflective and critical about their design approaches and practices.

Overall, critical self-reflection is a valuable method for improvement of both design pedagogy and practices.

The digital design marginalization framework

The Digital Design Marginalization framework (Sin et al., 2021) proposes that digital marginalization is the “pushing away”, whether intentional or not, of a defined group of users from a digital or online service or system, where the exclusion has additional, indirect, and long-lasting social consequences on that particular user group. For example, cashless retail stores that exclusively accept electronic/digital payments risk marginalizing socioeconomically disadvantaged users (Wick, 2019). Furthermore, online food ordering apps that are incompatible with screen readers not only excludes blind or low-vision users from using these online services, but also reinforces existing social inequalities (McKee, 2019). In each of these cases, digital online services lead to offline social consequences for users.

Designers can contribute, even unintentionally, to digital marginalization through their design practices and design choices they make (Sin et al., 2021). For instance, not establishing risk mitigation plans or inadequate consideration of all the social actors interacting with the primary user may result in the technology pushing others away from the primary user, thereby exacerbating social isolation and loneliness. Additionally, designs for older adults that over rely on external tech support can contribute to family tensions leading to social isolation and loneliness. Many activities across the spectrum of digital design research and practice have the potential to harm users. In turn, the DDM framework (Sin et al., 2021) helps one understand the ways in which digital interface design leads to the exclusion of potential users and contributes to digital marginalization. The DDM framework serves as an additional tool to existing practices and approaches adopted around ethics and inclusivity by institutions when training designers. The DDM framework, as well as this paper, aims to support, as examples, the increasing call for reflexivity in action research (Hayes, 2011) and the need to take care of risks exacerbating peoples' vulnerability, as supported by Sensitive HCI (Waycott et al., 2015).

In practice, the DDM framework has been shown to be promising in retrospective application for identifying design gaps leading to social consequences. Sin et al. (2021) demonstrated the use of the DDM framework to identify gaps in a digital design of their own creation and evaluation (an application for older adults to socially connect with others), as well as one that was externally developed (accessibility features in mobile devices). This use of the DDM framework revealed gaps in existing designs and the design processes used to generate them. It is still not known how the DDM framework may be used to predict issues in designs that have yet to be put “in the wild”. However, the retrospective use of the framework to reflect on design parallels our aims to reflect on our own design curricula and pedagogy after it has already been employed in the classroom.

The DDM framework holds promise for identifying, through self-reflection, existing designs that could lead to unintended social consequences. By examining one's design practices and projects through the lens of the DDM, one can better anticipate, articulate, and take action to prevent or mitigate, the harms of unintended social consequences.

Case studies

In order to identify the ways in which our design curricula may be contributing to downstream digital marginalization through design, we reflected upon our courses for which we are instructors through the lens of the Digital Design Marginalization (DDM) framework (Sin et al., 2021). In this section, we present our reflections in the form of case studies. For each case study, we provide the context of the course, the instructor's reflection of the course through the lens of the DDM, and a discussion of the final conclusions of the reflection and actions or intervention that would be taken for future iterations of the course resulting from the reflection.

Each case study is written by a separate author. The seven authors are from two different universities across four departments affiliated with diverse disciplines (computer science, information sciences, communication and media). In aggregate, the authors teach a variety of Human-Computer Interaction-related courses tailored to their disciplinary affiliation, across undergraduate, professional masters, and PhD levels. The individual teaching experiences cover a wide range, from 2 years to almost 25 years of teaching. In the following subsections, all use of the plural pronoun “we” or possessive adjective “our” refer to the co-author who contributed the respective case study (as opposed to all the authors of the paper).

Our goal with these case studies is to present an introspective reflection on how we were able to use the DDM framework, rather than to verify whether the DDM framework can be used as a tool for reflection or to validate the DDM's applicability. In other words, we aimed to show the types of gaps that can be identified by using the DDM as a reflection tool. The following case studies are meant to serve as examples of the use of the DDM framework for reflection on pedagogy. This paper is additionally an invitation to other educators to consider using the DDM in their self-reflections. This paper is a start of this process already, as only two of the authors on the paper are experts in the DDM framework, with the remaining authors (from different universities) invited to join with their reflections. Only one of the case studies is from an expert of the DDM framework.

Case study #1: Uncritically teaching potentially ableist usability testing methods

Context

The first case study we reflect on is drawn from the context of a third-year undergraduate introductory course in user experience design and human-computer interaction (HCI). While this particular course is delivered in a social science department, it is part of a program of study that aims to prepare students in both understanding the issues connecting technology with society and in designing (interactive) technologies that are better positioned to address such issues. The department itself is highly interdisciplinary and offers several programs of study at the intersection of technology studies, social sciences, and humanity-based media communication studies. The particular program of study where the introductory HCI course (that is subject of this case study) is offered bridges the gaps between computer sciences and social sciences. Students in this program take several foundational courses related to the general principles of design, media and interface design, video and graphic production, but also upper-year courses that introduce them to advanced topics in user experience design. The course we focus on here is a third-year course with an attendance of 80 students split in two sections. Students take this course after completing several more practical oriented courses in their second year (aimed at giving them the skills to handle design production and media creation). The course is the first to introduce students to the more rigorous study of HCI, and follows a curriculum common to such courses (Munteanu and St-Cyr, 2018). The curriculum covers topics such as understanding users (conducting user observations), developing user requirements grounded in observational studies, low fidelity and wireframe sketching, paper prototyping, early usability testing with paper prototypes, iterative prototype development and testing.

Reflections in pedagogy

We reflected on the core concepts behind the DDM framework, such as, among others, the notion of design approaches that are not obviously exclusionary but may lead to certain user groups being “pushed away” from the design that is created. We then inspected our own course materials and pedagogical artifacts that we employ in this course, and analyzed them from the perspective of the DDM framework. Among the several usability methods we teach in this course, one immediately stood out in this regard: the use of the think-aloud protocol for low-fidelity (paper prototype) usability testing, which is a key method we teach. We were somewhat familiar with prior concerns about think aloud potentially being an ableist usability method (Chandrashekar et al., 2006; Waugh, 2019). However, when we revisited, under the lens of DDM, how we teach this method, it became clear that the consequences of using such a method are not confined to inclusion/exclusion of students in the class, but to how students may use this method in their future design activities post-graduation, and the implications this has on those designs leading to marginalization.

When we teach the think aloud protocol, the limitations we convey to the students are about mechanical and procedural issues, such as users being too absorbed in the interaction to comprehensively verbalize their thoughts and actions. Thus, the limitations we discuss are intrinsic to the workings of this method. However, when we reflected on this through the DDM lens, it became clear that we never discuss the issues of think-aloud in relation to inclusivity. For example, we do not consider how users with visual or speech impairments may engage with this method (which were documented in literature such as Chandrashekar et al. (2006); Waugh (2019). Additionally, this has prompted us to consider other situation where the use of think aloud could be problematic–for example, older adults may face increased challenges with the think aloud protocol due to the cognitively taxing nature of the task (Neves et al., 2015). Our reflection with the DDM has helped us consider whether we are teaching students a design technique that may lead to (future) designers unwittingly ignoring specific groups of users and thus creating designs that may “push away” such users.

This reflection helped us realize that the uncritical use of methods such as think aloud goes against some of the other principles we teach in this class. For example, we teach students that, as a designer, “you are not the user” as a way to help them focus on users, instead of interpreting users' needs through the designer's own biases. However, for practical and pedagogical reasons, students often conduct usability testing in class, with their colleagues participating in usability testing sessions (we elaborate on this in the next subsection). Using DDM as a self-assessment or critique of our own teaching materials helped us realize that, in the end, these contribute to further entrenching exclusionary practices that students may later on apply to their post-graduation design professional practice. It has also helped us see that we may not fully subscribe to our own principle of “you are not the user”. A similar reflection became apparent for using personas as a usability method–which is discussed in a further case study later in this paper.

Case study #2: Course-based community partner relationships

Context

This case study is about a first-year masters level design course of about 60 students studying user experience design in a professional program. In this course, students are asked to work in groups and partner with community organizations to redesign the organizations' websites. Students conduct multiple phases of user research (through e.g., requirements gathering, storyboard validation, card sorts, tree sorts), culminating in a mid- to high-fidelity prototype website designs that are graded primarily on their information architecture (the main topic of the course).

Reflections in pedagogy

Reflecting upon the course through the lens of the DDM revealed some examples where broader social consequences may come into play due to issues of access and power. These are demonstrated through the projects of two student groups within the course.

One group partnered with a community organization that worked to provide shelters for the homeless. The organization's website was aimed at multiple audiences, including potential volunteers, potential donors, current volunteers, and the homeless. Students needed to prepare a website design that was usable for and could serve the needs of all of these user groups. An effective design relied on students' success in identifying, interviewing, and engaging with members of the multiple stakeholder groups. However, access to any users apart from current volunteers proved to be, unsurprisingly, difficult. Firstly, a strong relationship with the organization needed to be in place for the staff to be willing to connect students with the homeless that use the website. This was challenging to do given the short timeframe (3 months) of the course. Secondly, those who worked at the organization themselves were not always there long-term, and thus often lacked the long-term, trusting relationships with the homeless people in question.

In lieu of not being able to access the stakeholder type in question, students were allowed to find a stand-in of someone to be similar. In the case of the volunteers, for instance, the students often simply found someone in their own social networks (often family or friends) to interview. This reinforces existing stereotypes or biases and often led to students designing for personas that spoke to generalizations or stereotypes. While tolerable in a classroom context, this practice can both perpetuate stereotypes and often miss the point of partnering with organizations serving vulnerable populations; that is, to provide a voice to those who don't have one (in the spirit of “don't write about us without us”). The fact that we take shortcuts in a time-bound college course context is unsurprising; however, one might imagine that it begins the creation of “bad habits” that would perpetuate as these students go on to work in the industry.

A second instance involved a group designing a website for an indigenous organization focused on creating a “historical record” of indigenous experiences in Canada. This organization was not well-funded, and completely volunteer run (while noting that these volunteers have day jobs elsewhere). In contrast, our educational institution is a historically “white” institution, and the students partnering with the organization formed a group of 5 to 6 people, of which was large but not unusual in a college context). Even at the time, it occurred to us that the organization may become uncomfortable engaging with and being interviewed by that many students at once. Dynamics could have been at play in terms of the numbers (lots-to-one) and power (historically powerful-to-historically powerless). The dynamics of the relationship risks social harm or perpetuating social inequalities if not handled carefully. Even when such dynamics were adequately considered and managed, there was also the issue of building enough trust to be able to effectively engage with the stakeholders themselves. Furthermore, due to student schedules, some of them might not have been able to attend the interviews themselves and had to rely on second-hand recounts or transcripts from their groupmates. This risked them not fully understanding the nuances required to be considerate designers for their project. Finally, it was important to recognize the issue of historical exploitation that is being perpetuated with this community project approach in this course. Namely, the students themselves are working with the community partners in order to receive a grade. This is, in many ways, an inappropriate pairing with this type of community partner–even if the students, deep down, wish to work with this partner for all of the right reasons. So, where does this leave community partners like this?

In short, “community partners” for this course is a great idea for the students. It means the students receive the chance to work with someone outside of their own immediate circle, learn some domain knowledge in a space outside of their norm, and work with “real” organizations. Yet, upon reflection through the lens of the DDM, it becomes clear that this community-partner approach primarily only works for “conventional, non-marginalized community partners.” As we saw with the two examples, working with marginalized groups demands time to build trust, and a careful approach that is fundamentally non-exploitative.

While the students' intentions are well-intended, the structure of the course (i.e., 3 months) means that there is insufficient time to build rapport with some community partner organizations. In practice, this suggests that “conventional non-marginalized community partners” benefit the most from such partnerships–firstly because they are easier to access, and secondly because they are easier to work with within the constraints of a term of work. As a result, digital marginalization may be at play in the context of the course, and students may become accustomed to taking ethical shortcuts in future design practice. This reflection raises the need for increased guidance on the part of the instructor or equity experts to better equip future students with the skills to partner with community organizations serving vulnerable populations sensitively, effectively, and responsibly.

Case study #3: Civic engagement–personas and empathy

Context

This case study is grounded in a course that is part of a program focused on digital design for students majoring in management at a large Canadian university. This course is the first in a series and is offered at the second-year level. It focuses on teaching the fundamentals of user-centered design, including user research techniques such as interviewing and contextual inquiry, and the development of various design deliverables such as personas and paper prototypes. This is scaffolded by the production of a term project intended to help local citizens to become more engaged with their community and make informed choices in a meaningful part of their life.

Ongoing experience with term projects shows that students struggle to engage with specific audiences. An audit of the Fall 2019 projects showed that about two-thirds of the 25 projects aimed at a general public audience, with typical topics such as “climate change awareness” or “food waste”. Of those who chose a narrower audience, typical topics included tips for young drivers, student loan advice, and volunteerism for high school students in need of mandatory hours. In another term, a persona for a website about menstrual products for low-income women (a strong topic choice) nonetheless included a 29-year-old teacher, who, as an “older user is not a heavy technology user.” Taken as a whole, these demonstrate that there are standing issues with how we teach students to identify and empathize with a potential audience outside themselves (Bennett and Rosner, 2019).

Reflections in pedagogy

While user-centered design is intended to direct its practitioners toward an understanding of the humanity and fundamental needs of users, various critiques have also been made of the struggles with connecting to identified groups that are harder to work with such as children or older folks with dementia (Marti and Bannon, 2009). This struggle for practitioners is no doubt amplified for students without resources to connect with “actual users.” However, the trend to not even engage with thinking about narrower audiences reveals something deeper in the training process.

Students are introduced to conducting background research on their potential audience to learn about them. Examples of data sources such as national census data and industry white papers are covered. However, in courses and particularly during the pandemic years, empirical research with users has been limited. This helps to further explain the trend in topics to very familiar ones and the empathy gap on display for those dissimilar to themselves.

Reflecting on these outcomes through the lens of the Digital Design Marginalization framework has helped to describe the potential outcome of this issue more concretely. Even within the chosen topics as described above, specific and more interesting audiences were overlooked. While many projects looked at the facts and figures around climate change, individuals who would be directly affected were not identified or considered. Whether these would be low-income individuals or climate refugees coming to the region, the resulting designs were generic and failed to consider actual needs. This failure to imagine actual social actors affected by the chosen topic pushes away from marginalized humans and precludes meaningful interventions.

Case study #4: Defining the user: The danger of generalizations in design

Context

This case study considers a design course taught at the undergraduate level in a large Canadian university with a focus on the use of open data to address a design challenge. In the course, students are introduced to data portals made available by different levels of government and provincial representatives are guests of the class, meeting with students to discuss government mandates regarding the collection and sharing of public data as well as tips on how to best navigate the provincial data catalog. Students are asked to submit a brief report on some of data sets, where they consider the meta data and potential uses of the data in a design artifact.

In groups, students are tasked with making use of open data to address a broad design question: “how could open data be used to improve citizen engagement?” A series of scaffolded assignments take students through various ideation exercises, including brainstorming, mind mapping, story boarding, user personas and eventually prototyping. Government representatives return to the classroom in order to discuss the student proposals; almost all of which are digital products (either mobile or browser-based applications). Students then develop wireframe prototypes to better communicate their concept and produce an academic poster as their final deliverable. This poster articulates the goals of the applications, which open data sets would be utilized, and the target audience.

Reflections in pedagogy

When considered through the lens of Digital Design Marginalization, several issues emerge. Perhaps most importantly is that the “user” is conceptualized in broad terms which serves to mask or hide potential marginalization and that there is a lack of awareness from the earliest stages that the needs of various users would not be addressed. In the early stages of design, where students engage in defining a potential user, they tend toward imprecise or “fuzzy” terms. For example, it is not uncommon for the user to be characterized as “a student”, “between 18 and 29 years old”, “someone who likes the outdoors” or “someone seeking mental health related advice.” Such broad definitions serve to expand the potential of the design, which is not yet limited by specificity. However, the seeds of marginalization are present. Broad definitions, such as those listed above, carry assumptions regarding access to digital artifacts and the ability to interact with those artifacts.

In other words, a fictional user persona is a useful tool for articulating user motivations and behaviors, as well as serving as a communication conduit between members of the development team. Further, user personas serve to define a user outside of the development team. Unfortunately, a tool such as a user persona is too broad to capture the nuances at play in any design. A well-constructed user persona certainly appears “real”, but because they must represent a large user group, they have difficulty capturing specificity of all group members. In particular, accessibility issues are not considered in user personas, even where the design artifact is intended to address specific issues. This problem appears to be baked into the design technique itself; a user persona is not an exhaustive list of issues and concerns, but rather stands as an aggregation. It may contain multitudes, but it can't articular them all. Viewed through the lens of DDM, this is problematic. Defining the potential users of our design is a critical first step. But when such design building blocks miss the nuances that characterize each of us, there is a risk that the resulting artifact will lead to marginalization. Addressing such issues in the late stages of design can be difficult.

Furthermore, for undergraduate students, many of whom are exposed for the first time to design techniques, the issue of design marginalization exists at the beginning of their design education. But what about senior undergraduate students? By the fourth year, students are close to graduation and hopefully ready to start careers in design. Are they aware of digital design marginalization in their work? A brief consideration of the HCI curricula suggests that the answer is no. Design techniques taught in such a class and used by industry practitioners, tend toward generalization. While understandable, it leaves little room for potential users who fall outside of broad characterizations.

It is not enough for instructors to explicitly discuss accessibility issues and respect for various interests. For a variety of reasons, the final deliverable leaves little room for a careful consideration of who might be excluded based on the design. This appears to be a result of both the design techniques used and perhaps a result of the speed with which a design course unfolds. Given the requirements of a course totaling 24 h (over 12 weeks), the scaffolding process of design can leave inadequate room for reflection. This is an issue in the design profession and one that is mirrored in the course. Since students spend time exploring the data they must incorporate into their work, as well as time spent understanding the application of various design techniques, they may find themselves with inadequate time to fully consider the implications of their work. In a race to the finish line, concerns for the marginalized are jettisoned or likely not even considered. This is problematic as the course might represent the last opportunity to raise awareness of the implications of design work. Students will likely graduate with deficiencies and inadequate training necessary to consider the societal implications of their work.

Case study #5: Time is of the essence

Context

This case study is about a first-year master's level foundational Human-Computer Interaction (HCI) course. Upper-level undergraduate students have an option to enroll in this course as well. The class size for this course is typically between 25–30 students. The course aims to provide students with the skills to critically evaluate designs for accessibility and usability while understanding the typical underlying human processes related to memory and senses that explain why people have preferences for optimal system design. The students are taught key HCI concepts and assessed through a combination of individual and group assignments. User-centered design has a significant presence within the course, and the students learn to use various design, qualitative, and quantitative methodologies to elicit user requirements, distill them down into manageable information that can be shared with different stakeholders, and deliver low and high-fidelity prototype solutions.

Reflections in pedagogy

Although the course discusses the wider social consequences of poor design, it does not do so to the extent that would be acceptable through the lens of DDM. The course does emphasize the importance of accessibility and usability. In addition, we cover a discussion of guidelines, laws, and the consequences of poor design. However, there is a lack of discussion on the implications of exclusion beyond the argument that it is unfair not to be inclusive. DDM discusses a different perspective of “pushing away” instead of using the exclusion framing. The acknowledgment of “pushing away” would be an essential addition to reframing parts of the conversations with students on the course so that more nuance comes through concerning poor design outcomes.

Another aspect of the course that might be falling short is the completeness of stakeholder needs. We teach user profiles and personas, and the students should be identifying clear goals and motivations. Still, those are only as accurate as the data collected by the students in their ethnographic observations and contextual inquiry. Through the lens of DDM, it would be necessary to consider much more of the broader social consequences of design. There is an opportunity to encourage the students to include more user concerns related to the wider context of technology use. It is worth noting that the course does stress iteration within the design process, especially when evaluations highlight weak points that will need further refinement, but this does require time. The first case study described by Sin et al. (2021) observed that more time results in uncovering less obvious design issues. However, the luxury of time is challenging in an educational setting.

Design training happens best in practice—one can learn a lot through trial and error—and a possible limitation of the course (or any university course) is limited exposure to real users and time to engage with the process. In this class, we often emphasize to our students that the purpose of assignments in the course is to give them experience with each step of the design process, to understand what goes on from beginning to end. However, there is no expectation for the students to make ground-breaking discoveries. As welcome as this may be, it would be unfair to students if we were not considerate of the constraints they are working within, especially during COVID lockdown conditions. Students are often gaining experience on a small scale without many opportunities to follow up with their users. Under these conditions, the students will inevitably miss out on learning the limitations of their design work. Furthermore, there certainly is pressure falling on the person teaching the course to catch design issues contributing to potential digital marginalization. While anybody teaching a course will have experience and knowledge to share, the teacher will have their own biases and will not be an expert in all domains, limiting how many issues are caught when giving feedback.

Finally, even though this is a foundational course and there is an expectation the students will further refine their skills through more focused HCI courses in the degree program, there is undoubtedly a missed opportunity to start the conversation early about important issues raised by the DDM framework. A high percentage of our masters students are set on transitioning to an industry career after their degree, and we should be supporting them to become more socially conscious designers.

Case study #6: Sprint break: Finding space for speculative DDM evaluation in an intensive UXD fundamentals course

Context

In this case study, we consider an introductory first-year course in the user experience design stream for a two-year professional Masters program. The course is required for this stream, has an enrollment of about 150 students, and is the foundational introduction to design thinking for our students. It consists of an accelerated 4-week lecture portion followed by an 8-week design studio, in which groups of 4–6 students complete one cycle of research, ideation, prototyping, and evaluation of a digital product targeting students.

In order to seed student portfolios with a polished, presentable case study before they apply to internships, the course is intensely focused on industry design thinking methodologies and on the practicalities of getting the project completed: rigid templates, guided workshops, lean evaluation, and a short deliverables cycle ensure that students thoroughly practice the mechanics of the design thinking process. However, this intensive practical approach leaves almost no time for a holistic introduction to the design discipline and its role in shaping society. Important topics such as research ethics, accessibility, and evaluating marginalization are barely mentioned, and never practiced.

Project topics are constrained to the domain of improving university students' lives. This grants our students access to background research from our institution's student life organization, yields real, relatable problems to solve, and ensures a viable supply of research participants during the background research sprint. However, it also makes it easier for students to make assumptions about their target users by tacitly assuming them to be similar to themselves.

Reflections in pedagogy

Students typically identify important aspects of the student experience to improve, such as personal finance, health and fitness, social connectivity, and mental health. However, while they may be implicitly aware of marginalization and exclusions that already exist in these spaces, they are not guided to consider them in understanding their problem spaces.

Student projects are evaluated on their adherence to the process templates, producing legible deliverables, storytelling, and usability, but not on potential long-term impact, exclusion, or harm. Therefore, students are incentivized to present the successful, happy-path vision of their work, naturally gravitating toward safe, mainstream, undramatically helpful solutions: centralizing information on campus events, time management apps, mentorship matching services, etc. Because their target populations are not constrained and not examined for existing exclusions, their solutions are implicitly aimed at the median users our students imagine, while anyone else is invisibly excluded. Based on our grading schemes, students are trained to focus their self-evaluation on getting the templates right and convincing stakeholders of the benefits of their design. Upon successfully completing this project, students gain a sense of confidence and believe they understand how to perform the design process.

However, students have failed to grapple with three key issues. Firstly, they conclude that understanding a target population well enough to design for it only takes a week or so of lean research, whereas that is only the case because their problem space has been artificially constrained to be similar to their own experience. Secondly, having been taught design thinking as a series of replicable templates, they believe that following them will create a serviceable, usable product. When their process templates emphasize usability and appeal as the design's most important evaluation metrics, they tacitly omit considerations of access, equity, and marginalization. Finally, student projects often insert digital interfaces into previously non-digital, or less digital contexts, e.g., mentorship, networking, clubs and campus events, finding study space, using athletics facilities, etc., but they fail to consider the disparity in experiences for users that may be excluded from their new, digital solution.

In all, while students know in the abstract not to exclude, harm, or marginalize, the skills they are practicing do not challenge them to notice or counter these outcomes.

Case study #7: Pre-COVID hybrid design course with hearing and deaf or hard of hearing students

Context

This course was a graduate level user-centered design course, comprised mostly of Master of Science students, and some PhD students. It is an elective course in our program and provides more depth and context than the core course material that touches on similar topics (brainstorming, prototyping, etc.), typically taken in the second year of the Master's program. Approximately 26 students were enrolled in the course. In this pre-COVID hybrid in-person/online user-centered design course comprised of both hearing and deaf and hard of hearing students, students worked in groups on a semester-long design project, which culminated in a prototype and design specification. Project groups comprised of a mix of in-person and online and deaf or hard of hearing (DHH) and hearing students to increase engagement for online students and to encourage equitable communication practices, for example so that as much as possible all groups used Slack to communicate in the absence of interpreters. Through the course, students ideated, sketched, created paper prototypes, and finally user tested high-fidelity prototypes. They conducted a brief needs assessment interview and several feedback sessions, meeting with in-person users four times.

Reflections in pedagogy

Typically, such design courses introduce a design concept followed by a hands-on activity. All concepts and activities build on each other and culminate in a final prototype and specification. There is usually some effort to engage real-world users, either by requiring students to go out and find users, or by bringing users into the classroom.

Upon applying the DDM lens, we reflect on finding users for the “user-centered” portion of the design cycle: (1) if students are required to find users on their own, there may be issues as they are often likely to find friends, roommates, etc., people who are socially adjacent to them/their lives. This approach introduces the possibility of bias; students may never know how “strangers” might react to their designs. (2) If instructors connect students with users, again, bias can be introduced–as it may depend on user availability to meet with students (i.e., during class?), if compensation is involved, or if users are socially adjacent to the instructor (although the instructor could account for this a bit better than the students).

For the class we taught, there was bias toward finding users who were DHH (to counteract typical cases of not including DHH users at all)–however, in doing so we excluded the hearing user perspective (but, not the hearing designer perspective) as it relates to how behavior and interactions of hearing individuals influence technology use for DHH individuals. There is perhaps a missed opportunity to differentiate and better understand these roles. There is an opportunity to be intentional about who is included as users. Additionally, as a DDM reflection, our current focus of the course still only provides one view of disability and accessibility; specifically, we leave out other kinds of disabilities and so students may not realize how these perspectives can/should extend to accessibility and disability broadly. To address this, concerted effort should be made to contextualize accessibility as not just for a specific user group. If we teach this course again, we might be more intentional in discussing user bias, why it matters, and ways to mitigate it.

Case studies summary

This section presented reflections through the lens of the DDM on a variety of HCI courses. These courses differ on a number of dimensions, such as the education level of undergraduate (case studies 1, 3, and 4) and graduate levels (case studies 2, 5, 6, 7), whether the course is a core program requirement (case studies 1–6) or not (case study 7), and the expertise of the instructor with inclusive design in terms of expert (case studies 5 and 7), moderate (case studies 1, 3, and 4), and minimal expertise (case studies 2 and 6). The case studies also varied based on the core focus of the course such as introductory HCI (case studies 1 and 5), professional UX (case studies 2, 6, and 7), and civic engagement and open data (case studies 3 and 4). We aimed for this spread to illustrate the breadth of gaps that can be identified by applying the DDM framework in self-reflection, so as to demonstrate the relevance of this framework to not only instructors, but to program administrators and curricula coordinators as well. In the following section we provide an in-depth discussion of the pedagogical gaps identified by the DDM and the avenues through which DDM reflection can occur.

DDM in design pedagogy: A discussion

In the previous section, we presented seven case studies where we reflected on how our own design courses and curricula may be contributing to downstream—that is, once designers-in-training progress to industry practice–digital marginalization through design. These case studies demonstrate the application of the Digital Design Marginalization (DDM) framework to reflections on design pedagogy. Now, we build upon the case studies by discussing the types of design pedagogy gaps that can be revealed by the application of DDM to design pedagogy reflection. Then, we examine avenues for further operationalizing the reflection process through the DDM to benefit design pedagogy and the greater design community. Finally, we discuss future paths for continuing this work of reducing design-mediated social marginalization through design pedagogy through the DDM framework.

Design pedagogy gaps identified by the DDM framework

The examples presented in this paper cover a range of design courses from fundamentals to those more specifically emphasizing inclusive design practices. However, reflection through the DDM framework helped to uncover and articulate insights and possible interventions for the prevention of potential future design-mediated digital marginalization across all of the case studies. We present our case studies as examples that may start a longer-term discussion on how the DDM can be applied to reflect on and improve design education with downstream social consequences in mind. In aggregate, our reflections on our design courses demonstrate the suitability of the DDM framework in revealing at least three types of gaps in our pedagogy: how and what design methods and approaches are taught, how design courses are administered, and students' knowledge about design-implicated digital marginalization.

Gaps in how and what design methods and approaches are taught

Reflection through the DDM lens has demonstrated that the set of design techniques and approaches instructors add to students' “design toolkit” may themselves or in total encourage ableist methods of research and design, and lead to downstream social consequences. For instance, certain methods may not be wholly inclusive (e.g., the think-aloud protocol as illustrated in case study 1), or the ways in which the methods are applied in class may fall short of factoring in the social consequences of users of the design (e.g., shortcomings involving personas as raised in case studies 3, 4, and 5).

Gaps in how design courses are administered

The DDM framework has revealed potential flaws in instructors' administration of the course in terms of guidance in working with community partnerships and considering ethical issues in students' interactions with participants and users. Our reflection highlighted the impacts of the temporary nature of the course (e.g., as discussed in case study 2), limitations introduced by the duration of the course (e.g., as elucidated in several case studies including 2, 4, 5, and 6), and restricted access to target populations (e.g., as exemplified in several case studies including 2, 3, 5, and 6). Given these, students may require more support and guidance from the instructor to mindfully and adequately engage with community partners (e.g., as discussed in case study 2) and users (e.g., as discussed in case study 7), or critically evaluate their research and designs (e.g., as proposed in case studies 5 and 6).

Students' sensitivity and knowledge of design-implicated digital marginalization

Our reflections through the DDM framework highlighted that it is easy for students to graduate from their design programs without adequate tools, knowledge, or sensitivity to the implications of their work (e.g., as discussed in case studies 4 and 5). Moreover, in order to convey the weight of design choices, it may not be enough to only teach students about digital exclusion (as remarked upon in case study 5). The use of the DDM framework provided a language of marginalization that places names on the consequences of exclusion beyond only saying that users cannot use a design.

Avenues through which DDM reflection can occur

Digital Design Marginalization (DDM) captures the ways in which design choices and practices can result in digital marginalization, or in other words, social consequences for users. Sin et al. (2021) provided the theoretical framework of DDM and applied its lens to design. In turn, in this paper we demonstrate some benefits (as enumerated in the previous section) of applying the DDM lens to design pedagogy. Each case study is an illustration of a co-author's reflection through the DDM lens of their design course.

Although the main goal of this paper is not to provide the steps on how to practice reflection through the DDM lens, but rather to demonstrate the potential of the DDM to guide such reflection, these case studies demonstrate the suitability of the DDM lens as a discussion prompt for both solo reflection and group dialogue. This prompts the question of how might the process of reflecting on the social consequences of one's design pedagogy be operationalized? Grounded in the reflections and experiences described in this paper, we suggest three avenues for this: 1) through the educators themselves; 2) through program administrators or curricula coordinators across design courses; and 3) through institutional champions of equity and inclusion. We elaborate below on each of these avenues.

Reflection by instructors

On one hand, instructors (as we are ourselves) can use a tool such as the DDM framework to help guide reflection on the impact of their own design courses on downstream design-mediated digital marginalization. Many design course instructors pride themselves in applying on their own courses a practice that they teach within their own courses: iterative design. In other words, many design educators consciously and actively seek feedback from students and aim to improve the curriculum and delivery with each offering of the course. Reducing downstream digital marginalization through design is one additional dimension for which educators can consider when evaluating and adapting their courses.

Furthermore, for educators who are familiar with and teach concepts such as universal design, inclusive design, accessibility, and disability (e.g., the co-authors in charge of case studies 3, 5, and 7), the DDM framework can help articulate latent issues in curricula. The DDM framework can provide instructors with the language to articulate, for example in case 3, the consequences of generic designs that fail to consider actual needs. In turn, the DDM framework can help educators develop their soft skills in communicating with students about issues of inclusivity and marginalization and be more intentional about how users are included in class projects (e.g., as elaborated upon in case study 7). Lastly, the DDM framework can help educators reflect upon the existing HCI methods that they teach and reflect upon their potential problems of marginalization (such as in case study 1). This way, educators can question and re-evaluate how they teach and contextualize established HCI methods.

Reflection by program administrators and curricula coordinators

The DDM framework can be used as a tool by design program administrators and curricula coordinators (who often are also faculty members teaching the design courses, e.g., in case studies 1 and 3) to identify gaps in students' learning across the program with regards to design-mediated digital marginalization. These educators can use the DDM to consider, for example, whether or not students graduate the program with the training necessary to consider the societal implications of their work (e.g., as discussed in case studies 1 and 4), and to help address limitations posed by semester-long course structures to promoting and support student learning in these areas (e.g., as discussed in case studies 2 and 5). In our own reflections, we have found that students may not be graduating from their design programs with the tools, knowledge, or sensitivity necessary to understand and social implications of their work. In particular, course duration and the lack of time has been found in our reflections to be a key challenge. Although the lack of time should not be a limiting factor in trying to include inclusive design practices and pedagogy, our DDM reflections show that time constraints are systemic institutional issues that can lead to non-inclusive teaching. This can often be caused by program administrations' management of limited time resources.

Use by learning and academic support centers

Not only can the DDM framework be used by instructors and institutional divisions in charge of higher-level course structures, but it can be promoted as well by academic centers and services. Such divisions include but are not limited to institution-wide equity and accessibility offices, equity champions in teaching offices, and academic support centers. In such scenarios, members of these groups can provide instructors, program administrators, and curricula coordinators with materials to reflect on their educational materials through the DDM lens. Some of the case studies (1, 2, 3, and 6) also involved extended discussions over one to three 1-hour sessions between the lead author and the co-author in charge of the case study. The engagement and inspiration from such discussions may also transpire should the staff of these centers invite instructors and program instructors to meet with them.

Future considerations

Based on our experiences reflecting on our own course curricula and as educators in design ourselves, we suggest four paths to continue the work of reducing design-mediated social marginalization through design pedagogy: collecting case studies of design-based digital marginalization, creating guides to prompt educator reflections through the DDM of their courses, sharing these resources globally, and continuing the research started by this paper.

First, we suggest the creation of materials related to the topic of digital design-based marginalization from which educators may draw from for their course materials. Specifically, we suggest a public repository of case studies of digital marginalization through design and a collection of design evaluative questions informed by the DDM. Public common knowledge repositories could be organized to provide historic and recent examples of designs that have marginalized people, in similar style to the two case studies described by Sin et al. (2021). This could be helpful for facilitating discussion activities within a course where students are tasked with trying to identify problem scenarios with tech solutions. Evaluative questions could challenge students to try answer questions such as the following projects' designs [e.g., in the format of the Tarot Cards of Tech The Tarot Cards Of Tech, n.d.]. At the end of their project, we would ask students to reflect not only on the lessons they learned and the next potential steps in the project, but also on the limitations and exclusions of their work, and we would allocate grades to critical self-reflection.

These tools may serve as discussion activities within courses, where students could be tasked with identifying design-implicated social consequences within case studies and their own creations. Case studies can be discussed before group assignments to increase the likelihood that the students make more observant observation and ask deeper questions of their users to adhere to the DDM framework. Meanwhile, evaluative questions can prompt students to critically evaluate their designs and provide them with an additional criterion (i.e., the potential to marginalize users) by which to evaluate their projects.

Second, we recommend the creation of guides to facilitate educators' reflections and discussions of their courses through the lens of DDM. An example of a good starting point for this would be the Black Mirror Writers Room exercise (Klassen and Fiesler, 2022) which helps educators facilitate discussions on technology ethics. This can be adapted to DDM scenarios. These guides can sensitize design educators to the social implications of their pedagogy and encourage them to ask questions of themselves and each other about how they are teaching students design, as well as question how the educators themselves teach and contextualize established HCI methods. This process would empower educators to critically evaluate and improve their courses and pedagogical content in a socially conscious manner. Additionally, these guides can be used by champions of equity and inclusivity to promote design education for a more ethical and inclusive design future.

Third, we encourage the sharing of materials and resources created for the two recommendations above to be shared widely, through venues such as conferences, publications, and repositories of instructional materials (e.g., Call For Submissions | EngageCSEdu, n.d.). The global sharing of case studies and discussion points can not only promote more extensively the merits of reflecting on the social consequences of design pedagogy, but also invites global case studies of digital marginalization from which students may study design-implicated consequences.

Lastly, we suggest the continuation of the line of research exemplified in this paper. Specifically, this paper serves as a first step of seeing how instructors can reflect on their own teaching practice through the DDM framework. However, although educators can be reflective of the course and identify where it could be improved, students may have feedback given hands-on experience in the industry setting. Now that this paper has demonstrated how the DDM can help educators to expose gaps through self-reflection, we can follow up with a different study (involving a different methodological approach) to look at students who moved to industry. This future study can help round out the understanding of digital design marginalization in connection with design pedagogy.

Design choices and practices can contribute to digital exclusion, downgrading, and social consequences. Through frameworks like the DDM, one can identify places where design may be implicated in marginalizing people in society. These frameworks can also be applied to design pedagogy, and by purposefully critically evaluating the current ways in which design is taught to students, we are better equipped to intervene and contribute to a more equitable and inclusive design future.

Limitations

While the co-authors are based in different disciplines (computer science, information studies, communication and media) across different universities in two different countries, we still represent a western, North American perspective, and one that does not account for career colleges or other skills-oriented post-secondary schools outside of the university system. Finally, the insights presented in this paper are specific to curriculum introspection, and further methods (e.g., looking at on-the-job practices) will offer other insightful perspectives on how the DDM framework can be applied.

Conclusion

As it stands, it is easy for students to graduate from their design programs without the skills, knowledge, or sensitivity to address the social implications of their design practice. To minimize downstream social consequences of design practice, conscious attention must be paid on the implications of current design pedagogy. In this paper, we provided case studies of our reflections through the Digital Design Marginalization (DDM) framework on our own design courses. We present these reflections as case studies in order to discuss the types of shortcoming and opportunities that can be revealed through such self-reflection with respect to improvement in the training of design methods, the administration of design courses, and the course content related to issues of inclusivity. Based on our experiences with this reflection, we suggest incorporating the DDM as an additional reflection tool into the iterative improvements of design courses and into the agenda of existing champions of equity and inclusivity.

Data availability statement

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author/s.

Author contributions

JS contributed to the conception and design of the paper and wrote all of the sections except the individual case studies (Section Case study #1: Uncritically teaching potentially ableist usability testing methods onwards). CM, MN, VP, GT, KS, AT, and SS each contributed the text for one case study. All authors contributed to manuscript revision, read, and approved the submitted version.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Abreu, A. (2018). Why I won't “try on” disability to build empathy in the design process (and you should think twice…). Medium. Available online at: https://blog.prototypr.io/why-i-wont-try-on-disability-to-build-empathy-in-the-design-process-and-you-should-think-twice-7086ed6202aa (accessed November 24, 2021).

Battarbee, K., Suri, J. F., and Howard, S. G. (2015). Empathy on the Edge: Scaling and sustaining a Human-Centered Approach in the Evolving Practice of Design. Available online at: https://new-ideo-com.s3.amazonaws.com/assets/files/pdfs/news/Empathy_on_the_Edge.pdf (accessed November 24, 2021).

Bennett, C. L., and Rosner, D. K. (2019). “The promise of empathy: Design, disability, and knowing the “Other”,” in Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (New York, NY: Association for Computing Machinery), 1–13. doi: 10.1145/3290605.3300528

CrossRef Full Text | Google Scholar

Burnett, M., Peters, A., Hill, C., and Elarief, N. (2016a). “Finding gender-inclusiveness software issues with gendermag: a field investigation,” in Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, 2586–2598. doi: 10.1145/2858036.2858274

CrossRef Full Text | Google Scholar

Burnett, M., Stumpf, S., Macbeth, J., Makri, S., Beckwith, L., Kwan, I., et al. (2016b). GenderMag: a method for evaluating software's gender inclusiveness. Interact. Comput. 28, 760–787. doi: 10.1093/iwc/iwv046

CrossRef Full Text | Google Scholar

Byers, K. M., Elsayed-Ali, S., Jarjue, E., Kamikubo, R., Lee, K., Wood, R., et al. (2021). “Reflections on remote learning and teaching of inclusive design in HCI,” in 3rd Annual Symposium on HCI Education (EduCHI2021) 12.

Google Scholar

Call For Submissions | EngageCSEdu (n.d.). Available online at: https://engage-csedu.org/ (accessed November 23, 2021).,

Google Scholar

Chandrashekar, S., Stockman, T., Fels, D., and Benedyk, R. (2006). “Using think aloud protocol with blind users: A case for inclusive usability evaluation methods,” in Proceedings of the 8th International ACM SIGACCESS Conference on Computers and Accessibility, 251–252. doi: 10.1145/1168987.1169040

CrossRef Full Text | Google Scholar

Deszca, G., Munro, H., and Noori, H. (1999). Developing breakthrough products: challenges and options for market assessment. J. Oper. Manage. 17, 613–630. doi: 10.1016/S0272-6963(99)00017-0

CrossRef Full Text | Google Scholar

Dong, Y., Dong, H., and Yuan, S. (2018). “Empathy in design: A historical and cross-disciplinary perspective,” in Advances in Neuroergonomics and Cognitive Engineering, ed C. Baldwin. (Cham: Springer International Publishing) 295–304. doi: 10.1007/978-3-319-60642-2_28

CrossRef Full Text | Google Scholar

Doorley, S., Holcomb, S., Klebahn, P., Segovia, K., and Utley, J. (2018). Design Thinking Bootleg. Available online at: https://static1.squarespace.com/static/57c6b79629687fde090a0fdd/t/5b19b2f2aa4a99e99b26b6bb/1528410876119/dschool_bootleg_deck_2018_final_sm+%282%29.pdf (accessed November 24, 2021).

Google Scholar

Gray, C. M., Chivukula, S. S., Melkey, K., and Manocha, R. (2021). “Understanding “Dark” Design Roles in Computing Education,” in Proceedings of the 17th ACM Conference on International Computing Education Research (New York, NY: Association for Computing Machinery), 225–238. doi: 10.1145/3446871.3469754

CrossRef Full Text | Google Scholar

Gray, D. (2018). Updated Empathy Map Canvas. The XPLANE Collection. Available online at: https://medium.com/the-xplane-collection/updated-empathy-map-canvas-46df22df3c8a (accessed July 21, 2018).

Google Scholar

Grosz, B. J., Grant, D. G., Vredenburgh, K., Behrends, J., Hu, L., Simmons, A., et al. (2019). Embedded EthiCS: integrating ethics across CS education. Commun. ACM 62, 54–61. doi: 10.1145/3330794

CrossRef Full Text | Google Scholar

Hanson, V. L. (2007). “Inclusive thinking in computer science education,” in Proceedings of the 12th Annual SIGCSE Conference on Innovation and Technology in Computer Science Education, 3. doi: 10.1145/1268784.1268787

CrossRef Full Text | Google Scholar

Hayes, G. R. (2011). The relationship of action research to human-computer interaction. ACM Trans. Comput. Human Inter. 18, 15:1–15:20. doi: 10.1145/1993060.1993065

CrossRef Full Text | Google Scholar

Inclusive Design Research Centre (n.d.). Available online at: https://legacy.idrc.ocadu.ca/about-the-idrc/49-resources/online-resources/articles-and-papers/443-whatisinclusivedesign (accessed September 17 2020).

Google Scholar

Irani, L. C., and Silberman, M. S. (2016). “Stories we tell about labor: turkopticon and the trouble with “Design”.” in Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, 4573–4586. doi: 10.1145/2858036.2858592

CrossRef Full Text | Google Scholar

Klassen, S., and Fiesler, C. (2022). “ ‘Run wild a little with your imagination': Ethical speculation in computing education with black mirror,” in Proceedings of the 53rd ACM Technical Symposium on Computer Science Education. Vol. 1, 836–842. doi: 10.1145/3478431.3499308

CrossRef Full Text | Google Scholar

Koepfler, J. A., Stark, L., Dourish, P., Sengers, P., and Shilton, K. (2014). “Values and design in HCI education,” in CHI'14 Extended Abstracts on Human Factors in Computing Systems, 127–130. doi: 10.1145/2559206.2559231

CrossRef Full Text | Google Scholar

Larsen-Ledet, I., Bressa, N., and Vermeulen, J. (2019). “Reflections on teaching a mandatory hci course to computer science undergraduates,” in Proceedings of the 2019 EduCHI Symposium on HCI Teaching and Learning. 8.

Google Scholar

Lazem, S. (2021). HCI education of choice: On becoming critical and growing inclusivity. XRDS: Crossroads ACM Magaz. Student. 27, 46–49. doi: 10.1145/3456296

CrossRef Full Text | Google Scholar

Marti, P., and Bannon, L. J. (2009). Exploring user-centred design in practice: some caveats. Knowl. Technol. Policy. 22, 7–15. doi: 10.1007/s12130-009-9062-3

CrossRef Full Text | Google Scholar

Mattelmäki, T. (2006). Design probes. Aalto University. Available online at: https://aaltodoc.aalto.fi:443/handle/123456789/11829

Google Scholar

McKee, J. (2019). Domino's and the Web are Failing the Disabled. Wired. Available online at: https://www.wired.com/story/dominos-and-the-web-are-failing-the-disabled/ (accessed August 16, 2019).

Google Scholar

Munteanu, C., Molyneaux, H., Moncur, W., Romero, M., O'Donnell, S., and Vines, J. (2015). “Situational ethics: re-thinking approaches to formal ethics requirements for human-computer interaction,” in Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems - CHI'15. 105–114. doi: 10.1145/2702123.2702481

CrossRef Full Text | Google Scholar

Munteanu, C., and St-Cyr, O. (2018). Workshop on Renewing the HCI Curriculum: Bridging the Gap between Research, Industry Trends, and Curriculum Renewal within Human-Computer Interaction. Held in conjunction with the 2018 Graphic Interfaces Conferences – GI. Available online at: https://www.dgp.toronto.edu/hci-workshop/ (accessed November 24, 2021).

Google Scholar

Nario-Redmond, M. R., Dobromir, G., and Cobb, A. (2017). Crip for a day: The unintended negative consequences of disability simulations. Available online at: https://psycnet.apa.org/doiLanding?doi=10.1037%2Frep0000127 (accessed November 24, 2021).

PubMed Abstract | Google Scholar

Neves, B. B., Franz, R. L., Munteanu, C., Baecker, R., and Ngo, M. (2015). “ ‘My Hand Doesn't Listen to Me!': Adoption and Evaluation of a Communication Technology for the ‘Oldest Old',” in Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems - CHI'15, 1593–1602. doi: 10.1145/2702123.2702430

CrossRef Full Text | Google Scholar

Oleson, A., Mendez, C., Steine-Hanson, Z., Hilderbrand, C., Perdriau, C., Burnett, M., et al. (2018). “Pedagogical content knowledge for teaching inclusive design,” in Proceedings of the 2018 ACM Conference on International Computing Education Research, 69–77. doi: 10.1145/3230977.3230998

CrossRef Full Text | Google Scholar

Pillai, A. G., Baki Kocaballi, A., Wah Leong, T. A, Calvo, R., Parvin, N., et al. (2021). “Co-designing Resources for Ethics Education in HCI,” in Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems, 1–5. doi: 10.1145/3411763.3441349

CrossRef Full Text | Google Scholar

Putnam, C., Bradford, G., Rose, E. J., and Cheng, J. (2019). “Teaching accessibility: five challenges,” in Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. 6.

Google Scholar

Sin, J. L., Franz, R., Munteanu, C., and Barbosa Neves, B. (2021). “Digital design marginalization: new perspectives on designing inclusive interfaces,” in Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–11. doi: 10.1145/3411764.3445180

CrossRef Full Text | Google Scholar

Spitzberg, D., Shaw, K., Angevine, C., Wilkins, M., Strickland, M., Yamashiro, J., et al. (2020). “Principles at work: applying “design justice” in professionalized workplaces,” in CSCW'20. doi: 10.21428/93b2c832.e3a8d187

PubMed Abstract | CrossRef Full Text | Google Scholar

St-Cyr, O., MacDonald, C. M., Gray, C. M., Potter, L. E., Vasilchenko, A., Sin, J., et al. (2020). “EduCHI 2020: 2nd Annual Symposium on HCI Education,” in Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems, 1–8. doi: 10.1145/3334480.3375066

CrossRef Full Text | Google Scholar

The Tarot Cards Of Tech (n.d.). Available online at: http://tarotcardsoftech.artefactgroup.com/ (accessed November 7 2021).

Google Scholar

Waugh, D. (2019). Site unseen: website accessibility testing for academic libraries with visually-impaired users (Master's Thesis). University of North Carolina, Chapel Hill, NC, United States.

Google Scholar

Waycott, J., Wadley, G., Schutt, S., Stabolidis, A., and Lederman, R. (2015). “The Challenge of Technology Research in Sensitive Settings: Case Studies in “ensitive HCI”,” in Proceedings of the Annual Meeting of the Australian Special Interest Group for Computer Human Interaction. 240–249. doi: 10.1145/2838739.2838773

CrossRef Full Text | Google Scholar

Wick, J. (2019). Newsletter: Essential California: Why San Francisco banned cashless stores. Los Angeles Times. Available online at: https://www.latimes.com/newsletters/la-me-ln-essential-california-20190508-story.html

Google Scholar

Wright, P., and McCarthy, J. (2008). “Empathy and experience in HCI,” in Proceeding of the Twenty-Sixth Annual CHI Conference on Human Factors in Computing Systems - CHI'08. 637. doi: 10.1145/1357054.1357156

CrossRef Full Text | Google Scholar

Zitkus, E., Langdon, P., and Clarkson, P. J. (2013). Inclusive design advisor: understanding the design practice before developing inclusivity tools. J. Usability Stud. 8, 17. Available online at: http://uxpajournal.org/inclusive-design-advisor-understanding-the-design-practice-before-developing-inclusivity-tools/

Google Scholar