How to measure the accessibility maturity of organizations—A survey on accessibility maturity models for higher education

Maturity models are increasingly used to advance the processes of organizations, including Higher Education Institutions. In this paper, we review existing maturity models to analyze and optimize the accessibility of organizations. Therefore, we conducted a systematic literature research in the databases Web of Science, IEEE Xplore, BASE, ACM, and Google Scholar, resulting in 13 different maturity models. An additional web search on maturity models for accessibility found another 12 maturity models that we added to the results. Finally, we analyzed the 25 maturity models in more detail, specifically the indicators that each maturity model uses to measure accessibility. The most frequent indicators were “responsibility”, “competences & training”, and “monitoring”, with differences in the frequencies when separated by target group. Out of the 25 maturity models found, only 6 focused on Higher Education Institutions. None of the existing maturity models focuses on teaching and learning of accessibility explicitly.

1. Introduction

Article 24 of the UN Convention on the Rights of Persons with Disabilities, that came into force in 2008, demands access to education for people with impairments (United Nations, 2006). In the context of Higher Education, Middendorff et al. (2017) discovered that 11 % of German students report having a disability or chronic illness that has an aggravating effect on their studies. These disabilities include mobility, hearing, and vision impairments, chronic somatic illnesses, and mental illnesses. In a representative survey of more than 20,000 students with disabilities, two-thirds of students reported that their impairments had a major impact on their studies (Poskowsky et al., 2018). Hence, to support equal education opportunities for all, as required by law but also generally aimed for in an inclusive Higher Education environment, the accessibility of educational institutions needs to be improved.

In order to improve the processes of organizations in general, maturity models are tools that have become increasingly prominent over the last years. This development has also found its way into higher education institutions and includes bottom-up or top-down mechanisms (Hähne et al., 2020). A maturity model comprises a series of levels (from low maturity at one end to high maturity on the other). It describes a typical or desired development path for a specific object (Becker et al., 2009b). Maturity models can be used to analyze the current situation as well as identify potential for improvement (Becker et al., 2009b). Some maturity models also support comparisons between organizations.

The aim of this paper is to identify existing maturity models that analyze the accessibility especially of higher education institutions. While accessibility has been defined in many ways, in the following, we refer to a recent definition by ISO 9241-112:2017(E): “Extent to which products, systems, services, environments and facilities can be used by people from a population with the widest range of user needs, characteristics and capabilities to achieve identified goals in identified contexts of use. Note 1 to entry: Context of use includes direct use or use supported by assistive technologies”. (International Organization for Standardization, 2017). One of the first process description in the educational context is the index of inclusion (Booth and Ainscow, 2002).

To identify the existing accessibility maturity models, we formulate the following research questions:

• RQ1: What accessibility maturity models for higher education institutions exist?

• RQ2: How do accessibility maturity models differ depending on their target areas?

• RQ3: How do the maturity models measure accessibility maturity?

To answer these research questions, we conducted a systematic literature research in the databases Web of Science, IEEE XPlore, BASE, ACM, and Google Scholar to identify all maturity models that analyze the accessibility of organizations. The maturity models we found were then analyzed in more detail with regard to their target area, their structure, the process used for developing the models (the empirical foundation), and their evaluation process and results. To investigate how accessibility is measured by the maturity models, we categorized and analyzed the indicators that they use for measurement.

This paper is organized as follows. First, we describe our research methodology and the sources used for the study. We then present an overview of the results of the literature review and describe all 25 existing maturity models that we found which measure the accessibility maturity of organizations. Following the descriptions, we compare the models with one another, and provide an overview (presented as matrix) of the indicators that we determined were being used in the models. We follow up with a discussion of the study results and conclude with final remarks.

This paper aims to contribute to transparency and provide a resource for readers who are making decisions about which maturity model is most suitable for a particular organization.

2. Methods and materials

We performed a systematic literature review to identify maturity models for accessibility. The search took place in March and April 2022 in the Web of Science, IEEE Xplore, BASE, ACM, and Google Scholar databases. For the research the following two search strings were used, including terms in English and German:

• Search string 1: (Reifegradmodell OR “Reifegrad-Modell” OR “maturity model” OR “maturity grid”) AND (Inklusion OR Inclusion OR Accessibility OR Barrierefreiheit OR barrierefrei) AND (Universität OR university OR Hochschule OR “Higher Education” OR “Further Education”)

• Search string 2: (Reifegradmodell OR “Reifegrad-Modell” OR “maturity model” OR “maturity grid”) AND (Inklusion OR Inclusion OR Accessibility OR Barrierefreiheit OR barrierefrei)

The second string omits “Higher Education” and synonyms, so the results of the first search string were also included in the second search plus additional results that do not include Higher Education. Since Google Scholar found many results for search string 2 that were not relevant (more than 5 Mio in total), we only used search string 1 here and additionally made “university” and synonyms obligatory in the title to reduce the number of results. In BASE, different results were obtained when changing the order of arguments between “AND”, so for this search we first used search string 1 as shown above as well as both search strings 1 and 2, changing the order to have “Accessibility” and the synonyms in front of the string.

After performing the search, all duplicates were removed from the list of sources. In the subsequent screening of the titles and abstracts of the remaining sources, we assessed whether the paper contained a maturity model, whether the model was used for Higher Education or for other organizations (target areas), and whether it measured accessibility or anything else. Afterwards, we read the relevant papers in their entirety to extract the mentioned maturity models. Some papers described multiple maturity models, that did not all address accessibility. In that case only the relevant models were extracted and analyzed.

An additional search for maturity models for accessibility using Google with the search string “maturity model accessibility” revealed that there are also models for accessibility that are not mentioned in any scientific publication. In these search results, all sources describing a maturity model that assesses accessibility (independent of the target area) were collected and added to the results to provide a comprehensive analysis of the existing maturity models.

Additionally, we researched the identified models on the web to detect any additional information and sources. Finally, we analyzed the maturity models in more detail, specifically on the indicators that each uses to measure accessibility. This allowed us to look at their usage and showed the frequencies of the used indicators.

3. Literature review—overview

Figure 1 shows the steps that we conducted for the research as well as the number of papers found in each step. In the literature research we found a total of 1,883 sources. The differentiation by database is also shown in Figure 1. The removal of duplicates resulted in 1,506 papers, which were then categorized by screening title and abstract as described in the method. 519 of all papers included a maturity model, 234 of them targeting higher education institutions and 285 papers targeting other areas, like software organizations or the public sector. Only 12 out of the 519 papers that included at least one maturity model described an accessibility model, including four papers having higher education institutions as their target area. Subjects analyzed by the other models were, for example, Digital Transformation, e-Learning, e-Governance, Software Processes, ICT, Knowledge Management or Risk Management. From these 12 papers we extracted a total of 13 maturity models for accessibility. The additional search on the web yielded another 12 maturity models, that amounted to 25 different maturity models for accessibility in total.

FIGURE 1

Figure 1. Steps of the literature review and web search for accessibility maturity models with the number results.

4. Maturity models for accessibility

Table 1 summarizes the 13 maturity models found in the literature research. The table first names the extracted model, followed by the abbreviation used in Figure 1, the source(s) from the literature research, and the primary source. The 12 additional maturity models found by a web search are presented in Table 2.

TABLE 1

Table 1. Overview of the existing maturity models for accessibility from the literature research.

TABLE 2

Table 2. Additional maturity models for accessibility from a web search.

Figure 2 shows a historical overview of the maturity models published, to illustrate their connections and dependencies. One of the first maturity models, on which many subsequent models are based (and also some of the accessibility maturity models as Figure 2 shows), is the Capability Maturity Model Integration (CMMI) (Tocto-Cano et al., 2020). The CMMI was developed by the Software Engineering Institute to improve processes in organizations (Software Engineering Institute, 2009) on the basis of its predecessor, the Capability Maturity Model for Software, that was first published in 1991 (Weber et al., 1991). CMMI was first published in 2000 and specified for different areas in software, hardware and service development (e.g., development or acquisition) since then (Software Engineering Institute, 2009).

FIGURE 2

Figure 2. Historical overview of the existing maturity models found in the study. Arrows indicate that a maturity model was derived from a previous one, light-gray boxes show predecessor that are not meant for accessibility.

In the following subsections, all maturity models are briefly described with reference to the following characteristics: their target area, their structure, the process used for developing the maturity models (empirical foundation), and their evaluation process and results. The empirical foundation is assessed following Ahlemann et al. (2005), who identify empirical foundation as a quality indicator for maturity models and distinguish three levels of empirical foundation: 1. no (documented) empirical foundation, 2. case-based empirical foundation (based on individual cases), and 3. empirical groundwork (wide-scale empirical analysis with many subject matter experts or a broad analysis of empirical cases). In the following, the maturity models are categorized by their target areas in higher education institutions, organizations for digital products, public sector organizations, and all organizations (undefined). Later in this paper, we will also provide a comparison between the described models and their indicators.

4.1. Accessibility maturity models for higher education institutions

We found 6 maturity models, that were explicitly developed to analyze the accessibility maturity of higher education institutions. Two of these models were developed by universities on their own and meant to be used in these universities only. The others are developed in a more general way.

4.1.1. Inclusive Excellence Change Model (Scorecard) (IECM)

The IECM is the oldest model we found. It focuses on the topic inclusion and diversity, and was published in 2005 by Williams et al. (2005). The focus of IECM is not accessibility but rather cultural diversity (Williams et al., 2005). It is not structured as a typical maturity model as it has no stages but describes five aspects that influence inclusion. Hence, it can be construed as a framework that needs to be filled with specific goals, objectives, strategies, and measures when used. It was developed especially for educational institutions to self-assess their focus on inclusion, using for example, questionnaires, group discussions or data collection to get the information needed (Williams et al., 2005). We did not find any information on an empirical foundation of IECM, but it is based on many other maturity models as well as a literature review (Williams et al., 2005). Moreover, there is no evaluation documented.

4.1.2. Higher Education and Further Education Accessibility Maturity Model (HE and FE AMM)

The HE & FE AMM was published in 2006 by AbilityNet and developed by AbilityNet and McNaught Consultancy (Byrne-Haber et al., 2022). It was designed especially for Higher Education and Further Education Institutions and is structured as a matrix with five stages and nine dimensions (AbilityNet and McNaught, 2020). The stages are named Luck, Tokenism, Standards, Ownership, and Partnership (AbilityNet and McNaught, 2020). The data gathered in the HE & FE AMM are all qualitative, covering bottom -up and top-down processes. The model was developed on the basis of observations and conversations with leaders in Higher Education (McNaught, 2021) and we found no information about how it is evaluated. In addition to the published matrix, there is an online accessibility self-evaluation service (OASES) that helps higher education institutions to assess their accessibility maturity based on the HE & FE AMM (Ball et al., 2010).

4.1.3. ISOLearn Quality Label (ISOLearn QL)

The ISOLearn QL was developed by the ISOLEARN project consortium coordinated by the Aberta Universidade and published in 2016 (ISOLearn, 2016). It is a maturity model explicitly for higher education institutions and is structured in five stages and five dimensions (ISOLearn, 2016). To assess the maturity model, a questionnaire can be used (ISOLearn, 2016). The ISOLearn QL is based on empirical groundwork, with a project consortium of 8 different associations and universities providing an accurate trans-national analysis with a literature review, focus group data, and feedback loops (Caforio, 2018). The model has also been evaluated in several universities; however, the evaluation results were not published.

4.1.4. Smart Campus for All (SCampus4A)

The SCampus4A has not yet been published. However, data from a focus group that was charged with the model development in 2019 was published by the same organization as SCity4A (Evans and Thurston, 2019). This maturity model targets higher education institutions explicitly. The development of the model is still in progress; so far the version of the published results of the focus group consists of a matrix with six stages and six dimensions that is not yet filled with content (Evans and Thurston, 2019).

4.1.5. Capability Maturity Model (CMM)

The CMM was published without a date and is used and published by the California State University Accessibility Technology Initiative (CSU ATI, n.d.). There, it is used for the development and implementation of a strategy for increased accessibility. It combines a roadmap and a matrix of 6 levels and 3 dimensions, each including goals and success indicators (CSU ATI, n.d.). “The success indicators articulate the business processes that need to be implemented in support of the goals. Progress toward the goals and success indicators is measured by a set of status levels. Status levels are described by a set of criteria that must be met in order to move from one status level to the next” (CSU ATI, n.d.). For the application of CMM, documents are used and analyzed in a committee yearly (planning, making, and reporting). The empirical foundation is case-based with the California State University, as is the evaluation (CSU ATI, n.d.).

4.1.6. Maturity Model University of Arizona (MM-UniA)

The MM-UniA was developed on basis of the AMM and is used and published (no date known) by the University of Arizona (University of Arizona, n.d.). Therefore, we suggested that it is specialized for higher education institutions, most likely only for the University of Arizona explicitly, so it has a case-based empirical foundation. Like the AMM, it is structured as a matrix with five levels accordingly and includes eight dimensions (University of Arizona, n.d.). It is one of only two maturity models where the upper level introduces a new concept. No documented evaluation was found.

4.2. Accessibility maturity models for organizations for digital products

The nine maturity models described in this subsection are all developed to analyze organizations, that design and develop digital products, on their maturity to produce accessible products. This includes the development lifecycle with steps like planning, development, designing, and testing.

4.2.1. Accessibility Maturity Model Scorecard (AMMS)

The AMMS is assumed to be the predecessor of one of the best known maturity models for accessibility—the AMM (see later), as it was published in 2011 by the Business task force on accessible technology of the Business Disability Forum (Day, 2011). The target group of the AMMS are software-development organizations. It is structured as a matrix with 5 levels and 8 dimensions with qualitative indicators and conceptual level descriptions (Day, 2011). We found no information on an empirical foundation nor an evaluation, as it is an outdated version.

4.2.2. Accessibility Evolution Model (AEM)

The AEM is a maturity model developed by Microsoft for its own use with CMMI and DAMM as conceptual predecessor models (Lay-Flurrie, 2020). It was published in 2016 and is structured as a matrix with five stages and eight dimensions and conceptual stage descriptions (Lay-Flurrie, 2020). In contrast to other maturity models, the AEM defines neither any concrete indicators in the publication nor any information about how to use it. As the AEM is developed for Microsoft only, the empirical foundation is case-based on the individual Microsoft situation. Accordingly, the AEM has only been evaluated in a practical setting by Microsoft (and not in other organizations) and there are no findings published.

4.2.3. Maturity Model for Accessibility (MMA)

The MMA is a maturity model published in 2017 by Bailey and Gkatzidou for organizations to develop accessible products (Bailey and Gkatzidou, 2017). It uses a UX model from Renato Feijo as the conceptual predecessor model (Bailey and Gkatzidou, 2017). It has 6 stages arranged as stairs with no dimensions, where each level represents a new concept (Bailey and Gkatzidou, 2017). The levels of the MMA are not linked to concrete metrics, so the evaluation process is likely subjective and there are no instructions on how to collect data for the evaluation. There is no documented empirical foundation other than the model being based on the UX model and having been adapted toward accessibility. Regarding the evaluation of the model, a use case is mentioned, however, there are no documented results.

4.2.4. Usability and accessibility focused process improvement capability model (MODECUA)

The “Modelo para la determinación de la Capacidad de mejora de procesos centrados en la Usabilidad y la Accesibilidad” (MODECUA) is a capability maturity model for usability- and accessibility-centered processes, published in 2018 in Spanish by Quintal and Macías and based on various international quality standards (Quintal and Macías, 2018). It is used by organizations to optimize their processes to develop usable and accessible software products (Quintal and Macías, 2018). MODECUA is structured in 6 phases that follow processes, and 5 levels with a checklist (Quintal and Macías, 2018). The structure differs in comparison to other maturity models, as it has no dimensions, but rather process attributes and attribute practices per level with subsequent practice requirements (Quintal and Macías, 2018). For the development of MODECUA, various standards and maturity models were used as a basis (Quintal and Macías, 2018), but with no documented empirical foundation. There is also no documentation of results of their evaluation in a practical setting. In contrast to many of the other accessibility maturity models, MODECUA uses quantitative data analysis to assess the maturity.

4.2.5. Digital Accessibility Maturity Scorecard (DAMS)

The DAMS was developed by Hassell Inclusion Limited on the basis of ISO/IEC 30071-1:2019 (International Organization for Standardization, 2019), and was published in 2020 (Hassell Inclusion Limited, 2020). Its structure is a checklist with many questions categorized in 9 dimensions (Hassell Inclusion Limited, 2020). The result is 1 of 4 stages (Tiers), that are calculated by quantitative methods and collected via a questionnaire (Hassell Inclusion Limited, 2020). As the calculation process is not published, the users need to fill in the questions on the website and get an email with the results and possible changes for optimization. There is no empirical foundation nor evaluation documented for the maturity scorecard.

4.2.6. Accessibility Maturity Model (AMM-M)

The AMM-M was developed on the basis of the Usability Maturity Model (1998) and was published in 2020 (Michel, 2020). It was developed for the analysis of design processes of products in government agencies and is structured as a checklist with no dimensions (Michel, 2020). It contains items that are evaluated on a 4-likert-scale to calculate the actual stage out of 6 stages through interviews and the collection of quantitative data (Michel, 2020). The AMM-M is administered by calculating the responses of all team members separately, resulting in a final stage (Michel, 2020). The model and the process were evaluated in some interviews, workshops, and practical settings (Michel, 2020), without results having been published. No empirical foundation is documented other than changing some items in the underlying Usability Model.

4.2.7. Accessibility maturity model (iSoftStone) (AMM-iSoftSt)

The AMM-iSoftSt is a maturity model used to analyze accessibility issues in the software development life cycle, and was published in 2020 by iSoftStone (Baron, n.d.). It has only 2 + 1 dimensions in a matrix with five stages (Baron, n.d.). The analysis is meant to be self-assessed, with no instructions on how to collect the data. For the AMM-iSoftSt, neither a documented empirical foundation nor an evaluation was found.

4.2.8. Maturity Model for Digital Accessibility in Digital Product Organizations (MMAP)

The German “Reifegradmodell für digitale Barrierefreiheit in digitalen Produktorganisationen” (MMAP) was developed by Feistauer for organizations to optimize their production of accessible products, especially for the marketing division of a large company, and published in 2021 (Feistauer, 2021). It is based on various maturity models: AMM, CMMI, DAMS, DAMM, AEM (Feistauer, 2021). The structure is a matrix with five stages and 12 dimensions (Feistauer, 2021). For the data to be collected, Feistauer (2021) suggests group discussions or interviews with team members. MMAP was developed on a case-based empirical foundation with individual cases of three interviews in the company (Feistauer, 2021). For the evaluation, interviews and a simulation were conducted but with no concrete results (Feistauer, 2021).

4.2.9. W3C Accessibility Maturity Model (W3C AMM)

The first version of the W3C AMM was published in 2021 (Byrne-Haber et al., 2022) but it is still a work in progress and updated regularly. Its matrix has 4 stages with conceptual descriptions and 7 dimensions (Byrne-Haber et al., 2022). It is meant to be used by any organization; while higher education institutions are mentioned, they are not specifically in focus of W3C AMM (Byrne-Haber et al., 2022). There is no empirical foundation or evaluation documented at this time.

4.3. Accessibility maturity models for public sector organizations

The public sector organizations of the maturity models described in this section are public sector organizations in general as well as governments, cities, and libraries.

4.3.1. Policy-Driven Adoption for Accessibility (PDAA)

The PDAA is a maturity model developed by the National Association of State Chief Information Officers (NASCIO) and published in 2015 to analyze and optimize the ICT accessibility of government procurement processes (Ward, 2015). It is structured as a matrix with 3 stages and 6 dimensions with activity descriptions (Ward, 2015). As an addition to the matrix, some states (including Texas and Minnesota) have developed a questionnaire to determine the actual score quantitatively (Kuykendall, 2016). There is neither a documentation about an empirical foundation nor a validation of PDAA.

4.3.2. Smart City for All (SCity4A)

The SCity4A is a maturity model published in 2016 and developed for cities, however, it is found to be beneficial to other entities like universities (G3ict, 2019). It is structured as a checklist with five dimensions, subtopics, and items (G3ict, 2019). There are no stages, but the cities evaluate themselves by answering whether it is “happening today (not at all, partially, fully)”, what the priority level (1–3) and who the responsible party is. The items are activity descriptions and are meant to be evaluated by group discussions (G3ict, 2019). For the development of SCity4A, many experts were engaged to provide an empirical foundation (G3ict, 2019). It has also been evaluated in a practical setting with at least 3 cities (G3ict, n.d.), but no results of the evaluation were published.

4.3.3. Accessibility Maturity Matrix for Libraries (AMML)

The AMML was published in 2018 by German and Hartnett (2018) as a preliminary first version. It is based on the AMMS and applies to libraries (German and Hartnett, 2018). Like the AMMS, it is structured as a matrix with the equivalent 5 stages that are conceptually described, but it has only 7 dimensions (German and Hartnett, 2018). There is no information or instructions published on how to use the AMML, as its goal is to give a first idea of a structure of such a maturity model. There is no documented empirical foundation or evaluation.

4.3.4. Diversity and Inclusion Capability Maturity Model (D&I CMM)

The D&I CMM was developed based on literature and industry-based studies by Lundy et al. and was published in 2021 with the target groups being public sector organizations (Lundy et al., 2021). It is a matrix with five stages and five dimensions, with individual maturity levels for each dimension (Lundy et al., 2021). The development has a case-based empirical foundation with industry-based studies (Lundy et al., 2021), but there is no documented evaluation of the model.

4.4. Accessibility maturity models for all organizations (undefined)

All maturity models that have no specific target area are described in this section. These are either meant to be general for many different organizations or there was no explicit description of the target area for these maturity models.

4.4.1. Disability Equality Index (DEI)

The DEI is a disability inclusion rating tool for companies, first rolled out in 2012 as a joint initiative of Disability:IN and the American Association of People with Disabilities (AAPD) (Disability:IN, n.d.). It was developed based on expert groups from different companies and is structured as a questionnaire with 6 dimensions (AAPD and Disability:IN, 2022). There are no stages but organizations are classified as a “Best Place to Work for Disability Inclusion” if they have 80 or more points in the questionnaire (AAPD and Disability:IN, 2022). The included questions cover a wide array of an organization's areas, hence input from many stakeholders is needed to answer all questions. DEI measures quantitative as well as qualitative data. As many experts from different companies worked on the development of DEI, it is based on empirical work as described by Ahlemann et al. (2005). Its questionnaire is used on a yearly basis by a large number of companies, hence one can assume that the available questions and results are evaluated, but there are no evaluation findings published.

4.4.2. Digital Accessibility Maturity Model (DAMM)

The DAMM was published in 2015 by Level Access as an approach for measuring the degree of maturity a program has attained in managing and implementing accessibility (Level Access, 2015c). It is based on the work of the Business Disability Forum (AMMS) and structured as a matrix with 5 stages and 10 dimensions (Level Access, 2015a). For gathering the qualitative data in the analysis the publisher proposes using interviews, group discussions, documents, and other data, to get a holistic overview of the company or project (Level Access, 2015b). DAMM has no documented empirical foundation or evaluation.

4.4.3. Accessibility Maturity Model (AMM)

The AMM was published by the Business Disability Forum in 2016 and is assumed to be the successor of the AMMS from 2011. It is structured as a matrix with 5 levels and 10 dimensions that are based on the 10 points of the Accessible Technology Charter (ATC) (Business Disability Forum, 2021b). As in the AMMS, the level descriptions are conceptual. For a better analysis, Microsoft has developed an additional in-depth questionnaire to assess the AMM (Vermeersch, 2018). As the AMM is widely used, it has been evaluated in many practical settings (Caldwell, 2016). Nevertheless, there are no documented results of the evaluations and there is no description of an empirical foundation.

4.4.4. Seven Stages of Inclusion (7SI)

The 7SI maturity model by Ashton and McElvane was published in 2019 and developed with the Inclusive Leadership Index (ILI) and the Six Pillars of Diversity and Inclusion as a foundation (Ashton and McElvane, 2019). It has 7 levels in its matrix, that are divided into 3 phases (Ashton and McElvane, 2019). The levels have no dimensions and are conceptual descriptions. The 7SI was developed with input from over 500 companies over 8 years and therefore is empirically grounded (Ashton and McElvane, 2019). However, no documented evaluation could be found for 7SI, and it is not specified how to collect data when using the model.

4.4.5. Gartner Inclusion Index (GII)

The GII was published in 2021 and consists of a short questionnaire with 7 dimensions (Romansky et al., 2021). It allows organizations to get a holistic view of inclusion from their workforce by asking employees 7 questions (Romansky et al., 2021). There are no stages in this maturity model; the more the employees agree with these statements, the more inclusive the organization is deemed to be (Romansky et al., 2021). The model was developed through interviews with 30 executives (Romansky et al., 2021), providing a case-based empirical foundation. No documentation could be found about an evaluation.

4.4.6. Maturity Model Web Accessibility (MMWA)

For the Swiss “Reifegradmodell Web-Accessibility” (MMWA), it is unclear when it was developed and published by Hightech Zentrum Aargau AG and the Fachhochschule Nordwestschweiz FHNW (Threeway, n.d.). It was developed for all organization types to analyze the maturity in creating an accessible website. The conceptual predecessor models of the MMWA are various maturity models, e.g. CMMI and DAMM (Threeway, n.d.). The MMWA is structured as a checklist with five dimensions, 18 subtopics, and 63 indicators, where users answer each indicator on a 4-value Likert scale (Threeway, n.d.). The resulting stage is calculated using quantitative analysis with different weightings for the indicators to be assigned as one out of 5 possible stages. The resulting level descriptions are activity descriptions. Neither a documented empirical foundation nor evaluation was found in the documentation.

5. Comparison of the maturity models

Only six of the 25 maturity models found in this study were developed to support higher education institutions in optimizing their accessibility (see Section 4.1). The other maturity models were developed for organizations for digital products (9), public sector organizations like governments, cities, or libraries (4), or for all organizations with no target area specified (6). Interestingly, in the historical overview the first two maturity models for accessibility were models for higher education institutions (IECM in 2005 and HE-&FE-AMM in 2006)¹. After that, since 2011, many maturity models for other organizations were developed and only a few more maturity models for higher education institutions followed. Moreover, only 2 of the 6 maturity models for higher education were found through the literature research (ISOLearn QL and HE-&FE-AMM).This shows that most of these maturity models are not published but remain e. g., on the websites of the relevant institution.

None of the existing maturity models was developed to explicitly analyze and optimize the accessibility of teaching and learning about accessibility, but rather the maturity of accessibility in the whole institution. Some maturity models mention topics like student learning and curricula, but no maturity model solely focuses on this topic.

The structure of the maturity models differs significantly. We found 13 maturity models that are structured as a matrix with stages and dimensions (3 higher education models). Five maturity models include checklists, and three additional models revolve around questionnaires (1 higher education model). One of the higher education maturity models consists of a framework, and one is structured as stairs. Moreover, there are two maturity models that combine structures, one which combines a matrix and a questionnaire, and the other which combines a matrix with a roadmap (higher education model). For the number of stages in the maturity models, the majority have 5 stages (12 maturity models, 3 higher education models). Only three maturity models have 6 stages (2 higher education models), four models have 4 stages and only 1 model has 3 stages. The remaining five maturity models have no stages at all (1 higher education model).

As already mentioned in the descriptions of the maturity models, only two models (MM-UniA and MMA) have no level dependencies as upper levels make up new concepts. For all other maturity models, upper levels comprise lower levels. The analysis to ascertain the actual level of maturity differs between qualitative (19 maturity models, 5 higher education models) and quantitative (8 maturity models, 1 higher education model) approaches. Some of the models use both qualitative and quantitative indicators for the analysis.

Many of the maturity models do not have an empirical foundation, as described by Ahlemann et al. (2005). Only 4 maturity models (16 %: DEI, ISOLearn QL, 7SI, and SCity4A) had documented empirical groundwork with many experts. Another 8 maturity models (32 %: D&I CMM, AEM, MMAP, SCampus4A, MM-UniA, HE&FE-AMM, and GII) had a case-based empirical foundation. For the remaining 13 maturity models (52 %), no empirical foundation was documented. When separating the models by their target group, higher education models have an empirical foundation (either case-based or groundwork) for 83 % and no (documented) empirical foundation only for 17 % of the models. For the other maturity models, it is the other way around, where 63 % do not have a (documented) empirical foundation and 37 % have one.

Even fewer maturity models have been evaluated. None of the maturity models had documented results from an evaluation with methods such as prototypical implementation, case studies, practical settings, or surveys of users. Nine maturity models mentioned an evaluation but did not document the results of the evaluation (36 %: D&I CMM, AEM, MMAP, MMA, ISOLearn QL, MODECUA, SCity4A, AMM, DEI, AMM-M). The remaining 16 maturity models (64 %) provide no documented evaluation at all with 2 maturity models still being works in progress and therefore do not have an evaluation as yet. Separating the models by target area, the evaluation is even less for higher education models, as only 17 % documented any evaluation, 83 % had no (documented) evaluation at all. The other maturity models documented their evaluation more often, as 42 % of the non-higher education models have a documented evaluation.

6. Indicators of the existing maturity models

This section describes the indicators that the maturity models² use to evaluate and optimize accessibility in organizations. Table 3 shows a matrix where the maturity models are mapped to their indicators. During the study, we clustered the various metrics of the maturity models into indicators which we will describe in the following sections. For the categorization of the metrics, we first collected all metrics of the maturity models and sorted them inductively, resulting in these 34 indicators.

TABLE 3

Table 3. Matrix of maturity models and indicators.

6.1. Accessibility

The indicator accessibility includes the overall accessibility of the workplace. Here, different aspects like the accessibility of its IT systems (MM-UniA, AMMS, AMML, SCity4A, CMM, AMM, DEI), its used documents (AMM-iSoftSt, DAMM), and its web appearances (CMM) are assessed. Furthermore, the accessibility of the institution's communication is evaluated (W3C, AMM, DEI) as well as its level of physical accessibility (DEI, AMML).

6.2. Application process

The indicator application process contains topics such as an accessible application process (SCity4A, W3C AMM, DEI), which includes the platform of application (W3C AMM, DEI), job interviews, and job evaluation (W3C AMM). Furthermore, the DEI requests accommodation for applicants with disabilities. Another topic covered by the application process is the requirement of accessibility knowledge for new employees in the vacancy (MMAP).

6.3. Awareness

Awareness is measured by the maturity models in differing degrees of sensitization (SCity4A), specific events to raise awareness (SCity4A), communication of the relevance and importance of accessibility (SCity4A, AMM-M), and “accessibility as business as usual” (AMM).

6.4. Collaboration

AMM and IECM measure collaboration through existing cooperation between different sections of an institution to ensure accessibility. D&I CMM takes high performing and diverse thinking teams into account.

6.5. Communication

Communication is one of the most well represented indicators found in the different maturity models. Generally, the indicator can be divided into internal and external communication. For internal communication, the visibility of the following is assessed: support offerings and diversity activities (CMM, SCity4A), commitment to accessibility, standards, policy, requirements, goals, and strategy (DAMM, PDAA, SCity4A, IECM), diversity topics, importance, and know how (DAMS, CMM, IECM, ISOLearn QL), existing resources (SCity4A, W3C, AMM), and processes (CMM, PDAA). Additional factors are the frequency with which accessibility is displayed in education media (W3C AMM), the accessibility of internal communication itself (DAMM), and through which formats the communication is distributed, e.g., internal systems, the internet, direct contact, or print (W3C AMM).

With regard to public relations, it is investigated whether accessibility statements (DAMS) or documents (e.g., VPAT) (DAMM) can be found on the institution's website, if the public is informed on guidelines, funding, and implementation (SCampus4A), and how public relations in fields of diversity, equality of opportunities, and anti-discrimination are handled (SCity4A). Therefore, external communication focuses more generally on the external perception and effect of the organization (AMM), on marketing, press, and media work (AMM-iSoftSt, AEM), and the employment of external communication for reaching people with impairments (DAMS, DEI). Also, the organization of events concerning accessibility, diversity, and more is considered (AMM-iSoftSt).

6.6. Compensation of disadvantage

ISOLearn QL is the only maturity model checking for disadvantage compensation. Attention is paid to the joint development of approaches by students, teachers, and support staff. As examples of disadvantage compensation, ISOLearn QL mentions ongoing performance assessments instead of single final examinations, writing time extensions, and, if necessary, the division of a longer test.

6.7. Competences and training

The indicator competences and training can be separated into competences, which is the goal of the indicator, and training, which is one possibility of how to reach that goal. For competences, DAMS, HE-&FE-AMM, PDAA, MMWA, and W3C AMM all refer to the bundling, transfer, and generation of competences. MMWA focuses on the level of knowledge, understanding and implementation of accessibility guidelines, and how well general knowledge of this field is centralized. Measurements for competences can, for example, be certifications (AMM-iSoftStone) or evaluations of staff's knowledge (DAMS).

Trainings help to develop appropriate skills (AMM-M, MMAP, DEI, 7SI, AMM, MM-UniA, DAMM, MMWA, SCity4A, W3C AMM, AEM, AMM-M, and D&I CMM) and to create awareness (AMM, DAMM, AEM, AMM-M). W3C AMM, AMM-iSoftSt, and DAMM examine training schedules, while CMM and DAMS value whether trainings are available for different stakeholders. ISOLearn QL, MMWA, and 7SI review the regularity of trainings, and SCity4A audits if persons with disabilities are integrated as experts.

6.8. Consulting

The indicator consulting considers whether there exists a clear strategy for the provision of expert accessibility support (DAMS). With respect to a whole organization, AMM and MMWA focus on the employment of external consultants on accessibility. MM-UniA also mentions individual consultation as important.

6.9. Culture

The nine models mentioning culture evaluate whether a culture of accessibility or an inclusive culture is present at the institution in question (7SI, HE-&FE-AMM, AEM, D&I CMM, DEI, MMWA, GII). The IECM focuses on the psychological and behavioral climate which should be supportive of all students. W3C AMM recognizes shared beliefs and values as a factor of culture. The DEI also considers the support for disabled employees to self-identify as persons with impairments.

6.10. Curriculum

The indicator curriculum assesses the accessibility of courses with respect to accessible materials (CMM, ISOLearn QL) or accessible lectures (ISOLearn QL). The indicator is also used for available course content on accessibility and inclusion within the curriculum (IECM, SCity4A).

6.11. Customer focus

The indicator customer focus evaluates whether the user's accessibility requirements and needs are collected (AMM-M, MODECUA, DEI) and whether user satisfaction is considered (7SI, MODECUA). The DEI mentions methods such as focus groups, market research, and usability studies when working with a disability consultant or expert, in employee resource groups, affinity groups or a diversity council.

6.12. Diversity

The indicator diversity investigates, if the working force is diverse, for example if there are employees with impairments in the organization (AEM, SCity4A, DEI), if people with impairment work in all levels of the institution ‘s hierarchy (GII, IECM, SCampus4A, W3C AMM, D&I CMM, 7SI), or in the context of higher education institutions, if the student body is diverse (IECM).

6.13. Engagement

For engagement, some maturity models measure the internal engagement of an organization, like how inclusive, personalized, and multidirectional their engagement models are (SCampus4A) or how proactive their behavior is (HE- & FE-AMM). Most models focus on the external engagement of the organization, with respect to active input for organizations, events, or public forums (DEI, DAMS, DAMM, MMWA), membership in committees, unions, or initiatives (AMM-iSoftSt, DEI), working on norms and standards affecting accessibility (DAMS, IECM), or participating politically and judicially (IECM).

6.14. Feedback

The indicator feedback entails adequate complaint management (AMM, DAMM, PDAA, AMM-M, DEI), including a definition of the feedback and complaint process (DAMM, PDAA, AMM-M, DEI). Additionally, the DAMM focuses on feedback mechanisms on strategy and its documentation, whereas the SCity4A focuses on a feedback mechanism to identify areas for improvement and innovation in general.

6.15. Incentive systems

Incentive systems are mechanisms to motivate people or the whole organization regarding optimizing the level of accessibility. The maturity models mention different possibilities for incentive systems including material (SCity4A, DEI) or non-material (GII), for example certifications (SCity4A), symbols and badges of accessibility (DEI), rewards or recognition (GII).

6.16. Infrastructure

Regarding infrastructure, the SCampus4A investigates the equipment and furnishing of rooms, lecturers, students with respect to accessibility, as well as innovative technology like connected devices. Likewise, the SCity4A takes into account a clear technical infrastructure.

6.17. Innovation

The SCampus4A, AEM, DEI, and SCity4A evaluate the application of innovations to improve accessibility. The DAMS, in contrast, considers whether new product innovations are created via accessibility, how well digital accessibility is included into activities concerning digital trends and innovations, and if pioneers are actively followed to stay up to date in the field of accessibility. The SCity4A represents this indicator by how well innovations in the field of accessibility are promoted and supported.

6.18. Legal requirements

Legal requirements, i. e. compliance with legal standards, such as the WCAG or VPAT standards, is a metric collected by the DAMS, IECM, SCity4A, D&I CMM, MMAP, and SCampus4A. The DAMM breaks this down to federal, national, and municipal levels. The DAMM and MMAP also track whether the legal department consistently oversees compliance with legal standards.

6.19. Monitoring

Many maturity models assess the collection and evaluation of data in organizations: whether data is collected on how widely the offerings related to accessibility are used (DAMS, IECM, DAMM, W3C AMM, SCity4A), whether collected data is used to further develop the offerings (PDAA, SCampus4A, MM-UniA, CMM, D&I CMM, SCity4A, AEM, MODECUA), and how the lectures in higher education institutions are evaluated (ISOLearn QL). The DEI, PDAA, DAMM, MMAP, and ISOLearn QL consider whether monitoring results are published, while the MMWA, AMM, SCampus4A, AMM-iSoftSt, DAMM, MMAP, and AEM inquire about the impetus for continuous improvement of efforts.

6.20. Networking

Networking is related to the indicator communication and can be divided into internal and external exchange. Externally, it covers networking across institutional boundaries (AMM, SCity4A, DEI) and providing open-source materials (AMM). Further, this can include the existence of (external) partnerships (ISOLearn QL, SCampus4A), participation in peer groups (DEI), and leading associate partners to accessibility (AMM, SCity4A). The internal networking can be assessed according to available discussion forums (IECM, SCity4A) or networking in general (MMAP).

6.21. Onboarding

With onboarding, the maturity models assess whether the onboarding process itself is accessible (W3C AMM) and whether accessibility is a topic that is mentioned to new employees during the process. Possibilities for mentioning accessibility are onboarding programs (e.g., New Hire Orientation mentioned by DEI), materials about accessibility (DEI), training (MMAP, AMM, W3C AMM, DEI), or checking the competences of new employees in the onboarding process (MMAP). The CMM explicitly requires an onboarding with accessibility for all stakeholders (staff, student, faculty).

6.22. Participation

Many maturity models mention participation as relevant for accessibility. Participation means the inclusion of relevant stakeholders from all status groups in decision making processes and strategies (HE-&FE-AMM, IECM, DAMM, GII, AMM-M, ISOLearn QL, MODECUA). It is also relevant if people with disabilities are active in an advisory capacity (AMM, DAMM, DAMS, and D&I CMM).

6.23. Policy

The indicator policy tracks the existence of accessibility, disability, or diversity policies (HE-&FE-AMM, DAMS, PDAA, W3C AMM, MMWA, DEI, MODECUA). This includes the creation, deployment, operational conformance, and maintenance of the policy on an ongoing basis (DAMM). Additionally, there are other policies that should integrate accessibility, for example in the areas of recruitment and workplace adjustment (DAMS), social media (DAMS, W3C AMM), communication (W3C AMM), meetings and events (DAMS), procurement (HE-&FE-AMM, W3C AMM), IT (HE-&FE-AMM, PDAA), teaching and learning (HE-&FE-AMM), and marketing (HE-&FE-AMM).

6.24. Priority

The AEM, HE-&FE-AMM, D&I CMM, and MMA review whether it is ensured that accessibility is given a high priority throughout the institution but offer no concrete measurements.

6.25. Process

For the indicator process, both the existence of specific accessibility processes (DEI, MM-UniA, D&I CMM, AMM-M, AMMS, CMM) as well as the integration of accessibility into existing fundamental processes in other areas (PDAA, CMM, DAMM, MMAP, DAMS, D&I CMM, AMM-M) is assessed. Specific accessibility processes include for example accommodation processes for persons with disabilities (DEI, MM-UniA), processes for identifying and removing barriers (D&I CMM, AMM-M, AMMS), and for developing alternatives when barriers arise (CMM). Fundamental processes in other areas that should integrate accessibility are processes such as governance [AMMS, DAMM), risk management (AMMS, DAMM), compliance (DAMM), launch (DAMM, AMM(M)], and communication (DAMM). It is examined whether accessibility is consistently considered and documented (DAMM, CMM), whether these processes and their monitoring are continuously improved (DAMM, AMM-iSoftSt, MMAP), and generally whether there are both top-down and bottom-up processes (IECM).

6.26. Procurement

The indicator procurement can be broken down into three parts. The first part is about the need for accessibility requirements in the procurement process (CMM, DAMM, DAMS, PDAA, SCampus4A, W3C AMM, AMML) and for all purchased products to be accessible (DEI, CMM, W3C AMM). This should be documented in the procurement policy (DAMM, DAMS). The second part is the accessibility of suppliers. It is important to pay attention to whether suppliers implement and live accessibility, for example in their culture (MMAP, DAMS, AMMS, DEI). The MM-UniA, AMMS, DEI, and AEM require organizations to support suppliers in improving their accessibility, and, in addition to that, the DEI requires cooperation with companies that are owned by people with disabilities. The third part is the process of evaluating purchased systems on their accessibility (DAMS).

6.27. Product life cycle

The indicator product life cycle is only mentioned by maturity models for software organizations, not by those targeting higher education institutions. It measures whether and how requirements of accessibility are considered during the whole software development life cycle (AMM, DAMS, AMMS, AEM, AMML) and tested from the start (AMM, MMA, MMAP). It includes the process steps of planning, designing, coding, building, deploying, testing, and receiving feedback (AEM), and all parts of the software itself like features, design, and content (MMAP).

6.28. Resources

The indicator resources consists of several types of supports for improving accessibility (IECM, MMA). The IECM, AMMS, AMML, DEI, 7SI, AMM, MM-UniA, CMM, DAMM, MMWA, SCampus4A, and ISOLearn QL investigate the allocation of financial resources for accessibility measures. The AEM, IECM, AMM-M, and MM-UniA survey personnel resources. The AEM, IECM, and AMM-M check if technology resources are provided (i.e., tools and equipment). The AEM further assesses structural resources like processes and the IECM includes symbolic resources. Additionally, the indicator resources is mentioned by the SCity4A in terms of using existing resources and programs to provide access for persons with disabilities.

6.29. Responsibility

Responsibility focuses on people in the organization who are accountable for coordinating and implementing accessibility. One responsible party for accessibility is the leadership of the institution (7SI, IECM, MMWA, SCampus4A, AMM, MM-UniA, DAMS, DAMM, SCity4A, W3C AMM, AMMS, AEM, MMA, DEI). Other stakeholders are working groups (IECM, DEI) and the students (ISOLearn QL). In contrast, the MMAP sees everyone in the organization as co-responsible, an approach also seen with the D&I CMM, which mentions everyone in general and the leadership in particular. Some maturity models request a clear definition of responsible roles or departments (IECM, MMAP, MMWA, PDAA, CMM, DAMM, DAMS, HE-&FE-AMM, SCity4A). Another part of the responsibility are the governance structures that regulate committees, procedures, decision making powers, and the parties' relationships (7SI, MM-UniA, PDAA).

6.30. Return on Investment (RoI)

Three maturity models measure to what degree accessibility investments are subject to Return on Investment analysis (DAMM, DAMS, SCampus4A). The DAMS further specifies how expenses for accessibility are documented, and if their subsequent saving costs and the influence of accessibility on brand reputation is measured.

6.31. Standards

Standards need to be used in the organization (MM-UniA, SCampus4A, MODECUA) by building toward and testing against them (AEM), and need to be communicated within the whole organization (MM-UniA). The DAMM and AMMS request the documentation and publication of ICT accessibility standards for all types of products.

6.32. Strategy

The indicator strategy revolves around the existence of the topic accessibility in an organization's strategy, specifically an explicit accessibility strategy (AEM). This includes surveys, goal setting, plan of actions, and metrics (MMWA, SCampus4A, SCity4A, IECM, MM-UniA) as well as risk analysis and management (MODECUA), and is a cross-sectional task within the organization (AMM). The models also require a documentation of the strategy (CMM, ISOLearn QL, DAMM, MODECUA, AMML), its guidelines and policies (7SI, D&I CMM) or an established action plan, timeline, or roadmap (MMWA, SCampus4A, Scity4A, IECM).

6.33. Support

Support has various characteristics. The first aspect is support for implementing accessibility, for example, through a help desk, a support team, or training (AEM, DAMM). Further, available counseling for the implementers at the institution is considered (DAMS, DEI, IECM), as well as the provision of information materials such as guidelines and training materials to create accessible formats (CMM, DAMM). Another aspect is the support for people with disabilities (W3C AMM, ISOLearn QL, DEI).

6.34. Testing

The indicator testing concerns itself with the continuous testing and evaluation of digital products for their accessibility (AMM-M, MMA, CMM, AMM-iSoftSt, MMAP, W3C AMM). There are different ways of testing digital products, namely manual, program-based, and automated testing (AMM-iSoftSt, DAMM, DAMS, MMWA, MMAP, W3C AMM). The maturity models also differentiate between the products tested: the DEI focuses on testing internally and externally used products, the CMM mentions websites and course materials. Another important part of the testing mentioned by the DEI, MM-UniA, DAMM, MMWA, and SCity4A are user tests including people with disabilities and of Assistive Technologies. Lastly, the CMM also takes a clear responsibility for testing into its focus.

7. Frequencies of indicators included in the maturity models

Figure 3 shows the list of indicators, ordered in descending frequencies by the absolute numbers of maturity models mentioning them. Additionally, their percentage distribution is shown separated between maturity models targeting higher education institutions and other organizations (not higher education). For both types the 100 %-mark is highlighted by a line (for higher education n = 6, for others n = 19). The analysis of the indicators found in the maturity models allows detection of the indicators that were most frequently mentioned in the models. “Responsibility” is mentioned most, in 20 out of the 25 maturity models. Next, the indicators “competences & training” and “monitoring” are mentioned in 18 maturity models, followed by “resources” in 16 maturity models.

FIGURE 3

Figure 3. Frequencies of the indicators mentioned by maturity models in absolute numbers and percentage distribution.

Separating the maturity models for higher education institutions from those for other target areas, the most frequently included indicators differ. In models for higher education institutions, the most often mentioned indicators are “responsibility” (6 out of 6 models), “monitoring” (5 out of 6 models), “resources” (5 out of 6 models), and “strategy” (5 out of 6 models). For the other maturity models, the highest frequencies are for “competences & training” (14 out of 19 models), “responsibility” (14 out of 19 models), and “monitoring” (13 out of 19 models).

A comparison of the percentage distribution of the numbers of mentions per indicator, between higher education and other models, reveals large differences for some indicators (cf. Figure 3). Firstly, there are some indicators that are only included in higher education maturity models: “compensation of disadvantages” (1 model) and “curriculum” (3 models). On the other hand, some indicators are only mentioned in the maturity models for other organizations: “Application” (4 models), “awareness” (3 models), “customer focus” (4 models), “feedback” (6 models), “incentive systems” (3 models), and “product lifecycle” (7 models). Secondly, some indicators are mentioned more frequently by higher education models: “strategy” (higher education 83 %; others 47 %), “engagement” (HEI 50 %; others 21 %), “networking” (higher education 50 %; others 21 %), “responsibility” (higher education 100 %; others 74 %), “resources” (higher education 83 %; others 58 %), and “support” (higher education 50 %, others 26 %). On the side of the other models, only one indicator is mentioned more frequently than in higher education models: “policy” (others 37 %; higher education 17 %); except for the indicators that do not at all occur in higher education models.

8. Discussion

In this section, we return to the targeted questions regarding maturity models for higher education institutions and discuss them based on our study results.

8.1. What accessibility maturity models for higher education institutions exist (RQ1)?

In the literature research, we found 13 different maturity models for accessibility, only two of them addressing Higher Education Institutions. An additional web search increased the number to 25 maturity models. Out of the 25 maturity models, six specifically target higher education institutions (IECM, HE & FE AMM, ISOLearn QL, SCampus4A, CMM, MM-UniA³). Target areas of the other maturity models included, for example, software organizations, government, and cities. As the focus of this paper is on maturity models for higher education institutions, this discussion will continue by comparing higher education models with the others, not separating the other models by their target areas.

When looking at the historical overview of the maturity models found (see Figure 2), the first two maturity models for accessibility were models for higher education institutions in 2005 and 2006 and not maturity models for software organizations. This contrasts with the development of maturity models in general, that were first developed for software organizations (CMMI). Following these first two maturity models for accessibility, 5 years later, many other maturity models for accessibility with different target areas were published.

8.2. How do the accessibility maturity models measure accessibility maturity (RQ3) and how do they differ depending on their target areas (RQ2)?

On the one hand, the structure of the maturity models is an important aspect of how they measure accessibility. They include various structures like matrices (52 %), checklists (20 %), questionnaires (12 %), frameworks (4 %), or various combinations thereof (12 %). Looking at the higher education models, the distribution is very similar, except that they do not use checklists. The maturity models also differ in the number of stages they describe, varying from none over 3 to 6 different stages.

On the other hand, the content of the maturity models and therefore the mentioned indicators describe how they measure accessibility. The indicator “responsibility” is mentioned in most of the maturity models (20 out of 25). Often stated as well, are the indicators “competences & training” and “monitoring” (each by 18 models), followed by the indicator “resources” (16 models).

Separating the maturity models by their target areas, the included indicators differ. First, some indicators are either only mentioned by higher education models (“compensation of disadvantages” and “curriculum”) or by models for other organizations (“application”, “awareness”, “customer focus”, “feedback”, “incentive systems”, “product lifecycle”). Moreover, there are differences in the frequency with which some indicators are mentioned between the target areas. It would be worthwhile to carry out further research on whether these differences relate to the type and structure of the target organization. For some indicators such a relationship could be implied, for example, higher education institutions rarely develop their own products and therefore the indicator “product lifecycle” is less important than for software organizations. The same is true for the two indicators that are only included in higher education models (“compensation of disadvantage” and “curriculum”) since both are directly related to the work of higher education institutions. For the other indicators that are only included in other models, it is not as obvious why they are not covered in any of the higher education models. Here, a further analysis would be of value, to determine whether these indicators could be applied to higher education institutions as well, or if the differences in the maturity models are due to different types and structures of organizations.

A topic that was not mentioned by any of the maturity models was research on accessibility and inclusion, even though, especially for higher education institutions, it is a very prominent topic. We see important points there, for example, in accessible publications and publication catalogs or that research on accessibility and its funding should be more transparent. It would be worthwhile to explore the impact on the accessibility of a higher education institutions of research work and the influence it has, for example, on teaching methods.

8.3. Transferring maturity models

Looking at the differences in the mentioned indicators and discussing to what degree they are applicable across various organization types, the question also arises of whether maturity models in general are transferable. As most of the maturity models found in this study were developed on case-based knowledge and seldomly evaluated, it is unclear whether the resulting models are applicable to other organizations of the same type at all. Moreover, it is unclear whether they are transferable between different organization types or target areas, e. g. whether a maturity model for government can also be applied to a higher education institutions. Biberoglu and Haddad (2002) as well as Mettler and Rohner (2009) for example assume, that a transfer between large and small organizations could be problematic, as change may affect small organizations more than large organizations. Other contextual factors could be economic orientation, decision making or departmentalization (Mettler and Rohner, 2009). It is also questionable, whether these models are transferable internationally—especially those for higher education institutions. There is some research on the comparison of higher education institutions in different countries, for example the differences in policies (Goedegebuure, 1994), quality assurance (Billing, 2004), or the possibilities for studying abroad (Burn et al., 1990) which show that it is possible to compare higher education institutions internationally by using frameworks. However, Teichler (1996) points out that such international comparative studies always bear some problems, for example language barriers, a high amount of costs and effort, and sufficient field knowledge that is very time-consuming to acquire. Also, the results of Billing (2004) show, that a general model does not universally apply and adaptations are required. More research would be necessary to evaluate transferability, for instance by comparisons of structures and hierarchies or external factors influencing the organization.

8.4. Lack of empirical foundation and evaluation

The maturity models found in this research often did not have any (documented) empirical foundation. Most often they are case-based or their development was not documented at all. Looking at the higher education models in detail, we found that 83 % of the models had either a case-based or groundwork foundation. The ISOLearn QL did empirical groundwork, while the CMM, MM-UniA, HE & FE AMM, and SCampus4A are case-based. Only for the IECM was there was no development process documented at all. These results are considerably higher than the empirical foundation of the models for other target areas, which report on an empirical foundation for only 37 % of the models. This corresponds to the critique of Biberoglu and Haddad (2002), Becker et al. (2010) and Becker et al. (2009a), who criticize the poor theoretical basis of maturity models and that it is often not documented how these maturity models have been developed.

The development process of maturity models is described as either a bottom-up or top-down approach in the literature. Mettler (2010a) describes a development framework with a bottom-up approach using metrics that are already available in the organization. On the other hand, Becker, Knackstedt and Pöppelbuß (2009a) and de Bruin et al. (2005) describe a development framework for a top-down approach, in which developers first define maturity levels and then define assessment items (Mettler, 2010b). To the best of our knowledge, none of the accessibility maturity models makes use of a similar development framework.

Also, it is unclear for 64 % of the maturity models found in the research whether they have been evaluated at all, as the validation of maturity models is rarely documented and described. For none of the models in this paper could documented results of an evaluation be found. A description of a conducted evaluation was made by 36 % of the models, but they did not document any results. Looking at the higher education model, only the ISOLearn QL documented having conducted an evaluation, whereas the other 83 % did not document any evaluation. This corresponds to results from the literature: Wendler (2012) analyzed the available literature of maturity model research and found that 52% of the analyzed articles only described the development process and the model itself but did not conduct any validation. Furthermore, only 34% of the articles describing a validation dealt with their own maturity model, which underlines the fact, that most maturity models are developed without validation (Wendler, 2012). Helgesson et al. (2012) analyzed the literature according to methods for evaluating maturity models. Likewise to Wendler (2012), only 25% of the articles described an evaluation of a self-developed maturity model. Additionally, they determined, that when evaluating self-developed maturity models, most studies were quite small due to the amount of effort required.

Two aspects are important when validating maturity models: the correctness of measurement of how mature an organization is (Maier et al., 2012), and the ability of the maturity model to support an organization in increasing the level of maturity (Mettler and Rohner, 2009). There are multiple challenges in evaluating these models, for example output that is not easily measurable or the time needed to implement changes in an organization. There are multiple articles that deal with the evaluation of maturity models (for example Frank, 2006; Mettler and Rohner, 2009; Helgesson et al., 2012). For the specific field of accessibility maturity models, we found no literature dealing with how to effectively evaluate these models, suggesting that this is an important area for further research.

This lack of documentation of the procedural steps in the development of the existing maturity models and the lack of results from validations make the traceability of the models very difficult. Future development of new maturity models or revisions of existing maturity models should focus on reproducible steps in their development process as well as the validation to facilitate traceability.

8.5. Research gap in higher education institutions maturity models

Although this research found 25 maturity models for accessibility, with six of them targeting higher education institutions, there was no model that focused on the topic of student learning and teaching of accessibility. Only three of the existing maturity models include the indicator “curriculum” as content at all, which targeted including lectures and know-how transfer about accessibility in degree programs, as well as providing accessible lectures. No maturity model was found that focuses explicitly on the learning and teaching of accessibility or on the accessibility of lectures and studies, instead they all focused on the whole organization of higher education institutions. A maturity model focusing on student learning and teaching of accessibility would move the focus to the lower levels of the organizational structure. With such a model, it would no longer be necessary to involve the entire organization and get support from the leadership, because study programs or even teachers could apply this model independently and analyze their maturity on their own.

9. Conclusion

This review analyzed maturity models for accessibility in order to investigate which accessibility maturity models for higher education institutions already exist, how they differ from accessibility maturity models for other target areas, and how these maturity models for accessibility measure the accessibility maturity of organizations in detail. Therefore, we conducted a systematic literature research in the Web of Science, IEEE XPlore, BASE, ACM, and Google Scholar databases to identify all maturity models that analyze the accessibility of organizations. The maturity models found were analyzed in more detail by looking at their target group, their structure, the empirical foundation, and their evaluation process and results. To investigate how the accessibility is measured by the maturity models, we categorized and analyzed their indicators used for measurement.

The results show that there are 25 existing maturity models that assess the accessibility of organizations. Only 6 of them are focused on higher education institutions. To measure accessibility, most maturity models include the indicators “responsibility”, “competences & training”, “measurement” and “resources”. However, the included indicators vary for the maturity models of different target groups in occurrence as well as in frequency.

We also found a lack of empirical foundation and evaluation of the existing maturity models for accessibility, as none of them had documented evaluation results and only a few had an empirical foundation. This correlates with criticism on maturity models from the literature. Another research gap was found in the absence of a maturity model for teaching and learning accessibility, as all maturity models for higher education institutions solely focused on the organization in general instead of concentrating on a specific part of higher education institution structure. There is a need for more specific maturity models with detailed documentation of the development and validation process and results.

The results of this research show that there is a wide variety of maturity models for accessibility available and that this can make it difficult to decide which maturity model is most suitable for a particular organization. By describing the individual maturity models and the indicators they contain for measuring accessibility, this paper can contribute to transparency and provide a basis for decision-making.

Author contributions

NA, SK, CS, A-KB, AG, FN, and GZ contributed to conception and design of the study. NA conducted the literature review and analysis. NA, SK, and HC wrote the first draft of the manuscript. SK, HC, CS, and A-KB did critical review and editing of the manuscript. VK and GZ had the project management. All authors contributed to manuscript revision, read, and approved the submitted version.

Funding

This research was carried out in the SHUFFLE (Higher education initiative: digital accessibility for all) research project, funded by the Stiftung Innovation in der Hochschullehre. Opinions of the authors do not necessarily reflect those of the funding agency.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The handling editor KS is currently organizing a Research Topic with the author GZ.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnotes

1. ^Note: In this chapter, acronyms are used to reference accessibility maturity models that were introduced in chapter 4. Please refer to Tables 1, 2 for the meaning of the acronyms.

2. ^Note: In this chapter, acronyms are used to reference the accessibility maturity models that were introduced in chapter 4. Please refer to Tables 1, 2 for the meaning of the acronyms.

3. ^Note: In this chapter, acronyms are used to reference the accessibility maturity models that were introduced in chapter 4. Please refer to Tables 1, 2 for the meaning of the acronyms.

References

AAPD Disability:IN (2022). Disability Equality Index^® (DEI^®): 8th Annual/2022 Benchmark. Available online at: http://www.disabilityequalityindex.orgDEISurvey/Annual_DEI_Questions.pdf (accessed February 17, 2023).

AbilityNet (n.d.) HE FE Accessibility Maturity Model. Available online at: https://abilitynet.org.uk/accessibility-services/he-and-fe-accessibility-maturity-model (accessed February 17 2023).

AbilityNet McNaught (2020). A digital Accessibility Maturity Model. Available online at: https://abilitynet.org.uk/sites/abilitynet.org.uk/files/HE-FE-Accessibility-Maturity-Model-27052020.docx (accessed February 17, 2023).

Ahlemann, F., Schröder, C., and Teuteberg, F. (2005). Kompetenz- und Reifegradmodelle für das Projektmanagement: Grundlagen, Vergleich und Einsatz. (ISPRI-Arbeitsbericht, 2005,01). Osnabrück: ISPRI.

Google Scholar

Ashton, D. P., and McElvane, P. A. (2019). “Seven Stages of Inclusion (7SI) Maturity Model”, in Diversity Business Review: A Practical Thought Leadership Journal. p. 6–11. Available online at: https://online.fliphtml5.com/eunc/oini/#p=1 (accessed February 17, 2023).

Google Scholar

Bailey, C., and Gkatzidou, V. (2017). “Considerations for Implementing a Holistic Organisational Approach to Accessibility”, in Proceedings of the 14th International Web for All Conference, Web For All 2017 - The Future of Accessible Work, Perth Western Australia Australia, 2017. New York: ACM, pp. 1–4. doi: 10.1145/3058555.3058571

CrossRef Full Text | Google Scholar

Ball, S., McNaught, A., Watson, S., and Chnadler, A. (2010). “OASES: Online Accessibility Self Evaluation Service – A Web-Based Tool for Education Institutions to Self-Assess the Accessibility of Their Practice,” in Computers Helping People with Special Needs. (Lecture Notes in Computer Science), Miesenberger, K. et al. (eds.). Berlin, Heidelberg: Springer Berlin Heidelberg. p. 433–436. doi: 10.1007/978-3-642-14097-6_69

CrossRef Full Text | Google Scholar

Baron, J. (n.d.) Accessibility Maturity Model: Measuring Your Team's Software Development Best Practices. Available online at: https://www.isoftstoneinc.com/wp-content/uploads/library/iSoftStone-Accessibility-Maturity-Model.pdf (accessed February 17, 2023),

Google Scholar

Becker, J., Knackstedt, R., and Pöppelbuß, J. (2009a). Developing Maturity Models for IT Management. Bus. Inf. Syst. Eng. 1, 213–222. doi: 10.1007/s12599-009-0044-5

CrossRef Full Text | Google Scholar

Becker, J., Knackstedt, R., and Pöppelbuß, J. (2009b). Entwicklung von Reifegradmodellen für das IT-Management. Wirtschaftsinformatik. 51, 249–260. doi: 10.1007/s11576-009-0167-9

CrossRef Full Text | Google Scholar

Becker, J., Niehava,B Poeppelbbuss, J., and Simons, A. (2010). “Maturity Models in IS Research,” ECIS 2010 Proceedings. Available online at: https://aisel.aisnet.org/ecis2010/42.

Google Scholar

Biberoglu, E., and Haddad, H. (2002). A Survey of Industrial Experiences with CMM and the Teaching of CMM Practices. Lincoln: Faculty Publications.

Google Scholar

Billing, D. (2004). International comparisons and trends in external quality assurance of higher education: Commonality or diversity? Higher Educ. 47, 113–137. doi: 10.1023/B:HIGH.0000009804.31230.5e

CrossRef Full Text | Google Scholar

Booth, T., and Ainscow, M. (2002). Index for Inclusion: Developing Learning and Participation in Schools. Bristol: CSIE.

Google Scholar

Burn, B. B., Cerych, L., and Smith, A. (1990) Study Abroad Programmes. London: J. Kingsley (Higher education policy. Series 11, 1).

Google Scholar

Business Disability Forum (2021a). AMM-Scorecard-2021, 17 February. Available online at: https://businessdisabilityforum.org.uk/knowledge-hub/resources/tech-taskforce-accessibility-maturity-model/ (accessed February 17, 2023).

Google Scholar

Business Disability Forum (2021b). Business Disability Forum- Technology Taskforce - Accessibility Maturity Model (AMM).

Google Scholar

Byrne-Haber, S., Fazio, D., Sajik, J., and Ran, R. (2022). W3C Accessibility Maturity Model, 30 November. Available online at: https://w3c.github.io/maturity-model/ (accessed February 17, 2023).

Google Scholar

Caforio, A. (2018). “Building quality assurance frameworks for accessibility in education through ICT,” in The Online, Open and Flexible Higher Education Conference 2018 - Proceedings: Blended and Online Learning “Changing the Educational Landscape”, Ubachs, G., and Joosten-Adriaanse, F. (eds.) p. 33–43.

Google Scholar

Caldwell, D. (2016). Best practice guide: How to use the Accessibility Maturity Model (AMM). Available online at: https://businessdisabilityforum.org.uk/knowledge-hub/resources/tech-taskforce-accessibility-maturity-model/ (accessed February 17, 2023).

Google Scholar

CSU ATI (n.d.) Implementation Overview. Available online at: https://ati.calstate.edu/policy/implementation (Accessed February 17, 2023).

Google Scholar

Da Silva, P., and Alturas, B. (2015). “Web accessibility: Study of maturity level of Portuguese institutions of higher education,” in 2015 10th Iberian Conference on Information Systems and Technologies (CISTI), 2015 10th Iberian Conference on Information Systems and Technologies (CISTI), Aveiro, Portugal: IEEE. p. 1–7. doi: 10.1109/CISTI.2015.7170414

PubMed Abstract | CrossRef Full Text | Google Scholar

Day, P. (2011). e-A11: Workshop B - Building Your Accessibility Action Plan. Available online at: https://www.slideshare.net/Headstar/ea11-workshop-b-building-your-accessibility-action-plan (accessed February 17, 2023).

Google Scholar

de Bruin, T., Freeze, R. D., Kulkarni, U., and Rosemann, M. (2005). “Understanding the Main Phases of Developing a Maturity Assessment Model,” in ACIS 2005 Proceedings. Available online at: https://aisel.aisnet.org/acis2005/109 (accessed June 14, 2023).

Google Scholar

Disability:IN (n.d.) Disability Equality Index. Available online at: https://disabilityin.org/what-we-do/disability-equality-index/ (accessed February 17, 2023)

Google Scholar

Evans, S., and Thurston, J. (2019). Smart Campus Digital Inclusion Maturity Model for Higher Education G3ict.

Google Scholar

Feistauer, A. (2021). Reifegradmodelle für Web-Accessibility in der digitalen Produktentwicklung. Bachelor Thesis. Hochschule der Medien. Available online at: https://hdms.bsz-bw.de/frontdoor/deliver/index/docId/6667/file/BA-final.pdf

Google Scholar

Frank, U. (2006). “Evaluation of Reference Models,” in Reference Modeling for Business Systems Analysis. Pennsylvania: IGI Global.

Google Scholar

G3ict (2019). Smart Cities for All: Inclusive Innovation Playbook. Available online at: https://smartcities4all.org/wp-content/uploads/2019/05/I2-Playbook-XT.pdf (accessed February 17, 2023).

Google Scholar

G3ict (n.d.) Measuring Digital Inclusion in Your City or University. Available online at: https://smartcities4all.org/maturity-model-mamo/ (accessed February 17 2023).

Google Scholar

German, E., and Hartnett, E. (2018). “Disability Inclusion and Library Collections: Initiatives for Greater Access for All,” in What's Past Is Prologue: Charleston Conference Proceedings 2017. Charleston: Purdue University Press. doi: 10.5703/1288284316671

CrossRef Full Text | Google Scholar

Goedegebuure, L. C. J. (1994). Higher Education Policy: An International Comparative Perspective. Oxford: Published for the IAU Press by Pergamon Press.

Google Scholar

Hähne, C., Marquardt, G., Rudolph, E., Schmidt, H. J., Weber, G., and Wegner, G. (2020). Inklusion benötigt verschiedene. Prozesse: Aktivitäten und Strategien an der TU Dresden.

Google Scholar

Hassell Inclusion Limited (2020). ISO 30071-1 Digital Accessibility Maturity Scorecard. Available online at: https://hassellinclusion.scoreapp.com/ (accessed February 17, 2023).

Google Scholar

Helgesson, Y. Y. L., Höst, M., and Weyns, K. (2012). A review of methods for evaluation of maturity models for process improvement. J. Software. 24, 436–454. doi: 10.1002/smr.560

CrossRef Full Text | Google Scholar

International Organization for Standardization (2017). Ergonomics of Human-System Interaction Berlin: Beuth Verlag GmbH.

Google Scholar

International Organization for Standardization (2019). Information Technology — Development of User Interface Accessibility — Part 1: Code of Practice for Creating Accessible ICT Products and Services. Berlin: Beuth Verlag GmbH.

Google Scholar

ISOLearn (2016). Innovation and Social Learning in Higher Education Institutions. Available online at: https://www.uninettunouniversity.net/Data/Sites/1/GalleryImages/ISOLearn/Final/ISOLearn_Final_Publication.pdf (accessed February 17, 2023).

Google Scholar

Kuykendall, H. (2016). “Policy-Driven Adoption for Accessibility (PDAA): Adding Vendor Governance Evaluation to State Procurement,” in Mealey's™ Litigation Report: Cyber Tech & E-Commerce: LexisNexis.

Google Scholar

Lay-Flurrie, J. (2020). Accessibility Evolution Model: Creating Clarity in your Accessibility Journey. Available online at: https://blogs.microsoft.com/accessibility/accessibility-evolution-model/ (accessed February 17, 2023).

Google Scholar

Level Access (2015a). The Digital Accessibility Maturity Model: Core Dimensions. Available online at: https://www.levelaccess.com/the-digital-accessibility-maturity-model-core-dimensions/ (accessed February 17, 2023).

Google Scholar

Level Access (2015b). The Digital Accessibility Maturity Model: DAMM Audit – Preparation. Available online at: https://www.levelaccess.com/digital-accessibility-maturity-model-damm-audit-preparation/ (accessed February 17, 2023).

Google Scholar

Level Access (2015c). The Digital Accessibility Maturity Model: Introduction to DAMM. Available online at: https://www.levelaccess.com/the-digital-accessibility-maturity-model-introduction-to-damm/ (accessed February 17, 2023).

Google Scholar

Lundy, J., Keast, R., Wharston, B., Omari, M., Teo, S., and Bently, T. (2021). Utilising a capability maturity model to leverage inclusion and diversity in public sector organisations. Aust. J. Public Admin. 80, 1032–1045. doi: 10.1111/1467-8500.12521

CrossRef Full Text | Google Scholar

Maier, A. M., Moultrie, J., and Clarkson, P. J. (2012). Assessing organizational capabilities: reviewing and guiding the development of maturity grids. IEEE Trans. Eng. Manag. 59, 138–159. doi: 10.1109/TEM.2010.2077289

CrossRef Full Text | Google Scholar

McNaught, A. (2021). How to Measure Accessibility Maturity and Compliance in Higher Education. Available online at: https://abilitynet.org.uk/news-blogs/how-measure-accessibility-maturity-and-compliance-higher-education (accessed February 17, 2023).

Google Scholar

Mettler, T. (2010a). Supply-Management im Krankenhaus: Konstruktion und Evaluation eines konfigurierbaren Reifegradmodells zur zielgerichteten Gestaltung. Zugl.: Sankt Gallen, Univ., Diss.

Google Scholar

Mettler, T. (2010b). “Thinking in Terms of Design Decisions When Developing Maturity Models,” International Journal of Strategic Decision Sciences, 1(4), 76–87. doi: 10.4018/jsds.2010100105

CrossRef Full Text | Google Scholar

Mettler, T., and Rohner, P. (2009). “Situational maturity models as instrumental artifacts for organizational design,” in Proceedings of the 4th International Conference on Design Science Research in Information Systems and Technology - DESRIST '09, the 4th International Conference, Philadelphia, Pennsylvania, 7-8 May. New York, New York, USA: ACM Press,. p. 1.

Google Scholar

Michel, L. (2020). Using an Accessibility Maturity Model to Facilitate the Inclusion of Accessibility in Design Practices. Linköping: Linköping University, Department of Computer and Information Science.

Google Scholar

Middendorff, E., Apolinarski, B., Becker, K., Bornkessel, P., Brandt, T., Heißenberg, S., et al. (2017). Die wirtschaftliche und soziale Lage der Studierenden in Deutschland 2016.: Zusammenfassung zur 21. Sozialerhebung des Deutschen Studentenwerks durchgeführt vom Deutschen Zentrum für Hochschul- und Wissenschaftsforschung. Berlin: Bundesministerium für Bildung und Forschung (BMBF).

Google Scholar

Pickard, B. (2020). Challenging Deficit-Based Discourse in Higher Education Through a Social Connection Model of Responsibility: A Critical Disability Studies Perspective. Dissertation. Available online at: https://www.proquest.com/openview/4b57b1891ad4f21d558c01594616391b/1?pq-origsite=gscholar&cbl=2026366&diss=y (accessed June 14, 2023).

Google Scholar

Poskowsky, J., Fromme, C., Heißenberg, S., Zaussinger, S., and Brenner, J. (2018). Beeinträchtigt studieren - best2: Datenerhebung zur Situation Studierender mit Behinderung und chronischer Krankheit 2016/17. Berlin: Deutsches Studentenwerk.

Google Scholar

Quintal, C., and Macías, J. A. (2018). “A Capability Maturity Proposal for Usability and Accessibility Centered Processes,” in Proceedings of the XIX International Conference on Human Computer Interaction, Interacción 2018: XIX International Conference on Human Computer Interaction, Palma Spain, 2018. New York, USA: ACM, pp. 1–8. doi: 10.1145/3233824.3233836

CrossRef Full Text | Google Scholar

Quintal, C., and Macías, J. A. (2021). Measuring and improving the quality of development processes based on usability and accessibility. Universal Access in the Information Soc., 20, 203–221. doi: 10.1007/s10209-020-00726-7

CrossRef Full Text | Google Scholar

Romansky, L., Garrod, M., Brown, K., and Keo, D. (2021). How to Measure Inclusion in the Workplace. Available online at: https://hbr.org/2021/05/how-to-measure-inclusion-in-the-workplace (accessed February 17, 2023).

Google Scholar

Software Engineering Institute (2009). Brief History of CMMI. Available online at: https://resources.sei.cmu.edu/asset_files/Brochure/2009_015_001_28416.pdf (accessed February 17, 2023).

Google Scholar

Teichler, U. (1996). Comparative higher education: potentials and limits. Higher Educ. 32, 431–465. doi: 10.1007/BF00133257

CrossRef Full Text | Google Scholar

Threeway (n.d.) Reifegradmodell. Available online at: https://tools.3way.ch/de/Barrierefreiheit/Reifegradmodell (accessed February 17, 2023).

Google Scholar

Tocto-Cano, E., Cheung, W-H., Chow, S. K., Ip, M., Leung, S. Y. S., and Wong, R. M. Y. (2020). A systematic review of the application of maturity models in universities. Information. 11, 466. doi: 10.3390/info11100466

CrossRef Full Text | Google Scholar

United Nations (2006). Convention on the Rights of Persons with Disabilities and Optional Protocol. Available online at: https://www.un.org/disabilities/documents/convention/convoptprot-e.pdf (accessed February 17, 2023).

Google Scholar

Università Telematica Internazionale UNINETTUN (2016). ISOLearn: Innovation and Social Learning for HEI. Available online at: https://www.isolearn.net/en/isolearn.aspx (accessed February 17, 2023).

Google Scholar

University of Arizona (n.d.) Maturity Model. Available online at: https://itaccessibility.arizona.edu/about/maturity-model (accessed February 17 2023).

Google Scholar

Vermeersch, M. (2018). Best Practice Guide – Accessibility Maturity Model (AMM) – an in-depth questionnaire. Available online at: https://businessdisabilityforum.org.uk/knowledge-hub/resources/amm-questionnaire/ (accessed February 17, 2023).

Google Scholar

Ward, M. (2015). Accessibility in IT Procurement: Part 2: The PDAA Components. Available online at: https://www.nascio.org/wp-content/uploads/2019/11/NASCIO_-Accessability_In_IT_Procurment_Part_2a.pdf (accessed February 17, 2023).

Google Scholar

Weber, C. V., Garcia-Miller, S., Chrissis, M., and Bush, M. (1991). Key Practices of the Capability Maturity Model.

Google Scholar

Wendler, R. (2012). The maturity of maturity model research: a systematic mapping study. Info. Software Technol. 54, 1317–1339. doi: 10.1016/j.infsof.2012.07.007

CrossRef Full Text | Google Scholar

Williams, D. A., Berger, J. B., and McClendon, S. (2005). Toward a Model of Inclusive Excellence and Change in Post–Secondary Institutions. Washington, D.C.: Association of American Colleges and Universities.

Google Scholar