Why a “whack-a-mole” approach to widening participation won’t work - and what to do about it: a policy brief

Despite increasing access to higher education (HE) over the past two decades, students from disadvantaged backgrounds remain less likely to enter the most selective institutions, and less likely to achieve good outcomes. In England, providers that charge more than £6,000 a year tuition fees must have an Access and Participation Plan (APP) approved by the Office for Students (OfS). Plans aim to improve equality in student access, success, and progression. APPs typically focus on snapshots of equality gaps, inputs over outcomes, and individual institutions over collaborative efforts. These limitations encourage a “whack-a-mole” approach, reducing effectiveness of interventions. A more nuanced approach to evaluation is needed, together with increased and organized sector-wide collaboration, and acknowledgment of the dynamic operating context.

Introduction

Social mobility differentiation and outcomes in higher education (HE)

In England, the entry rate into HE has been increasing (of 15-year-olds in 2001/2, 33.6% entered HE by age 20 years vs. 47.1% in 2016/17: ONS, 2023). HE is no longer the preserve of the elite; the proportion of people attending from some of the most disadvantaged groups (lowest quintile of the Index of Multiple Deprivation) is now higher than those from the least disadvantaged (OfS, 2024a). Around 18% of students had been eligible for free school meals (FSM), compared to a baseline of 22% in the general population (OfS, 2024a). However, bigger disparities remain; in 2023 the access gap to high tariff providers (the most academically selective) between FSM eligible pupils and others reached its highest recorded level of 9.3 percentage points (Department for Education, 2023).

Policy context: access and participation plans (APPs)

Higher Education Institutions (HEIs) in England that charge more than the basic tuition fee cap (£6,000) must have an APP approved by the Office for Students (OfS, 2020). These plans represent commitments to actively improve equality of opportunity for students in terms of access, success, and progression to employment or further study. Plan durations vary, but most recently are expected to cover a 3–4-year period (OfS, 2023). Providers are encouraged to look at their own context, using supporting data (including that helpfully provided by the OfS), and evaluate their own performance on a range of equality of opportunity measures. Measures cover access, success and progression split by characteristics including disability, sex and measures of financial disadvantage. Where risks to equality of opportunity (typically equating to gaps between different groups) are identified, providers must explain how they will prioritize and address them. In a separate document, they indicate estimated spend on each set of activities proposed.

Policy challenges and considerations

Despite the changed context of access and the scope of APPs (or their predecessor access agreements) changing over time to place a greater emphasis on student success, and on the use of data, their requirements have changed relatively little. The APP development and approval process arguably suffers from several shortcomings in terms of its potential to meet its long term aims.

Failure to address a changing and complex landscape

Access and participation plan monitoring returns collect limited evidence of evaluation of impact and APP approval encourages evidence that suggests an intervention is likely to be effective. However, each new APP is not required to set out the time and resources that have gone into achieving the status quo, nor to consider potential future contexts such as rapidly increasing inflation. Indeed, the OfS may require providers to justify targets that focus on maintenance of progress, rather than addressing current gaps. For some providers this means they have no targets for some aspects of the APP, potentially resulting in the risk of cessation of the activity that has achieved their positive position. Like the fairground game of “whack-a-mole” (where the artificial “mole” pops up from one of several holes and needs to be hit back down with a rubber hammer - just to “reappear” in a different hole), this is ultimately inefficient, unproductive, and unlikely to lead to systemic change. It is also counter to the welcome new focus at the OfS of establishing “what works.” Alternatively, providers may continue doing what they are currently doing and then either overstretch themselves to do more or allocate too few resources to each intervention to make a meaningful impact. This is a particular risk whilst the real term value of tuition fee income is decreasing and a considerable number of providers are in financial crisis (OfS, 2024b).

As evidenced above, at a macro level access to HE for some disadvantaged groups now appears to be less of an issue than it has been previously. However, Crawford and van den Erve (2015) argued that it was not enough to simply encourage more people to go to university, but that what happens to them while they study and afterward needed more consideration. Indeed, those from disadvantaged backgrounds are less likely to continue, complete, attain a “good” degree, and progress to a graduate level destination (OfS, 2024a). Whilst this is recognized, the current approval process does not account for the interplay between access and success. A contributing factor is that disparities remain in access to the most selective providers, and to the most selective courses, including medicine (Department for Education, 2023; Medical Schools Council, 2023). However, intervention strategies can attend to more than one risk, and some providers do approach the issue holistically.

OfS expectations imply continuous and gradual closing of equality gaps (e.g., OfS, 2018a), but ever-increasing resources are required just to prevent them from increasing. For example, whilst financial support has been shown to be effective at improving continuation rates for students from low-income backgrounds (Moores and Burgess, 2022), awarding it requires repeated and significant annual investment from a tuition fee which is losing value in real terms. Moreover, institutions with large numbers of students from low-income backgrounds will have to spread available resources more thinly, resulting in smaller amounts of support per student (Murphy and Wyness, 2016; Wyness, 2016). Simultaneously, the real-terms value of any support awarded has been rapidly decreasing for students. In institutions where such financial support has been shown to be effective, gaps between students from disadvantaged backgrounds and their peers may now be reduced and therefore not appear a priority in analyses of gaps (see e.g., Moores and Burgess, 2022). This does not mean support is no longer needed and, in fact, the opposite is more likely to be true. Many of our “moles” need repeated and increasingly forceful “whacking.”

Although some gaps in student success are undeniably increased during HE, lack of progress in addressing gaps during compulsory education (pre-HE) may also have an impact on what HEIs are able to achieve. Some students experience structural disadvantages impacted by factors such as poverty and discrimination that APPs alone cannot reasonably be expected to address. The Sutton Trust (2024, p2) notes that ”The attainment gap first opens up before children even start at school, leaving lower income pupils behind…” and that since the pandemic, “the gap has widened considerably, with 10 years of progress now wiped out” (2024, p1). It therefore seems likely that HEIs will find it challenging to close gaps in the upcoming years unless this context is explicitly accounted for.

Short-term approaches and unsustainable practices

Notwithstanding the OfS (2022a) request to providers to submit a variation to APPs mid-cycle to respond to new priorities, the shift from an annual to a longer cycle for approval has been welcome. This has reduced burden on providers and encouraged longer-term thinking. However, the 3–4-year timescale is still relatively short and may discourage providers from addressing underlying issues, instead risking tokenistic practices. An example could include providing additional tutor support to close attainment gaps rather focusing on overall teaching and assessment across the institution. Moreover, for many interventions it may take a longer timescale to reveal their full impact. The most recent APP guidance (OfS, 2023) does suggest that strategies can include existing activities but refreshing interventions for new submissions does not align well with the “whole provider approach” that is ostensibly encouraged (OfS, 2021), instead further encouraging siloed and short-term approaches. In addition, from 2019/20 access and participation plans emphasized outcomes and impact over input (OfS, 2018b). This afforded opportunities to providers to deliver impact with less investment (and should have eliminated any need for performative spend), but also risked allocation of resources with little impact. Whilst evaluation has become an increasingly key requirement, the intention to measure providers on impact has not yet been fully realized and the OfS still requires cost estimates per intervention.

Insufficient sector-wide collaboration

The structure of the regulatory approach is focused on individual HEIs, whilst impact of access-based interventions may often be wider than this. Individual providers are accountable for their own performance, with some, albeit limited, reference to their geographical or disciplinary context. The regulatory guidance stresses the importance of collaboration, noting that providers should, “where appropriate” consider agreeing an intervention strategy “with other providers and third sector organizations” (OfS, 2023, p13). It seems likely that this guidance was intended principally for access, rather than student outcomes. The current system of non-collaborative approaches toward access, particularly when coupled with the increasing competition for students between providers, encourages conflation of outreach and marketing activities and risks the underestimation of the importance of good information, advice, and guidance (Summers et al., 2024b). This is particularly problematic in areas that are well served with HE provision. Collaborative approaches tend to be relatively limited and risk maintaining institutional silos and in 2024, the OfS further and significantly reduced funding for its collaborative access programme, Uniconnect (Times Higher Education, 2024).

From an evaluation perspective too, there are corollaries of this siloed approach. The underpinning logic of evaluation as described in the regulatory advice is to contribute to the “development of sector-wide knowledge of what works, for whom, and in what contexts” assumes that the current evaluation guidance produces generalizable knowledge. While evaluation is carried out in specific contexts, this is unlikely to be the case (see also Moores et al., 2023). The process of translating individual outcomes between institutional contexts is more complex than the guidance currently acknowledges, primarily because of the contextual heterogeneity of the sector. It also assumes that sufficient opportunities for sharing this knowledge exist. Currently, the two “official” conduits for evidence about what works are the OfS and TASO, the sector “what works” center. In the case of TASO, there has been historical bias toward counterfactual experimental designs (Type 3 causal) and continued conclusions of “no causal impact” in published reports. As Cartwright (2013) argues, these designs are more effective at identifying “it works somewhere” but do not provide “it-will-work-for-us” claims. Progress is therefore likely to be more rapid and allow for more meaningful and generalizable outcomes through collaboration.

Resourcing challenges

Since 2011, schools and colleges delivering compulsory education have received “pupil premium” funding for pupils with particular characteristics of disadvantage (largely based on numbers of pupils eligible for FSM). The aim of the funding is to improve attainment of disadvantaged students, and providers decide how to spend it. Gorard et al. (2021) suggested that overall, it has been a system that has worked and should be retained. In HE, the nearest equivalent is student premium funding, which is awarded based on risk categories (based on age, aimed for qualification, and non-retention associated with entry qualifications: OfS, 2022b). A supplementary element also considers the extent to which the risk categories intersect with the students coming from underrepresented areas, and there is a premium for disabled students. Moss (2023) calculated that in 2023/4, whilst around 5% of compulsory education budget was pupil premium and therefore ear-marked to reduce inequality, student premium funding in HEIs represented only 1.5% of teaching related income. Moreover, any additional uplift in income received directly from tuition fees from having an APP is provided by all students and is not associated with the number needing additional support. This means that providers with high numbers of disadvantaged students have more to do with the same proportion of income as those with less diverse cohorts. It may also mean that interventions are accessible universally, rather than targeted to areas of need - ensuring funding stretches across more than one priority group, or to all students.

Actionable recommendations

Introduce nuanced evaluations of current position

A nuanced approach to evaluation of provider access, success and progression is needed, that acknowledges the interplay between them. The Higher Education Policy Institute annually publishes a social mobility index for England (e.g., HEPI, 2023) which could be helpful and is available at both provider and course level (see also Britton et al., 2021). The index provides a weighted combination of measures of access, continuation, and graduate outcomes. However, it should be noted that this measure has been criticized for the inclusion of salary, because it fails to consider geographical context (providers in London feature heavily amongst the top ranked).

A snapshot analysis of a provider’s current position in terms of access, success and progression needs to be accepted as a progress point, with historical, current, and prospective contexts, rather than a starting point. For access and progression, it would be helpful to benchmark performance by region, using similar regions for comparison rather than only using gaps. For student success, it would be helpful to benchmark performance with providers whose intake is similar. Overall performance should be considered as well as gaps in performance to avoid incentives to decrease positive outcomes for some groups in order to reduce gaps, as opposed to improving performance for the targeted group. Providers may be performing well above benchmarks for disadvantaged groups, yet still have gaps, or not have gaps but show poor performance overall. Particularly if a whole institution approach is employed, institutions may be successful in raising the performance of all groups (which should not be discouraged). Additionally, some HEIs are almost entirely populated by students with at least one characteristic of disadvantage, making a gap approach problematic.

Foster whole-institution approaches

Rather than requiring detailed cost estimates for individual interventions, whole-institution strategies [as first recommended by Thomas (2017)] that are embedded and sustainable should be encouraged and recognized. Currently many of such strategies will not be explicitly included in the APP, nor necessarily evaluated and disseminated as good practice if successful (often because of the complex nature of evaluating them). These strategies should align with broader university objectives and be evaluated over longer cycles to measure true impact. One potential challenge of this approach is measuring progress and maintaining accountability, although arguably this is also a challenge under the current system (see OfS, 2022c for monitoring and outcome data). An advantage of a whole provider approach is that it potentially avoids the implication of a deficit associated with students targeted by an “intervention,” which can risk stigmatization and implies the existence of a problem that needs to be fixed. Instead, a structural model (where the institution assumes responsibility for gaps in outcomes e.g., the “attainment gap” change in language to the “awarding gap”) and inclusive approaches (embedded practice for all) is implemented. We have argued elsewhere that the “everyday” (such as embedding inclusive approaches to teaching and assessment) - difficult to label as an “intervention” in APPs - is likely to have the most significant impact on equality gaps (Moores and Summers, 2023). This could include things such as mode of delivery (e.g., Summers et al., 2024a; Summers et al., 2023) or attendance policies (see e.g., Moores et al., 2019).

Promote sector-wide collaboration

Access and participation work should be done collaboratively across the sector with institutions not measured purely on their own performance intake, but rather the performance uptake of a geographical area near to - or assigned to - them. At least in areas with good choice of provision, providers should be able to concentrate on what they are best at, e.g., some institutions might specialize in supporting mature students, so that they may contribute optimally to the issues at hand nationally. This kind of collaborative approach requires a top-down point of organization to coordinate, a suitable shared database, and staff to evaluate impact of each institution (or indeed the cumulative effects of impact from different institutions). In September 2024, the OfS announced a collaborative funding competition to improve equality of opportunity to encourage different providers as well as charities and third sector organizations to collaborate (OfS, 2024c). HEIs should also collaborate on evaluation of interventions. To produce useful recommendations sector-wide, we need to test the same interventions in multiple similar contexts to ensure that they are generalizable across contexts, or to understand the circumstances in which they will and will not work.

Address resourcing challenges

For sector-wide progress to be made on reducing gaps, the way in which these activities are funded needs consideration. More specifically, to be allocated according to actual disadvantage (e.g., based on previous FSM eligibility or household income), rather than entry qualifications. The fact that providers with large proportions of students from disadvantaged backgrounds have their resources stretched more thinly is particularly problematic, because they are in the best position to make the biggest impact on gaps nationally. At the same time, there is a need for the regulator to accept that provider plans may reflect less a lack of ambition and more a lack of resources. Targets set should first and foremost be realistic and justifiable based on impact of previous evaluations of interventions. This will help to avoid targets that can sometimes be a “triumph of hope over experience,” guided by the requirement to “show sufficient ambition.”

Conclusion

The recommendations outlined above suggest a significant shift in approach, requiring more nuanced evaluations, including recognition of context, embedded strategies, revisiting student premium funding and greater collaboration across the sector. While these changes arguably require substantial effort, many of the processes already exist, particularly in relation to the Teaching Excellence Framework (TEF). Merging APP and TEF requirements (perhaps also in conjunction with regulation of some conditions), could create a more streamlined and effective approach, ensuring that providers continue to make meaningful progress in promoting equality of opportunity for all students. Many of the data requirements are similar. Scotland has a similar system of “Outcome Agreements” (becoming Outcomes Framework and Assurance Model in 2425), which sits alongside their quality assurance and enhancement processes, although there is perhaps less regulatory emphasis on evaluation. However, meaningful differences between the APP and the TEF include that the TEF does not require estimates of investment, and that it is mainly retrospective. Nevertheless, institutions increasingly take a forward-looking approach to the TEF preparations, particularly in terms of evaluation of impact, so similarities are present. Whilst we have discussed above that estimating spend in a whole institution approach is difficult, it does ensure commitment of investment in this important area.

Author contributions

EM: Conceptualization, Writing – original draft, Writing – review and editing. JC: Writing – original draft, Writing – review and editing. LA: Writing – review and editing. LW: Writing – review and editing.

Funding

The author(s) declare financial support was received for the research, authorship, and/or publication of this article. The publication costs for this article were kindly funded by the College of Health and Life Sciences, Aston University Teaching Research Fund.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Britton, J., Drayton, E., and van den Erve, L. (2021). Which university degrees are best for intergenerational mobility? Research report November 2021. London: Institute for Fiscal Studies.

Google Scholar

Cartwright, N. (2013). Knowing what we are talking about: Why evidence doesn’t always travel. Evid. Policy 9, 97–112.

Google Scholar

Crawford, C., and van den Erve, L. (2015). Does higher education level the playing field? Socio-economic differences in graduate earnings. Educ. Sci. 5, 380–412. doi: 10.3390/educsci5040380

Crossref Full Text | Google Scholar

Department for Education (2023). Widening participation in higher education. London: Department for Education.

Google Scholar

Gorard, S., Siddiqui, N., and See, B. H. (2021). Assessing the impact of Pupil Premium funding on primary school segregation and attainment. Res. Papers Educ. 37, 992–1019. doi: 10.1080/02671522.2021.1907775

Crossref Full Text | Google Scholar

HEPI (2023). 2023 English social mobility index. Oxford: HEPI.

Google Scholar

Medical Schools Council (2023). MSC selection alliance annual report 2023: An update on the Medical Schools Council’s work in selection and widening participation. London: Medical Schools Council.

Google Scholar

Moores, E., and Burgess, A. P. (2022). Financial support differentially aids retention of students from households with lower incomes: A UK case study. Stud. High. Educ. 48, 220–231. doi: 10.1080/03075079.2022.2125950

Crossref Full Text | Google Scholar

Moores, E., and Summers, R. J. (2023). Evaluating the everyday. TASO blog. Available online at: https://taso.org.uk/news-item/evaluating-the-everyday/ (accessed July 22, 2024).

Google Scholar

Moores, E., Birdi, G. K., and Higson, H. E. (2019). Determinants of university students’ attendance. Educ. Res. 61, 371–387.

Google Scholar

Moores, E., Summers, R. J., Horton, M., Woodfield, L., Austen, L., and Crockford, J. (2023). Evaluation of access and participation plans: Understanding what works. Front. Educ. 8:1002934. doi: 10.3389/feduc.2023.1002934

Crossref Full Text | Google Scholar

Moss, A. C. (2023). The disconnect between quality and inequality: An analysis of the gaps in educational outcomes achieved by free school meal-eligible students in English higher education. HEPI Policy Note 46. Available online at: https://www.hepi.ac.uk/wp-content/uploads/2023/06/The-disconnect-between-quality-and-inequality.pdf (accessed August 28, 2024).

Google Scholar

Murphy, R., and Wyness, G. (2016). Testing means-tested aid (October 5, 2016). CESifo working paper series No. 6105. Munich: CESifo, doi: 10.2139/ssrn.2866923

Crossref Full Text | Google Scholar

OfS (2018a). Eliminate equality gaps within 20 years - office for students. Available online at: https://www.officeforstudents.org.uk/news-blog-and-events/press-and-media/eliminate-equality-gaps-within-20-years/ (accessed July 22, 2024).

Google Scholar

OfS (2018b). Access and participation review consultation (officeforstudents.org.uk). Available online at: https://www.officeforstudents.org.uk/media/6ad50390-eb57-4230-923a-0e0d53d672ac/access-and-participation-consultation-event-slides.pdf (accessed July 22, 2024).

Google Scholar

OfS (2020). Fees and other costs - office for students. Available online at: https://www.officeforstudents.org.uk/for-students/student-finance/value-for-money-as-a-student/fees-and-other-costs/ (accessed July 22, 2024).

Google Scholar

OfS (2021). How to evaluate and implement a whole-provider approach – Office for Students. Available online at: https://www.officeforstudents.org.uk/for-providers/equality-of-opportunity/evaluation/whole-provider-approach/ (accessed July 22, 2024).

Google Scholar

OfS (2022a). Advice on requests to vary access and participation plans to cover academic year 2023–24. Available online at: https://www.officeforstudents.org.uk/publications/advice-on-requests-to-vary-access-and-participation-plans-2023-24/. (accessed October 18, 2024).

Google Scholar

OfS (2022b). Guide to funding 2022-23. Available online at: https://www.officeforstudents.org.uk/media/7787/guide-to-funding_2022-23.pdf (accessed August 28, 2024).

Google Scholar

OfS (2022c). Access and participation plan data; monitoring data and outcomes 2020-21. Available online at: https://www.officeforstudents.org.uk/data-and-analysis/access-and-participation-plan-data/monitoring-data-and-outcomes-2020-21/ (accessed September 27, 2024).

Google Scholar

OfS (2023). Regulatory advice 6: How to prepare an access and participation plan: Supplementary access and participation plan guidance. Available online at: https://www.officeforstudents.org.uk/media/8973/regulatory_advice_6_how_to_prepare_an_access_and_participation_plan_dec2023.pdf (accessed August 27, 2024).

Google Scholar

OfS (2024a). Access and participation data dashboard. Available online at: https://www.officeforstudents.org.uk/data-and-analysis/access-and-participation-data-dashboard/data-dashboard/ (accessed September 01, 2024).

Google Scholar

OfS (2024b). Financial sustainability of higher education providers in England: 2024. Available online at: https://www.officeforstudents.org.uk/publications/financial-sustainability-of-higher-education-providers-in-england-2024/ (accessed August 29, 2024).

Google Scholar

OfS (2024c). Equality in higher education innovation fund – guidance for bidders. Available online at: https://www.officeforstudents.org.uk/publications/equality-in-higher-education-innovation-fund-guidance-for-bidders/ (accessed September 03, 2024).

Google Scholar

ONS (2023). Participation measures in higher education, Academic year 2021/22 - Explore education statistics – GOV.UK (explore-education-statistics.service.gov.uk). Available online at: https://explore-education-statistics.service.gov.uk/find-statistics/participation-measures-in-higher-education/data-guidance (accessed July 22, 2024).

Google Scholar

Summers, R. J., Burgess, A. P., Higson, H. E., and Moores, E. (2024a). How you teach and who you teach both matter: Lessons from learning analytics data. Stud. High. Educ. 3, 576–591.

Google Scholar

Summers, R. J., Fullwood, S., Sherlock, R., and Moores, E. (2024b). The importance of information, advice and guidance in widening access to higher education. Widen. Participat. Lifelong Learn. 26, 161–181. doi: 10.5456/WPLL.26.2.161

Crossref Full Text | Google Scholar

Summers, R. J., Higson, H. E., and Moores, E. (2023). The impact of disadvantage on higher education engagement during different delivery modes: A pre- vs. Peri-pandemic comparison of learning analytics data. Assess. Eval. High. Educ. 48, 56–66.

Google Scholar

The Sutton Trust (2024). General election policy briefing: Closing the attainment gap. Available online at: https://www.suttontrust.com/wp-content/uploads/2024/02/Closing-the-attainment-gap.pdf (accessed August 27, 2024).

Google Scholar

Thomas, L. (2017). A whole institution approach to widening participation: A summary report and recommendations. Available online at: https://www.officeforstudents.org.uk/media/3a07b57f-385d-4503-b441-a4331949f694/whole-institution-approach-to-wp-summary-report.pdf (accessed August 23, 2024).

Google Scholar

Times Higher Education (2024). University access scheme loses third of its budget in latest cuts. Available online at: https://www.timeshighereducation.com/news/university-access-scheme-loses-third-its-budget-latest-cuts (accessed August 30, 2024).

Google Scholar

Wyness, G. (2016). Deserving poor: Are higher education bursaries going to the right students? Educ. Sci. 6:5. doi: 10.3390/educsci6010005

Crossref Full Text | Google Scholar