A lack of meaningful human control for automated vehicles: pressing issues for deployment and regulation

The introduction of automated driving systems (ADS) presents significant regulatory and operational challenges to ensure safe and responsible deployment in mixed traffic environments. Despite much academic work and efforts of practitioners, these challenges remain open, requiring a transdisciplinary integration of perspectives. This paper draws on insights from a recent transdisciplinary workshop, highlighting the key issues in ADS deployment, including misalignment between regulations and system capabilities, emerging accident types, and gaps in driver understanding and training. Current regulations struggle to keep pace with the advancing capabilities of ADS, resulting in unclear accountability frameworks and inadequate safety measures. The concept of meaningful human control was used as a basis to identify issues. Workshop participants agreed that meaningful human control has an essential role to play to address the identified issues by ensuring that humans can adequately interact with ADS and that ADS are designed in a manner that ensures safe and responsible deployment with clear fail-safes and redundancy mechanisms. The paper advocates for meaningful human control through continuous driver and vehicle assessment, dynamic safety certifications, and stronger communication between regulators and manufacturers to ensure safe and responsible design, regulation and deployment of automated vehicles. Implementing these actions will strengthen ADS regulation and help navigate the ethical and operational complexities of automated driving systems.

1 Introduction

As automated driving systems (ADS) continue to evolve and become more integrated into road vehicle, the concept of meaningful human control (MHC) has emerged as a key principle in ensuring that these systems operate ethically, safely, and responsible. MHC refers to the idea that even when systems perform complex tasks autonomously, humans should retain the ability to influence or intervene in a way that ensures accountability and aligns the system’s actions with human intentions and values (Santoni de Sio and Van den Hoven, 2018). In the context of ADS, this means that human drivers, regulators, and other stakeholders must maintain meaningful control over vehicles, even if the systems operate autonomously (Calvert et al., 2024).

As vehicles take over more driving tasks, the role of the human driver transitions from an active operator to a passive supervisor. This shift presents several challenges, including the risk of overtrust in the automation (Dikmen and Burns, 2017), deskilling of drivers (Hopkins and Schwanen, 2021), and ambiguity in accountability during critical situations (Santoni de Sio and Mecacci, 2021). While regulations clearly state that a human driver remains responsible for the driving task at lower levels of automation (e.g., SAE Level 2), accountability becomes less clear as automation levels increases. The primary reasons for this are the lack of human drivers’ ability (Seppelt and Lee, 2019) and unclear control authority (Flemisch et al., 2012). For example, SAE Level 3 systems allow for conditional automation, meaning the vehicle drives autonomously under specific conditions if supervised by a human. However, a human driver is expected to remain vigilant and be ready to take over the control at all times, which has been shown to be an unrealistic demand to place on a human (Louw et al., 2015). Moreover, there are gaps in drivers’ knowledge of how ADS operate and drivers are not always aware of their responsibilities, leading to potential safety risks (Nordhoff et al., 2023). Therefore, while MHC offers a concept that can help design, develop, and evaluate automated vehicles (AVs) in a responsible manner, current practice on the road, in regulations, and in vehicle design falls short, which has already led to accidents and continues to produce unsafe situations. This highlights the need to bridge the divide between rigorous, theory-driven academic research and practical applications, which often lack grounding in formal principles.

This paper contributes by addressing this gap by drawing on insights from a recent transdisciplinary workshop involving diverse stakeholders from the ADS ecosystem, including industry, regulators, researchers, and drivers. The workshop focused on identifying critical challenges surrounding MHC in ADS, and discussing what is required to allow the concept of MHC to be brought to practice in automated driving. This paper systematizes the insights discussed in the workshop; by highlighting the challenges and proposing concrete actions to address those, the paper aims to contribute to the ongoing conversation about the future of ADS to help ensure that human control remains meaningful as vehicles become increasingly more automated. The rest of the contribution includes a short overview of the workshop setup (Section 2), a structured presentation of the main insights and findings of the workshop (Section 3), and a discussion of these findings in the context of meaningful human control (Sections 4, 5).

2 Workshop setup and participants

The workshop brought together 11 invited expert participants from a variety of organisations, including academia, applied research institutions, industry, and governmental agencies, all with a vested interest in ADS. The event was structured to encourage active participation and reflection on where stakeholders could exchange knowledge and identify the challenges and opportunities for implementing MHC in ADS, while also considering its relevance to their organizations and ongoing activities.

The workshop began with a presentation introducing the concept of MHC and its significance in ensuring how ADS can operate under ethical and accountable human control. This introduction set the stage for participants to reflect on how MHC impacts their work and the role their organizations could play in addressing the emerging issues surrounding AVs. Participants were then organized into smaller discussion groups, where they engaged in focused conversations about specific areas where MHC lacks. These discussions explored both the practical challenges of implementing MHC and the urgency of resolving these issues to ensure safe deployment of ADS. Each group was tasked with brainstorming ideas on how their respective organisations could contribute to enhancing MHC in ADS, while also considering the broader societal challenges. Later, the discussions shifted toward identifying research gaps and pilot project opportunities that could help advance MHC in real-world applications. Participants shared ideas on areas that need further exploration and what types of pilot initiatives could be launched to test and improve the integration of MHC into ADS. The focus was on developing actionable approaches that could be implemented across organisations and sectors to ensure a responsible approach to the increasing automation in driving systems. Throughout the workshop, detailed notes were taken to capture various insights and contributions, which were later summarized to provide a comprehensive overview of key challenges and proposed actions. This paper draws on these insights to offer recommendations for addressing the pressing issues in MHC for ADS, grounded in the practical experiences and perspectives shared by the workshop’s diverse group of stakeholders.

3 Findings

We structured the main findings from the workshop into two areas aligning with the main issues and main actions that arose from the discussions. On all of the points that were made, there was a general consensus, while prioritisation was not explicitly addressed. We note that the statements given below are the result of discussions of multiple stakeholders and do not necessarily align with the opinions of the authors.

4 Main issues

4.1 Deficiencies in drivers’ control over ADS at SAE L2-3

One of the most prominent issues discussed was that human drivers’ control over low and conditionally automated vehicles (SAE L2-3) is often not meaningful: the participants highlighted glaring mismatches between the demands of the task, the abilities of the drivers, and their understanding of the system. In particular, at SAE L2-3, human drivers often misunderstand their role to monitor vehicle performance, leading to over-reliance on automation. While Level 2 systems require drivers to remain fully engaged in the driving task, many drivers overtrust the system’s capabilities, assuming it can handle more than it is actually capable of. Level 3 systems exacerbate this, where the system can take control under certain conditions, but the driver is still expected to intervene if necessary — a demand that participants agreed is unrealistic and potentially dangerous. Moreover, it was highlighted that in many cases drivers are unaware of the vehicles’ capabilities and in some case are not even aware of which mode the vehicle is in (mode confusion). The deficiencies highlighted by the participants are in line with the studies reporting similar issues for real-world ADS such as Tesla Full-Self-Driving Beta (Suryana et al., 2024).

4.2 Changes in accident types and vehicle interactions

New types of accidents involving AVs are expected to occur. Unlike traditional vehicles, AVs exhibit different driving dynamics, which can lead to accident scenarios that were previously uncommon or non-existent. These new dynamics, such as the precision with which AVs follow traffic rules or their more cautious behaviour, often create unexpected interactions with human drivers who are accustomed to more flexible human driving patterns. This unfamiliarity increases the likelihood of accidents, particularly in mixed traffic environments where both AVs and human-driven vehicles share the road. AVs are, in essence, alien entities in an existing infrastructure designed for human drivers. Their rigid adherence to rules and slower decision-making processes can confuse human drivers and cause traffic flow disruptions, leading to rear-end collisions, lane-changing conflicts, and other types of incidents. This also leads to an ethical dilemma that AVs are expected to make roads safer and prevent accidents that would have occurred with humans drivers. However, they themselves may cause new types of accidents that would not have occurred with just humans, even though these in number may be fewer. In essence, even if AVs at some point become demonstrably safer than humans, it is unclear whether the public would accept that the new types of accidents are a price worth paying for this improvement.

4.3 Miscommunication, mis-trust and lack of drivers’ knowledge of ADS capabilities

The workshop revealed significant mistrust and miscommunication between manufacturers, regulators, and drivers. Manufacturers often overstate the capabilities of their ADS, leading drivers to expect more from the system than it can deliver. There is a lack of clear communication about system limitations. Automation can lead to overtrust and complacency [such as the case with the Tesla Autopilot and Full Self-Driving Beta systems (Nordhoff et al., 2023)], where drivers believe the system is more capable than it is. This misalignment of expectations poses significant safety risks. Additionally, there is a disconnect between regulations and vehicle capabilities, leading to inconsistencies in how systems are marketed, regulated, and perceived by users. This uncertainty and misalignment was mentioned to lead to increasing confusion and mistrust about what a vehicle can actually do and how automated control it can deal with. A lack of trust and acceptance from driver/users in critical situations could hold back the benefits of ADS, when driver decide to switch off systems they do not understand or trust. It was also mentioned that manufacturers appear to have insufficient knowledge of traffic systems and driver training.

4.4 Misalignment in driver training for ADS

A key point raised was the contradiction between current regulations and driver behaviour. Existing driving tests and assessments are outdated, focusing primarily on traditional vehicle control rather than preparing drivers to manage interactions with ADS. This has led to a gap in driver knowledge regarding the limitations of automation and how to intervene effectively. The mismatch between the level of automation and the skills required to oversee it safely creates a dangerous disconnect between regulation and real-world driving. In addition, deskilling is also feared where the reduced need for human control leads to a loss of critical driving skills. This also results in a lack of engagement, as drivers may become complacent, further reducing their ability to intervene when needed. A final concern relates to drivers that portray one type of driving style with training and for driving exams, but adjust to a less safer driving style after obtaining a license.

4.5 Insufficient redundancy in ADS for human failing

The lack of redundancy in ADS was identified as a major concern. Many participants noted that current systems lack sufficient fail-safes, meaning if one aspect of the system fails, there may be no adequate backup to maintain safe operation. Without redundancy, ADS are vulnerable to failures, particularly in critical situations where human intervention is expected but not guaranteed due to the driver’s diminished engagement. This lack of fallback mechanisms increases the vulnerability of the system to failures, whether due to technical issues or human error, which can lead to unsafe situations and accidents.

4.6 Ambiguous responsibility and accountability

A key concern raised during the workshop was the ambiguous division of responsibility between human drivers and automation systems, particularly in critical situations. Participants discussed the difficulty of clearly allocating responsibility in scenarios where the ADS is in control but requires human intervention. This not only blurs accountability in the event of an accident (i.e., makes it harder to determine who’s at fault) but also creates responsibility gaps (i.e., increases the likelihood of accidents in which the driver relied on ADS while ADS expected an intervention from the driver). Nominal liability, e.g., stating that under certain circumstances a specific actor is accountable, is not the same as where real responsibility or ethical accountability lies. This means that issue may not be properly tackled and may persist into the future.

4.7 Legal and policy gaps

The workshop highlighted significant gaps in current regulations, particularly in the operationalisation of MHC. Existing regulatory policies are often vague and inconsistently applied, with different regulators interpreting them in various ways. Participants pointed to the lack of clear legal guidance on how MHC should be integrated into ADS and assessed, and how drivers and manufacturers should be held accountable for ensuring safety.

4.8 Economic pressures outpacing regulation

The influence of economic pressures on ADS development was another mentioned concern. Participants noted that the rush to bring higher levels of automation to market is driven by competition and consumer demand, particularly in regions like China and the United States. This has led to premature claims about the capabilities of ADS, potentially compromising safety in favour of rapid innovation. The need for stronger regulatory oversight to ensure responsible development was emphasized, along with the risk of creating systems that outpace the regulatory framework.

5 Main actions

5.1 Create learning communities

To bridge the gap between stakeholders, participants suggested forming learning communities involving governments, manufacturers, researchers, and other industry players. These communities would facilitate knowledge sharing, collective problem-solving, and ongoing development of best practices for ADS. The collaborative approach would enable stakeholders to align their goals, particularly around safety standards and MHC operationalization. Practically speaking, building learning communities could be most productive when centred around ADS use-cases targeting specific environments (e.g., trucking, public transportation) rather than general-use ADS technology. Such case-specific communities can ensure a feasible trade-off between the complexity of the socio-technical system the ADS is operating within and the tangible societal benefits, as well as ensure meaningful involvement of all key stakeholders.

5.2 Introduce continuous driver assessment

It was widely agreed that static driving tests are insufficient for evaluating drivers’ ability to meaningfully control increasingly autonomous systems. Participants proposed continuous assessment mechanisms for drivers/users of ADS-equipped vehicles, ensuring that safety standards and driving skills evolve alongside the technology. This could include regular check-ins or performance-based evaluations that adjust as software updates or new systems are introduced. A modular approach to driver training could be an option here, similar to the continuing professional development training received by professionals who perform work that can either change or lead to deskilling over time. Another suggestion was made to allow drivers to make more extensive use of immersive driving simulators as part of training. This would allow a longer, more elaborate and extensive learning experience, with some similarities to simulator training used by airline pilots and truck drivers.

5.3 Safety certification for AVs

In addition to driver assessment, a strong need for an ongoing, dynamic safety certification for ADS itself was also discussed. Unlike traditional vehicles, ADS-equipped vehicles can exhibit behavior of their own, which needs to be evaluated and certified in order to allow the use of such vehicles on public roads. Furthermore, given that autonomous systems can receive frequent updates remotely, participants proposed the development of a real-time certification system that assesses not only the hardware but also software updates that affect system behaviour. This would ensure that changes to vehicle systems do not compromise safety and that human control remains viable at all times. Participants highlighted that concrete requirements for AV certification protocols are currently being developed by authorities in many countries, but require substantial fundamental research into appropriate behavior references for AVs as well as continuous assessment methodologies.

6 Discussion

The issues and actions presented in this paper highlight a number of aspects that are critical and urgent for the deployment of AVs on various levels. Meaningful human control does not even need to be explicitly considered to realise that these issue exist. However, seeing these aspects in the context of MHC helps identify and highlight the issues and, more importantly, how they can possibly be addressed. Hence, here we discuss the findings of the workshop in light of the two conditions for meaningful human control: tracking and tracing (Santoni de Sio and Mecacci, 2021).

The tracing condition of MHC focusses on which humans can exert meaningful control, how, and on what level (strategic, tactical, operational, ethical, etc.); furthermore, it implies that these humans have sufficient ability and knowledge to do so (Cavalcante Siebert et al., 2023). The workshop discussions highlighted numerous serious issues for this condition. Firstly, while conditionally automated vehicles demand that a driver monitors the system, the experts highlighted that reducing driver workload often leads to distraction. Drivers are unable to take back control in a timely and adequate fashion in many cases. More worryingly, drivers appear to often not realise what the capabilities and inabilities of their vehicles are and well as sometimes not realising which mode the vehicle is currently driving in. All of these issues lead to a profound disconnect between the drivers’ ability to control the system and their designated responsibility for the driving task. Improved driver training is seen as an essential step, as well as improving drivers’ understanding of vehicle capabilities and mode state through better human-machine interfaces. From an acceptance and trust perspective, users will need to gain confidence in these new types of vehicles, while aspects surrounding liability, responsibility, and accountability remain prevalent. Part of this might be the need to accept that AVs could lead to new types of accidents, while overall they are expected to lead to a much greater degree of safety. While legal liability will need to be explicitly detailed, solutions that facilitate forward-looking responsibility and accountability are equally important. These can aid stakeholders to exert more meaningful control, i.e., proactively act upon their responsibility to avoid undesirable outcomes. This can help increase safety and reduce the occurrence of situations that would require liability investigations in the first place.

The tracking condition for MHC considers the extent that an ADS can follow human intentions and reasons to act in a responsible and sufficiently intended manner. Much of this is embedded in the design of the vehicle, but also includes fall-back and remote intervention options. The behaviour of AVs is new in the traffic environment and will lead to different dynamics between vehicles. Predictability and human-like interaction with AVs will initially be important as human drivers adjust to the way AVs drive. Moreover, in some cases humans will not be able to respond to the same extent as an AV and accidents may occur as a consequence. When considering the abilities and safety of vehicles, and to what extent MHC is adhered to by the vehicle, law-makers and regulators have some real challenges ahead. While traditionally, a vehicle was a technical system controlled by a trained human who could display behaviour, now the vehicle also exhibits behaviour that previously never needed to be tested. Hence, regulators will now need to devise ways to evaluate vehicle behaviour that has been developed and programmed by car manufacturers, as well as how this vehicle behavior interacts with the behavior of the human user, and how these evolve over time. Increased redundancy in ADS is hence critical, both from the perspective of ADS fallbacks, but also as additional safety-net mechanisms for human ADS users. Creating vehicles that are not just safe, but are safe even when things go wrong is broadly seen as a necessity for future AV design and deployment.

The emphasis of many issues and actions discussed in this work is on the control of an AV by the human driver. However, the ability of other human agents in the AV ecosystem to exert control over the vehicle is equally important. There need to be ways for stakeholders – policy making institutions, vehicle manufacturers, road authorities, human road users surrounding the AVs, and many more – to meaningfully influence the AV behavior (and take appropriate responsibility) through design, regulation, and interaction. An important aspect in this regard is the interaction between AVs and traffic infrastructure. It is likely that increasing market penetration of ADS will lead in time to evolving road infrastructure (something that can provide a way for road authorities to exert meaningful control on AVs). Ensuring that this new infrastructure remains suitable for vehicles with and without ADS poses a substantial challenge. Relatedly, countries across the world differ greatly in how accommodating their current road infrastructure is towards the introduction of ADS. Designing ADS to be able to operate in environments with only the most basic infrastructure could help ensure a more fair distribution of benefits from ADS technology.

Many of the issues emphasized by the workshop participants point towards the need for new kinds of regulation which must take into account the complex socio-technical nature of the AV ecosystem. It is no longer feasible to assess and certify human drivers and vehicle technologies independently from each other. The challenge lies in both formulating clear capability criteria for human users of AVs which take into account the potential diversity of ADS technologies, as well as setting clear goalposts for ADS evaluation which involve the assessment of the interactions between ADS and humans in and outside of the AV. Relatedly, much of the existing legal liability mechanisms need to factor in these aspects. However, this requires deepening our fundamental understanding of the human-AV interaction and its ethical aspects.

We strongly recommend to view each of the main points offered in this paper through the lens of meaningful human control. This concept can potentially lead to solutions for each of the points and improve the overall human control and safety of ADS in practice. While other conceptual approaches undoubtedly can and will lead to improvements, none cover the scope of complex socio-technical issues surrounding AVs to the same extent that MHC does. Meaningful human control considers these issues from an integrated human-machine perspective including technical aspects, human behavioural abilities, and societal values.

7 Conclusion

This paper presents some of the current main challenges and opportunities that lie ahead for the development, deployment, and regulation of automated driving systems. With a focus on meaningful human control, we present insights gained from a transdisciplinary workshop involving key stakeholders from the AV ecosystem. The discussions underscored critical issues, including the lack of clear human control for conditionally automated vehicles, the emergence of new accident types, and the misalignment between ADS capabilities and driver expectations. Furthermore, the importance of addressing mistrust and miscommunication between manufacturers, regulators, and drivers is critical, as well as the need for redundancy in systems to ensure fail-safe operation. Key actions proposed include improving driver training, establishing continuous assessment mechanisms, and developing dynamic safety certification processes. Operationalising meaningful human control can help ensure that ADS operate safely, ethically, and under clear human control even as automation advances. This paper contributes to ongoing discussions by highlighting existing issues while offering actionable pathways and argues for the necessity of meaningful human control in increasingly autonomous driving.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Author contributions

SC: Conceptualization, Formal Analysis, Investigation, Methodology, Supervision, Writing–original draft, Writing–review and editing. AZ: Conceptualization, Formal Analysis, Investigation, Methodology, Supervision, Writing–original draft, Writing–review and editing.

Funding

The author(s) declare that financial support was received for the research, authorship, and/or publication of this article. This work is also supported by the Centre for Meaningful Human Control and the Automated Driving and Simulation (ADaS) lab at TU Delft.

Acknowledgments

This paper came about through the hosting of a transdisciplinary workshop at TU Delft in June 2025. The authors are grateful to the workshop participants for their time and expertise. The authors are also grateful to Ashwin George, Lucas Suryana, and Samir Mohammad, who assisted in setting up the workshop and took extensive notes.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Generative AI statement

The author(s) declare that no Generative AI was used in the creation of this manuscript.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Calvert, S. C., Johnsen, S., and George, A. (2024). “Designing automated vehicle and traffic systems towards meaningful human control,” in Research handbook on meaningful human control of artificial intelligence systems (Cheltenham, Unite Kingdom: Edward Elgar Publishing).

Google Scholar

Cavalcante Siebert, L., Lupetti, M. L., Aizenberg, E., Beckers, N., Zgonnikov, A., Veluwenkamp, H., et al. (2023). Meaningful human control: actionable properties for AI system development. AI Ethics 3, 241–255. doi:10.1007/s43681-022-00167-3

CrossRef Full Text | Google Scholar

Dikmen, M., and Burns, C. (2017). “Trust in autonomous vehicles: the case of Tesla Autopilot and summon,” in 2017 IEEE international conference on systems, man, and cybernetics (SMC) (IEEE), 1093–1098.

Google Scholar

Flemisch, F., Heesen, M., Hesse, T., Kelsch, J., Schieben, A., and Beller, J. (2012). Towards a dynamic balance between humans and automation: authority, ability, responsibility and control in shared and cooperative control situations. Cognition, Technol. and Work 14, 3–18. doi:10.1007/s10111-011-0191-6

CrossRef Full Text | Google Scholar

Hopkins, D., and Schwanen, T. (2021). Talking about automated vehicles: what do levels of automation do? Technol. Soc. 64, 101488. doi:10.1016/j.techsoc.2020.101488

CrossRef Full Text | Google Scholar

Louw, T., Kountouriotis, G., Carsten, O., and Merat, N. (2015). “Driver inattention during vehicle automation: how does driver engagement affect resumption of control?,” in 4th international Conference on driver Distraction and inattention (DDI2015), sydney: proceedings. ARRB group.

Google Scholar

Nordhoff, S., Lee, J. D., Calvert, S. C., Berge, S., Hagenzieker, M., and Happee, R. (2023). '(Mis-) use of standard Autopilot and Full self-driving (FSD) Beta: results from interviews with users of tesla's FSD Beta. Front. Psychol. 14, 1101520. doi:10.3389/fpsyg.2023.1101520

PubMed Abstract | CrossRef Full Text | Google Scholar

Santoni de Sio, F., and Mecacci, G. (2021). Four responsibility gaps with automated systems: why they matter and how to address them. Philos. Technol. 34, 1057–1084.

Google Scholar

Santoni de Sio, F., and Van den Hoven, J. (2018). Meaningful human control over autonomous systems, A philosophical account. Front. Robotics AI 5, 15. doi:10.3389/frobt.2018.00015

PubMed Abstract | CrossRef Full Text | Google Scholar

Seppelt, B. D., and Lee, J. D. (2019). Keeping the driver in the loop: dynamic feedback to support appropriate use of imperfect vehicle control automation. Int. J. Human-Computer Stud. 125, 66–80. doi:10.1016/j.ijhcs.2018.12.009

CrossRef Full Text | Google Scholar

Suryana, L. E., Nordhoff, S., Calvert, S. C., Zgonnikov, A., and Van Arem, B. (2024). “A meaningful human control perspective on user perception of partially automated driving systems: a case study of Tesla users,” in 2024 IEEE intelligent vehicles symposium (IV) (IEEE), 409–416.

Google Scholar