AUTHOR=Salvini Pericle , Reinmund Tyler , Hardin Benjamin , Grieman Keri , Ten Holter Carolyn , Johnson Aaron , Kunze Lars , Winfield Alan , Jirotka Marina TITLE=Human involvement in autonomous decision-making systems. Lessons learned from three case studies in aviation, social care and road vehicles JOURNAL=Frontiers in Political Science VOLUME=5 YEAR=2023 URL=https://www.frontiersin.org/journals/political-science/articles/10.3389/fpos.2023.1238461 DOI=10.3389/fpos.2023.1238461 ISSN=2673-3145 ABSTRACT=

This paper draws on three case studies to examine some of the challenges and tensions involved in the use of Autonomous Decision-Making Systems (ADMS). In particular, the paper highlights: (i) challenges around the shifting “locale” of the decision, and the associated consequences for stakeholders; (ii) potential implications for stakeholders from regulation such as the General Data Protection Regulation (GDPR); (iii) the different values that stakeholder groups bring to the “decision” question; (iv) how complex pre-existing webs of stakeholders and decision-making authorities may be disrupted or disempowered by the use of an automated system and the lack of evaluation of possible consequences; (v) how ADMS for non-technical users can lead to circumvention of the boundaries of intended system use. We illustrate these challenges through case studies in three domains: adult social care, aviation, and vehicle driver monitoring systems. The paper closes with recommendations for both practice and policy in the deployment of ADMS.