Working with robotic process automation: User experience after 18 months of adoption

This paper reports a study of User Experience (UX) with Robotic Process Automation (RPA), in the perspective of workers of EdP Brazil, a large electric utility company that operates in Brazil. RPA are software solutions for automating business processes that find increased interest of companies because they are inserted in workgroups as a co-worker, emulating human workers operating on GUI interfaces. Although the technology promises to drive a new wave of productivity in service companies, its impact on co-workers' experience is still unexplored. Based on projective interviews using the AXE (Anticipated eXperience Evaluation) protocol, after the first 18 months of RPA operation, the analysis of workers' collaboration with the robots has evidenced multiple facets of UX, technology acceptance and innovation adoption. For this case, RPA has provided an overall positive user experience mainly due to the perceived utility of the spared time, the upgrade in career opportunities and the pride for actively participating in the innovation adoption. Negative experience comes mainly from the lack of visibility that hinders robot management for efficiency and improvement. The methodology used in the study was successful in capturing the multifaceted workers' experience and is potentially useful to support user research in new expansion RPA projects.

1. Introduction

Robotic Process Automation (RPA) are software agents that automate clerical manual tasks by processing data. While robotic process automation technology has several, clearly defined benefits for the company, workers' experience with the robots are still not well documented in the literature. The IEEE Guide for Terms and Concepts in Intelligent Process Automation defines RPA as: “A preconfigured software instance that uses business rules and predefined activity choreography to complete the autonomous execution of a combination of processes, activities, transactions, and tasks in one or more unrelated software systems to deliver a result or service with human exception management” (Group, 2017).

RPA software, from now on designated simply as RPA, usually operate business information systems as a person would do, by retrieving data from business applications, filling forms and processing transactions under business rules (Alberth and Mattern, 2017). RPA software is configured to connect to an ERP system and/or other corporative systems, from which it extracts business data, replacing humans in well-defined manual and repetitive tasks.

As Wilcocks and colleagues state, “RPA software is ideally suited to replace humans for so called “swivel chair” processes; processes where humans take inputs from one set of systems (for example email), process those inputs using rules, and then enter the outputs into systems of record (for example Enterprise Resource Planning (ERP) systems)” (Willcocks et al., 2015).

RPAs can be grouped into three classes, or “generations”: the first one, labeled G1 RPA, automate tasks based on structured data found on systems databases. The second class, labeled G2 RPA, works from unstructured data such as text files based on machine learning techniques. G3 RPAs are cognitive platforms that can perform decision-making tasks (Ernst and Young, 2015).

G1 RPA is an interesting solution to lessen operational costs, as its adoption reduces operational costs with little impact on IT infrastructure. Robots interface with existing systems by emulating a human user, and they can be configured by business rule experts instead of IT personnel. The potential to use cognitive solutions in G2 and G3 RPA promises to increase transformation in the scenario of process automation in the next few years (Rutaganda et al., 2017). Effectively, according to MarketsandMarkets, the market for RPAs was estimated to grow over 30% between 2017 and 2022, mainly driven by “the ease of business processes provided by the robotic process automation, and convergence of robotic process automation with traditional business process industries” (MarketsandMarkets, 2017).

Claims toward adopting RPAs are based on several arguments: RPA potentially reduces FTE (full-time employees); RPA provides increased service quality due to the fact that robots do not make mistakes while doing what they are programmed to do: RPA increases efficiency due to the robots overall performance, superior to human's; RPA increases liability, because the transactions can be automatically documented according to compliance requirements; RPA provides uplifting of the human workforce, because being free of repetitive, tedious work, people can perform more valuable tasks such as listening to customers, analyzing the business and innovating.

Following the international trend EdP Brazil has started adopting G1 RPA in 2017 as part of the company effort to improve efficiency and reduce the risk of costs associated to non-compliance with the national energy agency rules. EdP Brazil implemented RPAs with Blue Prism™ technology in several business units, mainly in the financial areas. EdP Brazil is progressing in this process by developing its first G3 RPA (Vajgel et al., 2021).

Additionally, EdP led the foundation of the Brazilian Business Pact for Humanized Work Digitalization, which established principles to promote a human-centered process for robotization and digital transformation. These principles focus on empowering humans by education, inclusion, engagement, leadership development and compliance with ethical behavior. As a result of this pact, the authors, researchers from Universidade de São Paulo (USP) in Brazil and EdP Brazil employees in the Research, Development and Innovation area have cooperated to study workers' experience with RPA, under PROP&D, a R&D program regulated by the Brazilian Agency for Energy, ANEEL

In fact, while robotic process automation technology has several proven benefits for the company, user experience is a strategic factor for technology acceptance that is still poorly documented in the scientific literature.

Recent systematic literature reviews show that there is a growing interest in the area (Ivančić et al., 2019) as RPA is being massively applied to industry. A literature review by Syed et al. (2020) has pointed to several relevant research challenges. Yet, the user experience (UX) perspective of technology has not deserved a dedicated look in academic studies. Our own literature review showed that most of the publications about RPA belong to the business administration area, reporting on the success factors of RPA in business (Lacity et al., 2016; Rutaganda et al., 2017; Devarajan, 2018; Leshob et al., 2018) and on applications in different areas such as healthcare (Ratia et al., 2018), public administration (Houy et al., 2019), software engineering (Montero et al., 2019), and finance (Stople et al., 2017; Lewicki et al., 2019). Others are focused on identifying which tasks are suitable for being automated by RPAs (Vishnu et al., 2017). However, the effectiveness of innovation in a company depends not only on the technology, but also on factors that express how workers understand changes and cope with them.

Very few studies were found that considered the human factor with RPA in depth. Our literature search for related studies addressed the area of Human-Robot Interaction (HRI), which studies the interaction between robots and humans (Fontanillo Lopez et al., 2020; Schellen et al., 2021). Although the subject of relationship between humans and automata is not new, the study of HRI became more prominent in the 1980s, when behavior-based robotics started using distributed sense-response loops to generate appropriate responses to external stimuli. According to Goodrich and Schultz in their review on HRI, an HRI problem consists in understanding and shaping the interactions between one or more humans and one or more robots (Goodrich and Schultz, 2008). Chen and Barnes have reviewed the literature on human-agent interaction to identify the most critical issues that need to be addressed for such systems to be effective (Chen and Barnes, 2014). Yet although their discussion is applicable to RPA and RPA is a legitimate HRI problem, human-RPA interaction has not been studied yet in the HRI domain.

In the Computer-Supported Cooperative Work area, we found that Nauwerck and Cajander have published their preliminary results of a study of human introduction of robotic process automation (RPA) of financial support in a Swedish municipality focusing on workers' experience (Nauwerck and Cajander, 2019). However, the most comprehensive study found about workers' relationship with RPA was Katriina Juntunen's thesis from the Aalto University RPA adoption and acceptance processes in the Financial department of Stora Enso, a Finnish paper manufacturer (Juntunen, 2018). Her work produced an integrative framework in which she analyzed the organizational factors that influenced the adoption of RPA. This framework was helpful for our analysis in this paper, as we present in the following sections.

We have here attempted to understand how EdP workers relate to G1 RPAs after 18 months of its adoption. Our research question is stated as “After 18 months of the decision to adopt RPA in EdP, how do workers characterize their experience with RPA technology?” This research question is decomposed into three secondary questions:

RQ1: Which factors have influenced their positive experience with RPA?

RQ1: Which factors have influenced their negative experience with RPA?

RQ1: Which are the expectations of these workers regarding this technology evolution?

This paper is organized in 7 sections. In section 2, we present the theoretical background that supports this investigation. In Section 3, we describe the scenario of RPA application in EdP. In Section 4, we detail our research methodology. Section 5 contains the presentation and discussion of our findings regarding the UX framework. Section 6 presents our findings regarding the acceptance and adoption framework. Section 7 presents our discussion on findings as well as our reflections on how to improve workers' experience with RPAs.

2. Theoretical background

This research is based on theories and methods of the Human-Computer Interaction area, in which phenomena related to how people interact with technology are the core interest. At first, we have based our research on User eXperience (UX) frameworks, but the preliminary results suggested an expansion of analysis to broader organizational frameworks. In this section, we present the frameworks that guided our study.

2.1. User experience

Understanding UX is essential in modern design approaches. In the Human-Centred Design perspective, the design of an innovation must keep a close focus on humans, their needs and characteristics (FDIs, 2008).

Among the many definitions of UX, we pick Hassenzahl and Tractinsky's for its coherence with our analysis target: user experience is a consequence of a user's internal state (predispositions, expectations, needs, motivation, mood, etc.), the characteristics of the designed system (e.g., complexity, purpose, usability, functionality, etc.) and the context (or the environment) within which the interaction occurs (e.g., organizational/social setting, meaningfulness of the activity, voluntariness of use, etc.) (Hassenzahl and Tractinsky, 2006).

UX is unique to an individual. UX definition restrains the experience to those encountered with some technological artifact, be it a system, a product or a service. It is the result of the individual's encounters with the artifact, and is affected by prior experiences, expectations, cultural background and social context. UX refers to both the cognitive and emotional consequences of encounters.

As encounters can happen over time, UX varies due to exposure to systems. Roto, Law, Vermeeren and Hoonhout, early researchers on the concept of UX, have identified time spans of UX. People can have anticipated UX before their first encounter with the target technology, based on expectations, experience with other systems or information about the new technology. Momentary UX is related with one single encounter and to the feelings brought by that instantaneous interaction event. We refer to episodic UX when we consider a certain usage situation that may have happened in the past. Over time, the series of momentary experiences result in cumulative UX, which is defined as the reflection over the recollection of various episodes of usage. Interestingly, anticipated UX may relate to cumulative UX, because expectations are constructed based on previous experience (Roto et al., 2011b).

UX is influenced by factors, namely system, user and contextual factors (Roto et al., 2011b). Different UX authors identify different attributes for these factors. System factors are related to usability and quality-in-use attributes, as defined in ISO SQuaRE system (ISO, 2011). User factors may include prior knowledge and willingness to use the system as well as affective characteristics and personality traits. Contextual factors of different natures, from social and organizational environment to physical conditions and task specifications, can also shape experience.

The complexity of multiple influential factors added to the individual nature of UX makes experience design a difficult undertaking. However, it is possible to design for UX (Hassenzahl, 2013). Understanding influential factors is strategic knowledge; currently, companies are largely investing in UX to identify their customers' values, to propose new products, to make processes more efficient and to reduce waste.

There are several approaches to UX. Among different models and methods, Hassenzahl's framework for understanding UX is widely recognized as a comprehensive model, encompassing factors of different natures (Hassenzahl, 2003). Hassenzahl's model explains that a product is designed to present an Intended Quality. Intended qualities can be both pragmatic, that is, associated to what the system can do in both the functional and non-functional perspectives, and hedonic, associated to the emotional consequences that designers intend their users to experience. In real use, the designed intended quality is somehow perceived by users. The perceived pragmatic quality and the perceived hedonic quality can be different from the designed ones. They are compiled by a user's cognitive assessment into product attractiveness, which results in behavioral consequences (such as increased use) and emotional consequences (such as joy).

2.2. Methods for UX evaluation

Academic and professional literature also show several techniques and tools to evaluate UX. The website All about UX (https://www.allaboutux.org/all-methods, accessed on 07/30/2022) lists 81 different methods and techniques for evaluating UX. In this research, we chose the AXE-Anticipated eXperience Evaluation method because of its support to a qualitative in-depth interview, to the problem we had in hands. The AXE protocol is briefly described as follows.

AXE is a qualitative user research method developed by Lutz Gegner and Michael Runonen of Aalto University, in a cooperation project between Departments of Design and of Computer Science of Aalto University and Nokia Research Center, Helsinki. The protocol was developed for evaluating interactive concepts and early prototypes under the authors' assumption that “identifying important experiential aspects during very early phases of development can reduce costly changes but also provide a competitive edge.” (Gegner and Runonen, 2012).

The AXE protocol is based on psychological projective tests. Participants are shown pictures they associate with the product, system or service that is the target of evaluation. The pictures, predefined in the protocol, were selected to evoke the concepts of hedonic and pragmatic qualities, as well as attractiveness, from Hassenzahl's user experience model in the AttrakDiff questionnaire (Hassenzahl, 2003). Pictures are deliberately ambiguous so that participants can interpret them according to their personal background and beliefs, and express their “attitudes, opinions and self-concept” about the target product. As the AXE authors advocate, the activity of freely interpreting a picture helps remove the interference of the interviewer's words with the recall of the participant's experience.

Despite conceived as a method for evaluating experiences with early prototypes, our previous experience with AXE indicated that it is an interesting method also for cumulative UX, when the user recollects multiple periods of use (Roto et al., 2011a). The projective characteristic of the interview allows the moderator to place the focus on any moment in the experience timeline.

Also, we predicted that user experience with a new technology in a work environment with many context variables could have strong influence from organizational factors not captured by UX assessment methods with structured interviews, such as AttrakDiff. We understood that in-depth interviews could expose the totality of the experience. In previous studies with the AXE method, we observed that participants' interpretation of the images stimulated them to talk about what is relevant to them, even if not directly related to the hedonic and pragmatic qualities of the model underpinning the method.

AXE framework for analysis is based on Hassenzahl's UX model (Hassenzahl, 2003). There are three main categories, Perceived Product Features, Associated Attributes and (Anticipated) Consequences.

According to the protocol handbook (Gegner and Runonen, 2012), the analyst must classify under the Perceived Product Features category users' appreciation of the system look&feel, that is, opinions on Content, Interaction, Presentation and Functionality features.

The Associated Attributes category must be used to compile users' appreciation of system qualities. Opinions must be further separated into attributes associated to the system meeting Pragmatic Needs (Utility and Usability) and attributes associated to the system meeting users' Hedonic Needs (Stimulation, Identification and Evocation).

Reports on how the user felt attracted or changed their behavior are classified under Attractiveness and Behavioral Change subcategories of the (Anticipated) Consequences category.

Users may also express their perceptions as Suggestions or criticisms (Unwanted). Opinions about the overall concept are grouped into a Meta category.

2.3. Tecnhology acceptance and innovation adoption models

The trend toward RPA technology in companies has motivated researchers to study this technology from the point of view of technology acceptance models, as well as innovation adoption models.

Technology acceptance models intend to explain the user's decision about using or not a given technology. These frameworks focus on understanding the motivation of an individual to a certain behavior thus explaining the adoption of technology from an individual basis. Innovation adoption models propose a sequence of steps that an organization should go through in order to decide whether to adopt or reject a technology. They study how an idea perceived as new spreads through a social system and gets incorporated. Both kinds of models have constructs that help explain the relationship between users and an innovative technology. They were proven helpful to show the reasons why a technology may succeed or fail in their real application.

There are many contact points between UX models and technology acceptance models. Factors that provide a positive user experience can influence the acceptance behavior. Used in combination, the constructs of these models can be effective for understanding people's intention to use a technology (Prietch and Filgueiras, 2015; Al-Rahmi et al., 2019).

Our literature research on RPA technology adoption and acceptance resulted in one relevant publication, a Masters dissertation by Katriina Juntunen from Aalto University (Juntunen, 2018), in which she analyzed the intra-organizational adoption of RPAs in the Financial department of Stora Enso, a Finnish paper manufacturer.

In that work, Juntunen analyzed and compiled constructs from 8 models that explain adoption from the individual, social and managerial perspective: four user acceptance models, including the well-known Technology Acceptance Model (TAM), by Davis and colleagues (Davis, 1989) and the Unified Theory of Acceptance and Use of Technology (UTAUT), by Venkatesh and collaborators (Venkatesh et al., 2012); one innovation diffusion model, the Innovation Diffusion Theory (IDT) by Rogers (Rogers et al., 2014); and three change management models, which were included to provide a clear view on how management acts to promote innovation adoption.

Juntunen's composite model resulted in a structure for understanding the intra-organizational adoption of innovation. She proposes that influencing variables and key beliefs condition adoption.

Influencing variables, summarized in Table 1, were derived from all the cited models and include innovation attributes, organizational attributes, individual attributes and managerial facilitation attributes. In section V herein, we present the definition of each variable together with our interpretation of this variable in the context of our study.

TABLE 1

Table 1. Juntunen's composite model of technology acceptance and adoption-influencing variables (Juntunen, 2018).

The model also establishes key beliefs. The concept of belief is not clear in the literature and Juntunen does not define her position; however, based on the discussion by Österholm (2010), we assume that belief is the knowledge a person assumes to be true in the context of their actions, that is, the attitude toward innovation depends on the beliefs about the consequences of performing the behavior and evaluation of these consequences. She defines five influential beliefs that define whether individuals accept or resist the change, so that these beliefs should be influenced to promote the change:

• Perceived benefits, related to individuals' perception of outcomes and benefits of behavioral change.

• Perceived effort, related to individuals' perception of the number of resources to be dedicated to the change.

• Perceived social pressure and influence, that relates to the risk of being against the social tendency.

• Perceived need and appropriateness, that express the understanding of personal and task needs and the appropriateness of innovation.

• Perceived capabilities, which address how the individual evaluates self-efficacy and capability to perform, as well as resources availability.

Juntunen's framework complemented the UX framework as it explains several constructs that can influence experience but are encapsulated as “contextual factors.” In turn, as the paper reveals, the UX model complemented the adoption framework by explaining the human reasons behind constructs.

3. Scenario of RPA application

In this section, we describe the scenario in which we carried out our analysis. EdP Brasil has been consolidating its position as one of the largest companies in the electric energy area in Brazil, through strategic acquisitions of energy companies in the sector's privatization process. With this, the company inherited from the acquired companies business processes, teams, and systems, which need to go through revision and standardization. These activities are part of the company's digital transformation process, and in this context, the adoption of RPAs is seen as a necessary and urgent transformation.

When the introduction of RPA was initiated, business units were advised regarding the required characteristics of processes to be automated, considering the level and capabilities of the G1 RPA available to the organization. Automation candidate activities were those that fully matched requirements of data volume and execution time and presented a standardized execution process. Although the company had established processes at the value chain level, operational processes often lacked standardization and documentation, in which each employee performed the activity differently. Thus, before automation could proceed, business units was required to apply well-known strategies and models such as PDCA, Lean and 5W2H to standardize their candidate processes to automation. Our research was performed in the context of the client support activity in EdP Brazil which had gone through this effort. Two independent business processes automated with RPA technology were selected for analysis: the damage compensation process and the billing anomaly process. We describe them briefly in the following sections.

3.1. Damage compensation process

The damage compensation process (internally known as PID, from its Portuguese abbreviation) is responsible for compensating for damages caused by energy fluctuation to customers' electrical equipment. Electrical fluctuation can happen in the distribution network due to several causes, such as storms and equipment failure. In summer rainy seasons, the number of complaints due to electrical damage rises significantly. The company must respond to complaints in due time, as defined in Chapter XVI of the Brazilian ANEEL Regulatory Resolution 414/2010. Ṙobots are employed to scrutinize databases for evidence of matching between electrical incidents, affected areas and the customers' reports. Also, they organize communication with clients and follow the document exchanges for compensation, thus avoiding fines and economic penalties that result when the deadline for analysis and response to clients is not met.

3.2. Billing anomalies process

The second process deals with anomalies in billing accounting. Each day, a list of non-conformities in payments is detected by the billing system. Non-conformity causes are various; they can be divergence between values, error in barcode typing, errors in bank reports, wrong values, to cite a few. These cases are named anomalies. Robots diagnose and solve anomalies caused by known situations, performing analysis on several parameters and checking them in different systems. When the robot succeeds in classifying the anomaly in one of the known cases, it is corrected and cleared. If the robot is not capable of identifying the cause of anomaly, it reverts to manual.

4. Methodology

Our study was a qualitative research, guided by UX and technology adoption frameworks. We collected data on UX using AXE protocol with EdP employees, who are related to the robotized processes described in Section 3. Subjects were recruited by the company R&D managers based on their experience with the RPA technology. They are real users of the robot or process managers. Some of them participated in the robot configuration and deployment.

The AXE application has three phases: preparation, interview and analysis. In preparation phase, we provided two additional pairs of pictures that address specific features of the product as described in Section 4.2. Interviews took place in meeting rooms isolated from the sight and sound of the workplace, with only the participant and the interviewer (the first author of this paper) side by side at the table. Personnel and equipment were kept to a minimum: the interviewer, her laptop running a form for participant identification and her smartphone running an audio recording app.

The evaluation session was carefully explained in the context of the R&D project. Emphasis was given to the anonymization process of results in which the researcher removed names and text excerpts that could potentially identify the subject. Participants were informed that they were free to refuse to participate and invited to sign a consent form in case they agreed.

The submission of the research project to an Institutional Review Board was not required under the Brazilian regulation for ethics in research involving human participants (Brasil et al., 2016). This regulation exempts from submission and approval research projects that aim at the theoretical deepening of situations that emerge spontaneously from professional practice, provided they do not disclose data that can identify the subject.

Since the business process is well-known by the users, they were invited to recall a normal workday without the robot and a normal workday with the robot. Next, a welcome page was presented and read together. The second page had the instructions and the warm-up pair. The warm-up pair is easily interpreted as a concept of speed and performance. The interviewer presented two different interpretations of the picture to show that there is no right or wrong answer.

The pairs of pictures were then presented, and the interviewer prompted the participants to explain their choices and to discuss important matters as the interview proceeded. The interviews were informal and relaxed. Interviews were transcribed and temporized.

Segments were selected; snippets were coded using Grounded Theory for concepts (Corbin and Strauss, 2008), together with codes from both AXE and Juntunen's frameworks using Qualitative Data Analysis software.

4.1. Participants

We interviewed 10 workers of EdP Brazil, five males and five females. Five participants work in line with the robots, that is, their work activity requires continued contact with the robot. Two participants work occasionally in line with the robot, that is, they have other attributions, but they may be asked to work in the automated process when the workload is excessive. Three participants are managers: two are the owners of the selected processes and one is the manager of the automation robot initiative.

Three participants have finished graduate studies, six have finished college and one is a college student, working as intern in the team. College titles are varied, from humanities to engineering. Figure 1 shows the age distribution and years of service in the company.

FIGURE 1

Figure 1. Participants' age and years of service in EdP.

4.2. Pairs of pictures

Choice of image pairs followed the guidelines for selecting pictures in AXE handbook (Gegner and Runonen, 2012). In these guidelines, AXE authors recommend choosing pairs of images with at least two easily identifiable opposite attributes. Also, they advise that some ambiguity is preserved, so that they provide a lead for discussion. Additional image pairs should motivate users to talk about aspects of interest to the application project. Our research team had identified, in discussions leading up to the project, concerns from RPA managers about workers' experience with RPA being an obscure technology, and that it caused social tension by the prospect of unemployment. Pictures were picked from internet image files. The first pair of pictures was chosen to convey the opposition between company and individual; rich and poor; employee and customer. Thus, the left side showed a young man working at his computer while the right side showed houses in a low-income district. The second pair presented the surface of a moon full of craters in a dark space, in opposition to a stream of clear water being poured into a metal spoon. This pair intended to provoke the opposition between difficult and easy to learn; magical/obscure and explicit; unknown and known.

4.3. Text segmentation and coding

The transcript of interviews was segmented into 1,050 snippets, which were then coded using AXE and Juntunen's frameworks. The analysis of these data produced 100 facts, which are discussed in the following sections.

5. Results of UX evaluation

In this section, we present findings from the qualitative data analysis relative to UX. We used the AXE framework to disclose workers experience with RPA. Each finding is explained and evidenced by excerpts of interviews. The participants are referred to as (Si).

Statements are grouped by UX attributes, and those are grouped into categories according to the analysis framework.

AXE categories were used to group our findings. Since the experience totalizes several influential factors, often one experiential expression holds elements associated to more than one category. Our intention of grouping into categories only intends to organize the presentation of findings, and not to trace any solid line between categories. Table 2 summarizes the findings, detailed in the following sections.

TABLE 2

Table 2. User experience with RPA.

5.1. Perceived features

This category is related to the workers' perception on the robot's characteristics as an interactive system.

RPA is not a canonical interactive software—robots have no clear user interface that the user can manipulate and from which the user obtains information. The interface between robot and users is a file containing the identifiers of the transactions that must be executed by the robot, in chronological order. As the robot successfully performs the transaction, the corresponding register is removed from the list. Workers then observe the decrease in the number of records to assess robot operation. If an operation fails, the register is marked with an icon symbolizing that the transaction has been processed but terminated unsuccessfully. In the latter case, the user retrieves the identifier to analyze the reason for non-execution and, when possible, executes the activity manually. Usually, the reason is the failure of an application or a change in structure/layout of a site which the RPA accessed to perform some query. Also, the user receives a daily email with the list of transactions that are exceptions to the standardized process and that should be processed manually. Although rudimentary, there is interaction in the collaboration between humans and robots and, unquestionably, a resulting user experience.

5.1.1. Content

Under this category, we grouped findings associated to participants' view on the process information and on the ability of the automation to deal with the process. We assume that the “content” addressed by robots is the programmed task(s) it executes. Also under content, we list the findings related to the process even if not automated, because we assume that the process is subject to being eventually “contained” in robots.

Participants reported that the process became more organized after robots. This happened because automation was preceded by a process analysis, reengineering and standardization. Standardization of activities was perceived as beneficial to the understanding of process.

However, processes are perceived as complex; because robots encapsulate this complexity, information about processes is perceived as lengthy and confusing when workers have to deal with it: “Robots follow the tendency to make things easy, to bring a lot of information. This may be good, but it can also be bad because you may get lost among all the options you have” (S1). “Lots of information all the time, all messed up, and we have to organize ourselves to understand each piece” (S2).

Partially automated processes incur in workload being shared between robots and humans. Workers understand themselves as robot's redundancy, responsible for processing in case of failures. Robots alleviate the workload, but those workers allocated to the automated process still occasionally face a heavy workload, which is unpredictable because it may depend on external variables, such as the rainy season or calendar day, or contingency. “When the server went down, the robot did not work. Then our work was a bit heavier, because then we had to deal with everything” (S4).

Workers report they miss indicators of performance as an important piece of content. Present robot implementation lack visibility on its real performance compared to intended performance, as well as numbers that express false-positive and false-negative answers. “We were surprised because what it was supposed to have done, it did not do. … When you lead a team, you keep an eye on the team performance. The robot is now part of the team, so I must keep track of its performance. And today this is not clear to us” (S8). It is important to clarify that workers have a limited view of RPA KPIs such as execution rate, execution time and volume processed, which were defined and collected by the IT area responsible for the deployment of the robots.

5.1.2. Interaction

Under this category, we grouped those findings related to how participants describe the robot operability qualities, that is, how workers can control its operation.

From the workers' perspective, they understand themselves as robot's supervisors. They must keep track of the robot operation because in the end of the processing cycle, the robot informs the transactions it did not perform, so that workers can work on the remaining entries. “The guy at the end is following the movement of the robot, so he knows that the robot has a routine… As it finished doing the shutdown, it starts an email or a log, for that user who is monitoring, [as if it said] ‘look, I finished my activity' ” (S10).

5.1.3. Functionality

Under functionality, we categorized snippets that address the workers' perception on what the robot does or can do. Once G1 RPAs and humans work on the same process, the robots' functionality is well known to users. Our findings show robots not being allowed to fully perform their functionality.

Humans' work starts after the robot has finished processing and completes what robots could not manage, because humans have access to unstructured and complex information. The robot's speed to solve large amounts of transactions provides a positive experience, but because robots operate in a limited time window, sometimes it fails to complete their share of the work. Workers perceive that the robot is not as effective as it could be and that the robot operation could be managed by operational areas, who would tune its performance as needed. “It still does not comply with 100%, there are many things that are its duty; however, it does not do it, so we end up having to do the work that was its task” (S6).

Also, workers understand G1 as underused, for there are opportunities for more functionality and applications. “I think there are improvements… I'm not even thinking about a future generation of robots, but I think this one can still be adjusted” (S5).

5.1.4. Presentation

Under presentation, we categorized snippets that address the look and feel of the robot. Workers perceive presentation from their own perspective, as users, and from the clients' perspective.

From clients' perspective, workers understand that one of the robot's tasks is to communicate (via email) with clients. As cases processed by the robot issue immediate communication, workers assume that the robot's prompt communication provides a positive experience to clients. “Our client contacts us because somehow he understands that our service caused him harm, he suffered a loss. The more agile I am in responding to him, the better it is” (S7).

Workers express they were initially concerned with the clients' perception of robots. This concern is even greater with the new generation robots. Workers understand that clients' interaction with robots should be close to the interaction between humans. “Will it run properly for clients? Will they notice? Or will it go undetected?” (S1).

However, from the workers' own perspective, the robot is a black box, stealth by nature. Due to the complexity of business processes, workers miss visibility of the automated process rules. Although present workers know the process well, transparency in process execution can be beneficial to new workers and to process improvement. “I wanted not only to push a button and get a result, but to have something that ensured that the robot actually performed the steps and did what needed to be done, so that it could guarantee the result it provided” (S9).

Robots communicate the transactions it could not close by placing a lock in front of unprocessed entries. This icon is the key for humans understand that they can work on an entry because the robot has already checked it and was unable to solve it. An email is also sent with the information on the entries not processed. This regular communication is minimalist and effective for workers to observe that the robot is operating.

However, workers need indirect evidence to observe malfunctioning: the lack of communication indicates that the robot is inoperative. That is, in this case, supervision is done based on indirect evidence instead of clear status presentation. “We only know that it is working because the amount [of entries] is decreasing […] Then we check the count and the quantity is always the same, after half an hour, 40 minutes there is the same amount: Wait! There's something wrong” (S2). “We do not have this contact, we do not see the robot acting, we do not see it working, we do not have control over the robots. I think that's what we need” (S5).

Problems happen when workers spot errors in the robot operation, yet they cannot diagnose them, less likely act on correction. “But it fails, we know that because the protocol is there… for some unknown reason the robot has not picked it, maybe because of the amount…” (S6).

Robots thus fail in making their errors clear and visible.

5.2. Associated attributes

Under associated attributes, we compile those findings related to how workers perceive the system meeting their pragmatic and hedonic needs. Besides functionality and presentation, user experience with robots results in the perception of quality attributes, both present and desired. According to the Hassenzahl model, associated attributes can be of pragmatic and hedonic nature.

The Hedonic needs category expresses the perception of qualities associated with the satisfaction with the use and the to-be goals, while the Pragmatic needs category expresses the perception of usefulness and usability, and the to-do goals. In the Hassenzahl model, hedonic needs can be categorized into three groups: evocation, identification and stimulation. Pragmatic needs are categorized into usability and usefulness.

5.2.1. Evocation

Under evocation, we grouped those findings associated with the ability of the robot to stir participant's memories, and to represent values, events, relationships or thoughts that are important to the individual. Many different feelings were evoked by workers when talking about the robots, but undoubtedly, the most important value expressed by workers is the utility of released work time, for it provided an overall increase in life quality. The robot releases time previously used in repetitive tasks to more productive and pleasant usage. Also, it released workers from overtime, which was required when a high volume of work had to be processed to meet response deadlines: “The robot releases my time and I can work on other things, dedicate myself to groups, make other interactions, look for new things” (S1).

Workers were distressed and hopeless by the work before the robot was implemented. Back then, excessive workload came from different causes: from increased demand, resulting of system failures (anomalies process) or thunderstorm season (damage compensation process); from manual comparison of several data screens of non-integrated systems; from repetitive work that required focus and attention. In comparison to that scenario, the most common perception, expressed by all the participants, is that the presence of the robot evokes calmness, tranquility and comfort that is not translated into a complete relaxation, but instead as a positive feeling of engagement: “at the same time I have to pay attention to the process and the information it is sending me, I can be a bit calm because I can count on the robot” (S8).

Due to that, failure by the robot brings back the feelings of tedious, unpleasant work: “We divide the work, a stage for each employee and we stay focused practically all day on the same thing, a very operational work… All in all, it is quite dull” (S5).

The robot evokes the feeling of confidence in the process structure and on its work. Workers perceive the robots as trustworthy colleagues: “I think our tendency is to trust their work. To go blind, that's what we did. Today, we do not look at what it's doing or worry about if it's doing right or wrong. [...] The more you know, the more you trust” (S2).

In the beginning of their adoption process, robots evoked concern and surprise. Workers were concerned about the results of implementing the robot but were positively surprised by the robot being able to process over 70% of the entries, in the first days after its launching. “It was a surprise because, although we expected it to process 70%, 80%, I did not expect that in the early days” (S3).

Feelings of loneliness and remoteness were also reported. The work with the robot is perceived as distancing people because the person works with no human collaborator and because the robot is not physical. “I do not miss the contact with the robot, I really miss human contact. I am the kind of person who likes human contact, so if I have been developing an activity for a long time without having an interaction with another person, I miss that” (S9).

Robots evoke the expectation of unlimited possibilities of growth and new opportunities: “It can evolve more ... We can achieve much more” (S3).

5.2.2. Identification

According to Hassenzahl, “people also express their self through objects” (Hassenzahl et al., 2003). We grouped those findings related to how the work with robots contributes to users' identity or desired identity under this category.

We observed evidence of workers identifying themselves with the robots' maturity growth. Participation in a successful technological intervention brought a positive feeling of belonging and co-responsibility but also of superiority as we detail hereinafter.

Workers feel that they belong to the same organizational structure as the process and the robot. If the process gets more organized and defined, they feel professionally rewarded. “We are part of the process; we end up being inside the process. If it is a messy process, if it is not going well, it automatically reflects on the professional. If you are in a process that people understand as well structured, that gives return or provides data that is relevant to the company; this is a gain for us as professionals for the company” (S5).

Owing to the impact of automation on jobs, workers feel responsible for the development of new opportunities. “If I cannot prepare people for other things, if I do not provide new jobs, new opportunities, this can lead to people getting into a situation that we have caused, of unemployment, of having nothing to do” (S1).

Working with robots has awakened feelings of maturity growth in the teams. Robots were observed in the first days from release. While the robot's maturity grew, also did the workers' trust in its work and in their own: “It was something that we built and is solid, right?! We see that it is solid” (S7).

Workers perceive they are still smarter than robots. The ability to infer situations and make quick decisions based on patterns makes human work faster than robots', which take longer to analyze all the applicable business rules. “The robot processes the invoice item by item, that is, it takes a longer time than a person to analyse. Because the person already has the expertise, he/she takes a look, and he/she knows what to do” (S4).

Due to the successful implementation of the robot, the consequent praise received from other companies and internal groups, and because of the active participation in the implementation process, workers feel intensely proud of their achievement. “I managed to reverse the signal, in the sense that I previously had to beg for [RPA] to enter an area, now the area comes [to ask for RPA]. Then you see that it progressed a good way” (S10). Even those workers who were displaced from repetitive work to higher level activities also feel proud of their career. “Through the implementation of the robot, I was able to get here. It is a feeling of victory, of success” (S3).

5.2.3. Stimulation

Again, according to Hassenzahl, people strive for personal development. When products can support this development, they have a stimulating effect. We categorized findings about the stimulation experience, both positive and negative, under stimulation.

We identified factors for stimulation in the change from tedious and repetitive work to opportunities of personal improvement. Also, stimulus come from knowledge of the process facilitating increased control and potential influence in the company. Those released from monotonous activities are stimulated by their deeper understanding of the company's business process and structures and their ability to act. “Being able to act in these causes [of mistakes] is a bit complicated, and at the same time stimulating” (S3).

They refer to this stimulus as invigorating: “I feel invigorated by the opportunity to get out of something repetitive and do something new and think differently” (S7).

Workers see that working with automation represents a professional advantage. RPA is a trend and this knowledge favors their employability. “People who work in this area of robot development, AI, they will always be prominent because that is what the market was looking for” (S2).

In the daily activities, however, work can be frustrating when the robot does not perform the tasks it was intended to: “Sometimes it's frustrating… you expect it to do [the work] and it does not, it's kind of frustrating” (S6).

5.2.4. Usability

The perception of usability is one of the two categories under the perception of pragmatic qualities. We classified reports of users' perceptions on classic attributes of usability under this category.

Workers perceive that the robot's execution is free from errors. “So far, honestly, from the part of its work that we have analysed, I have not identified any flaws, no mistakes. About the work it performs, I have no doubt that it is a good job” (S5).

The robot is fast; it is perceived as an efficient machine that will evolve to faster, more efficient: “Its speed, the capability to execute […] and thinking that he might have an ever greater capacity, I think it will get more and more efficient, faster” (S1).

Robots are perceived as effective and resource efficient. “Robots are helping us, leveraging production with more effectiveness and a little better quality” (S8). “The technology works much faster and without errors, with fewer people - one managing the tool more than running [the process].” Robot's processing was observed to be accurate; however, in some cases, accuracy is a source of issues. “Many of, let's say, of the mishaps we had with the robot is the value review. Sometimes, on behalf of a cent the robot will leave [the entry] for the exception” (S2).

Owing to robotized process quality, other processes are impacted. Workers perceive the overall process quality as improving after the deployment of the robot. “After this process [PID] is finished, the customer can make a complaint. [...] So we also monitored the complaints process. And they have decreased in function of the quality that I applied to the other stages” (S1).

5.2.5. Usefulness

Under this category, we classify snippets that address the perception of robot as a useful tool. Because robots add to the teamwork, they are seen as teammates. “I understand that the robot is like a collaborator, it gave us more strength, more agility, so I think that looking from a global way it is adding, it is joining forces, assisting all the employees involved in the process” (S5).

Usefulness is compromised by business changes. Robots are not expected to solve all the different cases; however, they are also expected to continuously expand their functionality and process coverage. Robots must evolve and also follow the changes. “Nowadays, we have a lot of exceptions, there's a lot that it tried to deal with and that it could not” (S2).

Yet, robot improvement depends on the managerial decision to invest. “We did it, developed it and we're just using it. In more than a year, we did not move. It will depend on whether someone wants to invest in it” (S2).

5.3. Anticipated consequences

The model places attractiveness and behavior changes as categories of anticipated consequences, meaning that users, after their experience with the technology, may express likeliness of acceptance and anticipate their own future. In this study, the experience itself is not anticipated but cumulative, after 18 months of RPA adoption. Yet, participants have expressed their viewpoints about acceptance and their perception of the future, in the perspectives of technology evolution in the company and their personal destiny.

Participants foresee the broadening scope of robotization, with the robots working on non-digital or non-integrated sources of relevant data and the use of intelligent robots in more complex situations. These expectations are coherent with the newer generations of RPA. “I believe that in the future we will be closer to this situation because, with the improvement of robots, they will do exactly what we want, help us to produce more, identify more mistakes, work more closely with us” (S8).

In a more distant future, they believe that robots may evolve to resemble humans “Because from what I see today of the robots being developed, it more and more tends to meet the personal needs of the human being and is becoming more and more like the human being” (S9).

As expected, workers show concern about job loss in the future “With robotization, you reduce labor. I believe that with fewer people, with the help of robots, you will end up decreasing your number of people” (S6). However, they express their innocence of the unemployment process as they are expected to have pride in their accomplishment. “My expectation is that the team, the people who use it, present it with satisfaction, saying ‘look, this is what we did,' not with that other concern that you took a job, took space ...” (S7).

Workers perceive digital technologies as agents for changes in people behavior. On the one hand, the change environment stimulates an attitude of creative restlessness, in which stagnation and apathy are not welcome. “Even by the restlessness that I think we must have. When you've just seen one thing to improve you have to think about the next one, right?!” (S7). On the other hand, they foresee an increase in interpersonal distance: “I think people are losing some of their humanity, of this human coexistence” (S1). Change in careers is also expected as workers realize that new-generation robots will promote people growth due to innovation.

5.4. Complementary categories

The analytical framework proposed in the AXE protocol contains three categories to group common statements in which participants reveal their position toward the product (unwanted and suggestions) or some important information that does not address the concept under evaluation or the activity directly linked to it (meta). Once situations perceived as undesirable could be associated with perceptions of features or qualities, they were classified under their respective categories and category unwanted was suppressed from this analysis.

5.4.1. Suggestions

Several suggestions for improvement were collected from participants' interviews, which are useful for future expansion. Workers perceive other time-consuming activities that could be replaced by a robot, freeing more time for more valuable activities. They also suggested that the management of robots' performance should be in the hands of the operational team, which could more efficiently manage the robot's schedule. “If the scheduling was our task, because we know the amount every day, [we could change the schedule] and I think we would be adding even more value” (S5). Also, participants expect more frequent revisions in the operational procedures. “Because of these changes in procedures that happen all the time, I think we should periodically stop and check if something new has come up that we can include in the robot's activity, for example, to relieve some of the effort” (S2).

5.4.2. Meta

Regarding the meta category in AXE framework, our research found several statements that did not address the usage of the robot or its intrinsic characteristics, but instead, reveal a lot about the implementation and deployment process.

The richness of these findings has motivated us to extend our understanding of the UX concept to encompass managerial factors, which show that UX is strongly affected by contextual factors and by the collectivity of workers that share the organizational situation. In order to guide our understanding, we considered Juntunen's integrative framework categories, which are presented in the following section.

6. Results of acceptance and adoption

Juntunen's integrative framework for acceptance and adoption, presented in Section C, has guided our understanding of the contextual factors that conditioned user experience [21].

She explains that an RPA decision on adoption is a top-down movement (the primary adoption) which is followed by a series of actions toward internalization or second adoption. Acceptance, understood as workers' willingness to adhere to the adoption process, is influenced by several factors and beliefs. Her framework was useful to show the complex relationship between the contextual factors and their impact on user experience. We present the definition of each contextual factors and detail their manifestation in our study in the following sections.

6.1. Innovation attributes

The first category in Juntunen's model is the set of innovation attributes. Innovation attributes express constructs associated to the innovative characteristics of the product, service or idea being adopted. Innovation attributes can be closely related to the UX constructs of perceived features and associated attributes, especially those related to pragmatic qualities.

6.1.1. Relative advantage

Relative advantage reflects the superiority of the innovation over the previous idea it is replacing. There is a clear superiority of the robots over manual work, not only because of the released work time but also because of the increased productivity. Workers express that competitivity demands a quality leap that does not depend on hiring people. “I had collaborators with potential for a larger delivery or for participation in some projects. I could not allocate them because they were servicing the high-demand process. So, to have them participating we had to wait for the low-demand season” (S1). “We already had the desire to gain speed with repetitive processes, to become more and more competitive. There is a cost issue. Sometimes, to do more, you will not achieve this by hiring more people” (S7).

6.1.2. Complexity

Complexity refers to the perception of easiness to understand or use the technology.

The perception of technology being easy to use and to understand is strongly related to constructs of usability and user experience. Complexity is not perceived as associated to RPA but to the business process, and the RPA technology is perceived as a tool to reduce complexity, as processes are standardized before being automated. However, collaboration with the robots is affected by the lack of visibility of the robots' actions. Workers must develop strategies to cope with the lack of information on robots' work. “As the time span for the robot to do this analysis may not have passed, we always pick [entries from] the day before, so we are sure that the robot was able to do the analysis of all requests” (S8).

6.1.3. Trialability

Trialability is associated to how easily individuals can experiment with the innovation, thus, this category groups snippets that express how workers have experimented with the robots. Some of the participants had the opportunity to observe the robots from the first days of release. While the robot's maturity grew, also did the workers' trust in its work.“[Immediately after robot deployment] we would take everything it had processed the day before, and we checked if it had processed correctly, even to make the corrections at the beginning of the implantation. There's always something to be done, right? And then, from the moment we saw that its margin of error was very low, 1% or 2% of the amount that came in, we did not have the need to make this verification ... [...] So we spent a month checking if what it was doing was correct” (S2). Trialability was an important factor for acceptance, because several adjustments had to be made due to the several exceptions to business rules. “On paper it is one thing, but the moment you put it to work, you are faced with various situations. But I do not say it's luck, no, I believe it's trial and error. And then, you fix it and do it again until you adjust it the way you want it”. Modifiability is also perceived as an important factor, because workers observe that changes in business rules must be implemented quickly. “If you take too long to make [the adjustment] that you have identified, it becomes obsolete” (S7).

6.1.4. Observability

Under observability, we classified those snippets that express the workers' perception of the innovation being visible to others.

Robots are observable and demanded internally. Workers report that RPA is demanded from other business areas. “I managed to reverse the signal, in the sense that I previously had to beg for [RPA] to enter an area, now the area comes [to ask for RPA]. Then you see that it progressed a good way” (S10).

Outside the company, however, the technology is not visible by clients. Clients are unaware of changes and only notice its effects. Changes are then open to interpretation, as is the case when a client claimed the company did not analyze her case because the answer came too soon: “They [the clients] said "the proof that you do not even check if there has been an incident in the grid: now you respond within one workday” (S1).

6.1.5. Job-fit

Under this category, we grouped the findings associated to the perception that the innovation is compatible with the job it must perform. This is a relevant factor for acceptance in this case. Findings on associated attributes in the user experience framework showed that, in the early days of adoption, workers perceived robots' work as reliable and effective, freeing valuable time. “It theoretically does not do the wrong thing. What it is intended to do, it does, and does it well. What is assigned for it to do, it can do and it does” (S6). “When the robot was deployed, we had a very good time of having little work to do” (S2). However, job-fit can degrade if the robot is not updated and improved. “[The robot] solves the easy cases, at least for now” (S4). “The exceptions that we could not map are not processed by the robot, and this demand is passed on to us” (S5).

6.1.6. Voluntariness

This category includes the findings related to the innovation adoption being perceived as voluntary. Workers perceive that RPA adoption is a top-down, irreversible movement. “This is the first feeling we have when it comes to information that you will participate in a process to robotize your activity. Like a bitten apple” (S3). The technology is accepted as the innovation is seen as an opportunity for evolution. “It brings the feeling that I can improve, that I can move forward” (S3). “One has to try and reinvent oneself. Otherwise you, in fact, will be left behind” (S10).

6.2. Organizational attributes

The organizational context influences the adoption process. All the attributes in Juntunen's framework were found to be relevant, but two additional organizational attributes were observed to be relevant to the case and were added to this analysis: perception of impact on clients and process improvement.

6.2.1. Compatibility as social responsibility

The compatibility attribute should reflect the innovation matching organizational needs and values. We found that workers praise the value of social responsibility in automation, which mirrors the goals in the Pact for Humanized Work Digitalization.

Workers understand that automation raises the level of requirements for hiring. “Preparing for the digital world is a fairly complex social issue as to what is required in the labor market. If today we fail to supply the labor market with people for repetitive activities, let alone for technology” (S1).

6.2.2. Organizational norms

The organizational norms attribute reflects organizational and leaders' attitude toward change. Competitivity is seen as one of such factors. Workers express that competitivity demands a quality leap that does not depend on hiring people, and the adoption of robots is a question of embracing competitivity. “We already had the desire to gain speed with repetitive processes, to become more and more competitive. There is a cost issue. Sometimes, to do more, you will not achieve this by hiring more people. So I think it was a true evolution; this deployment was positive. This specific case was so successful that we presented it and as I said, other companies came to ask how it was done” (S7).

6.2.3. Innovativeness

Innovativeness reflects the perception that the organization is early in adopting the innovation. We found organizational innovativeness to be a relevant factor as workers are proud of the company being an early adopter of RPA in Brazil.

6.2.4. Resource factors

Resource factors reflect the availability of money, skills and cooperation. Under this attribute, we placed snippets related to the company's investment in the innovation process.

Workers are aware of the company investment in RPA and of the requirement of return. Workers acknowledge that technological advances must be economically justified. “[RPA] was a bet we all made. We focused on our results and the robots brought this to us. We bet on a machine process and it was not a roulette game because it was well thought out and we were sure we would make a profit on it” (S7).

Workers understands that RPA provides no FTE reduction but a change in duties and avoiding increased costs with labor. “Sometimes the investment is not just to add, but to maintain as well” (S7). “We often failed in the process due to the lack of workforce, due to the lack of people. Today because of robotization, we can execute all the steps” (S5). “If I am reducing man-hours, I am also reducing overtime. So it has a financial impact that is also expected by the company” (S10).

6.2.5. Impact on clients

As to any company, the interface with customers is sensitive for EdP and dealt with extreme care. Besides image issues, failures in this interface may result in fines imposed by the regulatory agency. Participants expressed their concern with the impact of technology adoption on their clients. In the studied cases, the interface from robot to customer are emails that report customer's request status. It is important to notice that similarly to the workers' experience with RPA, the interface is not solely responsible for the customers' experience. The effect of automation on processes outputs also produces impact on customers' experience. In general, they expect clients to be positively affected by RPA. “If it [the robot] finds an incident, it immediately reports it to the client [that it found an incident with a causal link]” (S1) “So we improve agility. It adds value to the company and to the consumer as well. If you have damaged equipment in your home [...] without knowing if you can fix it ... it is painful [...]” (S5). “…the company is not so susceptible to error, since a person can err more than a robot” (S8).

Conversely, workers acknowledge that because consumers are not aware of robot presence or do not understand their operation, they may misunderstand agility as negligence. “They [the clients] said “the proof that you do not even check if there has been an incident in the grid: now you respond within one workday"” (S1). Also, workers perceive that depending on socioeconomic factors, clients may experience difficulties in dealing with robots. As difficulties can come from clients not understanding the company's rules well or their own rights and duties in the contract, possibly such clients would benefit from human service. “I see there's a chance we can use robotization to get closer to our customers. But you must choose well which technology will be used, and how we will use it. I think that [robots] should be used to get to know our customer well, but to serve the customer, I think maybe not” (S8). “Because we did not clearly disclose the information to our clients, that bad situation [the client experienced], he will keep it for the rest of his life” (S6).

6.2.6. Process improvement

Process improvement was also perceived as an additional organizational factor, relevant in the studied case.

Workers attribute several improvements on business processes to RPA adoption. Improvement was clearly an outcome of the standardization needed for programming the robot activity but it is also reported to be a consequence of workers having free time to analyze the business process and its exceptions. As analytical workers, they can carry out their analysis and improvement beyond the context of one process and begin to look at relationships and mutual influence between processes. RPA is reported to promote synergy of workers toward process improvement. “When we get a fault that is not the robot's, I check what the problem in the process was, act on the root cause and make the correction” (S3). “[Out of the critical period of operational work] we can work more analytically. We can give more attention [to the process], apply improvements, analyse the data the process generates, we can share this data with other areas of the company. The quality of the process has improved significantly” (S5). “So an improper handling of an anomaly will lead to a damage compensation complaint. So today I go deep into that complaint to see what happened. When I worked on the billing anomaly process I couldn't see that I was making a mistake, so I could correct it” (S3). “Because with the repetitive process being done by the robot, it created an opportunity for people to be closer and to discuss more about the process” (S7).

6.3. Individual attributes

Individual attributes express personal characteristics influential to the process of change. Some of the attributes were not verified in our interviews: personality, age, gender and communication behavior.

6.3.1. Socioeconomic factors

Socioeconomic factors were expected to influence RPA adoption, as automation is often seen as a cause of workforce reduction. In the studied case, RPA was first seen as a workforce reduction policy, but participants reported that this idea has changed over time. “When robotization started, that was my feeling here, it was taking a piece out of me, out of my activity. I thought that way. [...] Damn, I'm going to lose my job” (S3). “We know there is the fear of robotization threatening job positions and everything else, but I see the robot actually as a member of staff ” (S7).

Job vulnerability manifests itself in different ways, as pressure for productivity and dissatisfaction with the present national political and economic scenario: “Because the tendency of all companies is to have fewer people doing more work. So you will always be more demanded for more activity. […] It is not only in this company, but in the market, in general” (S2). “There's a lot of stuff involved, even the situation in the country. Now, maybe, we will not even retire. We're going to work for the rest of our lives” (S2).

6.3.2. Innovativeness

Individual innovativeness is seen as a relevant factor in the studied case. Workers express that employability values are changing from knowledge background to innovativeness. “I think that for you to deal with technology, for you to deal with machine development, you have to have a gift ... You must like it, you must study hard, but you must have great creativity. […] That view, "Oh, you will only get a good job if you have a college degree” is changing. I think now it is: "if you show that you have something different” (S2).

RPA is not the first automation technology as end-user programming was found to be practiced before robots' adoption. “The staff ends up using Visual Basic language within Excel and do many things to automate our process” (S7).

6.3.3. Expertise

Individual expertise was not seen as a factor for adoption or acceptance of RPA, but as a factor for keeping up with future advances in technology. Workers appreciate the fact that working in the company forces them to catch up with technology trends. “If you're in the company, you end up adapting to technology and keeping up with growth” (S4).

6.4. Managerial facilitation

Managerial facilitation describes the management approach to facilitate and to accelerate the adoption. Managers and technical leaders exert influence over workers. In the present case, according to other cases reported in the literature, RPA adoption was decided at higher management levels and negotiated with the business areas. The snippets under this category describe the dissemination of the innovation idea throughout the company social structure.

6.4.1. Active participation

This category includes workers perception on their participation in RPA design.

The adoption process is clearly top-down, but workers see that RPA is a top-down movement that becomes accepted as people actively participate. RPA was first seen as a workforce reduction policy but as an inevitable progress in all companies, workers also expect to be under pressure for productivity. “When robotization started, that was my feeling here, it was taking a piece out of me, out of my activity. I thought that way. [...] Damn, I'm going to lose my job” (S3). “Because the tendency of all companies is to have fewer people doing more work. So you will always be more demanded for more activity.[…] It is not only in this company, but in the market, in general” (S2).

This view changed over time, as workers perceived that employability values are changing from knowledge background to innovativeness, or creative restlessness as they called their feeling. “Even by the restlessness that I think we must have. When you've just seen one thing to improve you have to think about the next one!” (S7).

Undoubtedly, the most influential managerial attitude in adoption was the considerate engagement of professionals, providing active participation of the workforce in RPA definition, programming and deployment. Workers participated by providing relevant information for development and changes and then verifying the robots' results until the technology was perceived as mature. “The end user was the main source of information for constructing this robot. So I think that's a key point because people advocate what they have the opportunity to participate in” (S7).

Workers see that their participation is still needed for robot improvement. “We just have to have time. […]to monitor [the process], to stop and say: ‘no, we can do this differently' and then, get in touch with the people that develop the robot” (S2).

6.4.2. HRM practices

Human Resource Management activities proved influential in removing adoption barriers and in promoting extrinsic motivation. Workers report that human resources spared by robots are being applied to purposeful tasks, which are more effective for the company and more rewarding to workers. They see strategic thinking, technology and innovation as advantageous skills in the new work market scenario, and clerical, repetitive work as less valued. They see themselves as responsible for developing technological professional competence. “There was a time when the market was looking for people. Today, not anymore. The market is looking for technology” (S2). “If you remove the robots, we will have to go back to all those operational, monotonous activities that require a lot of work and do not add much value, even to the professional. We cannot improve professionally by performing these activities” (S5). “It is characteristic of the company to invest in people. [… The company] provided that many people had a college degree. And what happens? These people want to get out of the operational positions and go to analyst positions. […] Consequently, the actual operational activity, if possible, must be robotized, so that individuals have the opportunity to do the analytical work that will bring benefits to the company and to them” (S7).

6.4.3. Management of information

This attribute is related to the channels for sharing information about the innovation.

Workers praise the considerate strategy for introducing innovation, which was anchored in straightforward and clear communication.

“I think people need to be clear, right? […] What helped me a lot was a conversation I had with the manager, who explained to me how things would happen and showed me that if I participated in the development process, it would be good for my career, I would get more visibility. And this really happened.” (S3) “I am very proud to work here, of using the methodology that was used, because of the concern that the company had to take people from our team to learn how to use technology, to have these people participating in this construction process and everything. I think that the success we have here [relates] to the architecture this project had, not only the technical architecture, but using cutting-edge technologies, building a structure, having people focused and resources in case it didn't work” (S7).

The initial communication strategy was successful, but workers are currently engaged in setting new communication channels that carry the necessary information for maintenance, monitoring and dissemination. “With the implementation of the robot, I saw the need to be in touch with the business areas, asking for feedback [on the robot]. How's it going? Is it satisfying your area? Does it help you in your daily life?” (S10).

Since there are similar groups in different Federation states, communication is seen as essential, yet workers perceive that communication to be still too reactive. “We do [make some change] here and do not communicate there or the guy does there [in another State unit] and does not communicate here. He is not isolated; he just does not interact” (S10).

6.4.4. Influence strategies

This attribute is related to strategies to informally influence workers toward adoption. Besides active participation, seen as the most effective organizational strategy toward adoption, organizational alignment is also considered a relevant factor. RPA introduction requires disclosure of information from different perspectives. “We are talking from the architecture to the availability of software, the involvement of investments so that people at the operational level also engage, they buy the idea. So I think there is a concept of working, of disclosing and aligning the expectation with everyone, which must involve the needs, from the top management, to those who are there in their routine” (S10).

7. Discussion

In this section, we summarize our findings and present our reflection on their meaning, as well as on how to improve workers' experience with RPAs.

7.1. Summary of findings

Our research question was stated as “After 18 months of the decision to adopt RPA in EdP, how do workers characterize their experience with RPA technology?”. This research question was decomposed into three secondary questions, which are answered by our results in the previous sections. Although the experience phenomenon is too complex for a Manicheistic evaluation, we compile those factors that can be accountable for a clear positive or negative experience.

7.1.1. Factor for positive experience

Workers perceive RPA positively because:

1. RPA promoted process reengineering and standardization of activities resulting in more organization and efficiency.

2. They see RPA as a worker under their supervision, thus they perceive themselves to be in control.

3. RPA prompt feedback to clients sends a good message of agility.

4. RPA communicates the results of its correct operation using a minimalist and objective language.

5. The time released by the RPA operation is valuable.

6. Workers experienced more tranquility in work after the robot implementation

7. Workers are free from repetitive work and can dedicate their time to more engaging activities.

8. Robots are trustworthy.

9. Robots were surprisingly effective from the first days.

10. Work with robotization makes workers more employable.

11. Workers belong to a successful team after RPA deployment.

12. Work with robotization makes workers more valuable to the company.

13. Better understanding of processes makes workers more valuable to the company.

14. Workers feel responsible for the development of new careers.

15. Workers feel they are still smarter than robots.

16. Robots are fast and reliable.

17. Robots are effective and resource efficient.

18. Robots are useful.

7.1.2. Factors of negative experience

Workers perceive RPA negatively because:

1. There is no visibility of the process executed by RPA.

2. Cooperative work between process and humans makes humans the robot's redundancy, thus resulting in unpredictable workload.

3. Workers miss performance indicators that can help faster identification of failures and unsolved cases in which human action is required.

4. Robots' operation in limited time windows results in not reaching full performance. In this case, the team suggest that robots should be managed by the operational team.

5. There are many other conditions and processes that could be allocated to robots.

6. RPA does not make its status visible.

7. RPA does not help diagnose operation errors which can result in fines.

8. Working with robots lessen human contact and increase loneliness.

9. Workers feel frustrated when they have to return to manual activities, mainly due to failures.

10. Robot's accuracy prevents it from closing issues.

11. RPA utility is compromised by business changes.

12. Robot improvement is dependent on managerial decision to invest.

7.1.3. Workers' expectation toward technology evolution

After RPA implementation, workers' expectations regarding the future of technology and their own destiny can be summarized as:

1. Robotization will address more processes and will integrate other sources of data.

2. Robots will become more intelligent.

3. Robots will have to behave as humans.

4. There will be job losses.

5. Workers will become more creative and active.

7.2. Reflecions on the findings

In this section, we reflect on the research findings. We observed, as expected, that the experience is not formed only with the contact with the technology but is strongly influenced by the context in which this technology is inserted. Thus, the combination of Hassenzahl's framework, which focuses on the experience with technology, and Juntunen's framework, that focuses on the adoption and acceptance of technology, together, promoted a valuable tool to understand the multifaceted experience.

It is interesting to reflect on the findings by the lens of Juntunen's construct of beliefs. Her research concluded that the adoption process is influenced through the beliefs of individuals. We notice that her five influential beliefs also helps in framing user experience.

First, perceived benefits were clearly related to positive experience and behavioral change. The most influential factor, expressed by the totality of participants, has been the utility of released time both for professional and personal purposes. As the robots freed them from extremely boring and discouraging tasks, they were able to envisage a better future of more challenging and rewarding work situations. We believe that this perception may have been beneficial for the introduction of new, more sophisticated RPA technology that followed the deployment the G1 RPA (Vajgel et al., 2021).

On the other hand, perceived social pressure was influential, but in different terms as defined in the model. The negative risk of being against the social tendency was not manifested because workers perceive themselves as pioneers in RPA adoption in their environment. Thus, social pressure is an influential factor considering early adoption as an opportunity or positive risk.

Perceived effort is clearly seen as influential, although the perception of effort and its consequences varied between participants. Participants that were released from tedious activities believed that the effort put in the development of robots was worth the consequences, but those workers who were allocated to complete the work left by the robots perceive that their individual resources were not spared by the technology. The negative experience with the opacity of RPA execution can be removed by the design of dashboards that presents in real time the robot execution status, operation schedule and reasons for non-performance. This design is not straightforward in the specific RPA technology and this is a relevant improvement opportunity.

Regarding perceived need and appropriateness, workers believe RPA was a cost-effective solution for the processes in which the technology was inserted. The negative experiences stemmed from the fact that process owners do not see the possibility of full appropriation of technology, appropriation seen as the capacity to captain the evolution of technology and its conformation to needs. They they realize that without proper ownership, the cost of deployment may be wasted. because process improvement and resulting changes are inevitable. Without the power of apropriation, robots would quickly become obsolete.

Just as the perceived effort, perceived capabilities were also influential but in different ways. While some participants expressed their capabilities being challenged by the technology, those who were actively engaged in the change believed the technology helped demonstrate their capacities.

8. Conclusions

A recent literature review pointed to several challenges in RPA research. Purposefully, research on human factors was left out from the list of research challenges: “We perceive these human aspects of RPA to be similar to other technology adoption challenges, which could be addressed by the plethora of prior and ongoing IT adoption literature; hence, we have not focused on these in our formation of the research agenda” (Syed et al., 2020).

However, user research has already been proved to be a valuable strategy to guarantee that the design is suitable to human needs, expectations, habits and organizational norms. Our methodology resulted in insights that can help the company to deliver better automation to their workers in the subsequent RPA projects (Vajgel et al., 2021), and we understand that it can be useful for other companies that value human-centered design.

8.1. Methodology, limitations and future work

Regarding the methodology used, we believe that the UX framework and Juntunen's model were complementary in guiding our understanding of the factors that conditioned user experience. The UX methodology and its analysis model provided a detailed view on workers' relationship with the technology, while the acceptance and adoption model helped organize and explain the organizational and managerial factors that underlie that experience.

As an exploratory, qualitative study, our findings are limited due to the number of respondents and their roles. An interesting extension of this study would be the confrontation of workers' opinion on automation pains and gains with that of high-level managers, who were not interviewed in this study. Also, we were restricted to two processes in EdP Brazil scenarios. However, our research method can be replicated in other facilities and with other stakeholders to provide a reliable photograph of the human factors involved in the experience with RPA.

We intend to broaden our view on RPA UX by exploring other scenarios and RPA technologies. However, because the nature of UX is contextualized, we will invest on methodologies for efficient analysis of specific situations. Despite the stated limitations, we believe that our effort toward understanding workers pains and desires can be useful for future implementations of RPAs.

Data availability statement

The raw data supporting the conclusions of this article are not readily available because they may identify the participants who provided the information. Requests for access to the data should be directed to the corresponding author and are not guaranteed to be granted.

Ethics statement

Ethical review and approval was not required for the study on human participants in accordance with the local legislation and institutional requirements. The patients/participants provided their written informed consent to participate in this study.

Author contributions

LF: investigation, methodology, writing-original draft, and supervision. PC, SA-S, ST, MS, and REQ: investigation, writing-review, and editing. VD: project administration, investigation, writing-review, and editing. All authors contributed to the article and approved the submitted version.

Funding

This work was supported by the Brazilian National Energy Power Agency (ANEEL)'s R&D program. REQ was supported by São Paulo Research Foundation (FAPESP) under grant 2019/21693-0.

Acknowledgments

The authors would like to thank the Brazilian National Energy Power Agency (ANEEL)'s R&D program; grant #2019/21693-0, São Paulo Research Foundation (FAPESP).

Conflict of interest

Authors ST, MS, and VD were employed by the company EdP Brasil.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Alberth, M., and Mattern, M. (2017). Understanding robotic process automation (RPA). J. Finan. Transform. 46, 54–61. Available online at: https://www.capco.com/Capco-Institute/Journal-46-Automation/Understanding-robotic-process-automation

Google Scholar

Al-Rahmi, W. M., Yahaya, N., Aldraiweesh, A. A., Alamri, M. M., Aljarboa, N. A., Alturki, U., et al. (2019). Integrating technology acceptance model with innovation diffusion theory: an empirical investigation on students' intention to use e-learning systems. IEEE Access 7, 26797–26809. doi: 10.1109/ACCESS.2019.2899368

CrossRef Full Text | Google Scholar

Brasil da Saúde, M., and de Saúde, C. N. (2016). “Resolução nº 510, de 7 de abril de 2016,” in Diário Oficial da União, 44–46.

Google Scholar

Chen, J. Y., and Barnes, M. J. (2014). Human-agent teaming for multirobot control: a review of human factors issues. IEEE Trans. Hum. Mach. Syst. 44, 13–29. doi: 10.1109/THMS.2013.2293535

CrossRef Full Text | Google Scholar

Corbin, J., and Strauss, A. (2008). Strategies for qualitative data analysis. Basics of qualitative research. Techn. Proc. Dev. Grounded Theory 3, 9781452230153. doi: 10.4135/9781452230153

CrossRef Full Text | Google Scholar

Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q. 13, 319–340. doi: 10.2307/249008

CrossRef Full Text | Google Scholar

Devarajan, Y. (2018). A study of robotic process automation use cases today for tomorrow's business. Int. J. Comput. Techn. 5, 12–18. Available online at: http://www.ijctjournal.org/Volume5/Issue6/IJCT-V5I6P3.pdf

Google Scholar

Ernst and Young. (2015). Robotic Process Automation White Paper. Rio de Janeiro: Ernst and Young Assessoria Empresarial Ltd.

Google Scholar

FDIs, I. (2008). 9241-210 (2009). Ergonomics of Human System Interaction-Part 210: Human-Centered Design for Interactive Systems (Formerly Known as 13407). Geneva: International Organization for Standardization (ISO).

Google Scholar

Fontanillo Lopez, C. A., Li, G., and Zhang, D. (2020). Beyond technologies of electroencephalography-based brain-computer interfaces: a systematic review from commercial and ethical aspects. Front. Neurosci. 14, 611130. doi: 10.3389/fnins.2020.611130

PubMed Abstract | CrossRef Full Text | Google Scholar

Gegner, L., and Runonen, M. (2012). “Anticipated experience evaluation handbook,” in Proceedings of 8th International Design and Emotion Conference, eds J. Brassett, P. Hekkert, G. Ludden, M. Malpass, and J. McDonnell (London).

Google Scholar

Goodrich, M. A., and Schultz, A. C. (2008). Human-Robot Interaction: A Survey. Boston, MA: Now Publishers Inc.

Google Scholar

Group, I. C. A. (2017). IEEE guide for terms and concepts in intelligent process automation.

Google Scholar

Hassenzahl, M. (2003). “The thing and i: understanding the relationship between user and product,” in Funology (Dordrecht: Springer), 31–42.

Google Scholar

Hassenzahl, M. (2013). “User experience and experience design,” in The Encyclopedia of Human-Computer Interaction 2 (Dordrecht).

Google Scholar

Hassenzahl, M., Burmester, M., and Koller, F. (2003). “Attrakdiff: ein fragebogen zur messung wahrgenommener hedonischer und pragmatischer qualität,” in Mensch & Computer 2003 (Wiesbaden: Springer), 187–196.

Google Scholar

Hassenzahl, M., and Tractinsky, N. (2006). User experience-a research agenda. Behav. Inf. Technol. 25, 91–97. doi: 10.1080/01449290500330331

CrossRef Full Text | Google Scholar

Houy, C., Hamberg, M., and Fettke, P. (2019). Robotic process Automation in Public Administrations. Bonn: Digitalisierung von Staat und Verwaltung.

Google Scholar

ISO, I. (2011). IEC25010: 2011 systems and software engineering-systems and software quality requirements and evaluation (square)-system and software quality models. Int. Organ. Stand. 34:2910. Available online at: https://www.iso.org/standard/35733.html

Google Scholar

Ivančić, L., Suša Vugec, D., and Bosilj Vukšić, V. (2019). “Robotic process automation: systematic literature review,” in International Conference on Business Process Management (Springer), 280–295.

Google Scholar

Juntunen, K. (2018). Influence of Contextual Factors on the Adoption Process of Robotic Process Automation (RPA): Case study at Stora Enso Finance Delivery. Uppsala: Uppsala University. p. 96. Available online at: https://www.diva-portal.org/smash/record.jsf?pid=diva2%3A1223866

Google Scholar

Lacity, M., Willcocks, L., and Craig, A. (2016). “Robotizing global financial shared services at royal DSM,” in The Outsourcing Unit Working Research Paper Series (London: Cham).

Google Scholar

Leshob, A., Bourgouin, A., and Renard, L. (2018). “Towards a process analysis approach to adopt robotic process automation,” in 2018 IEEE 15th International Conference on e-Business Engineering (ICEBE) (Xi'an: IEEE), 46–53.

Google Scholar

Lewicki, P., Tochowicz, J., and van Genuchten, J. (2019). Are robots taking our jobs? a roboplatform at a bank. IEEE Softw. 36, 101–104. doi: 10.1109/MS.2019.2897337

CrossRef Full Text | Google Scholar

MarketsandMarkets (2017). “Robotic Process automation market by process operation - global forecast 2022,” in Robotic Process Automation Market (Northbrook, IL).

Google Scholar

Montero, J. C., Ramirez, A. J., and Enríquez, J. G. (2019). “Towards a method for automated testing in robotic process automation projects,” in 2019 IEEE/ACM 14th International Workshop on Automation of Software Test (AST) (Montreal, QC: IEEE), 4–47.

Google Scholar

Nauwerck, G., and Cajander, A. (2019). “Automatic for the people: Implementing robotic process automation in social work,” in Proceedings of the 17th European Conference on Computer-Supported Cooperative Work-Demos and Posters. European Society for Socially Embedded Technologies (EUSSET) (Salzburg).

Google Scholar

Österholm, M. (2010). “Beliefs: a theoretically unnecessary construct?” in Sixth Congress of the European Society for Research in Mathematics Education. January 28th-February 1st 2009 (Lyon: Institut National de Recherche Pédagogique), 154–163.

Google Scholar

Prietch, S. S., and Filgueiras, L. V. L. (2015). “Technology acceptance evaluation by deaf students considering the inclusive education context,” in IFIP Conference on Human-Computer Interaction (Cham: Springer), 20–37.

Google Scholar

Ratia, M., Myllärniemi, J., and Helander, N. (2018). “Robotic process automation-creating value by digitalizing work in the private healthcare?” in Proceedings of the 22nd International Academic Mindtrek Conference (Tampere), 222–227.

Google Scholar

Rogers, E. M., Singhal, A., and Quinlan, M. M. (2014). Diffusion of Innovations. New York, NY: Routledge.

Google Scholar

Roto, V., Law, E., Vermeeran, A., and Hoonhout, J. (2011a). “UX white paper: Bringing clarity to the concept of UX,” in Dagstuhl Seminar Proceedings, eds J. Hoonhout, E. L. -C. Law, V. Roto, and A. Vermeeran (Dagstuhl: Schloss Dagstuhl – Leibniz-Zentrum fur Informatik).

Google Scholar

Roto, V., Law, E. L. -C., Vermeeran, A., and Hoonhout, J. (2011b). “10373 abstracts collection - demarcating user experience,” in Dagstuhl Seminar Proceedings (DagSemProc) eds J. Hoonhout, E. L. -C. Law, V. Roto, and A. Vermeeran (Dagstuhl: Schloss Dagstuhl – Leibniz-Zentrum fur Informatik), 1–26. doi: 10.4230/DagSemProc.10373.1

CrossRef Full Text | Google Scholar

Rutaganda, L., Bergstrom, R., Jayashekhar, A., Jayasinghe, D., Ahmed, J., et al. (2017). Avoiding pitfalls and unlocking real business value with RPA. J. Finan. Transform. 46, 104–115. Available online at: https://www.capco.com/Capco-Institute/Journal-46-Automation/Avoiding-pitfalls-and-unlocking-real-business-value-with-RPA

Google Scholar

Schellen, E., Bossi, F., and Wykowska, A. (2021). Robot gaze behavior affects honesty in human-robot interaction. Front. Artif. Intell. 4, 663190. doi: 10.3389/frai.2021.663190

PubMed Abstract | CrossRef Full Text | Google Scholar

Stople, A., Steinsund, H., Iden, J., and Bygstad, B. (2017). Lightweight it and the it function: experiences from robotic process automation in a norwegian bank. Bibsys Open J. Syst. 25, 1–11. Available online at: https://www.researchgate.net/publication/321319770

Google Scholar

Syed, R., Suriadi, S., Adams, M., Bandara, W., Leemans, S. J., Ouyang, C., et al. (2020). Robotic process automation: contemporary themes and challenges. Comput. Ind. 115, 103162. doi: 10.1016/j.compind.2019.103162

CrossRef Full Text | Google Scholar

Vajgel, B., Corrêa, P. L. P., Tóssoli De Sousa, T., Encinas Quille, R. V., Bedoya, J. A. R., Almeida, G. M. D., et al. (2021). Development of intelligent robotic process automation: a utility case study in brazil. IEEE Access 9, 71222–71235. doi: 10.1109/ACCESS.2021.3075693

CrossRef Full Text | Google Scholar

Venkatesh, V., Thong, J. Y., and Xu, X. (2012). Consumer acceptance and use of information technology: extending the unified theory of acceptance and use of technology. MIS Q. 36, 157–178. doi: 10.2307/41410412

CrossRef Full Text | Google Scholar

Vishnu, S., Agochiya, V., and Palkar, R. (2017). Data-centered dependencies and opportunities for robotics process automation in banking. J. Finan. Transform. 45, 68–76. Available online at: https://ideas.repec.org/a/ris/jofitr/1593.html

Google Scholar

Willcocks, L. P., Lacity, M., and Craigm, A. (2015). The IT Function and Robotic Process Automation. London: The London School of Economics and Political Science.

Google Scholar