Planning and Evaluating Remote Consultation Services: A New Conceptual Framework Incorporating Complexity and Practical Ethics

Establishing and running remote consultation services is challenging politically (interest groups may gain or lose), organizationally (remote consulting requires implementation work and new roles and workflows), economically (costs and benefits are unevenly distributed across the system), technically (excellent care needs dependable links and high-quality audio and images), relationally (interpersonal interactions are altered), and clinically (patients are unique, some examinations require contact, and clinicians have deeply-held habits, dispositions and norms). Many of these challenges have an under-examined ethical dimension. In this paper, we present a novel framework, Planning and Evaluating Remote Consultation Services (PERCS), built from a literature review and ongoing research. PERCS has 7 domains—the reason for consulting, the patient, the clinical relationship, the home and family, technologies, staff, the healthcare organization, and the wider system—and considers how these domains interact and evolve over time as a complex system. It focuses attention on the organization's digital maturity and digital inclusion efforts. We have found that both during and beyond the pandemic, policymakers envisaged an efficient, safe and accessible remote consultation service delivered through state-of-the art digital technologies and implemented via rational allocation criteria and quality standards. In contrast, our empirical data reveal that strategic decisions about establishing remote consultation services, allocation decisions for appointment type (phone, video, e-, face-to-face), and clinical decisions when consulting remotely are fraught with contradictions and tensions—for example, between demand management and patient choice—leading to both large- and small-scale ethical dilemmas for managers, support staff, and clinicians. These dilemmas cannot be resolved by standard operating procedures or algorithms. Rather, they must be managed by attending to here-and-now practicalities and emergent narratives, drawing on guiding principles applied with contextual judgement. We complement the PERCS framework with a set of principles for informing its application in practice, including education of professionals and patients.

Context: the Shift to Remote Consultations in the UK

On 30th July 2020, it was announced that all healthcare consultations in the UK should henceforth be “remote by default,” not just during the pandemic but indefinitely (1). Remote services had been introduced in March 2020 to manage the spread of COVID-19 and reduce the burden on the National Health Service (NHS). Patients seeking an appointment with their general practitioner (GP) had to make contact by electronic form or telephone before getting a return call from a clinician (2). In secondary care, much routine outpatient activity was canceled or undertaken remotely (3).

In the UK as elsewhere, the expansion of phone, video, and e-consultations were part of a wider pandemic-driven shift to technology-mediated care (4). These changes included directing patients to approved websites for self-management, an expanded telephone and electronic advice service featuring a bespoke COVID-19 Clinical Advice Service (CCAS) to supplement NHS 999 (for emergency calls) and NHS111 (for urgent and out-of-hours care), expansion of electronic prescribing, and introduction of virtual wards for oximetry monitoring (3).

Primary care clinicians welcomed the infection control benefits, empty waiting rooms, and slackening of red tape that accompanied the initial shift to remote (5). But they also warned of an uncomfortable “brave new world” characterized by fewer consultations overall, loss of continuity of care, threats to the clinician-patient relationship, inequalities of access, and clinical risks (6–9). Lay media coverage of remote consultations mirrored this pattern, with an initial positive response followed by stories of inaccessibility, missed diagnoses and patients feeling “fobbed off” with phone calls (10, 11). Whereas, politicians and the press emphasized the transformative potential of new technologies, most remote consultations before and during the pandemic occurred by telephone (3, 12–15).

Policy talk about remote health services was typically technology-focused, depicting these as state-of-the-art, high-quality, efficient, and safe (16). Arguments against remote forms of care tended to be patient-focused, highlighting possible disbenefits such digital exclusion. The benefit-harm balance of remote care is fundamentally an ethical issue. Technology-focused arguments often imply a broadly utilitarian ethical standpoint—that an efficient, remote-by-default service will minimize (though not eliminate) suffering, maximize overall benefit to the population and free up clinician time to create further benefits. Patient-focused arguments are more deontological, focusing more on what good care for the individual patient means, especially the clinician's duty to provide care for every patient to the best of his or her professional ability. However, critiques of remote modes of consulting can also be defended using utilitarian arguments, since clinical care is complex, relationship-based, nuanced, and emergent. If the consultation is narrowly transactional and fails to capture these wider dimensions, it is suboptimal, leading to inefficiency and exacerbating unfairness.

To guide and theorize our research on remote consultation services, our team set out to develop a conceptual framework. We began with a generic framework which some of us had developed previously, called NASSS—non-adoption, abandonment, and challenges to scale-up, spread and sustainability of technology-supported services (17). The NASSS framework has been widely used across a range of settings for evaluating and explaining the fortunes of various kinds of health technology projects [see for example (18–20)] and draws on complexity theory (21). NASSS explores the dynamic interaction between multiple domains (the technology, the people and so on) in a complex system and how these domains and their interdependencies have evolved—and are likely to evolve further—over time. There are theoretical parallels with other complexity-informed implementation and evaluation frameworks including i-PARIHS (22), normalization process theory (23), and the consolidated framework for implementation research (24).

But whilst NASSS was our theoretical starting point, we quickly discovered that it did not fully explain all of our empirical data—especially findings around the clinical relationship and the ethics of allocating appointment type. By adapting NASSS to fit our emerging empirical findings and relevant literature, we sought to develop a framework which would guide both the prospective planning and real-time evaluation of remote consultation services. Importantly, both NASSS and the adaptation described here (PERCS) are explanatory frameworks to guide a holistic interpretation of a complex and evolving phenomenon. They do not offer predictive certainty and are not intended to be applied formulaically.

The remainder of this paper is structured as follows. In section Empirical Context: Mixed-Method Studies of Remote Care Before and During the Pandemic, we describe the methods, study designs, sampling frames and datasets from our studies on remote consultation services in the UK. We highlight the elements of those datasets—especially qualitative research with staff and patients and an online Delphi study—which directly informed the theoretical work presented in this paper. In section Findings, we present and explain the PERCS framework and a set of guiding principles to inform its application. In section Discussion we contextualize our findings in a wider literature review and discuss implications for policy, practice, education and research.

Empirical Context: Mixed-Method Studies of Remote Care Before and During the Pandemic

Overview and Data Sources

Prior to the pandemic, we studied the organizational challenges associated with roll-out of video consultations across multiple clinical directorates in the UK's largest acute hospital trust (25–27), including sub-studies on physical examination by video (28, 29). We also undertook contract research for the Scottish Government to evaluate the national roll-out of video consultations—an initiative that was driven partly by the policy goal of reducing carbon footprint and travel costs from remote settings (30). Others in our team have studied help-seeking behavior in urgent care settings, including NHS 999 and NHS111 (31, 32). Insights from these studies informed our theoretical work.

Since the pandemic began, we have been involved in three separately-funded but theoretically related case studies. Details of ethics approvals are given at the end of the paper, and full empirical reports of these studies are in preparation for publication elsewhere. All studies were of mixed-methods design but predominantly qualitative, using interviews, ethnography, and documentary analysis to generate and follow an emerging story of change, using quantitative data to illustrate and enrich the story.

First, we were funded by the Scottish Government (June–October 2020) to extend our evaluation of the video consultation service (branded “Near Me”) to cover the early months of the pandemic to August 2021 (33). This study covered both primary and secondary care. It included 60 h of ethnographic observation; 223 interviews with healthcare staff, patients, and national-level stakeholders (policymakers, professional leaders, industry); quantitative analysis of automated activity reports on over 69,000 consultations (including over 18,000 patient assessments of consultation quality); and analysis of policy documents and implementation plans.

Second, we were funded by the UK Research and Innovation COVID-19 Emergency Fund from June 2020 to November 2021 for a study called Remote by Default, which addressed remote care in general practice. This study involves interviews (over 100 to date) with healthcare staff, patients and national-level stakeholders, as well as following four locality case studies in south London, Oxfordshire, Devon, and south Wales. Especially relevant to the development of PERCS were four online focus groups involving 19 participants (clinicians, support staff, and patients), four facilitated cross-sector workshops (held via Zoom) which brought together ~160 national policymakers, clinicians, patients, and other stakeholders, and a four-round Delphi study (described in detail below) of ethical principles and decisions relating to remote consulting.

Third, we were funded by a medical charity from June 2020 to July 2021 to study the roll-out of video consultations across the UK. The Health Foundation Video Consulting (HFVC) study involved a quantitative survey of current practice (to over 800 NHS staff), qualitative follow-up interviews with a sample of 40 of these (repeated longitudinally with a sub-sample of 20 as the pandemic unfolded), interviews with 10 patients, and two group discussions involving 15 patient and public representatives. This study also included 7 locality case studies of video consulting services—four in secondary care (in London, Norfolk, Oxfordshire, and Cumbria) and three on group video clinics in primary care (in England, Scotland, and Wales).

In each of these studies, our research question addressed the individual-, organizational-, and system-level challenges to introducing remote consultation services at pace and scale and routinizing such services. We used an embedded virtual researcher-in-residence model: each case study had an assigned member of the research team who built relationships with key informants, developed an understanding of local issues and contingencies, and coordinated data collection and feedback. An external advisory group with a lay chair and patient representation met 4-monthly.

Developing the PERCS Framework

In all the above studies, we undertook an initial phase of data management and familiarization. Interviews, focus groups, ethnographic field notes, and workshop write-ups were transcribed, de-identified, entered onto NVivo software (or, in one sub-study, an Excel spreadsheet for pandemic-related practical reasons) and broadly coded (for example under headings such as “staff attitudes” or “technical infrastructure”). Quantitative data (e.g., waiting times, uptake rates and trends, survey responses) were analyzed using descriptive statistics. For each case study, narrative synthesis was used to pull together an initial familiarization document based on the first ~3 months' data; this document was refined iteratively as data collection progressed.

Researchers discussed their ongoing findings in 2-weekly team meetings. Initially we used NASSS (described above) to organize and theorize emerging findings, but when this proved inadequate for explaining some of our key findings, we began to adapt it into the PERCS framework. Particularly germane to this revision of our previous theoretical work were the transcripts and field notes from focus groups and workshops involving clinicians, support staff, and patients who discussed the challenges of remote consulting in the real world.

Across our in-pandemic datasets, a number of themes were evident which had received little or no emphasis in pre-pandemic research on remote consultations (see Discussion for summary of that literature). For example:

- Access to care, especially patients' difficulty getting the kind of appointment they wished and conflict with support staff associated with this;

- Remote clinical assessment of patients, especially those with a complex clinical picture such as evolving symptoms, co-morbidities, learning difficulties, cognitive impairment or vision or hearing difficulties;

- Clinical risk management and safety-netting, especially in acutely unwell patients (e.g., deciding which patients with suspected COVID-19 to see in person or send to hospital);

- Continuity of care, especially how to create coherence over time when different consultation modalities and more than one clinician were involved;

- The patient's home and family context, notably concerns about privacy and safety (e.g., whether patients are safe from eavesdropping or coercion);

- Digital exclusion, including the impact of poor-quality technologies, low digital literacy and material aspects of the home environment (and, even more so, being homeless);

- Organizational efficiency such as the nature of, and reasons for, “double-handling” (e.g., when patients were transferred from e-consultation to phone or from phone to video or face-to-face);

- Staff wellbeing, especially stress and loss of motivation from lack of face-to-face contact and a perception that consultations have become more transactional;

- Technical issues, including technical infrastructure; interoperability (especially interfacing with the electronic patient record) and in patients' homes (e.g., linking with multiple devices), and the wasted time and stress caused by unreliable internet access and technologies;

- Sociotechnical issues such as use of workarounds—for example, patients strategically downplaying symptoms on e-consultations to avoid an algorithm-driven diversion to a call handling service;

- New forms of clinician-patient interaction, such as the growing importance of asynchronous messaging between patients and clinicians (in the form of SMS, e-consultations, and emailing); and the unfamiliarity and strangeness of communication via video, including some people's dislike of seeing their own face or body on video and a sense of inappropriate intimacy (e.g., a doctor's discomfort at seeing a patient part-naked in a bedroom rather than on an examination couch);

- Knock-on effects on vulnerable groups, for example the impact on people with drug dependency (e.g., reduced in-person pick-ups for controlled drugs has resulted in fewer opportunities for direct clinician-patient engagement);

- New opportunities for inter-organizational collaboration e.g., for clinicians to engage in multiagency work to prepare and plan patient care; and

- The national regulation and procurement context, which shaped not only what technology might be available and affordable in any organization, but also who could use it and for what.

These themes were interdependent, illustrating the “complex system” aspect of the phenomenon we were researching. They informed development of draft PERCS framework as we used it to manage and synthesize data both within locality case studies and in cross-case analysis. We discussed this emerging work in workshops with our professional and lay advisory group members. Further refinements were made iteratively as we progressed with our analysis.

Two prominent themes in our data were organizations' digital maturity in providing remote consultations and the need for proactive measures to improve digital inclusion. We drew out different qualitative dimensions of digital maturity from our data, and also constructed a semi-quantitative digital maturity scale, incorporating work by others (34–37). Given that previous attempts at introducing a comprehensive digital maturity scale for the UK NHS had been abandoned as unworkable (36, 38), we deliberately favored a short, pragmatic five-point scale over a detailed, exhaustive one. We also teased out qualitative dimensions of organizations' efforts at digital inclusion and included reference to these in the digital maturity scale. To be classed as fully digitally mature, an organization thus had to address digital inclusion as well as install and use advanced technology.

Developing the Guiding Principles

Evidence from policy announcements and our elite policymaker interviews revealed a relatively confident vision of an efficient, safe and accessible remote consultation service delivered mostly through state-of-the art digital technologies and implemented via rational allocation criteria and quality standards. But our front-line interviews, ethnographic observation, surveys, and service usage statistics revealed that, as in pre-pandemic times, the telephone was the dominant technology used and that decision-making at multiple levels—system-level strategic decisions about setting up remote consultation services, administrative decisions to allocate particular kinds of appointment (phone, video, e-, face-to-face), and clinical decisions when consulting remotely—was fraught with inherent contradictions and tensions. There were tensions, for example, between quality and efficiency, between demand management and patient choice, and between the needs and preferences of some patients and those of other patients in the context of limited staffing and material constraints (e.g., availability of safe waiting areas which limited face-to-face appointments during the pandemic). These tensions led to both large- and small-scale ethical dilemmas for managers, support staff, and clinicians. Few of these dilemmas could be resolved by resort to standard operating procedures or algorithms.

The contradictions and tensions in our data reflected clinical practice more generally. As Hunter (39) has argued, clinical decisions are governed not by hard and fast rules but by shared rules of thumb or guiding principles which she calls maxims, some of which are contradictory (e.g., “ignore the anecdotal” but “listen to the patient's story”). Maxims, which tend to be passed on orally from more to less experienced practitioners, encapsulate shared understanding and wisdom; they suggest a potential way forward but are high-level enough to be flexibly applied. Maxims and other high-level guiding principles require an understanding of the circumstances in which the rule should be applied rather than formulaic replication. Through experience, reflection on practice and discussion with more experienced colleagues, clinicians learn which guiding principle to use—and hence, what is the right thing to do—in a particular situation.

The situational application of such guiding principles has an important time dimension. With time, one constellation of symptoms, signs, and contextual influences will evolve to a subtly different constellation, generating clues as to the nature of the illness and its likely regression or further progression. A raised temperature observed at a single time point may persist, settle or become a swinging pyrexia. Only when this longitudinal pattern emerges do the relevant maxims to guide next steps become salient. Whilst Hunter focused narrowly on clinical maxims, case-based ethical reasoning more broadly—whether, for example, to invoke child protection measures or ask a social prescriber to get involved in someone's life—operates along similar lines. We need to know the story, its context, and how it is unfolding over time.

At the time of our fieldwork, there were few established guiding principles for provision of remote services (exceptions include “see all young infants face-to-face promptly” and “don't provide end-of-life care remotely”). To develop some more, we focused on a subset of our data relating to ethical tensions. We were aware of the many different ways in which ethical dilemmas in healthcare may be theorized—including utilitarian, rights-based, fairness or justice, virtue ethics, and the common good (40), and took the view that each of these philosophical lenses might prove useful in different scenarios. We also sought to consider ethical issues at individual-, organizational-, and system-level.

To develop and refine some ethical principles to guide application of the PERCS framework, we used the Delphi method—a well-established semi-structured approach to working toward consensus among experts (41). Steps include defining a problem, selecting a panel of experts (including, in this case, both clinicians and service users), supplying a summary of evidence and outlining key uncertainties, collecting quantitative (numerical scores), and qualitative (free text) data on a set of statements, feeding scores and comments back to panel members and repeating until residual disagreement cannot be resolved. Advantages of this method include practicality (it can be done online, asynchronously, without specialist tools), anonymity (participants know the average group score but not individuals' scores), and iteration (through feedback, outliers are prompted to either defend their response to the group or change it) (42, 43).

To prepare background evidence for the Delphi panel, we began by examining the tensions in our empirical data through various ethical lenses (presented in Findings). In this way, TG and RR developed a “long list” of 60 draft principles. We organized these into 30 partially contradictory pairs (for example, “when consulting remotely, deal directly with the patient and ensure privacy” and “relatives or carers may provide technical, linguistic, or physical assistance in remote consultations”) and grouped them under four categories (e.g., practice organization, matching appointment type with patient needs).

The four-round Delphi study was conducted virtually and asynchronously. Participants were recruited from our field sites, our advisory group and their networks, and a social media invitation (Twitter). We recruited and obtained emailed consent from an initial sample of 69, though 19 of these did not attempt any items (mostly because they found the exercise too difficult). The characteristics of those who participated are shown in Table 1; to avoid appearing intrusive we did not ask those identifying as patients or carers for details of occupation or illnesses.

TABLE 1

Table 1. Sample for Delphi study of underpinning principles.

For round 1, we provided participants with a summary of research on remote consultations, guidance from professional and regulatory bodies (e.g., Royal College of General Practitioners, General Medical Council) and instructions. We invited them to consider each pair of guiding principles on three dimensions—whether they belonged in the assigned category; whether to include this principle as worded or in an amended form (scored on a five-point scale from “definitely include” to “definitely exclude”); and suggestions for improving wording. Since the list was long, we split it into two parts, each sent to half the participants in round 1.

Participants were given 3 weeks to complete the survey and were sent two reminders. Responses were collected on the survey platform Survey Monkey (pandemic-related home-working precluded access to more specialized tools). The software generated a single document with the spread of quantitative scores and collated free-text comments. We analyzed qualitative data thematically and through discussion. An initial plan to analyse quantitative data statistically was abandoned when it became clear that presenting our draft principles in deliberately contradictory pairs had led to confusion and irritation (which we discuss briefly in the Findings section). We used people's free text suggestions to drop many principles (merging those that were near-duplicates), improve wording and introduce several new principles suggested by participants.

In round two, we abandoned the attempt to present the principles in contradictory pairs. We circulated a simple list of 25 principles and asked participants to repeat the scoring exercise on two dimensions—whether each item should be retained (quantitative, five-point scale) and whether its wording should be changed (qualitative, free text). We used descriptive statistics to chart the level of agreement with each principle, and thematic analysis to organize free-text comments. This process was repeated in a third round. In a short fourth round, a single item (the only one on which 80% agreement had not yet been reached) was circulated to confirm a minor rewording suggested by several participants.

Findings

The PERCS Framework for Planning and Evaluating Remote Consultation Services

The PERCS framework is shown in Figure 1. As noted above, it was adapted from the previously-published NASSS framework (17). We consider each PERCS domain in turn.

FIGURE 1

Figure 1. The PERCS (Planning and Evaluating Remote Consultation Services) framework and underpinning principles of healthcare quality and ethics.

The reason for consulting considers not just the illness but why the patient wishes to be seen—or why the clinician wishes to see them—now. It draws attention to urgency and rate of progression, and to what the patient wants and expects (e.g., advice, treatment, referral). Relevant to this domain is the vast biomedical evidence base on the origin, progression and treatment of diseases and risk states, and sociological and psychological evidence on why people approach health services at particular points in their illness (44). Help-seeking for urgent care is a social process comprising illness work (to make sense of symptoms), moral work (to justify choice of service and help-seeking behavior) and navigation work (to access services) (31). Some reasons for consulting are clinician- or system-driven (e.g., invitations for screening or long-term condition monitoring). Clinical allocation criteria, perhaps built into algorithms, may suggest (though not determine) a type of appointment.

Several examples from our dataset illustrated the subtleties of the reason for consulting and why rigid algorithms or allocation criteria may prove too brittle to guide practice. Clinicians described over-riding such systems (for example when triaging e-consultations) because they had a gut feeling that a seemingly minor aspect of the history could indicate serious illness (45) or because they had safety-netting concerns that a particular patient should be checked more frequently than the recommended interval. They talked about the need to be alert to the possibility of the “doorknob phenomenon” —that people sometimes seek a consultation for one reason but bring in the real reason late in the encounter, hence withholding key information at the triage stage. The following quote illustrates the complex and subtle nature of gut-feeling reasoning (the doctor is highly reflexive and pulls together small fragments of the history and aspects of the patient's personality and past behavior, to justify their heightened concern):

“there was somebody recently who was, you know, feeling very low, young man had kind of quite a long history of off and on of things … not always … minimizing it … very, you know, quite … I work in deprived areas quite … you know, quite, quite a high powered job, and in his 30s, and anyway I ended up thinking there's something more here, and so I brought him down to the surgery […] you know how you get a sense of stuff” (GP, HFVC_BND)

The patient domain, which strongly influences the reason for consulting, includes the patient's (or parent's) attitudes toward illness in general and remote consulting in particular. It embraces the patient's identity, values, personality traits, and socio-cultural background—features a sociologist might call habitus (46)—, their health beliefs (44), health literacy and digital literacy, socio-demographic characteristics (47, 48) and their personal experience of illness or disability. All these factors may help explain why the patient seeks a particular mode of consultation—and also why a clinician or support staff member may decide that it is in the patient's best interests (or not) to be seen face-to-face, as the following quote illustrates:

“I had a really interesting conversation with an 80-year-old with diverticular disease and he'd been phoning up every couple of weeks about his abdominal pain, which you routinely get with diverticular disease and there's no real cure and it's really just about managing his bowels. But nobody has sat down with him and drawn a section of the bowel and shown little pockets of diverticulae and how that's not gonna change and what he needs to do is to keep his bowels moving. So he doesn't understand what's going on for him and me trying to explain something over the phone like that is really hard to an 80-year-old who is losing every third or fourth word because his hearing is not brilliant anyway.” (GP, RBD_FM3)

The clinical relationship domain considers the clinician-patient interaction, including what might be called non-transactional aspects of the consultation—for example, the role of the clinician as professional witness to suffering (49), or what Balint, writing in a more medically-dominated era, called “the doctor as the drug” (50). It addresses the level of trust and positive regard that can be developed and maintained in different types of consultation. It also embraces the clinician's—and perhaps also the administrative team's—knowledge of the patient and their illnesses and consulting patterns. This knowledge may have been built over many years (or, alternatively, may be recent and fragmented). One of the most frequent and consistent themes in our dataset was a sense from both clinicians and patients that the clinical relationship is necessarily built—at least initially—face-to-face. For example:

“I think it's hard to build relationships over video and it's even harder over a phone call, OK? To really get to know somebody so that they trust you and you trust them and it's working. So, a lot of my phone calling is fine because it's people I've known for years because I've been in the same practice for eighteen and a half years and [distortion]. my patients.” (GP, HFVC_IT)

The home and family domain incorporates considerations of how the material features, physical layout and symbolic spaces of the home influence key issues such as privacy and comfort when consulting remotely (51). Our datasets include examples of patients who had no home or whose homes were small, crowded, lacking privacy (notably houses of multiple occupation, but also many family homes under conditions of lockdown) or not connected to the Internet. One GP expressed concern that a patient was consulting from the lavatory because this was the only quiet and private space available. Another described connecting to a teenager's bedroom where a parent may have been present but off camera. This domain includes socio-cultural aspects of family life such as family structure, English fluency and the expectations placed on different members, and the extent to which family and neighbors contribute to the “lay consultations” that either promote or prevent a subsequent attempt to access formal health care (31, 52). It also includes wider social determinants of health, notably the level of socio-economic deprivation and the education and digital literacy of family members who may or may not be able to support the patient. More prosaically, consulting behavior patterns previously learnt in the context of the clinic must now be transposed to the home space, which may produce dissonance, as the following quote illustrates:

“Well I had to like do. say to him, ‘Look, sit down, please sit down,' and then he. I just felt so daft and like, ‘Look, the doctor's going to tell you off if you don't sit down,' because of course [doctor] wasn't present. But that did kind of work. He kind of sat down and then he looked at her. But having him sat down and then like the others [siblings], you know, running around playing, is hard to prevent him from going off and playing.” (Mother talking about trying to do video consultation with young child, HFVC_BV)

Aspects of the home and family domain—especially material spaces and privacy—may also be relevant to staff working from home during or beyond the pandemic. The only available space for one (female) GP to consult from was the kitchen table, necessitating complex negotiations with other family members about access to the food supply.

The domain of technologies includes aspects of design—such as aesthetic appeal, functionality, technical performance and ease of use—as well as dependability. Telephone, for example, has less functionality video but is more dependable and familiar. Some software products (e.g., accuRx) potentially allow the clinician to instantly convert a phone consultation to video in real time, thus avoiding double-handling, but products which lack this functionality require a new appointment.

The technology domain also incorporates supply chain and costs—including up-front investment, maintenance and repair, and the risks to the service if a key supplier pulls out of the market or raises prices (thereby embracing the “value proposition” domain from the NASSS framework). Costs of technologies are typically unevenly distributed across the healthcare system. Telephone advice services, underpinned by advanced computer decision support systems, can be costly to set up and maintain and require new work skills to operate (32); their introduction may generate supply-induced demand (53). During the pandemic, some locality-based clinical commissioning groups became tied into commercial contracts with video software providers whose products were approved at speed at the height of the crisis and offered “free” or low-cost for the first year.

The staff domain includes a quantitative workforce component (are there enough of the right kind of staff?) as well as a qualitative component relating to staff capabilities, confidence, attitudes, and well-being. Staff attitudes are grounded in professional norms and values including deeply-held concerns about quality and safety of care. Many staff—rightly or wrongly—still considered face-to-face consulting to be the “gold standard” professional norm:

“They all miss face-to-face because that's what they were trained to do, they all work very, very closely with the children.” (Children's community services manager, HFVC_IC)“

And then also the barriers from, I think, healthcare professionals, so, 'How could I possibly have a difficult discussion over a video consult? How could I possibly discuss do not resuscitate [by video]? How could I break bad news when I can't touch my patients? How would I show empathy without touch? How can I be near and yet far? How could that be possible? You know, I need good eye contact, I need, you know, more non-verbals, I need to see the home, I need to see the person,' those sorts of thoughts maybe.” (Palliative care consultant, HFVC_TQ)

Our findings affirmed published claims that remote consulting is cognitively burdensome (54), excludes the physical touch on which the therapeutic relationship has traditionally depended (55), threatens continuity of care (7), brings greater clinical uncertainty, and makes the personal care afforded by the extended, face-to-face “long consultation” more difficult (8, 56, 57). For example:

“… it's maybe a slippery slope, you know, I'm trying my best in my consultations, I'm really focusing. I'm really tired from listening on the phone, because it is exhausting. I'm really tired from these Zoom meetings, because it's very different to have this sort of like overload of, you know, concentration and physically trying to do it, overload of information. And you feel like, you're failing, but actually, you're … but you can't do anything else to say, when you feel like you're not giving as good as services, as you know, somebody who was properly trained in all this, but you feel quite inadequate that, you know, but yet, you feel well, at least I'm doing something. But I need to be safe, and what are the unknown unknowns…?” (Primary care nurse, RBD_IM)

Our interviews with less experienced clinicians confirmed published concerns from clinical trainees that the format of digital consultations has made it harder for them to observe and consult alongside more experienced practitioners, thereby accumulating the tacit knowledge that enables them to manage clinical risk confidently (58), as the following quote (reflecting on the collective learning that had happened pre-pandemic) illustrates:

“…we were having clinical meetings in the surgery. We [had] a new doctor who joined us, who is young, who [had] just started, who [was] very hot on a weekly meeting that is mandated. And then she always [had] a sweet way of saying, Well, I'm the youngest here. I want to start with my worrying cases. case number one, case number two, what would you do here? What would you do there? And then, of course, we [would] all contribute.” (GP, RBD_OS)

These complex effects of the remote modality on front-line staff added to the huge pandemic-related toll on their morale and well-being, which included the physical and emotional stress of clinical work, the personal risk of infection and its complications, and personal bereavements and other losses (59–61).

The healthcare organization domain incorporates the organization's general innovativeness and its readiness for a particular innovation (62). Innovative organizations tend to be large, competently led and with clear strategic vision, functionally differentiated, non-hierarchical and with adequate slack (people and resources that can be channeled into new projects to get them up and running). Readiness generally involves both top and middle-management support, absence of opponents, and a formal assessment of innovation-system fit (e.g., a business case), though such requirements were often relaxed during the pandemic. This domain also includes the complex question of how technologies become—or why they fail to become—taken up and maintained in organizations (23). Within the organization, the innovative technology or service model needs to be actively implemented and “normalized” or “routinized” (i.e., made business-as-usual), replacing one familiar and comfortable set of interactions with one that feels unfamiliar and awkward, as this quote illustrates:

“I think we all miss. you didn't realize it at the time, but actually there's something very comforting about just having a [face-to-face] morning surgery booked and. we all moan about it but, you know, it's actually. you don't realize until it's gone actually how cocooned and in your zone you felt. And I guess it's how practices organize themselves.” (GP, HFVC TU)

Normalization includes supporting staff to make sense of the new technology in the context of their work; engaging them to participate; coordinating their implementation efforts; and monitoring benefits and costs (23). Each of these phases can be time-consuming and exhausting, particularly when implementing technologies across multiple organizations (32). Small wonder that healthcare staff (both clinical and administrative) described themselves as “maxed out” and “knackered,” with little capacity for learning the new work routines needed for remote care.

Another aspect of successful organizational-level innovation is the need for staff members to champion and support the innovation. An example from our dataset is the administrative staff member who won an award for supporting the spread and scale-up of video consultations in community services. This digitally-literate individual grasped the vision for the new service, became an early champion for it and a super-user (someone to whom others felt they could go for help). They influenced fellow staff members to give it a try (including calling in favors), developed bespoke training for colleagues and patients, and pushed for measures to help embed the change (e.g., better digital platforms) with support from management. Slow adoption of remote services in some other organizations or departments could often be attributed to absence of such individuals: nobody opposed the innovation, but nobody enthused about it either.

The wider system domain incorporates the powerful phenomenon of inter-organizational influence and learning, in which early-adopting organizations pass on insights and resources to those coming on stream later, as well as attention to how policy context may support—or interfere with—organizational innovation (62). An example of cross-system influence at national level was how Scotland, which had been an early and successful adopter of video consulting across the country, supported the other UK jurisdictions (Wales, England, and Northern Ireland) with advice and re-usable resources (e.g., protocols, patient information). Learning from models of good practice elsewhere tended to be a positive experience, whereas public release of performance data—in which each organization or department is compared competitively with others (for example in the proportion of consultations undertaken remotely)—was viewed negatively by our participants.

An important aspect of the policy context for remote consultations during the pandemic was the use of emergency powers to slacken red tape in relation to approved suppliers and purchasing rules, allowing organizations to effectively obtain and use any system deemed locally appropriate and workable (63).

“…essentially, we were given the word from the Trust that had come down from the region saying, 'Look, all is forgiven so to speak. For the time being, till we get a proper governance structure in place, just anything goes.'” (Service manager, HFVC_QA)

Our interviewees depicted these changes positively—indeed, they considered that the changes would have been impossible or much delayed without them—but they also expressed concern about what would happen when organizations returned to business as usual, since the longer-term trade-offs between convenience and regulatory control (e.g., data privacy) are unclear. Profit-oriented providers offering only a limited range of remote services and targeted primarily at low-risk patients such as young professionals were part of the landscape before the pandemic, competing with local GP practices. Their success during the pandemic at a time of regulatory laissez-faire raises questions about quality, safety, and equity of care going forward (64).

The wider system domain also includes policies that affect remote services indirectly. An example is cross-government measures to address the climate emergency through a range of green policies. NHS England, for example, recently announced a vision for a “net zero [carbon] NHS,” including a major contribution of new modalities for delivering care (65). The evidence base for achieving greener health services by means of remote care is limited and contested, but one study suggests that a substantial reduction in carbon footprint could be made (66).

Another aspect of the wider system domain in PERCS is underpinning infrastructure. Even before the pandemic, there was a strong policy push in the UK to build and strengthen the enabling technical infrastructure for a digitally-supported NHS (67–70), resulting in some relatively state-of-the-art elements such as the Health and Social Care Network (referred to as “The Spine”), along with various local and regional legacy components (some of which posed challenges relating to coverage and interoperability). But as we found, not everywhere in UK has even basic broadband connection:

“we're in rural Northern Ireland, there is still areas of the peninsula that haven't got Wi-Fi; they have got no sort of connection to things. Broadband speeds in many parts of Northern Ireland are one megabyte per second. So, just video consultation just physically doesn't work.” (Service manager, HFVC_QA)

Infrastructure also includes human elements (e.g., organizational roles, routines, and relationships), shared understandings and historical path-dependencies (such as legacy technologies and commercial contracts), and the regulatory and professional standards noted above (27, 37). As Gkeredakis et al. (4) have pointed out, during the pandemic a sense of urgency, relaxing of regulations and policy investment in technological solutions produced many new technological components but there was not time to strengthen either technical or human infrastructure, yet successful embedding and use of these novel solutions will be “contingent upon the openness, distributedness, recombinability, re-programmability, and accessibility of digital technologies” (page 2).

The initial draft of the PERCS framework consisted of the seven domains listed above, along with a temporal domain in which the dynamic interaction between all the other domains is followed over time, using narrative as a summarizing and synthesizing tool. One England-based clinician, for example, reflected that whilst video services had been broadly successful during the pandemic, there was little enthusiasm from her colleagues to continue this mode, and she doubted the service would last beyond the pandemic. She planned to return to Scotland where there was a long-term strategy and vision to extend video-based clinical services.

We subsequently added two side panels to the PERCS framework to place special emphasis on digital maturity (of the organization) and digital inclusion (for the population it serves). Whilst these concepts are to some extent subsumed within the central domains, the terms are increasingly widely used in healthcare circles and are worth exploring in their own right.

The theme of digital inclusion recurred in our data. For example:

“We have pockets of significant deprivation in [City name] and people just don't have the IT capability or the connectivity. We've had a lot of difficulty around connectivity, and I am not IT, so I'm not sure if that is largely down to the device that a patient or service user has, or whether it's down to the actual Internet or, you know, whatever the infrastructure is for. do you know what, I actually don't know the terminology but you know what I mean?” (Hospital clinician, HFVC_EY)

“…particularly with the with the outreach population, it's so easy to ignore them. You know, they are not jumping up and down and making a fuss and they can't jump up and down and make a fuss because they can't fill the forms in, because they've got to have internet access to fill the form in to complain. Yeah, it's really it's quite insidious. Really, how it, how it excludes people, and then it stops people complaining about it.” (GP, RBD_CK50023)

Digital inclusion should be considered in relation to inequalities more generally. Tudor Hart's (71) inverse care law states that people most in need of health care are least likely to seek it or receive it; the law reflects two mutually-reinforcing phenomena—worse health in deprived localities and barriers to accessing healthcare in those same localities (47, 48, 72, 73). Patients who already suffer the multiple jeopardy of poverty, low health literacy, poor housing, weak social networks, psychological stress (e.g., from fear of crime) and—for some—language and cultural discordance now face an additional problem of digital inequalities, defined as differential access to healthcare depending on digital access, digital literacy or both (74). These barriers are part of a wider digital shift whereby many aspects of life—insurance, banking, local government, education, travel, and holidays, and many more—are increasingly presented to the citizen, client or customer as “digital first,” thereby excluding (partially or wholly) those unable to access them in this way (75).

The digital divide needs to be studied at a granular level, not merely in terms of the presence or absence of Internet access (76) but also in terms of how much bandwidth, data, IT literacy and skills, and power (e.g., over who in the household has use of the computer or smartphone) people have (77, 78). Digital inclusion is high on the policy agenda in the UK and elsewhere (79, 80). Some groups (e.g., the neuro-atypical and some people with mental health conditions) are said to feel more comfortable with remote consultations than face-to-face, though anecdotal evidence outweighs rigorous research on this topic (81).

Drawing on previous research and policy work (76, 78, 80, 82–84) as well as our own empirical data, we conceptualized digital inclusion as requiring three kinds of measures. First, diversity of provision, to ensure that patients and carers are able to select from a wide range of options to suit their particular access needs and preferences; such options would ideally be co-designed with patients and carers. Second, digital access support—either directly from the organization or indirectly by signposting to third-party or community providers (libraries, community networks)—providing access to the equipment needed for remote services; help to acquire and use the skills to engage with digital health care and support to create coherence between multiple providers offering digital services in different ways. This might include upskilling individuals but also, where necessary, major infrastructural projects to improve broadband connection to remote localities. Third, provision of non-digital alternatives for people who are unable or currently unwilling to access care remotely.

Examples of digital inclusion efforts in our dataset included flexibility to accommodate patient preferences (some teenagers, for example, preferred to consult with the video switched off; some parents preferred to video their child doing a task and send the video to the therapist rather than the child perform live to camera), signposting to local digital literacy programmes, and making it clear to patients that face-to-face appointments were available on request. Co-design with digitally excluded groups is recommended by advocacy organizations (82). However, our dataset included numerous examples of capacity constraints which limited organizations' abilities to undertake co-design work, provide a high degree of flexibility or provide sufficient face-to-face slots to accommodate all requests.

Healthcare organizations participating in our studies were at different levels of digital maturity (for which a dictionary definition is “a measure of an organization's ability to create value [financial and non-financial] through digital”), as the following contrasting quote illustrates:

“When the COVID hit, obviously we had access to video consulting and we got on with it, but it very quickly petered out. So, we went to total telephone triage; we did some video consults. We got pictures sent to us via email and things like that. The functionality of our platforms … was generally quite poor; had a lot of problems connecting. So yeh, we did it for a bit, but I would say that we never exceeded probably ten percent of what we were doing actually using video. And now I don't use video at all. Only one of my partners is continuing to use it now and again.” (GP, HFVC_YN)

Our digital maturity scale is shown in Table 2. It has three linked elements. First, readiness—the extent to which the organization has the strategic alignment, leadership and resources needed to plan and deliver an appropriate range of remote services, including measures to address digital inequalities. Second, capability—the extent to which such services, including digital inclusion measures, are already technically present and up and running. Third, organizational infrastructure—the extent to which the underpinning material, regulatory, and human resource frameworks are in place to support further development of remote services.

TABLE 2

Table 2. Digital maturity scale for healthcare organizations in relation to remote services.

Findings on Practical Ethics of Remote Consultations

Evident in our interviews with clinicians were the four widely-cited principles of practical medical ethics—beneficence (act in the patient's best interests), non-maleficence (do no harm), autonomy (respect the patient's right to choice and self-determination), justice (treat people fairly, especially when allocating scarce resources). These principles were originally proposed by Beauchamp (who leant toward a consequentialist or utilitarian position, focusing on outcomes of decisions for the patient and others) and Childress (a deontologist focusing more on the clinician's duties) (85), and extended to consider scope of application by Gillon (86). Beauchamp and Childress (87) later added four further principles based on behavioral norms—veracity (tell the truth), privacy (ensure no intruders or eavesdroppers), confidentiality, and fidelity (avoid conflicts of interest such as the potential for personal profit). All these principles featured prominently—often as professional norms assumed to be self-evident—in our empirical data. They are reflected in guidance such as the General Medical Council's Duties of a Doctor (88), to which some clinical interviewees referred.

Additional ethical tensions related to staff well-being and redistribution of the work of caring (often shifting to the patient or lay carer). These themes resonated with writing by feminist philosophers (89–92) on the ethics of care, including the hidden work and emotional labor of low-status workers such as receptionists and unpaid carers. Held's taxonomy (91) divides the ethics of care into personal (focusing on an individual's commitment and accountability to the person they are caring for), political (focusing on various kinds of inequality in caring—such as fairness, human rights, and inclusivity) and global (caring for the planet and its sustainability—and hence caring for future generations).

A related concept is what May et al. (93) have called burden of treatment—the additional burden placed on the patient when they are asked to “self-manage” —with the sickest and most vulnerable carrying the greatest burden. Burden of treatment also includes the effort needed from the patient to access services (31, 93), which technologies (especially e-consultations) could potentially alleviate. Pols (92) has developed what she calls an empirical ethics of [technology-supported] care which considers technologies not as inanimate tools which make the clinical interaction more or less efficient (and as “cold” artifacts that contrast with “warm” human care) but as relational actors which, if creatively selected and “tinkered with” to fit specific situations, can enhance and support the personal ethical commitments and warm care relations between people.

Finally, our empirical data raised organizational- and system-level ethical issues, many of which could be mapped either to the US Institute of Medicine's (94) six dimensions of quality—safety, efficiency, patient-centeredness, timeliness, effectiveness, and equity—or to more recent work on organizational and system resilience—for which redundancy may be needed to help weather stress—and sustainability (95, 96). While these are quality principles rather than ethical principles, they draw implicitly on many of the ethical principles described above and have become widely used by those planning, developing, and evaluating services. These various ethical and quality frameworks were added as underpinning pillars to the PERCS diagram.

Of the 50 participants who completed round 1 of the Delphi survey, many put in free text comments that the paired principles were “contradictory” and “confusing.” This had been our intention, but it was clear that whilst clinicians work comfortably with contradictory principles in their day to day work, they experienced strong cognitive dissonance when asked to confront them in a desktop exercise without a real, practical situation in mind.

All but one of the 25 revised principles in round 2 were supported in principle by 80% or more respondents, but many had suggestions for changes in wording (e.g., be less paternalistic, reduce jargon, split a maxim into two). All but one of the 26 revised principles in round 3 were supported by 80% or more participants with only minor suggestions for further revisions to wording; the final item was supported by 71%, with several respondents all suggesting omission of a particular phrase. After re-circulating this one item in a brief fourth round, there was high agreement on all 26 principles, which are listed in Table 3.

TABLE 3

Table 3. Guiding principles to inform application of the PERCS framework.

Discussion

Summary of Main Findings

We have used selected elements of a large empirical dataset, drawn from multiple UK-based empirical studies, of the introduction and scale-up of remote consultation services both before and during the pandemic to develop the Planning and Evaluating Remote Consultation Services framework. PERCS, developed as an explanatory framework for analyzing research findings, has 7 domains—the reason for consulting, the patient, the clinical relationship, the home and family, technologies, staff, the healthcare organization, and the wider system—and considers how these domains evolve over time; it also focuses on the organization's digital maturity and digital inclusion efforts.

The PERCS framework enabled us to identify and explore how dynamic interactions between individual-, organizational-, and system-level factors influenced how remote consultation services were established and delivered (or not) in different local and regional settings. It also allowed us to surface a key paradox of the pandemic, namely the mismatch between policy vision and practical reality. Both during and beyond the pandemic, policymakers envisaged an efficient, safe, and accessible remote consultation service delivered through state-of-the art digital technologies and implemented via rational allocation criteria and quality standards. In contrast, our empirical data revealed that strategic decisions about setting up remote consultation services, allocation decisions for appointment type (phone, video, e-, face-to-face), and clinical decisions when consulting remotely were fraught with contradictions and tensions, leading to both large- and small-scale ethical dilemmas that were either unique to the remote modality or greatly exacerbated by it. These dilemmas—which included how far to use technological triage to manage demand in an under-funded, under-staffed system; when to welcome support from the patient's relatives and friends in a remote consultation; when to accept a compromised physical examination rather than bring the patient in for a full in-per son assessment; how far informational continuity might compensate for relational continuity; and how much time and resource to put into mitigating digital inequalities—could not be resolved by standard operating procedures or algorithms. Rather, dilemmas had to be managed emergently by attending to here-and-now practicalities. To complement the PERCS framework, we used a Delphi process to construct a set of principles for informing the practical ethics of its application.

Our data affirm previous research (described below) which showed that the challenges of establishing and running remote consultation services operate at multiple levels: political (various interest groups may gain or lose from the introduction of new service models), economic (costs and benefits may be unevenly distributed across the system), organizational (remote consulting requires new roles and workflows), technical (dependable links and high-quality audio and images are needed), relationally (because of altered interpersonal interactions), and clinical (patients are unique; some examinations require contact; and clinicians have deeply-held habits, dispositions, and norms).

Strengths and Limitations

Our work to develop a multi-level theoretical framework for remote consultation services has three main strengths. First, we built on previous frameworks for organizational and technological innovation which had focused on the dynamic interaction between multiple influences in a complex system (17, 62); these have been widely-used and stood the test of time. Second, the combined empirical dataset which we used to refine and extend those frameworks was large and diverse; it included several pre-pandemic studies as well as a government-funded evaluation and two in-pandemic research studies, incorporating extensive primary data from surveys, ethnographic observations, interviews, focus groups, cross-sectoral workshops, and a Delphi panel. Third, PERCS was extensively revised as we worked through analysis of the different datasets until all elements of the data could be explained with reference to the framework.

There are, however, several limitations to this work. All our empirical studies were conducted in the UK, so the PERCS framework and linked guiding principles may not be transferable to contexts very different from the UK without further adaptation (we invite collaborations from researchers in contrasting settings). Pandemic restrictions made ethnography impossible for the later months of the study, and whilst we obtained research ethics approval to video and audiotape clinical consultations, this aspect of the work has so far proved impossible in practice. Hence our current insights are based more on what people said was happening than on direct observation. The Delphi exercise to develop guiding principles for applying the PERCS framework was undertaken on a relatively small sample and should be replicated and also tested prospectively.

PERCS is an explanatory framework, not a deterministic or predictive tool. In other words, the framework and guiding principles help prompt the development of rich narratives and contextualized explanations, including ideas about what could or might happen in the future. They are not intended to determine fixed relationships between variables or firm predictions about what will happen.

Comparison With Previous Research

Findings from our in-pandemic research on remote consultation services contrast sharply with research on such services undertaken before the pandemic which—in retrospect—was incomplete, skewed and lacking granularity. This earlier research, summarized briefly below, also failed to capture the complexity, messiness and associated ethical tensions of remote consultations across a range of settings once these modalities move from being evaluated in a tightly-controlled trial setting to contributing a major part of mainstream services.

A key driver for earlier research had been the hypothesis that remote models would increase efficiency of care. For this reason, many studies had emphasized measures of efficiency including repeat appointments, staff workload (including knock-on workload for other sectors), length of consultation, and the proportion of remote appointments that get converted to face-to-face (thereby double-handling a problem), as well as addressing technical feasibility, user satisfaction, clinical quality and safety, and operational considerations (97). Study designs had included randomized controlled trials, qualitative interviews, mathematical modeling, and detailed micro-analysis of verbal interactions and physical movements. Study participants had been carefully selected, usually excluding anyone considered high-risk.

Previous research on telephone consultations is surprisingly sparse and supports no firm conclusions, though several studies have suggested that double-handling may reduce efficiency (32, 98–102). There was very little research on e-consultations prior to the pandemic, and findings were limited (12, 103–107). One quantitative survey followed up 756 e-consultations in general practice and found that most generated either a telephone (32%) or face-to-face (38%) consultation (108). In contrast, there have been dozens of randomized controlled trials comparing video with face to face appointments in low-risk patients with stable long-term conditions (109–114). Overall, patients randomized to video did no worse clinically and were no less satisfied than those randomized to usual care, and that costs (where measured) were similar. However, almost all primary studies were underpowered and had highly-selected samples. Qualitative studies had indicated high patient and staff satisfaction but frequent technical problems (113–115).

Before the pandemic, there had been very few case studies of the introduction of remote consultation services in real-world settings. Most such studies lacked descriptive detail; both the clinical condition and the technology were often described in bland and brief ways, omitting the nuances that might have explained variations in findings across studies (116). The few detailed real-world studies in the literature suggest that efforts to implement remote consultation services even without an ongoing pandemic are logistically complex, resource-intensive and often stymied by technical and regulatory challenges (18, 26, 27, 115, 117, 118). Levels of remote consulting outside the research setting were very low pre-pandemic (12, 13, 119). Reasons for this included perceived increase in workload and stress, confidentiality concerns, technical problems, and demographics (e.g., the elderly were often cited as a group who found technology difficult).

Peer-reviewed research on the process and impact of the shift to remote consultation services during the pandemic is limited at the time of writing. Observational studies documented a sharp increase in remote consultations (telephone, e- and video, in that order) during the first wave in the UK (14, 15). Many findings from these studies aligned with our own: for example, that staff experienced the shift as organizationally and professionally challenging but considered it justified for safety reasons; they later reported feeling tired and under pressure as well as concerned about the loss of in-person care, threats to the therapeutic relationship, potential for missed diagnoses especially in deprived and vulnerable groups, and a rising backlog of unmet need. In surveys of staff and patients, most described their experience of technology-mediated care as “positive” but also questioned whether remote clinical care was as good as face-to-face (3). The most common concerns voiced by NHS staff were inadequate technological infrastructure and the fact that new technologies do not work for everyone. Other studies have described problems of increased antimicrobial prescribing (120), reduced ordering of diagnostic investigations (121) and delayed diagnosis of cancer (122, 123) attributed to the pandemic-related shift to remote consultations.

Implications for Policy, Practice, Education, and Further Research

We anticipate that the multi-level PERCS framework will have a number of uses and benefits. First, we hope it will help researchers, as well as planners and policymakers, conceptualize the introduction and delivery of remote consultation services as complex interventions in complex systems rather than as tools or technologies with predictable impact and fixed effect sizes. Many interacting factors need to be taken into account, and the fortunes of remote consultation programs will unfold differently in different circumstances. Those who lead healthcare organizations may wish to set goals to improve their digital maturity and their efforts at digital inclusion. Second, we hope that clinicians will find the framework useful when considering how best to deliver excellent care to the populations they serve—especially when managing risk. The best kind of consultation for a particular patient at a particular time, taking account of the needs of other patients and staff, cannot be decided formulaically, but we hope that the guiding principles will help inform ethical allocation decisions and high-quality remote consulting. Third, we propose that use of the framework by those designing systems of care may not only reduce digital inequalities, but also lead to reductions in wider inequalities in care and burden of treatment. Finally, we believe the framework could prove useful in both undergraduate and postgraduate education, especially for promoting rich learning through reflection on practice.

We recommend further research into the application of the framework and the principles, especially the in-depth analysis of hard cases which raise particular ethical challenges.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The studies involving human participants were reviewed and approved by NHS East Midlands Leicester Central Research Ethics Committee (REC ref 20/EM0128; IRAS ID: 283196 and subsequent amendments). The patients/participants provided their written informed consent to participate in this study.

Author Contributions

All authors contributed to the empirical studies. TG conceptualized the framework, drafted the framework with RR and (to a lesser extent) other authors, and drafted the paper. TG led the Delphi study with support from RR and GW. RR is a case site lead for Remote by Default, led the focus groups and key cross-sector workshops, and wrote vignettes. SS leads a work package on Remote by Default, and provided theoretical expertise and led senior stakeholder interviews. RB is a case site lead for Remote by Default. CP leads a work package for Remote by Default. TG, RR, RB, LM, and SW contributed clinical, some social science and technology expertise, and other authors contributed social science and technology expertise. TG is guarantor. All authors selected and supplied empirical data which illustrated and shaped the PERCS framework. All authors from RB to GW contributed approximately equally and are listed in alphabetical order. All authors have read and approved the final manuscript.

Funding

This research was funded from the following sources: Scottish Government (Technology Enabled Care Programme), National Institute for Health Research (BRC-1215-20008), UK Research and Innovation via ESRC (ES/V010069/1), Wellcome Trust (WT104830MA), and The Health Foundation, an independent charity committed to bringing about better healthcare for people in the UK (2133488). Additional support was provided to RB by the NIHR through the Peninsula Applied Research Collaboration (PenARC).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

We thank the research participants including patients, healthcare staff, and wider stakeholders who all gave generously of their time while working under considerable pressure. We thank Charlotte Thompson-Grant for administrative support.

References

1. Hancock M. The Future of Healthcare (Speech, 30th July). London: UK Government (2020).

2. NHS England. Advice on How to Establish a Remote ‘Total Triage' Model in General Practice Using Online Consultations. London: NHS England (2020).

Google Scholar

3. Horton T, Hardie T, Mahadeva S, Warburton W. Securing a Positive Health Care Technology Legacy from COVID-19. London: Health Foundation (2021).

Google Scholar

4. Gkeredakis M, Lifshitz-Assaf H, Barrett M. Crisis as opportunity, disruption and exposure: exploring emergent responses to crisis through digital technology. Inform Organ. (2021) 31:100344. doi: 10.1016/j.infoandorg.2021.100344

CrossRef Full Text | Google Scholar

5. Marshall M, Howe A, Howsam G, Mulholland M, Leach J. COVID-19: a danger and an opportunity for the future of general practice. Br J Gen Pract. (2020) 70:270–1. doi: 10.3399/bjgp20X709937

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Khan N, Jones D, Grice A, Alderson S, Bradley S, Carder P, et al. A brave new world: the new normal for general practice after the COVID-19 pandemic. BJGP Open. (2020) 4:bjgpopen20X101103. doi: 10.3399/bjgpopen20X101103

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Gray DP, Freeman G, Johns C, Roland M. Covid 19: a fork in the road for general practice. BMJ. (2020) 370:m3709. doi: 10.1136/bmj.m3709

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Swinglehurst D, Dowrick C, Heath I, Hjörleifsson S, Hull S, Misselbrook D, et al. ‘Bad old habits'… and what really matters. Br J Gen Pract. (2020) 70:485–6. doi: 10.3399/bjgp20X712745

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Greenhalgh T, Rosen R. Remote by default general practice: must we, should we, dare we? Br J Gen Pract. (2021) 71:149–50. doi: 10.3399/bjgp21X715313

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Mroz G, Papoutsi C, Greenhalgh T. ‘From disaster, miracles are wrought': a narrative analysis of UK media depictions of remote GP consulting in the COVID-19 pandemic using Burke's pentad. Med Hum E-public. (2021). doi: 10.1136/medhum-2020-012111. [Epub ahead of print].

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Mroz G, Papoutsi C, Rushforth A, Greenhalgh T. Changing media depictions of remote consulting in COVID-19: analysis of UK newspapers. Br J Gen Pract. (2021) 71:e1–9. doi: 10.3399/BJGP.2020.0967

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Brant H, Atherton H, Ziebland S, McKinstry B, Campbell JL, Salisbury C. Using alternatives to face-to-face consultations: a survey of prevalence and attitudes in general practice. Br J Gen Pract. (2016) 66:e460–6. doi: 10.3399/bjgp16X685597

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Atherton H, Brant H, Ziebland S, Bikker A, Campbell J, Gibson A, et al. Alternatives to the face-to-face consultation in general practice: focused ethnographic case study. Br J Gen Pract. (2018) 68:e293–300. doi: 10.3399/bjgp18X694853

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Murphy M, Scott LJ, Salisbury C, Turner A, Scott A, Denholm R, et al. Implementation of remote consulting in UK primary care following the COVID-19 pandemic: a mixed-methods longitudinal study. Br J Gen Pract. (2021) 71:e166–77. doi: 10.3399/BJGP.2020.0948

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Joy M, McGagh D, Jones N, Liyanage H, Sherlock J, Parimalanathan V, et al. Reorganisation of primary care for older adults during COVID-19: a cross-sectional database study in the UK. Br J Gen Pract. (2020) 70:e540–7. doi: 10.3399/bjgp20X710933

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Greenhalgh T, Procter R, Wherton J, Sugarhood P, Shaw S. The organising vision for telehealth and telecare: discourse analysis. BMJ Open. (2012) 001574. doi: 10.1136/bmjopen-2012-001574

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Greenhalgh T, Wherton J, Papoutsi C, Lynch J, Hughes G, A'Court C, et al. Beyond adoption: a new framework for theorizing and evaluating nonadoption, abandonment, and challenges to the scale-up, spread, and sustainability of health and care technologies. J Med Internet Res. (2017) 19:e367. doi: 10.2196/jmir.8775

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Greenhalgh T, Wherton J, Papoutsi C, Lynch J, Hughes G, A'Court C, et al. Analysing the role of complexity in explaining the fortunes of technology programmes: empirical application of the NASSS framework. BMC Med. (2018) 16:66. doi: 10.1186/s12916-018-1050-6

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Abimbola S, Patel B, Peiris D, Patel A, Harris M, Usherwood T, et al. The NASSS framework for ex post theorisation of technology-supported change in healthcare: worked example of the TORPEDO programme. BMC Med. (2019) 17:233. doi: 10.1186/s12916-019-1463-x

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Dyb K, Berntsen GR, Kvam L. Adopt, adapt, or abandon technology-supported person-centred care initiatives: healthcare providers' beliefs matter. BMC Health Serv Res. (2021) 21:1–13. doi: 10.1186/s12913-021-06262-1

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Greenhalgh T, Papoutsi C. Studying complexity in health services research: desperately seeking an overdue paradigm shift. BioMed. (2018) 16:95. doi: 10.1186/s12916-018-1089-4

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Harvey G, Kitson A. PARIHS revisited: from heuristic to integrated framework for the successful implementation of knowledge into practice. Implement Sci. (2015) 11:1–13. doi: 10.1186/s13012-016-0398-2

PubMed Abstract | CrossRef Full Text | Google Scholar

23. May CR, Mair F, Finch T, Macfarlane A, Dowrick C, Treweek S, et al. Development of a theory of implementation and integration: normalization process theory. Implement Sci. (2009) 4:29. doi: 10.1186/1748-5908-4-29

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implementat Sci. (2009) 4:50. doi: 10.1186/1748-5908-4-50

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Shaw S, Wherton J, Vijayaraghavan S, Morris J, Bhattacharya S, Hanson P, et al. Advantages and limitations of virtual online consultations in a NHS acute trust: the VOCAL mixed-methods study. Health Serv Deliv Res. (2018) 6:1–135. doi: 10.3310/hsdr06210

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Greenhalgh T, Shaw S, Wherton J, Vijayaraghavan S, Morris J, Bhattacharya S, et al. Real-world implementation of video outpatient consultations at macro, meso, and micro levels: mixed-method study. J Med Internet Res. (2018) 20:e150. doi: 10.2196/preprints.9897

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Greenhalgh T, Wherton J, Shaw S, Papoutsi C, Vijayaraghavan S, Stones R. Infrastructure revisited: an ethnographic case study of how health information infrastructure shapes and constrains technological innovation. J Med Internet Res. (2019) 21:e16093. doi: 10.2196/16093

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Seuren LM, Wherton J, Greenhalgh T, Cameron D, A'Court C, Shaw SE. Physical examinations via video for patients with heart failure: qualitative study using conversation analysis. J Med Internet Res. (2020) 22:e16694. doi: 10.2196/16694

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Seuren LM, Wherton J, Greenhalgh T, Shaw SE. Whose turn is it anyway? Latency and the organization of turn-taking in video-mediated interaction. J Pragmat. (2021) 172:63–78. doi: 10.1016/j.pragma.2020.11.005

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Wherton J, Greenhalgh T. Evaluation of the Attend Anywhere / Near Me Video Consulting Service in Scotland, 2019-20. Edinburgh: Scottish Government (2020). Available online at: https://www.gov.scot/publications/evaluation-attend-anywhere-near-video-consulting-service-scotland-2019-20-main-report/pages/2/ (accessed October 16, 2020).

Google Scholar

31. Turnbull J, Pope C, Prichard J, McKenna G, Rogers A. A conceptual model of urgent care sense-making and help-seeking: a qualitative interview study of urgent care users in England. BMC Health Serv Res. (2019) 19:481. doi: 10.1186/s12913-019-4332-6

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Turnbull J, Pope C, Rowsell A, Prichard J, Halford S, Jones J, et al. The work, workforce, technology and organisational implications of the ‘111' single point of access telephone number for urgent (non-emergency) care: a mixed-methods case study. Health Serv Delivery Res. (2014) 2:1–135. doi: 10.3310/hsdr02030

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Wherton J, Greenhalgh T. Evaluation of the Near Me video Consulting Service in Scotland During COVID-19, 2020. Edinburgh: Scottish Government (2021).

Google Scholar

34. McCulloch P, Altman DG, Campbell WB, Flum DR, Glasziou P, Marshall JC, et al. No surgical innovation without evaluation: the IDEAL recommendations. Lancet. (2009) 374:1105–12. doi: 10.1016/S0140-6736(09)61116-8

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Flott K, Callahan R, Darzi A, Mayer E. A patient-centered framework for evaluating digital maturity of health services: a systematic review. J Med Internet Res. (2016) 18:e75. doi: 10.2196/jmir.5047

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Johnston DS. Digital maturity: are we ready to use technology in the NHS? Future Healthc J. (2017) 4:189. doi: 10.7861/futurehosp.4-3-189

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Star SL. The ethnography of infrastructure. Am Behav Sci. (1999) 43:377–91. doi: 10.1177/00027649921955326

CrossRef Full Text | Google Scholar

38. NHS England. The Forward View Into Action: Planning for 2015/16. London: NHS England (2014).

Google Scholar

39. Hunter K. “Don't think zebras”: uncertainty, interpretation, and the place of paradox in clinical education. Theor Med. (1996) 17:225–41. doi: 10.1007/BF00489447

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Markkula Center for Applied Ethics. A Framework for Ethical Decision Making. Santa Clara, CA: Santa Clara University (2015).

41. Boulkedid R, Abdoul H, Loustau M, Sibony O, Alberti C. Using and reporting the Delphi method for selecting healthcare quality indicators: a systematic review. PLoS ONE. (2011) 6:e20476. doi: 10.1371/journal.pone.0020476

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Khodyakov D, Chen C. Response changes in Delphi processes: why is it important to provide high-quality feedback to delphi participants? J Clin Epidemiol. (2020) 125:160–1. doi: 10.1016/j.jclinepi.2020.04.029

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Belton I, Macdonald A, Wright G, Hamlin I. Improving the practical application of the Delphi method in group-based judgment: a six-step prescription for a well-founded and defensible process. Technol Forecast Soc Change. (2019) 147:72–82. doi: 10.1016/j.techfore.2019.07.002

CrossRef Full Text | Google Scholar

44. Rosenstock IM. Why people use health services. Milbank Q. (2005) 83:1–32. doi: 10.1111/j.1468-0009.2005.00425.x

CrossRef Full Text | Google Scholar

45. Smith CF, Drew S, Ziebland S, Nicholson BD. Understanding the role of GPs' gut feelings in diagnosing cancer in primary care: a systematic review and meta-analysis of existing evidence. Br J Gen Pract. (2020) 70:e612–21. doi: 10.3399/bjgp20X712301

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Bourdieu P. Outline of a Theory of Practice. Cambridge University Press (1977). doi: 10.1017/CBO9780511812507

CrossRef Full Text | Google Scholar

47. Jansen T, Hek K, Schellevis FG, Kunst AE, Verheij RA. Socioeconomic inequalities in out-of-hours primary care use: an electronic health records linkage study. Eur J Public Health. (2020) 30:1049–55. doi: 10.1093/eurpub/ckaa116

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Marmot M. An inverse care law for our time. BMJ. (2018) 362:k3216. doi: 10.1136/bmj.k3216

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Frank AW. Just listening: narrative and deep illness. Famil Syst Health. (1998) 16:197. doi: 10.1037/h0089849

CrossRef Full Text | Google Scholar

50. Balint M. The doctor, his patient, and the illness. Lancet. (1955) 265:683–8. doi: 10.1016/S0140-6736(55)91061-8

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Mallett S. Understanding home: a critical review of the literature. Sociol Rev. (2004) 52:62–89. doi: 10.1111/j.1467-954X.2004.00442.x

CrossRef Full Text | Google Scholar

52. McKenna G, Rogers A, Walker S, Pope C. The influence of personal communities in understanding avoidable emergency department attendance: qualitative study. BMC Health Serv Res. (2020) 20:887. doi: 10.1186/s12913-020-05705-5

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Pope C, Turnbull J, Jones J, Prichard J, Rowsell A, Halford S. Has the NHS 111 urgent care telephone service been a success? Case study and secondary data analysis in England. BMJ Open. (2017) 7:e014815. doi: 10.1136/bmjopen-2016-014815

PubMed Abstract | CrossRef Full Text | Google Scholar

54. Ambrose L. Remote consulting: recognising the cognitive load. Br J Gen Pract. (2020) 70:295–5. doi: 10.3399/bjgp20X710213

PubMed Abstract | CrossRef Full Text | Google Scholar

55. de Zulueta P. Touch matters: COVID-19, the physical examination and 21st century general practice. Br J Gen Pract. (2020) 70:594–5. doi: 10.3399/bjgp20X713705

PubMed Abstract | CrossRef Full Text | Google Scholar

56. Brown VT, Gregory S, Gray DP. The power of personal care: the value of the patient–GP consultation. Brit J Gen Pract. (2020) 70:594–5. doi: 10.3399/bjgp20X713717

PubMed Abstract | CrossRef Full Text | Google Scholar

57. Dawnay G. Is this really doctoring? Br J Gen Pract. (2020) 70:455–5. doi: 10.3399/bjgp20X712445

PubMed Abstract | CrossRef Full Text | Google Scholar

58. Neve G, Fyfe M, Hayhoe B, Kumar S. Digital health in primary care: risks and recommendations. Br J Gen Pract. (2020) 70:609–10. doi: 10.3399/bjgp20X713837

PubMed Abstract | CrossRef Full Text | Google Scholar

59. Trump BD, Linkov I. Risk and resilience in the time of the COVID-19 crisis. Environ Syst Decis. (2020) 40:171–3. doi: 10.1007/s10669-020-09781-0

PubMed Abstract | CrossRef Full Text | Google Scholar

60. Vera San Juan N, Aceituno D, Djellouli N, Sumray K, Regenold N, Syversen A, et al. Mental health and well-being of healthcare workers during the COVID-19 pandemic in the UK: contrasting guidelines with experiences in practice. BJPsychol Open. (2021) 7:E15. doi: 10.1192/bjo.2020.148

PubMed Abstract | CrossRef Full Text | Google Scholar

61. Franklin J. Loneliness: an incommunicable disease? Br J Gen Pract. (2020) 70:463. doi: 10.3399/bjgp20X712541

PubMed Abstract | CrossRef Full Text | Google Scholar

62. Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. (2004) 82:581–629. doi: 10.1111/j.0887-378X.2004.00325.x

PubMed Abstract | CrossRef Full Text | Google Scholar

63. NHS Digital. Advice on Using Video Consultation Systems. London: NHS Digital (2020).

64. Salisbury C, Quigley A, Hex N, Aznar C. Private video consultation services and the future of primary care. J Med Internet Res. (2020) 22:e19415. doi: 10.2196/19415

PubMed Abstract | CrossRef Full Text | Google Scholar

65. NHS England. Delivering a Net Zero NHS. London: NHS England (2021).

Google Scholar

66. Holmner Å, Ebi KL, Lazuardi L, Nilsson M. Carbon footprint of telemedicine solutions-unexplored opportunity for reducing carbon emissions in the health sector. PLoS ONE. (2014) 9:e105040. doi: 10.1371/journal.pone.0105040

PubMed Abstract | CrossRef Full Text | Google Scholar

67. NHS England. Five Year Forward View. London: NHS England (2014).

68. NHS England. NHS Long Term Plan. London: NHS England (2019).

69. Monitor Deloitte. Digital Health in the UK: An Industry Study for the Office of Life Sciences. London: Deloitte (2015).

70. UK Government. The Digital Transformation Portfolio. London: UK Government (2019).

71. Hart JT. The inverse care law. Lancet. (1971) 297:405–12. doi: 10.1016/S0140-6736(71)92410-X

PubMed Abstract | CrossRef Full Text | Google Scholar

72. Mercer SW, Guthrie B, Furler J, Watt GC, Hart JT. Multimorbidity and the inverse care law in primary care. BMJ. (2012) 344:e4152. doi: 10.1136/bmj.e4152

PubMed Abstract | CrossRef Full Text | Google Scholar

73. Rolewicz L, Keeble E, Paddison C, Scobie S. Are the needs of people with multiple long-term conditions being met? Evidence from the 2018 General Practice Patient Survey. BMJ Open. (2020) 10:e041569. doi: 10.1136/bmjopen-2020-041569

PubMed Abstract | CrossRef Full Text | Google Scholar

74. Veinot TC, Mitchell H, Ancker JS. Good intentions are not enough: how informatics interventions can worsen inequality. J Am Med Inform Assoc. (2018) 25:1080–8. doi: 10.1093/jamia/ocy052

PubMed Abstract | CrossRef Full Text | Google Scholar

75. Schou J, Pors AS. Digital by default? A qualitative study of exclusion in digitalised welfare. Soc Policy Administr. (2019) 53:464–77. doi: 10.1111/spol.12470

CrossRef Full Text | Google Scholar

76. Office of National Statistics. Exploring the UK's Digital Divide. London: ONS (2019).

Google Scholar

77. Hilbert M. The bad news is that the digital access divide is here to stay: domestically installed bandwidths among 172 countries for 1986–2014. Telecomm Policy. (2016) 40:567–81. doi: 10.1016/j.telpol.2016.01.006

CrossRef Full Text | Google Scholar

78. Ramsetty A, Adams C. Impact of the digital divide in the age of COVID-19. J Am Med Inform Assoc. (2020) 27:1147–8. doi: 10.1093/jamia/ocaa078

PubMed Abstract | CrossRef Full Text | Google Scholar

79. NHS England. Implementing Phase 3 of the NHS Response to the COVID-19 Pandemic. London: NHS England (2020).

80. NHS Digital. How We Can Support Digital Inclusion. London: NHS Digital (2020).

81. Ellison KS, Guidry J, Picou P, Adenuga P, Davis TE. Telehealth and autism prior to and in the age of COVID-19: a systematic and critical review of the last decade. Clin Child Fam Psychol Rev. (2021) 1–32. doi: 10.1007/s10567-021-00358-0

PubMed Abstract | CrossRef Full Text | Google Scholar

82. Stone E, Nuckley P, Shapiro R. Digital Inclusion in Health and Care: Lessons Learned from the NHS Widening Digital Participation Programme. Leeds: Good Things Foundation (2020).

83. O'Connor S, Hanlon P, O'Donnell CA, Garcia S, Glanville J, Mair FS. Understanding factors affecting patient and public engagement and recruitment to digital health interventions: a systematic review of qualitative studies. BMC Med Inform Decis Mak. (2016) 16:1–15. doi: 10.1186/s12911-016-0359-3

PubMed Abstract | CrossRef Full Text | Google Scholar

84. Williams H, Whelan A. An Investigation Into Access to Digital Inclusion for Healthcare for the Homeless Population. Hastings: Seaview (2017). Available online at: https://amhp.org.uk/app/uploads/2018/11/Digital-Inclusion-and-Homeless-People.pdf (accessed March 7, 2021).

85. Beauchamp TL, Childress JF. Principles of Biomedical Ethics. Oxford: Oxford University Press (1979).

Google Scholar

86. Gillon R. Medical ethics: four principles plus attention to scope. BMJ. (1994) 309:184. doi: 10.1136/bmj.309.6948.184

PubMed Abstract | CrossRef Full Text | Google Scholar

87. Beauchamp T, Childress J. Principles of biomedical ethics: marking its fortieth anniversary. Am J Bioethics. (2019) 19:9–12. doi: 10.1080/15265161.2019.1665402

PubMed Abstract | CrossRef Full Text | Google Scholar

88. General Medical Council. Duties of a Doctor. London: GMC (2020). Available online at https://www.gmc-uk.org/ethical-guidance/ethical-guidance-for-doctors/good-medical-practice (accessed October 5, 2020).

Google Scholar

89. Tronto JC. Moral Boundaries: A Political Argument for an Ethic of Care. London: Routledge (1993).

Google Scholar

90. Fisher B, Tronto J. Toward a feminist theory of caring. In: Circles of Care: Work and Identity in Women's Lives. Albany, NY: State University of New York Press (1990). p. 35–62.

Google Scholar

91. Held V. The Ethics of Care: Personal, Political, and Global. Oxford: Oxford University Press on Demand (2006).

Google Scholar

92. Pols J. Towards an empirical ethics in care: Relations with technologies in health care. Med Health Care Philos. (2015) 18:81–90. doi: 10.1007/s11019-014-9582-9

PubMed Abstract | CrossRef Full Text | Google Scholar

93. May CR, Eton DT, Boehmer K, Gallacher K, Hunt K, Macdonald S, et al. Rethinking the patient: using Burden of Treatment Theory to understand the changing dynamics of illness. BMC Health Serv Res. (2014) 14:281. doi: 10.1186/1472-6963-14-281

PubMed Abstract | CrossRef Full Text | Google Scholar

94. Institute of Medicine (Us) Committee on Quality of Healthcare in America. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academies Press (2001).

Google Scholar

95. Doorn N, Gardoni P, Murphy C. A multidisciplinary definition and evaluation of resilience: The role of social justice in defining resilience. Sustain Resil Infrastruct. (2019) 4:112–23. doi: 10.1080/23789689.2018.1428162

CrossRef Full Text | Google Scholar

96. Braithwaite J, Wears RL, Hollnagel E. Resilient health care: turning patient safety on its head. Int J Qual Health Care. (2015) 27:418–20. doi: 10.1093/intqhc/mzv063

PubMed Abstract | CrossRef Full Text | Google Scholar

97. O'Cathail M, Sivanandan MA, Diver C, Patel P, Christian J. The use of patient-facing teleconsultations in the national health service: scoping review. JMIR Med Inform. (2020) 8:e15380. doi: 10.2196/15380

PubMed Abstract | CrossRef Full Text | Google Scholar

98. Thompson-Coon J, Abdul-Rahman A-K, Whear R, Bethel A, Vaidya B, Gericke CA, et al. Telephone consultations in place of face to face out-patient consultations for patients discharged from hospital following surgery: a systematic review. BMC Health Serv Res. (2013) 13:128. doi: 10.1186/1472-6963-13-128

PubMed Abstract | CrossRef Full Text | Google Scholar

99. Downes MJ, Mervin MC, Byrnes JM, Scuffham PA. Telephone consultations for general practice: a systematic review. Syst Rev. (2017) 6:128. doi: 10.1186/s13643-017-0529-0

PubMed Abstract | CrossRef Full Text | Google Scholar

100. Campbell JL, Fletcher E, Britten N, Green C, Holt TA, Lattimer V. Telephone triage for management of same-day consultation requests in general practice (the ESTEEM trial): a cluster-randomised controlled trial and cost–consequence analysis. Lancet J. (2014) 384:1859–68. doi: 10.1016/S0140-6736(14)61058-8

PubMed Abstract | CrossRef Full Text | Google Scholar

101. Newbould J, Ball S, Abel G, Barclay M, Brown T, Corbett J, et al. A ‘telephone first' approach to demand management in English general practice: a multimethod evaluation. Health Serv Deliv Res. (2019) 7:17. doi: 10.3310/hsdr,07170

PubMed Abstract | CrossRef Full Text | Google Scholar

102. Salisbury C, Murphy M, Duncan P. The impact of digital-first consultations on workload in general practice: modeling study. J Med Internet Res. (2020) 22:e18203. doi: 10.2196/preprints.18203

PubMed Abstract | CrossRef Full Text | Google Scholar

103. Huxley CJ, Atherton H, Watkins JA, Griffiths F. Digital communication between clinician and patient and the impact on marginalised groups: a realist review in general practice. Br J Gen Pract. (2015) 65:e813–21. doi: 10.3399/bjgp15X687853

PubMed Abstract | CrossRef Full Text | Google Scholar

104. Banks J, Farr M, Salisbury C, Bernard E, Northstone K, Edwards H, et al. Use of an electronic consultation system in primary care: a qualitative interview study. Br J Gen Pract. (2018) 68:e1–8. doi: 10.3399/bjgp17X693509

PubMed Abstract | CrossRef Full Text | Google Scholar

105. Huygens MW, Swinkels IC, Verheij RA, Friele RD, Van Schayck OC, De Witte LP. Understanding the use of email consultation in primary care using a retrospective observational study with data of Dutch electronic health records. BMJ Open. (2018) 8:e019233. doi: 10.1136/bmjopen-2017-019233

PubMed Abstract | CrossRef Full Text | Google Scholar

106. Edwards HB, Marques E, Hollingworth W, Horwood J, Farr M, Bernard E, et al. Use of a primary care online consultation system, by whom, when and why: evaluation of a pilot observational study in 36 general practices in South West England. BMJ Open. (2017) 7:e016901. doi: 10.1136/bmjopen-2017-016901

PubMed Abstract | CrossRef Full Text | Google Scholar

107. Marshall M, Shah R, Stokes-Lampard H. Online consulting in general practice: making the move from disruptive innovation to mainstream service. BMJ. (2018) 360. doi: 10.1136/bmj.k1195

PubMed Abstract | CrossRef Full Text | Google Scholar

108. Farr M, Banks J, Edwards HB, Northstone K, Bernard E, Salisbury C, et al. Implementing online consultations in primary care: a mixed-method evaluation extending normalisation process theory through service co-production. BMJ Open. (2018) 8:e019966. doi: 10.1136/bmjopen-2017-019966

PubMed Abstract | CrossRef Full Text | Google Scholar

109. Chongmelaxme B, Lee S, Dhippayom T, Saokaew S, Chaiyakunapruk N, Dilokthornsakul P. The effects of telemedicine on asthma control and patients' quality of life in adults: a systematic review and meta-analysis. J Allergy Clin Immunol Pract. (2019) 7:199–216.e11. doi: 10.1016/j.jaip.2018.07.015

PubMed Abstract | CrossRef Full Text | Google Scholar

110. Gentry MT, Lapid MI, Rummans TA. Geriatric telepsychiatry: systematic review and policy considerations. Am J Geriatr Psychiatry. (2019) 27:109–27. doi: 10.1016/j.jagp.2018.10.009

PubMed Abstract | CrossRef Full Text | Google Scholar

111. Hong Y, Lee SH. Effectiveness of tele-monitoring by patient severity and intervention type in chronic obstructive pulmonary disease patients: a systematic review and meta-analysis. Int J Nurs Stud. (2019) 92:1–15. doi: 10.1016/j.ijnurstu.2018.12.006

PubMed Abstract | CrossRef Full Text | Google Scholar

112. Kelson J, Rollin A, Ridout B, Campbell A. Internet-delivered acceptance and commitment therapy for anxiety treatment: systematic review. J Med Internet Res. (2019) 21:e12530. doi: 10.2196/12530

PubMed Abstract | CrossRef Full Text | Google Scholar

113. McFarland S, Coufopolous A, Lycett D. The effect of telehealth versus usual care for home-care patients with long-term conditions: a systematic review, meta-analysis and qualitative synthesis. J Telemed Telecare. (2019) 27:69–87. doi: 10.1177/1357633X19862956

PubMed Abstract | CrossRef Full Text | Google Scholar

114. Thiyagarajan A, Grant C, Griffiths F, Atherton H. Exploring patients' and clinicians' experiences of video consultations in primary care: a systematic scoping review. BJGP Open. (2020) 4:bjgpopen20X101020. doi: 10.3399/bjgpopen20X101020

PubMed Abstract | CrossRef Full Text | Google Scholar

115. Donaghy E, Atherton H, Hammersley V, McNeilly H, Bikker A, Robbins L, et al. Acceptability, benefits, and challenges of video consulting: a qualitative study in primary care. Br J Gen Pract. (2019) 69:e586–94. doi: 10.3399/bjgp19X704141

PubMed Abstract | CrossRef Full Text | Google Scholar

116. James HM, Papoutsi C, Wherton J, Greenhalgh T, Shaw SE. Spread, scale-up, and sustainability of video consulting in health care: systematic review and synthesis guided by the NASSS framework. J Med Internet Res. (2021) 23:e23775. doi: 10.2196/23775

PubMed Abstract | CrossRef Full Text | Google Scholar

117. Odendaal WA, Watkins JA, Leon N, Goudge J, Griffiths F, Tomlinson M, et al. Health workers' perceptions and experiences of using mHealth technologies to deliver primary healthcare services: a qualitative evidence synthesis. Cochrane Database Syst Rev. (2020) 3:CD011942. doi: 10.1002/14651858.CD011942.pub2

PubMed Abstract | CrossRef Full Text | Google Scholar

118. Wherton J, Shaw S, Papoutsi C, Seuren L, Greenhalgh T. Guidance on the introduction and use of video consultations during COVID-19: important lessons from qualitative research. BMJ Leader. (2020) 4:120–3. doi: 10.1136/leader-2020-000262

CrossRef Full Text

119. Hammersley V, Donaghy E, Parker R, McNeilly H, Atherton H, Bikker A, et al. Comparing the content and quality of video, telephone, and face-to-face consultations: a non-randomised, quasi-experimental, exploratory study in UK primary care. Br J Gen Pract. (2019) 69:e595–604. doi: 10.3399/bjgp19X704573

PubMed Abstract | CrossRef Full Text | Google Scholar

120. Han SM, Greenfield G, Majeed A, Hayhoe B. Impact of remote consultations on antibiotic prescribing in primary health care: systematic review. J Med Internet Res. (2020) 22:e23482. doi: 10.2196/23482

PubMed Abstract | CrossRef Full Text | Google Scholar

121. Crawford SM, Evans C, Edwards H, Zoltowski A. Requests from primary care for chest X-ray and CA125 measurements during the COVID-19 emergency: an observational study. Clin Med. (2021) 21:e45–7. doi: 10.7861/clinmed.2020-0638

PubMed Abstract | CrossRef Full Text | Google Scholar

122. Hiom S. How Coronavirus Is Impacting Cancer Services in the UK. London: Cancer Research UK (2020).

123. Helsper CW, Campbell C, Emery J, Neal RD, Li L, Rubin G, et al. Cancer has not gone away: a primary care perspective to support a balanced approach for timely cancer diagnosis during COVID-19. Eur J Cancer Care. (2020) 29:e13290 doi: 10.1111/ecc.13290

PubMed Abstract | CrossRef Full Text | Google Scholar