Doreen Tembo
Elizabeth Morrow
Louise Worswick
Debby Lennard- 1Wessex Institute, University of Southampton, Southampton, United Kingdom
- 2Independent Researcher, Research Support Northern Ireland, Killyleagh, Ireland
- 3NHS England, Taunton, United Kingdom
- 4Public Member of National Institute for Health Research Evaluation Trials and Studies Coordinating Centre Patient and Public Involvement Reference Group, University of Southampton, Southampton, United Kingdom
Background and Rationale: Internationally, the idea of “co-production’ has become more popular in health research because of the promise of partnership between researchers and patients to create research that focuses on patients’ needs. Patient and public involvement (PPI) at an early stage in deciding what research should be funded, can improve the quality and impact of research. However, professional power over the process places limits on the public practising their participatory rights for involvement in commissioning research that affects them and can leave members of the public feeling unheard or excluded, particularly within the context of early phase applied health research.
Aim: This article explores whether and how the public can be involved in the co-production of research commissioning early on in the process, with a focus on the power relations that pervade basic and early phase translational applied health research.
Methods: An exploratory literature review of international peer-reviewed and gray health research literature using structured searches of electronic databases and key search terms.
Results: There is very little literature that critically evaluates how PPI is embedded into the early phases of the commissioning process. The field of basic or early translational applied research appear to be particularly challenging. Four themes which emerged from the review are: reasons for PPI in research commissioning; benefits of PPI at strategic levels of research commissioning; contributions of patients and members of the public; improving PPI in research commissioning.
Conclusion: Although the public are being consulted at some stages of the research commissioning process, it is evident that the process of determining research priorities and agendas is far from being widely co-produced. Moving PPI from a consultative paternalistic model to a collaborative partnership model should be a priority for commissioners. Significant changes to communication, practices, systems, structures, or cultures that exclude patients and the public from contributing in meaningful ways, are needed to fulfill the potential of co-produced models of research commissioning.
Introduction
The Promise of Co-production
Internationally, the idea of “co-production” has become more popular in health research because of the promise of partnership between researchers and patients to create research that focuses on patient's needs. Patient and Public Involvement (PPI) at an early stage in deciding what research should be funded can improve the quality and impact of research. However, internationally there are very few examples of research commissioners involving patients or the public in decisions about research. This can leave members of the public feeling unheard or excluded by professionals.
Research commissioning is the most important stage of the research process for patients and the public to be involved as it gives the greatest potential to shape research agendas and to influence research funding (Oliver, 1996). However, research on decision making about future research priorities shows this rarely involves patients or the public. Decisions are more often made on the basis that technical rationalization of what research should be done, is more applicable than what is important to end users of research outputs.
Internationally in health services research PPI is widely recognized as being essential to the development of quality health services that are fit for purpose (Minogue and Girdlestone, 2010). Compared to health service delivery, PPI in health research management is globally a more recent movement and set of practices (Abrahams et al., 2004; Elberse et al., 2012; Gagnon et al., 2014; NIHR, 2015; PCORI, 2018).
Involving patients and the public in research, and especially in the early phases of research commissioning, such as research question or topic identification, priority setting, prioritization, and developing calls or advertisements for funding is thought to be crucial to overcome differential priorities between research funders, pharmaceutical companies and researchers, and the priorities of clinicians, patients and the public (Caron-Flinterman et al., 2005; Crowe et al., 2015). The consequences, as Chalmers and Glasziou (2009) describe of poor involvement of relevant stakeholders such as clinicians and patients in priority setting is an estimated avoidable waste of 85 per cent of global health research funding (Minogue et al., 2018).
Defining PPI and Co-production
The history of involving the public in service provision in the UK, one of the earliest adopters of PPI, was catalyzed by the rise in consumerist thinking in the 1960s and 1970s, and democratic or rights-based approaches that arose thereafter (Ridley et al., 2002). Under the UK Health and Social Care Act 2001 publicly-funded organizations have a duty to involve the public in the planning and provision of health services.
In the UK in 2006 the National Institute for Health Research (NIHR) was established with a mandate to involve patients and the public in commissioning and delivering publicly-funded applied health research. The organization “consumers in research” now known as INVOLVE, a national advisory group for PPI, also joined the NIHR in the same year. A legacy of this organization is its widely used definition of PPI, which we utilize in this paper:
“Research being carried out “with” or “by” members of the public rather than “to”, “about” or “for” them” (INVOLVE1).
There is variation internationally in definitions, models and ways of thinking about PPI. There is for example no agreed nomenclature with participation, engagement and involvement often being used interchangeably. There is also great variation, dependant on the country's historical development of democracy, in the mix of institutionalized vs. contestory forms of involvement in healthcare (Slutsky et al., 2016). Within the UK context, involvement within health research funding tends be embedded within institutionalized mechanisms and processes.
Theoretically there are different levels at which people can be involved, as highlighted in Hogg's (1999) models of involvement in service development which closely relate to the INVOLVE levels of involvement in research (consultation, collaboration, user-led and co-production). Paternalistic models of involvement, assume that professionals know best, and hence lend themselves to involvement at the consultative level. The Partnership models of involvement lend themselves more to collaborative approaches to involvement. The Consumerist model describes consumers in charge or user-driven or controlled involvement. Finally the Autonomy model emphasizes the importance of valuing individuals and the different perspectives patients and professionals bring, and is closely aligned with involvement at the co-produced level as defined by INVOLVE (Hickey et al., 2018).
Co-produced research harnesses the principles of sharing of power, including all perspectives and skills, respecting values and the knowledge of all those working together on the research, reciprocity and building and maintaining relationships. However, this understanding of co-production, while acknowledged to be valuable, has been criticized as being idealistic given current cultural, institutional and regulatory constraints (Madden and Speed, 2017; Green and Johns, 2019; Paylor and McKevitt, 2019).
Previous Research
The evidence base for PPI, and especially effective co-produced approaches in the early phases of research commissioning is underdeveloped (Nilsen et al., 2006; Oliver et al., 2008), especially when compared to PPI elsewhere in research (Shippee et al., 2015) or health services commissioning (Sheaff et al., 2015). A rapid review carried out by Manafò et al. (2018), which we include in this review, utilized rapid review methodology to explore existing evidence around the different approaches that could be utilized to enable PPI in priority setting in health ecosystems and health research. There is a need to further explore these and other different approaches and mechanisms, and the influence and impact PPI might have in the early phases of the research commissioning context (Staniszewska et al., 2011). This information could inform innovative collaborative and co-produced approaches which maximize the benefits of PPI through the research commissioning process.
PPI is perceived to be particularly challenging in the commissioning of clinical research, which might not have direct relevance to human health or patient outcomes, due to its early placement in the applied health research translational pathway (Caron-Flinterman et al., 2005; Dobbs and Whittaker, 2006). Some researchers might assume that patients may be put off engaging in such research due to finding science boring, irrelevant, or intimidating (Dobbs and Whittaker, 2006). Other researchers may be apprehensive because PPI can mean a different way of working that challenges established notions of professionalism (Thompson et al., 2009).
Concerns about tokenism and meaningful PPI are found throughout the research literature but are used as a catchall term that may not fully convey the limiting forces of professional power. Tokenism can be defined as the policy or practice of making only a symbolic effort to involve people (Domecq et al., 2014) or failure to develop approaches that enable people to contribute in meaningful ways (Supple et al., 2015). Unequal power relations between experts and the public can be challenging for both parties, and co-production and power sharing may be an unfulfilled ideological goal.
Aims of the Review
The aim of this exploratory literature review was to draw on international health research literature to explore some of the contextual complexities and the potential challenges of PPI in the early stages of research commissioning, with a particular focus on early translational applied health research.
The questions we explored were (a) whether and how the public can be involved in the co-production of knowledge in research commissioning? (b) What are the specific challenges in the context of basic and early phase translational applied health research? The paper draws on the findings of the exploratory literature review to address these questions.
We used the notion of co-production to consider how research might overcome differentials in power between professional and public members, which may limit meaningful PPI. Drawing on examples and findings from the literature, in the discussion, we suggest possible ways forward for innovation and improvement of meaningful PPI.
Our focus is the potentially challenging field of commissioning early phase applied health research because it is here that commissioning is far less likely to involve PPI than in the later phases of the “bench-to-bedside' research process (Callard et al., 2012). The reasons for which we will also explore.
Methods
Approach
An exploratory literature review was carried out between May and August 2018. Owing to the disparate and scarce nature of evidence on PPI in research commissioning, a systematic review was unlikely to yield useful results that can inform practice. Therefore, an exploratory approach was chosen to seek out relevant published literature to allow us to consider the issues and challenges of PPI in research commissioning. The method is illustrated by Figure 1.
Figure 1. Flow chart of exploratory review method.
We sought information about how to enable meaningful and effective approaches to involvement, as well as clarification about the meaning of tokenistic PPI in this context. We were interested in learning about ways of working that enable patients/public representatives to contribute to decision-making processes and the types of impact that PPI can have. The study team included two public contributors who were consulted throughout study.
Inclusion/Exclusions
The review explored issues about PPI in the commissioning of health research, including health services, health care, public health, clinical, and biomedical research. Included articles were those that addressed issues about: (i) any type of patients and public groups involved and their roles e.g., public reviewers, patient representatives or lay members, (ii) contexts of involvement in stages of the commissioning process, (iii) approaches to involvement, for example commenting on commissioning materials or involvement in face-to-face meetings, informing decisions, or shared decision-making practices, (iv) evidence of influence or impact of involvement on commissioning decisions, practices, or outcomes.
We sought journal articles (including empirical studies and literature reviews) and gray literature (including reports, discussion papers, commentary, and opinion pieces) where these offered useful insights and learning and were published in the English language.
Due to the limitations of time and resources we excluded articles published in other languages. We excluded articles that did not relate to health research commissioning, for example PPI in commissioning social care research or health professional education.
Search Strategy
The search strategy was to identify relevant evidence and information using:
• web-based searches of Web of Science, Google Scholar and PubMed to search the international scholarly literature; explore related works, citations, authors, and publications; and the retrieval of documents through online libraries or on the web.
• searches of the INVOLVE Evidence Library for gray literature e.g., PhD studies, organizational reports, and bibliographies.
• searches for NIHR unpublished reports and documents relating to PPI in commissioning.
Key Search Terms
Searches used the key term “patient and public involvement in research commissioning' and variations on the term (e.g., patient involvement in funding agencies). A comprehensive search drew on the search terms used by Brett et al. (2014) in their systematic review of the impact of PPI. It combined sets of terms including and relating to patient and public involvement (consumer, citizen, client, carer, lay, service users, survivor, stakeholder, family, relative); type of involvement (particp*, collaborat*, engage*, partner*, consult*, evaluat*) and commissioning (funding agencies, research briefs, research funding, identifying research priorities, research priority setting, scoping review). MeSH terms were used to expand the searches (patients, public, economics, research, funding).
Data Extraction
Identified articles deemed to be relevant to the aim of the review were retrieved in full for analysis. Data were extracted into themed categories in Microsoft Word and key data extracted included the following: the author; the year and country; the aims or focus of the article; the methods used for PPI; the type of patients or groups of the public involved; key issues, findings or implications.
Analysis
The approach to the analysis was to explore and identify themes in the data (Denzin and Lincoln, 2005) reflecting the aims of the review to explore some of the contextual complexities and the potential challenges of PPI in the early stages of research commissioning. We read each article and considered the main issues raised in relation to the questions of whether and how the public can be involved and specific challenges associated with involvement in the commissioning context. As issues were identified, these were given a code (a title phrase or word representing the issue), and in this way a code framework was developed from the data to indicate patterns across the data (Braun and Clarke, 2006). Codes were grouped together into emerging themes (purpose, benefits, challenges, facilitators) with links to the original sources (Denzin and Lincoln, 2005). In the analysis the notion of co-production was used as a lens through which to consider issues of power (Hickey et al., 2018) between professionals and public members. For example we looked for examples of power sharing in the data, e.g., new roles and responsibilities of PPI members, evidence of shared decision-making, and approaches to supporting positive interactions and communication. Tables were used to present synthesized themes and links to original sources.
Rigor
A study protocol for the review was developed and revised by team members, including identification of databases to be searched and key search terms. Strategies for minimizing biases in the search strategy were as follows. (a) One team member independently cross-checked a sample of 20 returned papers against included/exclusion criteria. (b) Members of the team discussed and reached agreement on the importance of emerging themes in the analysis. (c) Inclusion and use of gray literature to extend the searches beyond peer reviewed articles.
Results
The review identified 74 relevant papers, reports and articles about PPI in health research commissioning. The results of the review confirmed the lack of published material specifically around PPI in the early phases of the commissioning processes of early phase applied health or basic health research. The review did yield results on PPI in commissioning of applied health research that was further along the translational pathway. Here we present summary results of the main findings with some representative references to the body of literature from the review.
The structure of the results is presented according to four themes that emerged:
• Reasons for PPI in research commissioning
• Benefits of PPI at strategic levels of research commissioning
• Contributions of patients and members of the public
• Improving PPI in research commissioning.
Reasons for PPI in Research Commissioning
The review demonstrated that PPI in research commissioning predominantly operated within a paternalistic model, with public members being consulted rather than more inclusively involved in the commissioning processes as co-creators of knowledge and co-producers of commissioning decisions and processes. Reasons for PPI were rarely given or explained, which could reflect the fact that PPI is often a requirement of being awarded central funding in the UK context. However, this is not the case in other countries or for all health research that is funded by other means.
Benefits of PPI at Strategic Levels of Research Commissioning
Despite operating within a paternalistic environment, several benefits to involving patients and the public, beyond getting them to provide views about priorities for research, were identified in the literature. These are summarized in Table 1 and include research priorities becoming more relevant to users; broader perspectives being brought into commissioning decisions; research being more likely to be ethical, inclusive and fair; the contribution of public contributors' skills and knowledge to commissioning decisions; and encouragement of PPI in funded research.
Table 1. Benefits of PPI in research commissioning.
Contributions of PPI Members
The review also highlighted specific activities and contributions patients and public members make to the overall commissioning process. These have been summarized in Table 2 and include identifying topics, prioritizing topics, assessment, review of evidence, synthesizing results, and writing research briefs.
Table 2. Contributions of PPI members to research commissioning.
Improving PPI in Research Commissioning
The review discovered that new priority setting projects are being developed around the world (in the UK, US, Australia, Netherlands, and Canada) to build partnerships between patients and professionals (Bragge et al., 2011; Gagnon et al., 2014; Tong et al., 2015; Pratt et al., 2016; Ghisoni et al., 2017; Abma, 2018; JLA, 2018; Manafò et al., 2018) (see Table 3).
Table 3. Priority setting approaches that involve patients and the public.
Manafò et al. (2018) review of these priority-setting approaches concluded they are inclusive and objectively based, while being specific to the priorities of stakeholders engaged in the process. Key limitations identified were a lack of evaluation data on the success and extent to which patients were engaged, issues pertaining to feasibility of stakeholder engagement, coordination, communication, and limited resources.
Evaluation of nine projects that used the Dialogue Model (Abma et al., 2015) found patient involvement in agenda-setting is not automatically followed by patient involvement in programming and implementation. The authors recommend that support is needed during the process to organize patient involvement and adapt organizational structures like review procedures. Facilitating factors for success of the model include the importance of ownership; the value of dialogue for personal and mutual understanding; relational empowerment and critical awareness raising among patients; the importance of responsibility, responsiveness and trust; support in working with co-researchers; and the issue of representation (Abma, 2018).
Gagnon and colleagues of the Canadian Health Technology Assessment (HTA) programme have generated a conceptual framework for interventions to promote patient involvement in the early stages of HTA (Gagnon et al., 2014). Outcomes of PPI are evaluated with patients and their representatives using interviews and observations. These priority-setting projects and activities are promising but more needs to be done to test them out in different research funding contexts and particularly in early translational applied health research commissioning.
The review found examples of ways to facilitate PPI in commissioning, which could be utilized for the identification and prioritization stages of the process within early stage applied health research. In summary, these are:
• Planning for meaningful involvement all the way through the commissioning process (Oliver et al., 2004, 2009)
• Finding ways to expand opportunities for wider and effective participation and engagement with the public (Willis, 1995; Abelson et al., 2003; Oliver et al., 2008; INVOLVE, 2012; Morrow et al., 2013; Rikkers et al., 2015; Franck et al., 2018; Rawson et al., 2018; Simpson et al., 2018; Truitt et al., 2018)
• Building positive attitudes toward PPI as well as positive relations between stakeholders (Pittens et al., 2014; Abma et al., 2015; Abma, 2018). This could be facilitated by developing guidance, training and support for patient and the public contributors, Chairs of commissioning bodies and teams, including opportunities for shared learning (Boote et al., 2002; Caron-Flinterman et al., 2005; Oliver et al., 2008; INVOLVE, 2012)
• Encouraging organizations to assess the quality and impact of public involvement in commissioning (Oliver et al., 2015)
• Supporting commissioning teams to assess and provide feedback about processes and outcomes (O'Donnell and Entwistle, 2004; Howe et al., 2017).
Discussion
Variation in Opportunities for PPI in Commissioning
The review reveals a story of PPI opportunities for involvement in commissioning that ranges from ineffectual tokenism to meaningful co-creation of knowledge. Our findings suggest that while some research funders are fully committed to PPI at every stage, others have not given sufficient consideration to the benefits of PPI identified in this review. Indeed our findings do little to contest previous observations that commissioners may be concerned that PPI will distort research agendas (O'Donnell and Entwistle, 2004). Improved utilization of the review identified activities that patient and public representatives can be involved in by research funders would move the involvement model and levels from one of paternalism and consultation to one that is partnership-based and collaborative.
Most research funding organizations are open to asking patients to submit their views about priorities for research (e.g., a website where people can make suggestions for research), and some organizations go out and engage patients and groups of the public about their views about research needs. While the review highlighted novel and effective approaches to priority setting that include patients and the public, it also demonstrated that there is relatively little evidence, beyond identification and prioritization of research topics (e.g., James Lind Alliance Priority Setting Partnerships), of wide-spread co-production or co-creation in the development of prioritized research areas and funding calls.
The study by van Bekkum et al. (2016) which looked at ten UK agencies that fund health or medical research found involvement was not routinely incorporated into the planning of funding calls and there was little evidence of PPI being driven by democratic imperatives or rights-based arguments. Agencies and commissioning groups working within specific areas of health and medicine tend to promote particular definitions and practices which determine the boundaries in which researchers in these areas understand and practice PPI (van Bekkum et al., 2016). Professionals may be generally in favor of PPI but may believe that ultimately decisions about which research gets funded should be made by the professionals who are held accountable for these decisions (Oliver et al., 2004).
There are some strong examples of how the public can be involved in the co-production of knowledge in research commissioning. For example, some UK research funders, such as the NIHR and Medical Research Council, and US funders such as the Patient Centered Outcomes Research Institute, have research management frameworks for PPI which may include patients and members of the public being asked to review documentation that support prioritization of research topics or act as members of research prioritization committees (Oliver et al., 2009). However, even within this framework, it appears that some commissioning activities (e.g., defining assessment criteria, reviewing evidence, synthesizing results, writing documents for the consideration by committees, and funding decisions) may be undertaken by professionals without public input. Power is therefore balanced more toward researchers and funding organization staff than patients and public representatives. This is often the case for basic and the early applied health research commissioning context.
The Effects of Power Differentials
The review did not identify literature that focused on early stage commissioning processes for basic or early phase applied health research. The literature reveals some of the specific challenges in the context of basic and early phase translational applied health research. Perhaps most significant, is that professional skepticism and resistance manifest in subtle yet powerful ways that can limit co-production to a pipe dream (Chase et al., 2000). Even though the usefulness of patients' experiential knowledge alongside professional and clinical knowledge is widely accepted (Boote et al., 2002; Brett et al., 2014), it can be less clear how to integrate this type of knowledge into decision-making (Caron-Flinterman et al., 2005), to share ownership of decisions, and to assess decision-making effectiveness (Entwistle and O'Donnell, 2003). Researchers and funders may therefore employ tokenistic PPI, especially within the UK context where PPI is either increasingly encouraged or mandated.
The literature indicates that tokenism can be caused by lack of awareness or resistance to involvement amongst professionals, but can also be caused by practices, systems, structures or cultures that exclude patients and the public from contributing in meaningful ways (Supple et al., 2015). The technical nature of early phase translational research and the bureaucratic nature of commissioning may be a reason why the public are excluded from some commissioning activities. However, the literature demonstrates that public contributors' understanding of the technical clinical subjects, the language and science are not a necessary barrier to involvement.
When investigating patient and public involvement in biomedical research, Caron-Flinterman et al. (2005) asserted that training may support patients and the public to understand highly scientific or technical research. Further widespread use of non-technical language by professionals and plain English summaries may better enable involvement. Training for commissioning teams could cover inclusion strategies in patient–expert partnerships thereby enabling a better platform for both parties to effectively communicate and contribute to collaborative or co-produced approaches (Elberse et al., 2011).
Areas for Innovation and Improvement
Commissioning research requires informed judgements to be made about what research is important, and could lead to potentially significant results and impactful outcomes (Oliver et al., 2009). A sole focus on PPI as a participatory right endangers the involvement process into becoming a tokenistic activity that is consultative at best. Previous discursive papers on PPI suggests three different lines of thinking about the reasons for PPI in commissioning. These are: moral (to assure participative rights to involvement) (Boote et al., 2002; Coulter, 2004), methodological (to improve the quality and relevance of research to society) (Fisher, 2002; Chalmers and Glasziou, 2009), and impact (health, political, legislative, economic and societal impact). Moral or rights-based arguments suggest that PPI should be integral to research from the earliest stages as an intrinsic participatory right (Boote et al., 2002; Coulter, 2004). Methodological and impact based motivations, on the other hand, do not necessarily recommend involvement through the whole processes where it does not add value. It remains imperative that commissioners embed the moral or participatory rights-based driver as a key underlying factor that propels involvement in the system. Additionally an effective commissioning system must also consider and harness the methodological and impact drivers and benefits of PPI, such as those identified in this review, to create buy-in from all stakeholders.
Increasingly commissioning bodies are recognizing that the issue of what constitutes a rational discourse for future research, is a complex interplay of issues about how principles of patient need and rights translate into research contexts. Arguments against PPI warn against the lack of objectivity, possible bias, and individual self-interest of members of the public when it comes to making decisions about the allocation of research funds. Notions of the rights of the public to participate in all areas of health care—captured in the phrase “nothing about us without us”—are undermined by the apparent irrationality of involving members of the public in rational decisions about the allocation of research funds based on gaps in the evidence base and the feasibility, methods, and merit of the science in question.
Preoccupation with representation issues and concerns about the professionalization of lay members has directed too much attention to questions about the effectiveness of individual PPI representatives. Instead, PPI could be improved by examining the presuppositions and validity dimensions of everyday communication (normalized discourse) between professionals and PPI members. In relation to PPI in research commissioning this could include using reflective studies, to activate reflection on the unease, tensions and concerns about tokenism.
Improving opportunities for PPI requires the provision of meaningful spaces for dialogue, exchange and decision-making that suit different types of professionals and PPI representatives, as well as the public more generally. Early explicit exploration of different PPI roles and contributions with members of the public may assist effective participation and satisfaction. Singular PPI models are unable to effectively respond to the pluralism in experiences, values and opinions that different members of society hold.
Much could be gained from the involvement of third sector groups with local, regional or sector-wide views. Other approaches could be e-consultation or crowd-sourcing research topics and prioritizing them with a virtual public and professional community of practice, democratic prioritization (through voting), use of social media, or holding James Lind Alliance style priority-setting and consensus-building exercises to identify and prioritize areas of future focus (Rawson et al., 2018; Simpson et al., 2018; Truitt et al., 2018). In their review, Oliver et al. (2008) suggest a particularly fruitful method for involving the public in setting large-scale research agendas. The method was a combination of collaboration and consultation, with lay people taking leading roles in consulting peers in their networks.
There is a need for more innovative thinking about ways to relate to “seldom heard” and “hard to reach” populations, such as black and minority ethnic groups and persons with disabilities, by diversifying languages and mechanisms of communication. Creating mechanisms for engagement in commissioning that are more inclusive of diversity (e.g., by age, gender, ethnicity, socioeconomic background, and other characteristics) and reach out to wider groups of patients and the public (e.g., different experiences of health and illness, different patient groups, carers and those who are well) can help to stimulate interest and participation in commissioning. Combining different approaches can bring a more diverse range of people and their perspectives and views to the commissioning process that are more representative of diverse service user needs and priorities (Oliver et al., 2008).
More could be done to find ways to talk about complex technical ideas and research methods in accessible plain English (and to celebrate those professionals who find comprehensible expression), and to raise awareness of behavior that intimidates, side-lines or stigmatizes individuals. If we do not want PPI to be tokenistic in this area, it is important to develop policy, standards, guidance, roles, training, information, communication technologies and digital platforms (e.g., websites and social media) to support patient and public involvement in different research commissioning activities.
It is vital for people who find themselves occupying positions of power in the commissioning system to turn a critical eye toward the system. Research areas that appear to be far removed from immediate patient benefit due to being positioned early in the applied research translational pathway, especially need to better engage the public. Those in power should seek to show how the system is responsive to societal needs, for example showing the impact of commissioned research on patients or other beneficiaries (Pramesh et al., 2016). Therefore, a key issue for funders going forward is how to build capacity to adapt and absorb change brought about through co-production and the co-creation of new ways of commissioning.
Limitations
This review does not cover some of the practical challenges of PPI funders may face, including access and issues of reimbursement and payment. These issues, in different contexts, have been explored elsewhere in the literature and guidance to overcome some of these challenges is available from INVOLVE (Snape et al., 2014). The main limitation of the review is the focus on professionally defined commissioning approaches and models. It does not include lay groups taking the initiative through user-led research, or commissioning practices of user-led research organizations.
Limitations of the literature reviewed are the deficit of high-quality research studies (no trials were identified), the reliance on literature reviews, and small-scale evaluation studies carried out on single units or programmes. While international literature was included, differences in language and terminology of involvement, engagement and participation between countries are a limitation of the searches. Including languages other than English would have reduced bias but this was not possible within the limited resources for the review.
Conclusions
Although the public are involved in some countries, at some stages of the research commission process, it is clear that the process of agreeing research priorities is a long way from being co-produced and can be tokenistic. Tokenism can be caused by lack of awareness or resistance to involvement amongst professionals, but it can also be caused by highly structured commissioning systems, technically defined subject areas, and tasks that may exclude patients and the public from contributing in meaningful ways.
Addressing concerns about tokenism requires commissioners to critically reflect on current PPI practices and to devise ways of working that are meaningful and worthwhile for everyone involved. PPI could change from a minimal and minor role to a true partnership role, if improvements were made to communication, practices, systems, structures and cultures that stop patients and the public from contributing in meaningful ways.
If we want to avoid tokenism in PPI, it is important that commissioning organizations develop mechanisms to enable commission teams to secure the involvement of patients and the public through a range of options for engagement and involvement, including use of face-to-face methods and digital platforms. New, more distributed approaches to commissioning could be based on collaboration or partnership models, which bring together patients, carers and clinicians to create truly co-produced research agendas.
Data Availability
No datasets were generated or analyzed for this study.
Author Contributions
All authors are part of the project team comprising the larger work programme from which this paper draws its data. All authors contributed to the development of the paper, with DT and EM taking primary leadership in drafting and co-authors (LW, DL) providing detailed feedback on drafts.
Funding
The literature review was conducted as part of a work programme around an internal evaluation in the UK National Institute of Health Research Evaluation Trials and Studies Coordinating Center (NETSCC).
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Acknowledgments
We would like to thank members of project team of the NETSCC internal evaluation and in particular Alice Hawliczek (NETSCC) and Jane Putsey (Public Member).
Footnotes
1. ^INVOLVE. What is public involvement in research? Retrieved from: http://www.invo.org.uk/find-out-more/what-is-public-involvement-in-research-2/
References
Abelson, J., Eyles, J., McLeod, C. B., Collins, P., McMullan, C., and Forest, P. G. (2003). Does deliberation make a difference? Results from a citizens panel study of health goals priority setting. Health Policy 66, 95–106. doi: 10.1016/S0168-8510(03)00048-4
Abma, T. (2005). Patient participation in health research: research with and for people with spinal cord injuries. Qual. Health Res. 15, 1310–1328. doi: 10.1177/1049732305282382
Abma, T. (2018). Dialogue and deliberation: new approaches to including patients in setting health and healthcare research agendas. Action Res. 1476750318757850. doi: 10.1177/1476750318757850
Abma, T., Pittens, C. A. C. M., Visse, M., Elberse, J. E., and Broerse, J. E. W. (2015). Patient involvement in research programming and implementation: a responsive evaluation of the dialogue model for research agenda setting. Health Expect 18, 2449–2464. doi: 10.1111/hex.12213
Abrahams, N., Adhikari, R., Bhagwat, I. P., Christofides, N., Djibuti, M., Dyalchand, A., et al. (2004). Changing the debate about health research for development. international health research awards recipients. J. Public Health Policy 25, 259–287. doi: 10.1057/palgrave.jphp.3190028
Andejeski, Y., Breslau, E. S., Hart, E., Lythcott, N., Alexander, L., Rich, I., et al. (2002). Benefits and drawbacks of including consumer reviewers in the scientific merit review of breast cancer research. J. Womens Health Gend. Based Med. 11, 119–136. doi: 10.1089/152460902753645263
Boote, J., Dalgleish, M., Freeman, J., Jones, Z., Miles, M., and Rodgers, H. (2014). ‘But is it a question worth asking?’ a reflective case study describing how public involvement can lead to researchers' ideas being abandoned. Health Expect 17, 440–451. doi: 10.1111/j.1369-7625.2012.00771.x
Boote, J., Telford, R., and Cooper, C. (2002). Consumer involvement in health research: a review and research agenda. Health Policy 61, 213–236. doi: 10.1016/S0168-8510(01)00214-7
Brady, L., and Preston, J. (2017). Evaluating the Extent and Impact of Young People's Involvement in National Institute for Health Research (NIHR) Studies: An Assessment of Feasibility. Report of a project commissioned by the James Lind Initiative. Retrieved from http://generationr.org.uk/?p=1375 (accessed July 9, 2018).
Bragge, P., Clavisi, O., Turner, T., Tavender, E., Collie, A., and Gruen, R. L. (2011). The global evidence mapping initiative: scoping research in broad topic areas. BMC Med. Res. Methodol. 11:92. doi: 10.1186/1471-2288-11-92
Braun, V., and Clarke, V. (2006). Using thematic analysis in psychology. Qual. Res. Psychol. 3, 77–101. doi: 10.1191/1478088706qp063oa
Brett, J., Staniszewska, S., Mockford, C., Herron-Marx, S., Hughes, J., Tysall, C., et al. (2014). Mapping the impact of patient and public involvement on health and social care research: a systematic review. Health Expect 17, 637–650. doi: 10.1111/j.1369-7625.2012.00795.x
Callard, F., Rose, D., and Wykes, T. (2012). Close to the bench as well as at the bedside: involving service users in all phases of translational research. Health Expect 15, 389–400. doi: 10.1111/j.1369-7625.2011.00681.x
Caron-Flinterman, J. F., Broerse, J. E. W., and Bunders, J. F. G. (2005). The experiential knowledge of patients: a new resource for biomedical research? Soc. Sci. Med. 60, 2575–2584. doi: 10.1016/j.socscimed.2004.11.023
Chalmers, I., and Glasziou, P. (2009). Avoidable waste in the production and reporting of research evidence. Lancet 374, 86–89. doi: 10.1016/S0140-6736(09)60329-9
Chase, D., Milne, R., Stein, K., and Stevens, A. (2000). What are the relative merits of the sources used to identify potential research priorities for the NHS HTA programme? Int. J. Technol. Assess. Health Care 16, 743–750. doi: 10.1017/S0266462300102028
Coulter, A. (2004). Perspectives on health technology assessment: response from the patient's perspective. Int. J. Technol. Assess. Health Care 20, 92–96. doi: 10.1017/S0266462304000856
Crocker, J. C., Boylan, A. M., Bostock, J., and Locock, L. (2017). Is it worth it? patient and public views on the impact of their involvement in health research and its assessment: a UK-based qualitative interview study. Health Expect. 20, 519–528. doi: 10.1111/hex.12479
Crowe, S., Fenton, M., Hall, M., Cowan, K., and Chalmers, I. (2015). Patients', clinicians' and the research communities' priorities for treatment research: there is an important mismatch. Res. Involv. Engage. 1:2. doi: 10.1186/s40900-015-0014-7
Denzin, N., and Lincoln, Y. (2005). “Introduction: the discipline and practice of qualitative research,” in The Sage Handbook of Qualitative Research 3rd ed, eds N. K. Denzin, and Y. S. Lincoln (Thousand Oaks, CA: Sage, 1–32.
Dobbs, T., and Whittaker, I. (2006). Patient and public involvement in basic science research - are we doing enough? BMJ Opin. Available online at: https://blogs.bmj.com/bmj/2016/05/11/ppi-in-basic-science-research-are-we-doing-enough
Domecq, J. P., Prutsky, G., Elraiyah, T., Wang, Z., Nabhan, M., Shippee, N., et al. (2014). Patient engagement in research: a systematic review. BMC Health Serv. Res. 14:89. doi: 10.1186/1472-6963-14-89
Elberse, J. E., Caron-Flinterman, J. F., and Broerse, J. E. (2011). Patient-expert partnerships in research: how to stimulate inclusion of patient perspectives. Health Expect 14, 225–239. doi: 10.1111/j.1369-7625.2010.00647.x
Elberse, J. E., Pittens, C. A., de Cock Buning, T., and Broerse, J. E. (2012). Patient involvement in a scientific advisory process: setting the research agenda for medical products. Health Policy 107, 231–242. doi: 10.1016/j.healthpol.2012.05.014
Entwistle, V., Calnan, M., and Dieppe, P. (2008). Consumer involvement in setting the health services research agenda: persistent questions of value. J. Health Serv. Res. Policy 13(Suppl. 3), 76–81. doi: 10.1258/jhsrp.2008.007167
Entwistle, V., and O'Donnell, M. (2003). Research funding organisations and consumer involvement. J. Health Serv. Res. Policy 8, 129–131. doi: 10.1258/135581903322029458
Fisher, M. (2002). The role of service users in problem formulation and technical aspects of social research. Soc. Work Educ. 21, 305–312. doi: 10.1080/02615470220136885
Franck, L. S., McLemore, M. R., Cooper, N., De Castro, B., Gordon, A. Y., Williams, S., et al. (2018). A novel method for involving women of color at high risk for preterm birth in research priority setting. J. Vis. Exp. 131:56220. doi: 10.3791/56220
Fudge, N., Wolfe, C. D. A., and McKevitt, C. (2007). Involving older people in health research. Age Ageing 36, 492–500. doi: 10.1093/ageing/afm029
Gagnon, M. P., Candas, B., Desmartis, M., Gagnon, J., Roche, D. L., Rhainds, M., et al. (2014). Involving patient in the early stages of health technology assessment (HTA): a study protocol. BMC Health Serv. Res. 14, 273–273. doi: 10.1186/1472-6963-14-273
Gagnon, M. P., Desmartis, M., Lepage-Savary, D., Gagnon, J., St-Pierre, M., Rhainds, M., et al. (2011). Introducing patients' and the public's perspectives to health technology assessment: a systematic review of international experiences. Int. J. Technol. Assess. Health Care 27, 31–42. doi: 10.1017/S0266462310001315
Gamble, C., Dudley, L., Allam, A., Bell, P., Goodare, H., Hanley, B., et al. (2014). Patient and public involvement in the early stages of clinical trial development: a systematic cohort investigation. BMJ Open 4:e005234. doi: 10.1136/bmjopen-2014-005234
Ghisoni, M., Wilson, C. A., Morgan, K., Edwards, B., Simon, N., Langley, E., et al. (2017). Priority setting in research: user led mental health research. Res. Involve. Engage. 3:4. doi: 10.1186/s40900-016-0054-7
Gooberman-Hill, R., Horwood, J., and Calnan, M. (2008). Citizens' juries in planning research priorities: process, engagement and outcome. Health Expect 11, 272–281. doi: 10.1111/j.1369-7625.2008.00502.x
Green, G., and Johns, T. (2019). Exploring the relationship (and power dynamic) between researchers and public partners working together in applied health research teams. Front. Sociol. 4:20. doi: 10.3389/fsoc.2019.00020
Greenhalgh, T., Ovseiko, P. V., Fahy, N., Shaw, S., Kerr, P., Rushforth, A. D., et al. (2017). Maximising value from a United Kingdom biomedical research centre: study protocol. Health Res. Policy Syst. 15:70. doi: 10.1186/s12961-017-0237-1
Hailey, D., and Nordwall, M. (2006). Survey on the involvement of consumers in health technology assessment programs. Int. J. Technol. Assess. Health Care 22, 497–499. doi: 10.1017/S0266462306051427
Hewlett, S., Wit, M., Richards, P., Quest, E., Hughes, R., Heiberg, T., et al. (2006). Patients and professionals as research partners: challenges, practicalities, and benefits. Arthritis Rheum. 55, 676–680. doi: 10.1002/art.22091
Hickey, G., Brearley, S., Coldham, T., Denegri, S., Green, G., Staniszewska, S., et al. (2018). Guidance on Co-producing a Research Project. Southampton: INVOLVE.
Howe, A., MacDonald, H., Barrett, B., and Little, B. (2006). Ensuring public and patient participation in research: a case study in infrastructure development in one UK research and development consortium. Primary Health Care Res. Develop. 7, 60–67. doi: 10.1191/1463423606pc269oa
Howe, A., Mathie, E., Munday, D., Cowe, M., Goodman, C., Keenan, J., et al. (2017). Learning to work together – lessons from a reflective analysis of a research project on public involvement. Res. Involve. Engage. 3:1. doi: 10.1186/s40900-016-0051-x
Husereau, D., Boucher, M., and Noorani, H. (2010). Priority setting for health technology assessment at CADTH. Int. J. Technol. Assess. Health Care 26, 341–347. doi: 10.1017/S0266462310000383
Hutchison, K., Rogers, W., and Entwistle, V. A. (2017). Addressing deficits and injustices: the potential epistemic contributions of patients to research. Health Care Anal. 25, 386–403. doi: 10.1007/s10728-016-0323-5
INVOLVE (2012). Tip Sheets: Recruiting Members of the Public to Get Involved in Research Funding and Commissioning Processes. Available online at: http://www.invo.org.uk/wp-content/uploads/2012/04/Recruitment-tips-sheet.pdf (accessed July 9, 2018).
JLA (2018). James Lind Alliance Priority Setting Partnerships. Available online at: http://www.jla.nihr.ac.uk/news-and-publications/psp-articles-and-publications.htm (accessed July 9, 2018).
Lindenmeyer, A., Hearnshaw, H., Sturt, J., Ormerod, R., and Aitchison, G. (2007). Assessment of the benefits of user involvement in health research from the warwick diabetes care research user group: a qualitative case study. Health Expect 10, 268–277. doi: 10.1111/j.1369-7625.2007.00451.x
Madden, M., and Speed, E. (2017). Beware zombies and unicorns: toward critical patient and public involvement in health research in a neoliberal context. Front. Sociol. 2:7. doi: 10.3389/fsoc.2017.00007
Manafò, E., Petermann, L., Vandall-Walker, V., and Mason-Lai, P. (2018). Patient and public engagement in priority setting: a systematic rapid review of the literature. PLoS ONE 13:e0193579. doi: 10.1371/journal.pone.0193579
Manikam, L., Shah, R., Reed, K., Santini, G., and Lakhanpaul, M. (2017). Using a co-production prioritization exercise involving South Asian children, young people and their families to identify health priorities requiring further research and public awareness. Health Expect 20, 852–861. doi: 10.1111/hex.12524
McCormick, S., Brody, J., Brown, P., and Polk, R. (2004). Public involvement in breast cancer research: an analysis and model for future research. Int. J. Health Services 34, 625–646. doi: 10.2190/HPXB-9RK8-ETVM-RVEA
Menon, D., and Stafinski, T. (2011). Role of patient and public participation in health technology assessment and coverage decisions. Expert Rev. Pharmacoecon. Outcomes Res. 11, 75–89. doi: 10.1586/erp.10.82
Minogue, V., Cooke, M., Donskoy, A.-L., Vicary, P., and Wells, B. (2018). Patient and public involvement in reducing health and care research waste. Res. Involve. Engage. 4:5. doi: 10.1186/s40900-018-0087-1
Minogue, V., and Girdlestone, J. (2010). Building capacity for service user and carer involvement in research: the implications and impact of best research for best health. Int. J. Health Care Qual. Assur. 23, 422–435. doi: 10.1108/09526861011037470
Moran, R., and Davidson, P. (2011). An uneven spread: a review of public involvement in the national institute of health research's health technology assessment program. Int. J. Technol. Assess. Health Care 27, 343–347. doi: 10.1017/S0266462311000559
Morgan, L. J., Chambers, R., Banerji, J., Gater, J., and Jordan, J. (2005). Consumers leading public consultation: the general public's knowledge of stroke. Fam. Pract. 22, 8–14. doi: 10.1093/fampra/cmh709
Morris, C., Shilling, V., McHugh, C., and Wyatt, K. (2011). Why it is crucial to involve families in all stages of childhood disability research. Dev. Med. Child Neurol. 53, 769–771. doi: 10.1111/j.1469-8749.2011.03984.x
Morrow, E., Cotterell, P., Robert, G., Grocott, P., and Ross, F. (2013). Mechanisms can help to use patients' experiences of chronic disease in research and practice: an interpretive synthesis. J. Clin. Epidemiol. 66, 856–864. doi: 10.1016/j.jclinepi.2012.12.019
NIHR (2015). Going the Extra Mile: Improving the Nation's Health and Wellbeing Through Public Involvement in Research. Available online at: https://www.nihr.ac.uk/patients-and-public/documents/Going-the-Extra-Mile.pdf (accessed July 7, 2018).
NIHR BRCU (2017). NIHR Biomedical Research Centres and Units Annual Reports 2016/17. Available online at: https://www.nihr.ac.uk/about-us/how-we-are-managed/managing-centres/nihr-central-commissioning-facility/ccf-ppi/ppie-annual-reports.htm (accessed July 9, 2018).
NIHR CLAHRC (2017). Collaboration for Leadership in Applied Health Research and Care East of England. Patient and Public Involvement in Research Handbook. Available online at: http://www.clahrc-eoe.nihr.ac.uk/wp-content/uploads/2018/01/CLAHRC-EoE-PPI-IN-Research-Handbook_December-2017.pdf (accessed July 8, 2018).
Nilsen, E. S., Myrhaug, H. T., Johansen, M., Oliver, S., and Oxman, A. D. (2006). Methods of consumer involvement in developing healthcare policy and research, clinical practice guidelines and patient information material. Cochrane Database Syst. Rev. 3:Cd004563. doi: 10.1002/14651858.CD004563.pub2
O'Donnell, M., and Entwistle, V. (2004). Consumer involvement in decisions about what health-related research is funded. Health Policy 70, 281–290. doi: 10.1016/j.healthpol.2004.04.004
Oliver, S. (1996). The progress of lay involvement in the NHS research and development programme. J. Eval. Clin. Pract. 2, 273–280. doi: 10.1111/j.1365-2753.1996.tb00057.x
Oliver, S., Armes, D. G., and Gyte, G. (2009). Public involvement in setting a national research agenda: a mixed methods evaluation. Patient 2, 179–190. doi: 10.2165/11314860-000000000-00000
Oliver, S., Clarke-Jones, L., Rees, R., Milne, R., Buchanan, P., Gabbay, J., et al. (2004). Involving consumers in research and development agenda setting for the NHS: developing an evidence-based approach. Health Technol. Assess 8, 1–148, iii–iv. doi: 10.3310/hta8150
Oliver, S., and Gray, J. (2006). A Bibliography of Research Reports About Patients', Clinicians' and Researchers' Priorities for New Research. London: James Lind Alliance.
Oliver, S., Liabo, K., Stewart, R., and Rees, R. (2015). Public involvement in research: making sense of the diversity. J. Health Serv. Res. Policy 20, 45–51. doi: 10.1177/1355819614551848
Oliver, S., Rees, R. W., Clarke-Jones, L., Milne, R., Oakley, A. R., Gabbay, J., et al. (2008). A multidimensional conceptual framework for analysing public involvement in health services research. Health Expect. 11, 72–84. doi: 10.1111/j.1369-7625.2007.00476.x
Parsons, S., Thomson, W., Cresswell, K., Starling, B., and McDonagh, J. E. (2017). What do young people with rheumatic disease believe to be important to research about their condition? a UK-wide study. Pediatr. Rheumatol. Online J. 15:53. doi: 10.1186/s12969-017-0181-1
Paylor, J., and McKevitt, C. (2019). The possibilities and limits of “Co-producing” research. Front. Sociol. 4:23. doi: 10.3389/fsoc.2019.00023
PCORI (2018). How We Select Research Topics. Available online at: https://www.pcori.org/research-results/about-our-research/how-we-select-research-topics (accessed July 9, 2018).
Pittens, C. A. C. M., Elberse, J. E., Visse, M., Abma, T. A., and Broerse, J. E. W. (2014). Research agendas involving patients: Factors that facilitate or impede translation of patients' perspectives in programming and implementation. Sci. Public Policy 41, 809–820. doi: 10.1093/scipol/scu010
Pramesh, C. S., Venkataramanan, R., Suvarna, V., Goel, N. S., Lakshman, S., Venkatesh, V., et al. (2016). Involvement of general public in biomedical research. Perspect. Clin. Res. 7, 152–155. doi: 10.4103/2229-3485.192029
Pratt, B., Merritt, M., and Hyder, A. A. (2016). Towards deep inclusion for equity-oriented health research priority-setting: a working model. Soc. Sci. Med. 151, 215–224. doi: 10.1016/j.socscimed.2016.01.018
Rawson, T. M., Castro-Sanchez, E., Charani, E., Husson, F., Moore, L. S. P., Holmes, A. H., et al. (2018). Involving citizens in priority setting for public health research: implementation in infection research. Health Expect. 21, 222–229. doi: 10.1111/hex.12604
Resnik, D. (2001). Setting biomedical research priorities: justice, science, and public participation. Kennedy Inst. Ethics J. 11, 181–204. doi: 10.1353/ken.2001.0017
Rhodes, P., Nocon, A., Booth, M., Chowdrey, M. Y., Fabian, A., Lambert, N., et al. (2002). A service users' research advisory group from the perspectives of both service users and researchers. Health Soc. Care Commu. 10, 402–409. doi: 10.1046/j.1365-2524.2002.00376.x
Ridley, J., and Jones, L. Scottish Health Feedback. (2002). User and Public Involvement in Health Services: A Literature Review. Edinburgh: Project Report. NHS Scotland/Scottish Executive.
Rikkers, W., Boterhoven de Haan, K., Lawrence, D., McKenzie, A., Hancock, K., Haines, H., et al. (2015). Two methods for engaging with the community in setting priorities for child health research: who engages? PLoS ONE 10:e0125969. doi: 10.1371/journal.pone.0125969
Royle, J., and Oliver, S. (2004). Consumer involvement in the health technology assessment program. Int. J. Technol. Assess. Health Care 20, 493–497. doi: 10.1017/S0266462304001412
Shah, S. G., and Robinson, I. (2007). Benefits of and barriers to involving users in medical device technology development and evaluation. Int. J. Technol. Assess. Health Care 23, 131–137. doi: 10.1017/S0266462307051677
Sheaff, R., Charles, N., Mahon, A., Chambers, N., Morando, V., Exworthy, M., et al. (2015). NHS commissioning practice and health system governance: a mixed-methods realistic evaluation. Health Serv. Deliv. Res. 3. doi: 10.3310/hsdr03100
Shippee, N. D., Domecq Garces, J. P., Prutsky Lopez, G. J., Wang, Z., Elraiyah, T. A., Nabhan, M., et al. (2015). Patient and service user engagement in research: a systematic review and synthesized framework. Health Expect 18, 1151–1166. doi: 10.1111/hex.12090
Simpson, S., Cook, A., and Miles, K. (2018). Patient and public involvement in early awareness and alert activities: an example from the United Kingdom. Int. J. Technol. Assess. Health Care 34, 10–17. doi: 10.1017/S0266462317004421
Slutsky, J., Tumilty, E., Max, C., Lu, L., Tantivess, S., Hauegen, R. C., et al. (2016). Patterns of public participation: opportunity structures and mobilization from a cross-national perspective. J. Health Organ. Manag. 30, 751–768. doi: 10.1108/JHOM-03-2016-0037
Smith, E., Ross, F., Donovan, S., Manthorpe, J., Brearley, S., Sitzia, J., et al. (2008). Service user involvement in nursing, midwifery and health visiting research: a review of evidence and practice. Int. J. Nurs. Stud. 45, 298–315. doi: 10.1016/j.ijnurstu.2006.09.010
Smith, E., Ross, F. M., Mackenzie, A., and Masterson, A. (2005). Developing a service-user framework to shape priorities for nursing and midwifery research. J. Res. Nurs. 10, 107–118. doi: 10.1177/136140960501000101
Snape, D., Kirkham, J., Britten, N., Froggatt, K., Gradinger, F., Lobban, F., et al. (2014). Exploring perceived barriers, drivers, impacts and the need for evaluation of public involvement in health and social care research: a modified Delphi study. BMJ Open 4:e004943. doi: 10.1136/bmjopen-2014-004943
Staley, K. (2009). Exploring Impact: Public Involvement in NHS, Public Health and Social Care Research. Eastleigh: INVOLVE.
Staniszewska, S., Adebajo, A., Barber, R., Beresford, P., Brady, L.-M., Brett, J., et al. (2011). Developing the evidence base of patient and public involvement in health and social care research: the case for measuring impact. Int. J. Consum. Stud. 35, 628–632. doi: 10.1111/j.1470-6431.2011.01020.x
Supple, D., Roberts, A., Hudson, V., Masefield, S., Fitch, N., Rahmen, M., et al. (2015). From tokenism to meaningful engagement: best practices in patient involvement in an EU project. Res. Involve. Engage. 1:5. doi: 10.1186/s40900-015-0004-9
Thompson, J., Barber, R., Ward, P. R., Boote, J. D., Cooper, C. L., Armitage, C. J., et al. (2009). Health researchers' attitudes towards public involvement in health research. Health Expect 12, 209–220. doi: 10.1111/j.1369-7625.2009.00532.x
Tong, A., Chando, S., Crowe, S., Manns, B., Winkelmayer, W. C., Hemmelgarn, B., et al. (2015). Research priority setting in kidney disease: a systematic review. Am. J. Kidney Dis. 65, 674–683. doi: 10.1053/j.ajkd.2014.11.011
Truitt, A. R., Monsell, S. E., Avins, A. L., Nerenz, D. R., Lawrence, S. O., Bauer, Z., et al. (2018). Prioritizing research topics: a comparison of crowdsourcing and patient registry. Qual. Life Res. 27, 41–50. doi: 10.1007/s11136-017-1566-9
van Bekkum, J. E., Fergie, G. M., and Hilton, S. (2016). Health and medical research funding agencies' promotion of public engagement within research: a qualitative interview study exploring the United Kingdom context. Health Res. Policy Syst. 14:23. doi: 10.1186/s12961-016-0093-4
Viswanathan, M., Ammerman, A., Eng, E., Garlehner, G., Lohr, K. N., Griffith, D., et al. (2004). Community-based participatory research: assessing the evidence. Evid. Rep. Technol. Assess. (Summ). 99:1–8. doi: 10.1037/e439622005-001
Keywords: patient and public involvement, public engagement, co-creation of knowledge, co-production, research commissioning, research priority setting, citizen participation, biomedical
Citation: Tembo D, Morrow E, Worswick L and Lennard D (2019) Is Co-production Just a Pipe Dream for Applied Health Research Commissioning? An Exploratory Literature Review. Front. Sociol. 4:50. doi: 10.3389/fsoc.2019.00050
Received: 13 February 2019; Accepted: 27 May 2019;
Published: 24 June 2019.
Edited by:
Annette Louise Boaz, Kingston University, United KingdomReviewed by:
Alison O'Shea, Kingston University, United KingdomHana Asfour, Parallel Perspective Consulting (Q Perspective), Jordan
Copyright © 2019 Tembo, Morrow, Worswick and Lennard. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Doreen Tembo, d.tembo@soton.ac.uk