Leveraging patient experience data to guide medicines development, regulation, access decisions and clinical care in the EU

Patient experience data (PED), provided by patients/their carers without interpretation by clinicians, directly capture what matters more to patients on their medical condition, treatment and impact of healthcare. PED can be collected through different methodologies and these need to be robust and validated for its intended use. Medicine regulators are increasingly encouraging stakeholders to generate, collect and submit PED to support both scientific advice in development programs and regulatory decisions on the approval and use of these medicines. This article reviews the existing definitions and types of PED and demonstrate the potential for use in different settings of medicines’ life cycle, focusing on Patient-Reported Outcomes (PRO) and Patient Preferences (PP). Furthermore, it addresses some challenges and opportunities, alluding to important regulatory guidance that has been published, methodological aspects and digitalization, highlighting the lack of guidance as a key hurdle to achieve more systematic inclusion of PED in regulatory submissions. In addition, the article discusses opportunities at European and global level that could be implemented to leverage PED use. New digital tools that allow patients to collect PED in real time could also contribute to these advances, but it is equally important not to overlook the challenges they entail. The numerous and relevant initiatives being developed by various stakeholders in this field, including regulators, show their confidence in PED’s value and create an ideal moment to address challenges and consolidate PED use across medicines’ life cycle.

Introduction

Recent years have seen an increased focus on patient engagement and on data collected directly from patients, with the objective of implementing patient-centric decision-making across the medicines’ life cycle. This paradigm-shift from disease-centered to patient-centered originates from the notion that patients are the experts in their own experience regarding their conditions and treatments (1). Moreover, since patients are the end users of medicines, it is fair to assume that patients should participate more in the decisions that directly affect their medical care. One way of supporting this principle is by integrating Patient Experience Data (PED) into medicines development early on, so these data can be analyzed and submitted to regulators for the scientific evaluation on benefit–risk leading to approval of a medicine. Analysis of these data can also generate evidence regarding patients’ experiences, preferences, needs and reported outcomes that could not only guide more downstream processes leading to medicines reimbursement and access decisions by healthcare systems, but also optimize clinical care (2).

Various stakeholders, including regulators, have been concentrating their efforts into examining obstacles to the integration of PED in medicines development and also regulatory decision-making (3). As an example, the European Medicines Agency (EMA) organized a workshop in 2022 (4) to identify ways to improve evidence generation through PED, which would support a key priority for the EU Medicines Regulatory Network to increase patient access to medicines. A consensus on what is considered PED in the EU was reached with the collaboration of regulators, patients, healthcare professionals, industry representatives and academics, and a tentative EU definition of PED was proposed as “data collected via a variety of patient engagement activities and methodologies to collect patients’ experience of their health status, symptoms, disease course, treatment preferences, quality of life and impact of health care” (4).

In this context, it is also crucial to clarify the definition of Patient Engagement (PE) and how important it is for exploiting the collection and use of PED. The 2022 EMA workshop defined PE as “all activities involving interaction with patients to gather their experience on disease, preferences, outcomes and treatments” (4). Accordingly, augmented patient engagement through validated methodologies could, in turn, increase the quantity, quality and use of PED in regulatory processes, such as scientific advice on companies’ proposed plans to develop new medicines and further decisions on benefit–risk assessment of medicines (2, 4). Indeed, the scientific advice provided by EMA to medicine developers on their study design plans at early stages in medicines development is an example of the added value of capturing patients views in regulatory processes, and how these can help shape and have concrete impact on medicines’ development plans. A fifth of the scientific advices with regulatory recommendations provided by EMA to companies between 2017 and 2020 were shaped following patient input. Additionally, concerning the cases where no modifications based on patient input were made, patients agreed with the Agency’s advice on the proposed development plans in 90% of these cases (5). Thus, patients’ experience might add value to early advice regulatory processes, and offers an opportunity for more systematic use of the different types of PED in regulatory decision-making, underscoring the need for further promotion.

Therefore, this review aims to map the existing PED definitions, data collection methodologies and application of different types of PED, namely Patient-Reported Outcomes (PRO) and Patient Preferences (PP) and illustrate how these data can be used in the different phases of medicines’ life cycle. Additionally, and considering the recent advances in the field, some challenges and opportunities regarding the implementation of PED in regulatory decision-making will be discussed, focusing on aspects of guidance, methodological limitations and digitalization.

Types of patient experience data and collection methods

Patient engagement

Patient engagement in the development and regulation of medicines is an increasingly important element that aims to incorporate the views of patients throughout medicines’ life cycle, from research and development to regulatory approval and post-marketing surveillance. This approach recognizes patients as key actors whose insights and perspectives can significantly contribute to the safety, efficacy and acceptability of medicines (4–6). Although patient engagement is a relatively new field, experience to date allows us to describe some of the ways in which patient engagement is being integrated into the development and regulation of medicines (4).

By participating in the design of clinical trials, patients can provide valuable input at the design stage, helping researchers to prioritize outcomes that are most relevant to patients, select patient-friendly study protocols and improve recruitment strategies. Patients can also help develop informed consent materials to ensure that the information provided to participants is clear, understandable and responsive to patient concerns and preferences (7).

During the drug development process, patient advocates can work with pharmaceutical companies, regulatory agencies and even with the academic sector to provide insights into the lived experience of their condition, including treatment preferences, unmet needs and tolerability of potential side effects. This input can inform drug development strategies and decision-making processes (2, 5). In particular, regulatory agencies are increasingly involving patients in scientific advisory groups, public hearings and consultations to gather patient perspectives on benefit–risk assessments, prescribing requirements and post-marketing surveillance plans. Patient engagement continues after regulatory approval through initiatives such as patient registries, adverse event reporting systems and other measures that enable monitoring of treatment effectiveness and safety in real-world settings (2, 5).

Health technology assessment (HTA) agencies are also increasingly involving patients in their processes, taking into account their experiences with medical interventions, their views on health-related quality of life measures, and value assessments that inform reimbursement decisions (2). Particularly, the introduction of the Joint Clinical Assessment procedure in Europe, a centralized European framework for assessing clinical evidence within HTA assessments, is expected to promote PE in the HTA evaluation. In this line, EUnetHTA issued a guidance about patient engagement in 2023 (“Guidance on Patient and Healthcare Professional Involvement”) establishing a framework for the involvement of external experts, in which patients are included (8). The EU Health Technology Assessment regulation, including Joint Clinical Assessment, represents the most significant change in the European healthcare landscape in recent years. However critically, pricing and reimbursement decision-making remains the responsibility of individual member states.

To improve the incorporation of patient engagement in decision-making, a collaborative, multistakeholder approach is essential. An example of such collaborative efforts is the FDA/EMA Patient Engagement Cluster, established in 2016 to facilitate regular exchange of best practices regarding patient engagement throughout the medicine’s life cycle (9).

In 2018, the European Patients’ Academy on Therapeutic Innovation (EUPATI) issued a set of guidance documents to support the implementation of patient engagement among key stakeholders. Firstly, the organization clarifies how the term “patient” can be use more precisely to reflect different inputs, by establishing definitions for “individual patients,” “carers,” “patient advocates,” “patient organization representatives” and “patient experts.” Additionally, for regulators and regulatory agencies, the EUPATI suggests patient participation through initiatives such as: a network of patient organizations; a forum established within the regulatory authority composed by patient organizations; the creation of a group of individual patients expert in their disease and its treatment; cooperation in the field of communication for information dissemination; a dedicated program for capacity-building, namely about the regulatory system, and financial support for patients contributing to the regulator’s activities (10).

By integrating patient engagement into the development and regulation of medicines, stakeholders can ensure that medical products are developed, evaluated and regulated in a way that reflects patients’ needs, preferences and values. This patient-centered approach promotes transparency, accountability and trust in the healthcare system, ultimately resulting in improved health outcomes and quality of life for patients (4, 6, 10).

Patient-reported outcomes

According to the EMA, Patient Reported Outcomes (PROs) “refer to a health/treatment outcome reported directly by the patient without the interpretation of a clinician or another person” (4). This implies that the patient’s experience is captured without modification or interpretation by a healthcare professional or anyone else (11, 12).

PRO data are collected through patient-reported outcome measures (PROMs). PROMs are instruments (e.g., questionnaires or diaries) filled out by patients themselves as a self-report and have to be validated (12, 13). Therefore, PROMs may provide helpful information regarding subjective outcomes/concepts such as symptoms (e.g., pain, fatigue, or nausea), mental functioning, physical functioning, well-being, adherence to treatment, satisfaction with treatment, and treatment preferences (11–13).

PROMs can be classified as generic or disease-specific. Generic PROMs can be used irrespectively of disease or patient, and they aim to measure single or multidimensional health concepts such as cognitive function, performance status, symptoms (including pain). Additionally, generic PROMs can also be applied in healthy subjects, for example, to evaluate overall health, a nontreatment intervention study, or a method to assess a PROM. Since they can apply to various patient populations, PROMs enable the comparison across different treatments or groups of patients under different environments/contexts and conditions. Nevertheless, generic PROMs lack sensitivity, can be less responsive to change and may fail to capture relevant disease-specific aspects when compared to disease-specific PROMs (12–14).

On the other hand, disease-specific PROMs provide a quantification of the impact of a specific disease. These instruments tend to be of higher relevance and responsiveness to changes, providing more comprehensive information on specific aspects of a disease. Nevertheless, they do not allow comparisons with general population data, across different patient populations and distinct diseases or conditions (11, 12, 14–17). Thus, it is fundamental to select PROMs (generic or disease-specific) that are fit-for-purpose considering the study’s objectives and the characteristics of the population subject to analysis (12). Table 1 summarizes the main advantages, limitations as well as gives examples of both generic and disease-specific PROMs.

Table 1

Table 1. Advantages, limitations and examples of generic and disease-specific Patient-Reported Outcome Measures.

Considering both advantages and limitations of each category of PROMs, some authors recommend the combined administration of generic and disease-specific PROMs for a more thorough appraisal of the outcomes to be studied (14). Hybrid measurement systems, also known as modular packages, use a generic health measure plus complementary disease-specific instruments. For example, the FACIT (Functional Assessment of Chronic Illness Therapy) system comprises a generic Health-Related Quality of Life (HRQL) measure and complementary disease-specific subscales. Most importantly, this modular approach may also improve comparability of PRO data (14, 15, 18).

At a data collection level, PROMs can be a questionnaire, a self-report, or an interview, among other strategies; in either case, only the patient’s answers are considered, unless this is not possible, in which case caregivers may report these outcomes (12). The development of a PROM should start with the construction of a conceptual framework in which simpler concepts can be properly clustered into more complex groups for measurement purposes (11, 19). This conceptual framework for the development of a PROM should consider, at its core, the desired claim, and it should not only be developed based on literature review, experts’ and physicians’ knowledge, but also stem from patient input and experience.

During this development process, patient input is vital to adapt and confirm the conceptual framework and can be obtained through a variety of methods, such as exploratory patient interviews, focus groups and other feedback strategies (19–21). Patient advocates and experts can attest if the concept is meaningful; suggest other domains to be included in the instrument; provide feedback to develop an instrument measure that consider diminishing missing data strategies; share cultural and linguistic details to be considered; advise on the medium and technologies used in the data collection phase; and even participate in the dissemination of the PROM, promoting its use among their networks (19). Thus, involving more differentiated patient input might be of added value in this development phase of a PROM (19, 21). Overall, the PROM development framework will provide the rationale regarding what outcomes to measure and how this is to be accomplished. It should also determine the target population and research application, which will, in turn, define the instrument’s characteristics (20, 22, 23).

Measurement property testing is also required, which will confirm, prior to the creation of the instrument itself, that the findings drawn from the application of the PROM are valid and relevant (19, 21). Content, criterion and construct validity correspond to three measurement properties that are often assessed. Table 2 summarizes the definitions of these measurement properties and presents some examples of their application (24).

Table 2

Table 2. Examples of the practical application of content, criterion and construct validity.

Content validity corresponds to the degree to which the instrument accurately assesses the intended concept it was designed to measure (24). To prove its content validity, robust evidence should be gathered demonstrating that the instrument measures its purpose and that its items and domains are appropriate regarding the concept intended to be measured, population, and use (24). When it comes to a PROM, the qualitative work with patients is relevant since a PROM is meant to measure important concepts from the patient’s perspective. This way, the PROM should be built according to the patient’s comprehensiveness and perspective, and not according to clinicians and other stakeholders. Criterion validity, a related concept, must also be addressed as it intends to describe how the score of a particular instrument relates to a gold standard, demonstrating how similar the instrument used is to the gold standard instrument (24).

Construct validity corresponds to another type of measurement properties and is established as the extent to which a score given through one instrument matches consistently with the hypothesis proposed, regarding the claims being measured, and represents the relationship between the score and the theoretical claim (24, 26). A PROM must also show reliability, a measurement property which also contributes to its validity. Reliability means that the instrument will reproduce consistent results over time, if applied at different time points, for example throughout a clinical trial (24).

These instruments should also be able to detect if clinically important changes have occurred in an outcome, according to the study’s objectives. The degree to which a PROM can detect the changes of the measures over time should be well defined in advance (28, 29). Therefore, another relevant measurement property of the PROM is the minimal clinically importance difference (MCID), which can be defined as a measure of the smallest change in an outcome that patients perceive as important and may therefore require a change in patient’s management (30). To illustrate this concept, while MCID has been used to help healthcare professionals to assess if an intervention might cause a clinically important change, applying it to PROM allows patients to participate in the definition of what they find to be a meaningful change (30). The use of MCID is progressively expanding to comprehend the clinical effectiveness of a specific treatment, establish clinical practice guidelines, and accurately interpret trial outcomes. Despite this, different methods for MCID calculation (e.g., anchor-based methods, distribution-based methods) can result in heterogeneous results which can make the evaluation of the treatment’s effectiveness difficult. Consequently, research has been developed to calculate MCID thresholds for common PROMs to obtain more accurate results (30, 31).

After having reached the initial stages of PROM development, cognitive interviews are usually carried out in order to check how the items incorporated in the PROM are assessed by individual patients or carers in terms of clarity and relevance (32, 33). These cognitive interviews with patients then allow researchers to compare if the understanding of the instrument matches the idea that instigated its development and whether issues of literacy, jargon, technical language, or culture-specific constructs exist. The research team organizes these interviews until a saturation of information is reached, i.e., there is no additional information if more interviews were conducted. Consequently, the results and content obtained from the cognitive interviews is then considered during a revision phase where items can be altered accordingly (33). Patient organizations may play a crucial role in identifying individual patients or carers that consent to participate in these cognitive interviews.

Finally, PROM’s revision does not end in the development phase. During its use, additional clinical data, other patients’ inputs and considerations regarding changes of the environment can be gathered to instigate new revision processes, assuring the quality parameters that a PROM needs to reach. As an example, if a cognitive PROM’s item assesses the patients’ capacity to make a phone call, this indicator might need to be reviewed to start considering a scenario where patients use a smartphone, instead of a traditional landline phone, since both represent different cognitive abilities (33).

Patient preferences

Patient Preferences (PP), in the medicines’ regulatory setting, can be summarized as instruments capable of indicating “how desirable or acceptable is to patients a given alternative or choice among all the outcomes of a given medicine” (4). PP information may complement traditionally collected safety and efficacy data, ensuring that the patients’ preferences, needs, and values will guide decision-making based on what patients are disposed to consent in terms of benefits and harms. Therefore, it is anticipated that patient-centered decisions will be more trusted by patients, clinicians, and the general public. Such decisions might also increase patient satisfaction, by meeting their needs and expectations, and ultimately improve their adherence to the treatment, leading to better health outcomes and effectiveness (34).

In this context, PP can generate different inputs when compared to PRO data. Notwithstanding, PP information is able to illustrate what outcomes are a priority and what medical needs are required to be met. Moreover, PP might also be able to promote and highlight the need of shared decision-making, resulting in outcomes more relevant to patients and optimized transparency (35). Concretely, a study by Fifer et al. in the context of multiple myeloma illustrates that patient preferences may vary from patient to patient, while professionals involved in patient care may place different value on patients’ preferred treatment outcomes (36).

PP information can be collected through mainly two different methods. Preference exploration methods (qualitative) collect in-depth descriptive data about patient experiences, perspectives, and the treatment attributes that are most important to them. Qualitative research is usually unstructured or semi-structured, such as individual interviews, focus groups and open-ended survey questions (37, 38). In turn, patient preference elicitation methods (quantitative) quantify patient preferences in a structured manner. They collect numerical data that assesses the relative weights assigned to different attributes and which compromises patients are willing to accept, allowing to statistically detect preference heterogeneity. Discrete choice experiments (DCE), threshold technique, swing-weighting, and best-worst scaling (BWS) are some examples of preference elicitation methods (39). Table 3 summarizes some examples of PP exploration and elicitation methods.

Table 3

Table 3. Examples of PP collection methods.

In the context of an HTA study to establish the value of PP studies, a research program conducted by the National Institute for Health and Care Excellence (NICE) and the Myeloma UK patient charity identified DCE as the most used approach for eliciting patient preferences, due to its robustness and similarity to real-life decisions (40). This method is used in several studies to explain or predict a choice from a set of mutually exclusive alternatives (40–42). In the DCE approach, patients are asked to choose their preferred option, based on the attributes and their respective levels (effectiveness, duration of treatment benefit, risk of mild and/or serious side effects and mode of administration, for example). There may also be an “opt-out” option which allows the patient not to choose any alternative if neither are acceptable, providing a more accurate picture of the expected uptake of that treatment (37, 41–44). Therefore, this method is useful not only to assess the relative importance of treatment attributes, but also to measure the highest level of risk a patient would be willing to tolerate in return for a specific benefit (43).

Nevertheless, DCE is not universally applicable. For example, it is not a suitable method when there are many attributes to consider, when patients cannot process a large amount of information or require approaches of easier comprehension, or in case of small samples which do not allow valid statistical analysis, such as rare diseases (45, 46).

Although, PP collection methods can be categorized as either exploration or elicitation methods, they can also be classified as revealed-preference or stated-preference. Revealed-preference methods rely on observing real-world choices and behaviors to draw conclusions. Examples of revealed-preference methods encompass patient-preference trials and direct inquiries within clinical trials. Conversely, in stated-preference methods, patient preferences are elicited through hypothetical experiments. Stated-preference comprises methods such as direct assessment questions, DCE, threshold technique, conjoint analysis, and BWS (45).

Most patient-preference studies in healthcare research use stated-preference methods. Revealed-preference methods are only possible for existing products in the market, and thus are not applicable to novel medicines that are not yet available. Due to the hypothetically nature of stated-preferences, it has to be assumed that patients would actually choose the options they say they would. While revealed-preferences can avoid hypothetical bias, they still are subject to other bias, such as individual financial considerations. Furthermore, revealed-preference methods often are unable to infer the relative weights of individual attributes (37, 47, 48).

Patient experience data use in medicines’ life cycle

Patient reported outcomes

Medicine development and regulatory approval

PRO data obtained from clinical trials can inform the medicine development phase (e.g., study design), market authorization and post-authorization monitoring process, supporting the determination of benefit–risk balance and labeling claims in different moments of assessment (49). The patient perspective can complement traditional endpoints, such as objective clinical outcomes and laboratory parameters, which may not always fully capture the impact of a treatment. For instance, PRO data may help in disclosing treatment-related symptoms that need to be addressed or support choice between two medicines with a similar efficacy profile (50). As an example, overall survival (OS) is a common outcome assessed in randomized controlled trials (RCTs) to measure the clinical benefit of an experimental intervention, especially in oncology. However, this evaluation overlooks the experience of the patient during treatment (51). Consequently, these studies often include in the analysis the use of a PRO to assess health-related quality of life (HRQoL). Thus, a more holistic approach of the treatment might be evaluated, complementing other physiologic or clinical endpoints (48). PRO data can also be integrated into medical product labeling to provide information on safety and tolerability, and to support specific claims of treatment benefits. This can be relevant considering that the patients’ have first-hand knowledge of the effects of a certain medicine (12, 13, 52).

Several initiatives have been developed to promote best methodological practice for use of PRO in clinical trials. The PROTEUS Consortium, a project funded by the pharmaceutical industry, has identified core documents on designing PRO protocols (53), selecting PROM (50), analyzing PRO data (54), reporting PRO findings (55), among other aspects. These tools have been aggregated and described in detail in the PROTEUS Handbook (56). Selected PRO related guidance documents and tools are briefly described in the table below (Table 4).

Table 4

Table 4. Selected documents and tools for PRO use in medicine development and approval.

In recent years, regulators have been paying more attention to PROs, especially in their medicine’s approval processes due to the rising interest of this area of knowledge. A review of 497 European Public Assessment Reports (EPAR) of authorized medicines and 19 EPARs of withdrawn medicines, published from 2017 to 2022, found that 48.3% and 52.6% stated use of PRO, respectively. In this study, PROs were mostly considered as secondary (53.3%) and exploratory endpoints (18.8%); in 32.5% of the cases, the PROs used were related to general quality of life; and PRO use was particularly low in some therapeutic areas, such as infectious diseases (15.2%) (58).

Considering an example from the oncology field, the results from a study conducted to evaluate the use of PROs for the approval of oncology medicines in the EU, from 2017 to 2020, demonstrated that out of the 104 clinical trials conducted for the approval of the 76 medicines studied, PROs were considered as a secondary endpoint in 57.7% and exploratory in 29.8%. These 76 medicines corresponded to 128 indications, however only in 22 of them the PROM use was mentioned in section 5.1 of the Summary of Product Characteristics (SmPC). This emphasizes how the contribution of PROs to evidence accepted for establishing the positive benefit–risk assessment of a new medicine can be challenging, mainly due to aspects related to study design, PROM selection and missing data (59).

Finally, a study looking into establishing the use of PROs in the approval of orphan medicines, revealed that, in the European context, PRO use in orphan medicines was lower when compared with all medicines’ approval. Nonetheless, when compared to the results from a similarly conducted study in the United States of America, the authors state that FDA approvals included significantly less PROMs in their processes in comparison with the EMA approvals, during the study period (60). This could be due to a relatively longer experience within the European context following the publication in 2005 of the HRQoL guidance, which suggests that regulatory guidance can stimulate the use of PROs (60, 61).

Health technology assessment

It is also necessary to progress toward a more patient-centric evaluation of health technologies (62, 63). In other words, some authors argue that there should be a reframing of the concept of health value that includes what patients perceive as value (63). In the HTA setting, the perception of value differs from the need of positive results of critical endpoints assessed by regulators. Particularly, HTA focus on comparative analysis to guarantee that the intervention reimbursed has the ability to diminish burden in the system, while facilitating access. In this sense, there is an opportunity to use PROMs in HTA to gather information on a treatment’s added value, which, in turn, can inform market access, reimbursements, and pricing negotiations. Incorporating PRO data in HTA can be essential in assessing the effectiveness and value of health technologies, and ultimately improve efficiency in resource allocation (28, 62).

A study analyzed HTA appraisal reports that contained PROs as endpoints between January 2018 and March 2020 and that were submitted to the Canadian, French, German, Scottish and British HTA agencies. PRO data were found in 77% (48 out of 62) of the reimbursement submissions (62 medicinal products assessed in total). PROs were included in 23 appraisals as a primary or a key secondary endpoint and 43% of these assessments (10 out of 23) received approval for reimbursement from at least three agencies. However, the authors report that most of the PRO data submitted received unfavorable assessments from various agencies. The main reasons listed were the absence of a predefined analysis for responders, utilization of a non-validated tool for collecting PROs, uncertainty in PRO measurements and meaningful changes in scores. Thus, the study showed that there is a considerable variability in HTA assessment of PROs and there is room for companies to better prepare their submissions (29).

Real-world setting

Conventionally, PRO data are more often collected in clinical trials to support regulatory, HTA and clinical decision-making. However, stakeholders are broadening their interest in better understanding the patient perspective in the real-world (routine clinical care) setting where PRO data collection has been of added value (17, 64). Considering a clinical care example, a study on patients who suffered acute stroke discharged from a tertiary care hospital demonstrated that the results of a clinician-reported outcome did not align with the patients’ perception obtained through PRO 3 months after the event (65).

Furthermore, real-world PRO data can also inform the early stages of medicine development. By using PROs to gather insights on the natural history of disease, disease burden and unmet needs, researchers can select the most suitable endpoints for the subsequent clinical trials. Finally, PROs may be a source of evidence in early access, compassionate use, and off-label use contexts (66).

A review of all publicly available data on post-authorization safety studies (PASS) protocols submitted to the Pharmacovigilance Risk Assessment Committee (PRAC), EMA and European Network of Pharmacovigilance and Pharmacoepidemiology (ENCePP) repositories, from 2012 to 2015, demonstrated that PRO use among regulatory medicines post-authorization safety assessment is still low. Almost half of the electronic register of PASS (EU-PAS) entries had the protocols available, of which only 14% included PRO data. Such PROs assessed disease burden, symptoms and quality of life (64).

In 2023, a study using qualitative interviews to assess the degree of PRO implementation in the real-world setting found consensus that the use of PRO in routine clinical care setting is not yet well established. The main challenges identified were lack of infrastructure and resources to collect PRO systematically in the real-world setting and few financing opportunities (67).

Patient preferences

Medicine development and regulatory approval

Patient preferences (PP) information can address the relative weights between benefits and risks and what would be the maximum acceptable risk for a given health benefit, with such information being particularly useful in more complex benefit–risk evaluations for early access. Some reports suggest that information generated may be included on the product label to inform the patients regarding benefits and risks (39, 41, 68). The current regulatory thinking is that PP information can help healthcare professionals to identify situations of high preference heterogeneity and where an in-depth understanding of individual preferences is needed.

Despite the recognition of PP’s potential in regulatory decision-making, they are not yet systematically integrated in medicines’ life cycle, particularly at the medicine approval stage (39). In the literature, around 30 methods for incorporating PP are described, and some authors suggest that this number causes uncertainty for stakeholders when choosing a method and thus limiting its usage (69, 70). Additionally, in the 2022 workshop organized by the EMA, stakeholders recognized that PP studies might not have been used much in decision-making because they are complex and time consuming, and there is a lack of methodological harmonization. Stakeholders also encourage the development of PP tools based on practical experience to avoid rigidity in the implementation process of these methodologies and also of more guidance that might diminish this implementation’s hurdle (4, 70).

In 2015, the Innovative Medicines Initiative (IMI) launched the Patient Preferences in Benefit–Risk Assessments during the Drug Life Cycle project (Project PREFER), a collaborative effort between public and private entities such as the industry, academia, patient organizations, and a HTA organization. PREFER aimed to explore when and how PP should be considered to enhance decision-making process by regulatory and HTA organizations (71). This 6-year project was divided into work packages. The methodology work package was responsible for investigating the concerns stakeholders may have about using PP studies and provided recommendations on which methodologies should be used. The case study work package has conducted several studies based on the recommendations previously released. Finally, the recommendations work package launched a set of experience-based recommendations based on their work for PP inclusion throughout the medicines’ life cycle. Consequently, these recommendations are anticipated to aid in the formulation of guidelines for industry, regulatory and HTA institutions. From this project arose several publications, training materials, webinars to increase stakeholders’ familiarity with PP studies, and operational guidance and additional resources to assist in the design and implementation of these studies (71).

Health technology assessment

PP could inform HTA on non-health attributes that are not captured by traditional assessment tools (34, 39, 68, 72). In particular, PP can be useful in cost-effectiveness analysis balancing clinical outcomes and Quality-Adjusted Life Years (QALYs) and possibly in reducing uncertainty when it comes to adherence in the real-world setting (39).

Several HTA bodies, such as NICE in the UK, the German HTA body IQWiG, and the Belgian HTA body KCE, have shown growing interest in using patient preferences for HTA, and have engaged in projects to advance the field of PP assessment for HTA (40). However, despite this recognition, integration of PP in HTA processes remains a challenge and has not yet been widely used (72). Representativeness in the HTA process seems to be one of the limiting factors that hinder greater application of PP data, so more studies should be carried out in this regard (39).

Real-world setting

Throughout the post-marketing phase, PP may inform patient acceptability of a given therapy in routine clinical care settings, extensions of indications, post-marketing surveillance, specific treatment opportunities, need for optimizing existing medical products, and product innovation (39, 68, 73). For instance, as stated in the PREFER recommendations, a preference study could be set up to understand the patient acceptance of rare but serious safety signals observed in the post-approval phase. Finally, investigating revealed preferences in the real-world setting could add to the information provided by stated preferences and potentially improve external validity (43). Nonetheless, there are not yet many examples of the real application of PPs in the real-world setting, according to the literature. This limited incorporation of PP might be due to PP’s methodological challenges related to capturing and using real-world data for decision-making (limited access to databases and registries, data quality, complex analytical methodologies and confounding, for example) (74).

Challenges and opportunities

Regulatory guidance

As stated so far, the potential and number of applications of PED are vast. Moreover, all stakeholders seem to be aware of the importance of PED as a vehicle to develop a more patient-centric approach in healthcare and in the medicines’ regulatory field, in particular. This fact might be demonstrated by the number of past and ongoing efforts to leverage PE and PED in regulatory decision-making. In a review in 2023, 53 relevant global regulatory and HTA initiatives regarding the use of PED were highlighted (2). Additionally, this review showed that the majority of these initiatives (13) were of an international nature, followed by a tie between North America and Europe, which contributed equally with 11 initiatives (2).

In the European setting, EMA is demonstrating how this patient-centric vision has been put into practice by engaging patients and their representatives in its scientific committees’ and also in its public hearings (6). The “Multi-stakeholder workshop: Patient experience data in medicines development and regulatory decision-making” is also a recent evidence of the continuation of this vision and has provided essential multi-stakeholder reflections on how to progress integrating PED in the EU and addressing challenges in implementation (4, 75). In this context, it is also worth noting the possibilities of increasing the adoption of PED offered by the development of the European Health Data Space (76) and the reform of pharmaceutical legislation, both activities in progress at the time of writing this manuscript (77, 78).

In the US, FDA has developed an initiative called “Patient-Focused Drug Development” (PFDD) under the 21st Century Cures Act and The Food and Drug Administration Reauthorization Act of 2017 Title I. The PFDD initiative aimed at developing four guidances about methodological considerations that should be contemplated when submitting PED to FDA. Prior to these guidances, public workshops were organized and as Supplementary material, FDA published two other documents “Submitting Clinical Trial Datasets and Documentation for Clinical Outcome Assessments Using Item Response Theory” and “Submitting Patient-Reported Outcome Data in Cancer Clinical Trials” (79). Other stakeholders have also been able to share some feedback on these initiatives. In particular, various organizations (pharmaceutical industry, patient organizations, public institutions, professional associations) have provided comments on the methodological challenges related to PED as requested by the FDA (80). In addition, the Biotechnology Innovation Organization (BIO) also issued a white paper with recommendations for the PED table included in FDA review documents (81).

It is clear that regulatory agencies, namely EMA and FDA are engaged in efforts to create a more prepared ecosystem for the challenges posed. The main initiatives of interest that intend to promote and facilitate the use of different types of PED, developed by the two agencies, both in Europe and in the US, are briefly presented in Supplementary Table 1.

In the HTA landscape, the implementation of the Joint Clinical Assessment in the European setting presents an opportunity for PE and, consequently, for the use of PED. The guidance on Patient and Healthcare Professional Involvement advocates for the inclusion of individual patients with collective disease expertise as external experts, thereby allowing them to provide insights about treatment pathways, evaluate study designs and support the definition of the PICO framework for assessments. According to the same guidance, data derived from this involvement is expected to be collected through questionnaires, interviews and consolidated meetings (8). Despite the limited large-scale experience with these guidelines, there will be no need for an extended wait for feedback, as the Joint Clinical Assessment becomes mandatory for oncology medicines and advanced therapy medicinal products (ATMPs) in the beginning of 2025 (82).

Despite these efforts, stakeholders are calling for more regulatory guidance to be developed. On one hand, the medicines developers are asking for harmonization of the concepts, definition of standard methods and investing in collecting these data, and for the data to have a concrete role in decision-making processes. On the other hand, patient advocates and patient organization representatives and other stakeholders are demanding more transparency on how PED is evaluated and whether/why it is accepted as evidence to establish the benefit/risk so that they can assess the impact of their involvement in shaping development plans and regulatory decision-making (1, 4, 35, 83).

Methodological aspects

Methodological aspects still represent a considerable challenge for most stakeholders, which is one of the reasons why more guidance is requested. Missing data is a well-known problem in PED, particularly in PRO analysis, and it is defined by the SISAQOL-IMI Consortium, a collaborative project supported by both public and private organizations under the Innovative Health Initiative (IHI), as any “data that would be meaningful for the analysis of a given research objective or estimate but were not collected” (54). Therefore, the study protocol should describe how missing data will be accounted for in the analysis. The proportion and reasons for missing values should also be reported. For example, if only patients who feel better can complete the questionnaire, it will likely introduce a selection bias, resulting in misleading results (54, 84). The high rate of missing data has been attributed to the logistical difficulty and time-consuming implementation of PRO assessment in clinical trials, which is why it is often deprioritized (85). To overcome the barriers associated with missing data and improve completion rates, the scientific literature recommends several strategies, including: assuring that patients understand the purpose of the assessment and value the utility of PRO data; giving clear instructions to participants, and ensuring that physicians are also aware of the value of PROs and receive tutoring on the compilation and interpretation of data originated by PRO; assigning a person responsible for PRO data collection in a given study, checking for completeness of questionnaires, and ascertaining the reasons for missing values (13, 79, 86). The SISAQOL Consortium has issued a set of recommendations on how to manage missing data (54).

Regarding PP, researchers are developing efforts to establish which PP methods are more suitable for each stage in the medicine life cycle. From the 33 PP exploration and elicitation methods reviewed, a sample of 13 propitious reference exploration and elicitation methods were selected, which corresponded to the ones that were more likely to meet decision-makers’ criteria according to the study’s inquired experts. This thoughtful approach could assist both researchers and decision-makers in selecting PP instruments appropriate for the intended purpose, thereby reducing the initial burden associated with the decision-making process (70). This study is a good illustration of how stimulating the academic community can aid regulatory guidance to leverage the use of PED.

Participant burden for the collection of PED is another complex issue associated with the methodological inherent aspects, since it depends not only on the frequency and timing of assessment, but also on the trial duration, length and/or formatting of the instrument, mode of administration (paper, telephone or web-based), literacy level, the complexity of instructions, and disease severity and/or treatment toxicity (87, 88). An excessive respondent burden may result in unwillingness to complete the questionnaires and, ultimately, will result in missing data. Therefore, extensive, and time-consuming surveys are generally unpractical (88, 89).

However, patient education might be a tool to mitigate the challenge previously mentioned, since educating individual patients or even carers may enhance their understanding of the importance of filling in a questionnaire regardless of its size. Moreover, this practice can minimize the risk of information bias. In turn, the challenge is to provide sufficient information on the patients’ role and the purpose of their input without being too burdensome, and without creating selection bias (68). Engaging with patients involved in representative associations, who are generally better informed due to training programs provided by patient organizations they are part of, may be an enticing solution to overcome the question of whether patients are properly educated.

It is worth highlighting an initiative by the International Council on Harmonization (ICH) that could represent a great opportunity, at a global level, to develop tools both capable of harmonizing PED concepts and understandings, and of responding to the methodological limitations raised by researchers and stakeholders. Concretely, in 2021, the ICH issued a reflection paper entitled “Proposed ICH Guideline Work to Advance Patient Focused Drug Development,” anticipating new ICH guidelines (90). In addition to identifying critical areas where integrating the patient’s perspective could enhance drug development and inform regulatory decision-making, this reflection paper outlines specific strategies for the development of two new ICH guidelines: (1) a guideline on “what to measure in a clinical trial, including refining the set (list) of important impacts and concepts from patients, to select, modify or develop clinical outcome assessments (COAs) that can demonstrate change and define endpoints and meaningful change”; (2) a guideline on “methods for elicitation/collection, analysis, reporting and application of qualitative or quantitative assessments of the relative desirability or acceptability to patients of specified alternatives or choices among outcomes or other attributes that differ among the alternatives” (90). When implemented, it is expected that the guidelines will have the capacity of fostering global harmonization of PED, so these can be collected, analyzed and used in a way that is methodologically robust and suited for patient groups collecting PED, the pharmaceutical industry, regulatory agencies and even healthcare systems overall (90).

Digitalization

Technology and digitalization provide a unique opportunity for boosting passive and active patient data collection. For instance, some applications provided by personal devices (smartphones and smartwatches) can passively collect relevant data to inform a trial. These devices can unlock even greater potential when combined with medical devices or active data collection methods to fully understand patients’ experiences with a disease and/or treatment. However, ethical constraints, such as collection of sensitive data and GDPR, may present challenges that need to be overcome (91).

The possibilities offered by mobile health (mHealth) have increased over the last few years with the expanding use of phones and wearables. Consequently, regulatory agencies and medicine developers have been considering them for several phases of product development, such as clinical trials and pharmacovigilance activities. Since technological devices have lots of versatility, they have the benefit of being adaptable to a wide range of concepts of interest. Moreover, they can collect tremendous amounts of data in real time, when compared with traditional data collection measures (92), while reducing the burden of data collection on the patient. Although, PED coming from these advanced technologies could be of relevance in clinical trials setting, understanding how these data can be validated, used and interpreted is paramount when submitting applications/scientific data for regulatory assessment (66). In post authorization environments, mHealth can be used for pharmacovigilance purposes. For example, IMI WEB-RADR launched a mobile phone app that enables healthcare professionals and patients to report spontaneous AE reports directly to the competent authorities. Another example is the Med Watcher app, supported by the FDA, which has been used to improve the efficiency of reporting individual case safety reports (93).

Digitization also includes using social media as a potential data source for PED. Social media provide an easy platform for people and patients to exchange information, experiences, and opinions, along with the rise of new possibilities for patients to get more information on their diseases and treatments, and has contributed to the increase of new online patient groups who discuss health-related matters and that can also interact with healthcare providers and even academics (94). An example of social media contributing patient experience to define burden of a new disease in a crisis setting is Long COVID, where the patient experience reported in Twitter helped to start patient advocacy and scientific discussion on this new condition (95). Social media may hold potentially valuable safety information in several contexts (96–98), detecting undisclosed signals, capturing less frequently reported AE, or detect AE earlier than traditional methods. In addition to signal detection, social media could also detect the potential for abuse or misuse of certain medications, evaluate acceptability and risk perception by patients, as well as provide a validation tool for signals reported in other surveillance systems (99–102).

Yet, the validity of information posted online by patients requires careful analysis and interpretation, as patients may misinterpret and/or misreport their diagnosis, clinical outcomes, symptoms, or treatment regime (98, 103, 104). Given the volume and complexity of this data, the development and standardization of more effective and robust mining and processing methods is necessary and is yet to be established. The enhancement of mining strategies may provide higher quality data, potentially broaden the scope and utility of social media, provide more meaningful results, and reduce the burden of storage and analysis (100–102). Moreover, the population using social networks cannot be seen as representative of the overall population, since the demographics of individuals using these channels are barely known and may differ across different social media networks (96, 105). Although in recent years, users aged over 75 are getting more engaged with social media, users tend to be younger adults, with higher education and less ill or functionally disabled (96, 105).

In conclusion, digitalization has boosted the development of mHealth, and the processing of data extracted from social media. To fully use this data in regulatory decision-making, compliance with ethical and legal considerations need to be ensured; a sound regulatory framework should be in place and any methodological challenges that arise need to be overcome through further research (93). Despite all the additional challenges, the use of these data offers an opportunity to engage patients in a completely new way, providing insights that may have never been obtained before with conventional data collection methods.

Conclusion

This review tries to present an overview of the status of PED and potential for use in different settings of medicines’ life cycle at the time of this publication, including discussing challenges and opportunities to maximize regulatory use of PED. All parties acknowledge the importance of ensuring that patients’ views, values and preferences about the effects of a medicine are an essential part of the information at the early stages of the development and in any subsequent decision about its authorization and use.

Regulatory agencies and other stakeholders recognize PED integration into decision-making as a strategic priority, acknowledging the paramount importance of placing the patients’ preferred outcomes and their perspectives at the forefront of medicines development and evaluation. Increasing patient engagement initiatives have not only encouraged discussions on the use of PED, but also hold the potential to promote the generation and collection of such valuable data. Furthermore, these patient engagement initiatives could contribute to patient advocacy strategies, which in turn could enhance the medicines regulatory field by providing new and thoughtful insights. This creates a self-sustained cycle of sharing, promoting and developing good practices regarding the use of PED among the different stakeholders involved, including patient organizations. These organizations play a crucial role in liaising these initiatives and serving as a focal contact point between all interested parties.

Still, there is a long way to go to reach optimal use of PED in every stage of a medicine’s life cycle. This literature review indicates that PROs are the type of PED where experience to date is more established and are more advanced in implementation, quite possibly due to their intrinsically more quantitative nature. In addition, current evidence indicated that there is more research and regulatory guidance on PROs than for other types of PED, especially in the medicine development and regulatory approval settings. Nevertheless, PRO data are no substitute for PP data, which bring a different contribution to the scene by allowing patients, based on their own benefit–risk analysis, to opt for a particular choice. Even so, the evidence seems to suggest that PP data needs further research to achieve greater potential for its use in regulatory processes.

Several initiatives are ongoing at a global level, implemented by the various stakeholders, particularly by regulatory agencies, to enhance the use of PED in decision-making, while taking into consideration how PED can also contribute to HTA and clinical decisions. However, stakeholders are calling for additional efforts as there are challenges that have not yet been overcome. It is important to harmonize both the materials already published and the knowledge extracted from them, while also presenting solutions to the methodological challenges inherent to these data’s nature. In this context, the publication of the ICH guidelines could mark the first step along this path. On the other hand, digitalization has put the discussion of generating PED through mHealth and social media on the table. Although these bring their own challenges, the opportunity offered by these methodologies cannot be neglected in order to generate data that otherwise, i.e., conventionally, would not be obtained. The academic community could also play a crucial role in solidifying the landscape around PED, given the transversal added value it provides, especially in terms of addressing methodology-based obstacles. Hence, further research on the development and validation of adequate methods should be encouraged. Ongoing work in the EU on a reflection paper in PED is expected to encourage stakeholders to collect and submit PED to medicines regulators, offering the existing pathways of scientific advice and qualification of novel methodologies as the best platform to discuss early a company’s study design development plans using PED.

Although still under consultation, the new EU’s pharmaceutical legislation proposal (106) emphasizes the need for increased participation by patients in decision-making and treatment optimization. Once adopted, it will be important to review how the new provisions will impact the evolution of the patient experience data field and how PED will fit into any future regulatory processes and outcomes. What is beyond doubt is the unanimous agreement of all stakeholders that high quality data reflecting the direct experience from patients can be meaningful for regulatory purposes and in healthcare decisions if robust methodologies are used to collect and analyze these data. To get there, we should leverage the current momentum to solidify the PED framework in order to guarantee its success in the realm of regulatory decision-making. With all the initiatives outlined and planned, and by fostering a collaborative and constructive spirit among stakeholders, challenges will turn into opportunities, leveraging PED use in regulatory decision-making.

Author contributions

DA: Formal analysis, Investigation, Writing – original draft. DU: Project administration, Validation, Writing – review & editing. RG-Q: Project administration, Validation, Writing – review & editing. AA: Investigation, Writing – review & editing. JB: Project administration, Supervision, Validation, Writing – review & editing. PV: Supervision, Validation, Writing – review & editing. NB: Conceptualization, Writing – review & editing. BS: Conceptualization, Supervision, Validation, Writing – review & editing. CT: Conceptualization, Methodology, Project administration, Supervision, Validation, Writing – review & editing.

Funding

The author(s) declare that financial support was received for the research, authorship, and/or publication of this article. The European Medicines Agency supported the open-access publication fee for this manuscript. Diogo Almeida is funded by the European Union’s Horizon Europe Research and Innovation Actions under grant no. 101095479 (More-EUROPA).

Acknowledgments

The authors would like to thank to Catarina Branco, Catarina Silva, João Moreira, and Sofia Lopes for the bibliographic research and previous input to the contents of this paper, and Francesco Pignatti and Chantal Quinten for the review and content input.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Author disclaimer

The views expressed in this article are the personal views of the author(s) and may not be understood or quoted as being made on behalf of or reflecting the position of the regulatory agency/agencies or organizations with which the author(s) is/are employed/affiliated.

Supplementary material

The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fmed.2024.1408636/full#supplementary-material

References

1. Bergerum, C, Engström, AK, Thor, J, and Wolmesjö, M. Patient involvement in quality improvement – a ‘tug of war’ or a dialogue in a learning process to improve healthcare? BMC Health Serv Res. (2020) 20:1115. doi: 10.1186/s12913-020-05970-4

Crossref Full Text | Google Scholar

2. Bertelsen, N, Dewulf, L, Ferrè, S, Vermeulen, R, Schroeder, K, Gatellier, L, et al. Patient engagement and patient experience data in regulatory review and health technology assessment: a global landscape review. Ther Innov Regul Sci. (2023) 58:63–78. doi: 10.1007/s43441-023-00573-7

Crossref Full Text | Google Scholar

3. Hansen, MB, Nørgaard, LS, and Hallgreen, CE. How and why to involve patients in drug development: perspectives from the pharmaceutical industry, regulatory authorities, and patient organizations. Ther Innov Regul Sci. (2019) 7:216847901986429. doi: 10.1177/2168479019864294

Crossref Full Text | Google Scholar

4. Willgoss, T, Escontrias, OA, Scrafton, C, Oehrlein, E, Livingstone, V, Chaplin, FC, et al. Co-creation of the global patient experience data navigator: a multi-stakeholder initiative to ensure the patient voice is represented in health decision-making. Res Involv Engagem. (2023) 9:92. doi: 10.1186/s40900-023-00503-9

Crossref Full Text | Google Scholar

5. Murphy, A, Bere, N, Vamvakas, S, and Mavris, M. The added value of patient engagement in early dialogue at EMA: scientific advice as a case study. Front Med. (2022) 8:855. doi: 10.3389/fmed.2021.811855

Crossref Full Text | Google Scholar

6. European Medicines Agency. Engagement framework: European medicines agency and patients, consumers and their organisations. (2022). Available at: https://www.ema.europa.eu/en/documents/other/engagement-framework-european-medicines-agency-and-patients-consumers-and-their-organisations_en.pdf (Accessed 22 January 2024).

Google Scholar

7. Stergiopoulos, S, Michaels, DL, Kunz, BL, and Getz, KA. Measuring the impact of patient engagement and patient centricity in clinical Research and Development. Ther Innov Regul Sci. (2019) 54:103. doi: 10.1177/2168479018817517

Crossref Full Text | Google Scholar

8. European Network for Health Technology Assessment. Guidance on patient and healthcare professional involvement. (2023). Available at: https://www.eunethta.eu/wp-content/uploads/2023/04/EUnetHTA-21-D7.2-Guidance-for-involvement-of-patient-and-clinical-expert-in-JSC-and-JCA-v1.0.pdf (Accessed 23 April 2024).

Google Scholar

9. European Medicines Agency. Terms of reference for the EMA/FDA cluster on patient engagement. (2016). Available at: https://www.ema.europa.eu/en/partners-networks/international-activities/cluster-activities (Accessed 16 February 2024).

Google Scholar

10. Haerry, D, Landgraf, C, Warner, K, Hunter, A, Klingmann, I, May, M, et al. EUPATI and patients in medicines Research and Development: guidance for patient involvement in regulatory processes. Front Med. (2018) 5:230. doi: 10.3389/fmed.2018.00230

Crossref Full Text | Google Scholar

11. Joseph, C, Kelly, Z, Andrew, B, Jose, A, Demissie, A, and Tara, S. Patient-reported outcomes measurement, implementation and interpretation. New York: Chapman and Hall/CRC (2016).

Google Scholar

12. Storf, M. The impact of FDA and EMA guidances regarding patient reported outcomes (PRO) on the drug development and approval process Wissenschaftliche Prüfungsarbeit. (2013) Available at: https://www.dgra.de/media/pdf/studium/masterthesis/master_storf_m.pdf (Accessed 16 February 2024).

Google Scholar

13. European Medicines Agency. Reflection paper on the use of patient reported outcome (PRO) measures in oncology studies. (2014). 1–9 p. Available at: https://www.ema.europa.eu/en/documents/scientific-guideline/draft-reflection-paper-use-patient-reported-outcome-pro-measures-oncology-studies_en.pdf (Accessed 16 February 2024).

Google Scholar

14. Cella, D, Hahn, E, Jensen, S, Butt, Z, Nowinski, C, Rothrock, N, et al. Patient-reported outcomes in performance measurement. Research Triangle Park (NC): RTI Press (2015).

Google Scholar

15. Australian Comission on Safety and Quality in Healthcare. (2016). Available at: https://www.safetyandquality.gov.au/sites/default/files/migrated/PROMs-Literature-Review-December-2016.pdf (Accessed May 14, 2024).

Google Scholar

16. Snyder, C, and Wu, AW. Users’ guide to integrating patient-reported outcomes in electronic health records. Baltimore, MD: Johns Hopkins University (2017).

Google Scholar

17. Nixon, A, Muehlhausen, W, and Wild, D. Patient reported outcomes: An overview. Torino: SEEd (2015).

Google Scholar

18. Giesinger, JM, Efficace, F, Aaronson, N, Calvert, M, Kyte, D, Cottone, F, et al. Past and current practice of patient-reported outcome measurement in randomized Cancer clinical trials: a systematic review. Value Health. (2021) 24:585–91. doi: 10.1016/j.jval.2020.11.004

Crossref Full Text | Google Scholar

19. Addario, B, Geissler, J, Horn, MK, Krebs, LU, Maskens, D, Oliver, K, et al. Including the patient voice in the development and implementation of patient-reported outcomes in cancer clinical trials. Health Expect. (2020) 23:41–51. doi: 10.1111/hex.12997

Crossref Full Text | Google Scholar

20. Eton, DT, Shevrin, DH, Beaumont, J, Victorson, D, and Cella, D. Constructing a conceptual framework of patient-reported outcomes for metastatic hormone-refractory prostate cancer. Value Health. (2010) 13:613–23. doi: 10.1111/j.1524-4733.2010.00702.x

Crossref Full Text | Google Scholar

21. Willke, RJ. Measuring the value of treatment to patients: patient-reported outcomes in drug development. Am Health Drug Benefits. (2008) 1:34–40.

Google Scholar

22. Frei, A, and Puhan, M. Assessment of patient-reported outcomes In: E Clini, A Holland, F Pitta, and T Troosters, editors. Textbook of pulmonary rehabilitation. Cham: Springer International Publishing (2018). 93–107.

Google Scholar

23. Rothman, ML, Beltran, P, Cappelleri, JC, Lipscomb, J, Teschendorf, B, and Sloan, JA. Patient-reported outcomes: conceptual issues. Value Health. (2007) 10:S66–75. doi: 10.1111/j.1524-4733.2007.00269.x

Crossref Full Text | Google Scholar

24. Terwee, CB, Bot, SDM, de Boer, MR, van der Windt, DAWM, Knol, DL, Dekker, J, et al. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol. (2007) 60:34–42. doi: 10.1016/j.jclinepi.2006.03.012

Crossref Full Text | Google Scholar

25. ISOQOL. Using patient-reported outcome measures for screening. (2023) Available at: https://www.isoqol.org/using-patient-reported-outcome-measures-for-screening/ (Accessed 16 February 2024).

Google Scholar

26. Weldring, T, and Smith, SMS. Article commentary: patient-reported outcomes (PROs) and patient-reported outcome measures (PROMs). Health Serv Insights. (2013) 6:HSI.S11093. doi: 10.4137/HSI.S11093

Crossref Full Text | Google Scholar

27. Higgins, J, Thomas, J, Chandler, J, Cumpston, M, Li, T, and Page, M. Cochrane handbook for systematic reviews of interventions version 6.4 (2023) Available at: www.training.cochrane.org/handbook (Accessed 22 December 2023).

Google Scholar

28. Brogan, AP, DeMuro, C, Barrett, AM, D’Alessio, D, Bal, V, and Hogue, SL. Payer perspectives on patient-reported outcomes in health care decision making: oncology examples. J Manag Care Spec Pharm. (2017) 23:125–34. doi: 10.18553/jmcp.2017.23.2.125

Crossref Full Text | Google Scholar

29. Sharma, G, Naujoks, C, Booneti, S, Lokho, J, and Kommineni, J. POSC260 evaluating the impact of patient reported outcomes data on HTA reimbursement decisions. Value Health. (2022) 25:S184–5. doi: 10.1016/j.jval.2021.11.898

Crossref Full Text | Google Scholar

30. Sedaghat, AR. Understanding the minimal clinically important difference (MCID) of patient-reported outcome measures. Otolaryngol Head Neck Surg. (2019) 161:551–60. doi: 10.1177/0194599819852604

Crossref Full Text | Google Scholar

31. Franceschini, M, Boffa, A, Pignotti, E, Andriolo, L, Zaffagnini, S, and Filardo, G. The minimal clinically important difference changes greatly based on the different calculation methods. Am J Sports Med. (2023) 51:1067–73. doi: 10.1177/03635465231152484

Crossref Full Text | Google Scholar

32. Brod, M, Tesler, LE, and Christensen, TL. Qualitative research and content validity: developing best practices based on science and experience. Qual Life Res. (2009) 18:1263–78. doi: 10.1007/s11136-009-9540-9

Crossref Full Text | Google Scholar

33. Rothrock, NE, Kaiser, KA, and Cella, D. Developing a valid patient-reported outcome measure. Clin Pharmacol Ther. (2011) 90:737–42. doi: 10.1038/clpt.2011.195

Crossref Full Text | Google Scholar

34. Huls, SPI, Whichello, CL, van Exel, J, Uyl-de Groot, CA, and de Bekker-Grob, EW. What is next for patient preferences in health technology assessment? A systematic review of the challenges. Value Health. (2019) 22:1318–28. doi: 10.1016/j.jval.2019.04.1930

Crossref Full Text | Google Scholar

35. Mühlbacher, AC, Juhnke, C, Beyer, AR, and Garner, S. Patient-focused benefit-risk analysis to inform regulatory decisions: the European Union perspective. Value Health. (2016) 19:734–40. doi: 10.1016/j.jval.2016.04.006

Crossref Full Text | Google Scholar

36. Fifer, SJ, Ho, KA, Ho, KA, Lybrand, S, Axford, LJ, and Roach, S. Alignment of preferences in the treatment of multiple myeloma—a discrete choice experiment of patient, carer, physician, and nurse preferences. BMC Cancer. (2020) 20:1–11. doi: 10.1186/S12885-020-07018-6/FIGURES/2

Crossref Full Text | Google Scholar

37. Ho, M, Saha, A, McCleary, KK, Levitan, B, Christopher, S, Zandlo, K, et al. A framework for incorporating patient preferences regarding benefits and risks into regulatory assessment of medical technologies. Value Health. (2016) 9:746–750. doi: 10.1016/j.jval.2016.02.019

Crossref Full Text | Google Scholar

38. van Overbeeke, E, Forrester, V, Simoens, S, and Huys, I. Use of patient preferences in health technology assessment: perspectives of Canadian, Belgian and German HTA Representatives. Patient. (2021) 14:119–28. doi: 10.1007/s40271-020-00449-0

Crossref Full Text | Google Scholar

39. van Overbeeke, E, Whichello, C, Janssens, R, Veldwijk, J, Cleemput, I, Simoens, S, et al. Factors and situations influencing the value of patient preference studies along the medical product lifecycle: a literature review. Drug Discov Today. (2019) 24:57–68. doi: 10.1016/j.drudis.2018.09.015

Crossref Full Text | Google Scholar

40. Bouvy, JC, Cowie, L, Lovett, R, Morrison, D, Livingstone, H, and Crabb, N. Use of patient preference studies in HTA decision making: a NICE perspective. Patient. (2020) 13:145–9. doi: 10.1007/s40271-019-00408-4

Crossref Full Text | Google Scholar

41. Holmes, EAF, Plumpton, C, Baker, GA, Jacoby, A, Ring, A, Williamson, P, et al. Patient-focused drug development methods for benefit–risk assessments: a case study using a discrete choice experiment for antiepileptic drugs. Clin Pharmacol Ther. (2019) 105:672–83. doi: 10.1002/cpt.1231

Crossref Full Text | Google Scholar

42. Wortley, S, Flitcroft, K, and Howard, K. What is the role of community preference information in health technology assessment decision making? A case study of colorectal cancer screening. Int J Technol Assess Health Care. (2015) 31:241–8. doi: 10.1017/S0266462315000367

Crossref Full Text | Google Scholar

43. Innovative Medicines Initiative. Prefer Recommendations: Why, when and how to assess and use patient preferences in medical product decision-making. (2022). Available at: https://zenodo.org/records/6592304/files/PREFER%20Recommendations%20v1_1.pdf?download=1 (Accessed 14 May 2024).

Google Scholar

44. Willems, D, Hinzpeter, E-L, Van der Zee, HH, Sayed, CJ, Ingram, JR, Beaudart, C, et al. Patient preferences in the Management of Hidradenitis Suppurativa: results of a multinational discrete choice experiment in Europe. Patient Patient Center Outcomes Res. (2023) 16:153–64. doi: 10.1007/s40271-022-00614-7

Crossref Full Text | Google Scholar

45. Myeloma, UK, National Institute for health and care excellence. Measuring Patient Preferences: An exploratory study to determine how patient preferences data could be used in health technology assessment (HTA)—Project report (2019).

Google Scholar

46. Whitty, JA, de Bekker-Grob, EW, Cook, NS, Terris-Prestholt, F, Drummond, M, Falchetto, R, et al. Patient preferences in the medical product lifecycle. Patient Patient Center Outcomes Res. (2020) 13:7–10. doi: 10.1007/s40271-019-00400-y

Crossref Full Text | Google Scholar

47. Johnson, FR, and Zhou, M. Patient preferences in regulatory benefit-risk assessments: a US perspective. Value Health. (2016) 19:741–5. doi: 10.1016/j.jval.2016.04.008

Crossref Full Text | Google Scholar

48. Chen, H, Taichman, DB, and Doyle, RL. Health-related quality of life and patient-reported outcomes in pulmonary arterial hypertension. Proc Am Thorac Soc. (2008) 5:623–30. doi: 10.1513/pats.200802-020SK

Crossref Full Text | Google Scholar

49. Kluetz, PG, O’Connor, DJ, and Soltys, K. Incorporating the patient experience into regulatory decision making in the USA, Europe, and Canada. Lancet Oncol. (2018) 19:e267–74. doi: 10.1016/S1470-2045(18)30097-4

Crossref Full Text | Google Scholar

50. Reeve, BB, Wyrwich, KW, Wu, AW, Velikova, G, Terwee, CB, Snyder, CF, et al. ISOQOL recommends minimum standards for patient-reported outcome measures used in patient-centered outcomes and comparative effectiveness research. Qual Life Res. (2013) 22:1889–905. doi: 10.1007/s11136-012-0344-y

Crossref Full Text | Google Scholar

51. Korn, EL, Freidlin, B, and Abrams, JS. Overall survival as the outcome for randomized clinical trials with effective subsequent therapies. J Clin Oncol. (2011) 29:2439–42. doi: 10.1200/JCO.2011.34.6056

Crossref Full Text | Google Scholar

52. International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use. Outcome of public consultation on ICH reflection paper on patient-focused drug development (PFDD). (2021). 1–5. Available at: https://admin.ich.org/sites/default/files/2021-06/ICH_ReflectionPaper_PFDD_OutcomeOfPublicConsutlation_2021_0527.pdf (Accessed 25 January 2024).

Google Scholar

53. Calvert, M, Kyte, D, Mercieca-Bebber, R, Slade, A, Chan, AW, and King, MT. Guidelines for inclusion of patient-reported outcomes in clinical trial protocols the spirit-pro extension. JAMA. (2018) 319:483–94. doi: 10.1001/jama.2017.21903

Crossref Full Text | Google Scholar

54. Coens, C, Pe, M, Dueck, AC, Sloan, J, Basch, E, Calvert, M, et al. International standards for the analysis of quality-of-life and patient-reported outcome endpoints in cancer randomised controlled trials: recommendations of the SISAQOL consortium. Lancet Oncol. (2020) 21:e83–96. doi: 10.1016/S1470-2045(19)30790-9

Crossref Full Text | Google Scholar

55. Patrick, D. Reporting of patient-reported outcomes in randomized trials: the CONSORT PRO extension. Value Health. (2013) 16:455–6. doi: 10.1016/j.jval.2013.04.001

Crossref Full Text | Google Scholar

56. The PROTEUS-Trials Consortium (Patient-Reported Outcome Tools: Engaging Users and Stakeholders). PROTEUS Handbook. Prepared by The University of Sydney Quality of Life Office for the PROTEUS-Trials Consortium. (2022). Available at: https://theproteusconsortium.org/wp-content/uploads/2022/04/The-PROTEUS-Trials-Consortium-Handbook-US-Version.pdf 2022. (Accessed 25 January 2024).

Google Scholar

57. Pinto, CA, Tervonen, T, Jimenez-Moreno, C, Levitan, B, Montse, G, Girman, C, et al. Current practices and challenges when submitting patient experience data for regulatory decisions by the US Food and Drug Administration: an industry survey. Patient Patient Center Outcomes Res. (2023) 17:147. doi: 10.1007/s40271-023-00653-8

Crossref Full Text | Google Scholar

58. Ciani, O, Meregaglia, M, Mario, BA, Brichetto, G, Conte, A, Gasperini, C, et al. Patient-reported outcome measures in drugs for neurological conditions approved by European medicines agency 2017-2022. Neurol Sci. (2023) 44:2933–7. doi: 10.1007/s10072-023-06825-6

Crossref Full Text | Google Scholar

59. Teixeira, MM, Borges, FC, Ferreira, PS, Rocha, J, Sepodes, B, and Torre, C. A review of patient-reported outcomes used for regulatory approval of oncology medicinal products in the European Union between 2017 and 2020. Front Med. (2022) 9:968272. doi: 10.3389/fmed.2022.968272

Crossref Full Text | Google Scholar

60. Jarosławski, S, Auquier, P, Borissov, B, Dussart, C, and Toumi, M. Patient-reported outcome claims in European and United States orphan drug approvals. J Mark Access Health Policy. (2018) 6:1542920. doi: 10.1080/20016689.2018.1542920

Crossref Full Text | Google Scholar

61. Jarosławski, S, Auquier, P, Borissov, B, Dussart, C, and Toumi, M. Low rates of patient-reported outcome claims for orphan drugs approved by the us food and drug administration. J Mark Access Health Policy. (2018) 6:1433426. doi: 10.1080/20016689.2018.1433426

Crossref Full Text | Google Scholar

62. Inotai, A, Jakab, I, Brixner, D, Campbell, JD, Hawkins, N, Kristensen, LE, et al. Proposal for capturing patient experience through extended value frameworks of health technologies. J Manag Care Spec Pharm. (2021) 27:936–47. doi: 10.18553/JMCP.2021.27.7.936

Crossref Full Text | Google Scholar

63. Pratt-Chapman, M, and Bhadelia, A. Patient-reported outcomes in health Economic decision-making: A changing landscape in oncology. In: Walter E, editor. Regulatory and economic aspects in oncology. Recent results in cancer research. Cham: Springer. 213.

Google Scholar

64. Engel, P, Almas, MF, De Bruin, ML, Starzyk, K, Blackburn, S, and Dreyer, NA. Lessons learned on the design and the conduct of post-authorization safety studies: review of 3 years of PRAC oversight. Br J Clin Pharmacol. (2017) 83:884–93. doi: 10.1111/bcp.13165

Crossref Full Text | Google Scholar

65. Sanchez-Gavilan, E, Montiel, E, Baladas, M, Lallanas, S, Aurin, E, Watson, C, et al. Added value of patient-reported outcome measures (PROMs) after an acute stroke and early predictors of 90 days PROMs. J Patient Rep Outcomes. (2022) 6:66. doi: 10.1186/s41687-022-00472-9

Crossref Full Text | Google Scholar

66. Low, CA. Harnessing consumer smartphone and wearable sensors for clinical cancer research. NPJ Digit Med. (2020) 3:140. doi: 10.1038/s41746-020-00351-x

Crossref Full Text | Google Scholar

67. Maruszczyk, K, McMullan, C, Aiyegbusi, OL, Keeley, T, Wilson, R, Collis, P, et al. Paving the way for patient centricity in real-world evidence (RWE): qualitative interviews to identify considerations for wider implementation of patient-reported outcomes in RWE generation. Heliyon. (2023) 9:157. doi: 10.1016/j.heliyon.2023.e20157

Crossref Full Text | Google Scholar

68. Janssens, R, Huys, I, Van Overbeeke, E, Whichello, C, Harding, S, Kübler, J, et al. Opportunities and challenges for the inclusion of patient preferences in the medical product life cycle: a systematic review. BMC Med Inform Decis Mak. (2019) 19:1–16. doi: 10.1186/s12911-019-0875-z

Crossref Full Text | Google Scholar

69. Soekhai, V, Whichello, C, Levitan, B, Veldwijk, J, Pinto, CA, Donkers, B, et al. Methods for exploring and eliciting patient preferences in the medical product lifecycle: a literature review. Drug Discov Today. (2019) 24:1324. doi: 10.1016/j.drudis.2019.05.001

Crossref Full Text | Google Scholar

70. Whichello, C, Levitan, B, Juhaeri, J, Patadia, V, Disantostefano, R, Pinto, CA, et al. Appraising patient preference methods for decision-making in the medical product lifecycle: an empirical comparison. BMC Med Inform Decis Mak. (2020) 20:114. doi: 10.1186/s12911-020-01142-w

Crossref Full Text | Google Scholar

71. PREFER. Patient Preferences. (2016). Available at: https://www.imi-prefer.eu/ 2016. (Accessed 15 February 2024).

Google Scholar

72. Chachoua, L, Dabbous, M, François, C, Dussart, C, Aballéa, S, and Toumi, M. Use of patient preference information in benefit–risk assessment, health technology assessment, and pricing and reimbursement decisions: a systematic literature review of attempts and initiatives. Front Med. (2020) 7:1–14. doi: 10.3389/fmed.2020.543046

Crossref Full Text | Google Scholar

73. Crossnohere, NL, Fischer, R, Crossley, E, Vroom, E, and Bridges, JFP. The evolution of patient-focused drug development and Duchenne muscular dystrophy. Expert Rev Pharmacoecon Outcomes Res. (2020) 20:57–68. doi: 10.1080/14737167.2020.1734454

Crossref Full Text | Google Scholar

74. Laurent, T, Lambrelli, D, Wakabayashi, R, Hirano, T, and Kuwatsuru, R. Strategies to address current challenges in real-world evidence generation in Japan. Drugs Real World Outcomes. (2023) 10:167–76. doi: 10.1007/s40801-023-00371-5

Crossref Full Text | Google Scholar

75. Schroeder, K, Neil, B, Scott, J, Deane, K, Dormer, L, Nair, D, et al. Building from patient experiences to deliver patient-focused healthcare Systems in Collaboration with patients: a call to action. Ther Innov Regul Sci. (2022) 56:848–58. doi: 10.1007/s43441-022-00432-x

PubMed Abstract | Crossref Full Text | Google Scholar

76. European Comission. Proposal for a regulation of the European Parliament and of the council on the European health data space. (2022). Available at: https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CELEX:52022PC0197 (Accessed 25 January 2024)

Google Scholar

77. European Comission. Proposal for a regulation of the European Parliament and of the council laying down union procedures for the authorisation and supervision of medicinal products for human use and establishing rules governing the European medicines agency, amending regulation (EC) no 1394/2007 and regulation (EU) no 536/2014 and repealing regulation (EC) no 726/2004, regulation (EC) no 141/2000 and regulation (EC) no 1901/2006. (2023). Available at: https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CELEX:52023PC0193 (Accessed 25 January 2024).

Google Scholar

78. European Comission. Directive of the European Parliament and of the council on the union code relating to medicinal products for human use, and repealing directive 2001/83/EC and directive 2009/35/EC. (2023). Available at: https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CELEX:52023PC0192 (Accessed 25 January 2024).

Google Scholar

79. Aaronson, N, Choucair, A, and Elliott, T. User’s guide to implementing patient-reported outcomes assessment in clinical practice. Int Soc Qual Life Res. (2011). 21:57. doi: 10.1007/s11136-011-0054-x

Crossref Full Text | Google Scholar

80. Regulations.gov. Methodological challenges related to patient experience data: Request for information and comments. (2023). Available at: https://www.regulations.gov/docket/FDA-2023-N-1506/comments. (Accessed 17 February 2024).

Google Scholar

81. BIO White Paper on FDA’s Statement of Patient Experience. (2021). Available at: https://www.bio.org/sites/default/files/2021-08/BIO_WHITE_PAPER_ON_FDAs_STATEMENT_OF_PATIENT_EXPERIENCE.pdf (Accessed 17 February 2024).

Google Scholar

82. Howells, Ruth. Navigating joint clinical assessment and the role of RWE in the new process. Clarivate (2024) Available at: https://clarivate.com/blog/navigating-joint-clinical-assessment-and-the-role-of-rwe-in-the-new-process/ (Accessed 23 April 2024).

Google Scholar

83. Ørjasæter Elvsaas, IK, Ettinger, S, and Willemsen, A. Patient involvement in relative effectiveness assessments in the European network for health technology assessment. Int J Technol Assess Health Care. (2021) 37:e24. doi: 10.1017/S0266462320002226

Crossref Full Text | Google Scholar

84. Lombardi, P, Marandino, L, De Luca, E, Zichi, C, Reale, ML, Pignataro, D, et al. Quality of life assessment and reporting in colorectal cancer: a systematic review of phase III trials published between 2012 and 2018. Crit Rev Oncol Hematol. (2020) 146:102877. doi: 10.1016/j.critrevonc.2020.102877

Crossref Full Text | Google Scholar

85. Tykodi, SS, Schadendorf, D, Cella, D, Reck, M, Harrington, K, Wagner, S, et al. Patient-reported outcomes with nivolumab in advanced solid cancers. Cancer Treat Rev. (2018) 70:75–87. doi: 10.1016/j.ctrv.2018.08.001

Crossref Full Text | Google Scholar

86. Rivera, SC, Kyte, DG, Aiyegbusi, OL, Slade, AL, McMullan, C, and Calvert, MJ. The impact of patient-reported outcome (PRO) data from clinical trials: a systematic review and critical analysis. Health Qual Life Outcomes. (2019) 17:1–19. doi: 10.1186/s12955-019-1220-z

Crossref Full Text | Google Scholar

87. US Food and Drug Administration. Patient-focused drug development guidance public workshop: Methods to identify what is important to patients select, develop or modify fit-for-purpose clinical outcomes assessments. (2018). Available at: https://www.fda.gov/drugs/news-events-human-drugs/patient-focused-drug-development-guidance-methods-identify-what-important-patients-and-select (Accessed 23 April 2024).

Google Scholar

88. Kalluri, M, Luppi, F, Vancheri, A, Vancheri, C, Balestro, E, Varone, F, et al. Patient-reported outcomes and patient-reported outcome measures in interstitial lung disease: where to go from here? Eur Respir Rev. (2021) 30:2021. doi: 10.1183/16000617.0026-2021

Crossref Full Text | Google Scholar

89. Aiyegbusi, OL, Cruz Rivera, S, Roydhouse, J, Kamudoni, P, Alder, Y, Anderson, N, et al. Recommendations to address respondent burden associated with patient-reported outcome assessment. Nat Med. (2024) 30:650–9. doi: 10.1038/s41591-024-02827-9

Crossref Full Text | Google Scholar

90. International Council on Harmonisation (ICH). ICH reflection paper: Proposed ICH guideline work to advance patient focused drug development. (2021). Available at: https://admin.ich.org/sites/default/files/2021-06/ICH_ReflectionPaper_PFDD_FinalRevisedPostConsultation_2021_0602.pdf (Accessed 5 January 2024).

Google Scholar

91. Reaney, M, Cline, J, Wilson, JC, and Posey, M. Generating relevant information from patients in the technology-enhanced era of patient-focused drug development: opportunities and challenges. Patient. (2021) 14:11–6. doi: 10.1007/s40271-020-00455-2

Crossref Full Text | Google Scholar

92. Hassanaly, P, and Dufour, JC. Analysis of the regulatory, legal, and medical conditions for the prescription of Mobile health applications in the United States, the European Union, and France. Med Devices Evid Res. (2021) 14:389–409. doi: 10.2147/MDER.S328996

Crossref Full Text | Google Scholar

93. European Medicines Agency. HMA-EMA joint big data taskforce-summary report summary report. (2019). Available at: https://www.ema.europa.eu/en/documents/minutes/hma-ema-joint-task-force-big-data-summary-report_en.pdf (Accessed 25 January 2024).

Google Scholar

94. Denecke, K, Bamidis, P, Bond, C, Gabarron, E, Househ, M, Lau, AYS, et al. Ethical issues of social media usage in healthcare. Yearb Med Inform. (2015) 24:137–47. doi: 10.15265/IY-2015-001

Crossref Full Text | Google Scholar

95. Callard, F, and Perego, E. How and why patients made long Covid. Soc Sci Med. (2021) 268:113426. doi: 10.1016/j.socscimed.2020.113426

Crossref Full Text | Google Scholar

96. Sarker, A, Ginn, R, Nikfarjam, A, O’Connor, K, Smith, K, Jayaraman, S, et al. Utilizing social media data for pharmacovigilance: A review. J Biomed Inform. (2015) 54:202–12. doi: 10.1016/j.jbi.2015.02.004

Crossref Full Text | Google Scholar

97. Lavertu, A, Vora, B, Giacomini, KM, Altman, R, and Rensi, S. A new era in pharmacovigilance: toward real-world data and digital monitoring. Clin Pharmacol Ther. (2021) 109:1197–202. doi: 10.1002/cpt.2172

Crossref Full Text | Google Scholar

98. Sloane, R, Osanlou, O, Lewis, D, Bollegala, D, Maskell, S, and Pirmohamed, M. Social media and pharmacovigilance: a review of the opportunities and challenges. Br J Clin Pharmacol. (2015) 80:910–20. doi: 10.1111/bcp.12717

Crossref Full Text | Google Scholar

99. Tricco, AC, Zarin, W, Lillie, E, Jeblee, S, Warren, R, Khan, PA, et al. Utility of social media and crowd-intelligence data for pharmacovigilance: a scoping review. BMC Med Inform Decis Mak. (2018) 18:1–14. doi: 10.1186/S12911-018-0621-Y/TABLES/4

Crossref Full Text | Google Scholar

100. Convertino, I, Ferraro, S, Blandizzi, C, and Tuccori, M. The usefulness of listening social media for pharmacovigilance purposes: a systematic review. Expert Opin Drug Saf. (2018) 17:1081–93. doi: 10.1080/14740338.2018.1531847

Crossref Full Text | Google Scholar

101. van Stekelenborg, J, Ellenius, J, Maskell, S, Bergvall, T, Caster, O, Dasgupta, N, et al. Recommendations for the use of social Media in Pharmacovigilance: lessons from IMI WEB-RADR. Drug Saf. (2019) 42:1393–407. doi: 10.1007/s40264-019-00858-7

Crossref Full Text | Google Scholar

102. Rees, S, Mian, S, and Grabowski, N. Using social media in safety signal management: is it reliable? Ther Adv Drug Saf. (2018) 9:591–9. doi: 10.1177/2042098618789596

Crossref Full Text | Google Scholar

103. Price, J. What can big data offer the pharmacovigilance of orphan drugs? Clin Ther. (2016) 38:2533–45. doi: 10.1016/j.clinthera.2016.11.009

Crossref Full Text | Google Scholar

104. Davies, W. Insights into rare diseases from social media surveys. Orphanet J Rare Dis. (2016) 11:151. doi: 10.1186/s13023-016-0532-x

Crossref Full Text | Google Scholar

105. McDonald, L, Malcolm, B, Ramagopalan, S, and Syrad, H. Real-world data and the patient perspective: the PROmise of social media? BMC Med. (2019) 17:1–5. doi: 10.1186/S12916-018-1247-8/PEER-REVIEW

Crossref Full Text | Google Scholar

106. European Comission. Reform of the EU pharmaceutical legislation. (2023). Available at: https://health.ec.europa.eu/medicinal-products/pharmaceutical-strategy-europe/reform-eu-pharmaceutical-legislation_en (Accessed 25 March 2024).

Google Scholar