Gov➔Value: How to combine reported quality experiences and patient-reported outcome measures. First results on vulvar cancer patients in an Italian Research Hospital

Introduction: Vulvar cancer (VC) accounts for <1% of cancers affecting the female gender. Clinical Pathways (CP) and Clinical Outcomes Monitoring are useful for providing high-quality care to these patients. However, it is essential to integrate them with the patient's perspective according to Value-Based Healthcare paradigms. Patient-reported outcome measures (PROMs) and patient-reported experience measures (PREMs) are tools for assessing outcomes and experiences with health care from the patient's perspective. The aim of this paper is to collect and synthesize PROMs and main stakeholders' experience on the VC CP, according to a value-based approach.

Materials and methods: To select the most appropriate instrument, a review was conducted on the main databases and official websites of specific institutions and organizations. In the second phase, a 2-round Delphi survey was conducted to assess the Reported Experience Measures (REMs) tool. Questions were evaluated according to four criteria (general relevance, evidence-based, measurability, actionability) and included if strong agreement was reached. A Principal Component Analysis (PCA) was executed. Cronbach's alpha and McDonald's omega were computed. Fisher's exact test and Wilcoxon rank sum test were used to compare ratings between groups. Descriptive statistics were performed for both PROMs and REMs instruments.

Results: For PROMs assessment, EORTC QLQ-C30 questionnaire was selected and administered to 28 patients. Global Health Status/Quality of Life and Functional Scales Scores were high or very high, while symptoms scale reported low or medium scores. The final REMs consists of 22 questions for professionals and 16 for patients and caregivers. It was administered to 22 patients, 11 caregivers, 5 physicians, 2 nurses and 1 clinical senior manager. PCA identified 4 components. Scale reliability was acceptable (α = 0.75 95% CI: 0.61–0.85; ω = 0.69; 95% CI: 0.54, 0.82). A statistically significant difference between the patient/caregiver group and the professionals was found for items 8 (follow-up), 10 (perceived quality), 12 (safety), and 16 (climate) (p = 0.02; p = 0.03; p < 0.001; p < 0.001, respectively).

Discussion: PROMs could provide new ways of intercepting patients' needs and feedback, thus acting on them. The proposed REMs tool would allow to detect information not available elsewhere, which, through Audit and feedback strategies, could lead to enhancement of healthcare experience, according to a value-based approach.

1. Introduction

Over the past few decades, neoplastic diseases have increased in incidence and prevalence, becoming one of the leading causes of death. However, some cancers are not very widespread. Vulvar cancer (VC) is one example. They account for about 5 percent of all cancers affecting the female genital tract (1). The annual incidence is 1–2/100,000 women. It is most frequently diagnosed in women aged 65–74 and accounts for <1% of cancers affecting the female gender (2, 3). Nevertheless, as the average life expectancy increases, cases of VC are likely to increase. To best deal with it, a drastic change in the organization of care pathways is required.

Patients with VC require multidisciplinary evaluation to design the best personalized clinical approach (4–6). This leads diverse health professionals to work and share their expertise and knowledge to create evidence-based decision-making according to the perspective of personalized medicine.

For these reasons, most hospitals have begun looking at new organizational paradigms to reshape hospital care delivery processes, moving away from the lines of traditional academic specialties and focusing primarily on patient needs (7). This is particularly true in the oncologic field where, in a Shared decision-making (SDM) context, patients and families are becoming more active, informed, and aware of the risks and benefits of various treatment options (8).

It involves the application of methods and tools to combine physician and manager perspectives to leverage the centrality of the person cared for, shifting from a “disease-centered” to a “person-centered” approach (9).

In 2022, Fondazione Policlinico Universitario A. Gemelli-IRCCS (FPG-IRCCS), a large tertiary care center located in Rome (Italy) and one of the largest Italian Oncological Centers, set up and implemented a VC critical pathway (CP). It encompasses the optimal care processes to improve quality and ensure that every care episode follows the most updated scientific evidence (10). In addition, consistent with the best updated scientific evidence, a multidisciplinary VC team was established in our institution. Structured around a core team and supplemented by a group of support specialists and a care manager, it is responsible for treatment strategies and individualized management. In their multidisciplinary tumor board meeting, about 260 cases are discussed annually (5). Finally, during the CP design phase, key performance indicators (KPIs) were selected and calculated to monitor the overall performance of the CP and ensure continuous improvement in the quality of care through audit & feedback (A&F) strategies.

However, an understanding of the experience and perspective of patients, caregivers and healthcare professionals involved in CP is missing in this context.

According to a value-based approach, this paper aims to collect and synthesize patient-reported outcomes and main stakeholders' experience on the CP for VC patients.

Specifically, the aim is to:

1) Collect and summarize the PROMs (Patient Reported Outcome Measures) of patients within the CP;

2) Measure, through validated questionnaires, the experience of various CP's stakeholders (patients, caregivers, physicians, nurses and managers), compare and assess the concordance of their perceptions regarding the issues of safety and quality of care.

2. Materials and methods

To develop the Gov → Value tool, a three-phase methodology was carried out: extensive literature review to identify a PROMs questionnaire for patients with VC; extensive literature review and Delphi validation of a Reported Experience Measures (REMs) questionnaire; pilot study.

2.1. Literature review concerning VC PROMs

In order to select a tool for measuring PROMs specifically in the case of patients with VC, validated in the Italian language, an extensive search of the main evidence-based and already validated questionnaires in the literature was carried out.

The main databases (PubMed, Scopus, Web of Science) and official websites of institutions and organizations with specific expertise in this field (AIOM, CIPOMO, EORTC, ICHOM, Istat) were consulted.

2.2. Literature review and Delphi validation of a REMs questionnaire

In the second stage, we scoured the scientific literature to identify relevant items for our REMs questionnaire. The main databases (PubMed, Scopus, Web of Science) were consulted. Based on the results of the review, we elaborated a set of items designed to assess experience as reported by various stakeholders (patients, caregivers, physicians, nurses, senior managers). A two-round Delphi survey was conducted to validate the final version of the REMs questionnaire. During the first round, a panel of experts was asked to express, for each question, their degree of agreement on a Likert scale of 1 to 3 (with 1 corresponding to the minimum-“Not Relevant” and 3 corresponding to the maximum-“Relevant”), based on the following four criteria:

- General Relevance.

- Support from scientific evidence.

- Measurability.

- Actionability.

The average of the four scores provided corresponded to the “overall” score, which was used to exclude items from the final set of indicators. In both rounds, the indicators with the lowest scores were excluded. The second round was also used to validate the final set of indicators.

The panel selection criteria for this study included at least one of the following: (i) publications on the topic of Clinical Governance; (ii) experience on the topic of Clinical Governance; (iii) knowledge and expertise of the phenomenon of Clinical Governance; and (iv) willingness and motivation to participate. All identified experts were contacted individually and asked for their willingness to participate in the Delphi process. Eleven experts, including healthcare managers, economists, and physicians, patient organization's representatives, were recruited. The team of experts was invited to complete the Delphi survey by email, through a Google Modules questionnaire. A cover letter explained the purpose, relevance, and usefulness of this survey. The answers were collected immediately and anonymously. At the end of the study and after the two Delphi Rounds, the instrument in its integrity (including all the questionnaires) was validated.

This methodology replicated one already applied by the team to another clinical setting (11).

The first Round of consultation started on the sixth of April 2022 and ended on the twentieth of April. The authors considered the following levels of agreement:

• “Strong Agreement”: “Overall” score of the item is equal to or more than 2.5 out of 3.0.

• “Agreement for Exclusion”: “Overall” score for each item is equal to or more than 2.0 out of 3.0.

In the presence of a “strong agreement for inclusion”, the indicator was included in the Second Round of the Survey. Items falling in the category “agreement for exclusion” were eliminated. The Second Round was structured as the First Round. For the final list of questions, the following levels of agreement were established:

• “Strong agreement for inclusion in the final list”: “Overall” score equal to or more than 2.5 out of 3.0.

• “Agreement for exclusion from final list”: mean of “Overall” score for each item <2.0 out of 3.0.

The Second Round of consultation started on the twenty-fifth of April 2022 and ended on the fifth of May 2022.

2.3. Pilot study

In the third stage, the final version of the Gov➔Value Tool (REMs questionnaire plus PROMs questionnaire) was tested in a sample of VC patients and their care team.

The pilot study was monocentric, taking place in the FPG-IRCCS VC outpatient setting, between May and June 2022.

Patients with the following ICD-9-CM codes: 184.4 (vulvar malignant tumor, unspecified), 196.5 (secondary and unspecified malignant tumors of the lymph nodes of the inguinal region and lower limb), 196.6 (secondary and unspecified malignant tumors of intrapelvic lymph nodes), 196.2 (secondary and unspecified malignant tumors of intra-abdominal lymph nodes), caregivers, nurses, physicians, and clinical managers were recruited from the VC CP of the FPG-IRCCS. Data regarding age, co-morbidities and demographics were collected face-to-face during clinical examinations performed by the care manager.

Given the pilot nature of our study, no standard sample sizing is necessary. Rules of practice for in-house pilot studies indicate 20 patients as the minimum sample (12).

2.3.1. Inclusion criteria

All patients that are more than 18 years old and with malignant VC diagnosed on the VC CP are enrolled in the study and invited to reply to the questionnaire.

The eligible caregivers, instead, must be more than 18 years old and assist patients diagnosed with malignant VC included in the CP.

The eligible healthcare professionals (physician/nurse/manager) work in the Vulvar Pathology outpatient setting.

All the people included in the study must give the informed consent to the processing of data for research purposes.

2.3.2. Exclusion criteria

Patients who are not able to understand the questions of the questionnaire (e.g., cognitive capabilities alteration, non-comprehension of Italian language) were excluded.

The caregivers excluded are those who are not able to understand the questions of the questionnaire.

Other patients and caregivers excluded are those with no informed consent to the processing of data for research purposes, as well.

2.3.3. Survey administration

REMs and PROMs questionnaires were collected in a self-completed manner between May and June 2022. For patients unable to complete the questionnaire on their own, who expressed their willingness to participate in the study, completion support was offered, to ensure equity of participation.

2.3.4. Processing of personal data

An informative note about the study was provided to respondents. Informed consent for the processing of data was required, complying with General Data Protection Regulation, 2018. In the module for informed consent collection, the freedom of withdrawing the consent in any moment is specified.

2.3.5. Institutional review board approval

The research protocol was approved by the Ethics Committee of the Fondazione Policlinico Gemelli, Rome.

2.3.6. Data analysis

2.3.6.1. PROMs questionnaire

A descriptive analysis of the collected data was performed. The scores obtained from the PROMs questionnaires, linearly transformed on a scale from 0 to 100, were summarized and reprocessed using appropriate statistical methodologies, as indicated in EORTC QLQ-C30 manual (13). Mean, standard deviation (SD), median and interquartile range (IQR) were used for quantitative variables.

2.3.6.2. REMs questionnaire

With regard to the REMs questionnaire, for categorical variables, absolute and relative frequency were provided, whereas mean, SD, median and IQR were used for quantitative variables. The Shapiro-Wilk test was applied to test the Gaussian distribution of the quantitative variables. A scale score was calculated by adding up individual items from the REMs questionnaire. A Principal Component Analysis (PCA) was run to collapse the questionnaire variables into a smaller number of principal components accounting for a large share of variance. The PCA was based on polychoric correlations, given the ordinal nature of data. A varimax rotation was applied, thus obtaining rotated components (RCs). Bartlett's test of sphericity and the Kaiser-Meyer-Olkin test were performed to check PCA's assumptions. Cronbach's alpha (>0.7 considered satisfactory) and McDonald's omega (>0.7 considered sufficient) were computed to assess questionnaire reliability. Missing data were handled through pairwise deletion, whenever possible. In order to compare ratings between groups, Fisher's exact test was used, based on indications on individual items' comparisons (14), while Wilcoxon rank sum test was used for the total scale. Effect size was reported, as well, by means of Cramer's V and r, reported with a 95% confidence interval. Effect size interpretation was based on commonly followed recommendations in published literature (15, 16).

Values of p < 0.05 were considered statistically significant. All statistical analyses were carried out in R software, version 4.2.0 (CRAN^®, R Core 2022) within the RStudio platform, version 2022.02.3 + 492 (2009–2022 RStudio, PBC).

3. Results

3.1. Literature review

3.1.1. PROMs

As a result of our literature review, EORTC QLQ-C30 questionnaire was selected (Appendix 1). It is already validated and consists of 30 areas comprising different scales, implemented to measure physical, psychological and social functions of cancer patients. The first 28 questions have four different answers: 1 = No; 2 = A little; 3 = A lot; 4 = Very Much.

The last two, instead, have a Likert scale from 1 to 7 as possible answers. The questionnaire is available at https://qol.eortc.org/questionnaire/eortc-qlq-c30/. Linear transformation was executed according to the dedicated manual (13) (Table 1).

TABLE 1

Table 1. Results of EORTC QLQ-C30 questionnaire.

3.1.2. REMs

As a result of the literature review, a questionnaire was defined as follows:

o Doctor: 22 questions (14 Quality; 8 Safety);

o Nurse: 22 questions (14 Quality; 8 Safety);

o Senior Manager: 22 questions (14 Quality; 8 Safety);

o Patient: 16 Questions (10 Quality; 6 Safety);

o Care manager: 16 questions (10 quality; 6 safety);

Some questions are identical, while others were reformulated considering the user of the questionnaire. 18 questions were ranked through a 4-point Likert score:1 = no; 2 = a little; 3= rather much; 4 = very much. 4 questions (items 6, 9, 10, 20) were ranked on a dichotomic basis: yes; no.

3.2. Delphi validation of REMs questionnaire

3.2.1. First round of consultation

Ten (91%) out of eleven experts recruited responded to the First Round.

The analytical results are reported by questions and by evaluation criterion (Annex 1).

All the sections (Physicians, Nurses, Senior Manager, Patients, Caregivers) were validated entirely in First Round and they were all included in the Second Round.

However, the “Nurses' quality Section” received the lowest scores on some items.

3.2.2. Second round of consultation

Participation in the consultation was completed by 10 out of 10 participants (100%) and considered valid.

As well as in the first round of consultation, all the questionnaire sections received a positive evaluation from the experts and were validated entirely.

The average of each dimension of each perspective is 2.7 in the Second Round.

The resulting Five-sections questionnaire is composed of 72 questions, divided as follows (Annex 1):

1. Physicians: 22 questions (14 Quality Section; 8 Safety Section);

2. Nurses: 22 questions (14 Quality Section; 8 Safety Section);

3. Senior manager: 22 questions (14 Quality Section; 8 Safety Section);

4. Patients: 16 questions (10 Quality Section; 6 Safety Section);

5. Caregivers: 16 questions (10 Quality Section; 6 Safety Section).

3.3. Pilot study

3.3.1. PROMs

Twenty-eight women with VC were identified during the study period, and response rate was 85.71% (N = 24). The median age was 64 (Interquartile Range = 22).

The majority of respondents were in post-treatment phase (follow-up), counting for the 95.83%.

The most frequent functional difficulties encountered by women mainly concerned their roles: they felt limited in their job (39.12%, N = 9 reported “a lot”/“very much” as level of limitation) and in their typical free-time activities (41.67%, N = 10 reported level “a lot”/“very much”). As concerns symptoms, difficulties in sleeping (39.12%%, N = 9 answered “a lot”/“very much”) and weakness were common (39.12%, N = 9 answered “a lot”/“very much”) and they frequently felt tired (41.67%, N = 10 answered “a lot”/“very much”).

On the status of their global health (QoL), on a range of 1/7, the women in a level ≥5 for “Health in the last 7 days” were 11 (45.83%). For an evaluation, instead, of their “Quality of life in the last 7 days” 10 women (41.67%) reported levels ≥5.

Table 1 reports results of PROMs administration.

3.3.2. REMs' results

Twenty-two patients were included in the pilot study. All of the available caregivers (n = 11) accepted to respond. Physicians (n = 5), nurses (n = 2) and a clinical senior manager (n = 1), with experience in the treatment of women with VC, were also interviewed.

With regard to REMs questionnaire, mean, standard deviation (SD), median and interquartile range (IQR) of each Likert item's responses, classified into 2 categories, namely patients/caregivers and professionals, are displayed in Table 2. As to dichotomic items, Table 3 reports absolute and relative frequency.

TABLE 2

Table 2. Mean, SD, median and IQR of responses to each Likert item, classified by respondent group (patient/caregiver or professional).

TABLE 3

Table 3. Percentage of responses to dichotomic items.

A radar plot summarizing mean responses provided by professionals and patients/caregivers is shown in Figure 1.

FIGURE 1

Figure 1. Radar plot showing mean scores for item topic, as assigned by professionals and patients/caregivers.

A barplot showing ratings for each item, classified by group of raters (patients/caregivers and professionals) is depicted in Figure 2.

FIGURE 2

Figure 2. Barplot showing distribution of Likert items' responses among various scores, classified by group of raters. Scores 1, 2, 3, and 4, respectively correspond to responses: no, a little, rather much, very much.

3.3.2.1. PCA

The first 4 components accounted for 85% of variance in the dataset (Table 4). Explanation of therapy risks, visit duration, information provided, safety and shared decision-making mainly contribute to RC 1. RC 2 mainly consists of the following variables: punctuality, patient-physician climate and accessibility in terms of ease of visit reservation. Clear description of adverse reactions and polytherapy risks largely contribute to RC 3. Provision of informative material, indications on follow-up steps and facility accessibility have the highest loadings on RC 4. RC 1 can be interpreted as general care quality features of the visit itself. RC 3 is mainly concerned with the experience related to medical therapy prescription. RC 2 consists of aspects related to the visit experience, such as punctuality and physician-patient climate. Lastly, RC 4 mostly has to do with follow-up relevant activities.

TABLE 4

Table 4. Item loadings on RCs and cumulative percentage of variance accounted for by RCs.

3.3.2.2. Reliability analysis

When considering 13 4-point Likert items administered to all respondents (patients, caregivers and professionals), Cronbach's alpha was satisfactory (α = 0.75; 95% CI: 0.61–0.85). McDonald's Omega was similarly sufficient (ω = 0.69; 95% CI: 0.54, 0.82).

Cronbach's alpha if an item is deleted ranged from 0.48 to 0.77. Omega if an item is deleted ranged from 0.54 to 0.75. Specifically, item 7 and item 11 would increase omega notably.

When stratified by RC, Cronbach's alpha was 0.87 (95% CI: 0.80–0.92) in RC1, 0.91 (95% CI: 0.83–0.96) in RC3, 0.6 (95% CI: 0.38–0.81) in RC2 and 0.49 (95% CI: 0.14–0.71) in RC4.

When applied to the 13 4-point Likert items administered to patients and caregivers, Cronbach's alpha was satisfactory (α = 0.73; 95% CI: 0.56–0.85), while McDonald's Omega was nearly sufficient (ω = 0.64; 95% CI: 0.43, 0.80). If an item is dropped, alpha ranged from 0.37 to 0.76, while Omega was in the range from 0.32 to 0.72. Still, items 7 and 11 were negatively affecting omega the most.

Considering the 18 4-point Likert items for the professionals' group, Cronbach's alpha was fairly high (α = 0.84; 95% CI: 0.62–0.96) and McDonald Omega was sufficient (ω = 0.82; 95% CI: 0.58−0.96). Alpha ranged from 0.82 to 0.86 and omega from 0.32 to 0.87, in the case of removing an item. Deletion of items 2 and 21 would increase omega the most.

3.3.2.3. Comparison between scores from patients/caregivers and professionals

Results from comparison between professionals' and patients/caregivers' responses to each item are reported in Table 5.

TABLE 5

Table 5. Results of comparison between responses to each item from the professionals' and patients/caregivers' groups.

A statistically significant difference between the patient/caregiver group and the professionals was found for items 8, 10, 12 and 16 (p = 0.02; p = 0.03; p < 0.001; p < 0.001, respectively). Effect size was medium for item 8 and 10 and large for items 12 and 16.

Items 1, 2, 3, 4, 5, 6, 7, 9, 11, 13, 14, 15 and the total scale show no statistically significant difference, when comparing responses from patients/caregivers with those from professionals.

A comparison between scores from each individual group of stakeholders (physicians, nurses, manager, patients, caregivers) led to statistically significant differences for items 8 (p = 0.002), 12 (p = 0.003) and 16 (p < 0.001).

4. Discussion

The main goal of our research was to collect and synthesize patients' and key healthcare stakeholders' experience on a CP dedicated to women with VC, as only a few tools have been developed in oncological care to explore these items (17).

In several countries, patient-reported outcome measures (PROMs) and patient-reported experience measures (PREMs) are widely used in research and performance evaluation (18). This item is crucial as healthcare organizations could use PREMs and PROMs to: (a) improve the quality of care on an individual level, via a patient-centered approach through the implementation of personalized care, notably due to consideration of patients' concerns and needs; (b) improve diagnosis of diseases and potentially reduce their severity, via more regular or systematic assessment of the effectiveness of care and monitoring of disease progression; (c) increase patient information, communication and shared medical decision-making (19), thus paving the way for precision and personalized medicine (20, 21).

The present work aims to move beyond the mere patient experience (PREM) assessment by including all the key CP stakeholders' viewpoint. By doing so, PREMs were integrated with doctors, nurses and clinical leaders and caregivers' perspective, thus becoming “Reported Experience Measures” (REMs).

As no validated questionnaires on REMs were available, a specific one was designed, constructed, validated through the Delphi methodology and administered.

The proposed tool aims to analyze the same phenomenon (comparable dimensions of quality and safety and all referred to the same care event) from different perspectives.

Such an approach would recall the application of lean tools to improve quality and safety of care in testing or diagnostics (22) by comparing quality and safety experiences based on five different perspectives. It would be a development of the “Go to the Gemba”, the Japanese word meaning “go to the place” through the processes of care (23), so as to implement quality improvement initiatives where concordance among the different perspectives lacks.

To this end, it becomes essential to find organizational solutions that combine a high degree of specialization, technical and scientific advances, multidisciplinary and multiprofessional coordination, and patient participation (24).

To pursue such an approach, an internal organization consistency is required: (a) first, a standardized CP has to be developed by a dedicated multidisciplinary team to provide “case by case”, high-quality diagnosis, and evidence-based decision-making in the context of personalized medicine; (b) secondly, a set of KPIs has to be defined by the hospital monitoring system. This, combined with an A&F system, creates a virtuous environment with the primary goal of improving health provider performance and healthcare outcomes.

To our knowledge, this is among the first few studies assessing the feasibility of collecting data on different stakeholders' experience at a given “point of care” event, within the patient CP, so as to assess the different perspectives on crucial dimensions of quality and safety and elicit information to improve oncological care.

Such requirements matched with the management of vulvar cancer, where, unlike other oncology conditions, such as Breast Cancer (25, 26), the patient's experience has been less studied.

Alimena et al. (27), who examined PROMs in a typical clinic population of vulvar cancer patients, administered the following validated tools: the European Organization for the Research and Treatment of Cancer Quality of life Questionnaire (EORTC QLQ-C30), the Patient Reported Outcome Measurement Information System (PROMIS) Emotional and Instrumental Support Questionnaires, and the Functional Assessment of Cancer Therapy-Vulvar (FACT-V) questionnaire.

In our study, as the other PRO questionnaires were not validated in Italian, we were able to use only the EORTC QLQ-C30 questionnaire, and as to REMs questionnaire, the one we defined was tested.

Our study has a number of strengths: it makes use of an innovative tool, in line with recent literature developments; it deploys a rigorous methodological approach throughout all of the study phases; it originally provides various perspectives on health care experience.

Our study, however, does not come without its limitations. First, by default, all studies on vulvar cancer patients have relatively small populations. In addition, as regards the PROMs questionnaire administration, differently from a previous study (27), in our study it was not possible to use disease-specific questionnaires because the Italian version was not available (e.g., PROMIS and FACT-V). As a consequence, we chose to use the European Organization for the Research and Treatment of Cancer Quality of life Questionnaire (EORTC QLQ-C30). It is a frequently used patient-reported outcome instrument to assess health-related quality of life of patients with cancer. However, it is not designed to stratify by comorbidities.

Further limitations include the fact that, in questionnaire administration, a paper version was preferred. Paper questionnaires involve several time-consuming and costly steps. However, this choice is mainly related to the mean age of the patients that distinguish our cohort. Indeed, older populations typically suffer from low digital literacy, even though the proportion of elderly using digital technology has increased exponentially (28, 29).

The Delphi methodology was used to validate the REMs questionnaire. This methodology is used to combine expert knowledge and opinion to arrive at an informed group consensus on a complex problem. However, some methodological limitations should be taken into account (such as starting with provided material and questions may not be representative, the process tends to eliminate extreme positions and force a middle-of-the-road consensus and is also vulnerable to high dropout rates due to the large time commitment required) (30).

Both PCA and reliability analysis would benefit from a larger sample (31, 32). Additionally, we used Cronbach's alpha, which is a widely used coefficient for reliability assessment, despite a wide range of limits, largely based on hard-to-meet assumptions. We still reported it in light of its popularity, but decided to complement it with McDonald's Omega, which is recognized as more accurate (33). When comparing ratings, we considered patients and caregivers as one group and professionals as another, in order to avoid a fragmentation of our small sample, while still maintaining a focus on the main perspectives at stake. Furthermore, ideally, we should collect data on each individual care episode from the various stakeholders involved. This would allow a more comprehensive and specific interpretation of data.

In addition, we are aware that our search focused on patient experiences in the outpatient care setting within the hospital, and not on the whole pathway, as needed in cancer care.

In terms of future perspectives, longitudinally-administered PROMs questionnaires allow clinicians to keep track of clinical outcomes as reported by patients, as a valuable addition to clinical performance monitoring systems. This tool could provide new ways of intercepting patients' needs and feedback, thus acting on them. With regard to REMs, A&F practice based on results from the proposed questionnaire might play a key role in improving professional practice. Thus, our tool would allow to detect information not available elsewhere, with potential enhancement of healthcare experience for both patients and professionals, according to a value-based approach.

Our team emphasize how Patient-reported outcome measures (PROMs) and patient-reported experience measures (PREMs) are complementary and necessary tools to improve quality and safety.

Depending on the scores reported from individual perspectives, ad hoc improvement actions, such as A&F interventions, will be taken if discrepancies or critical issues emerge. If designed optimally and used in the right context, A&F can play an important role in improving professional practice.

This is especially true in the field of oncology, where it increasingly plays a leading role through the creation of SDM processes (34).

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving human participants were reviewed and approved by Ethics Committee of Fondazione Policlinico Universitario A. Gemelli IRCCS. The patients/participants provided their written informed consent to participate in this study.

Author contributions

EM and CA: conceptualization, methodology, data processing, writing of original draft, and project management. AP: methodology, data processing, formal analysis, and writing of original draft. AB: conceptualization, review, editing of manuscript, and supervision. SF: data collection, review, editing of manuscript, and supervision. LG: recruitment of participants and data collection. GS, GG, and LT: review and supervision. All authors approved the contributions, read, and approved the final manuscript.

Acknowledgments

The authors would like to acknowledge Drs. Giuseppe Greco and Federica Rizzo, who contributed to data collection and methodology, as well as Edoardo Valentini, who collected data.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpubh.2022.1014651/full#supplementary-material

References

1. Alkatout I, Günther V, Schubert M, Weigel M, Garbrecht N, Jonat W, et al. Vulvar cancer: epidemiology, clinical presentation, and management options. Int J Womens Health. (2015) 7:305. doi: 10.2147/IJWH.S68979

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Mancini S, Bucchi L, Baldacchini F, Giuliani O, Ravaioli A, Vattiato R, et al. Incidence trends of vulvar squamous cell carcinoma in Italy from 1990 to 2015. Gynecol Oncol. (2020) 157:656–63. doi: 10.1016/j.ygyno.2020.03.013

PubMed Abstract | CrossRef Full Text | Google Scholar

3. U.S. Department of Health and Human Services, Centers for Disease Control and Prevention and National Cancer Institute. U.S. Cancer Statistics Data Visualizations Tool, based on 2021 submission data (1999–2019). (2022). Available online at: https://gis.cdc.gov/grasp/USCS/DataViz.html (accessed on August 8, 2022)

Google Scholar

4. Tagliaferri L, Garganese G, D'Aviero A, Lancellotta V, Fragomeni SM, Fionda B, et al. Multidisciplinary personalized approach in the management of vulvar cancer – the Vul. Can Team experience. Int J Gynecol Cancer. (2020) 30:932. doi: 10.1136/ijgc-2020-001465

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Garganese G, Tagliaferri L, Fragomeni SM, Lancellotta V, Colloca G, Corrado G, et al. Personalizing vulvar cancer workflow in COVID-19 era: a proposal from Vul. Can MDT J Cancer Res Clin Oncol. (2020) 146:2535–45. doi: 10.1007/s00432-020-03312-9

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Tagliaferri L, Lancellotta V, Casà C, Fragomeni SM, Ferioli M, Gentileschi S, et al. The radiotherapy role in the multidisciplinary management of locally advanced vulvar cancer: a multidisciplinary vulcan team review. Cancers. (2021) 13:5747. doi: 10.3390/cancers13225747

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Porter ME, Teisberg EO. Redefining Health Care: Creating Value-based Competition on Results. Boston, MA: Harvard Business School Press. (2006).

PubMed Abstract | Google Scholar

8. Carman KL, Dardess P, Maurer M, Sofaer S, Adams K, Bechtel C, et al. Patient and family engagement: a framework for understanding the elements and developing interventions and policies. Health Aff (Millwood). (2013) 32:223–31. doi: 10.1377/hlthaff.2012.1133

PubMed Abstract | CrossRef Full Text | Google Scholar

9. European Commission. Defining value in ‘value-based healthcare’. Opinion by the Expert Panel on effective ways of investing in Health (EXPH). 2019.

Google Scholar

10. Lawal AK, Rotter T, Kinsman L, Machotta A, Ronellenfitsch U, Scott SD, et al. What is a clinical pathway? Refinement of an operational definition to identify clinical pathway studies for a Cochrane systematic review. BMC Med. (2016) 14:35. doi: 10.1186/s12916-016-0580-z

PubMed Abstract | CrossRef Full Text | Google Scholar

11. de Belvis AG, Fratini A, Angioletti C, Morsella A, Ruggeri R, Pepe G, et al. How to define a quadruple aim framework to assess value in critical pathway of the patients with Clostridioides difficile infection. Eur Rev Med Pharmacol Sci. (2021) 25:4597–610. doi: 10.26355/eurrev_202107_26252

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Birkett MA, Day SJ. Internal pilot studies for estimating sample size. Stat Med. (1994) 13:2455–63. doi: 10.1002/sim.4780132309

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Fayers PM, Aaronson NK, Bjordal K, Groenvold M, Curran D, Bottomley A, et al. The EORTC QLQ-C30 Scoring Manual. 3rd Ed. Brussels: European Organisation for Research and Treatment of Cancer (2001).

Google Scholar

14. Clason D, Dormody TJ. Analyzing Data Measured by Individual Likert-Type Items. J Agric Educ. (1994) 35:31–5. doi: 10.5032/jae.1994.04031

CrossRef Full Text | Google Scholar

15. Cohen J. Statistical Power Analysis for the Behavioral Sciences. 2nd ed. New York: Routledge (1988). 567 p.

PubMed Abstract | Google Scholar

16. Cohen J, A. power primer. Psychol Bull. (1992) 112:155–9. doi: 10.1037/0033-2909.112.1.155

CrossRef Full Text | Google Scholar

17. Rudolph C, Petersen GS, Pritzkuleit R, Storm H, Katalinic A. The acceptance and applicability of a patient-reported experience measurement tool in oncological care: a descriptive feasibility study in northern Germany. BMC Health Serv Res. (2019) 19:786. doi: 10.1186/s12913-019-4646-4

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Kingsley C, Patel S. Patient-reported outcome measures and patient-reported experience measures. BJA Educ. (2017) 17:137–44. doi: 10.1093/bjaed/mkw060

CrossRef Full Text | Google Scholar

19. de Santé HA. Quality of care as perceived by patients—PROMs and PREMs indicators Panorama of experiences in other countries and main lessons learned. Saint-Denis, France: Haute Autorité de Santé (2021).

Google Scholar

20. McKee M, Merkur S, Edwards N, Nolte E. The Changing Role of the Hospital in European Health Systems. Brussels, Belgium : European Observatory on Health Systems and Policies (2020). 306 p.

Google Scholar

21. Krzyszczyk P, Acevedo A, Davidoff EJ, Timmins LM, Marrero-Berrios I, Patel M, et al. The growing role of precision and personalized medicine for cancer treatment. TECHNOLOGY. (2018) 06:79–100. doi: 10.1142/S2339547818300020

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Venner AA, Beach LA, Shea JL, Knauer MJ, Huang Y, Fung AWS, et al. Quality assurance practices for point of care testing programs: Recommendations by the Canadian society of clinical chemists point of care testing interest group. Clin Biochem. (2021) 88:11–7. doi: 10.1016/j.clinbiochem.2020.11.008

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Aij KH, Teunissen M. Lean leadership attributes: a systematic review of the literature. J Health Organ Manag. (2017) 31:713–29. doi: 10.1108/JHOM-12-2016-0245

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Maes-Carballo M, Martín-Díaz M, Mignini L, Khan KS, Trigueros R, Bueno-Cavanillas A. Evaluation of the use of shared decision making in breast cancer: international survey. Int J Environ Res Public Health. (2021) 18:2128. doi: 10.3390/ijerph18042128

PubMed Abstract | CrossRef Full Text | Google Scholar

25. van Egdom LSE, Lagendijk M, van der Kemp MH, van Dam JH, Mureau MAM, Hazelzet JA, et al. Implementation of value based breast cancer care. Eur J Surg Oncol. (2019) 45:1163–70. doi: 10.1016/j.ejso.2019.01.007

PubMed Abstract | CrossRef Full Text | Google Scholar

26. van Egdom LSE, Oemrawsingh A, Verweij LM, Lingsma HF, Koppert LB, Verhoef C, et al. Implementing patient-reported outcome measures in clinical breast cancer care: a systematic review. Value Health. (2019) 22:1197–226. doi: 10.1016/j.jval.2019.04.1927

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Alimena S, Sullivan MW, Philp L, Dorney K, Hubbell H., del Carmen MG, et al. Patient reported outcome measures among patients with vulvar cancer at various stages of treatment, recurrence, and survivorship. Gynecol Oncol. (2021) 160:252–9. doi: 10.1016/j.ygyno.2020.10.022

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Monica A, Andrew P. Tech Adoption Climbs Among Older Adults. Washington, D.C.: Pew Research Center (2017). Available online at: https://www.pewresearch.org/internet/2017/05/17/tech-adoption-climbs-among-older-adults/

Google Scholar

29. Oh SS, Kim KA, Kim M, Oh J, Chu SH, Choi J. Measurement of digital literacy among older adults: systematic review. J Med Internet Res. (2021) 23:e26145. doi: 10.2196/26145

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Donohoe HM, Needham RD. Moving best practice forward: Delphi characteristics, advantages, potential problems, and solutions. Int J Tour Res. (2009) 11:415–37. doi: 10.1002/jtr.709

CrossRef Full Text | Google Scholar

31. Bujang MA, Omar ED, Baharum NA. A review on sample size determination for cronbach's alpha test: A simple guide for researchers. Malays J Med Sci MJMS. (2018) 25:85–99.

PubMed Abstract | Google Scholar

32. Osborne J, Costello AB. Sample size and subject to item ratio in principal components analysis. Pr Assess Res Eval. (2004) 9:11. doi: 10.7275/ktzq-jq66

CrossRef Full Text | Google Scholar

33. McNeish D. Thanks coefficient alpha, we'll take it from here. Psychol Methods. (2018) 23:412–33. doi: 10.1037/met0000144

PubMed Abstract | CrossRef Full Text | Google Scholar

34. de Belvis AG, Pellegrino R, Castagna C, Morsella A, Pastorino R, Boccia S. Success factors and barriers in combining personalized medicine and patient centered care in breast cancer. Results from a systematic review and proposal of conceptual framework. J Pers Med. (2021) 11:654. doi: 10.3390/jpm11070654

PubMed Abstract | CrossRef Full Text | Google Scholar