mRNA-COVID19 Vaccination Can Be Considered Safe and Tolerable for Frail Patients

Background: Frail patients are considered at relevant risk of complications due to coronavirus disease 2019 (COVID-19) infection and, for this reason, are prioritized candidates for vaccination. As these patients were originally not included in the registration trials, fear related to vaccine adverse events and disease worsening was one of the reasons for vaccine hesitancy. Herein, we report the safety profile of the prospective, multicenter, national VAX4FRAIL study (NCT04848493) to evaluate vaccines in a large trans-disease cohort of patients with solid or hematological malignancies and neurological and rheumatological diseases.

Methods: Between March 3 and September 2, 2021, 566 patients were evaluable for safety endpoint: 105 received the mRNA-1273 vaccine and 461 the BNT162b2 vaccine. Frail patients were defined per protocol as patients under treatment with hematological malignancies (n = 131), solid tumors (n = 191), immune-rheumatological diseases (n = 86), and neurological diseases (n = 158), including multiple sclerosis and generalized myasthenia. The impact of the vaccination on the health status of patients was assessed through a questionnaire focused on the first week after each vaccine dose.

Results: The most frequently reported moderate–severe adverse events were pain at the injection site (60.3% after the first dose, 55.4% after the second), fatigue (30.1%–41.7%), bone pain (27.4%–27.2%), and headache (11.8%–18.9%). Risk factors associated with the occurrence of severe symptoms after vaccine administration were identified through a multivariate logistic regression analysis: age was associated with severe fever presentation (younger patients vs. middle-aged vs. older ones), female individuals presented a higher probability of severe pain at the injection site, fatigue, headache, and bone pain; and the mRNA-1237 vaccine was associated with a higher probability of severe pain at the injection site and fever. After the first dose, patients presenting a severe symptom were at a relevant risk of recurrence of the same severe symptom after the second one. Overall, 11 patients (1.9%) after the first dose and 7 (1.2%) after the second one required postponement or suspension of the disease-specific treatment. Finally, two fatal events occurred among our 566 patients. These two events were considered unrelated to the vaccine.

Conclusions: Our study reports that mRNA-COVID-19 vaccination is safe also in frail patients; as expected, side effects were manageable and had a minimum impact on patient care path.

Introduction

The currently authorized messenger RNA (mRNA)-corona virus disease 2019 (COVID-19) vaccines—m-RNA-1237 Moderna (1) and BNT-162b2 Pfizer BioNTech (2)—have been evaluated in clinical trials that excluded, in accordance with the current regulations, immunocompromised subjects and restricted participation to healthy volunteers. Frail patients were not considered in such pivotal studies despite being the subjects at greatest risk of COVID-19 complications and with the potential greatest advantage.

Patients diagnosed with solid or hematological malignancy or under immunosuppressive treatment due to rheumatological or neurological diseases are considered at high risk of COVID-19 complications and are categorized as frail (3–7). The intended acceptance of the COVID-19 vaccine in frail patients confirms the positive attitudes towards vaccination, but questions arise around the safety of these vaccines in the setting of immune alterations engendered by their diseases and/or therapies (8–14). One of the most typical reasons for vaccine hesitancy has been fear related to vaccine side effects and underlying disease worsening. Of note, vaccine hesitancy has been reported also in the general population (vaccine acceptance up to 86.1% among healthcare students or 77.6% in the general population), as documented by large multinational studies (15–17), with concern about safety among the most given reasons to refuse vaccine.

Over the past 8 months, several groups have tried to answer questions about the efficacy and safety of COVID vaccination in different cohorts of frail subjects. No safety concerns emerged, confirming the profile described in the series of healthy subjects (14, 18–46) and pointing out an acceptable safety profile. VAX4FRAIL (47) study aimed at assessing immune responses to vaccination in a large trans-disease cohort of patients with hematological malignancies, solid tumors, and neurological and rheumatological diseases. The study’s main objective was to assess prospectively the immunological response to the COVID-19 vaccination in these specific subgroups, characterizing the kinetics of the immune response to the vaccination and its persistence over time. Longitudinal, prospective evaluation of the safety profile was part of this trans-disease study. Herein, we report safety profile results as outlined in VAX4FRAIL trial.

Methods

Safety analysis was performed among patients enrolled in the VAX4FRAIL trial between t0 and t2 according to the protocol, t0 being the “time point 0” at first dose of vaccine and t2 the “time point 2” of the blood sampling 2–4 weeks after the second dose of vaccine (47). This is a national, multicentric observational prospective study conducted in Italy with the primary aim of assessing the immune response of COVID-19 vaccination in frail, immunocompromised patients.

Patients were considered for the current safety analysis if they met the general inclusion criteria: being ≥18 years of age, having received COVID-19 vaccination with mRNA vaccines (BNT-162b2 Pfizer-BioNTech or m-RNA-1237 Moderna vaccine), and having completed the health status assessment questionnaire after one of the two vaccine doses. Frail patients under evaluation were diagnosed with hematological malignancies, solid tumors, immune-rheumatological disease, and neurological disease. Detailed inclusion and exclusion criteria were previously reported (47), and disease stratification was conducted according to diagnosis and treatment of the primary disease.

The impact of the vaccination on the health status of patients was assessed through a questionnaire focused on the first week after each vaccine dose. We developed the questionnaire by listing the symptoms with the highest probability to be experienced by the patients. A 5-item Likert scale was derived from the Palliative Care Outcome Scale (POS). The POS is a questionnaire developed to assess the quality of life of patients with advanced disease. It was previously forward–backward translated from English, and it is validated both in English and in Italian (48).

The questionnaire was administered to the patients after the first vaccine dose (3–4 weeks after the first dose) and at the subsequent timing according to VAX4FRAIL protocol (2–4 weeks after the second dose). The questionnaire assessed how much the patient was troubled by eight symptoms (pain or swelling at the injection site, fatigue, headache, bone pain, fever, enlarged lymph nodes, skin rash, insomnia, diarrhea, and nausea or vomiting) (see Supplementary Appendix 1) The answers were graded according to a five-level scale (not at all, slightly, moderately, severely, and overwhelmingly). The patient was also asked to report and grade other symptoms possibly occurring after each vaccine dose and if he/she had to postpone or suspend therapies due to symptoms related to vaccination.

Statistical Methods

The prevalence of symptoms after each dose of the vaccine was reported by aggregating the answers in three levels: score 1, no symptom (scored not at all); score 2, moderate symptom (scored slightly or moderately); and score 3, severe symptom (scored severely or overwhelmingly).

We estimated the probability of the occurrence of a severe symptom versus no symptom or moderate symptom after the second dose of the vaccine according to the occurrence of a severe symptom after the first dose using positive and negative predictive values (PPV and NPV, respectively). We also estimated the agreement between severe vs. non-severe symptom between the two doses using Cohen’s kappa statistics that adjust for the probability of the agreement occurring by chance. According to Landis and Koch, values <0 indicates no agreement, values between 0 and 0.20 as slight, 0.21–0.40 as fair, 0.41–0.60 as moderate, 0.61–0.80 as substantial, and 0.81–1 as almost perfect agreement (49).

The probability of occurrence of a severe symptom (scored 1) versus no symptom or moderate symptom (scored 0) after one of the two doses of the vaccine according to age, sex, main diagnosis, and type of vaccine was estimated in a multivariate logistic regression analysis adjusting for all variables.

The probability of occurrence of a severe symptom (scored 1) versus no symptom or moderate symptom (scored 0) after one of the two doses of the vaccine was estimated within each group of diseases by different subgroups and in a multivariate logistic regression analysis adjusting for age, sex, main diagnosis, and type of vaccine.

Results

Patients

Between March 3 and September 2, 2021, 566 patients were enrolled in the VAX4FRAIL study and were eligible for this analysis. Among 566 patients who received the first dose of the mRNA vaccine, 105 received mRNA-1273, and 461 received BNT162b2. Among the 566 patients who received the first dose, 556 received the second one (Figure 1).

FIGURE 1

Figure 1 Patients’ disposition.

Overall, 488 patients after the first dose and 489 patients after the second dose were evaluable for safety evaluation having completed clinical evaluation and questionnaires. Sixty-eight patients after the first dose and 57 after the second dose did not fill the questionnaires. Consent withdrawn occurred in 8 cases after the first dose and 10 cases after the second one.

Patient characteristics are reported in Table 1. Most patients (N = 299, 52.8%) were aged between 51 and 70 years, and 317 (56.0%) were female. The cohort of patients with solid tumors comprised 191 patients (33.7%); the one with neurological diseases included multiple sclerosis and generalized myasthenia comprised 158 patients (27.9%). Overall, 131 patients were diagnosed with hematological malignancies (23.1%) and 86 with immunorheumathological diseases (15.2%).

TABLE 1

Table 1 Patient’s characteristics.

Vaccine-Related Adverse Events

Overall, 438 (77.3%) patients reported any grade adverse events after the first dose (62, 11.0% reported as severe) and 373 (65.9%) after the second dose (87, 15.4% reported as severe). A detailed analysis of adverse events and severity is reported in Table 2.

TABLE 2

Table 2 Occurrence of symptoms/adverse events over the week after vaccine administration.

After the first dose (both mRNA-1273 and BNT162b2), 53.9% of the patients reported moderate pain at the injection site and 6.4% reported severe pain: this was the most frequently reported complaint after vaccine administration. Interestingly, after the second dose—t2—50.3% and 5.1% reported this adverse event as moderate and severe, respectively.

At t1, 25.6% of patients reported fatigue as a moderate event and 4.5% as a severe one. We observed a slight increase at t2, with 35% of patients reporting fatigue as a moderate event and 6.7% as a severe one.

Bone pain and headache were reported as moderate by 13.7% and 9.8% of patients at t1, while 2.7% and 2.0%, respectively, reported the symptoms as severe. After the second dose, bone pain was reported as moderate by 23.3% and severe by 3.9% of patients, while headache was reported as moderate by 16% and severe by 2.9% of patients.

Only 5.9% of patients reported fever as relevant symptoms at t1 (moderate 4.7%–severe 1.2%), while at t2, the percentage of patients reporting this symptom slightly increased: 15.9% overall, with 10.0% moderate and 5.9% severe. Only a minority of patients—<2%—reported nausea, diarrhea, insomnia, skin rash, or enlarged lymph nodes as severe manifestation both at t1 and t2.

Among unsolicited adverse events, after the first dose, 5 patients reported chills (5/5 moderate), 3 itching (2 moderate, 1 severe), 2 gastrointestinal pain (2/2 moderate), 5 myalgia (5/5 moderate), 6 dizziness (3 moderate, 3 severe), 1 drowsiness of moderate degree, and 2 sweats (1 patient moderate and 1 severe). Similarly, after the second dose, 12 patients reported chills (11 moderate, 1 severe), 7 itching (5 moderate, 2 severe), 1 a moderate gastrointestinal pain, 8 myalgia (6 moderate, 2 severe), 9 dizziness (8 moderate, 1 severe), 1 drowsiness of moderate degree, and 2 sweats (1 patient moderate and 1 severe). After the second dose, one patient reported confusion and one dysesthesia. Among the 11 patients with hematological malignancies who received an allogeneic stem cell transplantation, graft versus host disease (GvHD) occurrence or reactivation was not reported after the first and/or the second dose.

The agreement between the occurrence of a severe symptom after the doses (versus no symptom or a moderate symptom) was rather poor with a kappa statistic ranging between 0.15 and 0.28. Of note, the absence of severe symptoms after the first dose strongly predicts the high probability of the absence of the same severe symptoms after the second dose (Table 2). Patients reporting severe fever after the first dose were more prone to develop the same severe symptom after the second one.

Multivariate Model for Occurrence of Severe Symptoms

A multivariate logistic regression—adjusted by age, sex, diagnosis, and vaccine—was performed to identify risk factors associated with the occurrence of severe symptoms after vaccine administration (Table 3).

TABLE 3

Table 3 Probability of occurrence of a severe symptom after one of the two doses of vaccine estimated through a multivariate logistic regression adjusted by age, sex, diagnosis, and vaccine.

Age was associated with severe fever presentation: younger patients (age 18–50 years) had a higher probability of occurrence of severe fever than middle-aged patients [age 51–70; odds ratio, 0.3; 95% confidence interval (CI), 0.1–0.7] or older ones (>/= 71; odds ratio, 0.1; 95% CI, 0.1–0.7)—p = 0.003.

Female patients presented higher probability of severe pain at the injection site (odds ratio, 2.9; 95% CI, 1.4–6.2; p = 0.005), severe fatigue (odds ratio, 2.3; 95% CI, 1.2–4.5; p = 0.016), severe headache (odds ratio, 7.6; 95%, 1.7–33.6, p = 0.007), and severe bone pain (odds ratio, 2.7; 95% CI, 1.1–6.5; p = 0.028).

Diagnosis (hematological diseases, solid tumors, neurological diseases, and rheumatological diseases) has no impact on the occurrence of severe fatigue, headache, bone pain, or fever but was associated with severe pain at the injection site: patients with a diagnosis of solid tumors reported severe pain less frequently than patients with hematological diseases (odds ratio, 0.3; 95% CI, 0.1–1.0), patients with a diagnosis of neurological or rheumatological disease have a higher probability of occurrence of severe pain at the injection site in comparison with patients with hematological disease (odds ratio for neurological diseases, 4; 95% CI, 1.4–11; the odds ratio for rheumatological diseases, 2.8; 95% CI, 1.1–7.5)—p < 0.001.

The probability of occurrence of a severe symptom versus no symptom or moderate symptom after one of the two doses of the vaccine was estimated within each group of diseases by different subgroups and in an adjusted multivariate logistic regression analysis (Supplementary Table 1). Notwithstanding that the only statistically significant differences were reported in the solid tumor cohort for the symptoms severe fatigue and headache, the small number of patients in each subgroup and the lack of correction for multiplicity make it necessary to consider these results with caution.

Finally, the conditional contribution made by the vaccine (mRNA-1237 versus BNT162b2) clearly showed that mRNA-1237 was associated with a higher probability of severe pain at the injection site (odds ratio, 0.3; 95% CI, 0.1–0.6; p = 0.001) and a higher probability of severe fever (odds ratio, 0.3; 95% CI, 0.1–0.7; p = 0.004).

Impact on Disease and Therapy

Overall, 11 patients (1.9%) after the first dose and 7 (1.2%) after the second dose required postponement or suspension of the disease-specific treatment due to the occurrence of symptoms associated with the vaccine. Among the 11 patients who required delays after the first dose, 7 were diagnosed with hematological disease and 4 with a solid tumor. Among the seven patients who required delays after the second dose, three were diagnosed with the hematological disease and four with a solid tumor.

No patients from the neurological diseases’ cohort or the rheumatological diseases’ cohort required treatment delay.

Fatal events, long-term sequelae, or hospitalization related to the vaccine was not registered. Two patients died from disease progression; the events were clearly reported by the investigator as not related to the vaccine administration.

Discussion

Questionnaires based upon open and/or closed questions administered through the web-based app, phone interview, self-reporting forms, and face-to-face visits were all utilized to collect adverse events after COVID-19 vaccinations in the effort to clarify the safety of vaccines better. Absence of a univocal method to grade and collect this information makes it almost impossible to directly compare the different experiences, but, of note, all the studies—both retrospective and prospective—are concordant in underlying the general perception of the absence of safety issue among distinct cohorts of frail patients (Table 4).

TABLE 4

Table 4 Literature review.

Providing information on the safety of COVID-19 vaccination in high-risk frail populations is a duty of the international scientific community. To date, several groups are trying to answer two fundamental questions: (i) if and how frail patients develop an effective response to the vaccine and (ii) if the safety profile is confirmed valid in this category both with reference to toxicity and with reference to the maintenance of control of the underlying disease.

Knowledge of COVID-19 vaccine-related side effects have a crucial role in the public decision regarding vaccination. Studies designed to monitor the safety and effectiveness of COVID-19 vaccines globally are ongoing, focusing on short-term side effects, the booster doses’ side effects, and the long-term safety and effectiveness (50).

Safety profile is confirmed acceptable in all the experience reported so far in frail patients.

More systemic and local side effects were observed after the second dose of vaccine than after the first dose. The most common local side effects were pain at the injection site, local rash, and local swelling, whereas the most common systemic side effects were muscle pain, fatigue, headache, fever, chills, gastrointestinal complications, and flu-like symptoms (18, 19, 22–24, 26–28, 30, 31, 36–38). Overall, the incidence of severe symptoms was low—<2.5% (19, 39, 41)—with authors reporting no severe grade 3–4 adverse events (20–22, 24, 33, 38, 42–46). In a national prospective cohort study evaluating outcomes in patients with hematological malignancies in Lithuania (25), the authors reported that adverse events were more common after the second dose, with fatigue being the most prevalent symptom (13%). No grade 4 adverse events were reported.

Among patients with multiple sclerosis, Lotan and colleagues (26) outlined how 15% of the participants reported new or worsening neurological symptoms following the vaccination, the most frequent being sensory disturbances (58.3%). Most symptoms occurred within the first 24 h after vaccination and resolved within 3 days. A total of 28 participants (77.8%) did not require any medication to treat their symptoms. Of note, no increased risk of relapse activity was noted across patients with multiple sclerosis (26, 27) or autoimmune inflammatory rheumatic diseases (29–31): Boekel and coworkers clearly showed how multivariable logistic regression analyses showed similar odds for any adverse event, systemic adverse events, or moderate or severe adverse events between patients and controls, which was consistent when patients with rheumatoid arthritis or multiple sclerosis were compared with healthy controls (36).

In our experience, cohort composition was clearly defined at enrolment of patients, identifying a subset of high-risk subjects for severe COVID-19. The monitoring strategy was defined per protocol and homogeneous across the four categories of diseases (hematological disease, solid tumors, neurological conditions, and rheumatological diseases).

Overall, our study confirms the positive safety profile reported by other authors in both prospective and retrospective analyses (Table 4). Incidence of severe adverse events after vaccine administration was generally low and <3%. Most frequent complaints were pain at the injection site (severe, 6.3%) and fatigue (severe, 4.5%) after the first dose; pain at the injection site (severe, 5.1%), fatigue (severe, 6.8%), bone pain (severe, 3.9%) and fever (severe, 6%) after the second dose.

Patients experiencing a severe symptom after the first dose were more likely to report the same after the second one. Similarly, the absence of a severe symptom after the first dose was significantly associated with an absence of the same symptoms also after the second dose. This observation should be taken into consideration during the counseling of patients: patients should be reassured about the possibility of adopting a preventive strategy to reduce the burden of symptoms and on the not-unexpected onset of the symptom itself.

Moreover, the definition of predisposing factors to an increased possibility of presenting specific adverse events (e.g., younger patients are more likely to experience fever) can help both in the implementation of preventive measures (e.g., pre-emptive administration of painkiller drugs or antifever), in the optimization of counseling (e.g., if a patient experienced adverse events after the first shot, he should be advised that the recurrence of the same after the second is very likely and that this is not unexpected; moreover, strategies to counterbalance this inconvenience can be applied), and in the planning of therapies and/or vaccination itself.

According to the design of the study, the patient population was selected for the intrinsic immunocompromised condition—related both to the underling disease and the given treatments. Very few patients in each subgroup experienced severe symptoms, and this makes difficult to draw any additional conclusion (e.g., correlation between disease–treatment–disease status and occurrence of severe adverse events). Pursuing the monitoring of frail patients will further enlighten possible influencing factors.

Of note, <2% of patients required a delay or a suspension of the ongoing or planned treatment of the underlying diseases due to vaccination. This pointed out the positive safety profile of the vaccine strategy; furthermore, this aspect should be discussed with patients to confirm the absence of impact on the whole therapy program. Whatever the underlying disease (a hematological cancer, a solid tumor, a rheumatological disease, or a neurological disease), the mRNA-COVID-19 vaccine should be considered a crucial step to allow a safe program of treatment more than a possible obstacle or danger to pursue the control of the disease, being the safety profile reassuring. Fatal events or hospitalization related to the vaccine were not registered in our cohort of high-risk frail patients.

Despite the absence of safety concerns emerging from our study, we suggest that it is mandatory to maintain constant and careful surveillance concerning post-vaccination adverse events. This can only contribute to the reliability that the vaccination strategy pursues and represents a cornerstone of general medical practice.

As outlined by several authors (3–9), one of the most common reasons for vaccine hesitancy was fear related to vaccine side effects and disease worsening: we can confirm that vaccine side effects were both manageable and in line with previously reported in the general population, without the occurrence of unexpected events and no concern related to worsening of the underlying disease or the need to delay treatment.

To further strengthen the safety of the vaccine in frail patients, we underline that our safety results are in line with what has been observed in the pivotal studies conducted in immunocompetent subjects. Safety assessments included monitoring of solicited local and systemic adverse events for 7 days after each injection for both mRNA-1273 (1) and BNT162b2 vaccine (2).

In the mRNA-1273 (1) group, the adverse events at the injection site (pain was the most common event) occurred after the first dose in 84.2% of subjects, and the severity was mainly grade 1 or 2. After the first dose, 54.9% of subjects reported systemic adverse events; after the second dose, this percentage increased up to 79.4%, and, similarly, the severity increased. Authors reported that both solicited injection site and systemic adverse events were more common among younger participants (18–<65 years of age) than among older participants (≥65 years of age).

Similarly, in the BNT162b2 (2) group, local reactogenicity (pain at the injection site) was reported frequently (85% of subjects <55 years and 71% of >55 years after the first dose, 78% and 66% respectively, after the second). The proportion of participants reporting local reactions did not increase after the second dose, and no participant reported a grade 4 local reaction. In general, local reactions were mostly mild to moderate in severity. Systemic events were reported more often by younger vaccine recipients (16–55 years of age) than by older vaccine recipients (more than 55 years of age) and more often after the second dose than the first dose. The most reported systemic events were fatigue and headache.

Conclusion

Frail patients who are candidates to mRNA-COVID-19 vaccination should be reassured about the safety profile of vaccine strategy: adverse events were in line with the report from the healthy cohort of subjects and national observatories, no evidence of worsening of the underlying disease was reported, and no concern on the adherence to the treatment program of the disease itself emerged from our prospective multicenter national study. Pursuing careful and timely monitoring of expected and unexpected adverse events represents a gold standard of modern medicine and can only support evidence-based medicine.

Data Availability Statement

All data produced in the present study are available upon reasonable request to the authors.

Ethics Statement

The protocol has been approved by national competent authorities (AIFA) and the ethics committee of the National Institute for Infectious Diseases Lazzaro Spallanzani (IRCCS). The patients/participants provided their written informed consent to participate in this study.

The VAX4FRAIL Study Group:

Principal Investigators (alphabetical order): Giovanni Apolone (Fondazione IRCCS Istituto Nazionale dei Tumori di Milano); Alberto Mantovani (IRCCS Istituto Clinico Humanitas, Milano).

Scientific Coordinator: Massimo Costantini (Fondazione IRCCS Istituto Nazionale dei Tumori di Milano).

Steering Committee (alphabetical order): Chiara Agrati (IRCCS Istituto per le Malattie Infettive Lazzaro Spallanzani, Roma); Giovanni Apolone (Fondazione IRCCS Istituto Nazionale dei Tumori di Milano); Fabio Ciceri (IRCCS Ospedale San Raffaele, Milano); Gennaro Ciliberto (IRCCS Istituto Nazionale Tumori Regina Elena, Roma); Massimo Costantini (Fondazione IRCCS Istituto Nazionale dei Tumori di Milano); Franco Locatelli (Università La Sapienza, Roma); Alberto Mantovani (IRCCS Istituto Clinico Humanitas, Milano); Fausto Baldanti (Fondazione IRCCS Policlinico San Matteo di Pavia); Aldo Morrone (Istituto Dermatologico San Gallicano IRCCS, Roma); Carlo Salvarani (Azienda USL-IRCCS Reggio Emilia); Nicola Silvestris (IRCCS Istituto Tumori “Giovanni Paolo II,” Bari); Fabrizio Tagliavini (Fondazione IRCCS Istituto Neurologico Carlo Besta, Milano); Antonio Uccelli (Ospedale Policlinico San Martino IRCCS, Genova); Pier Luigi Zinzani (IRCCS Azienda Ospedaliero-Universitaria di Bologna).

Disease Groups

1. Hematological Malignancies Referent: Paolo Corradini (Fondazione IRCCS Istituto Nazionale dei Tumori, Milano);

2. Solid Tumors Referent: Gennaro Ciliberto (IRCCS Istituto Nazionale Tumori Regina Elena, Roma);

3. Immunorheumatological Diseases Referent: Carlo Salvarani (Azienda USL IRCCS Reggio Emilia);

4. Neurological Diseases: Referent: Antonio Uccelli (Ospedale Policlinico San Martino IRCCS, Genova); Renato Mantegazza (Fondazione I.R.C.C.S Istituto Neurologico Carlo Besta (INCB), Milano).

Immunological Group

Referents: Chiara Agrati (IRCCS Istituto per le Malattie Infettive Lazzaro Spallanzani, Roma); Maria Rescigno (IRCCS Istituto Clinico Humanitas, Milano); Daniela Fenoglio (Ospedale Policlinico San Martino IRCCS, Genova);

Participants: Roberta Mortarini (Fondazione IRCCS Istituto Nazionale dei Tumori di Milano); Cristina Tresoldi (IRCCS Ospedale San Raffaele, Milano); Laura Conti (IRCCS Istituto Nazionale.

Tumori Regina Elena, Roma); Stefania Croci (Azienda USL IRCCS Reggio Emilia); Fausto Baldanti (Fondazione IRCCS Policlinico San Matteo di Pavia); Vito Garrisi (IRCCS Istituto Tumori “Giovanni Paolo II,” Bari); Fulvio Baggi (Fondazione IRCCS Istituto Neurologico Carlo Besta, Milano); Tiziana Lazzarotto (IRCCS Azienda Ospedaliero-Universitaria di Bologna); Fulvia Pimpinelli (Istituto Dermatologico San Gallicano IRCCS, Roma).

INMI Centralized Laboratory (INMI Lazzaro Spallanzani–IRCCS, Roma) (alphabetical order)

Enrico Girardi (Scientific Director), Aurora Bettini; Veronica Bordoni; Concetta Castilletti; Eleonora Cimini; Rita Casetti; Francesca Colavita; Flavia Cristofanelli; Massimo Francalancia; Simona Gili; Giulia Gramigna; Germana Grassi; Daniele Lapa; Sara Leone; Davide Mariotti; Giulia Matusali; Silvia Meschi; Stefania Notari; Enzo Puro; Marika Rubino; Alessandra Sacchi; Eleonora Tartaglia

Clinical Task Force

Paolo Corradini, Silvia Damian, Filippo de Braud (Fondazione IRCCS Istituto Nazionale dei Tumori di Milano); Maria Teresa Lupo-Stanghellini, Lorenzo Dagna, Francesca Ogliari, Massimo Filippi (IRCCS Ospedale San Raffaele: Milano); Giulia Piaggio (IRCCS Istituto Nazionale Tumori Regina Elena, Roma); Elena Azzolini, Chiara Pozzi, Luca Germagnoli, Carlo Selmi, Maria De Santis, Carmelo Carlo-Stella, Alexia Bertuzzi, Francesca Motta, Angela Ceribelli (IRCCS Istituto Clinico Humanitas, Milano);

Fausto Baldanti, Sara Monti (Fondazione IRCCS Policlinico San Matteo di Pavia); Aldo Morrone (Istituto Dermatologico San Gallicano IRCCS, Roma); Maria Grazia Catanoso, Carmine Pinto, Francesco Merli, Franco Valzania, Monica Guberti (Azienda USL-IRCCS Reggio Emilia);

Rosa Divella, Antonio Tufaro, Vito Garrisi, Sabina Delcuratolo, Mariana Miano (IRCCS Istituto Tumori “Giovanni Paolo II,” Bari);

Carlo Antozzi, Silvia Bonanno Rita Frangiamore, Lorenzo Maggi (Fondazione IRCCS Istituto Neurologico Carlo Besta, Milano); Antonio Uccelli, Paolo Pronzato, Matilde Inglese, Carlo Genova, Caterina Lapucci, Alice Laroni, Ilaria Poirè (Ospedale Policlinico San Martino IRCCS, Genova);

Marco Fusconi, Vittorio Stefoni, Maria Abbondanza Pantaleo (IRCCS Azienda Ospedaliero-Universitaria di Bologna).

Statistical Committee

Diana Giannarelli (IRCCS Istituto Nazionale Tumori Regina Elena, Roma).

e-CRF and Monitoring Referent:

Valentina Sinno, Serena Di Cosimo (Fondazione IRCCS Istituto Nazionale dei Tumori di Milano).

Project Managers of the Study:

Referents: Elena Turola, Azienda USL-IRCCS di Reggio Emilia.

Participants: Iolanda Pulice, Roberta Mennitto Fondazione IRCCS Istituto Nazionale dei Tumori, Milano); Stefania Trinca (IRCCS Ospedale San Raffaele, Milano); Giulia Piaggio (IRCCS Istituto Nazionale Tumori Regina Elena, Roma); Chiara Pozzi (IRCCS Istituto Clinico Humanitas, Milano);

Irene Cassaniti (Fondazione IRCCS Policlinico San Matteo, Pavia); Alessandro Barberini (Istituto Dermatologico San Gallicano IRCCS, Roma); Arianna Belvedere (Azienda USL-IRCCS Reggio Emilia);

Sabina Del Curatolo (IRCCS Istituto Tumori “Giovanni Paolo II,” Bari); Rinaldi Elena, Federica Bortone (Fondazione IRCCS Istituto Neurologico Carlo Besta, Milano); Maria Giovanna Dal Bello (Ospedale Policlinico San Martino IRCCS, Genova); Silvia Corazza (IRCCS Azienda Ospedaliero-Universitaria, Bologna).

Author Contributions

MC, VS, and ET had full access to all the data in the study and took responsibility for the integrity of the data and the accuracy of the data analysis. Concept and design: MTLS, SDC, MC, SM, NS, GA, and AMa. Acquisition, analysis, or interpretation of data: MTLS, SDC, MC, SM, and NS. Drafting of the manuscript: MTLS, SDC, SM, MC, and NS. Critical revision of the manuscript for important intellectual content: all authors. Statistical analysis: MC, DG, and ET. Administrative, technical, or material support: VS and ET. All authors contributed to the article and approved the submitted version.

Funding

This study has been financed by Italian Ministry of Health within Ricerca Corrente 2021-Special Projects-Vax4Frail.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

We are indebted for their precious support to this study the Research Director Dr. Giuseppe Ippolito, and the Deputy General Director for Health Research and Innovation Dr. Gaetano Guglielmi, from the Italian Ministry of Health. We would also like to thank the patients who will be enrolled in the VAX4FRAIL study and their families and all those who will be actively involved in their continuous care, study data collection and analysis, and ultimately in the scientific production that will result from this research.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fonc.2022.855723/full#supplementary-material

References

1. Baden LR, El Sahly HM, Essink B, Kotloff K, Frey S, Novak R, et al. Efficacy and Safety of the mRNA-1273 SARS-CoV-2 Vaccine. N Engl J Med (2021) 384(5):403–16. doi: 10.1056/NEJMoa2035389

CrossRef Full Text | Google Scholar

2. Polack FP, Thomas SJ, Kitchin N, Absalon J, Gurtman A, Lockhart S, et al. Safety and Efficacy of the BNT162b2 mRNA Covid-19 Vaccine. N Engl J Med (2020) 383(27):2603–15. doi: 10.1056/NEJMoa2034577

CrossRef Full Text | Google Scholar

3. Lee LYW, Cazier JB, Starkey T, Briggs SEW, Arnold R, Bisht V, et al. COVID-19 Prevalence and Mortality in Patients With Cancer and the Effect of Primary Tumour Subtype and Patient Demographics: A Prospective Cohort Study. Lancet Oncol (2020) 21(10):1309–16. doi: 10.1016/S1470-2045(20)30442-3

CrossRef Full Text | Google Scholar

4. Lee LY, Cazier JB, Angelis V, Arnold R, Bisht V, Campton NA, et al. COVID-19 Mortality in Patients With Cancer on Chemotherapy or Other Anticancer Treatments: A Prospective Cohort Study. Lancet (2020) 395(10241):1919–26. doi: 10.1016/S0140-6736(20)31173-9

CrossRef Full Text | Google Scholar

5. Chaudhry F, Bulka H, Rathnam AS, Said OM, Lin J, Lorigan H, et al. COVID-19 in Multiple Sclerosis Patients and Risk Factors for Severe Infection. J Neurol Sci (2020) 418:117147. doi: 10.1016/j.jns.2020.117147

CrossRef Full Text | Google Scholar

6. Salter A, Fox RJ, Newsome SD, Halper J, Li DKB, Kanellis P, et al. Outcomes and Risk Factors Associated With SARS-CoV-2 Infection in a North American Registry of Patients With Multiple Sclerosis. JAMA Neurol (2021) 78(6):699–708. doi: 10.1001/jamaneurol.2021.0688. Erratum in: JAMA Neurol

CrossRef Full Text | Google Scholar

7. Goldman JD, Robinson PC, Uldrick TS, Ljungman P. COVID-19 in Immunocompromised Populations: Implications for Prognosis and Repurposing of Immunotherapies. J Immunother Cancer (2021) 9(6):e002630. doi: 10.1136/jitc-2021-002630

CrossRef Full Text | Google Scholar

8. Rodríguez-Blanco N, Montero-Navarro S, Botella-Rico JM, Felipe-Gómez AJ, Sánchez-Más J, Tuells J. Willingness to Be Vaccinated Against COVID-19 in Spain Before the Start of Vaccination: A Cross-Sectional Study. Int J Environ Res Public Health (2021) 18(10):5272. doi: 10.3390/ijerph18105272

CrossRef Full Text | Google Scholar

9. Chan WL, Ho YT, Wong CK, Choi HC, Lam KO, Yuen KK, et al. Acceptance of COVID-19 Vaccination in Cancer Patients in Hong Kong: Approaches to Improve the Vaccination Rate. Vaccines (Basel) (2021) 9(7):792. doi: 10.3390/vaccines9070792

CrossRef Full Text | Google Scholar

10. Brodziak A, Sigorski D, Osmola M, Wilk M, Gawlik-Urban A, Kiszka J, et al. Attitudes of Patients With Cancer Towards Vaccinations-Results of Online Survey With Special Focus on the Vaccination Against COVID-19. Vaccines (Basel) (2021) 9(5):411. doi: 10.3390/vaccines9050411

CrossRef Full Text | Google Scholar

11. Hwang JK, Zhang T, Wang AZ, Li Z. COVID-19 Vaccines for Patients With Cancer: Benefits Likely Outweigh Risks. J Hematol Oncol (2021) 14(1):38. doi: 10.1186/s13045-021-01046-w

CrossRef Full Text | Google Scholar

12. Uhr L, Mateen FJ. COVID-19 Vaccine Hesitancy in Multiple Sclerosis: A Cross-Sectional Survey. Mult Scler (2021) 27:13524585211030647. doi: 10.1177/13524585211030647

CrossRef Full Text | Google Scholar

13. Ehde DM, Roberts MK, Humbert AT, Herring TE, Alschuler KN. COVID-19 Vaccine Hesitancy in Adults With Multiple Sclerosis in the United States: A Follow Up Survey During the Initial Vaccine Rollout in 2021. Mult Scler Relat Disord (2021) 54:103163. doi: 10.1016/j.msard.2021.103163

CrossRef Full Text | Google Scholar

14. Gaur P, Agrawat H, Shukla A. COVID-19 Vaccine Hesitancy in Patients With Systemic Autoimmune Rheumatic Disease: An Interview-Based Survey. Rheumatol Int (2021) 41(9):1601–5. doi: 10.1007/s00296-021-04938-9

CrossRef Full Text | Google Scholar

15. Riad A, Abdulqader H, Morgado M, Domnori S, Koščík M, Mendes JJ, et al. Global Prevalence and Drivers of Dental Students' COVID-19 Vaccine Hesitancy. Vaccines (Basel) (2021) 9(6):566. doi: 10.3390/vaccines9060566

CrossRef Full Text | Google Scholar

16. Yasmin F, Najeeb H, Moeed A, Naeem U, Asghar MS, Chughtai NU, et al. COVID-19 Vaccine Hesitancy in the United States: A Systematic Review. Front Public Health (2021) 9:770985. doi: 10.3389/fpubh.2021.770985

CrossRef Full Text | Google Scholar

17. Troiano G, Nardi A. Vaccine Hesitancy in the Era of COVID-19. Public Health (2021) 194:245–51. doi: 10.1016/j.puhe.2021.02.025

CrossRef Full Text | Google Scholar

18. Waissengrin B, Agbarya A, Safadi E, Padova H, Wolf I. Short-Term Safety of the BNT162b2 mRNA COVID-19 Vaccine in Patients With Cancer Treated With Immune Checkpoint Inhibitors. Lancet Oncol (2021) 22(5):581–3. doi: 10.1016/S1470-2045(21)00155-8

CrossRef Full Text | Google Scholar

19. So ACP, McGrath H, Ting J, Srikandarajah K, Germanou S, Moss C, et al. COVID-19 Vaccine Safety in Cancer Patients: A Single Centre Experience. Cancers (Basel) (2021) 13(14):3573. doi: 10.3390/cancers13143573

CrossRef Full Text | Google Scholar

20. Ram R, Hagin D, Kikozashvilli N, Freund T, Amit O, Bar-On Y, et al. Safety and Immunogenicity of the BNT162b2 mRNA COVID-19 Vaccine in Patients After Allogeneic HCT or CD19-Based CART Therapy-A Single-Center Prospective Cohort Study. Transplant Cell Ther (2021) 27(9):S2666–6367(21)01027-7. doi: 10.1016/j.jtct.2021.06.024

CrossRef Full Text | Google Scholar

21. Monin L, Laing AG, Muñoz-Ruiz M, McKenzie DR, Del Molino Del Barrio I, Alaguthurai T, et al. Safety and Immunogenicity of One Versus Two Doses of the COVID-19 Vaccine BNT162b2 for Patients With Cancer: Interim Analysis of a Prospective Observational Study. Lancet Oncol (2021) 22(6):765–78. doi: 10.1016/S1470-2045(21)00213-8

CrossRef Full Text | Google Scholar

22. Mahil SK, Bechman K, Raharja A, Domingo-Vila C, Baudry D, Brown MA, et al. The Effect of Methotrexate and Targeted Immunosuppression on Humoral and Cellular Immune Responses to the COVID-19 Vaccine BNT162b2: A Cohort Study. Lancet Rheumatol (2021) 3(9):e627–37. doi: 10.1016/S2665-9913(21)00212-5

CrossRef Full Text | Google Scholar

23. Pimpinelli F, Marchesi F, Piaggio G, Giannarelli D, Papa E, Falcucci P, et al. Fifth-Week Immunogenicity and Safety of Anti-SARS-CoV-2 BNT162b2 Vaccine in Patients With Multiple Myeloma and Myeloproliferative Malignancies on Active Treatment: Preliminary Data From a Single Institution. J Hematol Oncol (2021) 14(1):81. doi: 10.1186/s13045-021-01090-6

CrossRef Full Text | Google Scholar

24. Chevallier P, Coste-Burel M, Le Bourgeois A, Peterlin P, Garnier A, Béné MC, et al. Safety and Immunogenicity of a First Dose of SARS-CoV-2 mRNA Vaccine in Allogeneic Hematopoietic Stem-Cells Recipients. EJHaem (2021) 1:10. doi: 10.1002/jha2.242. Epub ahead of print

CrossRef Full Text | Google Scholar

25. Maneikis K, Šablauskas K, Ringelevičiūtė U, Vaitekėnaitė V, Čekauskienė R, Kryžauskaitė L, et al. Immunogenicity of the BNT162b2 COVID-19 mRNA Vaccine and Early Clinical Outcomes in Patients With Haematological Malignancies in Lithuania: A National Prospective Cohort Study. Lancet Haematol (2021) 8(8):e583–92. doi: 10.1016/S2352-3026(21)00169-1

CrossRef Full Text | Google Scholar

26. Lotan I, Wilf-Yarkoni A, Friedman Y, Stiebel-Kalish H, Steiner I, Hellmann MA. Safety of the BNT162b2 COVID-19 Vaccine in Multiple Sclerosis (MS): Early Experience From a Tertiary MS Center in Israel. Eur J Neurol (2021) 28(11):3742–8. doi: 10.1111/ene.15028

CrossRef Full Text | Google Scholar

27. Achiron A, Dolev M, Menascu S, Zohar DN, Dreyer-Alster S, Miron S, et al. COVID-19 Vaccination in Patients With Multiple Sclerosis: What We Have Learnt by February 2021. Mult Scler (2021) 27(6):864–70. doi: 10.1177/13524585211003476

CrossRef Full Text | Google Scholar

28. Allen-Philbey K, Stennett A, Begum T, Johnson AC, Dobson R, Giovannoni G, et al. Experience With the COVID-19 AstraZeneca Vaccination in People With Multiple Sclerosis. Mult Scler Relat Disord (2021) 52:103028. doi: 10.1016/j.msard.2021.103028

CrossRef Full Text | Google Scholar

29. Furer V, Eviatar T, Zisman D, Peleg H, Paran D, Levartovsky D, et al. Immunogenicity and Safety of the BNT162b2 mRNA COVID-19 Vaccine in Adult Patients With Autoimmune Inflammatory Rheumatic Diseases and in the General Population: A Multicentre Study. Ann Rheum Dis (2021) 80(10):1330–8. doi: 10.1136/annrheumdis-2021-220647

CrossRef Full Text | Google Scholar

30. Rotondo C, Cantatore FP, Fornaro M, Colia R, Busto G, Rella V, et al. Preliminary Data on Post Market Safety Profiles of COVID 19 Vaccines in Rheumatic Diseases: Assessments on Various Vaccines in Use, Different Rheumatic Disease Subtypes, and Immunosuppressive Therapies: A Two-Centers Study. Vaccines (Basel) (2021) 9(7):730. doi: 10.3390/vaccines9070730

CrossRef Full Text | Google Scholar

31. Braun-Moscovici Y, Kaplan M, Braun M, Markovits D, Giryes S, Toledano K, et al. Disease Activity and Humoral Response in Patients With Inflammatory Rheumatic Diseases After Two Doses of the Pfizer mRNA Vaccine Against SARS-CoV-2. Ann Rheum Dis (2021) 80(10):1317–21. doi: 10.1136/annrheumdis-2021-220503

CrossRef Full Text | Google Scholar

32. Felten R, Kawka L, Dubois M, Ugarte-Gil MF, Fuentes-Silva Y, Piga M, et al. Tolerance of COVID-19 Vaccination in Patients With Systemic Lupus Erythematosus: The International VACOLUP Study. Lancet Rheumatol (2021) 3():e613–5. doi: 10.1016/S2665-9913(21)00221-6

CrossRef Full Text | Google Scholar

33. Cherian S, Paul A, Ahmed S, Alias B, Manoj M, Santhosh AK, et al. Safety of the ChAdOx1 Ncov-19 and the BBV152 Vaccines in 724 Patients With Rheumatic Diseases: A Post-Vaccination Cross-Sectional Survey. Rheumatol Int (2021) 41(8):1441–5. doi: 10.1007/s00296-021-04917-0

CrossRef Full Text | Google Scholar

34. Connolly CM, Ruddy JA, Boyarsky BJ, Avery RK, Werbel WA, Segev DL, et al. Safety of the First Dose of mRNA SARS-CoV-2 Vaccines in Patients With Rheumatic and Musculoskeletal Diseases. Ann Rheum Dis (2021) 19:annrheumdis-2021-220231. doi: 10.1136/annrheumdis-2021-220231

CrossRef Full Text | Google Scholar

35. Boekel L, Kummer LY, van Dam KPJ, Hooijberg F, van Kempen Z, Vogelzang EH, et al. Adverse Events After First COVID-19 Vaccination in Patients With Autoimmune Diseases. Lancet Rheumatol (2021) 3(8):e542–5. doi: 10.1016/S2665-9913(21)00181-8

CrossRef Full Text | Google Scholar

36. Sattui SE, Liew JW, Kennedy K, Sirotich E, Putman M, Moni TT, et al. Early Experience of COVID-19 Vaccination in Adults With Systemic Rheumatic Diseases: Results From the COVID-19 Global Rheumatology Alliance Vaccine Survey. RMD Open (2021) 7(3):e001814. doi: 10.1136/rmdopen-2021-001814

CrossRef Full Text | Google Scholar

37. Ali A, Dwyer D, Wu Q, Wang Q, Dowling CA, Fox DA, et al. Characterization of Humoral Response to COVID mRNA Vaccines in Multiple Sclerosis Patients on Disease Modifying Therapies. Vaccine (2021) 39(41):6111–6. doi: 10.1016/j.vaccine.2021.08.078

CrossRef Full Text | Google Scholar

38. Cavanna L, Citterio C, Biasini C, Madaro S, Bacchetta N, Lis A, et al. COVID-19 Vaccines in Adult Cancer Patients With Solid Tumours Undergoing Active Treatment: Seropositivity and Safety. A Prospective Observational Study in Italy. Eur J Cancer (2021) 157:441–9. doi: 10.1016/j.ejca.2021.08.035

CrossRef Full Text | Google Scholar

39. Peeters M, Verbruggen L, Teuwen L, Vanhoutte G, Vande Kerckhove S, Peeters B, et al. Reduced Humoral Immune Response After BNT162b2 Coronavirus Disease 2019 Messenger RNA Vaccination in Cancer Patients Under Antineoplastic Treatment. ESMO Open (2021) 6(5):100274. doi: 10.1016/j.esmoop.2021.100274

CrossRef Full Text | Google Scholar

40. Shroff RT, Chalasani P, Wei R, Pennington D, Quirk G, Schoenle MV, et al. Immune Responses to Two and Three Doses of the BNT162b2 mRNA Vaccine in Adults With Solid Tumors. Nat Med (2021) 27(11):2002–11. doi: 10.1038/s41591-021-01542-z

CrossRef Full Text | Google Scholar

41. Di Noia V, Pimpinelli F, Renna D, Barberi V, Maccallini MT, Gariazzo L, et al. Immunogenicity and Safety of COVID-19 Vaccine BNT162b2 for Patients With Solid Cancer: A Large Cohort Prospective Study From a Single Institution. Clin Cancer Res (2021) 27(24):6815–23. doi: 10.1158/1078-0432.CCR-21-2439

CrossRef Full Text | Google Scholar

42. Linardou H, Spanakis N, Koliou GA, Christopoulou A, Karageorgopoulou S, Alevra N, et al. Responses to SARS-CoV-2 Vaccination in Patients With Cancer (ReCOVer Study): A Prospective Cohort Study of the Hellenic Cooperative Oncology Group. Cancers (Basel) (2021) 13(18):4621. doi: 10.3390/cancers13184621

CrossRef Full Text | Google Scholar

43. Lasagna A, Agustoni F, Percivalle E, Borgetto S, Paulet A, Comolli G, et al. A Snapshot of the Immunogenicity, Efficacy and Safety of a Full Course of BNT162b2 Anti-SARS-CoV-2 Vaccine in Cancer Patients Treated With PD-1/PD-L1 Inhibitors: A Longitudinal Cohort Study. ESMO Open (2021) 6(5):100272. doi: 10.1016/j.esmoop.2021.100272

CrossRef Full Text | Google Scholar

44. Shmueli ES, Itay A, Margalit O, Berger R, Halperin S, Jurkowicz M, et al. Efficacy and Safety of BNT162b2 Vaccination in Patients With Solid Cancer Receiving Anticancer Therapy - a Single Centre Prospective Study. Eur J Cancer (2021) 157:124–31. doi: 10.1016/j.ejca.2021.08.007

CrossRef Full Text | Google Scholar

45. Goshen-Lago T, Waldhorn I, Holland R, Szwarcwort-Cohen M, Reiner-Benaim A, Shachor-Meyouhas Y, et al. Serologic Status and Toxic Effects of the SARS-CoV-2 BNT162b2 Vaccine in Patients Undergoing Treatment for Cancer. JAMA Oncol (2021) 8:e212675. doi: 10.1001/jamaoncol.2021.2675

CrossRef Full Text | Google Scholar

46. Delvino P, Bozzalla Cassione E, Biglia A, Quadrelli VS, Bartoletti A, et al. Safety of BNT162b2 mRNA COVID-19 Vaccine in a Cohort of Elderly, Immunocompromised Patients With Systemic Vasculitis. Clin Exp Rheumatol (2021).

Google Scholar

47. Agrati C, Di Cosimo S, Fenoglio D, Apolone G, Ciceri F, Ciliberto G, et al. COVID-19 Vaccination in Fragile Patients: Current Evidence and an Harmonized Transdisease Trial. Front Immunol (2021) 12:704110. doi: 10.3389/fimmu.2021.704110

CrossRef Full Text | Google Scholar

48. Costantini M, Rabitti E, Beccaro M, Fusco F, Peruselli C, La Ciura P, et al. Validity, Reliability and Responsiveness to Change of the Italian Palliative Care Outcome Scale: A Multicenter Study of Advanced Cancer Patients. BMC Palliat Care (2016) 15:23. doi: 10.1186/s12904-016-0095-6

CrossRef Full Text | Google Scholar

49. Landis JR, Koch GG. The Measurement of Observer Agreement for Categorical Data. Biometrics (1977) 33(1):159–74.

Google Scholar

50. Riad A, Schünemann H, Attia S, Peričić TP, Žuljević MF, Jürisson M, et al. COVID-19 Vaccines Safety Tracking (CoVaST): Protocol of a Multi-Center Prospective Cohort Study for Active Surveillance of COVID-19 Vaccines’ Side Effects. Int J Environ Res Public Health (2021) 18(15):7859. doi: 10.3390/ijerph18157859

CrossRef Full Text | Google Scholar