The Development and Testing of Balanced Communication Material for a Population-Based Breast Cancer Screening Program

Organized breast cancer screening (BCS) programs rely on written communication materials for achieving participation and informing women about screening-related benefits and limits. In order to achieve informed decisions and to maximize reader acceptance, the Agency for Health Protection of the metropolitan area of Milan aimed at improving the communication materials of the local BCS program through a multiphase, mixed-method process. Multidisciplinary working groups drafted three sets of materials: postal letters, an informative leaflet, and a question-and-answer online set. Readability was assessed using the Italian language-tailored Gulpease index. Suitability and Comprehensibility were assessed using the SAM + CAM instrument. User perception was investigated through “think aloud” interviews in two consecutive purposive samples. Participants’ intention to participate in the program was also assessed. After each phase was completed, materials were readapted, and previous phases were repeated, to maintain the pre-defined Gulpease and SAM + CAM targets. During the quality improvement process, the overall mean Gulpease and SAM + CAM scores increased from 65.5 (s. d. 10.4) to 67.7 (s. d. 8.2) and from 78 (s. d. 5.6) to 83 (s. d. 4.1), respectively. In light of the results of the first round of interviews, materials underwent rewriting and layout revision, which was generally appreciated during the second round, with a non-significant increase in the intention to participate in the program. However, negative emotions and miscomprehension concerning overdiagnosis were frequently reported, although less frequent in the second round, after rewording of the text. The mixed-method multistep process involving all the relevant key players allowed a balance among the multifaceted aspects of communication.

Introduction

Screening through biennial mammography can reduce a woman’s risk of dying from breast cancer (BC) (Independent UK Panel on Breast Cancer Screening, 2012). Organized breast cancer screening (BCS) programs, such as the one provided by the local Agency for Health Protection in the metropolitan area of Milan, rely on written communication materials for the dual purpose of achieving participation and informing women about BCS-related benefits and limits, in line with best practice recommendations (Independent UK Panel on Breast Cancer Screening, 2012; European Commission Initiative on Breast Cancer, 2019). The use of an information leaflet (IL), usually combined with the invitation to attend the program, is crucial for obtaining informed consent and ensuring long-term attendance. However, studies have identified a lack of information in these materials, particularly on the potential unintended harms (Jørgensen and Gøtzsche, 2004; Giordano et al., 2005; Jørgensen and Gøtzsche, 2006), such as overdiagnosis, which women invited to do BCS are often unaware of (Hersch et al., 2013; Waller et al., 2013; Henriksen et al., 2015).

In recent years, a growing number of publications have investigated the provision of balanced information regarding BCS, aimed at the achievement of women’s empowerment along with the ability to make an informed choice. Providing balanced information regarding the benefits and unintended harms of BCS, overdiagnosis in particular, improves women’s ability to make an informed choice, but appears to negatively affect the intention to undergo BCS (Martínez-Alonso et al., 2017; Ivlev et al., 2017). The ethical duty of providing complete and balanced information should weigh heavier than nudging women to attend the program (Ploug et al., 2012).

To address these issues and to maximize readers’ acceptance, we aimed at improving the communication materials for our BCS program through an integrated process involving different health professionals as well as a sample of the intended audience. Development of materials involved repeated assessments of the readability, comprehensibility and communicative effectiveness (CE), as well as two rounds of user testing, in order to identify, and possibly correct, potential pitfalls in terms of comprehension and acceptability. The primary outcome of the present study was quality improvement, defined as the provision of balanced information in line with the International (US Department of Health and Human Services and Centers for Disease Control and Prevention, 2009; Independent UK Panel on Breast Cancer Screening, 2012; European Commission Initiative on Breast Cancer, 2019) and Italian best practices (Italian Group for Cervical Cancer Screening, 2009), while meeting the informative expectations of the intended audience, and maintaining the pre-defined quality targets. For this reason, CE was assessed as a separate, secondary outcome, only for investigational purposes, in order to avoid a purely nudge strategy, and to minimize the decisional conflict.

Methods

We used an integrative multi-phase, mixed-method approach to evaluate readability, suitability, comprehensibility and the CE of materials. The process was designed according to a framework for the evaluation of patient information materials (Garner et al., 2012).

Setting and Material Types

In the metropolitan area of Milan (about 3.5 million inhabitants), more than 100,000 women per year attend our BCS program. Biennial mammography screening is offered to women aged 50–74 years and, starting from 2019, annual screening is offered to women aged 45–49, previously screened only on request. All eligible women receive a personal invitation letter, together with an IL. Hospital-based screening services provide the mammograms and diagnostic assessment tests. Women with positive results are contacted by telephone and referred for diagnostic assessment, while results within the norm are notified by post.

We developed three sets of new communication materials. Postal letters (the first invitation letter, the notification letter for normal results, and the invitation letter to subsequent screening rounds) are mainly focused on brief communications and practical instructions. The IL, to be sent by post together with the first invitation letter, covers the purpose of BCS, a brief epidemiological overview on BC, a description of possible benefits and unintended harms (supported by selected numerical data), practical issues about the program and the mammography in particular, and contacts of the local BCS unit. The IL and the letters were updated from previously used versions. The question-and-answer set (Q&A), a new type of material to be published on a BCS-devoted page of the institutional web site, contains more detailed information, including the description of the dedicated paths for specific populations (breast implant carriers, women at hereditary risk or with history of BC). The subsequent versions of the materials are available in the Supplementary Materials S1, S2.

Study Design

The process of material development was carried out in an integrative flow through several distinct evaluation phases, as depicted in Figure 1. After each phase was completed, materials were readapted, in order to achieve pre-defined standards of readability, suitability and comprehensibility and to meet the target population’s expectations. At each revision of the materials, previous quality assessment phases were repeated, in order to maintain the pre-defined quality targets.

FIGURE 1

FIGURE 1. Timeline and procedures involved in the development of information materials. *Professionals involved in the first drafts included Public Health physicians, health visitors, specialized nurses and specialized health clerks belonging to the Agency’s Screening Unit. **Graphic layout was designed in collaboration with experts from the Agency’s Communication Unit. ***The interviews were conducted by trained Public Health physicians and medical residents (FD, DC, LC, NP and JB). Re-assessments on subsequent versions of the materials are depicted in the textured bars.

First Draft (Phase 0)

Multi-disciplinary working groups (Figure 1) drafted the first versions of the materials. Content, style and design complied with CDC recommendations for the development of informative material tailored to the target population’s literacy level (US Department of Health and Human Services and Centers for Disease Control and Prevention, 2009) and with the best practices promoted by the National Center for Screening Monitoring for written communication materials (Italian Group for Cervical Cancer Screening, 2009).

Readability Assessment (Phase 1)

We aimed to develop a text that would be easy to read for the target population. Readability tests are generally modeled on distributional parameters measuring linguistic aspects of a text (average number of words per sentence, average length of sentences, average word length). We used the Gulpease index, an Italian language-tailored readability test (Lucisano and Piemontese, 1988), which measures word length in letters, rather than in syllables as for English language-tailored readability scores. The score computation results in a value between 0 and 100: lower values indicate harder to read texts, with variable cut-off values depending on the level of education. We chose a Gulpease index ≥60 as a target, which corresponds to easy readability for a lower secondary educational level. In Italy, education is free and compulsory until 16 years of age starting from 2006 (Law No. 296, December 27, 2006), while for anyone born before 1952 or between 1952 and 1992, compulsory and free education lasted respectively at 14 and 15 years of age. Since lower secondary school ends at 14 years of age, the chosen threshold presumably covered the educational levels of the target population of the BCS program. As an Italian language-tailored complementary measure of the qualitative aspects of readability (structural complexity, grammatical correctness, meaning), we performed the DyLan READ-IT index, which estimates the global percentage of difficulty of the text. Both tests were computed using the DyLan Lab online text tool (National Research Council of Italy, 2011).

Suitability and Comprehensibility Preliminary Assessment (Phase 2)

We aimed to develop comprehensible materials tailored to the target populations’ health literacy (HL) level. The suitability and comprehensibility assessment of materials (SAM + CAM) is a semi-quantitative instrument for evaluating to what extent health communication materials fit with the HL skills of target readers (Helitzer et al., 2009). It includes 22 variables divided into six categories: Content, Literacy Demand, Numeracy, Graphic material, Layout/Typography, and Learning/Stimulation/Motivation. Materials with scores between 0 and 39% are considered “not suitable”, those between 40 and 69% “adequate”, while scores ≥70%, the cut-off value we have set as a target, qualify materials as “superior”. Two readers independently computed the SAM + CAM score for the first drafts of materials. Inter-rater reliability was assessed using Krippendorff’s alpha (Krippendorff, 2004). Overall reliability was inadequate (alpha = 0.549). Discrepant ratings (alpha <0.67) occurred for 8 out of 22 items. Disagreements were resolved by discussion until consensus was achieved between raters. For subsequent versions, the SAM + CAM score was performed by a single rater.

User Perception Assessment (Phase 3)

We aimed to investigate how the BCS target population interprets BCS information materials, identifying and editing confusing or difficult to read text areas. Our secondary, exploratory, objective was to analyze additional cognitive aspects of materials’ perception. We performed “think aloud” interviews in the administrative receptions of three hospitals located in the city center, in the suburbs and in the province, respectively (De Nard et al., 2021). The “think aloud” method aims at collecting thoughts, verbalized while reading a given written material aloud, that would otherwise have remained silent, revealing the metacognitive aspects of materials’ perception (Ericsson and Simon, 1980). We purposively recruited women aged 40–74 years with no BC history living in Milan’s metropolitan area. Participants were recruited until thematic saturation, defined as the point in data collection when new incoming data produces little or no new information to address the research question (Guest et al., 2006), was achieved. All participants read an informative sheet, adapted from Smith et al. (2015), describing our aims, privacy policy, and the instructions for the “think aloud” method. The informative sheet is available in the Supplementary Material S3. Participants previously completed a socio-demographic questionnaire and practiced the think aloud method on a control information material. HL was assessed using the Single Item Literacy Screener (Morris et al., 2006). Participants were assigned different materials according to their age, HL level, screening status and web-searching habits, to ensure the correspondence with the intended readership, and the inclusion in each subset of materials of women of different age groups and HL levels. We used a marked protocol: participants were prompted to verbalize every time they encountered a red dot, placed at the end of each meaningful text unit (Smith et al., 2015). After modifying the materials based on the collected data, a second round of interviews was conducted in the suburban hospital. During the second round, women with BC history were also included in order to assess the consistency of personal experience with the content of materials. All interviews were transcribed verbatim.

Communicative Effectiveness (Phase 4)

We assessed CE using a proxy measure of simulated behavior (Garner et al., 2012). Participants were asked the following question: “After reading the material, are you willing to participate in the program?”. Answers, when provided, were coded dichotomously, as “yes”, if the participant was willing to participate, or “no” if she were not willing to participate.

Data Analysis

Data regarding phases 1 and 2 was descriptively analyzed. Differences in socio-demographic variables between rounds (phase 3) were computed using Fisher’s exact test. The predictive value of socio-demographic variables and type of material in determining CE (phase 4) was investigated for each interview round by a univariate logistic regression model. Qualitative data collected during phase 3 was analyzed through inductive thematic analysis (TA) (Braun and Clarke, 2006).

Compliance With Ethical Standards

The study was performed in line with the latest amendment to the Helsinki Declaration (Rickham, 1964). According to local legislation (Lombardy Region regulation n. 3 of December 24, 2012), Public Health Agencies’ quality improvement projects are not required to seek ethical approval. Data collection and analysis were anonymous, so that participants are no longer identifiable. Informed consent was verbally acquired after participants read an informative sheet.

Results

Readability Assessment (Phase 1)

The overall mean Gulpease index improved throughout the quality improvement process from 65.5 (s. d. 10.4) for first drafts, to 67.7 (s. d. 8.2) for the final versions (Table 1). The most notable improvement was observed for the IL, whose first version showed insufficient readability (Gulpease index 56.1). The syntax was therefore simplified by reducing the length of the sentences and by a selective use of vocabulary, resulting in a final increase of 6.2 points in the Gulpease index. Similarly, the readability of the Q&A first draft was below the pre-defined threshold of 60 (Gulpease index 56.9), but it was improved by rewriting (Gulpease index 61–Round 1 version). However, readability was reduced following the first round of interviews, as new parts of a syntactically more complex text had been added (Gulpease index 60.2-final version, +3.3 points overall). The readability scores of the letters underwent minimal changes from the already adequate initial values, due to the shortness of the texts. As reported in Table 1, the READ-IT global level of difficulty decreased for almost all materials. The READ-IT index proved useful in identifying the most difficult areas or aspects (such as vocabulary or syntax) to read with the text.

TABLE 1

TABLE 1. Readability assessment (phase 1), Suitability and Comprehensibility assessment (phase 2). The achievement of predefined thresholds of the Gulpease index (≥60) and of the SAM + CAM total score (≥70), was required to move to the next phase. The letters tested during the first round were the final versions.

Suitability and Comprehensibility Preliminary Assessment (Phase 2)

During the quality improvement process, an increase in the overall mean SAM + CAM score was obtained, from 78.04 (s. d. 5.64) for first drafts to 83.02 (s. d. 4.14) for the final versions, indicating “superior” quality for all versions, with the best subscores for the Literacy Demand and Layout/Typography categories (Table 1). The most significant improvements concerned the IL, which rose by 14.3 points in the SAM + CAM score, and the Q&A (+10.6 points). The lowest ratings for the IL’s first draft were reported in the Graphics and Layout/Typography categories, which showed an increase of 50 and 16.6% respectively after the editing process. The categories showing the lowest scores for the first draft of the Q&A, that of Learning/Stimulation/Motivation and Content, were later improved (8.3 and 25% increase respectively). Noticeably, the final Q&A SAM + CAM score was further reduced to 80% after online publication, since the layout was adapted to the website standards. The letters’ SAM + CAM score remained unchanged during the process, with the lowest ratings observed in the Graphics and Learning/Stimulation/Motivation categories.

User Perception Assessment (Phase 3)

A total of 44 women participated: 34 women took part in the first round of interviews, involving all types of materials, while 10 were included in the second round, which covered only the IL and the Q&A, since the letters had been subject to negligible criticism. The socio-demographic characteristics of the participants are reported in Table 2. No significant difference in socio-demographic variables was found between the two rounds.

TABLE 2

TABLE 2. Participants’ characteristics: comparison between rounds.

The interviews yielded 443 utterances (336 in the first round, 107 in the second round). TA identified five themes within the narrative data: validation of the information provided, information preferences, negative emotions caused by BCS limits, disproportioned risk perception, and organizational preferences. These findings are fully reported elsewhere (De Nard et al., 2021). In the final Q&A layout, as a result of first round’s reported information preferences, we introduced the possibility of optionally reading specific sections (including insights into previously unaddressed topics) by clicking on the corresponding links. As for organizational preferences, complaints about the starting age and frequency of BCS, screening tests offered and waiting times were frequently reported. In particular, preconceptions regarding the age limits of BCS and the frequency of the mammography were frequently observed among younger interviewees: some of them reportedly received a clinical recommendation to undergo annual ultrasound and/or mammography screening starting at the age of 40 or earlier, even in cases without familiar BC history. In the final versions of the texts, we explained the quality standards and the evidence supporting the program’s organization. These changes were generally appreciated during the second round. Interestingly, some women stated that they would prefer not to be informed about the limits of BCS, including the risk of overdiagnosis and overtreatment, which in some cases elicited anxiety or disappointment, while other participants appreciated the information provided, while acknowledging that such content may cause concern. We observed a disproportion between the perceived risk of interval BC (often overestimated) and the perceived risk of overdiagnosis, usually underestimated, and almost always misunderstood. To address this issue, we replaced the definition of overdiagnosis (“In some cases, BCS could lead to the identification and treatment of slow growing, possibly benign, tumors”) with a simplified definition (“A small part of the tumors identified through BCS may not be dangerous to health. However, to date, it is not possible to know which ones, so a treatment is always proposed”), which was generally well-accepted during the second round, but in some cases still misunderstood. Of note, utterances attesting miscomprehension of the topics addressed by the materials were observed among women with different educational levels, including University level. The main changes in the materials’ texts between rounds are reported in Supplementary Material S4.

Communicative Effectiveness (Phase 4)

Thirty-five women answered the question concerning CE (25 in the first round, 10 in the second). All of them reportedly understood the contents of the materials. After the first round, 18 women (72%) stated that they were willing to participate in the program. A nonsignificant improvement in CE was observed for the second round (+8% reported willingness to adhere; p = 1, two-sided Fisher’s exact test). Using univariate logistic regression analysis on first-round data, women who were interviewed in the provincial hospital (vs. the urban hospital; OR = 16.5, 95% CI 1.09–25.1, pr (Z) = 0.043) or living in the province (vs. the city of Milan; OR = 30, 95% CI 2.58–348.77, pr (Z) = 0.007) showed a greater propensity to adhere to the program.

Distribution of the Materials

The Q&A and the IL have been available on the Agency’s official website www.ats-milano.it since December 2018; the web outline and the direct link are available in Supplementary Material S2. The IL and the letters have been distributed since March 2019. All materials include an invitation for readers to collaborate on a further improvement of the contents, and reference contacts.

Discussion

The development of our BCS program’s information materials involved synthesizing the evidence, providing understandable numerical data in line with the best practices, assessing readability and suitability, investigating comprehensibility and acceptability a priori and with the intended audience. The mixed-method multistep process involving all the relevant key players (including the target population) allowed a balance between the multifaceted aspects of communication. All interviewees reportedly understood the contents of the materials, and most of them were willing to participate in the program after reading the materials. The acceptability of information regarding the limits of BCS varied, although it seems to have improved after the interactive tailoring of the content of the Q&A online set.

The findings of our TA confirmed the observations reported by previous studies performed in different settings. In particular, we observed that the risk of overdiagnosis is unknown to women invited for BCS (Hersch et al., 2013; Waller et al., 2013; Henriksen et al., 2015), and that the concept of overdiagnosis itself is complex to explain within the limits of a brief text format (Waller et al., 2013). Communicating the limits of our BCS program often yielded to confusion, negative emotions (Henriksen et al., 2015), or even denial. As previously reported (Henriksen et al., 2015), these reactions can be interpreted in light of the cognitive dissonance theory, which describes how an internal debate, involving feelings of tension and uncertainty, is elicited by the confrontation with new information that contradicts pre-existing beliefs and attitudes (Festinger, 1951). Our study adds a further dimension to this finding, in that pre-existing beliefs and attitudes might be influenced (long before screening begins) by the circle of acquaintances, as previously reported (Henriksen et al., 2015), but also by information distributed by other sources. In the Italian context, private health companies, charities, and health professionals, provide information regarding BCS through various forms of communication, including the internet, which in some cases is inconsistent with the information officially provided by population-based BCS programs.

Readability, suitability and comprehensibility tests have proved feasible instruments to guide the development of health information materials according to the target population’s literacy demands. However, few studies investigated the readability of online health information in Italian (Dini et al., 2017). To our knowledge, this is the first study assessing Italian-language patient information material with the SAM + CAM tool. The use of readability, suitability and comprehensibility tests is not yet standardized in Italian language-base breast cancer screening communication materials, and further research is needed to assess its impact on the quality of patient decision making.

Developers of BCS information should therefore consider interactive and tailored communication strategies. In addition, the different providers of BCS information, including charities, private health providers and clinical specialists, should develop a shared communication strategy, in order to promote clarity and to avoid miscomprehension. Providing balanced information at an earlier age could possibly prevent the formation and persistence of preconceptions that could negatively affect informed decision making. However, further research is needed to address this issue.

Our study is affected by some limitations. First, we chose the intention to participate in the program as an indicator of CE, while we did not assess indicators of quality of decision making (knowledge and decisional conflict). Our findings regarding CE should be interpreted with caution, in light of the reduced statistical power due to the small sample size. In addition, we did not assess participants’ socio-economic status, previously described as a determinant of participation in cancer screening programs (Carrozzi et al., 2015). Lombardy is the richest region of the country and one of the richest regions in the EU, with a GDP per capita 26% higher than the EU average, and an unemployment rate 6.3% below the EU average (European Commission, 2020). However, socio-economic indicators are unequally distributed in the metropolitan area of Milan: compared to its province, the City of Milan shows higher average annual income, population density, employment rate and a higher prevalence of graduates (Lelo et al., 2018). In our study, women living or working in the city of Milan appeared to be less prone to adhere to our BCS program in respect to those living or working in the province. Although less socio-economically disadvantaged, women living or working in the city might be possibly more exposed to the private healthcare marketing offering annual screening starting at an earlier age and additional ultrasound screening, and this finding is worth future investigations.

Conclusion

A mixed-method approach, which combines a preliminary assessment of the multifaceted aspects of health communication, using readability, suitability and comprehensibility tools, and the qualitative assessment of acceptability and CE, has proven to be an appropriate strategy for developing BCS information material. Patient-centered methods that actively involve the target population allow the developers of patient information material to identify discrepancies between the writers’ intended meaning and the actual meaning, constructed by the readers. The findings of our qualitative analysis enabled us to communicate complex concepts, such as BC overdiagnosis, in a better framed and more understandable way, according to women’s expectations, while maintaining the purpose of helping informed decisions. In addition, the analysis of the narrative data collected through TA interviews provided additional insights into women’s organizational and information preferences, emotional aspects and individual value systems, as well as into potential urban inequalities that deserve further investigation. BCS information development is an ongoing process, not only due to the increasing body of evidence, but also because women’s behavior, risk perception and value systems might change over time. Repeated assessments using the described mixed-method framework could address these changes in a timely and appropriate manner.

Data Availability Statement

The raw data supporting the conclusion of this article will be made available by the authors, without undue reservation.

Author Contributions

FD, SD, SC, PS, and AS formulated the study protocol. AB and SD participated in the first drafts of the materials. FD performed readability analyses; while the SAM + CAM score was calculated by FD and DC. FD and SD redrafted the materials. FD, DC, NP, LC, and AB participated in the think aloud study as interviewers and analyzed the data. Regression analysis was performed by EL. FD drafted the paper. All authors collaborated in the redrafting of the paper and approved the final version of the paper.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

Our sincere thanks to our interviewees, to the medical directions of the hospitals where the interviews took place, and to all the personnel of the Screening Unit of the Agency for Health Protection of the Metropolitan area of Milan that participated in the development of the materials.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fcomm.2021.719555/full#supplementary-material

References

Braun, V., and Clarke, V. (2006). Using Thematic Analysis in Psychology. Qual. Res. Psychol. 3, 77–101. doi:10.1191/1478088706qp063oa

CrossRef Full Text | Google Scholar

Carrozzi, G., Sampaolo, L., Bolognesi, L., Sardonini, L., Bertozzi, N., Giorgi Rossi, P., et al. (2015). Cancer Screening Uptake: Association with Individual Characteristics, Geographic Distribution, and Time Trends in Italy. Epidemiol. Prev. 39, 9–18.

PubMed Abstract | Google Scholar

De Nard, F., Deandrea, S., Bastiampillai, A. J., Cavazzana, L., Carnevali, D., Principi, N., et al. (2021). Women’s Perception, Preconceptions and Information Preferences for the Limitations of Breast Cancer Screening and Overdiagnosis: a Think-Aloud Study on the New Information Materials of the Agency for Health Protection of the Metropolitan City of Milan. Epidemiol. Prev., in press.

Google Scholar

Dini, G., Bragazzi, N. L., D'Amico, B., Montecucco, A., Igwe, S. C., Brigo, F., et al. (2017). A Reliability and Readability Analysis of Silicosis-Related Italian Websites: Implications for Occupational Health. Med. Lav. 108, 167–173. doi:10.23749/mdl.v108i3.6083

PubMed Abstract | CrossRef Full Text | Google Scholar

Ericsson, K. A., and Simon, H. A. (1980). Verbal Reports as Data. Psychol. Rev. 87, 215–251. doi:10.1037/0033-295X.87.3.215

CrossRef Full Text | Google Scholar

European Commission Initiative on Breast Cancer (2019). Recommendations for the European Breast Cancer Guidelines. Available at: https://ecibc.jrc.ec.europa.eu/recommendations [Accessed May 30, 2021].

Google Scholar

European Commission. (2020). Internal Market, Industry, Entrepreneurship and SMEs: Regional Innovation Monitor Plus. Available at: https://ec.europa.eu/growth/tools-databases/regional-innovation-monitor/base-profile/lombardy [Accessed July 3, 2021].

Google Scholar

Festinger, L. (1951). A Theory of Cognitive Dissonance. Stanford: Stanford University Press.

Garner, M., Ning, Z., and Francis, J. (2012). A Framework for the Evaluation of Patient Information Leaflets. Health Expect. 15, 283–294. doi:10.1111/j.1369-7625.2011.00665.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Giordano, L., Rowinski, M., Gaudenzi, G., and Segnan, N. (2005). What Information Do Breast Cancer Screening Programmes Provide to Italian Women? Eur. J. Public Health 15, 66–69. doi:10.1093/eurpub/cki117

CrossRef Full Text | Google Scholar

Guest, G., Bunce, A., and Johnson, L. (2006). How Many Interviews Are Enough? Field Methods 18, 59–82. doi:10.1177/1525822X05279903

CrossRef Full Text | Google Scholar

Helitzer, D., Hollis, C., Cotner, J., and Oestreicher, N. (2009). Health Literacy Demands of Written Health Information Materials: An Assessment of Cervical Cancer Prevention Materials. Cancer Control 16, 70–78. doi:10.1177/107327480901600111

PubMed Abstract | CrossRef Full Text | Google Scholar

Henriksen, M. J. V., Guassora, A. D., and Brodersen, J. (2015). Preconceptions Influence Women's Perceptions of Information on Breast Cancer Screening: a Qualitative Study. BMC Res. Notes 8, 404. doi:10.1186/s13104-015-1327-1

PubMed Abstract | CrossRef Full Text | Google Scholar

Hersch, J., Jansen, J., Barratt, A., Irwig, L., Houssami, N., Howard, K., et al. (2013). Women's Views on Overdiagnosis in Breast Cancer Screening: a Qualitative Study. BMJ 346, f158. doi:10.1136/bmj.f158

PubMed Abstract | CrossRef Full Text | Google Scholar

Independent UK Panel on Breast Cancer Screening (2012). The Benefits and Harms of Breast Cancer Screening: an Independent Review. Lancet 380,1778–1786. doi:10.1016/S0140-6736(12)61611-0

PubMed Abstract | CrossRef Full Text | Google Scholar

Italian Group for Cervical Cancer Screening (2009). Writing about Screening, Available at: https://www.osservatorionazionalescreening.it/content/scrivere-di-screening [Accessed May 30, 2021].

Google Scholar

Ivlev, I., Hickman, E. N., McDonagh, M. S., and Eden, K. B. (2017). Use of Patient Decision Aids Increased Younger Women's Reluctance to Begin Screening Mammography: a Systematic Review and Meta-Analysis. J. Gen. Intern. Med. 32, 803–812. doi:10.1007/s11606-017-4027-9

CrossRef Full Text | Google Scholar

Jørgensen, K. J., and Gøtzsche, P. C. (2006). Content of Invitations for Publicly Funded Screening Mammography. BMJ 332, 538–541. doi:10.1136/bmj.332.7540.538

PubMed Abstract | CrossRef Full Text | Google Scholar

Jorgensen, K. J., and Gøtzsche, P. C. (2004). Presentation on Websites of Possible Benefits and Harms from Screening for Breast Cancer: Cross Sectional Study. BMJ 328, 148. doi:10.1136/bmj.328.7432.148

PubMed Abstract | CrossRef Full Text | Google Scholar

Krippendorff, K. (2004). Reliability in Content Analysis. Hum. Comm Res 30, 411–433. doi:10.1111/j.1468-2958.2004.tb00738.x

CrossRef Full Text | Google Scholar

Lelo, K., Monni, S., and Tomassi, F. (2018). Urban Inequalities in Italy: a Comparison between Rome, Milan and Naples. Jesi 6, 939–957. doi:10.9770/jesi.2018.6.2(31doi:10.9770/jesi.2018.6.2(31)

CrossRef Full Text | Google Scholar

Lucisano, P., and Piemontese, M. E. (1988). “GULPease: a Formula to Predict Readability of Texts Written in Italian Language”, in Scuola e Città 3, 56–68.

Google Scholar

Martínez-Alonso, M., Carles-Lavila, M., Pérez-Lacasta, M. J., Pons-Rodríguez, A., Garcia, M., Rué, M., et al. (2017). Assessment of the Effects of Decision Aids about Breast Cancer Screening: a Systematic Review and Meta-Analysis. BMJ Open 7, e016894. doi:10.1136/bmjopen-2017-016894

CrossRef Full Text | Google Scholar

Morris, N. S., MacLean, C. D., Chew, L. D., and Littenberg, B. (2006). The Single Item Literacy Screener: Evaluation of a Brief Instrument to Identify Limited reading Ability. BMC Fam. Pract. 7, 21. doi:10.1186/1471-2296-7-21

PubMed Abstract | CrossRef Full Text | Google Scholar

National Research Council of Italy (2011). Lab for Computational Models of the Dynamics of Language and Cognition. Available at: http://www.ilc.cnr.it/dylanlab/apps/texttools [Accessed July 1, 2020].

Google Scholar

Ploug, T., Holm, S., and Brodersen, J. (2012). To Nudge or Not to Nudge: Cancer Screening Programmes and the Limits of Libertarian Paternalism. J. Epidemiol. Community Health 66, 1193–1196. doi:10.1136/jech-2012-201194

CrossRef Full Text | Google Scholar

Rickham, P. P. (1964). Human Experimentation. Code of Ethics of the World Medical Association. Declaration of Helsinki. Br. Med. J. 2, 177. doi:10.1136/bmj.2.5402.177

PubMed Abstract | CrossRef Full Text | Google Scholar

Smith, S. G., Vart, G., Wolf, M. S., Obichere, A., Baker, H. J., Raine, R., et al. (2015). How Do People Interpret Information about Colorectal Cancer Screening: Observations from a Think‐aloud Study. Health Expect. 18, 703–714. doi:10.1111/hex.12117

PubMed Abstract | CrossRef Full Text | Google Scholar

US Department of Health and Human ServicesCenters for Disease Control and Prevention. (2009). Simply Put – A Guide for Creating Easy-To-Understand Materials. 3rd ed. Atlanta, GA: Centers for Disease Control and Prevention.

Waller, J., Douglas, E., Whitaker, K. L., and Wardle, J. (2013). Women's Responses to Information about Overdiagnosis in the UK Breast Cancer Screening Programme: a Qualitative Study: Table 1. BMJ Open 3, e002703. doi:10.1136/bmjopen-2013-002703

PubMed Abstract | CrossRef Full Text | Google Scholar