Understandability of passive smoking prevention leaflets in Japan

Purpose: There are few studies in Japan evaluating informational materials about passive smoking prevention with respect to health literacy. This study applied a range of health literacy assessment tools to assess the understandability of public-sector leaflets about passive smoking prevention.

Methods: We collected 26 anti-passive smoking leaflets published online by Japanese prefectural governments. We used three internationally recognized health literacy assessment tools to evaluate their understandability: the Clear Communication Index (CCI), the Suitability Assessment of Materials (SAM), and the Patient Education Materials Assessment Tool (PEMAT, further divided into scales for understandability and actionability).

Results: Although none of the assessed materials met the CCI's scoring threshold for “easy to understand,” the highest-scoring leaflet, which received a top-two ranking in three of the four scales (excluding the PEMAT understandability scale), was characterized by its conciseness and its easy-to-understand structure, including the use of checklists and flowcharts to engage reader interest. There was a significant correlation between CCI and SAM, and between SAM and the PEMAT understandability scale. However, the PEMAT actionability scale did not significantly associate with the other assessment scales.

Conclusions: The understandability of written informational materials about passive smoking prevention in Japan could be improved by referring to multiple assessment scales.

Introduction

Health literacy has been defined as the ability of people to “meet the complex demands of health in modern society” (Sørensen et al., 2012). Health literacy also refers to the ability of health professionals to narrow the gap between the understandability of health information and people's capacity to use that information (Rudd and Parnell, 2022). This concept is crucial in the prevention or management of lifestyle-related diseases (Aida et al., 2020), chronic illness (Paul et al., 2022), infectious diseases (Castro-Sánchez et al., 2016), and smoking (Fonseca et al., 2022).

Among these health promotion efforts, smoking prevention has been particularly emphasized, with the WHO establishing global guidelines such as the FCTC (Framework Convention on Tobacco Control) and MPOWER (a set of six effective tobacco control measures). These initiatives were part of worldwide efforts to curb tobacco use. Regarding information for the purpose of smoking prevention and cessation, previous research indicated that messages based on logic were effective for smokers with higher health literacy, while messages based on emotion were more effective for smokers with lower health literacy (Hoover et al., 2018). Health literacy plays a significant role in influencing how smokers respond to different risk messaging styles.

Japan's smoking rate of 16.7% (Ministry of Health Labour and Welfare, 2020) was slightly lower than the global smoking rate of 17% in 2020 (Vital Strategies, 2023). Additionally, smoking rates (National Cancer Center, 2019) in six regions of Japan showed regional differences: Tohoku/Hokkaido at 21.5%, Kanto at 18.0%, Chubu at 18.0%, Kinki at 17.3%, Chugoku/Shikoku at 17.0%, and Kyushu at 19.3%. However, Japan has made minimal use of warning messages on tobacco packaging compared with international standards and was relatively slow to implement passive smoking measures. On April 1, 2020, in anticipation of the 2020 Tokyo Olympics, the Health Promotion Act was extensively amended to bring Japan's passive smoking measures in line with global standards. The amendments primarily prohibited smoking indoors in places such as restaurants and mandated the display of signage for smoking areas. In response, local governments took on the responsibility of informing residents, through various media, about the content of the amendments and the health hazards of passive smoking. Given that Japan is considered to have low health literacy compared with other countries globally (Nakayama et al., 2015), developing and distributing these communications was a challenging task.

In Japan, there have been various efforts to improve health literacy around smoking and passive smoking. These include local governments producing informational materials and creating posters that account for regional characteristics. In addition, Kostagiolas et al. (2023) noted that information obtained from the internet and from family, relatives, friends, and colleagues was more effective for quitting smoking than advice from medical staff, healthcare professionals, and primary healthcare services. Smoking prohibition in both public and private workplaces was effective in reducing passive smoking at workplaces in Europe (Olivieri et al., 2019). The effectiveness of implementing such regulations depends on how well the public is informed. Despite the clear importance of widespread information provision, there have been only limited studies in Japan evaluating passive-smoking-related posters and leaflets with respect to health literacy.

The present study used a range of Japanese-language health literacy assessment tools to evaluate the understandability (including clarity, suitability, and ease of understanding for the intended audience) of passive smoking prevention-related informational materials (hereafter “leaflets”) published by prefectural governments across Japan.

Methods

Health literacy leaflets can inform more people about the dangers of passive smoking. We conducted a descriptive analysis of written materials issued by local governments in Japan, specifically, to evaluate the understandability of leaflets related to passive smoking prevention. As this study did not involve human research subjects, institutional authorities determined that ethics approval was not required.

Study materials

To obtain materials for the study, we used Google's search engine to find relevant leaflets relating to passive smoking available on the official website of each prefectural government and downloaded the files in PDF format. The validity of this approach is based on each prefecture's obligation to work on the prevention of passive smoking and to use digital tools in all aspects of governance, in accord with national directives. For publication dates, we recorded the date that each leaflet was last updated. Japan's 47 prefectures are generally grouped into six regions: Tohoku/Hokkaido, Kanto, Chubu, Kinki, Chugoku/Shikoku, and Kyushu; we used these six regional names as semi-anonymized identifiers for the origin of each leaflet to avoid rating individual prefectures. The intended audience of each leaflet was also evaluated based on the leaflet's vocabulary and style.

The inclusion criteria for leaflets were that they were original and had been created and published by the relevant prefectural government. In cases where the prefecture had created multiple leaflets on the topic, we used the version posted highest on its web page. We included materials composed of a single sheet of paper as well as booklets composed of multiple pages. We excluded reproductions of a national leaflet on preventing passive smoking, prepared by the Japanese Ministry of Health, Labor and Welfare, because these identical leaflets were not created independently by any prefecture.

Assessment tools

The leaflets were assessed using the Japanese versions of three internationally recognized assessment tools: the Clear Communication Index (CCI; Baur and Prue, 2014), Suitability Assessment of Materials (SAM; Noro, 2009), and Patient Education Materials Assessment Tool (PEMAT; Furukawa et al., 2022).

Clear Communication Index

The CCI is a measure of effective public communication published by the U.S. Centers for Disease Control and Prevention and consists of four sections and 19 items. This index is characterized by its evaluation of the behavioral recommendations, numerical values, and risks presented in the informational material being assessed. Scores are expressed as percentages, with materials scoring 90% or more rated as “easy-to-understand materials” (Baur and Prue, 2014). In this study, we used the Japanese version of the CCI (Goto et al., 2018), but also referred to the original version to avoid errors of interpretation.

Suitability Assessment of Materials

The SAM is characterized by its detailed rating of document layout and consideration of readers' emotional reactions (Noro, 2009). It contains 23 items in four sections, with each item evaluated on a three-point scale of 0, 1, or 2 points. Materials achieving a total score of 70–100% are rated as “superior,” 40–69% as “adequate,” and 0–39% as “not suitable” (Nomura et al., 2021). In this study, we used the Japanese version (Nakazato and Noro, 2006), while also referring to the original version. This was because some items were deemed to be mistranslations, with questions that were natural in the original English text being unnatural when rendered into Japanese. The print quality section of the layout assessment was excluded because we obtained the leaflets in digital format. Two questions, about passive vs. active tense and mentioning content before the main message, were inapplicable because Japanese grammatical rules and paragraph structure differ from those of English. Japanese sentences frequently use passive tense to add politeness to instructions and requests. In addition, Japanese writing style routinely places topic sentences at the end of their explanatory paragraphs. So, on these points, we assigned a score of 2 for all leaflets, because their style was consistent with what Japanese readers would expect.

Patient Education Materials Assessment Tool

The PEMAT is a tool developed by the Agency for Healthcare Research and Quality within the U.S. Department of Health and Human Services. It is characterized by its assessment of the extent to which the material prompts action from the reader. There are two variants: PEMAT-P, which is a measure of printable materials, and PEMAT-A/V, which is a measure of audiovisual materials. We used the Japanese version of PEMAT-P (Furukawa et al., 2022), consisting of 23 items in two separate scales—“understandable” (PEMAT-U) and “actionable” (PEMAT-A)—while also referring to the original version for the sake of fidelity to its intention.

Evaluation process

First, leaflets were independently evaluated by the first and second authors, who are students in an MD-PhD curriculum mentored by a specialist in health literacy. They used all three assessment tools as instructed by their mentor. The leaflets were printed in color on A4 paper to ensure consistency across the process. The results of the evaluation were recorded in an Excel table with a drop-down system to unify the input and prevent typographical errors. Second, both evaluators double-checked the data for any mistakes or omissions, and their respective results tables were combined to identify discrepancies. Lastly, two researchers (third and last authors, one of whom specializes in health literacy research) joined the process to examine discrepancies and determine a unified score where required. The evaluation scores made by each evaluator before discussion and finalization are shown in Supplementary Table 1. The consistency (percentage with the same score) between the two evaluators was calculated as 77% for CCI, 64% for SAM, and 78% for PEMAT (79% for PEMAT-U and 75% for PEMAT-A).

Statistical analysis

The characteristics and scores determined by the three tools were summarized descriptively for each leaflet. The scores were ranked, and key features of the top-ranked leaflets were identified. In addition, correlation among the three tools (or four scales, including PEMAT-U and PEMAT-A) was examined using Spearman's rank correlation coefficient. A p-value of 0.05 or less was considered statistically significant.

Results

Of the 47 prefectures, we identified 26 with leaflets that met the inclusion criteria, all of which were included in the analysis. Leaflets that were identified as targeting a general audience were mostly aimed at both smokers and non-smokers, with the intention of raising awareness of the side effects of tobacco on smokers and those around them. Leaflets targeting business managers and companies mainly sought to explain that the laws around passive smoking had changed because of April 2020 revisions to the Health Promotion Act. In addition, two leaflets were intended for children, and one was intended for pregnant women. In the leaflets for children, the main aims were to raise awareness on how to avoid passive smoking and how not to become a new smoker.

Table 1 presents the basic characteristics for each leaflet and their scores determined by each of the three assessment tools. The number of prefectures with leaflets that met the inclusion criteria was three out of seven in Tohoku/Hokkaido, six out of seven in Kanto, five out of nine in Chubu, three out of seven in the Kinki region, six out of nine in Chugoku/Shikoku, and three out of eight in Kyushu. The largest number of leaflets were intended for a general audience (14 leaflets, 53.8%), followed by business managers (nine leaflets, 34.6%). In the CCI assessment, none of the leaflets scored more than the 90% threshold for “easy-to-understand materials.” However, in the SAM evaluation, 15 leaflets met the criteria for “superior materials.”

Table 1

Table 1. Basic characteristics of leaflets and scores for each assessment scale.

Figure 1 shows the distribution of scores for each assessment scale as a box plot. The range of scores on the CCI was small compared with that of the other scales. Comparing the median scores, the PEMAT-U scale was the highest.

Figure 1

Figure 1. Comparison of the score ranges for each assessment scale.

Table 2 presents a comparison of the key features of the leaflets that achieved the two highest scores for each assessment tool. The Kyushu 3 leaflet, which received a top-two ranking in three of the four scales (excluding PEMAT-U), was characterized by its concise presentation of information and its easy-to-understand structure, including checklists and flowcharts to engage reader interest. In addition, the Chugoku/Shikoku 2 leaflet, which received a top-two ranking for CCI and SAM, presented its main message in a “Three things to do to protect yourself” format and strongly encouraged actions from readers. The Chubu 2 leaflet, which received high marks for SAM and PEMAT-U, used flowcharts to aid understanding. In contrast, the Tohoku/Hokkaido 1 leaflet, which received a bottom-two ranking in three of the four scales (excluding CCI), was deemed hard to understand because it had no pictures or illustrations and used many complicated words, including those with difficult kanji characters not designated for everyday use by the Japanese Ministry of Education.

Table 2

Table 2. Comparison of features of top-ranked leaflets by assessment scale.

Table 3 shows the degree of agreement between each assessment scale, calculated using Spearman's rank correlation coefficient. There was a significant correlation between CCI and SAM (it was 0.395), and between SAM and PEMAT-U (it was 0.576). PEMAT-A did not significantly associate with the other assessment scales.

Table 3

Table 3. Spearman's rank correlation coefficients between assessment scales.

Discussion

None of the leaflets we assessed met the 90% scoring threshold for “easy-to-understand materials” in the CCI, and only 15 met the 70% scoring threshold for “superior” in the SAM. The highest-ranked leaflets were concise, structured with checklist and flowcharts, and provided clear main messages that prompted action from readers. The leaflets could be improved from the perspective of health literacy by including a summary and using interactions (such as problems, questions, Q&A, and checklists) to engage reader interest.

Methodologically, the scores of the CCI and PEMAT were the most consistent between the two evaluators. A previous study also found that both PEMAT-U and PEMAT-A demonstrated good inter-rater reliability (Vishnevetsky et al., 2018). Both measurements are formulated with concise questions with simple scoring. The SAM evaluation items use relatively lengthy phrasing, and each score has a different statement. Nevertheless, it has unique items (e.g., emotional considerations) that are not included in the other assessment tools. It was noteworthy that there were significant correlations between CCI and SAM, and between SAM and PEMAT-U, but not between CCI and PEMAT-U. The reason for the relatively poor correlation between PEMAT-A and the other scales might be its specific focus on readers' actions and the small number of evaluation items. As previously reported, each tool evaluates different aspects of written information, and multifaceted evaluation using multiple measurements is recommended (Nomura et al., 2021).

Some limitations should be noted. First, only a small number of leaflets (26) were reviewed, and they were evaluated by trained students rather than experienced evaluators. In future research, a larger number of materials should be assessed, and the evaluators should continue to undergo training from health literacy experts to improve their skills. Second, the Japanese versions of the assessment scales included instances of unclear translations. We therefore needed to make frequent reference to the original English versions during the evaluation process because if the understanding of rating scales differs among raters, there could be large differences in scores, which would reduce objectivity.

Conclusions

Our assessment of a sample of 26 leaflets on passive smoking prevention issued by local governments in Japan found that most of these were not fully understandable or easy to use. These leaflets could be improved by including a summary of the main points and incorporating techniques to invite reader interaction, such as through flowcharts and checklists, as were found in some of the highest-scoring leaflets. Wider implementation of studies like ours could encourage local governments to integrate routine health literacy assessments into their development and publication processes for informational leaflets and other health communications.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Author contributions

TI: Writing – original draft, Formal analysis, Investigation, Methodology. YI: Writing – original draft, Formal analysis, Investigation, Methodology. YK: Writing – original draft, Writing – review & editing, Formal analysis. KN: Writing – review & editing, Supervision. HY: Writing – review & editing. AG: Writing – review & editing, Conceptualization, Funding acquisition, Methodology, Supervision.

Funding

The author(s) declare financial support was received for the research, authorship, and/or publication of this article. This research was funded by the Japan Society for the Promotion of Science Grants in Aid for Scientific Research (No. 20K10539, PI: HY).

Acknowledgments

We thank Sanae Sato and staff at Fukushima Medical University for implementing the study. We thank Oliver Stanyon for editing early drafts of the manuscript.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The reviewer TN declared a past co-authorship with the authors HY and AG to the handling editor.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fcomm.2024.1417494/full#supplementary-material

References

Aida, A., Svensson, T., Svensson, A. K., Chung, U. I., and Yamauchi, T. (2020). eHealth delivery of educational content using selected visual methods to improve health literacy on lifestyle-related diseases: literature review. JMIR Mhealth Uhealth 8:e18316. doi: 10.2196/18316

PubMed Abstract | Crossref Full Text | Google Scholar

Baur, C., and Prue, C. (2014). The CDC Clear Communication Index is a new evidence-based tool to prepare and review health information. Health Promot. Pract. 15, 629–637. doi: 10.1177/1524839914538969

PubMed Abstract | Crossref Full Text | Google Scholar

Castro-Sánchez, E., Chang, P. W. S., Vila-Candel, R., Escobedo, A. A., and Holmes, A. H. (2016). Health literacy and infectious diseases: why does it matter? Int. J. Infect. Dis. 43, 103–110. doi: 10.1016/j.ijid.2015.12.019

PubMed Abstract | Crossref Full Text | Google Scholar

Fonseca, A. D. G., Silva Junior, R. F. D., Bauman, C. D., Silva, L. L., Silva, C. S. O. E., and Martins, A. M. E. B.L. (2022). Validity and reliability of the smoking-related health literacy assessment scale. Einstein 20:eAO5156. doi: 10.31744/einstein_journal/2022AO5156

PubMed Abstract | Crossref Full Text | Google Scholar

Furukawa, E., Okuhara, T., Okada, H., Shirabe, R., Yokota, R., Iye, R., et al. (2022). Cross-cultural adaptation, and validation of the Japanese version of the patient education materials assessment tool (PEMAT). Int. J. Environ. Res. Publ. Health 19:15763. doi: 10.3390/ijerph192315763

PubMed Abstract | Crossref Full Text | Google Scholar

Goto, A., Lai, A. Y., Kumagai, A., Koizumi, S., Yoshida, K., Yamawaki, K., et al. (2018). Collaborative processes of developing a health literacy toolkit: a case from Fukushima after the nuclear accident. J. Health Commun. 23, 200–206. doi: 10.1080/10810730.2018.1423650

PubMed Abstract | Crossref Full Text | Google Scholar

Hoover, D. S., Wetter, D. W., Vidrine, D. J., Nguyen, N., Frank, S. G., Li, Y., et al. (2018). Enhancing smoking risk communications: the influence of health literacy and message content. Ann. Behav. Med. 52, 204–215. doi: 10.1093/abm/kax042

PubMed Abstract | Crossref Full Text | Google Scholar

Kostagiolas, P., Parnavela, S., and Theodorou, P. (2023). The impact of smokers' information-seeking behavior on smoking cessation. Adv. Exp. Med. Biol. 1425, 645–662. doi: 10.1007/978-3-031-31986-0_63

PubMed Abstract | Crossref Full Text | Google Scholar

Ministry of Health Labour and Welfare (2020). The National Health and Nutrition Survey in Japan, 2019. Available at: https://www.mhlw.go.jp/content/001066903.pdf (accessed December 1, 2023).

Google Scholar

Nakayama, K., Osaka W Togari, T., Ishikawa, H., Yonekura, Y., Sekido, A., et al. (2015). Comprehensive health literacy in Japan is lower than in Europe: a validated Japanese-language assessment of health literacy. BMC Publ. Health 15:505. doi: 10.1186/s12889-015-1835-x

PubMed Abstract | Crossref Full Text | Google Scholar

Nakazato, Y., and Noro, I. (2006). Applying SAM's readability component to the Japanese language. Hijiyama Daigaku Gendai Bunka Gakubu Kiyo 13, 79–83.

Google Scholar

National Cancer Center (2019). Cancer Registry and Statistics, 2019. Cancer Information Service. Available at: https://ganjoho.jp/reg_stat/statistics/data/dl/index.html (accessed Jun 28, 2024).

PubMed Abstract | Google Scholar

Nomura, M., Goto, A., Osawa, E., and Miura, H. (2021). Empirical evaluation of understandability and usability of health handbooks commonly used in Japan. Front. Commun. 6:728934. doi: 10.3389/fcomm.2021.728934

Crossref Full Text | Google Scholar

Noro, I. (2009). Study on the Appropriateness of Information Materials for Patienents Focusing on Understandability and Secure Feeling of Informed Consent Document (In Japanese). Japan: Tohoku University.

Google Scholar

Olivieri, M., Murgia, N., Carsin, A. E., Heinrich, J., Benke, G., Bono, R., et al. (2019). Effects of smoking bans on passive smoking exposure at work and at home. Eur. Commun. Respirat. Health Survey 29, 670–679. doi: 10.1111/ina.12556

PubMed Abstract | Crossref Full Text | Google Scholar

Paul, B., Kirubakaran, R., Isaac, R., Dozier, M., Grant, L., Weller, D., et al. (2022). Theory of planned behavior-based interventions in chronic diseases among low health-literacy population: protocol for a systematic review. Syst. Rev. 11:127. doi: 10.1186/s13643-022-02006-2

PubMed Abstract | Crossref Full Text | Google Scholar

Rudd, R. E., and Parnell, T. A. (2022). Health Literacy: Insights for Action. A Population Health Approach to Health Disparities for Nurses: Care of Vulnerable Populations. New York, NY: Springer Publishing.

Google Scholar

Sørensen, K., Van den Broucke, S., Fullam, J., Doyle, G., Pelikan, J., Slonska, Z., et al. (2012). Health literacy and public health: a systematic review and integration of definitions and models. BMC Publ. Health 12:80. doi: 10.1186/1471-2458-12-80

PubMed Abstract | Crossref Full Text | Google Scholar

Vishnevetsky, J., Walters, C. B., and Tan, K. S. (2018). Interrater reliability of the Patient Education Materials Assessment Tool (PEMAT). Patient Educ. Couns. 101, 490–496. doi: 10.1016/j.pec.2017.09.003

PubMed Abstract | Crossref Full Text | Google Scholar

Vital Strategies (2023). The Tobacco Atlas. Availabe at: https://tobaccoatlas.org (accessed December 1, 2023).

Google Scholar