Li Wang1
Yongjin Li- 1College of Humanities, Huzhou University, Huzhou, Zhejiang, China
- 2College of Life Sciences, Huzhou University, Huzhou, Zhejiang, China
Background: Thyroid cancer (TC) is becoming an increasing public health problem worldwide. TikTok is a global video-sharing social media app, which can be a source of information on TC. However, the information quality of these videos remains unknown.
Methods: A search of TikTok was performed with the term “甲状腺癌”and “甲状腺肿瘤” (“thyroid cancer” and “thyroid neoplasm” in Chinese). Videos included were independently assessed using six predefined questions for content scores and DISCERN scale for information quality, while the video popularity index (VPI) was also calculated. A correlation analysis was performed among duration, presence of animation, VPI, DISCERN scores, and content scores.
Results: A total of 56 videos were finally included, of which 49 were uploaded by physicians, 4 by health organizations, and 3 by hospitals. While 43 were real content videos, 13 were animated. The mean of DISCERN score and content score was 3.44 ± 0.72 and 5.19 ± 0.95, respectively. Good consistency was found between the two reviewers in terms of both DISCERN scores and content scores. The video duration and presence of animation were positively correlated with VPI, while DISCERN scores and content scores were not correlated with VPI.
Conclusion: The overall quality of TC-related videos was satisfactory, although the quality varies greatly depending on the type of source. Patients should be cautious when using TikTok as a source of TC-related information.
Introduction
Thyroid cancer is the most common malignancy of the endocrine system, with the differentiated thyroid cancers papillary and follicular accounting for the majority of new cases (1). Furthermore, TC is more likely to occur in women with an incidence of 2.5 times (2), and the five-year survival rate is 98.1% (3). An apparent increase in TC incidence was shown in recent studies (4, 5), which may be driven by a combination of increased surveillance, lifestyle, and environmental factors (6–8).
Due to an increase in health awareness, people have a greater demand for health information, especially for patients suffering from cancer. Patients are increasingly demanding to be involved in decision-making process (9). Health information could be provided by diverse sources besides professional physicians. Although such information is not a substitute for medical advice, it could reinforce knowledge and help patients to make personal decisions.
Social media is an important way to disseminate information. However, social media is also a major source of misinformation, especially health relevant information. Misinformation tends to spread faster and more broadly than verified information due to the diversity of users and lack of monitoring and censorship mechanisms. Several studies have shown the potential for social media to be misleading in disseminating health information (10–16). TikTok is a short video-sharing app that allows users to create and share videos on any topic. In China, TikTok reached the first position on the global mobile app download list in December 2021 (17). Compared with other social media, TikTok has become more popular because it is easy to produce and share short videos, including elements like music, animation, and various visual effects (18). TikTok is expanding its coverage of topics on many aspects of daily life, such as beauty and makeup, education, cooking, wellness and technology (19). Educational healthcare content has also become an important part of TikTok’s content ecosystem (20). However, the unregulated content of TikTok video and the lack of peer review process increases the likelihood of dissemination of inaccurate and large volumes of information with varying quality and credibility, raising a significant challenge in the provision of optimal healthcare. Given the increasing use of TikTok app by increasingly patients, physicians, and researchers as an effective channel for healthcare communication in recent years (21, 22), it thus is vital for patients and health care providers to understand the quality of information that patients are likely to find, and for health educators to make higher quality short video available.
Although TC is a disease with a good prognosis, receiving a cancer diagnosis is a sensational event for the general person. Correct dissemination of TC-related health information plays an important role in enhancing patient belief and maintaining good patient–physician relationships. However, to our knowledge, the quality of short video content regarding to TC has not yet been analyzed. To address this gap, this study aims to assess the quality of the short videos about TC from TikTok.
Methods
Search strategy
Keywords “甲状腺癌” and “甲状腺肿瘤” (“Thyroid cancer” and “thyroid neoplasm” in Chinese) was searched within TikTok app on March 20, 2022. The first 100 videos that appeared on each search were reviewed. A total of 56 videos were finally included for analyzing in this study after excluding videos with advertising content (n = 8), duplicated (n = 36), as shown in the Supplementary Figure S1.
Data collection
All selected videos were categorized into three groups according to source: physicians, hospital channel (non-profit organization), and health organizations (for-profit organizations). In order to confirm the authenticity of the included videos, we not only carefully checked the registration information of the video uploader’s account, but also accessed the official website of the hospital where the video author worked to confirm the author’s real identity. Information of videos was extracted and coded, including source of video, the content of the videos, the presence of animation, duration (in seconds), the upload date, and other viewer interactive quality markers including number of views, likes, and comments. The VPI was calculated using the formula “(number of likes/number of views) × 100)” (12).
Quality assessment of video information
Videos were assessed from two aspects: the quality of information and their contents. The quality of information was rated using an adapted DISCERN tool by Singh et al. (23) considering the following reasons: (1) It is a widely used tool for evaluating the quality of health information (24, 25); (2) It is useful for assessing information quality on other video-based platforms (e.g., YouTube) (26). This DISCERN tool has five questions in total, and each question is answered yes or no. Yes answer is 1 point and no answer is 0, and a maximum of 5 points can be obtained. DISCERN scores thus ranged from 1 = unacceptable, 2 = poor, 3 = acceptable, 4 = good, to 5 = excellent, higher scores indicate a greater video reliability (25). Questions used in the DISCERN scale are shown in Table 1.
Table 1. Assessment of reliability of videos on TC found on TikTok APP.
Content score was rated in terms of six predefined questions (definition of a disease, risk factors, evaluation, signs and symptoms, management, and outcomes; one point is given for covering 1–2 aspects, two points is for 3 aspects, three points is for 4 aspects, four points is for 5 aspects and five points is for full coverage. The rating criteria are the same as DISCERN scores). To facilitate statistical analysis, we recorded the basic information of each video (publication date; duration; views, likes, comments, shares; presence of animation or not) and basic information of the video publisher (account name, self-description, publisher identity). Two reviewers independently appraised the quality of the included videos.
Ethics statement
This study focused on the quality assessment of TikTok videos contributed and viewed by the public, so ethics committee approval was not required.
Statistical analysis
Statistical analysis was made using IBM SPSS Statistics for Windows, version 19.0. The statistical analysis methods used include descriptive statistical analysis, Pearson correlation analysis, and one-way ANOVA. The continuous variables were expressed as mean ± standard deviation, median (min-max), while nominal variables were given as frequency and percentage. p < 0.05 was considered statistically significant.
Results
Video characteristics
Of the 56 videos included in this study, 49 (87.5%) were uploaded by physicians, 4 (7.1%) by health organizations, and 3 (5.4%) by hospital channels. There wasn’t video from patients or their relatives. Forty three (76.8%) videos were real content videos, 13 (23.2%) were videos with animation.
Descriptive statistics for videos
The duration of the videos varied from 5 to 111 s. The least online days of video were 2 days prior to data collection, whereas the most online days were nearly 3 years. Median (min-max) number of views, likes, comments, collection, sharing, and VPI were 46,211 (578–11,088,000), 1378.5 (22–308,000), 83 (0–73,000), 82.5 (1–1775), 130.5 (15–5,796) and 2.5 (2–8.82) prior to data collection. There were no significant differences among different sources of video (physician, hospital channel, and health organization) regarding such video characteristics mentioned above except for VPI between health organization and hospital channel (p = 0.002), physicians (p < 0.001).
Quality assessment of video
The average DISCERN score given by the two reviewers was 3.44 ± 0.72, while the DISCERN score for individual reviewers was 3.5 ± 0.71 and 3.38 ± 0.73, respectively. A good agreement was observed, κ = 0.72 (p < 0.001). Regarding the source of videos, the videos contributed by hospital (4.0 ± 0.12) and physicians (3.4 ± 0.7) had higher DISCERN scores than the one by the health organization (2.3 ± 0.5). Significant quality difference was observed between health organizations and hospitals (p = 0.014), physicians (p = 0.006). More than 85% of videos have a DISCERN score greater than 3 (Table 2). In addition, the video content responded the predefined six questions to different degrees, as shown in Figure 1. The average content score given by the two reviewers was 5.19 ± 0.95, while for individual reviewers was 5.21 ± 0.91 and 5.16 ± 0.99, respectively. The κ-coefficient for content scores was 0.781 (p < 0.001), which indicated a good consistency between the two reviewers. The results also showed that most of videos sufficiently addressed definition (53/56, 94.6%), signs and symptoms (50/56, 89.3%), management (52/56, 92.9%), evaluation (51/56, 91.1%) and outcomes (54/56, 96.4%), while risk factors (32/56, 57.1%) were absent in 42.9% videos.
Table 2. General characteristics of the videos.
Figure 1. Percentage of videos addressing each TC topic.
The top 10 videos, the source of videos, the video content, the upload date, and VPI values are given in Table 3.
Table 3. General characteristics and VPI values of the top 10 videos.
Correlations between descriptive parameters
The duration of the video and presence of animation were positively correlated with VPI of videos (duration: r = 0.40, p = 0.002; animation: r = 0.52, p < 0.001). Both DISCERN score and content score were not correlated with VPI, and no correlation was noted between DISCERN score and content score, as shown in Figure 2.
Figure 2. Correlations considering animation, duration, VPI, DISCERN scores, and content scores. **p < 0.01.
Discussion
TikTok as information source
Video-based social media were important platforms for producing and disseminating health-related videos. Recent evidence indicates that TikTok has demonstrated strong communication potential during the COVID-19 pandemic (27), however, the role of TikTok in disseminating TC relevant information remains unclear.
Our results suggest that the overall quality of TC-related videos from TikTok was satisfactory. Videos included in this study have received approximately 16.74 million views since they were uploaded. These results indicated that TikTok is a new source of information on TC.
Information quality appraisal
TikTok app is primarily an entertainment-oriented application which may differentiate itself from other social media by publishing quirky videos rather than serious professional content (18). As the number of users grows, some professional knowledge sharing videos, such as medical education, have also been integrated into TikTok’s content ecosystem (21, 28). Our results here indicated that the quality of video on TC found on TikTok was satisfactory. These results were consistent with some previous studies about other disease videos found on TikTok (29–31), which found that the information provided in these videos was generally reliable.
Our results also show that the videos included in this study all covered predefined relevant questions to varying degrees, which may be due to the duration limit of TikTok videos (the duration of videos for new users is limited to 15 s, and longer videos require a certain number of fans and views) (32). The most frequently mentioned topic was the outcomes of TC, which may be due to the fact that this disease is the nature of cancer and is not generally known by the general population to have a low rate of malignancy. It is clearly helpful for patients to improve their risk perception of the prognosis of the disease by using TikTok to communicate such information to them.
TikTok app has a strict audit system for video publishers of health knowledge sharing (33). Therefore, there are no videos uploaded by patients or their relatives in the first 100 videos we search each time. Of the included 56 videos, 49 (87.5%) were uploaded by physicians, 4 (7.1%) by health organizations, and 3 (5.4%) by hospital channels. Videos published by physicians and hospitals had the higher quality which have been confirmed by many previous studies, while those from the health organizations for profit had the lowest quality (34, 35). These results indicated that physicians and government-sponsored platforms are more likely to publish high-quality information than for-profit organizations. However, the videos contributed by the hospital channel account for a mere portion of the total videos on TikTok. Considering the busy routine of physicians, we therefore suggest that public hospitals organize professionals to contribute more high-quality videos to patients and leverage the power of this social media channel to promote public health.
For videos found on TikTok, the video popularity attributes to many factors, such as the rules set by TikTok and common social interactions, even the psychological effects of the audience (36), which challenge to find videos from trustworthy sources. Therefore, since trustworthiness is not the only criterion for TikTok ranking, it is possible that videos from trusted sources will receive a lower ranking, while misleading and incorrect videos may receive a higher ranking. Notably, a new algorithm is needed to enable higher user acceptance and trusted videos remain in the top positions in order to help patients to access quality and trusted video resources.
Animated videos in previous study (10) were reported to be correlated positively with the number of views and negatively with the video duration. However, videos with animation had no advantage over non-animated videos in terms of number of views, DISCERN scores, or content scores in our study. This may be due to the small number of animated videos and the use of similar and repeated animation resources. In addition, a weakly positive correlation existed between duration, animation, and VPI. This may attribute to the type of video. Unlike entertainment videos, health knowledge sharing videos need longer duration to accommodate more information. TikTok has recently raised its video duration limit to 10 min (37), which is good for health-sharing videos.
Limitations
Limitations to this study should be mentioned. First, the search results on TikTok are dynamic over time; the results thus might vary with the use of different search dates and time. Second, the research results may vary according to the geographical location of the viewer. Third, video sampling retrieves only the first 100 in each search which can lead to inadequate coverage.
Conclusion
In this study, we analyzed the quality of information provided by TikTok video regarding TC. The overall quality of information based on DISCERN scores and content scores was satisfactory. Our finding also supports TikTok as a new source of information on TC, and therefore, physicians and hospitals should embrace this evolving technology and provide videos with higher quality to improve patients’ awareness about TC. Patients should also be cautious when watching TC relevant videos on TikTok due to the complexity of VPI.
Data availability statement
The original contributions presented in the study are included in the article/Supplementary material, further inquiries can be directed to the corresponding author.
Author contributions
LW and YL: Conceptualization and Methodology. JG and LX: Experiments and Data analysis. LW and YL: Writing. All authors contributed to the article and approved the submitted version.
Funding
This work was supported by Science and Technology Department of Zhejiang Province (LGC21H200001) and Huzhou Science and Technology Bureau (2020GZ29).
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Supplementary material
The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpubh.2023.1049728/full#supplementary-material
References
1. Brito, JP, Hay, ID, and Morris, JC. Low risk papillary thyroid cancer. BMJ. (2014) 348:g3045. doi: 10.1136/bmj.g3045
2. Enewold, L, Zhu, K, Ron, E, Marrogi, AJ, Stojadinovic, A, Peoples, GE, et al. Rising thyroid cancer incidence in the United States by demographic and tumor characteristics, 1980–2005. Cancer Epidemiol Biomark Prev. (2009) 18:784–91. doi: 10.1158/1055-9965.EPI-08-0960
3. Bibbins-Domingo, K, Grossman, DC, Curry, SJ, Barry, MJ, Davidson, KW, Doubeni, CA, et al. Screening for thyroid cancer: US preventive services task force recommendation statement. JAMA. (2017) 317:1882–7. doi: 10.1001/jama.2017.4011
4. Wang, J, Yu, F, Shang, Y, Ping, Z, and Liu, L. Thyroid cancer: incidence and mortality trends in China, 2005–2015. Endocrine. (2020) 68:163–73. doi: 10.1007/s12020-020-02207-6
5. Rosário, PW, Ward, LS, Carvalho, GA, Graf, H, Maciel, RM, Maciel, LM, et al. Thyroid nodules and differentiated thyroid cancer: update on the Brazilian consensus. Arq Bras Endocrinol Metabol. (2013) 57:240–64. doi: 10.1590/s0004-27302013000400002
6. Davies, L, and Welch, HG. Increasing incidence of thyroid cancer in the United States, 1973-2002. JAMA. (2006) 295:2164–7. doi: 10.1001/jama.295.18.2164
7. Pandeya, N, McLeod, DS, Balasubramaniam, K, Baade, PD, Youl, PH, Bain, CJ, et al. Increasing thyroid cancer incidence in Queensland, Australia 1982-2008 - true increase or overdiagnosis? Clin Endocrinol. (2016) 84:257–64. doi: 10.1111/cen.12724
8. Friedenreich, CM, Ryder-Burbidge, C, and McNeil, J. Physical activity, obesity and sedentary behavior in cancer etiology: epidemiologic evidence and biologic mechanisms. Mol Oncol. (2021) 15:790–800. doi: 10.1002/1878-0261.12772
9. Vahdat, S, Hamzehgardeshi, L, Hessam, S, and Hamzehgardeshi, Z. Patient involvement in health care decision making: a review. Iran Red Crescent Med J. (2014) 16:e12454–4. doi: 10.5812/ircmj.12454
10. Cassidy, JT, Fitzgerald, E, Cassidy, ES, Cleary, M, Byrne, DP, Devitt, BM, et al. YouTube provides poor information regarding anterior cruciate ligament injury and reconstruction. Knee Surg Sports Traumatol Arthrosc. (2018) 26:840–5. doi: 10.1007/s00167-017-4514-x
11. Goobie, GC, Guler, SA, Johannson, KA, Fisher, JH, and Ryerson, CJ. YouTube videos as a source of misinformation on idiopathic pulmonary fibrosis. Ann Am Thorac Soc. (2019) 16:572–9. doi: 10.1513/AnnalsATS.201809-644OC
12. Aydin, MA, and Akyol, H. Quality of information available on YouTube videos pertaining to thyroid cancer. J Cancer Educ. (2020) 35:599–605. doi: 10.1007/s13187-019-01502-9
13. Bai, G, Pan, X, Zhao, T, Chen, X, Liu, G, and Fu, W. Quality assessment of YouTube videos as an information source for testicular torsion. Front Public Health. (2022) 10:905609. doi: 10.3389/fpubh.2022.905609
14. Suarez-Lledo, V, and Alvarez-Galvez, J. Prevalence of health misinformation on social media: systematic review. J Med Internet Res. (2021) 23:e17187. doi: 10.2196/17187
15. Johnson, SB, Parsons, M, Dorff, T, Moran, MS, Ward, JH, Cohen, SA, et al. Cancer misinformation and harmful information on Facebook and other social media: a brief report. J Natl Cancer Inst. (2022) 114:1036–9. doi: 10.1093/jnci/djab141
16. Chou, W-YS, Oh, A, and Klein, WMP. Addressing health-related misinformation on social media. JAMA. (2018) 320:2417–8. doi: 10.1001/jama.2018.16865
17. Annie, A. TikTok won the top of the global app download list at Christmas. (2021); Available online at: https://blog-en.olivech.com/hots/6308/.
18. Wang, Y. Humor and camera view on mobile short-form video apps influence user experience and technology-adoption intent, an example of TikTok (DouYin). Comput Human Behav. (2020) 110:106373. doi: 10.1016/j.chb.2020.106373
19. Wright, C. Are beauty bloggers more influential than traditional industry experts? J Promo Commun. (2017) 5:303–22.
20. News, B. TikTok makes moves into education market. (2020); Available online at https://www.bbc.com/news/technology-53079625.
21. Zhu, C, Xu, X, Zhang, W, Chen, J, and Evans, R. How health communication via Tik Tok makes a difference: a content analysis of Tik Tok accounts run by Chinese provincial health committees. Int J Environ Res Public Health. (2019) 17:192:17. doi: 10.3390/ijerph17010192
22. Basch, CH, Fera, J, Pierce, I, and Basch, CE. Promoting mask use on TikTok: descriptive cross-sectional study. JMIR Public Health Surveill. (2021) 7:e26392. doi: 10.2196/26392
23. Singh, AG, Singh, S, and Singh, PP. YouTube for information on rheumatoid arthritis--a wakeup call? J Rheumatol. (2012) 39:899–903. doi: 10.3899/jrheum.111114
24. Khazaal, Y, Chatton, A, Cochand, S, Coquard, O, Fernandez, S, Khan, R, et al. Brief DISCERN, six questions for the evaluation of evidence-based content of health-related websites. Patient Educ Couns. (2009) 77:33–7. doi: 10.1016/j.pec.2009.02.016
25. Charnock, D, Shepperd, S, Needham, G, and Gann, R. DISCERN: an instrument for judging the quality of written consumer health information on treatment choices. J Epidemiol Community Health. (1999) 53:105–11. doi: 10.1136/jech.53.2.105
26. Gorrepati, PL, and Smith, GP. DISCERN scores of YouTube information on eczema treatments. J Am Acad Dermatol. (2021) 85:1354–5. doi: 10.1016/j.jaad.2020.11.007
27. Comp, G, Dyer, S, and Gottlieb, M. Is TikTok the next social media frontier for medicine? AEM Educ Train. (2020) 5:532. doi: 10.1002/aet2.10532
28. Kassamali, B, Villa-Ruiz, C, Mazori, DR, Min, M, Cobos, GA, and LaChance, AH. Characterizing top educational TikTok videos by dermatologists in response to TikTok and dermatology: an opportunity for public health engagement. J Am Acad Dermatol. (2021) 85:e27–8. doi: 10.1016/j.jaad.2021.02.051
29. Song, S, Xue, X, Zhao, YC, Li, J, Zhu, Q, and Zhao, M. Short-video apps as a health information source for chronic obstructive pulmonary disease: information quality assessment of TikTok videos. J Med Internet Res. (2021) 23:e28318. doi: 10.2196/28318
30. Li, Y, Guan, M, Hammond, P, and Berrey, LE. Communicating COVID-19 information on TikTok: a content analysis of TikTok videos from official accounts featured in the COVID-19 information hub. Health Educ Res. (2021) 36:261–71. doi: 10.1093/her/cyab010
31. Kong, W, Song, S, Zhao, YC, Zhu, Q, and Sha, L. TikTok as a health information source: assessment of the quality of information in diabetes-related videos. J Med Internet Res. (2021) 23:e30409. doi: 10.2196/30409
32. TTEP.Cn: Time limit requirements for TikTok. (2022). Available online at: http://www.ttep.cn/jy/93448.html.
33. CN-HEALTHCARE. TikTok statement: only these people and units can publish medical content. (2022); Available online at: https://www.cn-healthcare.com/article/20210323/content-552617.html.
34. Shires, CB, Wilson, CD, and Sebelik, MJGS. Thyroid surgery YouTube videos: estimating quality by surgeon characteristics and view rate. Gland Surg. (2018) 8:207–11. doi: 10.21037/gs.2018.10.01
35. Ocak, U. Evaluation of the content, quality, reliability and accuracy of YouTube videos regarding endotracheal intubation techniques. Niger J Clin Pract. (2018) 21:1651–5. doi: 10.4103/njcp.njcp_207_18
36. Fan, Y, Tang, S, and Zheng, B. Factors influencing the popularity of scientific short videos: a case study of TikTok account of Chinese Academy of Sciences. Stud. Sci. Pop. (2020) 4:55–62. doi: 10.19293/j.cnki.1673-8357.2020.04.007
37. Spangler, T. TikTok bumps up max video length to 10 minutes. (2022). Available online at: https://variety.com/2022/digital/news/tiktok-maximum-video-length-10-minutes-1235191773/.
Keywords: thyroid cancer, TikTok, health information, short video, social media
Citation: Wang L, Li Y, Gu J and Xiao L (2023) A quality analysis of thyroid cancer videos available on TikTok. Front. Public Health. 11:1049728. doi: 10.3389/fpubh.2023.1049728
Edited by:
Harshad Thakur, Tata Institute of Social Sciences, IndiaReviewed by:
Jane G. Payumo, Michigan State University, United StatesTijs Van Den Broek, VU Amsterdam, Netherlands
Copyright © 2023 Wang, Li, Gu and Xiao. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Yongjin Li, bGl5b25namluQHpqaHUuZWR1LmNu