Content analysis of conversations on Reddit: reactions to FDA’s ENDS prioritized enforcement policy

Introduction: On January 2, 2020, the FDA announced a policy focused in part on prioritizing enforcement of flavored (other than tobacco- or menthol-flavored) cartridge-based electronic nicotine delivery systems (ENDS) without premarket authorization.

Methods: We used a query to identify Reddit conversations relevant to the policy from January 2 to May 6, 2020. Our sample included 576 posts (46 posts and 530 accompanying comments). Two analysts coded posts for mentions of use behaviors (e.g., switching, quitting), purchasing behaviors (e.g., purchasing from retailer new to the user), and flavored products. We summarized frequencies of coded data and provided illustrative quotes.

Results: Only 21.0% (121/576) of posts mentioned use behavior. Switching behavior was the most common use behavior mentioned (50.4%, 61/121). Most switching behavior posts focused on ENDS-related switching (91.8%, 56/61). The most common ENDS-related switching behaviors mentioned were switching to an open tank (45.9%, 28/61) or device with refillable pods/cartridges (44.3%, 27/61); 8.2% (5/61) mentioned switching to disposables. Just 15.5% (89/576) of posts mentioned purchasing behavior, with the most common being purchasing from a retailer new to the user (32.6%, 29/89). Only 6.8% (39/576) of posts mentioned specific flavors.

Conclusion: Reddit posts about the policy commonly discussed switching to non-cartridge-based ENDS products, such as open tank systems or disposable devices, and purchasing products from different online sources that were still selling these products. Findings suggest that publicly available Reddit data can complement data from traditional sources (e.g., surveys, sales) to understand potential unintended consequences associated with policies by exploring the public’s reactions.

1 Introduction

The 2009 Family Smoking Prevention and Tobacco Control Act and subsequent 2016 “deeming rule” provided the U.S. Food and Drug Administration (FDA) authority to regulate the manufacture, distribution, and marketing of tobacco products, including electronic nicotine delivery systems (ENDS) (Food and Drug Administration, HHS, 2016). All deemed new tobacco products on the market as of August 8, 2016 were required to submit a premarket application by September 9, 2020. FDA maintains the ability to prioritize enforcement against any unauthorized new tobacco product.

On January 2, 2020 (revised April 2020), FDA issued final guidance to industry that announced a policy prioritizing, among other things, enforcement for some unauthorized flavored (e.g., fruit, mint) cartridge-based ENDS products that may appeal to young people (U.S. Food and Drug Administration, 2020a).¹ Companies had 30 days to “cease the manufacture, distribution, and sale of unauthorized flavored cartridge-based e-cigarettes (other than tobacco or menthol)” or risk FDA enforcement actions when the enforcement policy went into effect on February 6, 2020.²

Several studies examined responses to the policy. After policy implementation, adult JUUL users reported less use of mint-flavored ENDS pods (explicitly prioritized for policy enforcement) and more use of menthol-flavored ENDS pods (explicitly not prioritized for policy enforcement) (Yingst et al., 2020). Another study showed sales of mint-flavored prefilled ENDS cartridges decreased following the policy (Ali et al., 2020; Diaz et al., 2020). Between August 2019 and May 2020, mint-flavored prefilled cartridge sales decreased from 47.6 to 0.3% of all prefilled cartridge unit sales, while menthol- and tobacco-flavored prefilled cartridge sales increased from 10.7 to 61.8% and 22.8 to 37.1% of all prefilled cartridge unit sales, respectively (Ali et al., 2020). In the 8 weeks following the policy announcement, U.S. ENDS market share of mint-flavored ENDS decreased 82.8% while share of menthol-flavored ENDS increased 104.9% (Diaz et al., 2020). Around the time of and following policy implementation, sales also increased for disposable ENDS that had greater product capacity and nicotine strength and lower cost per milliliter (Diaz et al., 2023). Relatedly, survey research suggests that young adult users of flavored ENDS most commonly responded to state and local restrictions on all flavored ENDS by continuing to vape (Tam et al., 2024).

While sales and survey data are important for understanding policy effects, social media data can provide complementary information on how people share and seek information around policies to provide early signals for how consumers may respond to policies, including in-depth, timely insights into how policy influences perceptions and tobacco use behaviors (e.g., initiation, use, substitution, cessation). The Comprehensive Model of Information Seeking suggests that antecedent factors such as personal experience with and salience of an issue to an individual influence the utility of information held by an information carrier and that utility along with characteristics of the information carrier influence information seeking actions (Johnson et al., 1995). Factors in this model apply to information seeking behavior around ENDS-related policies on Reddit. First, policies influencing the availability of ENDS products are salient to ENDS users and on Reddit people engage in rich, descriptive dialogue in self-organized communities of interest (e.g., communities focused on ENDS use) indicating utility to ENDS users seeking information around these policies. Further, Reddit as an information carrier provides opportunities to engage in in-depth dialogue due to no character maximum and the platform’s interactive nature making it useful for understanding perceptions and behaviors. While Reddit may be less popular among teens and adults than other platforms (e.g., YouTube) (Anderson and Jiang, 2018; Auxier and Anderson, 2021), Reddit insights may allow for in-depth understanding of attitudes and reported behaviors around ENDS use. Reddit users also have “karma” provided based on “upvotes” (i.e., user action signaling approval), which is a characteristic of individual information carriers within Reddit and serves as an indicator of users’ level of influence and popularity and increases the likelihood that users’ content will be viewed by information seekers.

Several studies have explored how users seek information about ENDS-related policies on Reddit. One article analyzed Reddit conversations about vaping policies (Xu et al., 2021). Another study (Silver et al., 2022) analyzed themes related to policy circumvention in Reddit conversations around ENDS restrictions, including the ENDS prioritized enforcement policy. Previous studies have also examined Reddit discussions about ENDS flavors (Wang et al., 2015; Zhan et al., 2017; Chen et al., 2020; Lu et al., 2020; Luo et al., 2021), ENDS use (Brett et al., 2019; Zhan et al., 2019), ENDS use motivations (Sharma et al., 2016), purchasing and selling ENDS (Liu et al., 2020), ENDS product reviews (Allem et al., 2019), e-liquid ingredients (Li et al., 2016), and topic analysis (Barker and Rohde, 2019; Hu et al., 2021; Xu et al., 2021).

This study investigated public response to the ENDS prioritized enforcement policy by analyzing Reddit conversations to qualitatively describe ENDS use, switching between products, and purchasing behaviors, which is crucial to understanding reactions to the policy and has yet to be explored with Reddit data. Results will also contribute to the growing body of research examining responses to the policy (Ali et al., 2020; Diaz et al., 2020; Yingst et al., 2020) by providing qualitative information to contextualize these findings.

2 Methods

This study was determined non-human subjects research by RTI International’s Institutional Review Board.

2.1 Relevant Reddit terms

On Reddit, users share content as a “post.” A “subreddit” is a group dedicated to one topic. A “comment” is a post by a user replying to an initial post or other comment. A “comment thread” involves one or more users responding to a post comment, becoming its own discussion; there may be multiple threads in reaction to one initial post. “Karma” is provided for each user and is based on “upvotes” (i.e., user action signaling approval that is similar to a “like”) (Reddit, 2021).

2.2 Data source and sample

To identify relevant discussions, we searched for terms such as “flavor policy(−ies)” or “flavor(s) ban*” posted in subreddits with ENDS content from January 2 to May 6, 2020 (see Supplementary material for query and subreddits). We used the chosen search terms to explore conversations in ENDS-related subreddits to reduce irrelevant content. Posts were English-language only. Searches were conducted in Brandwatch, a platform that provides users access to social media data. We identified an initial sample of 396 posts (Figure 1). Data were collected and coded from September through November 2020.

Figure 1

Figure 1. Sampling of Reddit posts and comments about FDA ENDS prioritized enforcement policy.

To narrow the coding sample, we first selected posts published by users with the highest karma scores. We selected the top 100 posts based on karma scores, limiting one post per user to ensure the sample was not skewed by individual users’ perspectives. We prioritized posts based on the highest karma scores (Struik and Yang, 2021) to focus on popular posts with higher engagement.

For each of these 100 posts, we selected up to three comment threads with the highest number of upvotes to focus on comments with higher engagement. The number of comment threads in these posts ranged from zero to 42 (mean = 7; median = 4 [IQR: 2–8]). At this stage, we included all comments in selected threads for coding. This resulted in an initial sample of 100 posts with 986 comments from 245 comment threads.

After the initial prioritization, we excluded irrelevant posts and accompanying comments (e.g., focused on conversations outside the U.S.) (3 posts, 10 comments) and excluded posts or comments that the author deleted (0 posts, 21 comments). We coded the resulting sample of 97 posts and 955 comments.

To ensure our sample focused on conversations related to the federal ENDS prioritized enforcement policy, we removed individual posts and comments that discussed only state or local policies or did not reference the federal policy (51 posts, 425 comments). For example, if a post discussed a state-level policy it was excluded, but if accompanying comments discussed the federal policy those were included.

Our final analytic sample included 46 posts and 530 comments (Figure 1). Each post and comment were coded separately and treated as equivalent units of analysis, for an analytic sample of 576 posts and comments from 282 unique authors. Posts and comments are hereafter referred to as posts for brevity.

2.3 Coding

We developed a codebook to capture themes about use and purchasing behaviors and mentions of ENDS flavors. Themes were not mutually exclusive, meaning a post could be coded for multiple themes. Posts included discussion of anticipated, actual, or recommended behaviors (e.g., predicting other users’ behaviors in response to the policy). All posts in the sample were double coded by two trained coders using qualitative coding software, NVivo 12. A third adjudicator independently resolved discrepancies.

2.3.1 Use behaviors

Use behaviors were coded when Reddit users discussed behaviors related to using ENDS or other tobacco products in response to the policy.

Switching behaviors were coded when users discussed products they were switching to in response to the policy (e.g., other ENDS device types, other tobacco products). ENDS-related switching included: (1) switching to open tank devices (e.g., tank systems); (2) switching to devices with refillable pods or cartridges (explicitly described as refillable) that can be filled with one’s choice of e-liquid; (3) switching to disposable devices sold ready for use (prefilled and charged); (4) switching to prefilled cartridge-based ENDS (device and pods) with tobacco- or menthol-flavored pods/cartridges; (5) switching to zero-nicotine e-liquids; and (6) other ENDS-related switching. We also coded for switching behaviors related to other products, including switching to non-ENDS tobacco products and cannabis.

Other use behaviors coded were: (1) mixing own e-liquid (i.e., do-it-yourself [DIY]); (2) cutting back or quitting ENDS; (3) continued use of flavored (non-menthol and non-tobacco) cartridge-based ENDS (includes cartridges/pods sold separately); (4) other ENDS use behaviors; and (5) other tobacco use behaviors.

2.3.2 Purchasing behavior

Purchasing behaviors in response to the policy were coded as: (1) purchasing from a retailer new to the user; (2) purchasing from a non-U.S. jurisdiction; (3) purchasing extra (i.e., stockpiling) ENDS not mentioned in the policy; (4) purchasing extra (i.e., stockpiling) flavored (non-menthol and non-tobacco) cartridge-based ENDS with pods/cartridges; (5) purchasing from another user or noncommercial source; and (6) other purchasing behavior.

2.3.3 Flavor coding

Flavor was coded when a flavor was mentioned in relation to ENDS as follows: (1) flavored products included in the policy (i.e., flavored [other than menthol- or tobacco-flavored] pod/cartridge-based ENDS, including devices sold with cartridges/pods and cartridges/pods sold separately); (2) all other flavored products not included in that policy provision (i.e., all flavors of non-cartridge-based e-liquid, menthol- or tobacco-flavored pods, all flavors of disposable devices); and (3) other flavored ENDS, which included cases where a specific flavor was mentioned but device type was unclear.

2.4 Analysis

We summarized frequencies of coded themes and provided illustrative post examples. To protect privacy, we paraphrased examples or used partial quotes. We also described instances where behavior themes intersected with flavored product mentions.

3 Results

3.1 Use behavior

Twenty-one percent of total posts mentioned use, including switching and other use behaviors (Table 1).

Table 1

Table 1. Mentions of use behaviors based on a sample of Reddit posts about FDA’s ENDS prioritized enforcement policy, January 2, 2020–May 6, 2020^a.

3.1.1 Switching behavior

Among posts mentioning use behavior, switching was most common (50.4%). Within switching, ENDS-related switching was most common (91.8% of switching behaviors in posts). Within ENDS-related switching, discussion of switching to an open tank (45.9%) or refillable device (44.3%) was most common. Users often said they planned to switch devices or products in response to the policy and requested or provided recommendations. Users were occasionally in search of fruit- or mint-flavored products to replace previous cartridge-based ENDS. In a post about switching to an open tank, one user asked for recommendations to replace “Iced Mango” Zpods.

Switching to a disposable device appeared in 8.2% of switching posts: users posited that many would switch to disposables or discussed switching themselves. One user said that many teens had already switched from JUUL to disposable Puff Bar “to get good [flavors]” without maintaining a tank device. Mentions of switching to cartridge-based ENDS with tobacco- or menthol-flavored pods/cartridges (4.9%) were uncommon. Users occasionally described trying, but not liking, these flavors. Mentions of switching to zero-nicotine e-liquid were uncommon (1.6%).

Mentions of switching to other non-ENDS tobacco products (6.6%) and cannabis (1.6%) were uncommon. Users occasionally noted other people would begin smoking cigarettes again following the policy or that they themselves would smoke or dip.

3.1.2 Other use behaviors

The second most common use behavior was mixing e-liquid (i.e., DIY activities) (24.0% of use behavior posts). One user said they blended e-liquids (e.g., “mix ‘hot cocoa’ and ‘almond’”). Another user noted DIY is “easy and cheap” and others should try it; another user noted that there are “apps with hundreds of [DIY] recipes.”

Cutting back or quitting ENDS was discussed in 12.4% of use behavior posts. Users occasionally discussed how the policy resulted in cutting back or quitting ENDS, which some said would be beneficial because the habit was “expensive.” Users occasionally discussed quitting for health reasons; one noted ENDS were “taking my life away.”

Discussion of continued use of flavored (other than tobacco- or menthol-flavored) cartridge-based ENDS occurred in 8.3% of posts. Posts often discussed policy circumvention. One user said teens would “always find a way” to continue using restricted products via overseas or black market sources. Users occasionally mentioned using restricted flavors; one said they now use fruit-flavored “knock off pods… and hate it.”

Other ENDS use behaviors (i.e., use behaviors not in pre-defined themes) were discussed in 8.3% of posts. Users asked questions about the policy; mentioned stockpiling unrestricted products (e.g., nicotine e-liquids, coils) in preparation for potential new policies (e.g., advising freezing nicotine e-liquids for long-term storage); or noted general ENDS use. One user wanted to know whether the menthol flavor changed after the policy, saying that the menthol-flavored cartridges tasted different to them and they might seek alternatives.

Discussion of other tobacco use behaviors with ENDS use (i.e., dual use) was infrequent (5.8%). These posts often overlapped with switching to non-ENDS tobacco products. One user said they will continue using smokeless if “flavors do get banned” in ENDS.

3.2 Purchasing behavior

Overall, 15.5% of posts mentioned purchasing behavior (Table 2). The most frequent purchasing behavior was purchasing from a retailer new to the user (32.6%), including specifying retailers they were currently purchasing from or planned to purchase from or requesting suggestions (e.g., “wevapeusa.com,” “pricepointny”). One user noted “discounthookahandvape” sold policy-restricted mango- and mint-flavored cartridges.

Table 2

Table 2. Mentions of purchasing behaviors based on a sample of Reddit posts about FDA’s ENDS prioritized enforcement policy, January 2, 2020–May 6, 2020^a.

In 21.3% of purchasing posts, users mentioned purchasing products prioritized for enforcement from non-U.S. jurisdictions (e.g., Canada). Users asked for source recommendations; in response, users noted retailers outside the U.S. selling restricted products to U.S. customers.

We identified purchasing extra other ENDS products (i.e., products not included in the policy) in 21.3% of purchasing posts. Users noted concerns about supply, and suggested stocking up on nicotine, e-liquid, coils, and devices (e.g., open/refillable devices). One user stated that users will regret not stockpiling products containing nicotine assuming these products could be taxed.

Purchasing extra flavored (other than menthol- and tobacco-flavored) cartridge-based ENDS (device and pods) was mentioned in 13.5% of purchasing posts. Users encouraged product stockpiling (“find a flavor you enjoy and stock up”) as JUUL’s flavored cartridge production stopped. One user doubted the policy would go through but advised others to stock up. No mentions of purchasing from a direct source (e.g., another user or non-commercial source) were observed.

Other purchasing behaviors were common (36.0%), including discussing features of devices users purchased or recommended purchasing, general purchase behavior mentions, or purchasing normal quantities (i.e., not stockpiling) of unrestricted products (e.g., nicotine, mod parts). One user preferred tobacco flavors, and reported being unaffected by the policy.

3.3 Specific flavors

Table 3 describes specific flavor mentions (e.g., flavored, menthol, tobacco) within use and purchasing posts. Flavor mentions were infrequent (6.8% of total). Specific flavors were mentioned in 24.8% of use behavior posts (5.2% of total), 27.9% of switching behavior posts (3.0% of total), and in 16.9% of purchasing behavior posts (2.6% of total). Posts about use and purchasing that mentioned flavors often discussed previously described themes.

Table 3

Table 3. Mentions of specific flavor^a by use and purchasing behavior based on a sample of Reddit posts about FDA’s ENDS prioritized enforcement policy, January 2, 2020–May 6, 2020^b.

New themes emerged in posts mentioning specific flavors. One theme centered around seeking to clarify what flavors and devices the policy affected. One user asked whether fruit flavors of “Flair Envy” or “Cali” devices would remain available. Another user considered switching to refillable pods, asking if they were “still going to be legal.” A second theme involved asking for or providing flavor recommendations to replace restricted products. One user requested mint-flavored e-liquid recommendations; others recommended “Pod juice Jewel Mint,” “Jewel Mint Sapphire,” and “Arctic Air.” A third theme was users experimenting with, but disliking, new flavors. One user responded to those who recommended a brand of honeydew, stating “it was worse” than their previous honeydew flavor.

4 Discussion

4.1 Summary of findings

We analyzed Reddit conversations describing use and purchase behaviors in the 4 months following FDA’s announcement of a policy prioritizing enforcement of flavored (other than tobacco- and menthol-flavored) cartridge-based ENDS. Users most frequently talked about switching to other ENDS that were not the focus of this provision of the policy (i.e., open/refillable systems). This finding is consistent with Silver et al. (2022) research showing that consumers on Reddit discussed substitute products (e.g., disposable or refillable devices) in discussions around ENDS flavor policies, including the prioritized enforcement policy. Recent studies analyzed ENDS sales data after the policy and found increases in sales of ENDS products that were not the focus of the policy, including (1) sales of flavored (other than tobacco- or menthol-flavored) disposable products, menthol- and tobacco-flavored prefilled cartridges (Ali et al., 2020; Diaz et al., 2020) and (2) sales of all menthol-flavored ENDS (Ali et al., 2020; Diaz et al., 2020).

Users posted less frequently about switching to other ENDS not explicitly prioritized by the policy (e.g., cartridge-based devices with tobacco- or menthol-flavored pods or cartridges, any flavor disposable devices). This finding was a bit surprising, given that disposable devices are available in many flavors (Ramamurthi et al., 2023), but this may reflect the sample of Reddit users. Although we are unaware of data characterizing ENDS users posting on Reddit, 18% of U.S. adults aged 18 and older (36% aged 18–29; Auxier and Anderson, 2021) and 7% of youth aged 13–17 report Reddit use (Anderson and Jiang, 2018) while research shows that disposable ENDS are more popular among youth than young adults and adults (McCauley et al., 2022). Few users in our sample discussed switching to non-ENDS tobacco products (e.g., cigarettes) or cannabis.

After switching, the second most common use behavior mentioned was mixing one’s own liquids, several of which emanated from the r/DIY_eJuice subreddit (14%; 4/29), and potentially suggesting attempts to mimic restricted flavors. Fewer users discussed cutting back or quitting ENDS in relation to the policy.

In conversations about purchasing behaviors, users mentioned purchasing from a retailer they had not previously purchased from (e.g., specific online retailers), purchasing products from outside the U.S., and stockpiling products in anticipation of the policy. Silver et al. (2022) similarly showed that consumers on Reddit discussed purchasing products online from outside the U.S. in discussions around ENDS flavor policies, including the prioritized enforcement policy. Research shows that more comprehensive flavored tobacco product restrictions (i.e., policies that do not exempt certain product categories, flavors, or retailers) can help reduce unintended consumer responses, such as switching to exempt products or purchasing from alternative retailers observed here (Gammon et al., 2022). The most common purchasing-related mentions were not in a pre-defined (i.e., a priori) coding category, including discussions of device features, purchase recommendations, and general purchase behaviors.

Specific flavors were mentioned infrequently. Many posts mentioning flavor discussed continued use of or purchasing extra flavored (other than tobacco- or menthol-flavored) cartridge-based ENDS. Other themes around flavors included asking questions about or clarifying flavors/devices affected by the policy; asking for or providing flavor recommendations to replace restricted products; and trying, but disliking, new flavors as a result of the policy.

These qualitative analyses of Reddit data provide more detailed insights into the range of use and purchasing behaviors in response to the ENDS prioritized enforcement policy than traditional public health surveillance sources (e.g., surveys, sales data) and suggest that social media data can provide meaningful insights into how people gather and share information about behaviors associated with health policies. A previous study of Instagram and Twitter complements our findings, showing ENDS brands were not using Instagram and Twitter to market cartridge-based ENDS devices with any tobacco, menthol, or other flavor, but were advertising devices without mentioning flavor (e.g., open/refillable, disposable) (Guillory et al., 2024). Social media data, being available in real time, have the potential to reveal timely reactions to policies across many products and retailers, which may not be possible with other data sources. Recent studies have demonstrated the utility of using social media data to assess public perceptions of policies, for example illustrating Reddit users’ discussion of ENDS flavor policy circumvention (Silver et al., 2022), showing that Twitter users’ sentiment around ENDS changed (Lu et al., 2022) and that number of mentions about quitting vaping increased after the announcement of the enforcement policy (Xie et al., 2022).

Our findings highlight the usefulness of conducting ongoing monitoring of social media and other data sources, such as the approach employed by the Center for Rapid Surveillance of Tobacco (CRST, 2024) to monitor tobacco industry marketing and emerging products, to anticipate responses and potential adaptations to future tobacco policy changes. Such monitoring could inform messaging efforts on cessation or to counteract misinformation. It is important to acknowledge that while Reddit data can provide signals for surveillance and act as an information resource for users around policy events, it may provide an incomplete picture of policy responses and should be considered alongside signals from other data sources, including sales and surveys. For example, there were few mentions of switching to flavored (other than tobacco- or menthol-flavored) disposable products and menthol- and tobacco-flavored prefilled cartridges despite sales of these products increasing around this time period (Ali et al., 2020; Diaz et al., 2020). However, these studies did not capture sales of open/refillable devices, which suggests that Reddit data can fill a gap by highlighting responses related to products not captured in sales data.

4.2 Limitations

This study has several limitations. This study focuses on conversations among a limited sample of Reddit posts based on the query used and sampling method. Furthermore, Reddit users represent a subset of the population that is actively, publicly discussing and sharing information about ENDS. Thus, findings may not generalize to conversations among all Reddit users, or to users on other platforms. We also do not know the characteristics of Reddit users who posted about the policy (e.g., age, ENDS use status). We collected posts from September through November 2020 for a retrospective time period of January through May 2020; users may have deleted or altered posts between the time of posting and data collection and we acknowledge the age of the data in the present study as a limitation. The study period included the final policy announcement (January 2, 2020) and the start of the policy (February 6, 2020); conversations may differ between the announcement period (policy anticipation) and the policy implementation. This study did not examine conversations around the November 2018 initial statement by the FDA Commissioner detailing aspects of the guidance, the March 2019 publication of the draft guidance, or any actions taken against products that were not explicitly prioritized in this policy (U.S. Food and Drug Administration, 2020b). This study did not code for illicit retail sales in the U.S. or elsewhere. Additionally, discussions included past, present, future, or hypothetical behaviors and these posts may not represent actual behaviors. Use of paraphrasing may have subtly altered the meaning or interpretation of original quotes. Next, the majority of posts in our sample did not mention any use or purchasing behavior. Users more commonly discussed general topics such as opinions about the policy (e.g., skepticism, negative or positive attitudes, political sentiments about the policy), which were outside the study scope. Last, ENDS-related behaviors around use and purchasing change over time and the conclusions from this study may not apply to other time periods.

5 Conclusion

In response to the ENDS prioritized enforcement policy, Reddit posts in our sample commonly discussed switching to other ENDS products not included in this provision of the policy, namely devices with refillable pods/cartridges and open tank systems, and purchasing products from online sources new to the user that still sold restricted flavored products. These findings underscore the benefit of analyzing publicly available social media data from ENDS communities on Reddit to examine potential unintended consequences associated with a policy, and can complement findings from survey and sales data to better understand how the public may have responded to the policy.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Author contributions

JG: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Supervision, Visualization, Writing – original draft, Writing – review & editing. ST: Conceptualization, Data curation, Methodology, Supervision, Writing – original draft, Writing – review & editing. JB: Data curation, Formal analysis, Project administration, Writing – original draft. AR: Software, Supervision, Writing – original draft, Writing – review & editing. SL: Conceptualization, Data curation, Formal analysis, Investigation, Software, Supervision, Writing – original draft. AK: Conceptualization, Supervision, Writing – review & editing. JN: Conceptualization, Methodology, Supervision, Writing – review & editing. STL: Conceptualization, Investigation, Methodology, Supervision, Writing – review & editing. KS: Supervision, Writing – review & editing. JD: Project administration, Supervision, Writing – review & editing.

Funding

The author(s) declare that financial support was received for the research, authorship, and/or publication of this article. This work was supported by the U.S. Food and Drug Administration Center for Tobacco Products (contract no. HHSF223201810042B/75F40119F19002).

Acknowledgments

The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the U.S. Food and Drug Administration. D. Neveleff with FDA provided critical review and comment on the manuscript drafts.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fcomm.2024.1348158/full#supplementary-material

Footnotes

1. ^In November 2018, FDA’s commissioner made an initial statement proposing new steps to reduce youth access to ENDS, including reducing access to flavored ENDS (other than mint, menthol, tobacco) (U.S. Food and Drug Administration, 2018). Draft guidance for a prioritized ENDS enforcement policy was published in March 2019 Federal Register (2019).

2. ^The policy also prioritized enforcement of products for which the manufacturer failed to take adequate measures to prevent minors’ (under age 21) access to and products for which marketing targeted or likely promoted minors’ use.

References

Ali, F. R. M., Diaz, M. C., Vallone, D., Tynan, M. A., Cordova, J., Seaman, E. L., et al. (2020). E-cigarette unit sales, by product and flavor type - United States, 2014–2020. MMWR Morb. Mortal Wkly. Rep. 69, 1313–1318. doi: 10.15585/mmwr.mm6937e2

Crossref Full Text | Google Scholar

Allem, J. P., Majmundar, A., Dharmapuri, L., Unger, J. B., and Cruz, T. B. (2019). Insights on electronic cigarette products from reviews on the Reddit forum. Tob. Prev. Cessat. 5:29. doi: 10.18332/tpc/111533

Crossref Full Text | Google Scholar

Anderson, M., and Jiang, J. (2018). Teens, social media and technology 2018. Pew Research Center. Available at: https://www.pewresearch.org/internet/2018/05/31/teens-social-media-technology-2018/ (Accessed May 31, July 18, 2022).

Google Scholar

Auxier, B., and Anderson, M. (2021). Social media use in 2021. Pew Research Center. Available at: https://www.pewresearch.org/internet/2021/04/07/social-media-use-in-2021/ (Accessed April 7, July 18, 2022).

Google Scholar

Barker, J. O., and Rohde, J. A. (2019). Topic clustering of e-cigarette submissions among Reddit communities: a network perspective. Health Educ. Behav. 46, 59–68. doi: 10.1177/1090198119863770

Crossref Full Text | Google Scholar

Brett, E. I., Stevens, E. M., Wagener, T. L., Leavens, E. L. S., Morgan, T. L., Cotton, W. D., et al. (2019). A content analysis of JUUL discussions on social media: using Reddit to understand patterns and perceptions of JUUL use. Drug Alcohol Depend. 194, 358–362. doi: 10.1016/j.drugalcdep.2018.10.014

Crossref Full Text | Google Scholar

Chen, L., Lu, X., Yuan, J., Luo, J., Luo, J., Xie, Z., et al. (2020). A social media study on the associations of flavored electronic cigarettes with health symptoms: observational study. J. Med. Internet Res. 22:e17496. doi: 10.2196/17496

Crossref Full Text | Google Scholar

CRST (2024). Center for Rapid Surveillance of tobacco TIM alerts. Available at: https://tobaccocrst.org/alert-archive

Google Scholar

Diaz, M. C., Donovan, E. M., Schillo, B. A., and Vallone, D. (2020). Menthol e-cigarette sales rise following 2020 FDA guidance. Tob. Control. 30, 700–703. doi: 10.1136/tobaccocontrol-2020-056053

Crossref Full Text | Google Scholar

Diaz, M. C., Silver, N. A., Bertrand, A., and Schillo, B. A. (2023). Bigger, stronger and cheaper: growth in e-cigarette market driven by disposable devices with more eliquid, higher nicotine concentration and declining prices. Tob. Control. 1–6. doi: 10.1136/tc-2023-058033

Crossref Full Text | Google Scholar

Food and Drug Administration, HHS (2016). Deeming Tobacco Products To Be Subject to the Federal Food, Drug, and Cosmetic Act, as Amended by the Family Smoking Prevention and Tobacco Control Act; Restrictions on the Sale and Distribution of Tobacco Products and Required Warning Statements for Tobacco Products. Final rule. Fed Regist. 81:28973–29106.

Google Scholar

Federal Register (2019). Modifications to compliance policy for certain deemed tobacco products; draft guidance for industry; availability. U.S. Department of Health and Human Services Food and Drug Administration Center for Tobacco Products. Available at: https://www.federalregister.gov/documents/2019/03/14/2019-04765/modifications-to-compliance-policy-for-certain-deemed-tobacco-products-draft-guidance-for-industry

Google Scholar

Gammon, D. G., Rogers, T., Gaber, J., Nonnemaker, J. M., Feld, A. L., Henriksen, L., et al. (2022). Implementation of a comprehensive flavoured tobacco product sales restriction and retail tobacco sales. Tob. Control. 31, e104–e110. doi: 10.1136/tobaccocontrol-2021-056494

Crossref Full Text | Google Scholar

Guillory, J. G., Trigger, S., Ross, A., Lane, S., Kim, A., Nonnemaker, J., et al. (2024). Changes in industry marketing of electronic nicotine delivery systems on social media following FDA's prioritized enforcement policy: a content analysis of Instagram and twitter posts. Front. Comm. 8:1274098. doi: 10.3389/fcomm.2023.1274098

Crossref Full Text | Google Scholar

Hu, M., Benson, R., Chen, A. T., Zhu, S. H., and Conway, M. (2021). Determining the prevalence of cannabis, tobacco, and vaping device mentions in online communities using natural language processing. Drug Alcohol Depend. 228:109016. doi: 10.1016/j.drugalcdep.2021.109016

Crossref Full Text | Google Scholar

Johnson, J. D., Donohue, W. A., Atkin, C. A., and Johnson, S. (1995). A comprehensive model of information seeking. Sci. Comm. 16, 274–3030. doi: 10.1177/1075547095016003003

Crossref Full Text | Google Scholar

Li, Q., Zhan, Y., Wang, L., Leischow, S. J., and Zeng, D. D. (2016). Analysis of symptoms and their potential associations with e-liquids' components: a social media study. BMC Public Health 16:674. doi: 10.1186/s12889-016-3326-0

Crossref Full Text | Google Scholar

Liu, H., Li, Q., Zhan, Y., Zhang, Z., Zeng, D. D., and Leischow, S. J. (2020). Characterizing social media messages related to underage JUUL e-cigarette buying and selling: cross-sectional analysis of Reddit subreddits. J. Med. Internet Res. 22:e16962. doi: 10.2196/16962

Crossref Full Text | Google Scholar

Lu, X., Chen, L., Yuan, J., Luo, J., Luo, J., Xie, Z., et al. (2020). User perceptions of different electronic cigarette flavors on social media: observational study. J. Med. Internet Res. 22:e17280. doi: 10.2196/17280

Crossref Full Text | Google Scholar

Lu, X., Sun, L., Xie, Z., and Li, D. (2022). Perception of the food and drug administration electronic cigarette flavor enforcement policy on twitter: observational study. JMIR Public Health Surveill. 8:e25697. doi: 10.2196/25697

Crossref Full Text | Google Scholar

Luo, J., Chen, L., Lu, X., Yuan, J., Xie, Z., and Li, D. (2021). Analysis of potential associations of JUUL flavours with health symptoms based on user-generated data from Reddit. Tob. Control. 30, 534–541. doi: 10.1136/tobaccocontrol-2019-055439

Crossref Full Text | Google Scholar

McCauley, D. M., Gaiha, S. M., Lempert, L. K., and Halpern-Felsher, B. (2022). Adolescents, young adults, and adults continue to use e-cigarette devices and flavors two years after FDA discretionary enforcement. Int. J. Environ. Res. Public Health 19:8747. doi: 10.3390/ijerph19148747

Crossref Full Text | Google Scholar

Ramamurthi, D., Chau, C., Berke, H. Y., Tolba, A. M., Yuan, L., Kanchan, V., et al. (2023). Flavour spectrum of the puff family of disposable e-cigarettes. Tob. Control. 32, e71–e77. doi: 10.1136/tobaccocontrol-2021-056780

Crossref Full Text | Google Scholar

Reddit (2021). What is karma? Available at: https://reddit.zendesk.com/hc/en-us/articles/204511829-What-is-karma (Accessed July 18, 2022).

Google Scholar

Sharma, R., Wigginton, B., Meurk, C., Ford, P., and Gartner, C. E. (2016). Motivations and limitations associated with vaping among people with mental illness: a qualitative analysis of Reddit discussions. Int. J. Environ. Res. Public Health 14:7. doi: 10.3390/ijerph14010007

Crossref Full Text | Google Scholar

Silver, N., Kucherlapaty, P., Kostygina, G., Tran, H., Feng, M., Emery, S., et al. (2022). Discussions of flavored ENDS sales restrictions: themes related to circumventing policies on Reddit. Int. J. Environ. Res. Public Health 19:7668. doi: 10.3390/ijerph19137668

Crossref Full Text | Google Scholar

Struik, L., and Yang, Y. (2021). E-cigarette cessation: content analysis of a quit vaping community on Reddit. J. Med. Internet Res. 23:e28303. doi: 10.2196/28303

Crossref Full Text | Google Scholar

Tam, J., Jimenez-Mendoza, E., Buckell, J., Sindelar, J., and Meza, R. (2024). Responses to real-world and hypothetical e-cigarette flavor bans among US young adults who use flavored e-cigarettes. Nicotine Tob. Res. 26, 1113–1117. doi: 10.1093/ntr/ntad258

Crossref Full Text | Google Scholar

U.S. Food and Drug Administration (2018). Statement from FDA Commissioner Scott Gottlieb, M.D., on proposed new steps to protect youth by preventing access to flavor tobacco products and banning menthol in cigarettes.

Google Scholar

U.S. Food and Drug Administration (2020a). "FDA finalizes enforcement policy on unauthorized flavored cartridge-based e-cigarettes that appeal to children, including fruit and mint ".).

Google Scholar

U.S. Food and Drug Administration (2020b). "Shenzhen Uwell technology co., ltd. d/b/a DTD distribution Inc ".).

Google Scholar

Wang, L., Zhan, Y., Li, Q., Zeng, D. D., Leischow, S. J., and Okamoto, J. (2015). An examination of electronic cigarette content on social media: analysis of e-cigarette flavor content on Reddit. Int. J. Environ. Res. Public Health 12, 14916–14935. doi: 10.3390/ijerph121114916

Crossref Full Text | Google Scholar

Xie, Z., Ruan, J., Jiang, Y., Zhang, B., Chen, T., Luo, J., et al. (2022). Potential impact of FDA flavor enforcement policy on vaping behavior on twitter. Int. J. Environ. Res. Public Health 19. doi: 10.3390/ijerph191912836

Crossref Full Text | Google Scholar

Xu, Y. A., Kim, H. M., Wang, Y., Wu, J., Hong, T., and McLaughlin, M. (2021). “Public responses and concerns regarding vape bans on reddit: a longitudinal topic modeling approach” in Social computing and social media: Experience design and social network analysis. ed. G. Meiselwitz (Cham: Springer), 391–403.

Google Scholar

Yingst, J. M., Bordner, C. R., Hobkirk, A. L., Hoglen, B., Allen, S. I., Krebs, N. M., et al. (2020). Response to flavored cartridge/pod-based product ban among adult JUUL users: "you get nicotine however you can get it". Int. J. Environ. Res. Public Health 18. doi: 10.3390/ijerph18010207

Crossref Full Text | Google Scholar

Zhan, Y., Liu, R., Li, Q., Leischow, S. J., and Zeng, D. D. (2017). Identifying topics for e-cigarette user-generated contents: a case study from multiple social media platforms. J. Med. Internet Res. 19:e24. doi: 10.2196/jmir.5780

Crossref Full Text | Google Scholar

Zhan, Y., Zhang, Z., Okamoto, J. M., Zeng, D. D., and Leischow, S. J. (2019). Underage JUUL use patterns: content analysis of Reddit messages. J. Med. Internet Res. 21:e13038. doi: 10.2196/13038

Crossref Full Text | Google Scholar