Framing Science: How Opioid Research Is Presented in Online News Media

Matthias, Lisa; Fleerackers, Alice; Alperin, Juan Pablo

doi:10.3389/fcomm.2020.00064

ORIGINAL RESEARCH article

Front. Commun., 30 September 2020

Sec. Science and Environmental Communication

Volume 5 - 2020 | https://doi.org/10.3389/fcomm.2020.00064

Framing Science: How Opioid Research Is Presented in Online News Media

$\nLisa Matthias,$ Lisa Matthias^1,2^*

Alice Fleerackers^2,3

Juan Pablo Alperin^2,4^*

¹Political Science Department, John F. Kennedy Institute, Freie Universität Berlin, Berlin, Germany
²Scholarly Communications Lab, Simon Fraser University, Vancouver, BC, Canada
³Interdisciplinary Studies, Simon Fraser University, Vancouver, BC, Canada
⁴School of Publishing, Simon Fraser University, Vancouver, BC, Canada

Popular news media play an instrumental role in shaping public perception of issues like the opioid crisis. Using a detailed coding instrument, we analyzed how opioid-related scholarly publications were covered in 149 news stories published by nine major US and Canadian online news outlets at the height of the crisis. We find that a small proportion of available studies receive coverage, mostly within issue-focused rather than science communication news stories. While most studies are framed as established “facts,” stories rarely provide sufficient information for news consumers to critically evaluate the validity of the research. Potential implications for science communication and public perception of the opioid crisis are discussed.

Introduction

“This is not too bad,” Richard Sackler responded when he learned about 59 fatal overdoses his company's drug, OxyContin (Oxycodone, 2019), a prescription opioid, had caused in early 2001. “It could have been far worse” (James, 2019, p. 98).

Fast forward to today, when—after almost two decades of deceptive marketing of opioids and economic and social upheaval (Dasgupta et al., 2018; James, 2019)—North America's overdose problem has, indeed, become “far worse” (James, 2019, p. 98). Since Sackler's original statement, opioid-related overdoses have increased dramatically, contributing to 47,600 deaths in the United States (US) in 2017 alone (14.9 per 100,000 population; NIDA, 2019). The situation is just as dire in Canada, which saw a corresponding 4,100 fatal overdoses that same year (11.2 per 100,000 population; Special Advisory Committee on the Epidemic of Opioid Overdoses, 2019). But while both nations refer to the opioid crisis as a public health emergency (Ending America's Opioid Crisis, 2019; Health Canada, 2019), concrete actions toward harm reduction and treatment options have been limited (Casey, 2016; CFMS, 2018; GAO, 2018; Warren, 2018; Fischer et al., 2019; Frank and Haffajee, 2019). Moreover, public opinion remains divided about important aspects of the opioid crisis, with opinion leaders reporting conflicting views about the severity of the situation, how best to respond to it, and who is responsible (Nanos Research, 2017; American Psychiatric Association, 2018; Angus Reid Institute, 2018; Blendon and Benson, 2018; Manchikanti et al., 2018). Given the strong link between public opinion and policy adoption (Page and Shapiro, 1983; Cook, 2002; Burstein, 2003), understanding these divides, why they persist, and how to manage them have literally become questions of life and death. As an influential guide of public health discourse, the news media are ideally positioned to aid and shape public understanding and perception of the scientific evidence that can contribute to policies designed for addressing the opioid epidemic (Wallack et al., 1993; Maibach and Parrott, 1995; Seale, 2003). As such, this study investigates how American and Canadian online news media covered scholarly publications on opioid-related disorders in 2017 and 2018, at the height of the opioid crisis (NIDA, 2019; Special Advisory Committee on the Epidemic of Opioid Overdoses, 2019).

Literature Review

Previous studies have shown that heightened media attention to an issue can result in a direct rise of public concern—the so-called agenda setting function of the media (McCombs, 2004; Pew Research Center, 2014; Iyengar, 2015). By choosing to focus on certain issues and not others, news journalists play key roles in “setting the agenda” for what publics think about and which topics they deem important (McCombs, 2002; Entman, 2010). But the news media do more than influence whether publics think about certain issues; they also play a key role in shaping how they think about them. By presenting, or framing, news in particular ways—that is, “selecting and highlighting some facets of events or issues, and making connections among them” —journalists can “promote a particular interpretation, evaluation, and/or solution” to the issues they cover (Entman, 1993, p. 52). Conceptualized another way, frames act as “central organizing idea[s]” that journalists use to structure their stories (Gamson and Modigliani, 1989, p. 3), and that audiences can rely on to make sense of the information presented to them. By raising the salience of select ideas, framing activates certain interpretive schemas, or ways of thinking, in audience members' minds, encouraging them to act, think, or feel a particular way (Entman, 2010).

Especially within the context of science communication, the effects of such frames can be profound. Previous studies across a range of science topics have shown that different frames can influence public interest in science issues, perceptions about individuals and diseases (e.g., compassion, stigma), preferences for science-informed policies, as well as personal and political behaviors (Nisbet, 2009; Gallagher and Updegraff, 2012; Gollust et al., 2013; Riles et al., 2015). Some scholars have gone so far as to argue that, in many science debates, power has been determined by one group or individual's ability to define the nature of the problem and to suggest potential solutions—in other words, to frame the issue at hand (Nisbet et al., 2003; Nisbet and Huge, 2006).

The news media are a top source of science news for a majority of Americans (Pew Research Center, 2017), and can thus play a key role in framing scholarly publications for public consumption. Journalists can amplify frames used by scientists, “hype” novel findings, raise doubts concerning the validity of scientific findings or the morality of their implications, or reframe the debate scholarly publications are placed in (Lawrence et al., 2000; Caulfield, 2004; Nerlich and Halliday, 2007; O'Keefe et al., 2015; Stecula and Merkley, 2019). Such framing choices may be particularly important when it comes to health topics, as publics often rely on media coverage for information about diseases, medical screenings and diagnostic tests, illness prevention, and more (Stryker et al., 2008; Kealey and Berkman, 2010; Wilson et al., 2012). Furthermore, the nature and quantity of health news coverage can influence individuals' illness prevention and detection behaviors (Yanovitzky and Blitz, 2000; Chapman et al., 2005; Shen et al., 2015), use of healthcare services (Grilli et al., 2002), and treatment preferences (Chen and Siu, 2001; Shaffer et al., 2018), as well as policy-making processes (Lawrence et al., 2000).

Together, these findings suggest that the news media's framing of scholarly publications related to the opioid epidemic may have important implications for society, both for individuals and policy makers. This had been the case with regards to the news media's framing of the opioid epidemic more broadly, which, much like with previous drug epidemics related to crack cocaine and methamphetamine, saw people with opioid use disorders stigmatized in media reports in ways that are believed to have hindered policy implementation and harm reduction efforts (Lawrence et al., 2000; McGinty et al., 2018, 2019). Yet, contrary to media reporting on prior epidemics, coverage of the opioid epidemic has increasingly shifted toward a public health lens, which positions opioid use and related disorders as treatable health conditions, and away from the more traditional emphasis on law enforcement, which frames opioid misuse in terms of criminal activities (McGinty et al., 2016; Willis and Painter, 2019; Shachar et al., 2020). As both these frames promote two distinct intervention strategies—law enforcement solutions focusing on arrests and prosecutions and public health responses aimed at expanding treatment and harm reduction—these narratives matter. A recent study by Shachar et al. (2020) suggests that these differences in framing stem from the different perceptions of which demographic groups are most affected by the epidemic and the legal status of the substance involved; specifically, opioids, with the exception of heroin, are not illegal drugs but are available via medical prescription, and opioid users are perceived as being predominantly white (see also Netherland and Hansen, 2016).

However, while these studies provide important insights into the news media's framing of the opioid epidemic in general, they do not examine the specific ways in which journalists present scholarly publications related to the prevention, harm reduction, and treatment of opioid-related disorders—a key consideration given that publics often defer to the testimony of scientists and other experts when dealing with unfamiliar issues (Nisbet et al., 2003; Maier et al., 2014; Boutron et al., 2019). Notably, the opioid epidemic is not unique in this sense; while there has been a wealth of research exploring how health issues such as cancer, vaccines, and stem cells are framed in the news media (Nisbet et al., 2003; Nerlich and Halliday, 2007; Barry et al., 2011; Jung Oh et al., 2012; Hove et al., 2015; O'Keefe et al., 2015; Dan and Raupp, 2018), relatively few studies have investigated how scholarly research about those issues is framed (see Zehr, 2000; Antilla, 2005; Ruhrmann et al., 2015; Guenther et al., 2019 for exceptions). The limited research that has suggests that scholarly publications are often framed as being valid and trustworthy in news coverage but can also be framed as uncertain or controversial—especially when they relate to politicized topics (e.g., climate change, molecular medicine). Yet, no previous work has examined the media framing of scholarly publications that relate to the opioid epidemic—an important oversight, given that uncertainty and controversy frames can shift public opinions, perceptions of credibility, and behavioral intentions. For example, research on the portrayal of cancer research found that audiences perceived scientists and journalists as more trustworthy when the research was “hedged” by reporting study limitations (Jensen, 2008). Related research has found that framing scientific findings as controversial can also reduce public beliefs about the certainty of climate change and intentions to aid farmers and workers affected by its impacts (Corbett and Durfee, 2004; Gustafson and Rice, 2019), and undermine perceptions of the safety of routine childhood vaccines (Dixon and Clarke, 2013). Importantly, such framing effects appear to be especially potent when the science is communicated with a lack of context (Corbett and Durfee, 2004).

As such, understanding the ways in which scholarly publications about the opioid epidemic are presented to the public is critical to the larger understanding of how the news media are shaping public opinion on the subject. Importantly, doing so requires considering news media stories beyond dedicated health or science coverage. While such dedicated coverage offers one way in which publics can learn about such publications, news stories about broader topics may also play an important role. A survey by the Pew Research Center (2017) found that nearly half of US adults (49%) are “uninterested science news consumers” who access science news infrequently and usually by chance, and that only 16% of Americans intentionally consume science news a few times a week. Similarly, while specialized science publications are one way in which publics learn about science news, more generalized outlets appear to be a more important source for many readers (ibid.). As such, news stories focused on broader issues, such as policy development, economics, or personal experiences, may be better aligned with news audiences' existing habits and preferences than specialized science journalism coverage. Such issue-focused stories may thus be one effective media strategy “for ‘going broad’ with science-related content,” a recent recommendation for science communicators wishing to “generat[e] attention and interest among non-elite audiences” (Nisbet and Scheufele, 2009, p. 1775). However, to our knowledge, they are not often examined in media framing studies of science and health topics. The current study seeks to address this knowledge gap and extend the currently limited knowledge of how popular news media portray scholarly publications by examining the framing of an important but understudied topic: the prevention, harm reduction, and treatment of opioid-related disorders. To do so, it examines the framing of scholarly publications in both issue-focused and science communication stories, relying on three uncertainty-related frames that have been previously identified in studies of scientific news coverage, each of which is outlined below.

Valid Science Frame

Antilla describes the valid science frame as “not discuss[ing] skepticism of the research” (2005, p. 344). These news stories do not convey any opposing views or doubts toward the research findings or topic, such that the authors of the study in question act as the “primary definers” (ibid.) of the issue at hand. Ruhrmann et al. characterize the valid science frame by “its above-average frequency of depicting scientific certainty” (2015, p. 10). That is, scientific information is communicated using a neutral tone, without controversy, discussion, or evaluation of risks or benefits. Finally, Guenther et al. propose that framing science as valid and credible is characterized by “certain single result(s), sufficient data, (strong) methodological quality, results pointing in the same direction, successfully replicated findings, […and/or] highly experienced researcher(s)” (2019, p. 50). As part of their analysis, Guenther and colleagues also recorded relevant criteria to assess scientific evidence, such as information about the authors, publication venue, study design, sample sizes, or funding sources. The valid science frame was found to be prevalent in news coverage of an array of science topics, from climate change to molecular medicine (Antilla, 2005; Ruhrmann et al., 2015) and was most commonly found in news stories published within dedicated science sections (i.e., as opposed to general interest stories; Guenther et al., 2019). Prevalence rates vary by topic and study, but findings suggest that the valid science frame appears in about 25 to 70 percent of news stories covering science (Antilla, 2005; Ruhrmann et al., 2015; Guenther et al., 2019).

Uncertain Science Frame

The uncertain science frame tends to emphasize “insufficient data or some uncertainties associated with the research” (Zehr, 2000, p. 92). Likewise, Guenther et al. describe the uncertain frame as privileging “preliminary data, knowledge gap[s], [and/or poor] methodological quality” (2019, p. 50). In their study of stem cell media coverage, Nisbet et al. (2003) blend elements of the uncertain and controversial science frames into a single “Uncertain” frame. This frame includes a “focus on scientific uncertainty over efficacy or outcomes of stem cell–related research and applications,” but also incorporates elements that are not specifically related to the framing of the research itself, such as uncertainty surrounding the viability of stem cell lines and disputes about the benefits of using embryo stem cells. In the context of climate change, meanwhile, Stecula and Merkley (2019) operationalized the uncertainty frame as shedding doubt on the veracity of the Intergovernmental Panel on Climate Change (IPCC) climate consensus, by referring, for example, to a lack of understanding about the issue at hand. Importantly, neither Nisbet et al. (2003) nor Stecula and Merkley (2019) investigated the framing of scholarly publications specifically, focusing instead on the framing of a scientific topic more generally. Despite this limitation, aspects of both studies can be used to understand the uncertainty frame in more detail. This frame appears to be slightly less prevalent than the valid science frame, occurring in between 1 to 25 percent of news stories (Nisbet et al., 2003; Antilla, 2005; Stecula and Merkley, 2019).

Controversial Science Frame

Finally, previous research has defined the controversial science frame as one that emphasizes conflicts and disagreements. Zehr posits that this frame often relies on a “practice of interjecting and emphasizing controversy or disagreement among scientists” (2000, p. 90). Antilla (2005) expands on this definition, arguing that controversy can also be mediated by highlighting disagreements between scientists and those outside of academia, such as industry leaders. She remarks that, in the context of climate change, news stories employing a controversial frame often “included rhetoric from climate skeptics with known fossil fuel industry ties” (Antilla, 2005, p. 347) that called the research findings into question. Ruhrmann et al. describe this frame as presenting “conflicting scientific results, which journalists…discuss with above-average frequency” (2015, p. 10). Like Nisbet et al. (2003), Guenther et al. do not explicitly identify a controversial science frame, instead characterizing stories that highlight “contrasting findings of research, contrasting interpretation of [the] same data-set, [or] conflicting viewpoints of researchers” (2019, p. 50) as uncertain. While not every study explicitly reported the prevalence of the controversial science frame, it appears to be present in between ~16 and 27 percent of news stories (Antilla, 2005; Ruhrmann et al., 2015).

Although uncertain and controversial science frames are sometimes treated as one and the same, research suggests they affect audiences differently; specifically, while reading a news story that frames scientific findings as uncertain appears to have minimal negative effects (and may even improve public perceptions about scientists and medical professionals), encountering a controversial science frame has been found to decrease news readers' beliefs in scientific claims and judgments about the credibility of the message source (Jensen et al., 2011; Gustafson and Rice, 2019). As such, we treat the two frames as distinct for the purposes of this study.

Specifically, we build on previous work examining the portrayal of scientific research in the news to explore the following three research questions:

RQ1. How often was research on opioid-related disorders mentioned in online news stories published by popular American and Canadian news media between 2017 and 2018?

RQ2. What contextual information do journalists provide when referencing scholarly publications about opioid-related disorders?

RQ3. How are scholarly publications about opioid-related disorders framed?

Methodology

Sample Selection and Collection

To investigate the framing of scientific research studies about opioid-related disorders in particular, as opposed to the opioid crisis more broadly, we needed to identify news articles (hereinafter “stories”) that mentioned or linked to relevant scholarly publications (hereinafter “studies”; see Figure 1, below). To do this, we first identified relevant scholarly publications as relating to the prevention, harm reduction, or treatment of opioid-related disorders by querying PubMed, a database for biomedical and life sciences research. PubMed indexes publications according to Medical Subject Headings, or “MeSH terms,” which provide a consistent way of cataloging and searching the database when authors use different terms for the same concept or topic (e.g., opioid addiction and heroin addiction). The MeSH term “opioid-related disorders” (U.S. National Library of Medicine), includes abuse, addiction, and dependence on any kind of opioid or opiate (e.g., heroin, morphine, codeine), from which we selected the six subtopics that aligned most closely with prevention, harm reduction, and treatment efforts: diagnosis, drug therapy, prevention and control, rehabilitation, therapy, and epidemiology. These subtopics became the basis of our final query, to which we added the MeSH term “needle-exchange programs” (found under the sub-heading “Preventative Health Services”) due to their importance as a harm reduction strategy that is not covered by any of the other search terms (Fernandes et al., 2017). This search yielded 12,166 relevant research studies.

FIGURE 1

Figure 1. Sample size and conceptualization of studies, references, and stories. In this study, we define “studies” as scholarly publications referenced in news stories; “stories” as news articles that implicitly or explicitly refer to research studies; “references” as those parts of the news story that relate to the research study.

Next, we searched the Altmetric Explorer for news stories mentioning or linking to any of the scientific studies identified by our search (May 20, 2019). The Altmetric Explorer is a tool built for exploring the Altmetric database, which tracks online activity and news mentions of scientific research outputs by identifying direct links to publications (i.e., a hyperlink or a publication identifier, such as a DOI) and by mining news texts for details such as author names, journal titles, and study timeframes (Altmetric.com, 2015). Since the Altmetric Explorer allows for PubMed queries, we were able to use the same search terms as above to find news stories that included research references. We define research references as those parts of a news story that clearly relate to a relevant, identifiable scholarly publication (e.g., a hyperlink to a publication and/or sentences, paragraphs, quotes discussing its results or implications). We limited our query to the most widely used news sources by Canadians and Americans (Alexa Internet, n.d.)—BBC, Breitbart, CBC, CNN, Fox News, Global News, the New York Times, Vice, and the Washington Post—and to news stories published in 2017 and 2018. Following this approach, we identified 212 news stories that included 310 references to 175 unique studies about opioid-related disorders (a story could include more than one reference to research). We collected and read all 212 news stories to verify each research reference manually. During this process, we removed 86 research references (27.7%): 35 duplicates (references identified by Altmetric more than once); 34 references in stories not written in English; 11 references in stories that were no longer available online; and 6 mismatched references (i.e., the reference in the news story was not identical to the one identified by Altmetric). Our final dataset consisted of 149 news stories that included 223 mentions of 164 unique studies.

To understand what kind of studies received media attention, we recorded basic background information about each study in our sample using data provided by Altmetric. Specifically, for each reference, we noted the study's digital object identifier (DOI) as well as the publication venue and year. Based on the DOIs, we also determined whether each study was freely available on the publisher website, in an institutional repository, or not publicly available. This was done using Unpaywall, a database containing the open access status of publications (Piwowar et al., 2018).

For further context, we queried the Media Cloud Explorer, an open-source platform for media analysis (MIT Center for Civic Media and Berkman Klein Center for Internet and Society)¹, to find out how the number of news stories mentioning scholarly publications compared to the overall volume of news coverage on the opioid crisis and opioid-related disorders during the same time frame. We set the search parameters to the same nine news sources and timeframe as before, using the following search terms: “(opioid OR heroin OR morphine) AND (addiction OR abuse OR epidemic OR crisis OR dependence OR overdose OR overdoses).” This search resulted in 9,565 news stories from which we removed 965 duplicate news stories, leaving a total of 8,600 stories.

Content Analysis

Since we were interested in the framing of scholarly publications in particular, we manually extracted only those parts of the news story that directly referred or linked to one of the studies in our sample. Following a directed content analysis approach (Hsieh and Shannon, 2005), we coded these extracts (hereinafter “references”) to identify which frames journalists used most often when covering research on opioid-related disorders. We also coded whether the news stories included any bibliographic information about the study, such as author name(s), affiliation(s), or publication venue as well as any details about the study design, such as sample size, timeframe, or other relevant details, such as the type of data (e.g., face-to-face interviews, nationally representative survey data), sample specifications (e.g., gender, age), geographical location (e.g., Rhode Island), or details about the analytical methods (e.g., cost-benefit analysis). Zooming out to examine each reference in the broader context of the news story, we determined the thematic focus of the news story—that is, whether it primarily reported on research findings or on some broader issue—as well as whether the story had first been published by the news source itself or by some other outlet (i.e., aggregate content). Finally, we noted whether each story included a link to the research study it mentioned.

To perform the frame analysis, we applied a deductive approach that relied on a detailed coding instrument (available in the Supplementary Materials) which was developed by drawing from previous research investigating valid, uncertain, and controversial science news frames (Zehr, 2000; Antilla, 2005; Ruhrmann et al., 2015; Guenther et al., 2019) and refined for the topic at hand. While deductive framing analysis may overlook certain frames that have not been identified in previous research (Matthes and Kohring, 2008), what the method lacks in flexibility it makes up for in terms of reliability, validity, and efficiency; relative to inductive approaches, deductive framing analysis tends to be easier to replicate and validate, can be applied to larger samples, and can more easily detect differences between media outlets (Semetko and Valkenburg, 2000; Van Gorp, 2010).

The three frames presented earlier, and included in the codebook, were operationalized as follows:

Valid Science

We defined the valid science frame as presenting scientific research as sound and trustworthy. Specifically, we proposed that this frame omits any potential scientific uncertainty and controversy regarding the study, such as poor methodological quality or contrasting research findings. We operationalized this frame using two indicators: portrayal of scholarly publications as trusted facts and an emphasis on positive scientific credibility. When journalists present research findings as trusted or established facts, they often do so without further evaluation or discussion (cf. Antilla, 2005; Ruhrmann et al., 2015). At the extreme, this can manifest as a news story that does not directly refer to a study at all, but merely cites it in the form of a hyperlink. In doing so, the findings are represented as established “truths,” rather than as outcomes of a scientific study with limitations requiring deeper consideration. Moreover, journalists can allude to positive scientific credibility by mentioning researcher affiliations or journal titles (cf. Guenther et al., 2019).

Uncertain Science

The uncertain science frame, on the other hand, presents scientific results as tentative or requiring additional research (cf. Zehr, 2000; Nisbet et al., 2003; Guenther et al., 2019). It cautions against viewing the research findings as trusted facts, often by raising doubts about the scientific credibility of the research results or the researchers themselves, emphasizing conflicts of interest (e.g., industry funding) or methodological shortcomings, such as small sample sizes (Zehr, 2000; Guenther et al., 2019; Stecula and Merkley, 2019). We operationalized this frame through two indicators: emphasis on preliminary findings and questioning of scientific credibility.

Controversial Science

Finally, the controversial science frame was conceptualized as injecting or emphasizing controversy and disagreement surrounding the reported study, either from within or beyond the academic sphere. We operationalized this frame with two indicators: contrasting research findings (Zehr, 2000; Ruhrmann et al., 2015; cf. Guenther et al., 2019) or conflicting viewpoints about the topic at hand (cf. Antilla, 2005). Contrasting research findings highlighted conflicts from within the scientific community, such as when research findings departed from those of other studies. Conflicting viewpoints, on the other hand, originated outside of the academy, such as when scientific consensus did not translate into political consensus or when a personal anecdote presented in a news story differed from the research findings.

Using Nvivo 12 Pro, each research reference was coded for the presence of each of the six indicators by one of the authors (LM; see Table 1, below), as per Syed and Nelson (2015). If indicators of multiple frames were present, we determined the dominant frame as follows: controversy superseded uncertainty, and both superseded validity (cf. Antilla, 2005). That is, stories could only be deemed uncertain if there was no indicator of controversy and would be deemed valid only if the research reference did not highlight any uncertainties or controversies. We chose this hierarchy because previous research suggests these frames affect audiences differently. Kortenkamp and Basten (2015) show that news readers rated discredited scientists (e.g., those portrayed as having a conflict of interest) as less credible by readers than non-discredited scientists. Moreover, presenting scientific certainty, or validity, can increase the audience's interest in science, whereas uncertainty does not (Retzbach et al., 2013). While an emphasis on uncertainty seems to have minimal effects, controversy can negatively affect health behaviors, beliefs in scientific consensus and in scientific claims more generally, and judgments about the credibility of the message source (Jensen et al., 2011; Kienhues et al., 2011; Dixon and Clarke, 2013; Gustafson and Rice, 2019).

TABLE 1

Table 1. Frame operationalization.

While coding for the frames of each reference, we also noted whether the news story included any bibliographic information, such as author name(s), affiliation(s), or publication venue, as well as information about the study design or process, such sample size, timeframe, or any other details pertaining to the research (e.g., type of data, methods, sample specifications).

We assessed inter-coder reliability using a second coder (AF) and random subsamples at various stages of the coding process (Bennett et al., 2008, p. 206). For these subsamples, we first itemized all references in our sample (1–226) and then used a random number generator to select references (Urbaniak and Plous, n.d.). After developing the initial codebook, both coders coded a subsample of 15 references; we discussed disagreements and revised the codebook to better reflect nuances between individual codes and to separate them more clearly. We repeated this process once more before coding the full sample. Table 2 presents the results of the final inter-coder reliability test, which consisted of 45 research references (20% of the full sample) that were coded by both coders independently. We measured inter-coder agreement using Krippendorff's Alpha, which allows for any number of coders or variables and corrects for small sample sizes (Krippendorff, 2004). Our results were above the generally accepted threshold of ·7 (Lombard et al., 2002), with most codes exceeding ·85. Agreement was lowest for “positive credibility” and “conflicting viewpoints”; after reviewing the results for these two codes, we realized the discrepancies did not come from actual disagreement between coders but from a lack of clarity in the codebook. The codebook was revised accordingly and used for the rest of the analysis.

TABLE 2

Table 2. Inter-coder reliability.

After determining which frames journalists used to present the individual references, we coded for certain characteristics of the news stories. First, we categorized the news stories in our sample according to their broad thematic focus—science communication or another issue. Science communication stories primarily focus on reporting scientific research whereas issue-focused stories place the primary focus on a larger issue, in which a scholarly publication is referenced, but not the main focus of the story. Prioritizing the thematic focus of the news story over style, we classified “health communication” stories (Schiavo, 2011) accordingly as either science communication or issue-focused and categorized editorial and opinion pieces as issue-focused stories (Bader, 1990). For this part of the analysis, we used only the headline and the first paragraph, as this is where journalists typically present the main topic of a news story (Pöttker, 2003; Burscher et al., 2016). Second, we coded whether the news story was an original story—that is, it had been first published by the news outlet itself—or a republished story that had been previously published by another source, such as a press agency. For this part of the analysis, we checked the beginning and header as well as the end and the footer or the news story, depending on where the news source listed authorship information. Third, we coded whether the news story contained a link to the research study, either within the body of the news story or at the end of it.

Statistical Methods

Throughout the results we present the result of various binary logistic regressions to estimate the relative probability of various outcome variables (e.g., that references are framed as valid or that they are accompanied by bibliographic or study details) in relation to various predictor variables (e.g., that stories are focused on science communication or that they have not been reposted from elsewhere). The calculated models are all in the form (Y = 1) = β₀ + β₁x, where Y is a binary outcome variable coded as 1 if at the studied variable was observed for a reference, and 0 otherwise, and where x is the predictor, coded as 1 if the story where the reference is observed is classified according to the variable in question, and 0 otherwise. Calculations were performed using the Python statsmodels package (Seabold and Perktold, 2010).

Results

How Often Was Research on Opioid-Related Disorders Mentioned in News Stories?

From 2017 to 2018, the nine news sources collectively published 149 news stories that included 223 references to 164 unique studies. Comparing this number to the results of the Media Cloud query for news stories from the same sources that broadly relate to the opioid crisis (n = 8,600), these results indicate that less than two percent stories mentioned or linked to scientific research on the prevention, harm reduction, and treatment of opioid-related disorders.

In relation to the published research on opioid-related disorders, the proportion of studies that get mentioned by the news media is similarly low. Of the 11,028 records that matched our PubMed query for research records corresponding to the prevention, harm reduction, and treatment of opioid-related disorders in January 2019, the majority of scientific studies were not referenced by any of the selected news outlets in 2017 or 2018, and only 164 studies (1.5%) appeared in at least one of news story in our sample. Of these, 129 (78.6%) studies were referenced only once, 22 (13.4%) twice, and 13 (7.9%) three or more times.

The most recently published studies received the most coverage, with more than half of the references published between 2016 and 2018 appearing in at least one story (n = 84, 51.2%). Over a quarter (n = 46, 28%) of the studies were free to read at the publisher's website, and an additional 36 (22%) were freely available in an institutional or subject repository. In total, slightly more than half of the studies (n = 84, 51%) were freely available for journalists and the public to read.

Not every news source in our study referenced scholarly publications to the same degree or in the same way. We examined the number and proportion of references found in science communication (as opposed to issue-focused stories) and original news stories (as opposed to reposted stories), as well as references framed as valid science or with at least one bibliographic or one study design detail (Table 3). However, since these differences cannot be meaningfully studied with this sample, the remaining analysis is performed using the collection of stories and research references as a whole.

TABLE 3

Table 3. Research references by news source.

What Contextual Information Do Journalists Provide When Referencing Scholarly Publications About Opioid Related Disorders?

Most references to studies about opioid-related disorders appeared in the context of a larger news issue (i.e., issue-focused stories, n = 170, 76.2%), while only a quarter of them (n = 53, 23.8%) appeared in stories dedicated to science communication. The vast majority of the references appeared in original stories (n = 188, 84.3%), with only 15.7% of references (n = 35) appearing in stories that had been previously published by another source. Most references included a hyperlink to the original study (n = 192, 86.1%). However, in more than 70% of republished cases (n = 25), the original news story included an outbound link to the study, but the republished version did not.

The majority of references did not include bibliographic information about the research study—such as author names, affiliations, or journal titles—which can function as heuristic cues for scientific credibility and enable readers to locate the original study (Table 4). One in four research references included author affiliations (n = 56, 25.1%), one in five named the authors (n = 45, 20.2%), and 39 (17.5%) named both. References that noted journal titles were equally rare (n = 43, 19.3%). Likewise, most references did not provide any information about the research design or process, and only reported the findings (n = 187, 84%; Table 5). Merely 36 references include any details about the study design, most frequently sample sizes (n = 27, 75%) or study timeframes (n = 19, 52.8%). Various other details, such as the methodological approach or sample specifications, were equally rare (n = 20, 55.6%). Generally, such references provided only one or two study details (n = 16, 44.4% and n = 13, 36.1%, respectively), and more often appeared in science communication stories (stories: n = 26, 52%; references: n = 26, 49%) than in issue-focused ones (stories: n = 9, 9%; references: n = 10, 6%). A logistic regression model confirms that mentions found in science communication stories were statistically more likely to be accompanied by both bibliographic details (odds ratio [OR] = 11.14, p < 0.001) and study details (OR = 13.92, p < 0.001) when compared to mentions found in issue-focused stories.

TABLE 4

Table 4. Research references that provided bibliographic information about the study, grouped by frame.

TABLE 5

Table 5. Research references that included details about the research design and methodology, grouped by frame.

How Are Scholarly Publications About Opioid-Related Disorders Framed?

The vast majority of the 223 research references in our sample framed science as valid (n = 184, 82.5%). The uncertain (n = 17, 7.6%) and controversial (n = 22, 9.8%) science frames appeared much less frequently (see Tables 4, 5).

Valid Science

The majority of the references in our sample framed science as valid—that is, presented the research as trustworthy, certain, and credible. For example, Rao (2017) informs Vice readers about a new study (Martins et al., 2017) that “looked at the rising number of heroin users in the country, and [found] most of them, again, are middle-aged white men without a college education.” Rao presents the research findings as trusted facts and gives no reason to question the validity or credibility of the science. Rao does this by including several credibility markers, such as the lead author's institution and the journal the study was published in. Notably, the news story omits any limitations or uncertainties about the study and instead provides quotes from the lead author of the study with a possible explanation of the findings. In Rao's story, a study author offers their view on the racial dimension of the opioid crisis by stating that “doctors are more likely to prescribe painkillers to white people, since there is still a stigma that minority patients will abuse drugs [and that this] has actually made it easier for white Americans to get addicted to opioids simply because of access” (Rao, 2017).

Many of these news stories simply linked to the study without elaborating on it further. This applied to 79 (42.9%) of 184 references with a valid science frame. We consider these instances as valid science because this journalistic strategy eliminates questions of doubt or uncertainty and instead uses scientific research as an authoritative source to verify facts and claims presented in the news story. Such references appeared to be useful when journalists included statistics, such as “drugs that are now killing more people than AIDS did at its peak” (Szalavitz, 2017c) or “France reduced heroin overdoses by nearly 80 percent by making buprenorphine easily available starting in 1995” (The Editorial Board, 2018). They were also used to provide examples, such as “syringe vending machines have existed in Berlin since 1988 and have been in France, Australia and Puerto Rico for years” (Farah, 2017). In other cases, these hyperlinks served to support the journalist's statements in the simplest way possible: by offering curious readers opportunities to read additional resources regarding elements of the story that are of particular interest to them without detracting from the main narrative of the story.

We view these instances, however, as a distinct type of the valid science frame, because, unless readers choose to hover over or click on these links, it remains unclear to them whether the link leads to a scholarly publication or to some other source. In more than half of these cases, the hyperlinked text consisted of only one or two terms (n = 45, 57%), making it almost impossible to guess what kind of content the link referred to. For example, Radcliffe (2018) writes (all links underlined, links to studies also in bold): “It's also important to remember that relapse isn't a failure. People with an opioid use disorder will often relapse along the road to recovery.”

This phenomenon was particularly evident in cases where links to studies were located near links leading to non-scholarly resources. Such was the case in the story “A huge sigh of relief on health care” by Garrett (2017):

A call to insert provisions allowing employers to conduct genetic tests on job applicants, eliminating all funding for Planned Parenthood, questioning the wisdom of “forcing” men to pay for insurance coverage of maternity care, childbirth and newborn services, undermining access to treatment for opioid addiction, telling poor Americans that they need to choose between buying the latest iPhone and purchasing insurance—these and countless more tone-deaf statements and initiatives from Republican politicians had the nation crying foul.

Other references provided some indication that the hyperlink might lead to a scholarly publication, using phrases like “research” or “study” (n = 105, 58.7%). See for example, this excerpt from Szalavitz (2017b), which weaves six research references into just two sentences: “In the case of MAT [medication-assisted treatment], the “assisted” part is even more absurd because research shows that requiring counseling and other hurdles like frequent urine testing adds little additional benefit beyond that of the medication itself. Even adding super-intensive services does not make a real difference: Fundamentally, what's most important is the medication, according to a Cochrane Review, a strict analysis of the data.”

Strikingly, the valid science frame predominantly presented research findings without context (n = 80, 43.5%). By omitting information about the study design, the valid science frame conveyed validity by implying broad scientific agreement (n = 33, 17.9%)—like Frakt's story 2017, which points to “several systematic reviews and meta-analyses of therapies for opioid addiction found that methadone therapy reduced criminal activities related to heroin use”—and by emphasizing scientific credibility by stating author affiliations (n = 42, 22.8%) or journal titles (n = 30, 16.3%). Only a few valid science references noted author names (n = 33, 17.9%) or included comments by independent experts (n = 6, 3.2%).

Only 25 (13.6%) references provided further details about the study, such as information about the sample size (n = 21, 84%) or timeframe (n = 14, 56%). Of those references that did include contextual details, 16 (64%) appeared in science communication stories, like “Opioid use before surgery may lead to problems after: Study,” a syndicated news story originally published by UPI (2017) that informs readers that the reported study was based on “200,000 middle-class people who had hysterectomies, bariatric surgery, hernia repair and reflux surgery and spent at least one night in the hospital over a 42-month period.” While, as we noted above, references found in science communication stories were generally more likely to contain study details, these same stories cannot be said to be more or less likely to frame scholarly publications as valid. That said, references in science communication stories were significantly less likely to be presented as just a link, without any additional information, when compared to issue-focused stories (OR = 0.30, p < 0.01).

Uncertain Science

The uncertain science frame—which portrays scholarly publications as tentative and cautions against viewing study findings as trusted facts—was the least common frame in our sample (n = 17, 7.6%). About a third of all instances of this frame occurred within a single sentence (n = 5). These references appeared most frequently in issue-focused stories that were void of details about how the study was conducted or who conducted it (n = 12, 70% of uncertain references). Only science communication stories provided additional information about the study background or design (n = 6, 35.3%), or included comments by the authors or independent experts (n = 5, 29.4%).

Uncertainty was most often conveyed by noting a lack of existing research (n = 7, 41.2%). For example, in a story discussing the effectiveness of jail-based methadone treatment, Williams (2017) cautions that “few studies have measured the outcomes of” these treatments, however “a 2014 Australian study found fewer overdose deaths after release.” Similarly, Blaszczak-Boxe (2017) notes that “more research should be done” to investigate the potential benefits of marijuana for opioid withdrawal and that “one study ‘is not sufficient to make sweeping conclusions about the potential efficacy of CBD to inhibit heroin craving and drug use in addicted individuals.’”

In other cases, stories questioned the scientific credibility of the studies themselves (n = 6, 35.3%). For example, Szalavitz (2017a) introduces uncertainty by noting potential issues with the sample size and funding sources, writing that “five mostly small and industry-funded clinical trials have found Vivitrol to be effective in treating opioid problems—at least for those who choose to stick with it.”

Examples such as these shed doubt on the certainty of the scholarly publications presented, potentially decreasing publics' intentions to support or try new treatments for opioid dependence (e.g., Vitriol, methadone). However, the uncertainty frame can also function as a warning and a call for greater awareness for unintended consequences of drug misuse, as in Lieber's (2018) reference to a case report of “two patients who died after taking large doses of loperamide [an over-the-counter medication for diarrhea] in an effort to curb their opioid withdrawals” suggests. In this story, Lieber notes that “the authors speculated that the deaths resulted not from the opioid itself but from other ingredients in the medication that are toxic to the heart.”

Controversial Science

One in 10 research references framed science as controversial (n = 22), highlighting conflicting viewpoints about the research study or its implications. This frame appeared most often in issue-focused news stories (n = 17, 77.3%).

In some cases, controversy was conveyed by highlighting conflicting viewpoints within the scientific community, for example, by noting that study findings departed from previous research, or that research studies had produced mixed results (n = 8, 36.4%). This can be seen in Rogers (2017):

As public health and addiction experts scramble to find a way to curb this epidemic, a new debate is emerging over what, if any, role cannabis can play in preventing addiction, and helping addicts recover. Some argue it can provide relief of the chronic pain that often leads people to opioids in the first place, while other worry it could cause recovering addicts to fall off the wagon… Women who reported using recreational cannabis while in MMT [Methadone Maintenance Therapy] were 82 percent more likely to also use illicit opioids, according to Zielinski's study, published last month in Biology of Sex Differences… Other researchers have found that cannabis use has no association with how well someone manages to stay off opioids while receiving MMT.

In other cases, the controversial science frame emphasized conflicting views about the study from beyond the academic sphere, much like in Antilla's work on climate change research (2005). Nearly one in six references that used the controversial frame highlighted differences between the views of scientists and other actors, such as parents or medical practitioners. While these “other actors” were not policymakers, we combined both reference groups in our analyses, as both represent conflicting opinions from outside of the scientific community, not from within it. These instances most often highlighted disagreements between scientists and policymakers (n = 10, 45.5%). Russell (2018), for example, pits Canadian Health Minister Christine Elliot's views against those of “experts and advocates”:

“There is evidence on both sides,” Elliot said during question period Monday. “We need to make sure that we review all of the evidence to understand what is happening. What is happening that is saving lives? What else can we do to save more lives? Are there other examples that we should be looking at besides supervised injection clinics?” Experts and advocates on the frontlines of the opioid epidemic, however, say there is already an overwhelming body of evidence to show that the overdose-prevention and safe injection sites save lives and help get people into treatment… One study published in 2011 examined overdose mortality in the two years before and after a supervised injection site opened in Vancouver in 2003 known as InSite. It found the fatal overdose rate in the area decreased by 35 per cent.

References such as these, which emphasize conflicting findings about the effectiveness of treatment and harm reduction strategies for opioid-dependence, could have similar effects on audience beliefs and intentions as the uncertain science references discussed above. Yet, given that the controversial science frame has been found to be more influential than the uncertain science frame (Jensen et al., 2011; Kienhues et al., 2011; Dixon and Clarke, 2013; Gustafson and Rice, 2019), we would expect these effects to be amplified. For example, these references could negatively affect public beliefs in the effectiveness of cannabis use in opioid-dependence treatments or supervised injection sites as a form of harm reduction, as well as support for related public health policies.

Across the various forms of controversy, stories seldom included details about the mentioned study (n = 6, 27.3%; Table 4). Details were, however, more common when the controversy occurred within the scientific community (n = 4, 60%), rather than between the scientific community and outside actors (n = 2.20%).

Discussion

The primary aim of this study was to use research about opioid-related disorders as a case study to better understand how popular online news media outlets frame and contextualize scholarly publications in their stories. Our results indicate that, at least regarding opioid-related disorders, research is not frequently referenced but that, when it is, it is most often framed as valid science. Notably, this framing is most often achieved not by highlighting scientific credibility or the rigor of the research methods, but by omitting study details and other contextual information—sometimes by merely providing a link to the published study. Finally, we find that research on opioid-related disorders is more often included in issue-focused stories than in science communication stories—referenced in passing rather than as the focus of the story itself.

Despite the severity of the opioid crisis, we found that only a small proportion of available research on opioid-related disorders received coverage in major news media. In 2017 and 2018, the online news outlets we analyzed referred to relevant research in less than two percent of their opioid-related news stories. The relatively low level of coverage, as well as the limited use of research references in the news stories, is perhaps unsurprising considering various disruptions in the media environment in the last two decades—technological advances, novel ways of news production and consumption, and sinking ad revenues (Waldman, 2011; Anderson et al., 2016; Barthel, 2017)—that have led to the decline of specialized reporting, including science journalism (Dunwoody, 2014; Schäfer, 2017). Still, one might expect that research studies would feature more prominently in news stories covering a public health crisis as urgent as the opioid epidemic, especially since sharing expert knowledge has been shown to play an important role in shaping public opinion, informing policy, and guiding effective interventions in this area (Lawrence et al., 2000; McCann, 2008; Tieberghien, 2014).

When the news outlets we studied did mention scholarly publications, we found that online news journalists overwhelmingly portrayed them as valid science, implying that results were credible and trustworthy. These findings align with research exploring the news media's framing of health and science issues, which has previously documented the prevalence of the valid science frame relative to the uncertain and controversial frames (Antilla, 2005; Ruhrmann et al., 2015; Guenther et al., 2019) as well as a general tendency for journalists to minimize uncertainty when reporting on science (e.g., Dumas-Mallet et al., 2018; Ponce de Leon et al., 2019) and uncritically report messages found in peer-reviewed journal articles (Bubela and Caulfield, 2004; Caulfield, 2004). Our study demonstrates that such practices prevail despite the politicized nature of the opioid crisis and the evolving research into new drug treatments and prevention measures (Bell et al., 2018; Carew and Comiskey, 2018; Crist et al., 2018; Koller et al., 2019; Volkow et al., 2019). While the opioid crisis itself remains controversial among Americans and Canadians (Angus Reid Institute, 2018; Blendon and Benson, 2018), our results suggest that scholarly publications surrounding the crisis do not—at least from the perspective of the nine major news outlets analyzed in this study. Potential reasons for this are the different demographic groups affected by the opioid crisis, which vary in class and race compared to previous drug epidemics, and the availability of opioids as prescription medication, both of which could have an impact on how causes of and solutions to opioid abuse are defined (Hansen and Netherland, 2016; Shachar et al., 2020).

Scholars have often noted a lack of context—or even “hyping” —of scientific research findings in mainstream news coverage (Caulfield, 2004; James et al., 2016; Wilkins, 2016). We find evidence of a similar tendency in the news outlets we studied, with notable differences between the three frames we analyzed. Overall, opioid-related research was seldom contextualized, with few stories including details or bibliographic information about the studies they referenced. But the valid science frame, in particular, was often communicated by omitting details about the research referenced, rather than highlighting them. Journalists appeared to communicate scientific validity not by emphasizing the rigor of the methods (or lack thereof), but by simply reporting research findings without elaboration—at the extreme, providing nothing more than a link to the published study. The controversial science frame, by comparison, provided more details about the study methodology or the researchers behind it, but also emphasized divergent views on the results presented. References employing the uncertain science frame similarly included more contextual information than those using the valid science frame, but most often highlighted uncertainty by emphasizing the preliminary nature of the research. These findings align with Entman's (1993, p. 54) assertion that “Most frames are defined by what they omit as well as include,” while also extending existing research exploring media framing of scholarly publications by providing detailed insights into how the valid, uncertain, and controversial science frames are used in online news stories.

The implications of framing science as valid without providing context about the scholarly publication itself are unclear. Of course, from the journalist's point of view, omitting study details is often necessary; space for discussion is limited in online news media—be that because of word counts or readers' attention spans (Liu, 2005; Athreya and Mouza, 2017)—and few journalists have time to dedicate to providing in-depth coverage (MacLaughlin et al., 2018). Indeed, it is possible that the journalists authoring these stories may not have intended to frame the study as valid at all, but rather to provide a relatively “neutral” account of the findings. However, by leaving out information about the authors and research design, these issue-focused stories offer few opportunities for readers to critically engage with or evaluate the research presented to them. Instead, the lack of content has the effect, whether intended or not, of portraying the science as valid by encouraging readers to “trust or believe in the science without knowing what science” (Taylor, 2010, p. 234). Doing so could have the opposite effect and, instead diminish readers' trust in the news coverage (Jensen et al., 2017). Indeed, some degree of uncertainty is inherent to the scientific method itself (Priest, 2018), and as Gustafson and Rice (2019, p. 19) argue, “science communicators often should communicate uncertainties to maximize accuracy and follow ethical principles.” Regardless of whether presenting scholarly publications without being explicit about their inherent uncertainty influences trust in journalism (or in science more broadly), the implications of this journalistic approach need to be considered. Given that a majority of initial biomedical studies are later invalidated by other research (Dumas-Mallet et al., 2016), the news media's uncontextualized reporting of opioid-related research findings could lead to public support of health policies that promote experimental treatment or prevention interventions that have not yet been sufficiently tested. Conversely, portraying findings as tentative or preliminary when in fact they are supported by a larger body of research could perpetuate a lack of support for effective treatments (see, for example, McGinty et al., 2020).

Finally, we find that references to opioid-related scholarly publications are more commonly found in issue-focused stories than those dedicated to communicating the results of research, suggesting that news consumers often encounter scientific information when not actively seeking it out. This aligns with existing research documenting the ongoing decline of dedicated science news coverage (Dunwoody, 2014; Schäfer, 2017), as well as the high proportion of “uninterested science news consumers” who most often encounter science news by chance (Pew Research Center, 2017). On the one hand, embedding scholarly publications in coverage focused on broader issues, such as policy development, economics, or personal anecdotes, may be an effective way of engaging these “uninterested” audiences (Nisbet and Scheufele, 2009). (Indeed, this journalistic approach may be particularly appropriate for an issue such as opioid-dependence, which has social, economic, political, and cultural dimensions, as well as scientific ones). On the other hand, these issue-focused stories tended to provide even less context about the research they referenced than science communication stories. That is, unlike science communication stories, issue-focused stories seldom included details like author names, affiliations, or journal titles. They were also less likely to provide any details about the study design or methods, and more likely to provide only a link to the scholarly publications they referenced. These findings extend previous research on science reporting, which has similarly documented a lack of detail on how research studies are covered in mainstream news coverage (Pellechia, 1997; Zimmerman et al., 2001; Maillé et al., 2010; Schwitzer, 2013) by demonstrating key differences between journalistic practices in science communication stories and issue-focused stories, and how these differences relate to the way science is framed for public news consumers. While it was outside the scope of this paper to formally assess journalists' motivations to reference scholarly publications, journalists appeared to often use references to scholarly publications as a way to verify their claims or provide further information to an element covered in these issue-focused news stories, as has been seen in previous studies examining the use of hyperlinks in online health stories (Karlsson and Sjøvaag, 2018; Stroobant and Raeymaeckers, 2019).

Our study is not without limitations. First, although today's online news landscape is diverse—with variation between news outlets in terms of funding models, ideological orientation, audience profiles, and medium—we did not consider these differences in our sample selection or analysis. Instead, we present a broad overview of how scholarly publications about opioid-related disorders are framed by the most widely used news outlets in Canada and the US. To offer a more nuanced perspective and avoid potential biases, future research should choose news sources that are suitable for comparative analysis and explore how different source factors might influence the framing of opioid-related scholarly publications and other research more broadly. Second, by narrowly focusing our content analysis on references to scholarly publications, our study does not address how the effects of different frames might be influenced by or interact with the frames of the story as a whole, or the larger narrative that it is part of. Future studies should examine the effects of the three science frames on audiences' perceptions, attitudes, and behaviors, especially in the presence of intersecting frames. Likewise, more work is needed to explore the effects of providing, or not providing, details about the research design and process in news stories, especially as they pertain to issues of public concern, such as the opioid crisis.

Data Availability Statement

The data that support the findings of this study has been made openly available through the Harvard Dataverse at https://doi.org/10.7910/DVN/TAGFBL.

Author Contributions

The original idea for this study was conceived by LM who worked with AF on the final conceptualization and methodology under the supervision of JA. LM collected and curated the data. LM and AF developed the detailed coding instrument and carried out the formal analysis. JA secured funding and resources. All authors co-wrote the manuscript.

Funding

This research was supported by the Social Sciences and Humanities Research Council of Canada through an Insight Grant: Understanding the Societal Impact of Research Through Social Media (435-2016-1029).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

We thank Esteban Morales and Asura Enkhbayar for their assistance and valuable input during the research process, as well as Betsy Leimbigler, Laura Moorhead, Lauren Maggio, and Michelle Riedlinger for their helpful comments on previous drafts of the manuscript. Finally, we would like to thank Altmetric for access to the Research References data.

Footnote

1. ^MIT Center for Civic Media and Berkman Klein Center for Internet and Society Media Cloud Explorer. Available online at: https://explorer.mediacloud.org/#/home (accessed March 9, 2020).

References

Alexa Internet (n.d.). Top Sites in United States. Available online at: https://www.alexa.com/topsites/countries/US (accessed February 22, 2019).

Altmetric.com (2015). Altmetric. Available online at: https://web.archive.org/web/20190702125939/https://www.altmetric.com/about-us/ (accessed July 17, 2019).

American Psychiatric Association (2018). APA Public Opinion Poll – Annual Meeting 2018. Available online at: http://web.archive.org/web/20190721201304/https://www.psychiatry.org/newsroom/apa-public-opinion-poll-annual-meeting-2018 (accessed July 23, 2019).

Anderson, C. W. Jr., Downie, L., and Schudson, M. (2016). “How has digital technology been changing the news—and journalism?,” in The News Media: What Everyone Needs to Know (New York, NY: Oxford University Press), 62–66.

Angus Reid Institute (2018). Opioids in Canada: One-in-Eight Have Family or Close Friends Who Faced Addiction. Angus Reid Institute. Available online at: https://web.archive.org/web/20190723055249/http://angusreid.org/opioid-crisis/ (accessed July 23, 2019).

Antilla, L. (2005). Climate of scepticism: US newspaper coverage of the science of climate change. Glob. Environ. Change 15, 338–352. doi: 10.1016/j.gloenvcha.2005.08.003