AUTHOR=Tattersall Andy , Carroll Christopher 

TITLE=What Can Altmetric.com Tell Us About Policy Citations of Research? An Analysis of Altmetric.com Data for Research Articles from the University of Sheffield

JOURNAL=Frontiers in Research Metrics and Analytics

VOLUME=2

YEAR=2018

URL=https://www.frontiersin.org/journals/research-metrics-and-analytics/articles/10.3389/frma.2017.00009

DOI=10.3389/frma.2017.00009

ISSN=2504-0537

ABSTRACT=<sec id="ST1"><title>Background</title><p>There is a growing interest in using and analyzing altmetric data for quantifying the impact of research, especially societal impact (Bornmann, <xref ref-type="bibr" rid="B3">2014</xref>, Thelwall et al., <xref ref-type="bibr" rid="B28">2016</xref>, Haunschild and Bornmann, <xref ref-type="bibr" rid="B12">2017</xref>). This study therefore aimed to explore the usefulness of <uri xlink:href="http://Altmetric.com" xmlns:xlink="http://www.w3.org/1999/xlink">Altmetric.com</uri> data as a means of identifying and categorizing the policy impact of research articles from a single center (the University of Sheffield).</p></sec><sec id="ST2"><title>Method</title><p>This study has only included published research articles from authors at the University of Sheffield and indexed in the <uri xlink:href="http://Altmetric.com" xmlns:xlink="http://www.w3.org/1999/xlink">Altmetric.com</uri> database. Altmetric data on policy impact was sourced from <uri xlink:href="http://Altmetric.com" xmlns:xlink="http://www.w3.org/1999/xlink">Altmetric.com</uri> following a data request and included citations up until February 2017. Supplementary <uri xlink:href="http://Altmetric.com" xmlns:xlink="http://www.w3.org/1999/xlink">Altmetric.com</uri> data, including news media, blogs, Mendeley saves, and Wikipedia citations, were also gathered.</p></sec><sec id="ST3"><title>Results</title><p><uri xlink:href="http://Altmetric.com" xmlns:xlink="http://www.w3.org/1999/xlink">Altmetric.com</uri> data did enable the identification of policy documents that cited relevant articles. In total, 1,463 pieces of published research from authors at the University of Sheffield were found to be cited by between 1 and 13 policy documents. 21 research articles (1%) were listed as being cited in five or more policy documents; 21 (1%) in four policy documents; 50 (3%) in three documents; 186 (13%) in two documents; and 1,185 (81%) in one document. Of those 1,463 outputs, 1,449 (99%) were journal articles, 13 were books, and 1 was a book chapter (less than 1%). The time lag from the publication of the research to its citation in policy documents ranged from 3 months to 31 years. Analysis of the 92 research articles cited in three or more policy documents indicated that the research topics with the greatest policy impact were medicine, dentistry, and health, followed by social science and pure science. The <uri xlink:href="http://Altmetric.com" xmlns:xlink="http://www.w3.org/1999/xlink">Altmetric.com</uri> data enabled an in-depth assessment of the 21 research articles cited in five or more policy documents. However, errors of attribution and designation were found in the <uri xlink:href="http://Altmetric.com" xmlns:xlink="http://www.w3.org/1999/xlink">Altmetric.com</uri> data. These findings might be generalizable to other institutions similar in organizational structure to The University of Sheffield.</p></sec><sec id="ST4"><title>Conclusion</title><p>Within the limitations of the current text-mining system, <uri xlink:href="http://Altmetric.com" xmlns:xlink="http://www.w3.org/1999/xlink">Altmetric.com</uri> can offer important and highly accessible data on the policy impact of an organization’s published research articles, but caution must be exercised when seeking to use this data, especially in terms of providing evidence of policy impact.</p></sec>