Abstract:BackgroundCitation of research in policy documents has been suggested as an indicator of the potential longer-term impacts of research. We investigated the use of research citations in childhood obesity prevention policy documents from New South Wales (NSW), Australia, considering the feasibility and value of using research citation as a proxy measure of research impact.MethodsWe examined childhood obesity policy documents produced between 2000 and 2015, extracting childhood obesity-related references and coding these according to reference type, geographical origin and type of research. A content analysis of the policy documents examined where and how research was cited in the documents and the context of citation for individual research publications.ResultsOver a quarter (28%) of the policy documents (n = 86) were not publicly available, almost two-thirds (63%) contained references, half (47%) cited obesity-related research and over a third (41%) of those containing references used unorthodox referencing styles, making reference extraction laborious. No patterns, in terms of the types of documents more likely to cite research, were observed and the number of obesity research publications cited per document was highly variable. In total, 263 peer-reviewed and 94 non-peer-reviewed obesity research publications were cited. Research was most commonly cited to support a policy argument or choice of solution. However, it was not always possible to determine how or why individual publications were cited or whether the cited research itself had influenced the policy process. Content analysis identified circumstances where research was mentioned or considered, but not directly cited.ConclusionsCitation of research in policy documents in this case did not always provide evidence that the cited research had influenced the policy process, only that it was accessible and relevant to the content of the policy document. Research citation across these public health policy documents varied greatly and is unlikely to be an accurate reflection of actual research use by the policy agencies involved. The links between citation and impact may be more easily drawn in specific policy areas or types of documents (e.g. clinical guidelines), where research appraisal feeds directly into policy recommendations.

Science cited in policy documents: Evidence from the Overton database

Overton -- A bibliometric database of policy document citations

How relevant is climate change research for climate change policy? An empirical analysis based on Overton data

Societal and scientific impact of policy research: A large-scale empirical study of some explanatory factors using Altmetric and Overton

How many scientific papers are mentioned in policy-related documents? An empirical investigation using Web of Science and Altmetric data

Openness and Knowledge Sharing of Economics,Sociology,Law and Politics:A Comparative Study on Citations of Chinese Social Science Journals

Assessing the influence of research quality on policy citations: Quantitative analysis finds non‐academic factors more likely to influence how papers get cited in SDG policy

A Bibliometric Study on Sustainable City Based on Natural Science and Social Science Research

Binary black-hole evolutions of excision and puncture data

The citation advantage of foreign language references for Chinese social science papers

Global Performance and Development on Sustainable City Based on Natural Science and Social Science Research: A Bibliometric Analysis.

Bibliometric analysis of status quo and hotspots of health policy research under the concept of One Health based on CiteSpace

The citation impact of social sciences and humanities upon patentable technology

Policy documents as sources for measuring societal impact: How often is climate change research mentioned in policy-related documents?

Does citation matter? Research citation in policy documents as an indicator of research impact – an Australian obesity policy case-study

Bibliometrics Analysis Of Science And Technology Policy In China

Citation advantage of COVID-19 related publications

Highly cited articles in the Education and Educational Research category in the Social Science Citation Index: a bibliometric analysis

Does knowledge have a half-life? An observational study analyzing the use of older citations in medical and scientific publications

Do open access journal articles experience a citation advantage? Results and methodological reflections of an application of multiple measures to an analysis by WoS subject areas

Computer Science Papers in Web of Science: A Bibliometric Analysis