Research with integrity – referencing work is way more than just a tick-box exercise

I once ran into a spot of bother with referencing.

During a brief course in the history and philosophy of science, being unfamiliar with both the style of footnotes and many of the authors, I took the short cut of adding references to my essay based on how they appeared in the footnotes of the books I was reading. I saw lots of references to ‘Ibid’ (who I assumed was one of the plethora of ancient Greek philosophers) and duly cited them as “Ibid, p. 365”.

I had a feeling this might not be quite right, but I was a bit too embarrassed to check because I assumed I was supposed to know how this worked already.

What I didn’t know until much later was that “Ibid” is just Latin shorthand for “in the same source” and it’s used to save space in footnotes when you’re quoting multiple times from the same manuscript.

This all came back to me recently when I was thinking about how researchers learn to reference while preparing tutorials for first year PhD students. Like so many things in research, what seems obvious once you know it can be completely incomprehensible until someone lets you in on the secret.

A surprisingly large number of papers continue to be cited after they have been retracted.

With that in mind, I wanted to share a few things related to referencing that I’ve found out over the last couple of years which would have been useful to me while I was still in the lab.

Avoiding accidently citing retracted papers

One way to preserve the integrity of the scientific record is the retraction of work that contains significant errors, fraud or other types of research misconduct. However, a surprisingly large number of papers continue to be cited after they have been retracted.

The website Retraction Watch keeps a record of the top 10 most highly cited retracted papers, some of which have been referenced more times after retraction than before (sometimes referred to as ‘zombie papers’). Similarly, a report in the Economist in 2021 found that the 20,000 papers in the Retraction Watch database had been cited in 95,000 articles after their retractions, and these articles were cited in 1.65m further papers. It also seems that many of these citations do not mention that the paper has been retracted.

There can be perfectly good reasons to cite a retracted paper (more on this later) but there are some useful tools to help avoid doing so by accident. I encourage researchers at the CRUK Manchester Institute to use the latest versions of reference managers Endnote, Papers or Zotero (which is also free and open-source) because they will automatically flag a paper that is known to be retracted. Using tools like this can help you as an author to make an informed decision about what to do next in relation to these works.

Referencing retracted papers appropriately

If you choose to cite a retracted paper (perhaps to reflect on how the field has moved on, or comment on an issue raised through the retraction), there are tips on how to do this over at Retraction Watch.

The core principle is to ensure the retraction is noted, and ideally to cite the retraction notice, which often has a separate digital online identifier (DOI). This might seem obvious, but case studies of retracted papers found mostly positive citations of these works, suggesting that those citing them may not have known they were retracted – or at least were treating the work in the same way they would before the retraction.

Standing on the shoulders of giants

This referencing of retracted papers highlights a wider point, which is that referencing is not neutral.  When researchers add a reference, they are conveying some degree of status on that research.  While much effort is being made to move away from reductionist metrics of judging scientific success, citations are part of the currency of promotion and grant success. Even as these metrics are improved, showing how your work relates to what has gone before will always be part of good scientific practice.

Therefore, adding a reference should not be a passive act, but one taken with consideration of its impact. This is particularly important in terms of representation in research. A study from 2021 – “Gender Disparity in Citations in High-Impact Journal Articles” – found that papers in a series of medical journals with women as the primary author were cited less often than those with men as the primary author, while articles with women as both primary and senior authors had approximately half the number of citations as those authored by men as both primary and senior authors. The authors suggest some reasons for this, including reduced visibility of female researchers, implicit bias, and higher levels of self-citation from male researchers.

A 2020 preprint suggests similar effects might be at work to account for a detected racial and ethnic imbalance in citations. One way researchers can engage with this is by taking some time to review reference lists to see whether they reflect the people who are active in your field, not just the most prominent or famous names.

Citation is not a neutral act

I recently came across this 2017 editorial in Nature Genetics – “Neutral citation is poor scholarship” – where the editors examined citations in recent papers. They found the most common type of reference was of a “neutral, flavourless or unexamined” kind, such as “this field exists (refs. 1–20)”.  While that might seem reasonable to represent a wide scope of activity, the piece argues that references should be used with more context and detail, showing how they fit into the argument being built in the paper, and appropriately reflecting their contribution to the field so far.

Footnote

Referencing can seem like a tick-box exercise but viewed the right way it becomes an opportunity to lend support to other researchers; engage in constructive debate about scientific literature; draw attention to work that might otherwise be missed; and credit those whose contributions have furthered your progress.

If it ever seems complicated or confusing, don’t be embarrassed about seeking advice from those around you – including library services and journal editors. As someone (but probably not Ibid) once said: “There are no stupid questions”.

Author:
Dr Andrew Porter is Research Integrity and Training Adviser at Cancer Research UK Manchester Institute.