It is amusing to observe the love-hate relationship that we, as scientists, appear to be developing with new generation artificial intelligence and especially with Large Language Models (LLMs) such as ChatGPT, Gemini and Claude.

LLMs are real-world manifestations of our geekiest childhood Sci Fi fantasies. Remarkably intuitive and easy to use, they appear to ‘understand’ our less-than-perfectly-formed questions. They can summarise and synthesize information for us, making the practice of dredging through hundreds of pages of PubMed or Google results seems unbearable. They allow us to explore any topic and can respond to us no matter our line of thought.

Let’s face it, they are a huge time and effort saver. Disillusionment rapidly sets in however when we experience their faults. From fake citations masquerading as real references to erroneous information presented with confidence. How many of us have recoiled when reviewing a paper harbouring the tell-tale junk of AI-writing; or receiving LLM-generated reviewer comments?

We are being swamped by AI-generated content that is endangering the very foundations of objectivity and empirically derived facts. For many of us, our Sci Fi dreams have morphed into nightmares. We become, rightly, suspicious of everything and increasingly discouraged.

Yet not all scientists appear to have the same negative reaction – as evidenced by the growing AI generated ‘information’ being spread in scientific circles.

Use the tools correctly…

Who is to blame? In my opinion, the blame lies squarely on our shoulders. The misuse of a tool is rarely the fault of the instrument itself but rather that of the wielder. Like with any tool, we must understand how they work, what their appropriate use is, and we must use them responsibly.

Traditional AI/machine learning algorithms – ubiquitous in science – are heavily constrained to specific inputs and outputs and fully supervised by humans. Foundation models such as LLMs, on the other hand, are given vast, unstructured, uncurated data.

Importantly, LLMs are not trained to be fact-retrieval algorithms. They are generative, probabilistic predictors of the next items in a sequence. They will generate (not retrieve) the most reasonable follow-up in response to prompts. Given they were trained on books, articles and general language constructs generated by humans, this follow-up will frequently be factually correct. But is not necessarily so, and not by design.

Humans behave much the same way. Complete the following sentence: “We finish each other’s …”. Probabilistically, the most likely word is ‘sentences’ unless you have a 6-year-old in your life in which case it will more likely be ‘sandwiches’. But perhaps in your specific context you meant ‘stories’.

Like human responses, we can guide LLMs to predict a more appropriate output by giving them context and constraints. In this setting, a fabricated but plausible citation provided by an LLM makes perfect sense. The concept of AI ‘hallucination’ is a human construct that we use to describe our surprise when we misuse AIs and get an answer we did not expect. In the example above, ‘sandwiches’ is not a hallucination, neither would ‘cupcakes’ be.

Responsible use will yield great results

LLMs and other foundation models provide powerful tools in our scientific armoury. We can use them to explore patterns in data that would be hidden in noise. We can use them to generate hypotheses that we can then test with appropriate controls.

Indeed, even in our everyday interactions with them, we must provide such controls. We must always question both ourselves, and the responses we are getting; can we verify information using an alternative method? Can we ask the LLM to provide a link to a primary source and verify that primary source ourselves?