A radical project: video game design meets cancer research

What does a video game designer have to do with cancer research?  

Well, until recently, our answer to that question may have been quite different.   

But since 2017, when Professor Greg Hannon and his team of international researchers were awarded £20m to develop a new way for scientists to study tumours, the worlds of cancer research and virtual reality (VR) have collided. The result is unlike anything that has ever been seen in either field. Incredibly, the same techniques that make video games so popular are now advancing our understanding of cancer. 

Make it real

This is thanks to Cancer Grand Challenges, a funding initiative co-founded by Cancer Research UK and the US National Cancer Institute. It brings together diverse international teams of world-class researchers to answer some of the most complex questions in human health.  

For the tumour mapping challenge, the IMAXT team has combined existing VR techniques with entirely new approaches to build a system for creating 3D representations of individual tumours.  

These are exact models of unprecedented size and detail, and can be viewed, explored and studied from the inside using VR. Moreover, the team has also created a unique VR cancer lab called Project Theia, where researchers can examine these models and do their research together, no matter where in the world they actually are.   

Owen Harris

The new perspective IMAXT’s VR software offers could give doctors and scientists the understanding they need to develop new ways to diagnose and treat cancer, stopping it from spreading and coming back.    

And now, on 1st December 2022, after seeing exceptional responses in trials earlier this year, the IMAXT team has released its software to the wider academic (non-commercial) cancer research community.  

To mark the occasion, we spoke to Owen Harris, the lead video game designer for IMAXT, about working alongside scientists on such an innovative project. 

An unusual request   

This story starts when Harris was a child. “I was playing video games and, like many kids at the time, I was enamoured by the medium,” he explains. “And one day I realised they were made by people like me – they weren’t just these magical artefacts.” From there, Harris’ experience gets a lot less typical.  

He went on to study video game design at college and, after a few years in the industry, got involved with a science project of a different ilk. “I had been working on an intervention for anxiety and depression, studying the effects of soothing music and visuals as well as slow breathing exercises,” he explains. “So I have some experience working with scientists. And I think that is one of the ways that Greg Hannon found me.”  

But even Harris was surprised when he first heard about IMAXT. 

“An email arrived in my inbox from Greg,” he recalls. “And I thought it was a scam because it sounded so radical and strange.” Still, it was too interesting to ignore. Harris and Hannon got on a phone call, and the rest is history.   

“Greg explained that he wanted to visualise breast cancer and use virtual reality to generate new treatments for it. At the time, my aunt, who I was very close with and was one of the most important people in my life, had her breast cancer come back. So I was very motivated to get involved.”  

The forest and the trees 

Biopsies of tumours play a crucial role in a person’s cancer journey. By giving doctors a close up look at a cancer, they reveal indispensable information that helps guide diagnosis and determine decisions around treatment.   

But to fully understand cancer, we need to know even more about tumours. That means investigating everything about them: from the types and numbers of different cells inside, to where and how they’re distributed and exactly what they’re doing to help the cancer survive and grow, as well as what’s going on in the surrounding tissue.

While biopsies are extremely helpful, current technologies look at cells individually, rather than in the context of their surrounding tissue and environment. This offers vital information about the elements that make up a tumour, but a thorough overview of how the cells interact is missing.     

[Cancer researchers] never actually get to see what they’re working on; they never actually get to visually and viscerally see the results of the things that they’re doing.

That’s where VR comes in. By making it possible to look at tumours as they appear in the body, and not just as isolated clumps of data about individual cells, IMAXT could take analysing biopsies to the next level. 

“If you were looking for a specific tree in a forest, or wanted to understand a particular type of relationship that tree has with its environment, would you rather look at a photograph of the forest or go for a walk through it?” asks Harris. 

“Looking at a thing and being inside of it are so viscerally and hugely different.” Studying tumours from the inside out can reveal hidden information about their biology, which could help us work out better ways of treating them. “And that’s what we are trying to facilitate.” 

How does the software work?

“Because everything was happening at the same time, we started developing the software before the other parts of the team had actually generated the first samples,” Harris explains. 

“First of all, we started thinking about the environment of the virtual lab and the sort of place that would be comfortable to spend several hours doing visual analytical work. And we started by just trying to develop a nice space to be in.”

The team’s ambition was to give scientists a far more spacious and welcoming virtual lab than they have when they interact with their data on traditional computer screens. In fact, Project Theia swaps out the spreadsheet for an entire studio.

Dr Dario Bresson, head of the IMAXT lab, investigates an early-stage breast cancer through VR.

“We approached this by thinking about a cook in a professional kitchen, or a sculptor working in their art studio, who has this expansive space and all the tools that they might need to do reflective and creative work,” says Harris.

But it’s not easy to create something so novel. “Nothing like this had ever existed before, so we had to make a lot of guesses. We had to use a lot of intuition to lay the space out and think about what tools would be needed, as well as where things should be – and even how bright the space should be.” 

Like most things, the virtual lab evolved once the team was able to see how scientists used the original version of the interface. “And it was through that process that making version 2 became very different. Now we know what the data looks like.”  

Moving from game design into research 

Breakthroughs don’t just happen overnight. Research takes time. To Harris, that’s a key difference between making video games and designing tools for scientists. 

“There is a tremendous pressure in the video game industry to produce and ship content as quickly as possible. This is the reason video games end up being so similar. In a scientific project there is generally more time to ideate, experiment, try different paths and find the best solutions to problems. The process is slower in pace but deeper in results.” 

Some differences made for more of a challenge. Perhaps the biggest one for Harris was the amount of the data cancer research generates.  

Tumours are incredibly complex. That’s part of the reason a VR tool is so useful for studying them. But it’s also why it’s so hard to make one. In a video game you might have to get hundreds or thousands of things moving onscreen, but in the case of IMAXT, the team had to deal with tens, or even hundreds of thousands of pieces of data at once – from individual cells to tiny molecules within them.

Nonetheless, cancer research and video game design also have quite a bit in common. When Harris describes it, developing a video game sounds a lot like running a scientific experiment. 

“When designing video games, you’re trying things out, you’re seeing what works,” he explains.  “You have a theory or an idea, and you test it, you see if it’s the right way to go. That was easy to slot into a research setting.” 

From spreadsheets to headsets

The response of scientists trying this tech out for the first time has been, in Harris’ words, “delightful”. 

Eliane Brechbuel, a Phd student, tries out Project Theia.

Some of the best reactions have come from people who study single cells, working with huge amounts of very specific data. “We were talking to some students who were describing how, currently, they just have to work in these really huge spreadsheets and do this kind of analysis through programming interfaces,” he explains.  

“They never actually get to see what they’re working on; they never actually get to visually and viscerally see the results of the things that they’re doing.” 

Changing that makes studying tumours a lot easier. As they get used to the IMAXT platform, researchers will be able to do more analysis on more samples in less time, all while working more closely with colleagues from around the world. 

But it’s about a lot more than creating a better working environment. Importantly, by making it possible for researchers to visualise their data in a different way, IMAXT could give them new ways of interpreting and using their findings.  

“Hopefully, giving researchers these tools and datasets will enable them to make real discoveries and breakthroughs,” says Harris. There’s potential to more closely study why some cancers come back in very aggressive ways and why others do not, and to predict how cancers might return or develop. “Maybe eventually we’ll even get to the place where this is used to analyse specific patients and tumours.” 

With researchers now uploading their samples, strapping on their headsets and stepping into the virtual lab, that’s beginning to look more and more like it could become reality. 

Find out more about the IMAXT team on the Cancer Grand Challenges website. 

The software can be accessed for free for academic non-commercial research only under the terms of an Academic Licence at Suil.ie.