Mixed Reality and Medicine: Virtual Practice Makes Perfect

Mixed reality is becoming an increasingly useful tool for surgeons, but can it also help to train the next generation of doctors?

Prestigious medical institutions such as Imperial College London are using HoloLens devices in operating theatres to spot key blood vessels, bones, and muscles helping to make procedures quicker and safer.

Patients who have suffered an accident may have open wounds that require reconstructive surgery. Skin and blood vessels are taken from a healthy part of the body and used to cover the wound, enabling it to close and heal properly. A vital step in the process is connecting the blood vessels of the “new” tissue with those at the site of the wound, so oxygenated blood can reach the area.

Surgeons have traditionally used a handheld ultrasound scanner to find vessels under the skin by detecting the movement of blood. However, this is very time-consuming and still requires some guesswork as to where the vessels are and their path through body tissue, explained Dr Dimitri Amiras – a Consultant Musculoskeletal Radiologist at Imperial College Healthcare NHS Trust – as he walked me through the demo.

This latest technology allows surgeons to take CT scans that have previously been completed and overlay 3D digital models of them onto a patient’s limb during reconstructive surgery. It has been used to help surgeons successfully move blood vessels from one part of the body to another in order to help open wounds heal more quickly.

At the Global Education and Skills Forum (GESF) in Dubai I got a chance to try it out first-hand, and it was pretty amazing, specially knowing that the Holograms I was looking at were based on real CT scans. Patients so far have included a 41-year-old man who injured his leg in a car crash, an 85-year-old woman who fractured her fibula and a person who developed an infection that required surgery.

“Mixed reality offers a new way to find these blood vessels accurately and quickly by overlaying scan images onto the patient during the operation. This technology allows us to experience the data that we have collected from patients before their operation in the most realistic and natural way. You look at the leg and essentially see inside of it; you see the bones and the course of the blood vessels,” he said.

Dr Philip Pratt, a Research Fellow in the Department of Surgery and Cancer at Imperial, said using the HoloLens is allowing surgeons to understand a patient’s unique anatomy very quickly and accurately and to collaborate more efficiently in teams as multiple surgeons wearing the headsets can also see what their colleagues are specifically looking at.

As well as a powerful visualization and collaboration tool for working surgeons, immersive technologies can also help solve a growing problem: the global shortage of such professionals, specially in the developing world. Training facilitated by immersive technologies can be instrumental in addressing the chronic shortage of trained professionals in areas like teaching, healthcare and the digital economy.

Bassem F Hashash, from augmented reality training and collaboration platform Proximie, says an estimated five billion people worldwide do not have access to surgery. This is due partly to the chronic lack of specialists available to perform such procedures, and the difficulty and expense involved in training new surgeons.

Training at scale is a real challenge, especially in areas such as healthcare, as you can only fit a small number of students in an operating room, and observation and practice is how one learns the crucial skill-set necessary in such professions. Yet with immersive technology, it’s possible for experts and students to virtually transport themselves into settings which allow them to flexibly collaborate and train. They can, in other words, become a democratizing force in education, by enabling students to virtually access experiences that would otherwise be out of their reach.

According to Richard Vincent of Fundamental VR who was also showcasing his haptic VR application at GESF (which let me have a great time literally messing around someone’s insides), it isn’t a matter of virtual training being “better,” but of enabling the benefits of experiential learning to become more widely accessible, allowing training to move beyond the limited model of “see one, do one, teach one” and opening up the possibilities for students to practice and perfect their skills in a safe and accurate environment.

In developing his own HoloLens application, Vincent has seen evidence that working with mixed reality tends to trigger different parts of the brain through immersion, so that content is assimilated more quickly and “sticks” with learners for longer, hence its potential to add to, enhance, and accelerate learning. And Moore’s law he adds, means that his company’s current training simulation system costs one tenth of what it did a few years prior. He fully expects that costs will compress rapidly over the next five to ten years.

“As the technology progresses it will become more affordable and more integrated into our everyday lives,” said Dr. Amiras. “Just as most people here are carrying some sort of smartphone, the next step will be some sort of augmented reality device.” 

“This is the new interface between computers and humans. What happens with technology is that when it works well, it disappears,” agrees Vincent. “Immersive Technology in the future won’t be about a bulky headset. What we have today is the equivalent of the Motorola phone of 1987. It will be glasses and contacts. This might feel scary now but in 10 years time it will feel as natural as carrying a smartphone in your pocket does now.”

All this means that people all over the world will be able to easily access realistic experiences and simulations, learning and practicing new skills in entirely new ways. Much in the same way as the widespread adoption of the internet facilitated broader access to knowledge, immersive technologies can enable broader and more egalitarian access to experience itself, and that’s pretty exciting.

Image Credit: Andrew Trotman / The Varkey Foundation / FundamentalVR

About the Scout

Alice Bonasio

Alice Bonasio runs the Tech Trends blog and contributes to Ars Technica, Quartz, Newsweek, The Next Web, and others. She is also writing VRgins, a book about sex and relationships in the virtual age. She lives in the UK.

Send this to a friend