A HoloLens visualization tool enables surgeons to treat chronic heart conditions more effectively
At this year’s Imagine Cup, Microsoft hosted dozens of student teams from across the world. They showcased an incredibly broad range of products developed to solve practical problems that ranged from getting rescue workers to disaster zones quickly to helping farmers manage the health of their livestock, improving water quality, battling fake news, and holding politicians to account.
And given the company’s vision—which they set out at this year’s Build conference—of developing a Mixed Reality ecosystem, it’s hardly surprising that we also saw a few interesting HoloLens projects cropping up as well. One of those was developed by three Biomedical Engineering students from Xidian University in China, who came up with the idea after meeting at their school’s Innovation Lab.
The students had the opportunity during the course of their studies to work with children suffering from congenital heart disease, and found that often a child’s heart was so small that standard examination and diagnosis techniques were entirely unsuitable for them. In fact, in some cases these standard adult procedures could cause serious damage to their tiny and already fragile organs.
This of course caused many problems not only during examination, but consequently later in surgery, as surgeons were not supplied with ideal levels of information about a patient’s heart. The students therefore decided to develop a practical tool that addressed some of those issues. The result was Surgical Assistant, a Windows medical imaging processing platform that allows for both 2D and 3D visualization.
“With our platform we can localize a lesion, and observe all the details of that lesion in a non-invasive way,” explained Maoqing Yang from the Surgical Assistant Team.
Like all this year’s Imagine Cup projects, Surgical Assistant was deployed on Azure, but it additionally based its computer diagnosis and decision-making algorithms on Azure Machine Learning Studio.
Such cloud-based healthcare-based virtual applications are certainly a booming area, and disciplines like surgery in particular—which are highly visual and where building up the right skillset requires many hours of practice—are ideally suited for such solutions, which allow for realistic practice with none of the risks associated with practicing on live patients.
There are indeed a host of examples emerging of Mixed Reality applications being used to train surgeons, many of which are seizing upon the ability of the HoloLens to overlay holograms onto real-world backdrops, as well as its collaborative capabilities. Unlike in VR, the user is able to interact with colleagues and teachers (in the same room or remotely) at the same time as viewing holographic content, which is also shared so that all participants are looking at the same thing in real time.
This works to closely replicate the dynamics of an operating room, and is already yielding positive results, which will likely improve dramatically once haptic technology develops further to allow for fine-tuning of factors around touch, pressure and feel which are crucial in surgery.
Although the Surgical Assistant team were prevented by visa issues from attending the finals at Microsoft’s Headquarters in Redmond, WA, they still worked with several senior company staff to improve and develop the structure of the product, and are optimistic about the prospects of deploying and scaling their product.
They now plan on continuing to develop the product in partnership with more hospitals and deploying the tool to help may more young patients suffering from such chronic heart conditions.
“We have developed this system for over 2 years, and we have tested it in a hospital environment,” said Yang. “Now we want to let more people know about our work and collaborate with whoever is interested to solve these problems.”
