Nature: Virtual-reality applications give science a new dimension

By David Matthews

As I put on a virtual-reality (VR) headset, the outside world disappears. A cell fills my visual field, and as I crane my neck, I can see it from several angles. I stick my head inside to explore its internal structure. Using hand controllers, I dissect the cell layer by layer, excavating with a flick of the wrist to uncover tiny, specialized structures buried beneath the surface.

VR isn’t new, but interest in the technology has boomed since 2016, when gamers and a handful of scientists introduced several high-quality, relatively inexpensive commercial headsets to the public. A similar surge has emerged in augmented reality (AR), a related technology that uses a see-through visor or smartphone screen to layer objects on top of real surroundings.

Compared with VR, visualization software for AR headsets is less advanced. Mark Hoffman, chief research information officer at Children’s Mercy Kansas City, a hospital in Missouri, has experimented with viewing proteins and computed tomography (CT) scans using Microsoft’s HoloLens — a kind of visor with a built-in computer that projects 3D objects over the real world.

Surgeons at Children’s Mercy are exploring the use of AR to view CT scans of patients’ hearts before an operation, he says. Hoffman uses a step-by-step approach to make such data viewable using the HoloLens. The surgeon can explore the tissue by projecting it onto a fixed point in space — say, in the middle of the room. But if they turn their head, the image disappears and they see only what is actually there. “They walk into the ventricle or the atrium of the heart, and maybe they’ll see that, for a particular child, the entry point of a blood vessel is not where it normally would be.” 

Read the full article via Nature

Learn more about the Children's Research Institute