While the expected Vision Pro mixed reality headset from Apple Inc. is being marketed. Pro is currently in consumer use, experts in surgical simulation, immersive technologies and a surgeon who used Google Glass in the operating room envision a bright future for the device in healthcare.
Richard Vincent, founder and CEO of FundamentalVR, which designs immersive surgical simulation software to run on virtual reality headsets and handheld haptics, and Sam Glasenberg, co-founder and CEO of Level Ex, which makes medical video games to train surgeons, expressed excitement about the uses. The potential for Apple Vision Pro to train surgeons, among other possibilities in healthcare. (See also “Minute Insight: FundamentalVR raises $20M in Series B, continues development of HapticVR for surgical training” – Medtech Insight, August 11, 2022.)
Richard Vincent, CEO, FundamentalVR
BasicVR
“It's a great headphone, and headphones continue to evolve very quickly,” Vincent said. Medtech Insight. “Meta Quest 3.” [mixed-reality headset] What was issued at the end of last year was a big step forward, and this is another big step forward. It has some amazing capabilities…and our use case is in healthcare [is in] Training, education and skills transfer where we want to bring people to proficiency through virtual reality experience.
“The Vision Pro offers some specific advantages over other headsets out there,” Glasenberg noted. “You won't be able to use it in the operating room yet, but from a training and practice perspective, it's very useful.”
Executives find that many surgeons tend to be Apple fans. Vincent said several surgeons have asked him if they can try Vision Pro before its launch on February 2.
Rafael Grossman, a trauma surgeon at Portsmouth Regional Hospital in New Hampshire, known as the first surgeon to bring Google Glass into the operating room in 2013, also expects interest in the healthcare sector to be high. “The fact that it is an Apple device will definitely help us increase the interest and penetration of this device – not only among the general public, but also around the world.” [health care] Industry,” he said.
Vision Pro is expected to have wide-ranging applications from surgical training, patient and health professional education, to connecting patients with loved ones at home through an immersive virtual experience. It should also help streamline cumbersome tasks in hospitals, including documentation of electronic medical records.
according to Bloomberg NewsApple executives pointed to healthcare, training and education as key areas of technology that are still emerging. Mike Rockwell, Apple's vice president of technology development group, was quoted as saying that surgeons often find it difficult to look at displays during operations, and this is where Vision Pro can bring it all together and hopefully improve outcomes.
Outstanding features
Experts Medtech Insight People who spoke agreed that the Vision Pro stands out from other VR headsets on the market when it comes to the quality of its displays, high resolution, eye tracking and gesture-based controls, and its transition between immersive virtual experiences and real-world viewing. , a feature known as “crossing over”.
Apple markets the Vision Pro as “spatial computing.” It is designed to be a device that allows users to create a virtual office space and do real work. Users can open virtual windows, web browsers, applications, videos, and images and move them around in space. To navigate, the user simply looks at the icon — the device accurately tracks eye movement — and, by pinching their thumb and finger together, selects the icon.
The displays inside the Vision Pro represent a huge leap forward, Vincent said, as they are sharp enough to read even small text. He described the eye tracking as “exceptional” and the movement between mixed reality and the real world as “seamless.” “If you raise your hands up, you can see your hands in the virtual reality space,” he noted.
Glasenberg also praised the Vision Pro's eye tracking feature, which is extremely important for training future surgeons. “It helps you understand, 'Are medical professionals looking where they're supposed to be looking? Are they catching the things they're supposed to be hunting?'” Glasenberg said. (See also 'Medical video games for smarter surgeons and healthier astronauts; Level Ex CEO offers Demo” – Medtech Insight, December 15, 2022.)
Vision Pro-Powered Haptics
FundamentalVR, based in London, UK, combines virtual reality (VR) with haptics, allowing surgeons in training to feel virtual objects and experience, for example, the sensation of cutting or drilling without actually being in the operating room. FundamentalVR's clients are primarily medical device and pharmaceutical companies who purchase an annual subscription to access the company's platform and then distribute it to their clients, who tend to be health systems, surgeons, nurses and other healthcare workers.
FundamentalVR's software is hardware-agnostic, and the startup has been working closely with Apple over the past two-three months to better understand the Vision Pro ecosystem and how it can fit into the FundamentalVR platform and be used alongside other devices.
“It's still very early in the ecosystem around Vision Pro, but we've been able to navigate through it,” Vincent said. FundamentalVR has developed an application for Vision Pro, currently available on an “invite only” basis to its customers in the App Store, to simulate catheter placement.
“The thing that interests us, and I think it interests Apple as well, is the combination of the visual precision of the Vision Pro as well as the kinetic touch,” he said. “We're using Vision Pro to power a haptic device, so we're removing a piece of the infrastructure that we normally use, which is really exciting. So, today, if the team were to share any images of our current setup, you'd see three things — the headset, you'll see a laptop “Or a Mac, and you'll see touch devices. With Vision Pro we're getting rid of the laptop or the Mac. You don't need that anymore.”
By eliminating the computer, this can reduce the cost of setting up FundamentalVR with Vision Pro. However, the current price of $3,500 for the Vision Pro headset is seven times greater than the price of the Meta Quest 3 headset. Regardless of pricing, Vincent does not expect that customers who currently use other headsets will easily switch to the Vision Pro.
“There will be use cases where it's absolutely perfect, and there will be other cases where it doesn't make sense to spend that kind of money,” Vincent said. He expects the fastest adoption of Vision Pro will be in situations where touch and photography really matter.
The possibility of surgery is possible, but it is currently limited
While Vision Pro has potential for use by surgeons in the operating room, Glasenberg and Vincent believe there are factors that could limit that use. Maintaining sterility could be an issue, they said, and Glasenberg believes that “augmented reality passthrough” could be difficult during surgery, because data still passes through cameras, not glass.
“In an operating room where you have very high contrast — very bright lights, you have a lot of monitors, you have a lot of very bright reflections — VR traversal becomes an issue,” he said. “Cameras don't do a good enough job at what we call the dynamic range of the operating room. Surgeons can see the MRI data shown to patients, but now it's difficult to see the patient and see the instruments.
Sam Glasenberg, CEO of Level Ex
Previous level
Grossman dismissed concerns about sterility, noting that unlike a device that comes into contact with a patient and thus needs to be sterile, “anything on my head is not sterile at all.”
As for the problematic crossing, the surgeon believes that the limitations can be overcome by making adjustments to the environment in the operating suite.
“It depends on the type of surgery you have. It depends on the type of device you use. It's all somewhat adjustable.” “The operating room is a very bright place sometimes. Sometimes it is very dark, depending on the type of surgery you are having. This is not a major issue. You can always play with that.”
Grossman believes the modifications to the Vision Pro may make it more attractive for surgeons to use in the operating room. “It's still a huge device…so I think that's a factor, and the two-and-a-half-hour battery life is a factor as well.”
The bigger issue, he said, is ensuring the device meets regulatory considerations such as keeping patient health data secure and private.
One company that is really ahead of the curve when it comes to using headphones in the surgical suite is Magic Leap. The Florida-based company announced in January 2023 that it had received IEC 60601 certification for its second-generation Magic Leap 2 augmented reality headset, giving it the green light for use in the operating room and other clinical settings.
Magic Leap reportedly secured about $590 million from the Saudi sovereign wealth fund last month, just weeks before the launch of the Apple Vision Pro.
“Magic Leap 2 doesn't do augmented reality bypass,” Glasenberg explained. “With Magic Leap 2, it's like you're looking through a pair of glasses with digital content projected onto the lenses. Magic Leap 2 has its own limitations. Its digital content has a very limited field of view compared to Apple Vision Pro.
Wider applications
Grossman believes Vision Pro's potential in healthcare goes much further.
Dr. Rafael Grossman
By creating a fully immersive experience, patients can be taken into the operating room before surgery to help them better understand what to expect during surgery.
“Imagine the power of this tool to reduce anxiety and fear of undergoing surgery,” he said.
He also discussed how the device can be used to enhance communication and communication between providers, patients and their families, and promote empathy and compassion.
“Imagine [virtually] “Bringing a patient in the ICU to their family members in the living room and seeing the whole family together and how the sick or dying patient would feel to have the last view of the family gathering at home, all of that speaks to the patient and saying goodbye.”
It can also be used to improve and simplify tasks, such as documentation of electronic medical records. Instead of having to enter data into a computer that is moved around the hospital, Grossman envisions a future in which health professionals use “private computing” to retrieve data and take notes with the help of generative artificial intelligence, while walking around the hospital or while talking to patients in their homes. Rooms. By reducing stressful tasks, it will also help alleviate burnout, which is a serious problem among healthcare workers, he added.
“The possibilities are endless…. [to] Change the way we do computing, change the way we interact with physical screens with this device. He believes the healthcare community will be very receptive to these use cases, citing his own research.
“Last year I finished my master's degree [degree] I earned my PhD in Health Delivery Science at Cedars-Science and my final project for this program was a qualitative analysis of clinical providers' admissions using a head-mounted display to interact with patients making hospital rounds. Of the approximately 20 doctors, most are nurses [in the study]”90% said they couldn't wait to use it.”
