EyeSight, Environments, and eye-tracking, oh my!
Yesterday, Apple revealed Apple Vision Pro, the company’s first spatial computing device. Available early next year, this high-end mixed reality headset promises a variety of game-changing features designed to revolutionize the way we work, play, and connect with others.
This includes everything from 3D “Personas” powered by machine learning technology to spatial video capture, all of which are powered by a one-of-a-kind Apple Silicon two-chip design.
We received quite a bit of information about the device yesterday during Apple’s 2023 Worldwide Developer Conference. The following are just five features we found particularly exciting.
EyeSight
Arguably the most unique feature of Apple Vision Pro, EyeSight is a new technology that gives the headset a “transparent” feel. The entire front of the headset is composed of a three-dimensionally formed laminated glass. Anytime someone approaches the headset while it’s being worn, the device uses eye-tracking technology to generate a 3D replica of your eyes.
The technology can also inform others in your vicinity what you’re up to in your headset. Not only does EyeSight look incredibly cool, but, according to Apple, this will allow for more natural interactions with those outside of your headset. This kind of technology could be crucial in developing a mainstream mixed reality ecosystem.
Personas
Apple Vision Pro will allow you to connect with coworkers remotely through a number of apps, such as Zoom. Those on mobile or desktop will appear as 2D tiles, while those connecting via Apple Vision Pro headsets will appear as realistic 3D avatars, referred to by Apple as “Personas.”
After enrolling with Apple, you’ll be able to scan your face with Apple Vision Pro to automatically generate a Persona. During video calls, your avatar will mimic your facial expressions and hand movements with an impressive level of accuracy thanks to machine learning technology.
Environments
Apple Vision Pro is a mixed reality device, which means it’s capable of projecting virtual content over the real world. That said, when you need a break from reality, you can turn the Digital Crown to increase the intensity of your “Environments,” 3D volumetric captures of real-world locations brought to life in stunning clarity.
This looks like an excellent way to isolate yourself from the outside world so you can focus on your in-headset activities, whether it be your next work assignment or the latest episode of your favorite Disney+ show.
Spatial Videos & Photos
Apple Vision Pro features Apple’s first 3D camera, allowing you to capture 180-degree stereoscopic videos and photos with ease and access them anytime from your photo library on iCloud. According to Apple, this technology will allow you to relive past memories anytime, anywhere.
As if that weren’t cool enough, you can also view panoramic photos captured on your iPhone device and immerse yourself in wide-angle shots.
Eye-Tracking
Eye-tracking technology powers a number of exciting features on Apple Vision Pro. This includes foveated rendering, an optimization technique that reduces the image quality in your peripheral vision in order to reduce the rendering workload. Long story short, the headset improves the graphic quality of wherever you’re looking.
Eye tracking can also be used to navigate the primary menu. All you have to do is focus your attention on a widget and tap your fingers together to select it. There are also the Facetime and video conferencing calls we mentioned earlier; eye tracking adds a new level of expression to in-headset video calls.
For more information on Apple Vision Pro, check out our full coverage of yesterday’s announcement here. You can also check out the official Apple website for even more details.
Feature Image Credit: Apple