Developer Uses VR To Build AR Experiences That Mimic Real-World Physics

Lee Vermeulen is breaking new ground with his experimental AR prototypes. 

“Palmer Luckey is a tinkerer,” is how Time Magazine described the creator of the Oculus VR headset back in August of 2015. Since that article, Luckey’s “tinkering” has expanded into a full-blown technological revolution, earning him a whopping 2.3 billion USD when he sold his VR company to Facebook.

Like Luckey, Toronto-based developer Lee Vermeulen is also a tinkerer. However, where as Palmer Luckey explored the possibilities with VR technology, Vermeulen explores the possibilities of augmented reality, and how AR experiences grounded in real-world physics can co-exist with reality. It’s not just about placing a virtual object in a real-world location; it’s about creating an AR experience that fits and reacts to the environment in a natural manner.

If you explore Vermeulen’s YouTube and Twitter pages, you can see work dating back to 12 years ago. It’s his most recent work, however, that has been garnering a lot of attention from both the AR & VR communities as of late.

Vermeulen – who is also the co-founder of the company Alientrap Games, and the creator of the VR sandbox tool Modbox –  has been posting videos of AR experiences that explore how fluid physics, radial gravity, trajectory prediction, mirrors, and soft body physics could exist virtually in the real-world in a way that looks and acts realistically.

This includes everything from an ‘AR Mirror World’ that exists in within a real-life mirror and ‘AR Photo Freeze’, to AR slime and bubbles that react to real-world objects with realist physics.

In an interview with VRScout, Vermeulen talked about his work and what led him to AR. “It was after I tried Tilt Brush that I saw the possibilities of what 6DOF controllers could bring,” said Vermeulen, adding, “It brought a sense of depth and spatial awareness in a way that didn’t exist before.”

Having that type of control in a virtual environment opened up so many possibilities for Vermeulen. “Interaction has always been my main focus.” Unfortunately, most AR devices don’t offer the control that VR does. It’s all hand gestures, and it’s not initiative.

ZedMini camera mounted to HTC Vive / Image Credit: Lee Vermeulen

One of the criticisms of the first HoloLens was the fact that hand gesture functionality featured a rather intimidating learning curve; it didn’t feel natural, and it wasn’t very accurate. Moving, pinning, or creating AR content required precise tracking and practice.

When it comes to Microsoft’s next generation HoloLens 2, Vermeulen says, “I’m hopeful for the next HoloLens but if it doesn’t have good controllers, then I feel that is not at all an advancement.”

When asked about why he focuses more on AR, Vermeulen says, “I don’t want to completely escape reality, I don’t want to be in a complete virtual world. I just want to augment my world.”

According to Vermeulen, this basis of thought is why VR hasn’t caught on the way many industry veterans had anticipated. “Most people don’t want to escape or be blinded from the world for a long period of time.”

When watching Vermeulen’s videos, you can see that in some of his experiments he is using VR controllers in an AR environment.  That’s because he builds his AR experiences through his VR tool Modbox, explaining that if he wants to place an AR switch on his wall, it’s just easier to do it directly in AR rather than having to build it in the Unity Editor and then trying to position it correctly on a second screen. “It’s just naturally quicker to build AR directly in AR.”

In a blog post entitled Prototyping future AR in VR, Vermeulen breaks down the VR headset he uses to create his unique AR experiences. The primary component is a ZedMini camera hooked up to an HTC Vive headset, that allows for AR pass-through while he is in VR. Though it’s not perfect, Vermeulen states that it does allow for good proof-on-concept videos.

One interesting video from Vermeulen’s YouTube page shows him positing an AR robot using the HTC Vive’s hand controllers. Using a voice command, he is then able to turn off the lights to his home – blending AR, VR, AI, and smart home technology into a single experience.

The video received a lot of feedback from curious internet denizens wondering what possible use-case scenarios this project could facilitate. Why couldn’t they just use a simple voice assistant device, such as an Amazon Echo or Google Home? For Vermeulen, that wasn’t the point of the experiment. “The point was to show the scripting possibilities of what you could do with an avatar, but I also wanted to show the possibilities of connecting AR to IoT and what that means, being able to visual seeing your lights be turned off seemed to make it feel natural.”

Sometimes you do things for the sake of just seeing what happens. Of course, this is what tinkering is – altering existing technologies, products, and concepts in the hopes of creating something new.

Without this type of curiosity, we wouldn’t have the pacemaker, the microwave, or potato chips! Can you imagine a world without potato chips?!

Vermeulen is a tinkerer, but he is also modern-day explorer. His work is steeped in the exploration of how AR can be pushed into new directions by blending it with VR, IoT, AI and smart homes. At the end of the day, however, Vermeulen just wants to excite you with the possibilities surrounding AR technology.

“More people excited about AR = quicker development of AR.”

Vermeulen’s YouTube channel is chock full of interesting projects and prototypes guaranteed to get you hyped on the future of augmented reality; we highly suggest your subscribe. If you’re interested in taking Vermeulen’s approach to creating AR experiences in a blended environment, you’ll need a ZedMini camera and a copy of Modbox, which is available on the Steam store.

About the Scout

Bobby Carlton

Hello, my name is Bobby Carlton. When I'm not exploring the world of immersive technology, I'm writing rock songs about lost love. I'd also like to mention that I can do 25 push-ups in a row.

Send this to a friend