Beat Reality transforms any environment into a music-reactive art installation.
Available on the Vive Focus — HTC’s China-only replacement for its discontinued Daydream headset — Beat Reality generates a colorful filter that allows you to experience any locale with literal rose-tinted glasses.
Created by Enea Le Fons and Anthony “SkarredGhost” Vitillo as a result of their #30DaysInVR collaboration, the app presents a mixed reality (MR) overlay that adjusts color and brightness based on sound. For example, playing percussive music makes your entire view come to life, as if everything in front of you were made of pulsing neon filaments.
While it’s only been released on the Vive Focus so far, Western audiences may soon have access to Beat Reality and/or similar visualizer apps with the proliferation of MR passthrough cameras on standalone headsets, similar to features we’ve seen on hardware such as Magic Leap One Creator Edition, Oculus Quest and the Lenovo Mirage Solo.
Taking a deeper dive into the creation of Beat Reality, I interviewed co-creator Anthony Vitillo over a series of emails. In this interview, Vitillo shares tips for other immersive developers and the challenges the creators faced during development.
VRSCOUT: For somebody who isn’t familiar with MR, how would you describe Beat Reality?
VITILLO: “Beat Reality is a Mixed Reality experience that turns the world into your dance floor wherever you are; at home, in the office, etc. You can turn on your stereo, raise the volume, put your Vive Focus on and launch Beat Reality to enjoy the music as if you were in a discotheque. The experience will show the reality around you, but modified so that it beats following the rhythm of your music. It’s a synesthetic experience that will mix your visuals with what you hear.
I’ve tried it in the office putting on some 90s dance music and it was [a lot of fun]. Well, maybe not for the people in the nearby offices, but for me [it was] for sure. I could see the edges of the world around me, all pumping with bright colors following [the music], a bit like when you are in a disco and everything is dark — [minus] the spotlights that cast light following the music.
Of course it can also be used in actual dance floors: various underground DJs that have tried it, have found it amazing. Also artists find it interesting, because it can modify your reality in an original way. It’s a new way to [enjoy music].”
“I’ll define it as ‘An innovative portable discotheque.’ A new way to enjoy music. With some magic, [it] can make you have the sensation of being in a futuristic dancefloor wherever you are.”
I understand this isn’t your first time writing an mixed reality app. What did you learn from your previous project?
“Well, I have a failed startup on my shoulder[s]. It was called Immotionar, and it was focused on full-body virtual reality using Kinect. Everybody was amazed by our product and we won some awards, but we failed. The reasons are many, and I have described them in our post mortem, but one of them was that we spent too much time creating something that people were not interested in buying. The lesson is: start small, throw some little apps and ideas in the wild and adjust them with user feedback. And if things do not work, abandon them immediately. Fail as fast as you can. Especially with VR, [where] succeeding is harder than in other more mature sectors.
And this is what we are doing with Beat Reality: it is not a monster project with thousands of features, but a simple experience that does few things, and do[es] them well. We have lots of ideas to improve it, but we are also waiting to see the [user feedback] to decide what [the priorities are]. So, my philosophy is now doing [things slowly], one step after the other, investing only in projects and features [that] people are really interest[ed] in using.”
What inspired you to write Beat Reality?
“Well, the actual inventor is Enea Le Fons, the guy [behind] #30DaysInVR. We became friend[s] during the #30DaysInVR initiative and sometimes we talk about XR together. He is a creative and [always has] tons of ideas. He is also very fond of underground dance music and makes jam sessions, DJ sets and all kind[s] of stuff.
One day, among other things, he told me that it would have been cool to see [the world as being made of edges] pumping [to] the rhythm of music. I think he had no idea that some days later I could come up with an actual prototype of this […] with the Vive Focus. He got super excited; I’ve rarely seen him so happy. After that, we started iterating that prototype to transform it into something that could be amazing for our users.”
The Vive Focus controller, as seen from inside Beat Reality.
Are you planning to port the app to the Oculus Quest or Mirage Solo?
“Maybe. I’ve heard that Google is opening the Daydream platform to MR apps and also Oculus is working on some kind of passthrough for the Quest, even if the 4 cameras are positioned in the corners and […] are in an ideal arrangement for [Mixed Reality passthrough]. Carmack has said they are working on this. Let’s see what the god of gaming will do.
We’re not planning, [for sure], to port it in the very short term. As I’ve said, we are now more focused on getting the first [feedback] from Viveport users. But in the future… [sic]”
“There’s almost no documentation. When looking for problems related to Vive Focus on Google, I always only find my own blog as an answer. It’s like this meme. Maybe using Baidu and searching in Chinese would be a little better, but I am still not at [a] master Chinese level.
The Vive Wave SDK is interesting, but it is a new project and needs time to become mature. And the Focus should be launched in the Western market to be able to create a greater community around it.”
Is there any hidden feature of Beat Reality that you’d like readers to know about?
“Well, we are launching an update that will introduce two new modes: one to have a pixelated vision and the other to have a Matrix-like ASCII vision. If you use Beat Reality for some minutes [sic], you will unlock these two new amazing modes.
Then something that people don’t always realize is that it is possible to shoot photos and GIFs inside the app! [By] connecting the Focus to the PC, it is possible to download them and share them with your friends on social media. Enea has made a lot of GIFs and they are really incredible. He [shoots] GIFs everywhere: in the car, in the house, alone, with friends, etc… you can find them at this GIPHY channel and see how this Beat Reality-vision can really make you feel reality in a new way.”
What lessons did you learn from writing Beat Reality that you would share with other developers?
“Well, I’d advise to try to break things. This is something that I love making… [sic] they give me an SDK and the first thing that I think is: how can I make with it something that is not allowed with it? And then I [go] crazy [trying] to realize it. With the Vive Focus, I already made a Unity Plugin to preview live in the headset what you are doing inside Unity. I made a plugin to make AR for the focus. And now I’m developing some Mixed Reality apps (Beat Reality, but also ‘Enter The Matrix’ that makes you see the world […] as in the movie The Matrix).
I’m also experimenting with the beta of the Gesture SDK for the Focus, and I’ve already broken the rules shooting a selfie with the Vive Focus. When Mister President (Alvin Wang Graylin) saw the last one, he told me ‘Nice, but that thing has been done [sic] by us to stay on your head, not in your hand!’
So, I advise [every one] of you to spend some time each week to break the rules and try to do something different. And then share what you have made with the community […] breaking the rules requires creative solutions and a lot of trial and error. So, you have to [put in] a lot of effort to find the right road to do stuff. For instance, the first idea on how to make the edge vision would have required a complicated process that employed the use of OpenCV, but that would have had a terrible framerate on the headset. But then, after various thoughts and trials, I found the right path with a completely different GPU method.
Don’t give up, continue pushing, try every possible road to arrive at the solution. In the end either you realize that what you are trying to do is impossible (in this case, do something else) or you will solve it and you will feel a great satisfaction. VR is a new field; there [are] a lot of new things we can create with it!”
What are your closing thoughts on the launch of Beat Reality?
“It has been a very interesting ride, and however it will go, I would have been happy [for] having done it. At the moment I’ve heard no negative feedback on Beat Reality, apart from it feeling a bit too little, but that’s a happy problem, because it means that people are liking this alpha version and want more from it.
I really hope that people will understand how it is important to exploit Mixed Reality (something that has been underlined also by Oculus at its OC5 event) and how it is important to create VR applications that are not just games and do not try always to follow the rules.”
Is there anything else you’d like to add?
“I want to thank you, Gabriel, for having given me the opportunity to have this interview; Enea for the wonderful collaboration we had together; my (business) partner Massimiliano Ariani for the help he gave us; and Mister President for being so inspiring and supportive during the whole development of the project.”
—
Beat Reality launched on September 14 for the Vive Focus. If you’re able to get your hands on one, the app is free on the Viveport app market and you can play with it immediately. Also, you’re invited to go ahead and follow Anthony Vitillo (aka SkarredGhost) on his official blog over at https://skarredghost.com, where he posts all kinds of useful mixed reality development information in his regular blog posts.