Real-time AR text translation in a VR space.
This past May during the 2019 I/O Developer Conference, Google unveiled several new features heading to their Google Search and Lens applications. These additions included 3D animated models accessible directly through Google Search, the ability to scan restaurant menus for additional information on recommended dishes, as well as automatic tip calculations.
Arguably the most exciting addition, however, is a translation tool that uses your smartphone’s camera to convert any physical text—whether it be a street sign, restaurant menu, pamphlet, or any other written or printed writing—into your preferred language in real-time.
This same usefulness can apply to VR environments as well, as proven by an indie developer who last week managed to import a working version of Google’s AR-based tool into an interactive VR space.
In a video posted to Twitter by user @Phasedragon, the developer can be seen tinkering with a wrist-mounted version of Google Lens while in VR, using the app to translate a variety of text—including a wooden sign, pamphlets, and several banners—from Korean to English.
“I just hooked together a few apps, didn’t really do anything myself other than try a bunch to see which ones worked,” stated the modest creator in a follow-up tweet. “Sparkocam to capture the desktop and export as a virtual webcam, android studio emulator to run google lens, and OVR toolkit to display it in VR.”
Phasedragon states that he attempted the project using Microsoft Translate initially, but eventually made the switch to Google Lens as a result of Translate’s incompatibility with AR.
With social VR applications such as VRChat, Rec Room, Sansar, and other multiperson immersive experiences rising in popularity—both domestically and overseas—existing VR player bases will only continue to become more diverse. As more and more cultures begin to enter the ecosystem, translation devices such as Google Lens may not just be a useful commodity, but a necessary one.
Featured Image Credit: Phasedragon
