Over a dozen years of audacious augmented play beyond the beloved brick.
For over a dozen years LEGO has augmented our world with audacious augmented play beyond the beloved brick.In addition to creating experiences that “play well,” LEGO continuously experiments with emerging technologies to shape what the near future of play could be, thoughtfully creating experiences that change the way people live and interact with the physical and digital world.
Over the years we have seen significant changes in augmented reality (AR) technology. Continuous advancements with ARKit, AR Core, and Lidar combined with increased adoption in WebAR, the ongoing evolution of AR glasses, projection mapping, and gesture-based experiences have made immersive technology more accessible for creators as well as everyday people.
Let’s build our time machine of choice— perhaps the LEGO Back to the Future – DeLorean Time Machine, Dr. Who Phone Box, or H. G. Wells’s classic time machine—and reflect on how LEGO has augmented our world over the years using immersive technology.
Life of George (2011)
In 2011, LEGO unveiled their first fully integrated digital-to-physical gaming experience, Life of George. Powered by brick recognition software, Life of George challenges players to build physical models, scan them into the digital game and receive a score based on accuracy and speed. Developed byEye Cue, the experience features 12 levels and multiple ways to play: single-player, “pass-and-play” competitive, and creation mode. Life of Geroge was retired by the LEGO Group in 2015.
Ultra Agents (2014)
In 2014, LEGO launched Ultra Agents, a spy-themed experience that had players battling baddies brick-by-brick through a series of graphic novels, mini-games, missions, and secret builds powered by a free companion app. LEGO teamed up with Trigger to bring the world of Ultra Agents to life by merging physical and digital play, optimized by LEGO’s AppBrick technology.
The experience featured a “conductive brick” that the app recognized when pressed onto the screen, allowing kids to interact with LEGO directly on a tablet for the first time. Wired described Ultra Agents as, “one of the most forward-thinking lines the Danish company has ever created.” Ultra Agents retired in 2015.
LEGO Fusion (2014)
In 2014, LEGO announced LEGO Fusion, a play experience that combines traditional LEGO brick play with familiar app-based game themes. Each LEGO FUSION experience consists of a distinct set of LEGO bricks, a corresponding free downloadable app, and the FUSION capture plate, a small brick building plate with a printed design that enables a smartphone or tablet’s camera to identify the size and colors of the LEGO bricks built onto it.
In response to game prompts, children, build vertically in 2-D on the FUSION capture plate, enabling the app to ‘see,’ import, and transform the creation into 3-D in the digital world. LEGO Fusion retired from the LEGO Group in 2015.
LEGO AR Studio (2017)
In 2017, theCreative Play Lab, LEGO Group’s department that explores the future of play, launched LEGO AR Studio. Developed with Apple’s ARKit, the unique creative tool allows kids to play with digital versions of some of the most popular LEGO sets, such as the LEGO City Train Set, LEGO City Fire Station, and LEGO Ninjago Movie, in real-world scenes using AR technology that mixes them with physical LEGO models.
LEGO BrickHeadz Builder (2018)
LEGO BrickHeadz Builder is an AR play experience for fans of all ages, from the beginner stage to the free builder, that allows you to build with digital LEGO bricks, personalize your own BrickHeadz, and bring the characters to life in the digital world. In this Android-only app experience, discover LEGO BrickHeadz characters and objects, by solving the play puzzles in the ‘Magic Book’, explore virtual LEGO models, unlock new characters and items through LEGO play. LEGO BrickHeadz Builder is currently retired from the Google Play store.
LEGO Playgrounds (2018)
In late 2018, LEGO Playgrounds launched as the first-ever app-based portal to discover and play with the latest LEGO mixed reality experiments. The first LEGO Playgrounds experiment featured the popular LEGO NINJAGO theme, offering an AR gaming experience that detects select physical LEGO Ninjago Dragon Hunters sets and brings them to life in the digital world. Players can choose their favorite Ninjago characters, share power-ups, compare high scores, and create unique gaming experiences. Multiplayer mode allows players to play in a shared space thanks to Apple’s ARKit technology, as showcased at WWDC. LEGO Playgrounds is currently retired and no longer available on the app store.
LEGO Hidden Side (2019)
LEGO Hidden Side is an AR-enhanced LEGO paranormal play theme where kids must turn a chilling haunted world back to normal, one ghost at a time. Kids build a series of “haunted” multi-layered building sets, then use the free AR companion app to solve interactive mystery challenges, hunt and catch ghosts, and explore new game challenges. With the all-new multiplayer mode, LEGO Hidden Side delivers fast-paced, competitive gameplay kids love coming back to, again and again. Whether you have the physical bricks or not, players can experience a little bit of Hidden Side magic by downloading the app Hidden Side was retired by LEGO in 2020 and the app is currently available.
LEGO – Through The Bricks (2020)
To tease the release of the anticipated Harry Potter ‘Diagon Alley’ LEGO sets, Makemepulse collaborated with LEGO to create an immersive world using 8th Wall’s web AR platform. Through The Bricks pays homage to the iconic scene in the first Harry Potter movie in which Hagrid takes Harry “through the wall” to experience Diagon Alley for the very first time. Fans can bring their favorite characters to life anywhere and build their own QR code to create a wondrous wizardry portal to the site.
LEGO Vidiyo (2021)
In 2021, LEGO Group and Universal Music Group (UMG) announced LEGO VIDIYO, a music video maker for kids, that combines LEGO bricks with AR tech that enables future music creatives to direct, produce, star in, and share their own music videos using chart-topping tracks from UMG’s unrivaled artists, including L.L.A.M.A, the first-ever human version of a LEGO Minifigure signed to a global record deal. LEGO VIDIYO uses AR to bring a series of BeatBoxes, Bandmates, BeatBits to life. Kids can rebuild their everyday world to the beat of their own imagination. A fun, kid-safe, and monitored interactive social network app that turns screen time into creative time, LEGO VIDIYO is an affordable, accessible play experience that sets the stage for the next generation of young music creatives.
LEGO Augmented Entertainment – Viewing & Gaming Experiences
LEGO Nexo Knights (2015)
In 2015, LEGO announced LEGO Nexo Knights. Set in the futuristic, high-tech world of the Kingdom of Knighton, the series centers around five young knights and their quest to battle Jestro, the Book of Monsters, and the Lava Monster Army. LEGO NEXO Knights bridged physical and digital play with building sets, an episodic television series, magazines, and a digital gaming app that delivered augmented live-action story-driven gameplay. To provide a consistent style and connection to Nexo Knights, LEGO incorporated Vuforia’s VuMark™ to create scannable Nexo Knight shields that appear on the physical LEGO box sets, magazines, and the TV show. In the TV series, viewers were able to scan new LEGO NEXO KNIGHTS powers directly to the app, further connecting their play experience.
In 2018, at the Nuremberg Toy Fair, LEGO announced the end of LEGO Nexo Knights.
FOX LEGO Masters WebAR Builder (2020)
To accompany FOX’s LEGO Masters television program, Trigger developed the FOX LEGO Masters Web AR Builder using 8th Wall’s innovative web AR platform, allowing viewers at home to build their own LEGO creations in AR alongside the show’s master builders. Each week, the show featured new challenges for viewers to compete for a chance to win tickets to LEGOLAND, a LEGO Masters Build Box, and be featured on official LEGO Masters social channels.
Connected Lenses (2021)
Snapchat and LEGO teamed up to introduce a new type of AR-enhanced Snapchat Lens called Connected Lenses, which allows people to collaboratively build LEGO creations together through Snapchat — even if they’re not in the same physical space or environment.
LEGO Augmented Retail, Spaces & Places – Real-World Experiences
LEGO Digital Box (2009)
In 2009, LEGO transformed retail through the power of AR with LEGO Digital Box. For the first time, LEGO shoppers could scan their selected LEGO kits to see how they would look after they are built. LEGO teamed up with Munich, Germany-based AR software developer Metaio, acquired by Apple in 2015, to bring the LEGO Digital Box to life. Today, the LEGO Digital Box continues to bring LEGO wonder to life across LEGO stores around the world.
LEGO In-Store Action Model Box Demo (2016)
To celebrate LEGO fandom, LEGO and Trigger, created a LEGO In-Store Action App treating fans and shoppers worldwide to special AR meet and greets with their favorite Minifigures. After downloading the app, kids can visit a LEGO store and watch as minifigs like Santa Claus, Batman, and Nexo Knight Clay spring to life, offering the ultimate augmented photo op. Select retailers, such as Target and Toys “R” Us, featured special AR experiences, enhancing 4-foot LEGO display model boxes with 3D animations and sounds to bring the retail displays to life.
LEGO House Fish Designer (2017)
In 2017, the LEGO House, aka “the home of the brick,” opened in Billund, Denmark. The heart of LEGO House is the experience zones. The Fish Designer, created in collaboration between LEGO and Trigger, allows guests to build a physical LEGO brick fish and then scan their custom design into a digital aquarium where it swimmingly comes to life with other guests’ fish. The LEGO Fish Designer was a 2018 SXSW Innovation Award winner and continues to be one of the highest performing digital experiences in the LEGO House.
Hidden Side WebAR (2020)
WebAR experiences continue to grow, offering developers a new way to create more playful, accessible experiences that don’t require a dedicated app in order to play. LEGO Retail collaborated with Trigger to develop the LEGO Hidden Side WebAR experience built on 8th Wall’s web AR platform, bringing AR and play into retail environments. Customers scan a QR code to experience their first taste of LEGO’s new AR-enabled product line, LEGO Hidden Side, and launch into web-based tap-and-shoot gameplay seeking out floating AR ghosts to capture.
NYC LEGO Flagship Store (2021)
LEGO launched its groundbreaking ‘retailtainment’ concept at its flagship NYC store last year, pushing the boundaries of traditional retail once again by creating a destination for kids, families, ADFOL (adult fans of LEGO), and others to explore, rejoice, and celebrate the beloved brick. Located on 5th Avenue in New York City, the installation features several first-of-a-kind immersive shopper experiences designed to inspire creativity and imagination.
Among the augmented physical-digital experiences, the NYC flagship includes The Brick Lab, a uniquely interactive 20-minute themed immersive experience with interactive animated content, lighting, sound, and music. The Personalisation Studio invites visitors to make a memento in LEGO form, either as a portrait from the Mosaic Maker or an iconic Minifigure from the LEGO Minifigure Factory and more.
Reflecting on LEGO Innovation
We reached out to LEGO’s Creative Play Lab, the creators of the future of LEGO Play, to learn more about LEGO’s perspective on augmenting our world.
According to Tom Donaldson, Senior Vice President, Creative Play Lab, “While the LEGO System in Play will always be at the heart of everything we do, augmented reality technologies have allowed us to wrap digital layers around the LEGO brick to extend play experiences into the digital world, and we have explored how we can best do so in various ways. Our experiences and learnings since we launched our first app featuring augmented reality in 2011 have helped pave the way for our AR play theme LEGO Hidden Side – merging simultaneous physical and digital play. LEGO VIDIYO is our latest augmented reality experience blending physical and digital play.”
When asked to reflect on the AR products and experiences he’s most proud of during his time at LEGO, he stated, “From those, I have personally helped develop, LEGO VIDIYO stands out as it showcases how augmented reality can be used to create unique digital experiences that put kids in charge of things that are happening in the real world. It taps into kids’ creativity in a powerful way. LEGO Hidden Side is another example that captures the best of traditional LEGO model building and adds real depth by allowing children to combine play elements from both the physical and digital world.”
Fast Forward >> What does the future hold?
Unfortunately, we do not have access to a LEGO brick-built crystal ball. What we do know, however, is that people continue to crave AR gaming experiences. The global AR gaming market is due to reach US$33.5 billion by 2027, up from US$3 billion in 2020. That’s an anticipated growth of over 1017% according to Research and Markets, 2020. On top of that, AR retail continues to skyrocket.
In a study by Snap and Deloitte Digital, there will be four key growth areas over the next year (media, shopping, gaming, and communications) and over the next five years, 74% of consumers believe that AR will be even more important in their lives.
We asked several industry leaders their perspectives on what the future holds for AR technology.
According to Tom Donaldson, Senior Vice President, Creative Play Lab, “Strengthening the physical part of augmented play will be key going forward. The power of the digital world is great, but we need to remember that the physical world is an incredible place.” He further noted, “Children are playing more fluidly than ever before. They move seamlessly between play in the real world and play in the digital world. Therefore, I believe that having a hand in both the physical and digital world with an increased focus on physical elements will be the most important trend in creating the best play experiences in the future.”
“Augmented Reality has become an essential part of the marketing and advertising mix,” says Tom Emrich, VP of Product at 8th Wall. “Brands and agencies are using WebAR to engage with consumers wherever they are and no matter what device they are on–all while not requiring an app to download.”
John Macbeth, Sales Director and Co-founder of Hoopla Digital states, “There’s been a significant escalation over the last 2-3 years from a point where brands were mostly just looking at app-based executions offering face masks or static models. Now we are seeing a real shift in consideration of how AR can be used from both a reach (with WebAR), & engagement perspective (with fully interactive experiences that transcend traditional brand to consumer opportunities). It’s going to get to a point soon where AR is a standard consideration in media planning, just like video or other common creative assets are now.”
“There are a number of areas to look at like markerless body tracking for fluid placement of retail apparel like watches, shoes, etc, multiplayer real-time gaming/interactive experiences, live streaming, greater accuracy & speed of environment scanning with the advancements of Lidar,” stated Macbeth.
Trigger has been partnering with LEGO for nearly a decade and has created over 30 mixed reality prototypes for kid testing as well as designed XR experiences across a variety of teams and themes including toys, apps, retail, and LEGO House, the LEGO museum in Billund, Denmark.
Jason Yim, CEO of Trigger states, “I think everyone understands that kids seem “all-in” when it comes to digital entertainment whether it be video, games, and toys. But from our LEGO experience, we’ve grown to understand that combining digital and physical in a meaningful and successful way is very difficult and time-consuming. Making a great physical toy is hard, making a great digital experience is hard, making something that combines the two is twice as hard. However, we are hoping that tools and technology improvements will make this combination easier.”
“A lot of the friction in an XR toy can be around very mundane tasks like logging into wifi, connecting pieces over Bluetooth, etc… that are no FUN for anybody, but have to be done and can taint the entire experience. But hopefully, these start to “just work”. When HMDs are safe and affordable for kids, that will change the industry. A phone as an extra play element still creates a barrier – hands-free will be the most natural way to play. (However, it will open up its own challenges to solve too),” stated Yim.
For nearly 90 years, LEGO taught us to “play well” through imagination and for more than a dozen years helped us create, play, and extend our world with the beloved brick. With web-based AR experiences on the rise and AR glasses set to explore over the next few years, expect to see even more augmented play in our everyday lives moving forward. Like LEGO, smart, future-focused brands will team up with innovative partners to test, learn, and create purposeful connected experiences that impact the way we play, learn, and interact with the world.
Image Credit: LEGO
