Volumetric capture adds a layer of depth to the pop legends latest performance.
Earlier this month Billboard Magazine held its annual awards ceremony in Las Vegas, Nevada, and while most of us are still swooning over that incredible performance by Korean demigods BTS, pop music icon Madonna is receiving her own fair share of praise thanks to an interesting live performance that used AR technology to bring several versions of the legend to life on-stage at once.
During the May 1st festivities at the MGM Grand, Madonna appeared on-stage opposite Maluma for a performance of her new song “Medellín” that included the artists usual helping of abstract visuals and seductive dancing; all of which enhanced by a variety of AR elements, including environmental set pieces as well as dancing clones of the pop star
Playing into the role of the mysterious Madame X, Madonna danced alongside a variety of AR personas, including a secret agent, a cha-cha instructor, a bride, and a musician. Brought to life using a combination of Unreal Engine and volumetric capture, these augmented holograms would appear sporadically throughout the performance, disappearing into puffs of smoke around the original Madonna seemingly at random.
The multifaceted performance was brought to life by Sequin, a new AR company founded by members of The Future Group—a team of virtual production specialists responsible for the incredible mixed reality flooding and blizzard reenactments used by The Weather Channel, as well as the 2018 live performance of Riot Games’ virtual k-pop girl group, K/DA.
“What’s new about this is that it’s a completely choreographed performance where Madonna and Maluma are dancing with four digital versions of Madonna in perfect choreography,” spoke cofounder Lawrence Jones in an interview with Engadget.
“The interesting thing about real-time visual effects in broadcast augmented reality [is that] a big portion of the work is happening in pre-production. All the creation of the assets, all of the animation, most of the lighting is all done ahead of time.”
In the end, three AR-capable cameras in total were used during the live performance. This included a standard front-of-house camera, a moving crane, and a wireless Steadicam; a similar setup to K/DA’s 2018 performance.
Madonna recorded her volumetric capture performances at a studio in London; meanwhile, a company in Canada produced the digital assets for the various AR weather effects. The team at Sequin then used a tool called Brainstorm to ensure all the live video feeds being fed into Unreal Engine timed up perfectly with the various digital broadcast objects being layered on top.
