Zubr has been working with volumetric video for almost a decade, creating content that has been experienced by over 100k people. If you’re not familiar with the term, volumetric video capture is a technique that captures a three-dimensional space – Zubr specialises in this to bring you real-world performances in AR, VR and holographic platforms.
This technique for creating immersive experiences is ideal for moving subjects, enabling the creator to capture the movement and geometry of performances with depth-sensing cameras. Volumetric video is the true-3D equivalent of conventional video. It captures the visual likeness of the subject just like ordinary video; but also grabs its geometric shape per frame. That means that viewers will see a 3D ‘hologram’ conveying the authentic shape and movement of a living subject, without intervention from digital artists or developers.
The best way to harness the strengths of volumetric video is to enable the audience to explore the content in realtime 3D. That means using video game technology to display the visuals dynamically, as opposed to rendering it out as an ordinary video which would waste its 3D properties. The Zubr team capitalises on their augmented and virtual reality expertise to create a perfect match with volumetric video. Zubr has countless examples of how they’ve executed this in a variety of projects, so if you’re still feeling bewildered as to how this works in practice, take a look at some of these outstanding projects.
From Every Angle - Bristol Proms
This project was a simulcast live production for two separate audiences. In 2013, From Every Angle created an enhanced live performance in the historical auditorium of the Bristol Old Vic Theatre and a simulcast, split-screen, cinematic presentation at the Watershed Cinema. Zubr’s co-founder, Jack Norris, was part of the VFX team that created the live volumetric visuals, a year before Zubr was founded.
This piece at the Bristol Proms by BDH was groundbreaking in how it used real-time volumetric capture alongside lasers and live camera feeds to create live digital visuals.
Jack reflects on his time working on this project: “It’s amazing now to think back to this project almost a decade later and realise that, in terms of technology and execution, there is barely a single thing about it that would be better if we did it now. At the time it was a moment of realisation for me; the potential of volumetric video, even when in a crude, triangular form, was enormous.”
Featuring Pianist Jan Lisiecki, Kinect cameras captured his extraordinary dexterity and exertion – augmented live into 3D animation. High-speed cameras caught intimate details of the performance, and it was a glimpse into the future for volumetric video. After this in 2014, Zubr was founded with the purpose of exploring volumetric video experiences. Jack undertook many experimental tests with various theatre and dance practitioners around Bristol and started to grow the team.
Experimental render modes, stunning environments and unprecedented audience viewpoints were all utilised to create compelling digital pieces.
Immersion Dance
In 2015, Zubr began an Arts Council-funded R&D project to explore the potential of hologram capture and augmentation of dance performances. Hosted by Bristol’s Pervasive Media Studio, Zubr worked with Julia Thorneycroft Dance to explore cutting-edge methods of dance capture that preserve the authentic physical data of the performance. Using Kinect cameras and smartphones to create a series of AR and VR demonstrators, viewers could enjoy dance performances from the comfort of their sofa. The finished pieces were showcased to families at the Bristol Children’s Hospital.
We the Curious capture suite
Back in 2016, Zubr teamed up with science centre We The Curious to create the world’s first automated volumetric capture suite. This project invited people to create their own 4D holograms in a unique immersive visitor attraction experience. The suite was wildly successful, capturing over 50,000 volumetric clips and was live non-stop for two years.
Jack tells us, “Our volumetric capture suite at We the Curious is still one of my favourite Zubr projects yet! Volumetric video was completely off the radar back then, and yet there we were watching whole families, couples and people of all ages making 3D captures of themselves with no staff needed to help them.
“It was completely organic and very much ahead of its time. In fact, it’s only now, six years later, that we’re seeing similar capture suites becoming available for commercial use. But We the Curious can always say that they had primary school children making their own volumetric captures before most creative industry professionals had even heard of it!”
Each visitor or group was recorded in a 10-second performance, which was then viewed on custom We the Curious virtual reality headsets as an animated 3D model. The system was fully automated and accommodated one capture per minute.
Volumetric music videos
Since 2016, Zubr has used volumetric video on a number of creative projects. The potential for volumetric video application spans far and wide, as Zubr has aptly demonstrated this through the diversity of its work. This includes collaboration with musicians to create unconventional music videos. The jagged visual style of many volumetric capture technologies is perfect for music videos, and avoids the pitfall of the ‘uncanny valley’ that often comes with 3D visuals that are too smooth and polished.
One example is this volumetric music video with Aïsha Devi, commissioned for Simple Things Festival, at which the 5G multi-user VR piece built around a hologram of the electronic artist was showcased. Working with the University of Bristol’s Smart Internet Lab, Zubr brought Aïsha Devi’s track I’m Not Always Where My Body Is into an ambisonic virtual reality music video, nestled amongst otherworldly visuals supplied by Polish creative duo, Pussykrew.
Having directed a volumetric video capture session of Aïsha, they transformed her performance into a holographic rendering as the centrepiece of the experience. Millennium Square played host to the VR piece loaded onto 5G-enabled VR headsets, where groups of 15 festival goers embarked on a shared musical trip.
What’s in store for the future?
It’s an exciting time for volumetric video creation and it has grown in popularity over the past decade as other companies have transitioned into the space, particularly in the South West. Most recently Condense announced the opening of their world’s first metaverse studio, meaning true-to-life live performances can be streamed directly into the virtual worlds and games platforms used by billions of people worldwide – without the need for VR headsets. The Bristol MyWorld project may bring crossover with volumetric video facilities for high-end TV and Film applications so many exciting developments still to come on that one.
For Zubr, its focus will now be directed to accessible, platform-agnostic and open source approaches to volumetric capture, led by the demands of its commercial projects. Jack reflects that “Back in 2014, we founded Zubr purely for exploring volumetric video. With the availability of depth-sensing cameras and high performance smartphones, that’s when the potential of volumetric content was really clear to us.
“In 2022, looking back on the 200 projects we’ve completed since then, volumetric video has become more of a niche area to us rather than the essential ingredient. Quite often it’s simply more appropriate to use rigged avatars, or even stereoscopic video. But we still love the raw, real-world capture aesthetic that you can only get from volumetric content, and we know that audiences are still enthralled by it too.”
Editorial Month
Sponsored By
Immersive Month
