Well, it has been an interesting week to say the least!
A couple of years back, we were blown away at Siggraph when we saw how Jon Favreau had used his ‘volume’ and virtual production technology in Unreal for a fantastic looking TV series like the Mandalorian. Ever since, we have been pushing gaming technology within our organisation whenever we saw an opportunity for it, and we built up our internal Unreal knowledge by testdriving it in small scale TV and online productions of all kinds.
In this case we used Unreal engine in combination with an SDI I/O card and a couple of motioncontrolled PTZ camera’s to bring several speakers to a live audience during 5 days of live keynote sessions. Our most ambitious use of the technology yet and the pressure was on ! We were not in Kansas anymore now.
The 3D coordinates from the PTZ streamed in real time to the unreal engine workstations and we used Composure and other Virtual Production tools and plugings in Unreal to put our speakers, filmed in a green key, in a fully customizeable unreal 3D environment that would adjust according to the zoom/xyz of the PTZ camera’s. We developed a lot of blueprint code to make things like colorspaceconversions work correctly and to ensure that any transitions we made during live broadcast, from virtual cameras to composure camera’s would be as seamless as possible. We also used a streamdeck (and several sdi to hdmi convertors) to make the whole setup as userfriendly as possible.
The last day of the week there was a kind of virtual afterparty with a live DJ and we took the opportunity to put all kinds of animated stuff (including niagara particles) and environments (like the moon) in the scene.
Framerate sometimes became an issue when too much stuff was happening in combination with animated dynamic lighting and the heavy composure lifting etc and we ran into some last minute issues with our postprocessing volumes etc. But in the end we were really happy with the final results and the stability of the whole setup was rocksolid.
Even our broadcast crew seemed impressed by the kind of quality outputted by this costeffective setup and the future potential of this very flexible and largely softwarebased setup was clear.
Next up; we really need to get our hands on some HQ LED walls, some motion capture suits and some DMX light stuff so that we can make our next iteration even cooler and more interactive. Hopefully one day, we can shoot our own Mandalorian series ;-)…..