Starting today, Insta360 8K Live Software is available to broadcasters everywhere.
History was made at this year’s NFL Pro Bowl, the annual all-star football game. But rather than a record-breaking endzone run, it was a VR football experience that made this year’s game special.
A team from Verizon, Radiant Images and Flight School Studio pulled off the world’s first 8K 360-degree live broadcast of an NFL game. The feed was integrated into ESPN’s live broadcast and projected into a dome 200 miles away in Miami. To make the historic broadcast possible, the team used Verizon’s 5G network and two Insta360 Pro 2 360 live streaming cameras.
The other key piece of the workflow was the newly launched Insta360 8K Live software. After testing and refinement at the hands of teams like Radiant Images, this professional VR live streaming software is now officially available to broadcasters everywhere.
Using Insta360 8K Live, broadcasters can stitch and broadcast immersive video in 8K, or insert high-res, reframed video feeds into a traditional SDI broadcast in real time. The NFL broadcast was among the first applications of this VR live streaming software, demonstrating its groundbreaking potential at a time when COVID-19 has underlined the need for new ways to enjoy live events.
“What got us really excited about this project was all the applications that it’s going to inspire in the future. Along with immersive broadcast, the ability to take a 360 feed and reframe it live and plug those individual shots into a traditional live TV broadcast is huge,” said Michael Shabun, VP of Marketing and Business Development at Insta360. “Broadcasters are realizing what a game-changer it is to have a professional 360 cam as part of their live coverage.”
We sat down with Michael Mansouri, co-founder of Radiant Images, to learn more about how this team of innovators achieved this VR football experience. Read on to find out what their workflow entailed and how you can realize your own 8K VR live streaming project.
1. VR live streaming the game with a multi-cam setup
VR live streaming the Pro Bowl from Orlando to a dome in Miami over 200 miles away using 5G was a new challenge for Mansouri. “To us, it's very challenging because no one has ever done this before, in this scale, in this kind of resolution, with this distance — and all on 5G,” Mansouri said.
The team decided to use two Insta360 Pro 2 360 live streaming cameras to film the game—one as a goal post cam and one mounted on the skycam.
“We needed a 360 live streaming camera that's very, very small, lightweight, can stream high resolution, easily communicate back and forth, and was very reliable,” Mansouri said.
The Insta360 Pro 2 camera weighs only 1.5 kilograms (3.3 pounds), making it easy to mount in a variety of positions for unique angles and shots. This versatility was important because Pro 2 and ESPN’s traditional TV camera had to share the same skycam. For the entirety of the game, the Insta360 Pro 2 captured 360-degree footage in crystal-clear 8K.
2. Stitching 360 live streaming camera feeds with Insta360 8K Live
The biggest challenge with live broadcasting 360-degree footage is stitching the video feeds captured from multiple lenses together at full quality in real time.
With the new Insta360 8K Live software, six video streams from Pro 2’s six lenses can be stitched in real time in 8K quality. A dynamic stitching algorithm analyzes content in real time to optimize stitch quality and placement, ensuring the feeds are merged seamlessly. The8K Live software is also ready to stitch together Insta360 Titan’s eight individual video feeds.
Insta360 8K Live outputs a 2D HDR VR live stream in 30 fps, using RTSP/RTMP/RTMPS protocols and H.265 HEVC codec. It also enables simultaneous recording during a live stream so you can save footage at full quality for future playback or post-processing.
To use the software, broadcasters simply connect Insta360 Pro 2 to a server PC with an Ethernet cable or router that supports Gigabit Ethernet transmission. During the Pro Bowl broadcast, the team connected the skycam-mounted Pro 2 with a Gigabit Ethernet cable, then converted it to fiber to connect it to their remote server PC about 1,000 feet away. The team’s server PC’s were also equipped with two RTX 2080 Ti graphics cards for optimum image processing.
Shabun explained one of the biggest challenges for Insta360 with this project was “developing a way to transmit and convert our signal into SDI, which is something that broadcasters demand. But our engineers worked hand-in hand with Radiant Images and other leading broadcast industry professionals to develop a workflow in time for the Pro Bowl.
Mansouri valued the close collaboration of Insta360’s team of engineers in helping to pull off the project. With the Insta360 8K Live software, two Blackmagic 8K Pro DeckLink cards and software to connect the dots, Pro 2’s signal was successfully converted into a standard SDI output for broadcasting.
3. Choosing camera feeds to broadcast with the switcher control panel
A team of professionals was waiting by at the switcher control panel to receive the two stitched SDI video feeds from each of the two Insta360 Pro 2 cameras. In real time, they followed the action on both feeds and selected the optimum footage to broadcast using a Blackmagic ATEM Constellation 8K Switcher.
A switcher allows you to view all incoming video feeds and direct the footage you’d like to broadcast to the appropriate destination. In this case, the team only had two Insta360 Pro 2 feeds to monitor, but your team could integrate more feeds at this stage, including both 360 and standard video.
You can also choose at this stage whether to reframe your 360 footage or broadcast the original 360-degree footage. ESPN decided to broadcast an equirectangular projection from the Pro 2, giving viewers at home a feel for the unlimited field of view fans in Miami were experiencing.
4. Transmission to the dome with Verizon 5G
Once the team had a couple of stitched 8K video feeds, the next step was to convert the footage for transmission to the dome. A software or hardware encoder can convert an SDI feed into codecs like the H.264 or H.265 (HEVC) VR live streaming protocols.
But they faced a roadblock because their encoders weren’t spec’d for transmitting an 8K feed.
To jump this hurdle, the team got clever. They decided to use TouchDesigner, a live visual manipulation software, to split the 8K x 4K sphere into easily transmittable quadrants. By encoding and transmitting the 8K sphere piece by piece, they were able to receive the stream at full quality in Miami, and then reconstruct a desired hemisphere for projection into the dome.
If it took you a few moments to digest that, you’re not alone. The upshot of the team’s technical acrobatics, though, was that they managed to use standard broadcast equipment to transmit an ultra-high-res video over a 200-mile distance at lightning speed.
The final piece of the puzzle was a Verizon 5G router, which allowed them to overcome traditional bandwidth limitations. As Mansouri says, “With 4G, we're limited on bandwidth. We can't send high resolution, and if we do, there's latency. So what 5G solves is immediate latency. It happens instantly.”
5. Projecting the VR live stream into the dome
Once footage had been transmitted to the dome, it was time to project it to the audience.
Footage needs to be decoded before projection. In this case, the on-site team at the dome used a hardware decoder to convert the feed back into SDI format. They then used TouchDesigner to reconstruct a 4K x 4K hemisphere (the half of the original 8K x 4K sphere that could be mapped to the dome’s interior), which was then carefully repositioned for projection.
As Mansouri explains, “You can't just put things into the dome. You have to de-warp and position the feed correctly inside the dome.”
The Miami team confirmed the projection mapping and then projected the feed onto the dome using six projectors. The number of projectors needed varies based on the size of the dome.
There was only a 3-second delay from the game to the dome, thanks to the 5G transmission. The VR football experience allowed the audience to truly watch the game in real time as if they were really at the game.
Danny Driggers, a Verizon engineer, described the experience watching the VR football experience inside the dome as very immersive, “You feel like you’re there. You can feel every bit of it. It’s absolutely the future.”
“You feel like you’re there. You can feel every bit of it. It’s absolutely the future.”
The possibilities are endless for 5G VR live streaming technology and 360 live streaming cameras. In addition to VR domes, immersive 360 footage can also be live streamed to VR headsets, mobile phones, desktops and tablets.
Click the link below to learn more about Insta360 8K Live and download a free trial of the software.
Keen to keep up to date on Insta360 stories? Keep an eye on our blog and sign up for our mailing list. You can also contact our Enterprise team directly to see how 360 cameras can benefit your business.
%20(1).svg)
%20(1).svg)
.svg)
