Optical Flow Explained: The Key to Seamless Image Stitching

The Insta360 Pro ensures a seamless stitch using the science of optical flow.

360-degree cameras face a thorny problem: How do you take visual data captured by multiple lenses, facing multiple directions, and then produce a final image that blends all of this information into one seamless whole?

Add on the fact that the various subjects of these lenses may be moving unpredictably — crossing from the field of view of one lens to another and then back again, hovering about the zone where two lenses overlap, or shifting between the foreground and background of a scene at different rates— and the problem gets a whole lot thornier.

It’s a little like trying to solve a jigsaw puzzle where the image on every piece is changing in real time. And it’s the reason you've probably seen shots that turn out like this:

Toronto's CN tower has seen better days. 

The Insta360 Pro solves this problem with a cutting-edge stitching technology that’s based on the science of optical flow.

Optical flow is a subject worthy of an army of PhD’s, but the basic idea is simple enough.

Instead of stitching together images according to a one-size-fits-all rule – essentially programming a computer to match dotted line A with dotted line B – the optical flow approach lets a computer keep track of the actual content of an image, down to the level of individual pixels.

Take the Insta360 Pro, with its six independent lenses, as an example. An optical-flow-based stitching algorithm allows for a granular comparison of the six images captured by the Pro's lenses, so that the images can be stitched together intelligently, according to their actual subjects.

In practice, this means that the Insta360 Pro can avoid taking shots like the one of Toronto's CN tower above, where a key object has fallen victim to an indiscriminate stitching algorithm. Have another look:

The problem is obvious to any human, but how do you explain it to an algorithm? If the stitching algorithm had been able to notice the stark shift in the arrangement of pixels at the seam where the tower is cut in half – a sudden jump from gray concrete to blue sky – it wouldn’t have made this mistake. By letting a computer actually “see” what’s happening in an image, optical-flow-based stitching avoids this type of problem.

In the case of video capture, the Insta360 Pro’s optical flow stitching algorithm works on an even more sophisticated principle. Not only does it compare the images captured by each of the Pro’s six lenses for any given frame of a video, but it also compares an image captured by a single lens to images captured by that same lens in adjacent frames of the video.

The upshot: Seamless stitching that works with the content of your shoot instead of against it. Compare the two shots below to see how optical flow can save your subjects:

Optical-flow-based image stitching adjusts a stitch according to the actual content of a shot. 

Optical flow stitching ensures that creators using the Insta360 Pro can transport their viewers into new worlds, worlds that look just like they should – seamless.

Keen to keep up to date on Insta360 stories? Keep an eye on our blog and sign up for our mailing list.