Optical Flow

Compute optical flow on image sequences.

Input

One sequence of 2D RGB images.

Output

2-channel image containing flow in x and y directions on channel 0 and 1 resp.

Description

Runs optical flow algorithms registered in the framework on the supplied image set between consecutive images. The parameters are:

Show live preview - If checked, shows a live preview of the optical flow computation.

Mask label - If set, the optical flow is computed only on the pixels with the specified label. All other pixels receive a flow and uncertainty of NaN. -1 means all pixels are considered.

Remove invalid flow - If checked, removes invalid flow values, i.e. those that map to pixels outside the second image’s dimensions, from the output.

Two learning based algorithms Optical Flow RAFT and Optical Flow SEARAFT are available:

  1. The algorithm Optical Flow RAFT has following parameters (although default settings work well):

Init from previous computation - Marking this will store the computed optical flow internally and subsequent computation will initialize the flow from the previously stored flow. Setting to true can help in better quality results in case of temporally smooth videos.

Number of refinements - Setting to a value greater than zero will perform that many optical flow refinement steps. Usually zero is enough. If required a value in range [1, 3] can further improve the results.

Downsample to train resolution - Downsamples the input images before computing flow to have a maximum dimension of 1024 (if input images are larger). This can allow for better results since the network was trained on similar resolution images.

  1. The algorithm Optical Flow SEARAFT is an improved version of Optical Flow RAFT. It also outputs flow uncertainty in addition to regular optical flow. It has the following parameters

Use larger network - Uses larger machine learning model which can give slightly better results than the smaller (default) one at the expense of more computation. Downsample to train resolution - Similar to Optical Flow RAFT.