使用cv2.calcOpticalFlowPyrLK()查找光流作为ndarray [英] To find optical flow as ndarray using cv2.calcOpticalFlowPyrLK()

问题描述

我需要使用Lucas Kanade光流找到视频的每2个相邻帧之间的光流.我正在为项目使用python和openCV.

I need to find the optical flow between every 2 adjacent frames of a video using Lucas Kanade optical flow. I'm using python and openCV for the project.

据我了解，卢卡斯·卡纳德(Lucas Kanade)是找到光流的稀疏方法.是否有密集的实现?如果是这样，如何在python中使用它?

As per my understanding Lucas Kanade is a sparse method to find optical flow. Is there a dense implementation of it? If so, how to use it in python?

使用cv2.calcOpticalFlowFarneback()(这是一种密集方法)，我们得到包含光流的ndarray(以下示例中的"flow")作为输出.

Using cv2.calcOpticalFlowFarneback(), which is a dense method, we get as output an ndarray ('flow' in below example) which contains the optical flow.

cv2.calcOpticalFlowFarneback(prev, next, pyr_scale, levels, winsize, iterations, poly_n, poly_sigma, flags[, flow]) → flow

是否可以使用cv2.calcOpticalFlowPyrLK()获得类似的输出?

Is there a way to get a similar output using cv2.calcOpticalFlowPyrLK()?

cv2.calcOpticalFlowPyrLK(prevImg, nextImg, prevPts[, nextPts[, status[, err[, winSize[, maxLevel[, criteria[, flags[, minEigThreshold]]]]]]]]) → nextPts, status, err

使用cv2.calcOpticalFlowPyrLK()(如上)时，获得的输出包含nextPts，该nextPts包含要跟踪的下一个点，但不直接给出2帧的光流.如果我从nextPts中减去prevPts，结果是两个帧之间的光流吗? 我在解释此链接中的calcOpticalFlowFarneback()的部分中找到了'prev(y，x)〜next(y + flow(y，x)[1]，x + flow(y，x)[0])': href ="https://docs.opencv.org/2.4/modules/video/doc/motion_analysis_and_object_tracking.html" rel ="nofollow noreferrer"> https://docs.opencv.org/2.4/modules/video/doc/motion_analysis_and_object_tracking.html 因此是个问题. (cv2.calcOpticalFlowPyrLK()和cv2.calcOpticalFlowFarneback的语法也来自此链接) 下面是我的实现方法.

When using cv2.calcOpticalFlowPyrLK() (above), the output obtained contains nextPts which contains the next points to track but it does not directly give optical flow of the 2 frames. If I subtract the prevPts from nextPts, is the result the optical flow between the two frames? I found ' prev(y,x)~next(y+flow(y,x)[1],x+flow(y,x)[0]) ' in the section explaining calcOpticalFlowFarneback() in this link: https://docs.opencv.org/2.4/modules/video/doc/motion_analysis_and_object_tracking.html hence the question. (The syntax for both cv2.calcOpticalFlowPyrLK() and cv2.calcOpticalFlowFarneback are also from this link) Below is my implementation of the same.

import cv2
import numpy as np
import os
import subprocess as sp

yuv_filename = 'can_0.yuv'
flow=[]

width, height = 320, 240

file_size = os.path.getsize(yuv_filename)
n_frames = file_size // (width*height*3 // 2)
f = open(yuv_filename, 'rb')


old_yuv = np.frombuffer(f.read(width*height*3//2), dtype=np.uint8).reshape((height*3//2, width))


# Convert YUV420 to Grayscale
old_gray = cv2.cvtColor(old_yuv, cv2.COLOR_YUV2GRAY_I420)


# params for ShiTomasi corner detection
feature_params = dict( maxCorners = 100,
                       qualityLevel = 0.3,
                       minDistance = 7,
                       blockSize = 7 )

# Parameters for lucas kanade optical flow
lk_params = dict( winSize  = (15,15),
                  maxLevel = 2,
                  criteria = (cv2.TERM_CRITERIA_EPS | cv2.TERM_CRITERIA_COUNT, 10, 0.03))


p0 = cv2.goodFeaturesToTrack(old_gray, mask = None, **feature_params)


for i in range(1,n_frames):
    # Read Y, U and V color channels and reshape to height*1.5 x width numpy array
    yuv = np.frombuffer(f.read(width*height*3//2), dtype=np.uint8).reshape((height*3//2, width))

    # Convert YUV420 to Grayscale
    gray = cv2.cvtColor(yuv, cv2.COLOR_YUV2GRAY_I420)

    # calculate optical flow
    p1, st, err = cv2.calcOpticalFlowPyrLK(old_gray, gray, p0, None, **lk_params)


    flow.append(np.subtract(p1,p0))
    good_old = p0[st==1]
    good_new = p1[st==1]


    # Now update the previous frame and previous points
    old_gray = gray.copy()
    p0 = good_new.reshape(-1,1,2)


f.close()

是否流过一个ndarray列表，其中包含输入视频的相邻帧之间的光流?

Is flow a list of ndarrays containing the optical flow between adjacent frames of the input video?

这是我得到的流量[7]的输出(随机选择)，但是我不确定它是否是光流量值.

This is the output I got for flow[7] (randomly selected) but I'm not sure if it is optical flow values.

[[[6.7138672e-03 4.5318604e-03]]

[[[ 6.7138672e-03 4.5318604e-03]]

[[-1.6220093e-02 1.9645691e-03]]

[[-1.6220093e-02 1.9645691e-03]]

[[-8.5296631e-03 1.8482208e-03]]

[[-8.5296631e-03 1.8482208e-03]]

[[-5.8441162e-03 1.5701294e-02]]

[[-5.8441162e-03 1.5701294e-02]]

[[7.5836182e-03 2.3475647e-02]]

[[ 7.5836182e-03 2.3475647e-02]]

[[-1.4129639e-02 1.6357422e-02]]

[[-1.4129639e-02 1.6357422e-02]]

[[4.4555664e-03 4.1809082e-03]]

[[ 4.4555664e-03 4.1809082e-03]]

[[5.6457520e-04 -9.5863342e-03]]

[[ 5.6457520e-04 -9.5863342e-03]]

[[2.8991699e-04 -3.0517578e-05]]

[[ 2.8991699e-04 -3.0517578e-05]]

[[-2.3452759e-02 -1.5502930e-02]]

[[-2.3452759e-02 -1.5502930e-02]]

[[-6.8283081e-03 3.3264160e-03]]

[[-6.8283081e-03 3.3264160e-03]]

[[-8.4381104e-03 7.7590942e-03]]

[[-8.4381104e-03 7.7590942e-03]]

[[5.7144165e-03 1.1177063e-02]]

[[ 5.7144165e-03 1.1177063e-02]]

[[-1.4160156e-02 2.1179199e-02]]

[[-1.4160156e-02 2.1179199e-02]]

[[-1.0498047e-02 8.0099106e-03]]

[[-1.0498047e-02 8.0099106e-03]]

[[-1.8310547e-04 2.8953552e-03]]

[[-1.8310547e-04 2.8953552e-03]]

[[4.4937134e-03 -2.0904541e-03]]

[[ 4.4937134e-03 -2.0904541e-03]]

[[-4.7698975e-02 3.7708282e-02]]

[[-4.7698975e-02 3.7708282e-02]]

[[6.3323975e-03 1.3298035e-02]]

[[ 6.3323975e-03 1.3298035e-02]]

[[-3.3233643e-02 -1.7229080e-02]]

[[-3.3233643e-02 -1.7229080e-02]]

[[7.5683594e-03 2.4566650e-03]]

[[ 7.5683594e-03 2.4566650e-03]]

[[-3.0364990e-03 3.4656525e-03]]

[[-3.0364990e-03 3.4656525e-03]]

[[-1.0345459e-02 -7.4539185e-03]]

[[-1.0345459e-02 -7.4539185e-03]]

[[1.3168335e-02 2.1423340e-02]]

[[ 1.3168335e-02 2.1423340e-02]]

[[-6.3476562e-03 -1.0681152e-02]]

[[-6.3476562e-03 -1.0681152e-02]]

[[1.5869141e-03 1.0375977e-03]]

[[ 1.5869141e-03 1.0375977e-03]]

[[2.1820068e-03 6.7329407e-03]]

[[ 2.1820068e-03 6.7329407e-03]]

[[-9.6130371e-03 2.9449463e-03]]

[[-9.6130371e-03 2.9449463e-03]]

[[-2.1362305e-03 8.5525513e-03]]

[[-2.1362305e-03 8.5525513e-03]]

[[-1.7547607e-04 2.1362305e-04]]

[[-1.7547607e-04 2.1362305e-04]]

[[2.9144287e-03 1.4343262e-03]]

[[ 2.9144287e-03 1.4343262e-03]]

[[2.9602051e-03 -7.1868896e-03]]

[[ 2.9602051e-03 -7.1868896e-03]]

[[-1.2878418e-02 5.0182343e-03]]

[[-1.2878418e-02 5.0182343e-03]]

[[-3.1585693e-03 -5.0544739e-05]]

[[-3.1585693e-03 -5.0544739e-05]]

[[1.0070801e-03 1.3740540e-02]]

[[ 1.0070801e-03 1.3740540e-02]]

[[6.7138672e-04 1.7852783e-03]]

[[ 6.7138672e-04 1.7852783e-03]]

[[-2.0568848e-02 -1.2943268e-02]]

[[-2.0568848e-02 -1.2943268e-02]]

[[-2.1057129e-03 4.5013428e-03]]]

[[-2.1057129e-03 4.5013428e-03]]]

还有没有一种方法可以使用Lucas Kanade方法获得光流，使其具有与输入帧相同的大小或尺寸?如果可以，怎么办?

Also is there a way to get optical flow using Lucas Kanade method such that it has the same size or dimensions as the input frames? If so how?

推荐答案

您可以通过为每个像素计算对应的点对，使用纯金字塔形Lucas Kanade方法估算密集的流场.为此:

You can estimate a dense flow field with the pure pyramidal Lucas Kanade method by computing for each pixel the corresponding point pairs. To do that:

• 忽略cv2.goodFeaturesToTrack

• 初始化包含图像所有像素位置的点列表p0

• ignore cv2.goodFeaturesToTrack

• initialize point list p0 that contains all pixel locations of the image

grid_y, grid_x = np.mgrid[0:prevImg.shape[0]:1, 0:prevImg.shape[1]:1]
p0 = np.stack((grid_x.flatten(),grid_y.flatten()),axis=1).astype(np.float32)

运行备用流量计算

run the spare flow computation

p1, status, err = cv2.calcOpticalFlowPyrLK(prevImg, currImg, p0, None)

通过计算组装稠密的流场并重塑运动矢量(p1-p0)，流的形状将为(h x w x 2)和prevImg(h x w x 1)

assemble dense flow field by computing and reshape motion vectors (p1 - p0) flow will have the shape (h x w x 2) and prevImg (h x w x 1)

flow = np.reshape(p1 - p0, (prevImg.shape[0], prevImg.shape[1], 2))

但是，这不是计算密集的光流场的最有效方法.在OpenCV中，您可以查看其他 DenseOpticalFlow方法(例如DIS，Farneback，RLOFDense，Dual-TVL1)

However, this is not the most efficient way to compute a dense optical flow field. In OpenCV you may take a look to other DenseOpticalFlow methods (e.g. DIS, Farneback, RLOFDense, Dual-TVL1)

这篇关于使用cv2.calcOpticalFlowPyrLK()查找光流作为ndarray的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！