使用FFmpeg，Python和OpenCV显示流 [英] Display stream with FFmpeg, python and opencv

问题描述

情况:我有一个连接到树莓派的basler相机，并且我试图将它的FFmpg直播到Windows PC的tcp端口，以监视相机前面发生的事情.

Situation : I have a basler camera connected to a raspberry pi, and I am trying to livestream it's feed with FFmpg to a tcp port in my windows PC in order to monitor whats happening in front of the camera.

有效的东西:我设法在树莓派上设置了一个python脚本，该脚本负责记录帧，将它们馈送到管道并将它们流式传输到tcp端口.从该端口，我可以使用FFplay显示流.

Things that work : I manage to set up a python script on the raspberry pi which is responsible for recording the frames, feed them to a pipe and streaming them to a tcp port. From that port, I am able to display the stream using FFplay.

我的问题:FFplay非常适合快速，轻松地测试您要行驶的方向是否正确，但是我想阅读"流中的每个帧，进行一些处理，然后使用opencv显示流.那，我还没做.

My problem : FFplay is great for testing out quickly and easily if the direction you are heading is correct, but I want to "read" every frame from the stream, do some processing and then displaying the stream with opencv. That, I am not able to do yet.

最少代表，这就是我在树莓派方面使用的代码:

Minimaly reprsented, that's the code I use on the raspberry pi side of things :

command = ['ffmpeg',
           '-y',
           '-i', '-',
           '-an',
           '-c:v', 'mpeg4',
           '-r', '50',
           '-f', 'rtsp',
           '-rtsp_transport',
           'tcp','rtsp://192.168.1.xxxx:5555/live.sdp']

p = subprocess.Popen(command, stdin=subprocess.PIPE) 

while camera.IsGrabbing():  # send images as stream until Ctrl-C
    grabResult = camera.RetrieveResult(100, pylon.TimeoutHandling_ThrowException)
    
    if grabResult.GrabSucceeded():
        image = grabResult.Array
        image = resize_compress(image)
        p.stdin.write(image)
    grabResult.Release() 

在我的PC上，如果我在终端上使用以下FFplay命令，它将起作用并实时显示流:

On my PC if I use the following FFplay command on a terminal, it works and it displays the stream in real time :

ffplay -rtsp_flags监听rtsp://192.168.1.xxxx:5555/live.sdp?tcp

在我的PC上，如果我使用以下python脚本，则流开始，但由于不确定如何解码，它在 cv2.imshow 函数中失败:

On my PC if I use the following python script, the stream begins, but it fails in the cv2.imshow function because I am not sure how to decode it:

import subprocess
import cv2

command = ['C:/ffmpeg/bin/ffmpeg.exe',
           '-rtsp_flags', 'listen',
           '-i', 'rtsp://192.168.1.xxxx:5555/live.sdp?tcp?', 
           '-']

p1 = subprocess.Popen(command, stdin=subprocess.PIPE, stdout=subprocess.PIPE)

while True:
    frame = p1.stdout.read()
    cv2.imshow('image', frame)
    cv2.waitKey(1)

有人知道我需要在这两个脚本中进行哪些更改才能使我工作吗?

提前感谢您的提示.

推荐答案

您可以从 p1.stdout 中读取已解码的帧，将其转换为NumPy数组，然后对其进行整形.

You can read the decoded frame from p1.stdout, convert it to NumPy array, and reshape it.

• 更改命令以获取 rawvideo 格式和BGR像素格​​式的解码帧:

• Change command to get decoded frames in rawvideo format and BGR pixel format:

 command = ['C:/ffmpeg/bin/ffmpeg.exe',
            '-rtsp_flags', 'listen',
            '-i', 'rtsp://192.168.1.xxxx:5555/live.sdp?tcp?',
            '-f', 'image2pipe',    # Use image2pipe demuxer
            '-pix_fmt', 'bgr24',   # Set BGR pixel format
            '-vcodec', 'rawvideo', # Get rawvideo output format.
            '-']

从 p1.stdout 读取原始视频帧:

 raw_frame = p1.stdout.read(width*height*3)

将读取的字节转换为NumPy数组，然后将其调整为视频帧尺寸:

Convert the bytes read into a NumPy array, and reshape it to video frame dimensions:

 frame = np.fromstring(raw_frame, np.uint8)
 frame = frame.reshape((height, width, 3))

现在，您可以显示调用 cv2.imshow('image'，frame)的框架.

Now you can show the frame calling cv2.imshow('image', frame).

该解决方案假定您事先知道了视频帧的大小( width 和 height ).

The solution assumes, you know the video frame size (width and height) from advance.

下面的代码示例包括使用 cv2.VideoCapture 读取 width 和 height 的部分，但是我不确定是否要这样做可以正常工作(由于'-rtsp_flags'，'listen'.(如果确实有效，则可以尝试使用OpenCV而不是FFmpeg进行捕获).

The code sample below, includes a part that reads width and height using cv2.VideoCapture, but I am not sure if it's going to work in your case (due to '-rtsp_flags', 'listen'. (If it does work, you can try capturing using OpenCV instead of FFmpeg).

以下代码是完整的工作示例"，使用公共RTSP Stream进行测试的

The following code is a complete "working sample" that uses public RTSP Stream for testing:

import cv2
import numpy as np
import subprocess

# Use public RTSP Stream for testing
in_stream = 'rtsp://wowzaec2demo.streamlock.net/vod/mp4:BigBuckBunny_115k.mov'

if False:
    # Read video width, height and framerate using OpenCV (use it if you don't know the size of the video frames).

    # Use public RTSP Streaming for testing:
    cap = cv2.VideoCapture(in_stream)

    framerate = cap.get(5) #frame rate

    # Get resolution of input video
    width  = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
    height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))

    # Release VideoCapture - it was used just for getting video resolution
    cap.release()
else:
    # Set the size here, if video frame size is known
    width = 240
    height = 160


command = ['C:/ffmpeg/bin/ffmpeg.exe',
           #'-rtsp_flags', 'listen',  # The "listening" feature is not working (probably because the stream is from the web)
           '-rtsp_transport', 'tcp',  # Force TCP (for testing)
           '-max_delay', '30000000',  # 30 seconds (sometimes needed because the stream is from the web).
           '-i', in_stream,
           '-f', 'image2pipe',
           '-pix_fmt', 'bgr24',
           '-vcodec', 'rawvideo', '-an', '-']

# Open sub-process that gets in_stream as input and uses stdout as an output PIPE.
p1 = subprocess.Popen(command, stdout=subprocess.PIPE)

while True:
    # read width*height*3 bytes from stdout (1 frame)
    raw_frame = p1.stdout.read(width*height*3)

    if len(raw_frame) != (width*height*3):
        print('Error reading frame!!!')  # Break the loop in case of an error (too few bytes were read).
        break

    # Convert the bytes read into a NumPy array, and reshape it to video frame dimensions
    frame = np.fromstring(raw_frame, np.uint8)
    frame = frame.reshape((height, width, 3))

    # Show video frame
    cv2.imshow('image', frame)
    cv2.waitKey(1)
  
# Wait one more second and terminate the sub-process
try:
    p1.wait(1)
except (sp.TimeoutExpired):
    p1.terminate()

cv2.destroyAllWindows()

样本框(只是为了好玩):

Sample frame (just for fun):

使用 FFprobe 读取宽度和高度:

如果您事先不了解视频分辨率，则可以使用FFprobe来获取信息.

When you don't know the video resolution from advance, you may use FFprobe for getting the information.

这是一个使用FFprobe读取 width 和 height 的代码示例:

Here is a code sample for reading width and height using FFprobe:

import subprocess
import json

# Use public RTSP Stream for testing
in_stream = 'rtsp://wowzaec2demo.streamlock.net/vod/mp4:BigBuckBunny_115k.mov'

probe_command = ['C:/ffmpeg/bin/ffprobe.exe',
                 '-loglevel', 'error',
                 '-rtsp_transport', 'tcp',  # Force TCP (for testing)]
                 '-select_streams', 'v:0',  # Select only video stream 0.
                 '-show_entries', 'stream=width,height', # Select only width and height entries
                 '-of', 'json', # Get output in JSON format
                 in_stream]

# Read video width, height using FFprobe:
p0 = subprocess.Popen(probe_command, stdout=subprocess.PIPE)
probe_str = p0.communicate()[0] # Reading content of p0.stdout (output of FFprobe) as string
p0.wait()
probe_dct = json.loads(probe_str) # Convert string from JSON format to dictonary.

# Get width and height from the dictonary
width = probe_dct['streams'][0]['width']
height = probe_dct['streams'][0]['height']

这篇关于使用FFmpeg，Python和OpenCV显示流的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！