使用基于状态的标签注释视频帧 [英] Annotating video frames with a label based on state

问题描述

我有一堆显示来自 Microsoft Kinect 的人体姿势的视频和深度图.

I have a bunch of videos and depthmaps showing human poses from the Microsoft Kinect.

我可以在视频中获得人类的骨架，但我想要做的是从这个骨架数据中识别出某个姿势.

I can get a skeleton of the human in the video but what I want to do is recognize a certain pose from this skeleton data.

为此，我需要用 0 或 1 注释视频中的每一帧，对应于坏姿势"和好姿势"，即该帧有一个二元状态变量.

To do that I need to annotate each frame in the videos with a 0 or 1, corresponding to "bad pose" and "good pose", i.e. the frame has a binary state variable.

我希望能够在 matlab 中播放 avi 文件，然后按空格键在这两种状态之间切换，同时将状态变量添加到一个数组中，给出视频中每一帧的状态.

I want to be able to playback the avi file in matlab and then press space to switch between these two states and simultaneously add the state variable to an array giving the state for each frame in the video.

matlab 中有没有可以做到这一点的工具?否则 matlab 不是限制，python、C++ 或任何其他语言都可以.

Is there a tool in matlab that can do this? Otherwise matlab is not a restriction, python, C++ or any other language is fine.

我一直在谷歌搜索，我发现的大部分内容是用多边形注释单个帧.我想以视频常规帧速率的一半来执行此操作.

I have been googling around, and most of the stuff I have found is to annotate individual frames with a polygon. I want to do this at maybe half the regular framerate of the video.

我使用了 miindlek 提供的解决方案，如果有人遇到这个问题，我决定分享一些东西.我需要在视频中查看我为每一帧分配的注释，因此我在显示视频时在视频的左上角做了一个小圆圈.希望这对以后的其他人有用.我还捕获了用 waitKey 按下的键，然后根据输出做一些事情.这允许在注释期间按下多个键.

I used the solution provided by miindlek and decided to share a few things if someone runs into this. I needed to see in the video what annotation I was assigning to each frame, so I made a small circle in the upper left corner of the video as I displayed it. Hopefully this will be useful for someone else later. I also capture the key pressed with waitKey and then do something based on the output. This allows for multiple keys to be pressed during the annotations.

import numpy as np
import cv2
import os
os.chdir('PathToVideo')

# Blue cicle means that the annotation haven't started
# Green circle is a good pose
# Red is a bad pose
# White circle means we are done, press d for that

# Instructions on how to use!
# Press space to swap between states, you have to press space when the person
# starts doing poses. 
# Press d when the person finishes.
# press q to quit early, then the annotations are not saved, you should only 
# use this if you made a mistake and need to start over.

cap = cv2.VideoCapture('Video.avi')

# You can INCREASE the value of speed to make the video SLOWER
speed = 33

# Start with the beginning state as 10 to indicate that the procedure has not started
current_state = 10
saveAnnotations = True
annotation_list = []
# We can check wether the video capture has been opened
cap.isOpened()
colCirc = (255,0,0)
# Iterate while the capture is open, i.e. while we still get new frames.
while(cap.isOpened()):
    # Read one frame.
    ret, frame = cap.read()
    # Break the loop if we don't get a new frame.
    if not ret:
        break
    # Add the colored circle on the image to know the state
    cv2.circle(frame,(50,50), 50, colCirc, -1)
    # Show one frame.
    cv2.imshow('frame', frame)
    # Wait for a keypress and act on it
    k = cv2.waitKey(speed)
    if k == ord(' '):
        if current_state==0:
            current_state = 1
            colCirc = (0,0,255)
        else:
            current_state = 0
            colCirc = (0,255,0)
        if current_state == 10:
            current_state = 0
            colCirc = (0,255,0)
    if k == ord('d'):
        current_state = 11
        colCirc = (255,255,255)

    # Press q to quit
    if k == ord('q'):
        print "You quit! Restart the annotations by running this script again!"
        saveAnnotations = False
        break

    annotation_list.append(current_state)

# Release the capture and close window
cap.release()
cv2.destroyAllWindows()

# Only save if you did not quit
if saveAnnotations:
    f = open('poseAnnot.txt', 'w')
    for item in annotation_list:
        print>>f, item
    f.close()

推荐答案

解决您的任务的一种方法是将 opencv 库与 python 结合使用，如本 教程.

One way to solve your task is using the opencv library with python, as described in this tutorial.

import numpy as np
import cv2

cap = cv2.VideoCapture('video.avi')

current_state = False
annotation_list = []

while(True):
    # Read one frame.
    ret, frame = cap.read()
    if not ret:
        break

    # Show one frame.
    cv2.imshow('frame', frame)

    # Check, if the space bar is pressed to switch the mode.
    if cv2.waitKey(1) & 0xFF == ord(' '):
        current_state = not current_state

    annotation_list.append(current_state)

# Convert the list of boolean values to a list of int values.    
annotation_list = map(int, annotation_list)
print annotation_list

cap.release()
cv2.destroyAllWindows()

变量 annotation_list 包含每个帧的所有注释.要在两种模式之间切换，您必须按空格键.

The variable annotation_list contains all annotations for each frame. To switch between the two modes, you have to press the space bar.

这篇关于使用基于状态的标签注释视频帧的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！