使用Python组合所有视频的功能 [英] Combining features of all videos using Python

问题描述

假设我有20个视频.它们具有相同的场景，尺寸，并且来自同一台摄像机.假设这20个视频中有一个有一个人在走来走去.其他所有视频基本上都是相同的(除了很小的自然变化，例如刮风的叶子等).

Let's assume I have 20 videos. They are of the same scene, dimensions and from the same camera. Let's assume one of those twenty videos has a person walking across. All the other videos are mostly the same (except for minor natural changes, like wind blowing leaves etc).

我正在寻找一种将所有20个视频合并为1个视频的好方法.合并是指叠加".每个视频的所有帧都相互重叠，但是以差异"表示.我想不出一个好方法. 到目前为止，这是我所拥有的:(简化代码).

I am looking for a good way to merge all the 20 videos into 1 video. By merge I mean "overlay". All frames of each video overlaid on top of each other, but in a way that "differences" show. I can't figure out a good way. Here is what I have so far: (Code simplified).

基本上，我要遍历20个视频，并将每个视频与下一个视频混合，并使用新创建的混合视频与下一个视频，依此类推.

Basically, I am looping through the 20 videos, and blending each video with the next one and using the newly created blended video with the next video and so on.

但是，因为我使用的是cv2.addWeighted，所以与人同行的视频几乎消失了(以20％的覆盖率覆盖50％).如何在保持显着像素差异"的情况下创建覆盖视频?我不知道哪个视频是不同的，这毫无价值，因此我无法创建蒙版.鉴于大多数视频大部分都是相似的，因此应该有一些方法可以使帧的功能保持显着不同.

However, because I am using cv2.addWeighted, the video with the person walking across, almost disappears (after 20 overlays at 50%). How do I create an overlay video where 'significant pixel differences' are maintained? It is worth nothing that I don't know which video is different - so I can't create masks. Given most of the videos are mostly similar, there should be some way to keep the features of frames that are significantly different.

videos = ['1.mp4', '2.mp4' , ...., '20.mp4']

for video in videos:
    fourcc = cv2.VideoWriter_fourcc(*'mp4v')
    out = cv2.VideoWriter('new-blended.mp4', fourcc, orig_fps, (width,height)) 
    vid = cv2.VideoCapture(video) # read current video file
    try:
        blend_vid = cv2.VideoCaptire('blended.mp4')
    except: 
        print ('No worries, this is the first video, we will create a blend now')
        blend_vid = None

    while True: # read all frames, blend
        succ, frame = vid.read()
        succ_b = False
        if blend_vid: succ_b, frame_b = blend_vid.read()
        if not succ_b and not succ: 
            break
        if succ_b:
            merged_frame = cv2.addWeighted(frame, 0.5, frame_b, 0.5, 0)
        else:
            merged_frame = frame
        out.write(merged_frame)
    try:
        os.remove('blended.mp4')
    except:
       pass # will fail the first time
    os.rename ('new-blended.mp4', 'blended.mp4')

添加更多上下文: 在这种特定情况下，背景"是一条车道.前景将是人们走进来的某些框架.我最感兴趣的是记录帧中的差异"并保存它们.为了提供更多背景信息，我们假设有20个视频，每个5分钟.每个视频都具有相同的场景，并在100分钟内顺序记录.我希望创建一个5分钟的视频，将20个视频合并(叠加)在一起，以保留关键差异".目的是帮助人们快速(在5分钟内)查看100分钟的车道视频，以查看是否有任何变化".

Adding more context: In this specific context, 'background' is a driveway. Foreground will be certain frames where people walk in an out. I am mostly interested in recording 'differences' in frames and preserving them. To give some more context, lets assume there are 20 videos, each of 5 minutes. Each video is of the same scene, recorded sequentially over 100 minutes. I'm looking to create a single video, of 5 minutes that combines (overlays) the 20 videos together that preserves 'key differences'. The goal is to help a person review 100 minutes of video of the driveway quickly (in 5 mins) to see if anything "changed".

推荐答案

感谢@Stephen Meschke的提示，我明白了这一点，并且一旦正确正确地实现它就很好了，这并不是一个很好的方法.做我想做的事. 背景"和前景"之间的区别并不是很好.

Thanks to the hint from @Stephen Meschke, I got it working and realized it's pretty good once you do it correctly its not really a good approach to do what I wanted to do. The difference between "background" and "foreground" isn't really good.

无论如何，这是我的代码.如果有人看到改进的方法，请告诉我:

Anyway, this is my code. If anyone sees ways to improve it, please let me know:

框架"是新视频中的框架. "frame_b"是在视频处理的每次迭代中创建的混合视频.

"frame" is the frame from the new video. "frame_b" is the blended video that gets created each iteration of video processing.

kernel_clean = cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(3,3))
kernel_fill = np.ones((20,20),np.uint8)

# get foreground objects from new frame
frame_mask = fgbg.apply(frame)
# clean noise
frame_mask = cv2.morphologyEx(frame_mask, cv2.MORPH_OPEN, kernel_clean)
# fill up foreground mask better
frame_mask = cv2.morphologyEx(frame_mask, cv2.MORPH_CLOSE, kernel_fill)

# remove grey areas, or set detectShadows=False in the extractor, which I learned later. However, removing shadows sometimes causes gaps in the primary foreground object. I found this to produce better results.
indices = frame_mask > 100
frame_mask[indices] = 255
# get only foreground images from the new frame
foreground_a = cv2.bitwise_and(frame,frame, mask=frame_mask)
# clear out parts on blended frames where forground will be added
frame_mask_inv = cv2.bitwise_not(frame_mask)
modified_frame_b = cv2.bitwise_and(frame_b, frame_b, mask=frame_mask_inv)
merged_frame = cv2.add(modified_frame_b, foreground_a)

这篇关于使用Python组合所有视频的功能的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！