旋转视频而不旋转AVCaptureConnection并在AVAssetWriter会话中间 [英] Rotating video without rotating AVCaptureConnection and in the middle of AVAssetWriter session

问题描述

我正在使用 PBJVision 来实现点击录制视频功能。图书馆不支持方向，所以我正在尝试设计它。从我看到的，旋转视频有三种方法 - 我需要帮助决定最佳前进方法以及如何实现它。请注意，在点击到记录段之间可能会发生轮换。因此，在录制会话中，方向被锁定为用户点击按钮时的方向。下次用户点击按钮进行录制时，应该将方向重新设置为设备的方向（因此生成的视频显示为正面朝上）。

I'm using PBJVision to implement tap-to-record video functionality. The library doesn't support orientation yet so I'm in the process of trying to engineer it in. From what I see, there are three ways to rotate the video - I need help on deciding the best way forward and how to implement it. Note that rotation can happen between tap-to-record segments. So in a recording session, the orientation is locked to what it was when the user tapped the button. The next time the user taps the button to record, it should re-set the orientation to whatever the device's orientation is (so the resulting video shows right-side-up).

这些方法在GitHub上的问题页面中进行了概述

The approaches are outlined in the issue page on GitHub as well

方法1
使用 setVideoOrientation旋转 AVCaptureConnection ： - 这会导致视频预览在每次切换时都会闪烁，因为这样可以切换实际的硬件。不酷，不可接受。

Method 1 Rotate the AVCaptureConnection using setVideoOrientation: - this causes the video preview to flicker every time it's switched, since this switches the actual hardware it seems. Not cool, not acceptable.

方法2
设置转换用于编写视频的 AVAssetWriterInput 对象的属性。问题是，一旦资产作者开始写作，转换属性就无法更改，因此这仅适用于视频的第一段。

Method 2 Set the transform property on the AVAssetWriterInput object used to write the video. The problem is, once the asset writer starts writing, the transform property can't be changed, so this only works for the first segment of the video.

方法3
使用以下内容旋转附加的图像缓冲区：如何直接在IOS 4中旋转CVImageBuffer图像而不转换为UIImage？但它一直在崩溃而且我甚至不确定我是不是正在吠叫正确的树。有一个异常被抛出，我无法真正追溯到比我正在使用 vImageRotate90_ARGB8888 函数错误的事实。

Method 3 Rotate the image buffer being appended using something like this: How to directly rotate CVImageBuffer image in IOS 4 without converting to UIImage? but it keeps crashing and I'm not even sure if I'm barking up the right tree. There's an exception that is thrown and I can't really trace it back to much more than the fact that I'm using the vImageRotate90_ARGB8888 function incorrectly.

我在上面链接的GitHub问题页面上的解释稍微详细一些。任何建议都会受到欢迎 - 说实话，我在AVFoundation上并不是很有经验，所以我希望有一些神奇的方法可以做到这一点，我甚至不知道！

The explanation is a bit more detailed on the GitHub issue page I linked to above. Any suggestions would be welcome - to be honest, I'm not hugely experienced at AVFoundation and so I'm hoping that there's some miraculous way to do this that I don't even know about!

推荐答案

根据 Apple的文档（物理旋转缓冲区确实带有性能成本，因此只有在必要时才请求轮换）。方法2适合我，但如果我在不支持转换元数据的应用上播放我的视频，则视频无法正常旋转。方法3就是我做的。

Method 1 isn't the preferred method according to Apple's documentation ("Physically rotating buffers does come with a performance cost, so only request rotation if it's necessary"). Method 2 worked for me but if I played my video on an app that doesn't support the transformation "metadata", the video isn't rotated properly. Method 3 is what I did.

我认为在您尝试直接从 vImageRotate传递图像数据之前它会崩溃... 到 AVAssetWriterInputPixelBufferAdaptor 。您必须首先创建 CVPixelBufferRef 。这是我的代码：

I think it's crashing for you before you're trying to pass the image data directly from vImageRotate... to the AVAssetWriterInputPixelBufferAdaptor. You have to create a CVPixelBufferRef first. Here's my code:

captureOutput：didOutputSampleBuffer：fromConnection：我在将帧写入适配器之前旋转帧：

Inside of captureOutput:didOutputSampleBuffer:fromConnection: I rotate the frame before writing it into the adaptor:

if ([self.videoWriterInput isReadyForMoreMediaData])
{
    // Rotate buffer first and then write to adaptor
    CMTime sampleTime = CMSampleBufferGetPresentationTimeStamp(sampleBuffer);
    CVPixelBufferRef rotatedBuffer = [self correctBufferOrientation:sampleBuffer];
    [self.videoWriterInputAdaptor appendPixelBuffer:rotatedBuffer withPresentationTime:sampleTime];
    CVBufferRelease(rotatedBuffer);
}

执行vImage旋转的引用函数是：

The referenced function that performs the vImage rotation is:

/* rotationConstant:
 *  0 -- rotate 0 degrees (simply copy the data from src to dest)
 *  1 -- rotate 90 degrees counterclockwise
 *  2 -- rotate 180 degress
 *  3 -- rotate 270 degrees counterclockwise
 */

- (CVPixelBufferRef)rotateBuffer:(CMSampleBufferRef)sampleBuffer withConstant:(uint8_t)rotationConstant
{
    CVImageBufferRef imageBuffer        = CMSampleBufferGetImageBuffer(sampleBuffer);
    CVPixelBufferLockBaseAddress(imageBuffer, 0);        

    OSType pixelFormatType              = CVPixelBufferGetPixelFormatType(imageBuffer);
    NSAssert(pixelFormatType == kCVPixelFormatType_32ARGB, @"Code works only with 32ARGB format. Test/adapt for other formats!");

    const size_t kAlignment_32ARGB      = 32;
    const size_t kBytesPerPixel_32ARGB  = 4;

    size_t bytesPerRow                  = CVPixelBufferGetBytesPerRow(imageBuffer);
    size_t width                        = CVPixelBufferGetWidth(imageBuffer);
    size_t height                       = CVPixelBufferGetHeight(imageBuffer);

    BOOL rotatePerpendicular            = (rotateDirection == 1) || (rotateDirection == 3); // Use enumeration values here
    const size_t outWidth               = rotatePerpendicular ? height : width;
    const size_t outHeight              = rotatePerpendicular ? width  : height;

    size_t bytesPerRowOut               = kBytesPerPixel_32ARGB * ceil(outWidth * 1.0 / kAlignment_32ARGB) * kAlignment_32ARGB;

    const size_t dstSize                = bytesPerRowOut * outHeight * sizeof(unsigned char);

    void *srcBuff                       = CVPixelBufferGetBaseAddress(imageBuffer);

    unsigned char *dstBuff              = (unsigned char *)malloc(dstSize);

    vImage_Buffer inbuff                = {srcBuff, height, width, bytesPerRow};
    vImage_Buffer outbuff               = {dstBuff, outHeight, outWidth, bytesPerRowOut};

    uint8_t bgColor[4]                  = {0, 0, 0, 0};

    vImage_Error err                    = vImageRotate90_ARGB8888(&inbuff, &outbuff, rotationConstant, bgColor, 0);
    if (err != kvImageNoError) 
    {
        NSLog(@"%ld", err);
    }

    CVPixelBufferUnlockBaseAddress(imageBuffer, 0);

    CVPixelBufferRef rotatedBuffer      = NULL;
    CVPixelBufferCreateWithBytes(NULL,
                                 outWidth,
                                 outHeight,
                                 pixelFormatType,
                                 outbuff.data,
                                 bytesPerRowOut,
                                 freePixelBufferDataAfterRelease,
                                 NULL,
                                 NULL,
                                 &rotatedBuffer);

    return rotatedBuffer;
}

void freePixelBufferDataAfterRelease(void *releaseRefCon, const void *baseAddress)
{
    // Free the memory we malloced for the vImage rotation
    free((void *)baseAddress);
}

注意：您可能希望对 rotationConstant使用枚举。类似的东西（不要用 MOVRotateDirectionUnknown 调用此函数）：

Note: You may like to use enumeration for rotationConstant. Something like that (don't call this function with MOVRotateDirectionUnknown):

typedef NS_ENUM(uint8_t, MOVRotateDirection)
{
    MOVRotateDirectionNone = 0,
    MOVRotateDirectionCounterclockwise90,
    MOVRotateDirectionCounterclockwise180,
    MOVRotateDirectionCounterclockwise270,
    MOVRotateDirectionUnknown
};

注意：如果您需要 IOSurface 支持，应该使用 CVPixelBufferCreate 不是 CVPixelBufferCreateWithBytes 的和字节的数据传递到它直接：

Note: If you need IOSurface support, you should use CVPixelBufferCreate instead of CVPixelBufferCreateWithBytes and pass bytes data into it directly:

NSDictionary *pixelBufferAttributes = @{ (NSString *)kCVPixelBufferIOSurfacePropertiesKey : @{} };
CVPixelBufferCreate(kCFAllocatorDefault,
                    outWidth,
                    outHeight,
                    pixelFormatType,
                    (__bridge CFDictionaryRef)(pixelBufferAttributes),
                    &rotatedBuffer);

CVPixelBufferLockBaseAddress(rotatedBuffer, 0);
uint8_t *dest = CVPixelBufferGetBaseAddress(rotatedBuffer);
memcpy(dest, outbuff.data, bytesPerRowOut * outHeight);

CVPixelBufferUnlockBaseAddress(rotatedBuffer, 0);

这篇关于旋转视频而不旋转AVCaptureConnection并在AVAssetWriter会话中间的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！