在OpenGLES2.0中将视频作为纹理 [英] Video as texture in OpenGLES2.0
问题描述
我想将视频作为纹理放置到OpenGLES 2.0 iOS中的对象上.
我用AVPlayerItemVideoOutput创建AVPlayer，设置
I want to place video as texture to object in OpenGLES 2.0 iOS.
I create AVPlayer with AVPlayerItemVideoOutput, setting
NSDictionary *videoOutputOptions = [NSDictionary dictionaryWithObjectsAndKeys:
[NSNumber numberWithInt:kCVPixelFormatType_32BGRA], kCVPixelBufferPixelFormatTypeKey,
[NSDictionary dictionary], kCVPixelBufferIOSurfacePropertiesKey,
nil];
self.videoOutput = [[AVPlayerItemVideoOutput alloc] initWithPixelBufferAttributes:videoOutputOptions];
比我每时每刻得到CVPixelBufferRef
CMTime currentTime = [self.videoOutput itemTimeForHostTime:CACurrentMediaTime()];
CVPixelBufferRef buffer = [self.videoOutput copyPixelBufferForItemTime:currentTime itemTimeForDisplay:NULL];
然后我使用此方法将其转换为UIImage
Then i convert it to UIImage with this method
+ (UIImage *)imageWithCVPixelBufferUsingUIGraphicsContext:(CVPixelBufferRef)pixelBuffer
{
CVPixelBufferLockBaseAddress(pixelBuffer, 0);
int w = CVPixelBufferGetWidth(pixelBuffer);
int h = CVPixelBufferGetHeight(pixelBuffer);
int r = CVPixelBufferGetBytesPerRow(pixelBuffer);
int bytesPerPixel = r/w;
unsigned char *bufferU = CVPixelBufferGetBaseAddress(pixelBuffer);
UIGraphicsBeginImageContext(CGSizeMake(w, h));
CGContextRef c = UIGraphicsGetCurrentContext();
unsigned char* data = CGBitmapContextGetData(c);
if (data) {
int maxY = h;
for(int y = 0; y < maxY; y++) {
for(int x = 0; x < w; x++) {
int offset = bytesPerPixel*((w*y)+x);
data[offset] = bufferU[offset]; // R
data[offset+1] = bufferU[offset+1]; // G
data[offset+2] = bufferU[offset+2]; // B
data[offset+3] = bufferU[offset+3]; // A
}
}
}
UIImage *image = UIGraphicsGetImageFromCurrentImageContext();
UIGraphicsEndImageContext();
CVPixelBufferUnlockBaseAddress(pixelBuffer, 0);
CFRelease(pixelBuffer);
return image;
}
结果是我从视频中获得了所需的帧:
As result i got required frame from video:
毕竟我尝试用
- (void)setupTextureWithImage:(UIImage *)image
{
if (_texture.name) {
GLuint textureName = _texture.name;
glDeleteTextures(1, &textureName);
}
NSError *error;
_texture = [GLKTextureLoader textureWithCGImage:image.CGImage options:nil error:&error];
if (error) {
NSLog(@"Error during loading texture: %@", error);
}
}
我在GLKView的update方法中调用了此方法，但结果黑屏，只有音频可用.
I call this method in GLKView's update method, but as result got black screen, only audio available.
谁能解释做错了什么?看起来我在纹理上做错了...
Can anyone explain whats done wrong? Looks like i'm doing something wrong with textures...
推荐答案
该问题最有可能是您发布的代码之外的其他地方.要检查纹理本身，请创建快照(Xcode中的功能)，然后查看是否可以在其中看到正确的纹理.在显示纹理对象时，也许坐标不正确或缺少一些参数，或者是您忘记启用某些属性或不存在着色器...
The issue is most likely somewhere else then the code you posted. To check the texture itself create a snapshot (a feature in Xcode) and see if you can see the correct texture there. Maybe your coordinates are incorrect or some parameters missing when displaying the textured object, could be you forgot to enable some attributes or the shaders are not present...
既然您已经很了解了，我建议您首先尝试绘制一个彩色正方形，然后尝试对其应用纹理(不是来自视频)，直到获得正确的结果.然后从视频中实现纹理.
Since you got so far I suggest you first try to draw a colored square, then try to apply a texture (not from the video) to it until you get the correct result. Then implement the texture from video.
这只是一个建议，因为您从视频中获取原始像素数据，应该考虑只创建一个纹理，然后使用纹理子图像功能直接用数据更新纹理，而不是对图像进行一些奇怪的迭代和变换. glTexSubImage2D将直接获取您的缓冲区指针并进行更新.
And just a suggestion since you are getting raw pixel data from the video you should consider creating only one texture and then use texture sub image function to update the texture directly with the data instead of doing some strange iterations and transformations to the image. The glTexSubImage2D will take your buffer pointer directly and do the update.
这篇关于在OpenGLES2.0中将视频作为纹理的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
