在OpenGl中显示YUV [英] Display YUV in OpenGl
问题描述
我无法显示NV12格式的原始YUV文件.
I am having trouble displaying a raw YUV file that it is in NV12 format.
我可以显示一个选定的帧，但是它仍然主要是黑色和白色，并带有一些粉红色和绿色的阴影.
I can display a selected frame, however, it is still mainly in black and white with certain shades of pink and green.
这是我的输出的样子
Here is how my output looks like
无论如何，这是我的程序的工作方式. (这是在cocoa/objective-c中完成的，但是我需要您在程序算法方面而非语法方面的专家建议.)
Anyways, here is how my program works. (This is done in cocoa/objective-c, but I need your expert advice on program algorithm, not on syntax.)
在程序执行之前，YUV文件存储在名为"test.yuv"的二进制文件中.该文件为NV12格式，这意味着将首先存储Y计划，然后对UV计划进行隔行扫描.我的文件提取没有问题，因为我对其进行了大量测试.
Prior to program execution, the YUV file is stored in a binary file named "test.yuv". The file is in NV12 format, meaning the Y plan is stored first, then the UV plan is interlaced. My file extraction has no problem because I did a lot testings on it.
在启动过程中，我创建了一个查找表，该表将二进制8位/字节/字符转换为受尊重的Y，U，V浮点值
During initiation, I create a lookup table that will convert binary/8 bites/a byte/chars into respected Y, U, V float values
对于Y平面，这是我的代码
For the Y plane this is my code
-(void)createLookupTableY //creates a lookup table for converting a single byte into a float between 0 and 1
{
NSLog(@"YUVFrame: createLookupTableY");
lookupTableY = new float [256];
for (int i = 0; i < 256; i++)
{
lookupTableY[i] = (float) i /255;
//NSLog(@"lookupTableY[%d]: %f",i,lookupTableY[i]);//prints out the value of each float
}
}
U平面查找表
-(void)createLookupTableU //creates a lookup table for converting a single byte into a float between 0 and 1
{
NSLog(@"YUVFrame: createLookupTableU");
lookupTableU = new float [256];
for (int i = 0; i < 256; i++)
{
lookupTableU[i] = -0.436 + (float) i / 255* (0.436*2);
NSLog(@"lookupTableU[%d]: %f",i,lookupTableU[i]);//prints out the value of each float
}
}
还有V查询表
-(void)createLookupTableV //creates a lookup table for converting a single byte into a float between 0 and 1
{
NSLog(@"YUVFrame: createLookupTableV");
lookupTableV = new float [256];
for (int i = 0; i < 256; i++)
{
lookupTableV[i] = -0.615 + (float) i /255 * (0.615*2);
NSLog(@"lookupTableV[%d]: %f",i,lookupTableV[i]);//prints out the value of each float
}
}
此后，我提取Y& UV计划并将它们存储在两个缓冲区yBuffer&中. uvBuffer
after this point, I extract the Y & UV plan and store them into two buffers, yBuffer & uvBuffer
这时，我尝试转换YUV数据并将其存储到名为"frameImage"的RGB缓冲区数组中.
at this point, I attempt to convert the YUV data and stored it into a RGB buffer array called "frameImage"
-(void)sortAndConvert//sort the extracted frame data into an array of float
{
NSLog(@"YUVFrame: sortAndConvert");
int frameImageCounter = 0;
int pixelCounter = 0;
for (int y = 0; y < YUV_HEIGHT; y++)//traverse through the frame's height
{
for ( int x = 0; x < YUV_WIDTH; x++)//traverse through the frame's width
{
float Y = lookupTableY [yBuffer [y*YUV_WIDTH + x] ];
float U = lookupTableU [uvBuffer [ ((y / 2) * (YUV_WIDTH / 2) + (x/2)) * 2 ] ];
float V = lookupTableV [uvBuffer [ ((y / 2) * (YUV_WIDTH / 2) + (x/2)) * 2 + 1] ];
float RFormula = Y + 1.13983f * V;
float GFormula = Y - 0.39465f * U - 0.58060f * V;
float BFormula = Y + 2.03211f * U;
frameImage [frameImageCounter++] = [self clampValue:RFormula];
frameImage [frameImageCounter++] = [self clampValue:GFormula];
frameImage [frameImageCounter++] = [self clampValue:BFormula];
}
}
}
然后我尝试在OpenGl中绘制图像
then I tried to draw the Image in OpenGl
-(void)drawFrame:(int )x
{
GLuint texture;
glGenTextures(1, &texture);
glBindTexture(GL_TEXTURE_2D, texture);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, YUV_WIDTH, YUV_HEIGHT, 0, GL_RGB, GL_FLOAT, frameImage);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glEnable(GL_TEXTURE_2D);
glBindTexture(GL_TEXTURE_2D, texture);
glRotatef( 180.0f, 1.0f, 0.0f, 0.0f );
glBegin( GL_QUADS );
glTexCoord2d(0.0,0.0); glVertex2d(-1.0,-1.0);
glTexCoord2d(1.0,0.0); glVertex2d(+1.0,-1.0);
glTexCoord2d(1.0,1.0); glVertex2d(+1.0,+1.0);
glTexCoord2d(0.0,1.0); glVertex2d(-1.0,+1.0);
glEnd();
glFlush();
}
所以基本上这是我的程序，位于坚果壳中.本质上，我读取二进制YUV文件，将所有数据存储在char数组缓冲区中.然后，我将这些值转换为它们受尊重的YUV浮点值.
so basically this my program in a nut shell. essentially i read the binary YUV files, stores all the data in a char array buffer. i then translate these values into their respected YUV float values.
我认为这可能是错误所在:在Y查找表中，我将Y平面标准化为[0,1]，在U平面中，标准化了[-0.435,0.436]之间的值，以及在V中飞机我将其标准化为[-0.615,0.615].我这样做是因为根据维基百科，这些是YUV值范围.
This is where I think the error might be: in the Y lookup table I standardize the Y plane to [0,1], in the U plane I standardized the values between [-0.435,0.436], and in the V plane I standardized it bewteen [-0.615,0.615]. I did this because those are the YUV value ranges according to wikipedia.
YUV到RGB的公式与Wikipedia中的公式相同.我还尝试了其他各种公式，这是唯一给出框架粗略外观的公式.任何人都可能敢于猜测为什么我的程序无法正确显示YUV帧数据.我认为这与我的标准化技术有关，但对我来说似乎还可以.
And the YUV to RGB formula is the same formula from Wikipedia. I have also tried various other formulas, and this is the only formula that gives the rough outlook of the frame. Anyone might venture to guess to why my program is not correctly displaying the YUV frame data. I think it is something to do with my standardization technique, but it seems alright to me.
我已经做了很多测试，并且100％确信该错误是由查找表引起的.我认为我的设置公式不正确.
给所有正在阅读的人的便条 这.最长的时间，我的框架 显示不正确，因为我 无法提取帧数据 正确.
A note to everyone who is reading this. For the longest time, my frame was not displaying properly because I was not able to extract the frame data correctly.
当我第一次开始编程时， 在一个剪辑的印象中 假设30帧，所有30个Y平面数据 首先写入数据文件 然后是UV平面数据.
When I first started to program, I was under the impression that in a clip of say 30 frames, all 30 Y planar datas are first written in the data file, followed then by UV plane datas.
我通过审判发现的 错误是对于YUV数据文件， 特别是NV12，数据文件是 以以下方式存储
What I found out through trial and error was that for a YUV data file, specifically NV12, the data file is stored in the following fashion
Y(1)UV(1)Y(2)UV(2)... ...
Y(1) UV(1) Y(2) UV(2) ... ...
@nschmidt
@nschmidt
我将代码更改为您建议的
I changed my code to what you suggested
float U = lookupTableU [uvBuffer [ (y * (YUV_WIDTH / 4) + (x/4)) * 2 ] ]
float V = lookupTableU [uvBuffer [ (y * (YUV_WIDTH / 4) + (x/4)) * 2 + 1] ]
这就是我得到的结果
and this is the result that i get
这是控制台上的打印行.我打印出Y，U，V和 正在转换并存储在frameImage数组中的RGB值
here is the print line from the console. i am print out the values for Y, U, V and RGB value that are being translated and stored on in the frameImage array
YUV:[0.658824，-0.022227，-0.045824] RGB最终版本:[0.606593,0.694201,0.613655]
YUV:[0.658824,-0.022227,-0.045824] RGBFinal:[0.606593,0.694201,0.613655]
YUV:[0.643137，-0.022227，-0.045824] RGBFinal:[0.590906,0.678514,0.597969]
YUV:[0.643137,-0.022227,-0.045824] RGBFinal:[0.590906,0.678514,0.597969]
YUV:[0.607843，-0.022227，-0.045824] RGBFinal:[0.555612,0.643220,0.562675]
YUV:[0.607843,-0.022227,-0.045824] RGBFinal:[0.555612,0.643220,0.562675]
YUV:[0.592157，-0.022227，-0.045824] RGB最终版本:[0.539926,0.627534,0.546988]
YUV:[0.592157,-0.022227,-0.045824] RGBFinal:[0.539926,0.627534,0.546988]
YUV:[0.643137,0.025647,0.151941] RGBFinal:[0.816324,0.544799,0.695255]
YUV:[0.643137,0.025647,0.151941] RGBFinal:[0.816324,0.544799,0.695255]
YUV:[0.662745,0.025647,0.151941] RGBFinal:[0.835932,0.564406,0.714863]
YUV:[0.662745,0.025647,0.151941] RGBFinal:[0.835932,0.564406,0.714863]
YUV:[0.690196,0.025647,0.151941] RGBFinal:[0.863383,0.591857,0.742314]
YUV:[0.690196,0.025647,0.151941] RGBFinal:[0.863383,0.591857,0.742314]
2009年7月13日更新
由于 nschmidt 的建议，问题终于得以解决.事实证明，我的YUV文件实际上是YUV 4:1:1格式.最初，我被告知它是YUV NV12格式.无论如何，我想与您分享我的结果.
The problem was finally solved thanks to the recommendation from nschmidt . It turns out that my YUV file was actually in YUV 4:1:1 format. I was originally told that it was in YUV NV12 format. Anyways, I would like to share my results with you.
此处输出
我的解码代码如下
float Y = (float) yBuffer [y*YUV_WIDTH + x] ;
float U = (float) uvBuffer [ ((y / 2) * (YUV_WIDTH / 2) + (x/2)) ] ;
float V = (float) uvBuffer [ ((y / 2) * (YUV_WIDTH / 2) + (x/2)) + UOffset] ;
float RFormula = (1.164*(Y-16) + (1.596* (V - 128) ));
float GFormula = ((1.164 * (Y - 16)) - (0.813 * ((V) - 128)) - (0.391 * ((U) - 128)));
float BFormula = ((1.164 * (Y - 16)) + (2.018 * ((U) - 128)));
frameImage [frameImageCounter] = (unsigned char)( (int)[self clampValue:RFormula]);
frameImageCounter ++;
frameImage [frameImageCounter] = (unsigned char)((int)[self clampValue:GFormula]);
frameImageCounter++;
frameImage [frameImageCounter] = (unsigned char)((int) [self clampValue:BFormula]);
frameImageCounter++;
GLuint texture;
glGenTextures(1, &texture);
glEnable(GL_TEXTURE_2D);
glBindTexture(GL_TEXTURE_2D, texture);
glTexParameteri(GL_TEXTURE_2D, GL_GENERATE_MIPMAP_SGIS, GL_TRUE);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, YUV_WIDTH, YUV_HEIGHT, 0, GL_RGB, GL_UNSIGNED_BYTE, frameImage);
//glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_WRAP_S,GL_CLAMP_TO_EDGE_SGIS);
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_WRAP_T,GL_CLAMP_TO_EDGE_SGIS);
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MAG_FILTER,GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MIN_FILTER,GL_LINEAR_MIPMAP_LINEAR);
glRotatef( 180.0f, 1.0f, 0.0f, 0.0f );
glBegin( GL_QUADS );
glTexCoord2d(0.0,0.0); glVertex2d(-1.0,-1.0);
glTexCoord2d(1.0,0.0); glVertex2d(+1.0,-1.0);
glTexCoord2d(1.0,1.0); glVertex2d(+1.0,+1.0);
glTexCoord2d(0.0,1.0); glVertex2d(-1.0,+1.0);
glEnd();
NSLog(@"YUVFrameView: drawRect complete");
glFlush();
基本上，我使用NSData提取原始文件.存储在char数组缓冲区中.对于YUV到RGB的转换，我使用了上面的公式，然后，将值固定为[0:255].然后我在OpenGL中使用了2DTexture进行显示.
essentially, I used NSData for the raw file extraction. stored in a char array buffer. For YUV to RGB conversion, I used the above formula, afterwards, I clamped the values to [0:255]. then I used a 2DTexture in OpenGL for display.
因此，如果将来要将YUV转换为RGB，请使用上面的公式.如果使用先前文章中的YUV到RGB的转换公式，则需要将RGB的值固定在[0:1]
So if you are converting YUV to RGB in the future, use the formula from above. If are using the YUV to RGB conversion formula from the earlier post, then you need to display the texture in GL_Float from the values for RGB are clamped between [0:1]
推荐答案
下一步尝试:) 我认为您的uv缓冲区不是交错的.看起来U值先出现，然后是V值数组.将行更改为
Next try :) I think you're uv buffer is not interleaved. It looks like the U values come first and then the array of V values. Changing the lines to
unsigned int voffset = YUV_HEIGHT * YUV_WIDTH / 2;
float U = lookupTableU [uvBuffer [ y * (YUV_WIDTH / 2) + x/2] ];
float V = lookupTableV [uvBuffer [ voffset + y * (YUV_WIDTH / 2) + x/2] ];
可能指示是否确实如此.
might indicate if this is really the case.
这篇关于在OpenGl中显示YUV的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
