表征我的RAW相机输出 [英] Characterizing my RAW camera Output
问题描述
豹纹成像M021相机。该公司并不真正支持Linux，因此相机只会输出原始数据。我需要它在Beagleboard上运行，所以这就是为什么我要努力使其工作。
他们的一个员工告诉我它在<一种href = http://www.fourcc.org/yuv.php#YUYV rel = nofollow noreferrer> YUY2格式（通常为每个像素16位），但是高4位始终为0，而较低的12位包含如下信息：
使用命令：
fswebcam --device / dev / video0-分辨率1280x720 --dumpframe test.raw
我得到的文件长1,843,200字节，即每像素2字节的1280x720图像（每个像素16位）。
但是，我真正能够正确显示图像的唯一方法是使用IrfanView的RAW图像显示并将其设置为每像素12位，未归一化，具有GR的Bayer模式和垂直翻转。我不知道为什么需要垂直翻转，因为其他设置将显示偏斜，怪异的图像，但不会翻转。然后我使用12 BPP并将其翻转。
我想因为高4位始终为0，实际上它是每个像素12位，而不是16位？
我需要知道文件中的字节真正发生了什么，以便自己编写转换算法（除非有人知道与IrfanView相同的开源程序）。
使用Python，我编写了一个非常快速的脚本，只需拉出Y组件并进行查看（期望图像的灰度版本），但是我得到了一个非常失真的版本。我的代码有问题吗？我是否以错误的顺序提取数据？ IrfanView中的未规范化到底是什么意思？为什么要使用RGB才能看到图像，需要GR Bayer模式设置？
withopen（'test.raw'，'r + b'）如f：
Y0 = []
U = []
Y1 = []
V = []
vals = [Y0，U， Y1，V]
val = f.read（1）
pixel = int.from_bytes（val，byteorder ='big'）
i = 0
vals [i] .append （像素）
，而val：
i + = 1
val = f.read（1）
if val！=：
pixel = int.from_bytes（ val，byteorder ='big'）
vals [i％4] .append（pixel）
k = 0
with open（ 1.test， w） as f：i在范围内（720）：
，j在范围内（640）：
f.write（str（Y0 [k]））
f.write （）
f.write（str（Y1 [k]））
f.write（）
k + = 1
f.write（ \n ）
生成的垃圾图片：
我会很感谢任何人对此的帮助或建议。
编辑：
更多可能有用的证据。如果我在matlab中运行此代码，只需将每2个字节视为精确的像素值：
fid = fopen（'test.raw '，'r'）;
[I，count] = fread（fid，[1280，720]，'uint16'）;
imagesc（I’）
colormap（gray）;
我得到此图像：
我仍然缺少颜色信息，因为我只是忽略了它。而且还是有点歪斜。但是看起来更好。如果放大，则图像歪斜的模式是好像素，黑色像素，好像素，黑色像素等。对照相机和颜色有更深入了解的人是否知道这表示什么？
编辑2：
在Mark Ransom的专家帮助下，我编写了一个不错的OpenCV脚本以读取数据，利用CV_BayerGR2RGB转换为RGB，然后查看图像。
#include< vector>
#include< iostream>
#include< stdio.h>
#include< opencv2 / opencv.hpp>
#include< opencv2 / highgui / highgui.hpp>
int main（）{
//每个像素由16位组成，高4位始终等于0
无符号字符字节[2];
//将数据保存在向量
std :: vector< unsigned short int>数据;
//读取相机数据
FILE * fp = fopen（ test.raw， rb）;
while（fread（bytes，2，1，fp）！= 0）{
//数据采用低位字节序，因此将第二个字节右移并连接第一个字节
data.push_back（bytes [0] |（bytes [1]<< 8））;
}
//用16位无符号整数制作矩阵1280x720
cv :: Mat imBayer = cv :: Mat（720，1280，CV_16U）;
//创建一个矩阵来保存RGB数据
cv :: Mat imRGB;
//将向量中的数据复制到一个不错的矩阵
memmove（imBayer.data，data.data（），data.size（）* 2）;
//将GR Bayer模式转换为RGB，并将其放入RGB矩阵！
cv :: cvtColor（imBayer，imRGB，CV_BayerGR2RGB）;
cv :: namedWindow（ Display window，cv :: WINDOW_AUTOSIZE）;
// * 15，因为图像较暗
cv :: imshow（ Display window，15 * imRGB）;
cv :: waitKey（0）;
返回0;
}
您拥有的证据，首先，每个字节的高4位为0，并且您可以从Irfanview中看到正确的彩色图片，这告诉我该员工不正确，并且该格式实际上不是YUY2-它是GRGB。
此代码应拉出绿色像素并将其加倍，以便您可以了解图像的外观。请注意，偶数行和奇数行将在以G和（R或B）开头之间交替。
with open（'test。 raw'，'r + b'）as f：
raw_g = []
对于y在范围内（720）：
对于x在范围内（1280 // 4）：
如果y％2：
r1，r0，g1，g0，b1，b0，g3，g2 = f.read（8）
else：
g1，g0，r1，r0， g3，g2，b1，b0 ＝ f.read（8）
g0 ＝ g0 ＜＜ 8 | g1
g2 = g2< 8 | g3
raw_g.extend（[g0，g0，g2，g2]）
ll可能想右移这些值>> 4 ，然后再将它们写到灰度文件中。
编辑：该代码还有一个错误（额外阅读），我没有意识到这些值是小端而不是大端。这些更改已应用到上面的代码示例中。
由于值似乎有点暗，我还决定应用伽玛校正-代替 n>> 4 我使用了 int（255.999 *（n / 4095.0）**（1 / 2.2））
恐怕de-Bayering超出了我在简短的StackOverflow答案中所能实现的范围。
I have a very curious problem with my Leopard Imaging M021 camera. The company doesn't really support linux, so the camera will only output raw data. I need it to run on a Beagleboard, so that's why I'm putting effort into getting it to work.
An employee of theirs has told me that it is in YUY2 format (usually 16 bits per pixel), but the high 4 bits are always 0, and the lower 12 bits contain information like so:
Using the command:
fswebcam --device /dev/video0 --resolution 1280x720 --dumpframe test.raw
I get a file 1,843,200 bytes long, meaning a 1280x720 image with 2 bytes per pixel (16 bits per pixel).
HOWEVER, the only way I've been able to actually get the image to display correctly is if I use IrfanView's RAW image display and set it to 12 bits per pixel, not normalized, with a bayer pattern of GR, and a vertical flip. I have no idea why the vertical flip is required, because the other settings will display a skewed, weird image but not flipped. Then I use 12 BPP and it is flipped.
I suppose since the high 4 bits are always 0, that causes it to in actuality be 12 bits per pixel rather than 16?
I need to know what is really going on with the bytes in the file in order to write the conversion algorithm myself (unless anyone knows of an open source program that does the same thing as IrfanView).
Using Python, I've made a really quick script to just pull the Y component out and view it (expecting a greyscale version of the image), but I get a very distorted version. Is there something wrong with my code? Am I pulling out the data in the wrong order? What the heck does "not normalized" mean in IrfanView? Why is the GR Bayer pattern setting required to see the image in RGB?
with open('test.raw', 'r+b') as f:
Y0 = []
U = []
Y1 = []
V = []
vals = [Y0, U, Y1, V]
val = f.read(1)
pixel = int.from_bytes(val, byteorder='big')
i = 0
vals[i].append(pixel)
while val:
i += 1
val = f.read(1)
if val != "":
pixel = int.from_bytes(val, byteorder='big')
vals[i % 4].append(pixel)
k = 0
with open("1.test", "w") as f:
for i in range(720):
for j in range(640):
f.write(str(Y0[k]))
f.write(" ")
f.write(str(Y1[k]))
f.write(" ")
k += 1
f.write("\n")
Resulting junk image:
I would appreciate anyone's help or advice on this.
EDIT:
Further possibly helpful evidence. If I run this in matlab, just treating every 2 bytes as the exact pixel value:
fid = fopen('test.raw', 'r');
[I, count] = fread(fid , [1280, 720], 'uint16');
imagesc(I')
colormap(gray);
I get this image:
I'm still missing color information, because I just disregarded it. And it's still skewed a little bit. But it looks better. If you zoom in, the pattern of the image skew is good pixel, black pixel, good pixel, black pixel, etc. Does someone more knowledgeable of cameras and colors know what that is indicative of?
EDIT 2:
With the expert help of Mark Ransom, I wrote up a nice OpenCV script to read in the data, utilize CV_BayerGR2RGB to convert to RGB, and view the image. It works!
#include <vector>
#include <iostream>
#include <stdio.h>
#include <opencv2/opencv.hpp>
#include <opencv2/highgui/highgui.hpp>
int main() {
// Each pixel is made up of 16 bits, with the high 4 bits always equal to 0
unsigned char bytes[2];
// Hold the data in a vector
std::vector<unsigned short int> data;
// Read the camera data
FILE *fp = fopen("test.raw","rb");
while(fread(bytes, 2, 1, fp) != 0) {
// The data comes in little-endian, so shift the second byte right and concatenate the first byte
data.push_back(bytes[0] | (bytes[1] << 8));
}
// Make a matrix 1280x720 with 16 bits of unsigned integers
cv::Mat imBayer = cv::Mat(720, 1280, CV_16U);
// Make a matrix to hold RGB data
cv::Mat imRGB;
// Copy the data in the vector into a nice matrix
memmove(imBayer.data, data.data(), data.size()*2);
// Convert the GR Bayer pattern into RGB, putting it into the RGB matrix!
cv::cvtColor(imBayer, imRGB, CV_BayerGR2RGB);
cv::namedWindow("Display window", cv::WINDOW_AUTOSIZE);
// *15 because the image is dark
cv::imshow("Display window", 15*imRGB);
cv::waitKey(0);
return 0;
}
The evidence that you have, first that each byte has the upper 4 bits as 0 and the fact that you can see a proper color picture from Irfanview, tells me that the employee was incorrect and the format isn't actually YUY2 - it's GRGB.
This code should pull out the green pixels and double them up so you can get an idea of what the image looks like. Note that the even and odd lines will alternate between starting with G and (R or B).
with open('test.raw', 'r+b') as f:
raw_g = []
for y in range(720):
for x in range(1280//4):
if y % 2:
r1, r0, g1, g0, b1, b0, g3, g2 = f.read(8)
else:
g1, g0, r1, r0, g3, g2, b1, b0 = f.read(8)
g0 = g0 << 8 | g1
g2 = g2 << 8 | g3
raw_g.extend([g0, g0, g2, g2])
You'll probably want to right-shift these values >> 4 before you write them out to a gray-scale file.
Edit: The code had one remaining bug (an extra read), and I didn't realize the values were little-endian instead of big-endian. Those changes are already applied to the code sample above.
Since the values seemed a little dark, I also decided to apply a gamma correction - instead of n >> 4 I used int(255.999 * (n / 4095.0)**(1/2.2))
I'm afraid de-Bayering is a little beyond what I can pull off in a short StackOverflow answer.
这篇关于表征我的RAW相机输出的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
