如何在顶点/片段着色器(金属)中使用3x3 2D变换 [英] How to use a 3x3 2D transformation in a vertex/fragment shader (Metal)
问题描述
我的任务很简单，但是显然我仍然不了解投影在着色器中的工作方式.我需要在一个纹理四边形(2个三角形)上进行2D透视变换，但是在视觉上看起来不正确(例如，梯形比CPU版本中的梯形稍微高些或拉伸得更多).
I have a supposedly simple task, but apparently I still don't understand how projections work in shaders. I need to do a 2D perspective transformation on a texture quad (2 triangles), but visually it doesn't look correct (e.g. trapezoid is slightly higher or more stretched than what it is in the CPU version).
我有这个结构:
struct VertexInOut
{
float4 position [[position]];
float3 warp0;
float3 warp1;
float3 warp2;
float3 warp3;
};
在顶点着色器中，我做类似的事情(texCoords是四角的像素坐标，而单应性以像素坐标计算):
And in the vertex shader I do something like (texCoords are pixel coords of the quad corners and homography is calculated in pixel coords):
v.warp0 = texCoords[vid] * homographies[0];
然后在片段着色器中像这样:
Then in the fragment shader like this:
return intensity.sample(s, inFrag.warp0.xy / inFrag.warp0.z);
结果不是我所期望的.我花了几个小时在此上，但我无法弄清楚. 放空
The result is not what I expect. I spent hours on this, but I cannot figure it out. venting
更新:
这些是CPU的代码和结果(又称预期结果):
These are code and result for CPU (aka expected result):
// _image contains the original image
cv::Matx33d h(1.03140473, 0.0778113901, 0.000169219566,
0.0342947133, 1.06025684, 0.000459250761,
-0.0364957005, -38.3375587, 0.818259298);
cv::Mat dest(_image.size(), CV_8UC4);
// h is transposed because OpenCV is col major and using backwarping because it is what is used on the GPU, so better for comparison
cv::warpPerspective(_image, dest, h.t(), _image.size(), cv::WARP_INVERSE_MAP | cv::INTER_LINEAR);
这些是GPU的代码和结果(又名错误的结果):
These are code and result for GPU (aka wrong result):
// constants passed in buffers, image size 320x240
const simd::float4 quadVertices[4] =
{
{ -1.0f, -1.0f, 0.0f, 1.0f },
{ +1.0f, -1.0f, 0.0f, 1.0f },
{ -1.0f, +1.0f, 0.0f, 1.0f },
{ +1.0f, +1.0f, 0.0f, 1.0f },
};
const simd::float3 textureCoords[4] =
{
{ 0, IMAGE_HEIGHT, 1.0f },
{ IMAGE_WIDTH, IMAGE_HEIGHT, 1.0f },
{ 0, 0, 1.0f },
{ IMAGE_WIDTH, 0, 1.0f },
};
// vertex shader
vertex VertexInOut homographyVertex(uint vid [[ vertex_id ]],
constant float4 *positions [[ buffer(0) ]],
constant float3 *texCoords [[ buffer(1) ]],
constant simd::float3x3 *homographies [[ buffer(2) ]])
{
VertexInOut v;
v.position = positions[vid];
// example homography
simd::float3x3 h = {
{1.03140473, 0.0778113901, 0.000169219566},
{0.0342947133, 1.06025684, 0.000459250761},
{-0.0364957005, -38.3375587, 0.818259298}
};
v.warp = h * texCoords[vid];
return v;
}
// fragment shader
fragment int4 homographyFragment(VertexInOut inFrag [[stage_in]],
texture2d<uint, access::sample> intensity [[ texture(1) ]])
{
constexpr sampler s(coord::pixel, filter::linear, address::clamp_to_zero);
float4 targetIntensity = intensityRight.sample(s, inFrag.warp.xy / inFrag.warp.z);
return targetIntensity;
}
原始图片:
更新2:
与通常的观点认为应该在片段着色器中进行透视划分相反，如果我在顶点着色器中进行划分(并且三角形之间没有扭曲或接缝)，则会得到与 更相似的结果，为什么?
Contrary to the common belief that the perspective divide should be done in the fragment shader, I get a much more similar result if I divide in the vertex shader (and no distortion or seam between triangles), but why?
更新3:
如果出现以下情况，我将得到相同(错误)的结果
I get the same (wrong) result if:
- 我将透视图分割移动到片段着色器
- 我只是消除了代码中的鸿沟
很奇怪，似乎没有发生鸿沟.
Very strange, it looks like the divide is not happening.
推荐答案
好的，解决方案当然是一个很小的细节:simd::float3的 division 表现得很疯狂.实际上，如果我在片段着色器中进行透视划分，如下所示:
OK, the solution was of course a very small detail: the division of simd::float3 behaves absolutely nuts. In fact, if I do the perspective divide in the fragment shader like this:
float4 targetIntensity = intensityRight.sample(s, inFrag.warp.xy * (1.0 / inFrag.warp.z));
有效！
这使我发现乘以预除浮点数与乘除浮点数是不同的.如果有人知道我们为什么可以揭开这个谜底，我仍然不知道其原因.
Which lead me to find out that multiplying by the pre-divided float is different than dividing by a float. The reason for this is still unknown to me, if anyone knows why we can unravel this mystery.
这篇关于如何在顶点/片段着色器(金属)中使用3x3 2D变换的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
