渲染脚本渲染比Android上的OpenGL渲染慢得多 [英] Render script rendering is much slower than OpenGL rendering on Android
问题描述
背景:
我想基于Android相机应用的代码添加实时滤镜.但是，Android相机应用程序的体系结构是基于OpenGL ES 1.x的.我需要使用着色器来自定义我们的过滤器实现.但是，将相机应用程序更新为OpenGL ES 2.0太困难了.然后，我必须找到其他一些方法来实现实时滤镜而不是OpenGL.经过研究，我决定使用渲染脚本.
I want to add live filter based on the code of Android camera app. But the architecture of Android camera app is based on OpenGL ES 1.x. I need to use shader to custom our filter implementation. However, it is too difficult to update the camera app to OpenGL ES 2.0. Then I have to find some other methods to implement live filter instead of OpenGL. I decided to use render script after some research.
问题:
我已经通过渲染脚本编写了一个简单过滤器的演示.它表明fps远低于OpenGL实现的fps.约5 fps和15 fps.
I have wrote a demo of a simple filter by render script. It shows that the fps is much lower than implementing it by OpenGL. About 5 fps vs 15 fps.
问题:
-
Android官方异地表示:RenderScript运行时将并行处理设备上所有可用处理器(例如多核CPU，GPU或DSP)上的工作，使您可以专注于表达算法，而不是安排工作或负载均衡.那么为什么渲染脚本的执行速度较慢?
The Android official offsite says: The RenderScript runtime will parallelize work across all processors available on a device, such as multi-core CPUs, GPUs, or DSPs, allowing you to focus on expressing algorithms rather than scheduling work or load balancing. Then why is render script implementation slower?
如果渲染脚本不能满足我的要求，还有更好的方法吗?
If render script cannot satisfy my requirement, is there a better way?
代码详细信息:
我和提问者在同一个团队中.我们要编写一个基于渲染脚本的实时滤镜相机.在我们的test-demo-project项目中，我们使用一个简单的过滤器:添加了叠加过滤器ScriptC脚本的YuvToRGB IntrinsicScript. 在OpenGL版本中，我们将相机数据设置为纹理，并使用着色器进行image-filter-procss.像这样:
Hi I am in the same team with the questioner. We want to write a render-script based live-filter camera. In our test-demo-project, we use a simple filter: a YuvToRGB IntrinsicScript added with a overlay-filter ScriptC script. In the OpenGL version, we set the camera data as textures and do the image-filter-procss with shader. Like this:
GLES20.glActiveTexture(GLES20.GL_TEXTURE0);
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, textureYHandle);
GLES20.glUniform1i(shader.uniforms.get("uTextureY"), 0);
GLES20.glTexSubImage2D(GLES20.GL_TEXTURE_2D, 0, 0, 0, mTextureWidth,
mTextureHeight, GLES20.GL_LUMINANCE, GLES20.GL_UNSIGNED_BYTE,
mPixelsYBuffer.position(0));
在RenderScript版本中，我们将摄影机数据设置为分配"，并使用脚本内核进行image-filter-procss.像这样:
In the RenderScript version, we set the camera data as Allocation and do the image-filter-procss with script-kernals. Like this:
// The belowing code is from onPreviewFrame(byte[] data, Camera camera) which gives the camera frame data
byte[] imageData = datas[0];
long timeBegin = System.currentTimeMillis();
mYUVInAllocation.copyFrom(imageData);
mYuv.setInput(mYUVInAllocation);
mYuv.forEach(mRGBAAllocationA);
// To make sure the process of YUVtoRGBA has finished!
mRGBAAllocationA.copyTo(mOutBitmap);
Log.e(TAG, "RS time: YUV to RGBA : " + String.valueOf((System.currentTimeMillis() - timeBegin)));
mLayerScript.forEach_overlay(mRGBAAllocationA, mRGBAAllocationB);
mRGBAAllocationB.copyTo(mOutBitmap);
Log.e(TAG, "RS time: overlay : " + String.valueOf((System.currentTimeMillis() - timeBegin)));
mCameraSurPreview.refresh(mOutBitmap, mCameraDisplayOrientation, timeBegin);
两个问题是: (1)RenderScript进程似乎比OpenGL进程慢. (2)根据我们的时间日志，使用固有脚本的YUV到RGBA的过程非常快，大约需要6毫秒；但是使用scriptC进行叠加的过程非常缓慢，大约需要180毫秒.这是怎么发生的?
The two problems are : (1) RenderScript process seems slower than OpenGL process. (2) According to our time-log, the process of YUV to RGBA which uses intrinsic script is very quick, takes about 6ms; but the process of overlay which uses scriptC is very slow, takes about 180ms. How does this happen?
这是我们使用的ScriptC的rs-kernal代码(mLayerScript):
Here is the rs-kernal code of the ScriptC we use(mLayerScript):
#pragma version(1)
#pragma rs java_package_name(**.renderscript)
#pragma stateFragment(parent)
#include "rs_graphics.rsh"
static rs_allocation layer;
static uint32_t dimX;
static uint32_t dimY;
void setLayer(rs_allocation layer1) {
layer = layer1;
}
void setBitmapDim(uint32_t dimX1, uint32_t dimY1) {
dimX = dimX1;
dimY = dimY1;
}
static float BlendOverlayf(float base, float blend) {
return (base < 0.5 ? (2.0 * base * blend) : (1.0 - 2.0 * (1.0 - base) * (1.0 - blend)));
}
static float3 BlendOverlay(float3 base, float3 blend) {
float3 blendOverLayPixel = {BlendOverlayf(base.r, blend.r), BlendOverlayf(base.g, blend.g), BlendOverlayf(base.b, blend.b)};
return blendOverLayPixel;
}
uchar4 __attribute__((kernel)) overlay(uchar4 in, uint32_t x, uint32_t y) {
float4 inPixel = rsUnpackColor8888(in);
uint32_t layerDimX = rsAllocationGetDimX(layer);
uint32_t layerDimY = rsAllocationGetDimY(layer);
uint32_t layerX = x * layerDimX / dimX;
uint32_t layerY = y * layerDimY / dimY;
uchar4* p = (uchar4*)rsGetElementAt(layer, layerX, layerY);
float4 layerPixel = rsUnpackColor8888(*p);
float3 color = BlendOverlay(inPixel.rgb, layerPixel.rgb);
float4 outf = {color.r, color.g, color.b, inPixel.a};
uchar4 outc = rsPackColorTo8888(outf.r, outf.g, outf.b, outf.a);
return outc;
}
推荐答案
Renderscript不使用任何GPU或DSP内核.谷歌故意模糊的文档助长了这是一个普遍的误解. Renderscript曾经具有到OpenGL ES的接口，但已过时，并且从未用于动画墙纸之外.如果可用，Renderscript将使用多个CPU内核，但我怀疑Renderscript将被OpenCL取代.
Renderscript does not use any GPU or DSPs cores. That is a common misconception encouraged by Google's deliberately vague documentation. Renderscript used to have an interface to OpenGL ES, but that has been deprecated and has never been used for much beyond animated wallpapers. Renderscript will use multiple CPU cores, if available, but I suspect Renderscript will be replaced by OpenCL.
看看Android SDK中的Effects类和Effects演示.它显示了如何使用OpenGL ES 2.0着色器将效果应用于图像，而无需编写OpenGL ES代码.
Take a look at the Effects class and the Effects demo in the Android SDK. It shows how to use OpenGL ES 2.0 shaders to apply effects to images without writing OpenGL ES code.
更新:
当我学到的问题多于问一个问题时，这真是太好了，在这里就是这种情况.您可以从缺乏答案的情况中看到，Renderscript很少在Google之外使用，因为其奇怪的架构忽略了OpenCL等行业标准以及有关其实际工作方式的几乎不存在的文档. 尽管如此，我的回答确实引起了Renderscrpt开发团队的罕见回应，该团队只包含一个实际上包含有关渲染脚本的有用信息的链接-PowerVR GPU供应商IMG的Alexandru Voica的这篇文章:
It's wonderful when I learn more answering a question than asking one and that is the case here. You can see from the lack of answers that Renderscript is hardly used outside of Google because of its strange architecture that ignores industry standards like OpenCL and almost non-existent documentation on how it actually works. Nonetheless, my answer did evoke a rare response from the Renderscrpt development team which includes only one link that actually contains any useful information about renderscript - this article by Alexandru Voica at IMG, the PowerVR GPU vendor:
那篇文章有一些很好的信息，这对我来说是新的.越来越多的人张贴评论，他们无法使Renderscript代码真正在GPU上运行.
That article has some good information which was new to me. There are comments posted there from more people who are having trouble getting Renderscript code to actually run on the GPU.
但是，我以为Renderscript不再由Google开发是不正确的.尽管我声明"Renderscript不使用任何GPU或DSP内核".直到最近才是真实的，我了解到自Jelly Bean发行版以来，这种情况已经发生了变化. 如果Renderscript开发人员之一可以解释这一点，那就太好了.或者即使他们有一个公开的网页对此进行了说明或列出了 实际支持哪些GPU，以及如何确定代码是否真正在GPU上运行.
But, I was incorrect to assume that Renderscript is no longer being developed at Google. Although my statement that "Renderscript does not use any GPU or DSPs cores." was true until just fairly recently, I have learned that this has changed as of one of the Jelly Bean releases. It would have been great if one of the Renderscript developers could have explained that. Or even if they had a public webpage that explains that or that lists which GPUs are actually supported and how can you tell if your code actually gets run on a GPU.
我认为Google最终将用OpenCL代替Renderscript，并且我不会花时间开发它.
My opinion is that Google will replace Renderscript with OpenCL eventually and I would not invest time developing with it.
这篇关于渲染脚本渲染比Android上的OpenGL渲染慢得多的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
