相机预览有效模糊 [英] Effective blurring of camera preview

问题描述

到目前为止我已经尝试过的:

What I've tried so far:

将每个帧转换为位图，并使用库对其进行模糊处理，然后将其放入 ImageView .显然太慢了-类似于 1 fps .

Convert every frame into bitmap, blur it with library and put it into ImageView which is in front of camera preview. Obviously was too slow - something like 1 fps.

然后我开始使用 RenderScript ，它会模糊每一帧，并且应将处理结果放置在 TextureView (这是摄像机的预览)中.

Then I started to use RenderScript which blurs every frame and result of processing should be placed in TextureView which is cover camera preview.

该方法的基本代码安全性:

Essential peaces of code of that approach:

BlurFilter

ScriptIntrinsicBlur.create(rs, Element.RGBA_8888(rs)).apply {
    setRadius(BLUR_RADIUS)
}
private val yuvToRgb = ScriptIntrinsicYuvToRGB.create(rs, Element.RGBA_8888(rs))
private var surface: SurfaceTexture? = null

private fun setupSurface() {
    if (surface != null) {
        aBlurOut?.surface = Surface(surface)
    }
}

fun reset(width: Int, height: Int) {
    aBlurOut?.destroy()

    this.width = width
    this.height = height

    val tbConvIn = Type.Builder(rs, Element.U8(rs))
            .setX(width)
            .setY(height)
            .setYuvFormat(android.graphics.ImageFormat.NV21)
    aConvIn = Allocation.createTyped(rs, tbConvIn.create(), Allocation.USAGE_SCRIPT)

    val tbConvOut = Type.Builder(rs, Element.RGBA_8888(rs))
            .setX(width)
            .setY(height)
    aConvOut = Allocation.createTyped(rs, tbConvOut.create(), Allocation.USAGE_SCRIPT)

    val tbBlurOut = Type.Builder(rs, Element.RGBA_8888(rs))
            .setX(width)
            .setY(height)
    aBlurOut = Allocation.createTyped(rs, tbBlurOut.create(),
            Allocation.USAGE_SCRIPT or Allocation.USAGE_IO_OUTPUT)

    setupSurface()
}

fun execute(yuv: ByteArray) {
    if (surface != null) {
        //YUV -> RGB
        aConvIn!!.copyFrom(yuv)
        yuvToRgb.setInput(aConvIn)
        yuvToRgb.forEach(aConvOut)
        //RGB -> BLURED RGB
        blurRc.setInput(aConvOut)
        blurRc.forEach(aBlurOut)
        aBlurOut!!.ioSend()
    }
}

MainActivity

override fun onCreate(savedInstanceState: Bundle?) {
    super.onCreate(savedInstanceState)
    setContentView(R.layout.activity_main)

    initQrScanner()
}

override fun onStart() {
    super.onStart()
    fotoapparat.start()
}

override fun onStop() {
    fotoapparat.stop()
    super.onStop()
}

private fun initQrScanner() {
    val filter = BlurFilter(RenderScript.create(this))
    tvWholeOverlay.surfaceTextureListener = filter

    fotoapparat = Fotoapparat
            .with(this)
            .into(cvQrScanner)
            .frameProcessor({
                if (it.size.width != filter.width || it.size.height != filter.height) {
                    filter.reset(it.size.width, it.size.height)
                }
                filter.execute(it.image)
            })
            .build()
}

activity_main.xml

<android.support.constraint.ConstraintLayout 
xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    xmlns:tools="http://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    tools:context="com.blur.andrey.blurtest.MainActivity">

    <io.fotoapparat.view.CameraView
        android:id="@+id/cvQrScanner"
        android:layout_width="match_parent"
        android:layout_height="match_parent" />

    <TextureView
        android:id="@+id/tvWholeOverlay"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        app:layout_constraintLeft_toLeftOf="parent"
        app:layout_constraintRight_toRightOf="parent"
        app:layout_constraintTop_toTopOf="parent" />

</android.support.constraint.ConstraintLayout>

不幸的是，它仍然很慢- 3-4 FPS .同样，模糊的叠加层也会旋转，但这是另一个问题.

And unfortunately it is still to slow - 3-4 FPS. Also blurring overlay is rotated, but it's another problem.

我已经在Github上创建了 测试项目 ，您可以在其中快速复制问题并检查如何进行优化.期待您的想法.

I've created test project on Github where you can quickly reproduce problem and check how it is possible to optimise. Looking forward for your ideas.

UPD 我可以通过在模糊之前缩小输入日期来提高性能.我将这些更改推送到测试存储库中.现在，即使在低端设备上，我也具有非常好的(15-20 FPS)性能，但是分辨率较低(例如HD)，而在FHD和UHD上却不够好((8-12 FPS).

UPD I was able to improve performance with scaling down input date before blurring. I pushed those changes to test repo. Now I have really good (15-20 FPS) performance even on low end devices, but with low res (HD for instance), and not good enough on FHD and UHD ((8-12 FPS).

推荐答案

我认为您可以采取的最佳方法是将 CameraView 替换为 TextureView 并将其用作相机预览.您可以在此处 https://developer.android.com上轻松找到示例./reference/android/view/TextureView.html ，以及此处 https://github.com/dalinaum/TextureViewDemo/blob/master/src/kr/gdg/android/textureview/CameraActivity.java .

I think the best approach you can take is to replace CameraView with a TextureView and use it as a camera preview. You can easily find examples how to do it here https://developer.android.com/reference/android/view/TextureView.html, and here https://github.com/dalinaum/TextureViewDemo/blob/master/src/kr/gdg/android/textureview/CameraActivity.java.

下一步是在 TextureView 上使用模糊着色器.您可以在此处找到示例: https://github.com/nekocode/blur-using-textureview/blob/master/app/src/main/java/cn/nekocode/blurringview/sample/BlurFilter.java

Next step is to use a blurring shader on your TextureView. You can find an example here: https://github.com/nekocode/blur-using-textureview/blob/master/app/src/main/java/cn/nekocode/blurringview/sample/BlurFilter.java

您可以使用此着色器或找到一些更漂亮的着色器.

You can use this shader or find some more nice-looking one.

如果您想使用现成的解决方案，可以看一下这个库: https://github.com/krazykira/VidEffects

In case if you want a ready to use solution, you can take a look at this library: https://github.com/krazykira/VidEffects

它允许您将着色器应用于视频视图"，因此，再次使用模糊着色器可以达到所需的效果.

It allows you to apply shaders to a Video View, so again, you can use a blurring shader to achieve a desired effect.

该库的示例: https://stackoverflow.com/a/38125734/3286059

编辑

因此，我从Nekocode分支了一个很棒的库，并添加了一个有效的模糊滤镜.请查看我的最新提交. https://github.com/Dimezis/CameraFilter

So I forked a great library from Nekocode and added a working blur filter. Please see my latest commit. https://github.com/Dimezis/CameraFilter

正如我所说的，TextureView + GL着色器.要启用模糊着色器，请单击右上角的菜单，然后选择相应的选项.

As I said, TextureView + GL shaders. To enable blur shader, click on menu in top right corner and select the corresponding option.

如果您需要更快/更好/更简单的模糊着色器，则可以在 https://www.shadertoy.com上搜索一个

If you need faster/better/simpler blur shader, you can search for one on https://www.shadertoy.com

在某些设备上，相机预览可能会翻转，但这是另一个要解决的问题.

On some devices you might camera preview flipped, but that's another problem to solve.

这篇关于相机预览有效模糊的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！