如何将图像格式YUV_420_888用于Google的MLKIT [英] How to use image format YUV_420_888 for MLKIT of Google

问题描述

ImageReader从相机预览中获取的每一帧都是格式为 YUV_420_888 的图像，我想将其用作MLKIT的输入.

ImageReader get each frame from camera preview is image with format YUV_420_888, i want to use it as input for MLKIT.

在Google文档中，我可以输入以下内容运行检测器:

In document of google, i can run detector with input is:

• 位图
• media.Image
• ByteBuffer
• ByteArray
• 文件

我尝试将 YUV_420_888 转换为上述对象，但失败了

I tried convert YUV_420_888 to above objects but I failed

推荐答案

我使用此功能将图像框架转换为NV21格式，并在

I used this function to convert the image frame to NV21 format and set the image type in metadata in FirebaseVisionImage

fun YUV_420_888toNV21(image: ImageProxy): ByteArray {

            val width = image.width
            val height = image.height
            val ySize = width * height
            val uvSize = width * height / 4

            val nv21 = ByteArray(ySize + uvSize * 2)
            val yBuffer = image.planes[0].buffer // Y
            val uBuffer = image.planes[1].buffer // U
            val vBuffer = image.planes[2].buffer // V

            var rowStride = image.planes[0].rowStride
            assert(image.planes[0].pixelStride == 1)

            var pos = 0

            //may need to flip the buffers if you get underflow exception

            if (rowStride == width) { // likely
                yBuffer.get(nv21, 0, ySize)
                pos += ySize

            } else {
                var yBufferPos = (width - rowStride).toLong() // not an actual position
                while (pos < ySize) {
                    yBufferPos = yBufferPos + (rowStride - width).toLong()
                    yBuffer.position(yBufferPos.toInt())
                    yBuffer.get(nv21, pos, width)
                    pos += width
                }
            }

            rowStride = image.planes[2].rowStride
            val pixelStride = image.planes[2].pixelStride

            assert(rowStride == image.planes[1].rowStride)
            assert(pixelStride == image.planes[1].pixelStride)

            if (pixelStride == 2 && rowStride == width && uBuffer.get(0) == vBuffer.get(1)) {
                // maybe V an U planes overlap as per NV21, which means vBuffer[1] is alias of uBuffer[0]
                val savePixel = vBuffer.get(1)
                vBuffer.put(1, 0.toByte())
                if (uBuffer.get(0).toInt() == 0) {
                    vBuffer.put(1, 255.toByte())
                    if (uBuffer.get(0).toInt() == 255) {
                        vBuffer.put(1, savePixel)
                        vBuffer.get(nv21, ySize, uvSize)
                        //Log.d("NV211",DataConverter.jsonify(nv21))
                        return nv21 // shortcut
                    }
                }

                // unfortunately, the check failed. We must save U and V pixel by pixel
                vBuffer.put(1, savePixel)
            }

            // other optimizations could check if (pixelStride == 1) or (pixelStride == 2),
            // but performance gain would be less significant

            for (row in 0 until height / 2) {
                for (col in 0 until width / 2) {
                    val vuPos = col * pixelStride + row * rowStride
                    nv21[pos++] = vBuffer.get(vuPos)
                    nv21[pos++] = uBuffer.get(vuPos)
                }
            }
            //Log.d("NV212",DataConverter.jsonify(nv21))
            return nv21
        }

这篇关于如何将图像格式YUV_420_888用于Google的MLKIT的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！