有没有办法裁剪 Image/ImageProxy(在传递给 MLKit 的分析器之前)? [英] Is there a way to crop Image/ImageProxy (before passing to MLKit's analyzer)?
问题描述
我将 CameraX 的分析器用例与 MLKit 的 BarcodeScanner 一起使用.我想裁剪从相机接收到的部分图像，然后再将其传递给扫描仪.
I'm using CameraX's Analyzer use case with the MLKit's BarcodeScanner. I would like to crop portion of the image received from the camera, before passing it to the scanner.
我现在正在做的是将 ImageProxy(我在分析器中收到的)转换为 Bitmap，裁剪它，然后将它传递给 BarcodeScanner.缺点是它不是一个非常快速和有效的过程.
What I'm doing right now is I convert ImageProxy (that I recieve in the Analyzer) to a Bitmap, crop it and then pass it to the BarcodeScanner. The downside is that it's not a very fast and efficient process.
我还注意到运行此代码时在 Logcat 中收到的警告:
I've also noticed the warning I get in the Logcat when running this code:
ML Kit 检测到您似乎将相机帧传递给检测器作为位图对象.这是低效的.请用用于 camera2 API 的 YUV_420_888 格式或(旧版)相机的 NV21 格式API 并将字节数组直接传递给 ML Kit.
ML Kit has detected that you seem to pass camera frames to the detector as a Bitmap object. This is inefficient. Please use YUV_420_888 format for camera2 API or NV21 format for (legacy) camera API and directly pass down the byte array to ML Kit.
最好不要进行 ImageProxy 转换，但是如何裁剪要分析的矩形?
It would be nice to not to do ImageProxy conversion, but how do I crop the rectangle I want to analyze?
我已经尝试过设置 Image (imageProxy.image.cropRect) 类的 cropRect 字段，但它似乎不会影响最终结果.
What I've already tried is to set a cropRect field of the Image (imageProxy.image.cropRect) class, but it doesn't seem to affect the end result.
推荐答案
是的，如果您使用 ViewPort 并将视口设置为您的 UseCases(imageCapture or imageAnalysis as here https://developer.android.com/training/camerax/configuration) 你只能得到关于裁剪矩形的信息，特别是如果你使用 ImageAnalysis(因为如果你使用 imageCapture，对于磁盘上的图像在保存之前被裁剪，它不适用于 ImageAnalysis，如果你使用 imageCapture 而不保存在磁盘上)和这里的解决方案我是如何解决这个问题的:
Yes, it's true that if you use ViewPort and set viewport to yours UseCases(imageCapture or imageAnalysis as here https://developer.android.com/training/camerax/configuration) you can get only information about crop rectangle especially if you use ImageAnalysis(because if you use imageCapture, for on-disk the image is cropped before saving and it doesn't work for ImageAnalysis and if you use imageCapture without saving on disk) and here solution how I solved this problem:
首先为用例设置视口，如下所示:https://developer.android.com/training/camerax/configuration
获取裁剪位图进行分析
override fun analyze(imageProxy: ImageProxy) {
val mediaImage = imageProxy.image
if (mediaImage != null && mediaImage.format == ImageFormat.YUV_420_888) {
croppedBitmap(mediaImage, imageProxy.cropRect).let { bitmap ->
requestDetectInImage(InputImage.fromBitmap(bitmap, rotation))
.addOnCompleteListener { imageProxy.close() }
}
} else {
imageProxy.close()
}
}
private fun croppedBitmap(mediaImage: Image, cropRect: Rect): Bitmap {
val yBuffer = mediaImage.planes[0].buffer // Y
val vuBuffer = mediaImage.planes[2].buffer // VU
val ySize = yBuffer.remaining()
val vuSize = vuBuffer.remaining()
val nv21 = ByteArray(ySize + vuSize)
yBuffer.get(nv21, 0, ySize)
vuBuffer.get(nv21, ySize, vuSize)
val yuvImage = YuvImage(nv21, ImageFormat.NV21, mediaImage.width, mediaImage.height, null)
val outputStream = ByteArrayOutputStream()
yuvImage.compressToJpeg(cropRect, 100, outputStream)
val imageBytes = outputStream.toByteArray()
return BitmapFactory.decodeByteArray(imageBytes, 0, imageBytes.size)
}
可能会降低转换速度，但在我的设备上我没有注意到差异.我在compressToJpeg方法中设置了100的质量，但是mb如果设置的质量低可以提高速度，需要测试.
Possibly there is a loss in conversion speed, but on my devices I did not notice the difference. I set 100 quality in method compressToJpeg, but mb if set less quality it can improve speed, it need test.
更新:21 年 5 月 2 日:
我找到了另一种方法，无需先转换为 jpeg，然后再转换为位图.这应该是一种更快的方式.
I found another way without convert to jpeg and then to bitmap. This should be a faster way.
将视口设置为以前的.
Set viewport as previous.
将 YUV_420_888 转换为 NV21，然后进行裁剪和分析.
Convert YUV_420_888 to NV21, then crop and analyze.
override fun analyze(imageProxy: ImageProxy) {
val mediaImage = imageProxy.image
if (mediaImage != null && mediaImage.format == ImageFormat.YUV_420_888) {
croppedNV21(mediaImage, imageProxy.cropRect).let { byteArray ->
requestDetectInImage(
InputImage.fromByteArray(
byteArray,
imageProxy.cropRect.width(),
imageProxy.cropRect.height(),
rotation,
IMAGE_FORMAT_NV21,
)
)
.addOnCompleteListener { imageProxy.close() }
}
} else {
imageProxy.close()
}
}
private fun croppedNV21(mediaImage: Image, cropRect: Rect): ByteArray {
val yBuffer = mediaImage.planes[0].buffer // Y
val vuBuffer = mediaImage.planes[2].buffer // VU
val ySize = yBuffer.remaining()
val vuSize = vuBuffer.remaining()
val nv21 = ByteArray(ySize + vuSize)
yBuffer.get(nv21, 0, ySize)
vuBuffer.get(nv21, ySize, vuSize)
return cropByteArray(nv21, mediaImage.width, cropRect)
}
private fun cropByteArray(array: ByteArray, imageWidth: Int, cropRect: Rect): ByteArray {
val croppedArray = ByteArray(cropRect.width() * cropRect.height())
var i = 0
array.forEachIndexed { index, byte ->
val x = index % imageWidth
val y = index / imageWidth
if (cropRect.left <= x && x < cropRect.right && cropRect.top <= y && y < cropRect.bottom) {
croppedArray[i] = byte
i++
}
}
return croppedArray
}
我从这里获得的第一个作物乐趣: Android:如何使用 CameraX 裁剪图像?
而且我还发现了另一种作物乐趣，似乎更复杂:
And I found also another crop fun, it seems that it is more complicated:
private fun cropByteArray(src: ByteArray, width: Int, height: Int, cropRect: Rect, ): ByteArray {
val x = cropRect.left * 2 / 2
val y = cropRect.top * 2 / 2
val w = cropRect.width() * 2 / 2
val h = cropRect.height() * 2 / 2
val yUnit = w * h
val uv = yUnit / 2
val nData = ByteArray(yUnit + uv)
val uvIndexDst = w * h - y / 2 * w
val uvIndexSrc = width * height + x
var srcPos0 = y * width
var destPos0 = 0
var uvSrcPos0 = uvIndexSrc
var uvDestPos0 = uvIndexDst
for (i in y until y + h) {
System.arraycopy(src, srcPos0 + x, nData, destPos0, w) //y memory block copy
srcPos0 += width
destPos0 += w
if (i and 1 == 0) {
System.arraycopy(src, uvSrcPos0, nData, uvDestPos0, w) //uv memory block copy
uvSrcPos0 += width
uvDestPos0 += w
}
}
return nData
}
我从这里获得的第二次收获乐趣:https://www.programmersought.com/article/75461140907/
如果有人能帮助改进代码，我会很高兴.
这篇关于有没有办法裁剪 Image/ImageProxy(在传递给 MLKit 的分析器之前)?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
