相机预览和OCR [英] Camera Preview and OCR

问题描述

我是android开发的新手-我正在使用Xamarin.

I am new to android development - I'm using Xamarin.

我试图编写一个应用程序来启动相机预览，然后不断扫描传入的帧中的文本(我使用的是NuGet的Xamarin.Tesseract).

I am trying to write an application that initiates the camera preview, and then constantly scans the incoming frames for text (I am using Xamarin.Tesseract from NuGet).

换句话说，我不想让用户拍张照片然后进行OCR分析，而是希望他们仅将摄像机对准带有文字的纸，我将继续进行OCR分析，直到我检测到要搜索的特定文本为止)，此时，我将向用户大加赞赏.

In other words, I don't want to make the user take a photo and then do the OCR analysis, instead I want them to just point the video camera at some paper with text on it, i'll continually do the OCR analysis until I detect the specific text I'm searching for) at which point I'll give a big thumbs up to the user.

这是我到目前为止所采用的方法:

This is the approach I have gone down so far:

1. 初始化相机并设置预览回调

1. Initialise the camera and set a preview callback

_Camera = Android.Hardware.Camera.Open();          
_Camera.SetPreviewCallback(this); 
_Camera.StartPreview();              

在回调中，获取代表当前帧的字节，并将其作为Xamarin.Tesseract的输入图像字节.

In the Callback, take the bytes representing the current frame and pass this as the input image bytes for Xamarin.Tesseract

public void OnPreviewFrame(byte[] data, Android.Hardware.Camera camera)
{        
await _TesseractApi.SetImage(data); /// this hangs                
string text = _Api.Text;
return text;          
} 

当前在将byte []传递到Tesseract API时挂起.我很确定这将是因为数组中的字节不是正确的编码，还是我从根本上不了解Camera api！

This currently hangs when passing the byte[] into the Tesseract API. I'm pretty sure it's going to be because the Bytes in the array are either the wrong encoding, or, i'm fundamentally not understanding the Camera api!

任何人都可以在书写方向上向我推一下吗?

Can anyone give me a nudge in the write direction?

推荐答案

查看

Looking at the code for TesseractApi.SetImage(byte[]), it is calling BitmapFactory.DecodeByteArray() which expects a valid Bitmap.

不幸的是，相机预览返回的是YUV图像，而 BitmapFactory 不支持.

Unfortunately, the camera preview is returning a YUV image, which BitmapFactory doesn't support.

这里是将YUV图像转换为JPEG的代码，然后您可以将其传递给Tesseract.

Here is code to transform the YUV image to a JPEG which you can then pass to Tesseract.

private byte[] ConvertYuvToJpeg(byte[] yuvData, Android.Hardware.Camera camera)
{
    var cameraParameters = camera.GetParameters();
    var width = cameraParameters.PreviewSize.Width;
    var height = cameraParameters.PreviewSize.Height;
    var yuv = new YuvImage(yuvData, cameraParameters.PreviewFormat, width, height, null);   
    var ms = new MemoryStream();
    var quality = 80;   // adjust this as needed
    yuv.CompressToJpeg(new Rect(0, 0, width, height), quality, ms);
    var jpegData = ms.ToArray();

    return jpegData;
}

这篇关于相机预览和OCR的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！