摄像机在多个视图上的预览-初始化/释放处理 [英] Camera preview on multiple views - initialize/release handling
问题描述
我将Android相机的用法封装到一个类（CameraAccess）中，该类使用不可见的SurfaceTexture作为预览并实现Camera.PreviewCallback。在此回调中，我得到当前帧的字节数组，然后在多个视图/片段上使用。
I encapsulated the usage of the Android camera into one single class (CameraAccess) that uses an invisible SurfaceTexture as preview and which implements the Camera.PreviewCallback. Within this callback I get the byte array of the current frame which I then want to use on multiple views/fragments.
我的问题是生命周期管理。通常，相机在单个View中使用，并在onSurfaceCreated和onSurfaceDestroyed中初始化/释放（请参见SurfaceHolder.Callback）。但是在我的场景中，我需要在多个视图上使用预览。每个视图都将自己添加为CameraAccess类的回调。
My problem is the life-cycle management. Usually the camera is used within a single View and initialized/release in onSurfaceCreated and onSurfaceDestroyed (see SurfaceHolder.Callback). But in my scenario I need to use the preview on more than one View. Each view adds itself as a callback to the CameraAccess class.
我想将CameraAccess作为成员加入Application类。但是，当您按下主页按钮时，该应用程序仍处于活动状态，但所有视图均被破坏。您将如何处理相机的初始化和释放？
I thought to put the CameraAccess as a member into the Application class. But when you press the home button then the Application is still alive but all views are destroyed. How would you handle the init and release of the camera?
推荐答案
这是我解决的方法。
您需要解决一些问题。首先，OpenCV矩阵的矩阵/颜色转换，这在Android中是非常昂贵的操作。因此，建议仅执行一次。此类还确保仅当UI请求位图时，原始矩阵才转换一次。
There are some problems you need to solve. First, the matrix/color conversions of the OpenCV matrix, which is a very expensive operation in Android. Therefore, it is recommended to do that only once. This class makes also sure that the original matrix is only converted once and only if the UI is requesting the bitmap.
import org.opencv.android.CameraBridgeViewBase.CvCameraViewFrame;
import org.opencv.core.Mat;
import android.graphics.Bitmap;
public interface CameraFrame extends CvCameraViewFrame {
Bitmap toBitmap();
@Override
Mat rgba();
@Override
Mat gray();
}
该类的实现如下：
private class CameraAccessFrame implements CameraFrame {
private Mat mYuvFrameData;
private Mat mRgba;
private int mWidth;
private int mHeight;
private Bitmap mCachedBitmap;
private boolean mRgbaConverted;
private boolean mBitmapConverted;
@Override
public Mat gray() {
return mYuvFrameData.submat(0, mHeight, 0, mWidth);
}
@Override
public Mat rgba() {
if (!mRgbaConverted) {
Imgproc.cvtColor(mYuvFrameData, mRgba,
Imgproc.COLOR_YUV2BGR_NV12, 4);
mRgbaConverted = true;
}
return mRgba;
}
@Override
public Bitmap toBitmap() {
if (mBitmapConverted)
return mCachedBitmap;
Mat rgba = this.rgba();
Utils.matToBitmap(rgba, mCachedBitmap);
mBitmapConverted = true;
return mCachedBitmap;
}
public CameraAccessFrame(Mat Yuv420sp, int width, int height) {
super();
mWidth = width;
mHeight = height;
mYuvFrameData = Yuv420sp;
mRgba = new Mat();
this.mCachedBitmap = Bitmap.createBitmap(width, height,
Bitmap.Config.ARGB_8888);
}
public void release() {
mRgba.release();
mCachedBitmap.recycle();
}
public void invalidate() {
mRgbaConverted = false;
mBitmapConverted = false;
}
};
Matrix转换由OpenCV Utils org.opencv.android完成。 Utils.matToBitmap（converted，bmp）; 。由于我们只想接收一次摄像机图像，但是要在多个视图上显示它，所以这是1：n的关系。 1是接收图像的组件（将在后面说明），而n是要使用该图像的任何UI视图。对于这些UI回调，我创建了此接口。
The Matrix conversion is done by the OpenCV Utils org.opencv.android.Utils.matToBitmap(converted, bmp);. Since we want to receive the camera image only once, but displaying it on multiple views, it is a 1:n relationship. The 1 is the component that receives the image (will be explained later), while the n is any UI view that wants to use the image. For those UI callbacks, I created this interface.
public interface CameraFrameCallback {
void onCameraInitialized(int frameWidth, int frameHeight);
void onFrameReceived(CameraFrame frame);
void onCameraReleased();
}
由 CameraCanvasView ，这是一个Android视图。 SurfaceView 和 SurfaceHolder 可以在 android.graphics 中找到。视图的真正UI是Surface。因此，创建表面（显示在显示屏上）后，视图将自己注册到 CameraAccess （后面显示的1：n关系中的1）。每当 CameraAcess 收到新的摄像机图像时，它将在所有已注册的回调中调用 onFrameReceived 。由于视图是这样的回调，因此它将从 CameraFrame 中读取位图并显示它。
It is implemented by the CameraCanvasView, which is an Android view. SurfaceView and SurfaceHolder can be found in android.graphics. The real UI of the view is the Surface. So, when the surface is created (show on the display), the view will register itself to the CameraAccess (the 1 from the 1:n relationship shown later). Whenever a new camera image is received by the CameraAcess it will invoke onFrameReceived on all registered callbacks. Since the view is such a callback, it will read the bitmap from the CameraFrame and display it.
public class CameraCanvasView extends SurfaceView implements CameraFrameCallback, SurfaceHolder.Callback {
Context context;
CameraAccess mCamera;
Paint paint = new Paint(Paint.ANTI_ALIAS_FLAG);
Rect mBackgroundSrc = new Rect();
public CameraCanvasView(Context context) {
super(context);
this.context = context;
SurfaceHolder sh = this.getHolder();
sh.addCallback(this);
setFocusable(true);
this.mCamera = CameraAccess.getInstance(context,
CameraInfo.CAMERA_FACING_BACK);
}
@Override
public void onCameraInitialized(int frameWidth, int frameHeight) {
}
@Override
public void onFrameReceived(CameraFrame frame) {
this.setBackgroundImage(frame.toBitmap());
}
@Override
public void onCameraReleased() {
setBackgroundImage(null);
}
@Override
public void surfaceCreated(SurfaceHolder arg0) {
this.setWillNotDraw(false);
this.mCamera.addCallback(this);
}
@Override
public void surfaceChanged(SurfaceHolder arg0, int arg1, int arg2, int arg3) {
}
@Override
public void surfaceDestroyed(SurfaceHolder arg0) {
this.mCamera.removeCallback(this);
}
public void setBackgroundImage(Bitmap image) {
this.mBackground = image;
if (image != null)
this.mBackgroundSrc.set(0, 0, image.getWidth(), image.getHeight());
else
this.mBackgroundSrc.setEmpty();
invalidate();
}
@Override
public void onDraw(Canvas canvas) {
canvas.drawColor(Color.BLACK);
if (mBackground != null && !mBackground.isRecycled())
canvas.drawBitmap(mBackground, mBackgroundSrc, boundingBox, paint);
}
}
最后，您需要使用相机手柄，即1我们的1：n关系。这是 CameraAccess 。它处理相机初始化并将其自身注册为回调，Android会在收到新帧时通知该回调。这是 android.hardware.Camera.PreviewCallback 。可以在Android本身中找到它的基类。 CameraAccess 还包含一个单独的 CameraAccessFrame 和一个单独的OpenCV矩阵。每当接收到新图像时，该图像都会放入现有的OpenCV矩阵中，从而覆盖矩阵值并使 CameraAcessFrame 无效，以通知与其绑定的任何UI元素。覆盖现有矩阵可为您节省用于释放和保留内存的内存操作。因此，请勿破坏并重新创建矩阵，而要覆盖它。值得一提的是， CameraAccess 是逻辑上不可见的组件，而不是可视的Android视图。通常，相机图像直接显示在UI元素上，而Android需要在其上渲染表面/视图。由于组件是不可见的，因此需要手动创建 SurfaceTexture 。相机将自动渲染为该纹理。
Finally, you need the camera handler, the 1 in our 1:n relationship. This is the CameraAccess. It handles the camera initialization and registers itself as a callback that is notified by Android whenever a new frame is received. This is the android.hardware.Camera.PreviewCallback. The base class for this can be found in Android itself. The CameraAccess also holds one single CameraAccessFrame with one single OpenCV Matrix. Whenever a new image is received, that image is put to the existing OpenCV matrix thus overwriting the matrix values and invalidating the CameraAcessFrame to notify any UI element, that is bound to it. Overwriting an existing matrix is saving you memory operations for freeing and reserving memory. Therefore, do not destroy and recreate a matrix, but overwrite it. It is important to mention, that the CameraAccess is a logical invisible component and not a visual Android view. Usually camera images are directly shown on UI elements and Android needs a surface/view to render on. Since my component is invisible, I need to create a SurfaceTexture manually. The camera will automatically render into that texture.
public class CameraAccess implements Camera.PreviewCallback,
LoaderCallbackInterface {
// see http://developer.android.com/guide/topics/media/camera.html for more
// details
final static String TAG = "CameraAccess";
Context context;
int cameraIndex; // example: CameraInfo.CAMERA_FACING_FRONT or
// CameraInfo.CAMERA_FACING_BACK
Camera mCamera;
int mFrameWidth;
int mFrameHeight;
Mat mFrame;
CameraAccessFrame mCameraFrame;
List<CameraFrameCallback> mCallbacks = new ArrayList<CameraFrameCallback>();
boolean mOpenCVloaded;
byte mBuffer[]; // needed to avoid OpenCV error:
// "queueBuffer: BufferQueue has been abandoned!"
private static CameraAccess mInstance;
public static CameraAccess getInstance(Context context, int cameraIndex) {
if (mInstance != null)
return mInstance;
mInstance = new CameraAccess(context, cameraIndex);
return mInstance;
}
private CameraAccess(Context context, int cameraIndex) {
this.context = context;
this.cameraIndex = cameraIndex;
if (!OpenCVLoader.initAsync(OpenCVLoader.OPENCV_VERSION_2_4_7, context,
this)) {
Log.e(TAG, "Cannot connect to OpenCVManager");
} else
Log.d(TAG, "OpenCVManager successfully connected");
}
private boolean checkCameraHardware() {
if (context.getPackageManager().hasSystemFeature(
PackageManager.FEATURE_CAMERA)) {
// this device has a camera
return true;
} else {
// no camera on this device
return false;
}
}
public static Camera getCameraInstance(int cameraIndex) {
Camera c = null;
try {
c = Camera.open(cameraIndex); // attempt to get a
// Camera
// instance
Log.d(TAG, "Camera opened. index: " + cameraIndex);
} catch (Exception e) {
// Camera is not available (in use or does not exist)
}
return c; // returns null if camera is unavailable
}
public void addCallback(CameraFrameCallback callback) {
// we don't care if the callback is already in the list
this.mCallbacks.add(callback);
if (mCamera != null)
callback.onCameraInitialized(mFrameWidth, mFrameHeight);
else if (mOpenCVloaded)
connectCamera();
}
public void removeCallback(CameraFrameCallback callback) {
boolean removed = false;
do {
// someone might have added the callback multiple times
removed = this.mCallbacks.remove(callback);
if (removed)
callback.onCameraReleased();
} while (removed == true);
if (mCallbacks.size() == 0)
releaseCamera();
}
@Override
public void onPreviewFrame(byte[] frame, Camera arg1) {
mFrame.put(0, 0, frame);
mCameraFrame.invalidate();
for (CameraFrameCallback callback : mCallbacks)
callback.onFrameReceived(mCameraFrame);
if (mCamera != null)
mCamera.addCallbackBuffer(mBuffer);
}
private void connectCamera() {
synchronized (this) {
if (true) {// checkCameraHardware()) {
mCamera = getCameraInstance(cameraIndex);
Parameters params = mCamera.getParameters();
List<Camera.Size> sizes = params.getSupportedPreviewSizes();
// Camera.Size previewSize = sizes.get(0);
Collections.sort(sizes, new PreviewSizeComparer());
Camera.Size previewSize = null;
for (Camera.Size s : sizes) {
if (s == null)
break;
previewSize = s;
}
// List<Integer> formats = params.getSupportedPictureFormats();
// params.setPreviewFormat(ImageFormat.NV21);
params.setPreviewSize(previewSize.width, previewSize.height);
mCamera.setParameters(params);
params = mCamera.getParameters();
mFrameWidth = params.getPreviewSize().width;
mFrameHeight = params.getPreviewSize().height;
int size = mFrameWidth * mFrameHeight;
size = size
* ImageFormat
.getBitsPerPixel(params.getPreviewFormat()) / 8;
mBuffer = new byte[size];
mFrame = new Mat(mFrameHeight + (mFrameHeight / 2),
mFrameWidth, CvType.CV_8UC1);
mCameraFrame = new CameraAccessFrame(mFrame, mFrameWidth,
mFrameHeight);
SurfaceTexture texture = new SurfaceTexture(0);
try {
mCamera.setPreviewTexture(texture);
mCamera.addCallbackBuffer(mBuffer);
mCamera.setPreviewCallbackWithBuffer(this);
mCamera.startPreview();
Log.d(TAG, "Camera preview started");
} catch (Exception e) {
Log.d(TAG,
"Error starting camera preview: " + e.getMessage());
}
for (CameraFrameCallback callback : mCallbacks)
callback.onCameraInitialized(mFrameWidth, mFrameHeight);
}
}
}
private void releaseCamera() {
synchronized (this) {
if (mCamera != null) {
mCamera.stopPreview();
mCamera.setPreviewCallback(null);
mCamera.release();
Log.d(TAG, "Preview stopped and camera released");
}
mCamera = null;
if (mFrame != null) {
mFrame.release();
}
if (mCameraFrame != null) {
mCameraFrame.release();
}
for (CameraFrameCallback callback : mCallbacks)
callback.onCameraReleased();
}
}
public interface CameraFrameCallback {
void onCameraInitialized(int frameWidth, int frameHeight);
void onFrameReceived(CameraFrame frame);
void onCameraReleased();
}
@Override
public void onManagerConnected(int status) {
mOpenCVloaded = true;
if (mCallbacks.size() > 0)
connectCamera();
}
@Override
public void onPackageInstall(int operation,
InstallCallbackInterface callback) {
}
private class PreviewSizeComparer implements Comparator<Camera.Size> {
@Override
public int compare(Size arg0, Size arg1) {
if (arg0 != null && arg1 == null)
return -1;
if (arg0 == null && arg1 != null)
return 1;
if (arg0.width < arg1.width)
return -1;
else if (arg0.width > arg1.width)
return 1;
else
return 0;
}
}
}
大多数 CameraAccess 中的代码是关于Android相机的初始化和处理的。我将不进一步说明如何初始化相机。那里有很多文档。
Most of the code in CameraAccess is about the initialization and handling of the Android camera. I will not explain any further how a camera is being initialized. There is plenty of documentation out there.
更新2020年5月12日：根据要求，我在相当长的代码中添加了更多详细信息和说明。如果对其他课程还有其他疑问，请告诉我。
Update 12. May 2020: On request, I added more details and explanation to the quite long code. In case of any further question about the other classes, let me know.
这篇关于摄像机在多个视图上的预览-初始化/释放处理的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
