Processing Android camera frames in real time

Question

I'm trying to create an Android application that will process camera frames in real time. To start off with, I just want to display a grayscale version of what the camera sees. I've managed to extract the appropriate values from the byte array in the onPreviewFrame method. Below is just a snippet of my code:

byte[] pic;
int pic_size;
Bitmap picframe;
public void onPreviewFrame(byte[] frame, Camera c)
{
    pic_size = mCamera.getParameters().getPreviewSize().height * mCamera.getParameters().getPreviewSize().width;
    pic = new byte[pic_size];
    for(int i = 0; i < pic_size; i++)
    {
        pic[i] = frame[i];
    }
    picframe = BitmapFactory.decodeByteArray(pic, 0, pic_size);
}

The first [width*height] values of the byte[] frame array are the luminance (greyscale) values. Once I've extracted them, how do I display them on the screen as an image? Its not a 2D array as well, so how would I specify the width and height?

Answer 1

You can get extensive guidance from the OpenCV4Android SDK. Look into their available examples, specifically Tutorial 1 Basic. 0 Android Camera

But, as it was in my case, for intensive image processing, this will get slower than acceptable for a real-time image processing application. A good replacement for their onPreviewFrame 's byte array conversion to YUVImage:

YuvImage yuvImage = new YuvImage(frame, ImageFormat.NV21, width, height, null);

Create a rectangle the same size as the image.

Create a ByteArrayOutputStream and pass this, the rectangle and the compression value to compressToJpeg():

ByteArrayOutputStream baos = new ByteArrayOutputStream(); yuvimage.compressToJpeg(imageSizeRectangle, 100, baos);

byte [] imageData = baos.toByteArray();

Bitmap previewBitmap = BitmapFactory.decodeByteArray(imageData , 0, imageData .length);

Rendering these previewFrames on a surface and the best practices involved is a new dimension. =)