Error with Camera2 API when calling session.setRepeatingRequest(captureRequestBuilder.build(), null, backgroundHandler)

Question

I am trying to make a camera preview app. When I call

session.setRepeatingRequest(captureRequestBuilder.build(), null, backgroundHandler)

I get this error:

E/HandlerDispatcher: IllegalArgumentException while invoking public abstract void android.hardware.camera2.CameraCaptureSession$StateCallback.onConfigured(android.hardware.camera2.CameraCaptureSession)
    java.lang.IllegalArgumentException: Each request must have at least one Surface target
        at android.hardware.camera2.impl.CameraDeviceImpl.submitCaptureRequest(CameraDeviceImpl.java:934)
        at android.hardware.camera2.impl.CameraDeviceImpl.setRepeatingRequest(CameraDeviceImpl.java:993)
        at android.hardware.camera2.impl.CameraCaptureSessionImpl.setRepeatingRequest(CameraCaptureSessionImpl.java:239)
        at anthe.rwca.org.robocup19.MainActivity$3.onConfigured(MainActivity.java:147)
        at java.lang.reflect.Method.invoke(Native Method)
        at android.hardware.camera2.dispatch.InvokeDispatcher.dispatch(InvokeDispatcher.java:39)
        at android.hardware.camera2.dispatch.HandlerDispatcher$1.run(HandlerDispatcher.java:65)
        at android.os.Handler.handleCallback(Handler.java:789)
        at android.os.Handler.dispatchMessage(Handler.java:98)
        at android.os.Looper.loop(Looper.java:164)
        at android.os.HandlerThread.run(HandlerThread.java:65)

This is my code:

CameraDevice.StateCallback stateCallback = new CameraDevice.StateCallback() {
        @Override
        public void onOpened(@NonNull CameraDevice camera) {
            cameraDevice = camera;
            createCaptureSession();
        }

        @Override
        public void onDisconnected(@NonNull CameraDevice camera) {

        }

        @Override
        public void onError(@NonNull CameraDevice camera, int error) {

        }
    };


    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);

        textureView = findViewById(R.id.textureView);
    }

    @Override
    protected void onResume() {
        super.onResume();
        textureView.setSurfaceTextureListener(new TextureView.SurfaceTextureListener() {
            @Override
            public void onSurfaceTextureAvailable(SurfaceTexture surface, int width, int height) {
                openBackgroundHandler();
                setupCamera();
                openCamera();
            }

            @Override
            public void onSurfaceTextureSizeChanged(SurfaceTexture surface, int width, int height) {

            }

            @Override
            public boolean onSurfaceTextureDestroyed(SurfaceTexture surface) {
                return false;
            }

            @Override
            public void onSurfaceTextureUpdated(SurfaceTexture surface) {

            }
        });
    }

    @Override
    protected void onPause() {
        super.onPause();
        closeBackgroundHandler();
    }

    private void setupCamera() {
        cameraManager = (CameraManager) getSystemService(CAMERA_SERVICE);
        try {
            cameraIDs = cameraManager.getCameraIdList();
            cameraID = cameraIDs[0];
            Log.i(TAG, String.valueOf(cameraIDs.length));
            Log.i(TAG, cameraID);
            CameraCharacteristics cameraCharacteristics = cameraManager.getCameraCharacteristics(cameraID);
            StreamConfigurationMap streamConfigurationMap = cameraCharacteristics.get(CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP);
            size = streamConfigurationMap.getOutputSizes(SurfaceTexture.class)[0];
        } catch (CameraAccessException e) {
            e.printStackTrace();
        }
    }

    @SuppressLint("MissingPermission")
    private void openCamera() {
        try {
            cameraManager.openCamera(cameraID, stateCallback, backgroundHandler);
        } catch (CameraAccessException e) {
            e.printStackTrace();
        }
    }

    private void openBackgroundHandler() {
        handlerThread = new HandlerThread("camera_app");
        handlerThread.start();
        backgroundHandler = new Handler(handlerThread.getLooper());
    }

    private void closeBackgroundHandler() {
        handlerThread.quit();
        handlerThread = null;
        backgroundHandler = null;
    }

    private void createCaptureSession() {
        SurfaceTexture surfaceTexture = textureView.getSurfaceTexture();
        surfaceTexture.setDefaultBufferSize(size.getWidth(), size.getHeight());
        Surface surface = new Surface(surfaceTexture);
        try {
            cameraDevice.createCaptureSession(Collections.singletonList(surface), new CameraCaptureSession.StateCallback() {
                @Override
                public void onConfigured(@NonNull CameraCaptureSession session) {
                    try {
                        CaptureRequest.Builder captureRequestBuilder = cameraDevice.createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW);
                        session.setRepeatingRequest(captureRequestBuilder.build(), null, backgroundHandler);

                    } catch (CameraAccessException e) {
                        e.printStackTrace();
                    }
                }

                @Override
                public void onConfigureFailed(@NonNull CameraCaptureSession session) {

                }
            }, backgroundHandler);
        } catch (CameraAccessException e1) {
            e1.printStackTrace();
        }
    }

I'm new to the Camera2 API so I have no idea what the error means. I looked around on the internet but I could not find a similar issue.

I would also like to know how to get the previewed frame in a bitmap or in an nv21 byte array, so I can do some image processing on it and display the processed image as a bitmap (or is there a way to directly apply some image processing to the previewed frame)? Thank you in advance.

Answer 1

Take a look at the Camera2Basic sample to help out with the general flow of things.

In this particular case - a capture session can have multiple target output Surfaces (one for preview, one for still capture JPEGs, for example). Each capture request can target any subset of those outputs (so the repeating request targets the preview output, while still captures target the JPEG output, etc).

So your capture request needs to have one or more Surfaces added as targets, and the Surfaces have to be one of the ones included in the session creation.

Since you only have one Surface in your session, just add

 captureRequestBuilder.addTarget(surface)

before the setRepeatingRequest call.

If you want to get a Java ByteBuffer of YUV data, you can add an ImageReader as an output target as well, for both the session and the capture request. Note that only the YUV_420_888 flexible YUV format is guaranteed to be supported, so you'll have to manually adjust it if you really need something in an NV21 layout (on any given device, YUV_420_888 may be NV21 under the hood, but there's no guarantee about that).