Tensorflow's app Camera Images are Blurry

Question

What I'm trying to achieve:

What I achieve w/ my tensorflow app:

Background: I'm using TensorFlow lite application that works badly because the images that are produced w/ the flash on are blurry and many similar white objects can't be distinguished. (The images are taken in a sort of a cup, which is a closed environment, this is why the flash is always on)

If you want to reproduce the project, you can follow the instructions below:

Set up the working directory

git clone https://github.com/tensorflow/examples.git

Open the project with Android Studio

Open a project with Android Studio by taking the following steps:

1. Open Android Studio. After it loads select " Open an existing Android Studio project".

2. In the file selector, choose examples/lite/examples/image_classification/android from your working directory to load the project.

In LegacyCameraConnectionFragment.java, in the function “onSurfaceTextureAvailable “

Add in line 90~ this code is added to turn on the flash all the time (it's off by default):

List<String> flashModes = parameters.getSupportedFlashModes();
if (flashModes.contains(android.hardware.Camera.Parameters.FLASH_MODE_TORCH))
{
  parameters.setFlashMode(parameters.FLASH_MODE_TORCH);
}

More info. about installation can be found here: https://www.tensorflow.org/lite/models/image_classification/overview#example_applications_and_guides

What I tried:

First, I tried tweaking the camera parameters:

            previewRequestBuilder.set(
                CaptureRequest.CONTROL_AF_MODE,
                CaptureRequest.CONTROL_AF_MODE_MACRO);

            previewRequestBuilder.set(CaptureRequest.FLASH_MODE,
                    CaptureRequest.FLASH_MODE_TORCH);

        parameters.setFlashMode(parameters.FLASH_MODE_TORCH);
        parameters.setFocusMode(Camera.Parameters.FOCUS_MODE_MACRO);
        parameters.set("iso","100");
        parameters.setJpegQuality(100);

Then,

I've tried to implement autofocus w/ the following code (Which seems to do some focus, but still the image almost stays the same)

 private Camera.AutoFocusCallback myAutoFocusCallback = new Camera.AutoFocusCallback() {

    @Override
    public void onAutoFocus(boolean arg0, Camera arg1) {
      if (arg0) {
        LegacyCameraConnectionFragment.camera.cancelAutoFocus();
      }
    }
  };

  public void doTouchFocus(final Rect tfocusRect) {
    try {
      Camera mCamera = LegacyCameraConnectionFragment.camera;
      List<Camera.Area> focusList = new ArrayList<Camera.Area>();
      Camera.Area focusArea = new Camera.Area(tfocusRect, 1000);
      focusList.add(focusArea);

      Camera.Parameters param = mCamera.getParameters();
      param.setFocusAreas(focusList);
      param.setMeteringAreas(focusList);
      mCamera.setParameters(param);

      mCamera.autoFocus(myAutoFocusCallback);
    } catch (Exception e) {
      e.printStackTrace();
    }

  }

  @Override
  public boolean onTouchEvent(MotionEvent event) {

    if (event.getAction() == MotionEvent.ACTION_DOWN) {
      float x = event.getX();
      float y = event.getY();
  Rect touchRect = new Rect(
          (int) (x - 100),
          (int) (y - 100),
          (int) (x + 100),
          (int) (y + 100));


  final Rect targetFocusRect = new Rect(
          touchRect.left * 2000 / previewWidth - 1000,
          touchRect.top * 2000 / previewHeight - 1000,
          touchRect.right * 2000 / previewWidth - 1000,
          touchRect.bottom * 2000 / previewHeight - 1000);

  doTouchFocus(targetFocusRect);
}

return false;

}

Third, I tried checking some repos: First repo is Camera2Basic

https://github.com/googlearchive/android-Camera2Basic

This repo produces the same bad results.

Then I tried OpenCamera's source code which can be found in source-forge.net:

https://sourceforge.net/projects/opencamera/files/test_20200301/

And the app produces really good results, but after a few days I still couldn't figure out which part should I take from there to make this work. I believe it has to do w/ the focus, but I wasn't able to understand how to take the code from there.

I also watched some YouTube videos and went over 10 posts here about Android's Studio Camera API v1 and v2 and tried to fix it on my own.

I've no idea how to continue, any ideas are highly appreciated.

Answer 1

You are using a very old implementation of the Camera API in LegacyCameraConnectionFragment, which is deprecated. You should use android.hardware.camera2, which is being used in the other tensorflow example, CameraConnectionFragment. Recently, CameraX was released to Beta, and there would probably be fewer examples for you to follow online, but some people are enjoying it already. More info about cameraX here.

Looks like CameraConnectionFragment.java is already using optimal settings for you?

// Auto focus should be continuous for camera preview.
previewRequestBuilder.set(
    CaptureRequest.CONTROL_AF_MODE,
    CaptureRequest.CONTROL_AF_MODE_CONTINUOUS_PICTURE);
    // Flash is automatically enabled when necessary.
    previewRequestBuilder.set(
       CaptureRequest.CONTROL_AE_MODE, CaptureRequest.CONTROL_AE_MODE_ON_AUTO_FLASH);

---

PS: I don't think you should hard-code or even 'fine-tune' your camera settings so that it only works in very small cases. Let the API do the work for you.