Bad performance ML Kit barcode scanning

Question

I'm using Googles ML Kit for barcode scanning, and gathered the code below from the examples and tutorials provided by Google. However, the performance is dramatic; it takes several seconds, can be 10, 15 seconds, to recognize a barcode. Is there any way to improve this?

Also, how can this be used with inverted bar codes? I found that I need to invert the image, however, if in the Analyzer I try to get image.bitmapInternal or image.byteBuffer, it is always null.

Build.gradle

 implementation 'com.google.mlkit:barcode-scanning:17.0.0'
 // CameraX core library using camera2 implementation
implementation "androidx.camera:camera-camera2:1.0.2"
// CameraX Lifecycle Library
implementation "androidx.camera:camera-lifecycle:1.0.2"
// CameraX View class
implementation "androidx.camera:camera-view:1.0.0-alpha31"

Then, in a fragment:

typealias BarCodeListener = (barCode: String) -> Unit
const val TAG = "ConnectorScanner"
[...]

override fun onResume() {
    super.onResume()
    cameraExecutor = Executors.newSingleThreadExecutor()
    startCamera()
}

private fun startCamera() {
    val cameraProviderFuture = ProcessCameraProvider.getInstance(requireContext())
    val resolution = Size(720, 1280)
    cameraProviderFuture.addListener({
        // Used to bind the lifecycle of cameras to the lifecycle owner
        val cameraProvider: ProcessCameraProvider = cameraProviderFuture.get()

        // Preview
        val preview = Preview.Builder()
            .setTargetResolution(resolution)
            .build()
            .also {
                it.setSurfaceProvider(binding.viewFinder.surfaceProvider)
            }

        imageCapture = ImageCapture.Builder()
            .setTargetResolution(resolution)
            .setCaptureMode(ImageCapture.CAPTURE_MODE_MAXIMIZE_QUALITY)
            .build()

        val imageAnalyzer = ImageAnalysis.Builder()
            .setBackpressureStrategy(ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST)
            .build()
            .also {
                it.setAnalyzer(cameraExecutor, BarCodeAnalyzer { barCode ->
                    if (BuildConfig.DEBUG) {
                        Toast.makeText(context, "Code: $barCode", Toast.LENGTH_LONG).show()
                    }
                    viewModel.onConnectorCodeScanned(barCode)
                    cameraProvider.unbindAll()
                })
            }

        // Select back camera as a default
        val cameraSelector = CameraSelector.DEFAULT_BACK_CAMERA

        try {
            // Unbind use cases before rebinding
            cameraProvider.unbindAll()

            // Bind use cases to camera
            cameraProvider.bindToLifecycle(
                this, cameraSelector, preview, imageCapture, imageAnalyzer
            )

        } catch (exc: Exception) {
            Log.e(TAG, "Use case binding failed", exc)
            showErrorDialog()
        }

    }, ContextCompat.getMainExecutor(requireContext()))
}

Image analyzer

private class BarCodeAnalyzer(private val listener: BarCodeListener) : ImageAnalysis.Analyzer {
    val options = BarcodeScannerOptions.Builder()
        .setBarcodeFormats(
            Barcode.FORMAT_DATA_MATRIX
        )
        .build()

    @SuppressLint("UnsafeOptInUsageError")
    override fun analyze(imageProxy: ImageProxy) {
        val mediaImage = imageProxy.image
        mediaImage?.let {
            val image =
                InputImage.fromMediaImage(it, imageProxy.imageInfo.rotationDegrees)
                val scanner = BarcodeScanning.getClient(options)

            scanner.process(image)
                .addOnSuccessListener { barcodes ->
                    if (barcodes.isNotEmpty()) {
                        barcodes.firstOrNull()?.rawValue?.let { barcode ->
                            Log.d(TAG, barcode)
                            listener(barcode)
                        }
                        imageProxy.close()
                    }
                }
        }
        imageProxy.close()
    }
}

Answer 1

With some luck, I found the solution to the performance issue, it's adding an OnCompleteListener and closing the images there. So the analyzer will be

scanner.process(image)
                .addOnSuccessListener { barcodes ->
                    if (barcodes.isNotEmpty()) {
                        barcodes.firstOrNull()?.rawValue?.let { barcode ->
                            Log.d(TAG, barcode)
                                listener(barcode)
                        }
                    }
                }
                .addOnCompleteListener {
                    imageProxy.close()
                }
        }

Now the scanning of the barcode is lightning fast!