How can I display a negative mode preview with CameraX in Android?

Question

I'm learning CameraX API, and CameraXBasic is a office sample code.

CameraFragment.kt in CameraXBasic project displays a real camera preview.

Now I hope to display a negative mode preview. How can I do with CameraX API? Is there a sample code?

CameraFragment.kt

private lateinit var viewFinder: TextureView

private fun bindCameraUseCases() {
    // Get screen metrics used to setup camera for full screen resolution
    val metrics = DisplayMetrics().also { viewFinder.display.getRealMetrics(it) }
    val screenAspectRatio = Rational(metrics.widthPixels, metrics.heightPixels)
    Log.d(TAG, "Screen metrics: ${metrics.widthPixels} x ${metrics.heightPixels}")

    // Set up the view finder use case to display camera preview
    val viewFinderConfig = PreviewConfig.Builder().apply {
        setLensFacing(lensFacing)
        // We request aspect ratio but no resolution to let CameraX optimize our use cases
        setTargetAspectRatio(screenAspectRatio)
        // Set initial target rotation, we will have to call this again if rotation changes
        // during the lifecycle of this use case
        setTargetRotation(viewFinder.display.rotation)
    }.build()

    // Use the auto-fit preview builder to automatically handle size and orientation changes
    preview = AutoFitPreviewBuilder.build(viewFinderConfig, viewFinder)

 ....

 CameraX.bindToLifecycle(
            viewLifecycleOwner, preview, imageCapture, imageAnalyzer)
}

Answer 1

This is an android.media.effect.Effect being applied to the SurfaceTexture (the PreviewConfig does not have any such properties). See EffectFactory.EFFECT_NEGATIVE:

/**
 * Applies negative film effect on image.<br/>
 * Parameters: scale (float): the degree of film grain.
**/
public final static String EFFECT_NEGATIVE = "android.media.effect.effects.NegativeEffect";

Here it is also explained (the Kotlin and Java documentation are differently organized).

However, you might need to use your own SurfaceTexture, because it else it is difficult to get the GLContext. This example is almost working, but the texture.attachToGLContext():

private EffectContext fxContext = null;
private EffectFactory fxFactory = null;
private Effect fx = null;

protected void applyNegativeEffect(SurfaceTexture texture, int width, int height) {
    if(this.fxContext == null) {
        // texture.attachToGLContext(texture.mSurfaceTexture);
        this.fxContext = EffectContext.createWithCurrentGlContext();
    }
    if(this.fxFactory == null) {
        this.fxFactory = this.fxContext.getFactory();
    }
    if(this.fx == null) {
        this.fx = fxFactory.createEffect(EffectFactory.EFFECT_NEGATIVE);
        this.fx.setParameter("scale", 1.0f);
        // this.fx.apply(0, width, height, 0);
    }
}

But SurfaceTexture (where long mSurfaceTexture would need to be exposed) reads:

/**
 * These fields are used by native code, do not access or modify.
**/
private long mSurfaceTexture;

---

Setting a shader on a GLSurfaceView might be easier to accomplish:

String shader = "#extension GL_OES_EGL_image_external : require\n"
    + "precision mediump float;\n"
    + "varying vec2 vTextureCoord;\n"
    + "uniform samplerExternalOES sTexture;\n"
    + "void main() {\n"
    + "  vec4 color = texture2D(sTexture, vTextureCoord);\n"
    + "  float colorR = (1.0 - color.r) / 1.0;\n"
    + "  float colorG = (1.0 - color.g) / 1.0;\n"
    + "  float colorB = (1.0 - color.b) / 1.0;\n"
    + "  gl_FragColor = vec4(colorR, colorG, colorB, color.a);\n"
    + "}\n";

And a camera2 example for OpenGL ES preview.

Getting the GLContext from CameraX abstraction is the actual problem, because one would need to build androidx.camera.core with a custom OpenGL preview, in order to apply the shader - or to get the input/ouput texId.

Update:

Implement a preview reads:

If you require direct access to the SurfaceTexture, such as to perform OpenGL rendering, see Manually create a SurfaceTexture. In these cases, pass in a SurfaceTexture to a Preview object using Preview.PreviewSurfaceProvider.

See issue issue #146957342, Futures.immediateFuture(surface) doesn't work anymore.