Chroma-Filtering video using GPUImage?

Question

I am attempting to display a video file with transparency inside my application using a transparency key (RGB: 0x00FF00, or full green) using @BradLarson's awesome GPUImage toolkit. However, I am experiencing some difficulties with the GPUImageChromaKeyFilter filter, and I don't quite understand why.

My source video file is available in my dropbox here (12 KB, 3 seconds long, full green background, just a square on the screen),

And I used the sample project titled SimpleVideoFilter.

This is the code I attempted to use (I simply replaced -viewDidLoad):

NSURL *sampleURL = [[NSBundle mainBundle] URLForResource:@"sample" withExtension:@"m4v"];
movieFile = [[GPUImageMovie alloc] initWithURL:sampleURL];

filter = [[GPUImageChromaKeyFilter alloc] init];
[filter setColorToReplaceRed:0 green:1 blue:0];
[filter setEnabled:YES];

[movieFile addTarget:filter];

GPUImageView *filterView = (GPUImageView *)self.view;
[filter addTarget:filterView];

[movieFile startProcessing];

According to the documentation (which is sparse), this should have the effect of replacing all of the green in the video. Instead, I get this as an output:

Which tells me that the video is playing (and thus it's copying to the application), but it doesn't seem to be doing any chroma keying. Why would this be? Do I need to manually set smoothing values & thresholds? I shouldn't, because the source only contains two colors (0x00FF00 and 0x000000).

I have tested this on the device as well, to no avail. Almost all other filters I attempt to use work, such as GPUImageRGBFilter, GPUImageSepiaFilter, etc. Could GPUImageChromaKeyFilter just be broken?

Any help with this would be appreciated, as at this point I'm scraping the bottom of the barrel for transparency on a video.

Answer 1

Similar to the original question, I wanted to put a green-screen video on top of a custom view hierarchy incl. live video. Turned out this was not possible with the standard GPUImage ChromaKey filter(s). Instead of alpha blending, it blended the green pixels with the background pixels. For example a red background became yellow and blue became cyan.

The way to get it working involves two steps:

1) make sure the filterview has a transparent background:

filterView.backgroundColor=[UIColor clearColor];

2) Modify GPUImageChromaKeyFilter.m

old: gl_FragColor = vec4(textureColor.rgb, textureColor.a * blendValue);
new: gl_FragColor = vec4(textureColor.rgb * blendValue, 1.0 * blendValue);

Now all keyed (for example green) pixels in the video become transparent and uncover whatever is below the filterview, incl. (live-)video.

Answer 2

I got it to work by using GPUImageChromaKeyBlendFilter and setting a background image. I'd like to know though if we can do without having to set a background. When I don't set the background, the movie shows as white, but adding the background renders fine...

filter = [[GPUImageChromaKeyBlendFilter alloc] init];
[(GPUImageChromaKeyBlendFilter *)filter setColorToReplaceRed:0.0 green:1.0 blue:0.0];
[(GPUImageChromaKeyBlendFilter *)filter setThresholdSensitivity:0.4];

UIImage *inputImage = [UIImage imageNamed:@"background.png"];
sourcePicture = [[GPUImagePicture alloc] initWithImage:inputImage smoothlyScaleOutput:YES];
[sourcePicture addTarget:filter];
[sourcePicture processImage];