Use SceneKit to render an CVOpenGLTextureRef

Question

I am using AVCaptureSession, and outputting a CVOpenGLTextureRef at 60fps, which I need to display using SceneKit.

I'm fairly sure I should be using the SCNProgramDelegate method bindValueForSymbol: but can't quite figure out the mechanism to bind the texture ref. The examples I have seen use a GLKTextureInfo object, and bind its name property with the call to glBindTexture().

What is the equivalent mechanism to do this with an CVOpenGLTextureRef?

Answer 1

A relatively easy solution may be to use AVCaptureVideoPreviewLayer. You can set it directly as the contents of the diffuse material of the geometry you want to display the frames on, and you get the benefits of all the optimisation work that Apple have (probably) done for this. AVFoundation and SceneKit will handle all the details for you.

If you are doing work on the CPU to the incoming buffer, there's also the lower level AVSampleBufferDisplayLayer class, which lets you pass CMSampleBufferRef objects, which you can build from a CVPixelBufferRef. It's a bit more difficult to feed and care for, but not impossible.