Converting CMSampleBuffer to CVMetalTexture in Swift with CVMetalTextureCacheCreateTextureFromImage

Question

I'm trying to get a simple render-camera-output-to-metal-layer pipeline going, and it works well enough in Objective-C (There's the MetalVideoCapture sample app) but there seems to be some formatting weirdness when I try to translate that to swift. My ultrasimple capture buffer looks like this (ignore the lack of sanitization...)

    func captureOutput(captureOutput: AVCaptureOutput!, didOutputSampleBuffer sampleBuffer: CMSampleBuffer!, fromConnection connection: AVCaptureConnection!) {
    var error: CVReturn! = nil
    let sourceImageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer)
    let width = CVPixelBufferGetWidth(sourceImageBuffer!)
    let height = CVPixelBufferGetHeight(sourceImageBuffer!)
    var outTexture: CVMetalTextureRef? = nil

    error  = CVMetalTextureCacheCreateTextureFromImage(kCFAllocatorDefault, videoTextureCache!, sourceImageBuffer!, nil, MTLPixelFormat.BGRA8Unorm, width, height, 0, &outTexture!)

    if error != nil {
        print("Error! \(error)")
    }

    let videoTexture = CVMetalTextureGetTexture(outTexture!)
    self.imageTexture = videoTexture!
}

Where videoTextureCache is var videoTextureCache: CVMetalTextureCache? = nil

But it gives me Cannot invoke 'CVMetalTextureCacheCreateTextureFromImage' with an argument list of type '(CFAllocator!, CVMetalTextureCache, CVImageBuffer, nil, MTLPixelFormat, Int, Int, Int, inout CVMetalTextureRef)'

The thing is, if I replace outTexture with nil it stops throwing the error but clearly that's not going to help me. According to the reference for the function I need UnsafeMutablePointer?> for that last value. Which I'm not sure how to get.

Answer 1

Try to allocate the your the textureCache in advance, here is what I use as the member variable:

var _videoTextureCache : Unmanaged<CVMetalTextureCacheRef>?

And then I allocate the textureCache in an initialization method via

CVMetalTextureCacheCreate(kCFAllocatorDefault, nil, _context.device, nil, &_videoTextureCache)

where _context.device is the MTLDevice. Then, in the captureOutput method, I use the following (beware, no error checking included here)

var textureRef : Unmanaged<CVMetalTextureRef>?
CVMetalTextureCacheCreateTextureFromImage(kCFAllocatorDefault, _videoTextureCache!.takeUnretainedValue(), imageBuffer, nil, MTLPixelFormat.BGRA8Unorm, width, height, 0, &textureRef)

I hope this helps!