How to mirror iOS screen via USB?

Question

I'm trying to mirror iOS device screen via USB connection to OSX. QuickTime does this fine, and I read this article with a code example: https://nadavrub.wordpress.com/2015/07/06/macos-media-capture-using-coremediaio/

However, the callback of CMIOStreamCopyBufferQueue is never called and I'm wondering what am I doing wrong?

Have anyone faced this issue and can provide a working example ?

Thanks.

Answer 1

Well.. eventually I did what Nadav told me in his blog - discover DAL devices and capture their output using AVCaptureSession like this:

-(id) init {

    // Allow iOS Devices Discovery
    CMIOObjectPropertyAddress prop =
    { kCMIOHardwarePropertyAllowScreenCaptureDevices,
        kCMIOObjectPropertyScopeGlobal,
        kCMIOObjectPropertyElementMaster };
    UInt32 allow = 1;
    CMIOObjectSetPropertyData( kCMIOObjectSystemObject,
                              &prop, 0, NULL,
                              sizeof(allow), &allow );

    // Get devices
    NSArray *devices = [AVCaptureDevice devicesWithMediaType:AVMediaTypeMuxed];
    BOOL deviceAttahced = false;
    for (int i = 0; i < [devices count]; i++) {
        AVCaptureDevice *device = devices[i];
        if ([[device uniqueID] isEqualToString:/*deviceUDID*/]) {
            deviceAttahced = true;
            [self startSession:device];
            break;
        }
    }

    NSNotificationCenter *notificationCenter = [NSNotificationCenter defaultCenter];

    // Device not attached - subscribe to onConnect notifications
    if (!deviceAttahced) {


        id deviceWasConnectedObserver = [notificationCenter addObserverForName:AVCaptureDeviceWasConnectedNotification
                                                                        object:nil
                                                                         queue:[NSOperationQueue mainQueue]
                                                                    usingBlock:^(NSNotification *note) {
                                                                        AVCaptureDevice *device = note.object;
                                                                        [self deviceConnected:device];
                                                                    }];

        observers = [[NSArray alloc] initWithObjects:deviceWasConnectedObserver, nil];
    }

    return self;
}

- (void) deviceConnected:(AVCaptureDevice *)device {
    if ([[device uniqueID] isEqualToString:/*deviceUDID*/]) {
        [self startSession:device];
    }
}

- (void) startSession:(AVCaptureDevice *)device {

    // Init capturing session
    session = [[AVCaptureSession alloc] init];

    // Star session configuration
    [session beginConfiguration];

    // Add session input
    NSError *error;
    newVideoDeviceInput = [AVCaptureDeviceInput deviceInputWithDevice:device error:&error];
    if (newVideoDeviceInput == nil) {
        dispatch_async(dispatch_get_main_queue(), ^(void) {
            NSLog(@"%@", error);
        });
    } else {
        [session addInput:newVideoDeviceInput];
    }  

    // Add session output
    videoDataOutput = [[AVCaptureVideoDataOutput alloc] init];
    videoDataOutput.videoSettings = [NSDictionary dictionaryWithObject: [NSNumber numberWithInt:kCVPixelFormatType_32BGRA] forKey: (id)kCVPixelBufferPixelFormatTypeKey];

    dispatch_queue_t videoQueue = dispatch_queue_create("videoQueue", NULL);

    [videoDataOutput setSampleBufferDelegate:self queue:videoQueue];
    [session addOutput:videoDataOutput];

    // Finish session configuration
    [session commitConfiguration];

    // Start the session
    [session startRunning];
}

#pragma mark - AVCaptureAudioDataOutputSampleBufferDelegate

- (void)captureOutput:(AVCaptureOutput *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnection *)connection {
    NSImage *resultNSImage = [self imageFromSampleBuffer:sampleBuffer];

    /*
     * Here you can do whatever you need with the frame (e.g. convert to JPG)
     */
}