cv：：垫子不匹配UIImageView宽度？

Question

我使用AVFoundation捕获视频帧，用opencv处理并在新iPad上的UIImageView中显示结果。opencv进程执行以下操作("inImg"是视频帧):

cv::Mat testROI = inImg.rowRange(0,100);
testROI = testROI.colRange(0,10);
testROI.setTo(255); // this is a BGRA frame.

然而，在框架的左上角，我得到的不是一个垂直的白色条(100行× 10 col)，而是从右上角到左下角的100条阶梯状的水平线，每条线长10像素。

经过一番调查，我意识到显示帧的宽度似乎比cv::Mat宽8个像素。(即第二行的第9个像素就在第1行的第1个像素的正下方。)

视频帧本身正确显示(行与行之间没有位移)。问题出现在AVCaptureSession。sessionPreset是AVCaptureSessionPresetMedium(帧行=480,cols=360)，但当它是AVCaptureSessionPresetHigh(帧行=640,cols=480)时不出现。

全屏显示360种颜色。(我尝试逐像素遍历和修改cv::Mat。像素1-360显示正确。361-368消失，369显示在像素1的正下方)。

我尝试了imageview的组合。contentMode (UIViewContentModeScaleAspectFill和UIViewContentModeScaleAspectFit)和imageview。clipstound (YES/NO)，但没有运气。

有什么问题吗?非常感谢。

我使用以下代码创建AVCaptureSession:

NSArray* devices = [AVCaptureDevice devicesWithMediaType:AVMediaTypeVideo];
if ([devices count] == 0) {
    NSLog(@"No video capture devices found");
    return NO;
}

for (AVCaptureDevice *device in devices) {
     if ([device position] == AVCaptureDevicePositionFront) {
           _captureDevice = device;
     }
}

NSError* error_exp = nil;
if ([_captureDevice lockForConfiguration:&error_exp]) {
    [_captureDevice setWhiteBalanceMode:AVCaptureWhiteBalanceModeContinuousAutoWhiteBalance];
    [_captureDevice unlockForConfiguration];
}
// Create the capture session
_captureSession = [[AVCaptureSession alloc] init];
_captureSession.sessionPreset = AVCaptureSessionPresetMedium;

// Create device input
NSError *error = nil;
AVCaptureDeviceInput *input = [[AVCaptureDeviceInput alloc] initWithDevice:_captureDevice error:&error];
// Create and configure device output
_videoOutput = [[AVCaptureVideoDataOutput alloc] init];
dispatch_queue_t queue = dispatch_queue_create("cameraQueue", NULL); 
[_videoOutput setSampleBufferDelegate:self queue:queue];
dispatch_release(queue); 
_videoOutput.alwaysDiscardsLateVideoFrames = YES; 
OSType format = kCVPixelFormatType_32BGRA;
_videoOutput.videoSettings = [NSDictionary dictionaryWithObject:[NSNumber numberWithUnsignedInt:format]forKey:(id)kCVPixelBufferPixelFormatTypeKey];

// Connect up inputs and outputs
if ([_captureSession canAddInput:input]) {
    [_captureSession addInput:input];
}
if ([_captureSession canAddOutput:_videoOutput]) {
    [_captureSession addOutput:_videoOutput];
}
AVCaptureConnection * captureConnection = [_videoOutput connectionWithMediaType:AVMediaTypeVideo];
if (captureConnection.isVideoMinFrameDurationSupported)
    captureConnection.videoMinFrameDuration = CMTimeMake(1, 60);
if (captureConnection.isVideoMaxFrameDurationSupported)
    captureConnection.videoMaxFrameDuration = CMTimeMake(1, 60);
if (captureConnection.supportsVideoMirroring)
    [captureConnection setVideoMirrored:NO];
[captureConnection setVideoOrientation:AVCaptureVideoOrientationPortraitUpsideDown];

当接收到一个帧时，调用以下命令:

- (void)captureOutput:(AVCaptureOutput *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnection *)connection
{
@autoreleasepool {
    CVPixelBufferRef pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer);
    OSType format = CVPixelBufferGetPixelFormatType(pixelBuffer);
    CGRect videoRect = CGRectMake(0.0f, 0.0f, CVPixelBufferGetWidth(pixelBuffer), CVPixelBufferGetHeight(pixelBuffer));
    AVCaptureConnection *currentConnection = [[_videoOutput connections] objectAtIndex:0];
    AVCaptureVideoOrientation videoOrientation = [currentConnection videoOrientation];
    CGImageRef quartzImage;
    // For color mode a 4-channel cv::Mat is created from the BGRA data
    CVPixelBufferLockBaseAddress(pixelBuffer, 0);
    void *baseaddress = CVPixelBufferGetBaseAddress(pixelBuffer);
    cv::Mat mat(videoRect.size.height, videoRect.size.width, CV_8UC4, baseaddress, 0);
    if ([self doFrame]) { // a flag to switch processing ON/OFF
            [self processFrame:mat videoRect:videoRect videoOrientation:videoOrientation];  // "processFrame" is the opencv function shown above
    }
    CIImage *ciImage = [CIImage imageWithCVPixelBuffer:pixelBuffer];
    quartzImage = [self.context createCGImage:ciImage fromRect:ciImage.extent];
    CVPixelBufferUnlockBaseAddress(pixelBuffer, 0);
    UIImage *image = [UIImage imageWithCGImage:quartzImage scale:1.0 orientation:UIImageOrientationUp];
    CGImageRelease(quartzImage);
    [self.imageView performSelectorOnMainThread:@selector(setImage:) withObject:image waitUntilDone:YES];

Answer 1

我假设您正在使用构造函数Mat(int _rows, int _cols, int _type, void* _data, size_t _step=AUTO_STEP), AUTO_STEP为0，并假设行步长为width*bytesPerPixel。

这通常是错误的—非常将行对齐到一些更大的边界是常见的。在这种情况下，360不是16的倍数，但368是;这强烈表明它正在对齐到16像素的边界(也许是为了帮助算法在16×16块中处理?)。

试

cv::Mat mat(videoRect.size.height, videoRect.size.width, CV_8UC4, baseaddress, CVPixelBufferGetBytesPerRow(pixelBuffer));