如何使用AVAssetReader在iOS上正确读取解码的PCM样本——当前解码不正确

Question

我目前正在处理一个应用程序，作为我计算机科学学士学位的一部分。该应用程序将关联来自iPhone硬件（加速度计、gps）的数据和正在播放的音乐。

该项目仍处于起步阶段，只进行了2个月的工作。

我现在需要帮助的是从itunes库中读取歌曲的PCM样本，并使用和音频单元播放。目前，我想工作的实现如下：从iTunes中选择一首随机歌曲，并在需要时从中读取样本，然后存储在缓冲区中，让我们称之为sampleBuffer。稍后在消费者模型中，音频单元（它有一个混合器和一个远程IO输出）有一个回调，我只需将所需数量的样本从sampleBuffer复制到回调中指定的缓冲区中。然后，我通过扬声器听到的东西与我所期望的不太一样；我能认出它在播放这首歌，但它似乎解码错误，而且噪音很大！我附上了一张图像，显示了前半秒（24576个样本@44.1kHz），这与正常的输出不相似。在我进入列表之前，我已经检查了文件是否损坏，类似地，我已经为缓冲区编写了测试用例（所以我知道缓冲区不会更改样本），尽管这可能不是最好的方法（有些人会争辩说要走音频队列路线），但我想对样本执行各种操作，并在歌曲完成之前更改歌曲，重新安排播放的歌曲等等。此外，音频单元中可能有一些不正确的设置，然而，显示样本的图形（显示样本解码不正确）是直接从缓冲区中获取的，因此我现在只想解决为什么从磁盘读取和解码不正确。现在我只是想通过工作来演一出戏。无法发布图片，因为stackoverflow是新手，所以这里有图片链接：https://i.stack.imgur.com/RHjlv.jpg

上市：

这是我设置将用于AVAssetReaderAudioMixOutput 的audioReadSetigns的地方

// Set the read settings
    audioReadSettings = [[NSMutableDictionary alloc] init];
    [audioReadSettings setValue:[NSNumber numberWithInt:kAudioFormatLinearPCM]
                         forKey:AVFormatIDKey];
    [audioReadSettings setValue:[NSNumber numberWithInt:16] forKey:AVLinearPCMBitDepthKey];
    [audioReadSettings setValue:[NSNumber numberWithBool:NO] forKey:AVLinearPCMIsBigEndianKey];
    [audioReadSettings setValue:[NSNumber numberWithBool:NO] forKey:AVLinearPCMIsFloatKey];
    [audioReadSettings setValue:[NSNumber numberWithBool:NO] forKey:AVLinearPCMIsNonInterleaved];
    [audioReadSettings setValue:[NSNumber numberWithFloat:44100.0] forKey:AVSampleRateKey];

现在，下面的代码列表是一个接收带有歌曲的persistent_id的NSString的方法：

-(BOOL)setNextSongID:(NSString*)persistand_id {
assert(persistand_id != nil);
MPMediaItem *song = [self getMediaItemForPersistantID:persistand_id];
NSURL *assetUrl = [song valueForProperty:MPMediaItemPropertyAssetURL];
AVURLAsset *songAsset = [AVURLAsset URLAssetWithURL:assetUrl 
                                            options:[NSDictionary dictionaryWithObject:[NSNumber numberWithBool:YES] 
                                                                                forKey:AVURLAssetPreferPreciseDurationAndTimingKey]];

NSError *assetError = nil;
assetReader = [[AVAssetReader assetReaderWithAsset:songAsset error:&assetError] retain];
if (assetError) {
    NSLog(@"error: %@", assetError);
    return NO;
}
CMTimeRange timeRange = CMTimeRangeMake(kCMTimeZero, songAsset.duration);
[assetReader setTimeRange:timeRange];
track = [[songAsset tracksWithMediaType:AVMediaTypeAudio] objectAtIndex:0];
assetReaderOutput = [AVAssetReaderAudioMixOutput assetReaderAudioMixOutputWithAudioTracks:[NSArray arrayWithObject:track]
                                                                            audioSettings:audioReadSettings];
if (![assetReader canAddOutput:assetReaderOutput]) {
    NSLog(@"cant add reader output... die!");
    return NO;
}
[assetReader addOutput:assetReaderOutput];
[assetReader startReading];
// just getting some basic information about the track to print
NSArray *formatDesc = ((AVAssetTrack*)[[assetReaderOutput audioTracks] objectAtIndex:0]).formatDescriptions;
for (unsigned int i = 0; i < [formatDesc count]; ++i) {
    CMAudioFormatDescriptionRef item = (CMAudioFormatDescriptionRef)[formatDesc objectAtIndex:i];
    const CAStreamBasicDescription *asDesc = (CAStreamBasicDescription*)CMAudioFormatDescriptionGetStreamBasicDescription(item);
    if (asDesc) {
        // get data
        numChannels = asDesc->mChannelsPerFrame;
        sampleRate = asDesc->mSampleRate;
        asDesc->Print();
    }
}
[self copyEnoughSamplesToBufferForLength:24000];
return YES;
}

以下是函数-（void）copyEnoughSamplesToBufferForLength:

-(void)copyEnoughSamplesToBufferForLength:(UInt32)samples_count {
[w_lock lock];
int stillToCopy = 0;
if (sampleBuffer->numSamples() < samples_count) {
    stillToCopy = samples_count;
}
NSAutoreleasePool *apool = [[NSAutoreleasePool alloc] init];

CMSampleBufferRef sampleBufferRef;
SInt16 *dataBuffer = (SInt16*)malloc(8192 * sizeof(SInt16));
int a = 0;
while (stillToCopy > 0) {
    sampleBufferRef = [assetReaderOutput copyNextSampleBuffer];
    if (!sampleBufferRef) {
        // end of song or no more samples
        return;
    }
    CMBlockBufferRef blockBuffer = CMSampleBufferGetDataBuffer(sampleBufferRef);
    CMItemCount numSamplesInBuffer = CMSampleBufferGetNumSamples(sampleBufferRef);
    AudioBufferList audioBufferList;
    CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer(sampleBufferRef,
                                                            NULL,
                                                            &audioBufferList,
                                                            sizeof(audioBufferList),
                                                            NULL,
                                                            NULL,
                                                            0,
                                                            &blockBuffer);
    int data_length = floorf(numSamplesInBuffer * 1.0f);
    int j = 0;
    for (int bufferCount=0; bufferCount < audioBufferList.mNumberBuffers; bufferCount++) {
        SInt16* samples = (SInt16 *)audioBufferList.mBuffers[bufferCount].mData;
        for (int i=0; i < numSamplesInBuffer; i++) {
            dataBuffer[j] = samples[i];
            j++;
        }
    }
    CFRelease(sampleBufferRef);
    sampleBuffer->putSamples(dataBuffer, j);
    stillToCopy = stillToCopy - data_length;
}
free(dataBuffer);
[w_lock unlock];
[apool release];
}

现在sampleBuffer将具有错误解码的样本。有人能帮我为什么会这样吗？这种情况发生在我的iTunes库中的不同文件（mp3、aac、wav等）上。任何帮助都将不胜感激，此外，如果您需要我的代码的任何其他列表，或者输出听起来像什么，我会根据请求附上它。在过去的一周里，我一直在尝试调试它，但在网上没有找到任何帮助——每个人似乎都在以我的方式进行设计，但似乎只有我有这个问题。

谢谢你的帮助！

Peter

Answer 1

目前，我还在进行一个项目，该项目涉及将iTunes库中的音频样本提取到AudioUnit中。

音频单元渲染回调包括在内，供您参考。输入格式设置为SInt16StereoStreamFormat。

我使用了MichaelTyson的循环缓冲区实现——TPCircularBuffer作为缓冲存储。非常易于使用和理解！！！谢谢迈克尔！

- (void) loadBuffer:(NSURL *)assetURL_
{
    if (nil != self.iPodAssetReader) {
        [iTunesOperationQueue cancelAllOperations];
        [self cleanUpBuffer];
    }
    NSDictionary *outputSettings = [NSDictionary dictionaryWithObjectsAndKeys:
                                    [NSNumber numberWithInt:kAudioFormatLinearPCM], AVFormatIDKey, 
                                    [NSNumber numberWithFloat:44100.0], AVSampleRateKey,
                                    [NSNumber numberWithInt:16], AVLinearPCMBitDepthKey,
                                    [NSNumber numberWithBool:NO], AVLinearPCMIsNonInterleaved,
                                    [NSNumber numberWithBool:NO], AVLinearPCMIsFloatKey,
                                    [NSNumber numberWithBool:NO], AVLinearPCMIsBigEndianKey,
                                    nil];
    AVURLAsset *asset = [AVURLAsset URLAssetWithURL:assetURL_ options:nil];
    if (asset == nil) {
        NSLog(@"asset is not defined!");
        return;
    }
    NSLog(@"Total Asset Duration: %f", CMTimeGetSeconds(asset.duration));
    NSError *assetError = nil;
    self.iPodAssetReader = [AVAssetReader assetReaderWithAsset:asset error:&assetError];
    if (assetError) {
        NSLog (@"error: %@", assetError);
        return;
    }
    AVAssetReaderOutput *readerOutput = [AVAssetReaderAudioMixOutput assetReaderAudioMixOutputWithAudioTracks:asset.tracks audioSettings:outputSettings];
    if (! [iPodAssetReader canAddOutput: readerOutput]) {
        NSLog (@"can't add reader output... die!");
        return;
    }
    // add output reader to reader
    [iPodAssetReader addOutput: readerOutput];
    if (! [iPodAssetReader startReading]) {
        NSLog(@"Unable to start reading!");
        return;
    }
    // Init circular buffer
    TPCircularBufferInit(&playbackState.circularBuffer, kTotalBufferSize);
    __block NSBlockOperation * feediPodBufferOperation = [NSBlockOperation blockOperationWithBlock:^{
        while (![feediPodBufferOperation isCancelled] && iPodAssetReader.status != AVAssetReaderStatusCompleted) {
            if (iPodAssetReader.status == AVAssetReaderStatusReading) {
                // Check if the available buffer space is enough to hold at least one cycle of the sample data
                if (kTotalBufferSize - playbackState.circularBuffer.fillCount >= 32768) {
                    CMSampleBufferRef nextBuffer = [readerOutput copyNextSampleBuffer];
                    if (nextBuffer) {
                        AudioBufferList abl;
                        CMBlockBufferRef blockBuffer;
                        CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer(nextBuffer, NULL, &abl, sizeof(abl), NULL, NULL, kCMSampleBufferFlag_AudioBufferList_Assure16ByteAlignment, &blockBuffer);
                        UInt64 size = CMSampleBufferGetTotalSampleSize(nextBuffer);
                        int bytesCopied = TPCircularBufferProduceBytes(&playbackState.circularBuffer, abl.mBuffers[0].mData, size);
                        if (!playbackState.bufferIsReady && bytesCopied > 0) {
                            playbackState.bufferIsReady = YES;
                        }
                        CFRelease(nextBuffer);
                        CFRelease(blockBuffer);
                    }
                    else {
                        break;
                    }
                }
            }
        }
        NSLog(@"iPod Buffer Reading Finished");
    }];
    [iTunesOperationQueue addOperation:feediPodBufferOperation];
}
static OSStatus ipodRenderCallback (
                                     void                        *inRefCon,      // A pointer to a struct containing the complete audio data 
                                     //    to play, as well as state information such as the  
                                     //    first sample to play on this invocation of the callback.
                                     AudioUnitRenderActionFlags  *ioActionFlags, // Unused here. When generating audio, use ioActionFlags to indicate silence 
                                     //    between sounds; for silence, also memset the ioData buffers to 0.
                                     const AudioTimeStamp        *inTimeStamp,   // Unused here.
                                     UInt32                      inBusNumber,    // The mixer unit input bus that is requesting some new
                                     //        frames of audio data to play.
                                     UInt32                      inNumberFrames, // The number of frames of audio to provide to the buffer(s)
                                     //        pointed to by the ioData parameter.
                                     AudioBufferList             *ioData         // On output, the audio data to play. The callback's primary 
                                     //        responsibility is to fill the buffer(s) in the 
                                     //        AudioBufferList.
                                     ) 
{
    Audio* audioObject   = (Audio*)inRefCon;
    AudioSampleType *outSample          = (AudioSampleType *)ioData->mBuffers[0].mData;
    // Zero-out all the output samples first
    memset(outSample, 0, inNumberFrames * kUnitSize * 2);
    if ( audioObject.playingiPod && audioObject.bufferIsReady) {
        // Pull audio from circular buffer
        int32_t availableBytes;
        AudioSampleType *bufferTail     = TPCircularBufferTail(&audioObject.circularBuffer, &availableBytes);
        memcpy(outSample, bufferTail, MIN(availableBytes, inNumberFrames * kUnitSize * 2) );
        TPCircularBufferConsume(&audioObject.circularBuffer, MIN(availableBytes, inNumberFrames * kUnitSize * 2) );
        audioObject.currentSampleNum += MIN(availableBytes / (kUnitSize * 2), inNumberFrames);
        if (availableBytes <= inNumberFrames * kUnitSize * 2) {
            // Buffer is running out or playback is finished
            audioObject.bufferIsReady = NO;
            audioObject.playingiPod = NO;
            audioObject.currentSampleNum = 0;
            if ([[audioObject delegate] respondsToSelector:@selector(playbackDidFinish)]) {
                [[audioObject delegate] performSelector:@selector(playbackDidFinish)];
            }
        }
    }
    return noErr;
}
- (void) setupSInt16StereoStreamFormat {
    // The AudioUnitSampleType data type is the recommended type for sample data in audio
    //    units. This obtains the byte size of the type for use in filling in the ASBD.
    size_t bytesPerSample = sizeof (AudioSampleType);
    // Fill the application audio format struct's fields to define a linear PCM, 
    //        stereo, noninterleaved stream at the hardware sample rate.
    SInt16StereoStreamFormat.mFormatID          = kAudioFormatLinearPCM;
    SInt16StereoStreamFormat.mFormatFlags       = kAudioFormatFlagsCanonical;
    SInt16StereoStreamFormat.mBytesPerPacket    = 2 * bytesPerSample;   // *** kAudioFormatFlagsCanonical <- implicit interleaved data => (left sample + right sample) per Packet 
    SInt16StereoStreamFormat.mFramesPerPacket   = 1;
    SInt16StereoStreamFormat.mBytesPerFrame     = SInt16StereoStreamFormat.mBytesPerPacket * SInt16StereoStreamFormat.mFramesPerPacket;
    SInt16StereoStreamFormat.mChannelsPerFrame  = 2;                    // 2 indicates stereo
    SInt16StereoStreamFormat.mBitsPerChannel    = 8 * bytesPerSample;
    SInt16StereoStreamFormat.mSampleRate        = graphSampleRate;

    NSLog (@"The stereo stream format for the "iPod" mixer input bus:");
    [self printASBD: SInt16StereoStreamFormat];
}

Answer 2

我想有点晚了，但你可以试试这个库：

https://bitbucket.org/artgillespie/tslibraryimport

使用此功能将音频保存到文件中后，可以使用MixerHost的渲染回调来处理数据。

Answer 3

如果我是你，我会使用kAudioUnitSubType_AudioFilePlayer播放文件，并使用单元渲染回调访问其示例。

或

使用ExtAudioFileRef将样本直接提取到缓冲区。