小贝子编程

c++中FFmpeg在转换音频时声音失真



我正在使用FFmpeg库来生成包含来自各种文件的音频的MP4文件，例如MP3, WAV, OGG，但我遇到了一些麻烦(我也将视频放在那里，但为了简单起见，我省略了这个问题，因为我已经得到了工作)。我当前的代码打开一个音频文件，解码内容并将其转换为MP4容器，最后将其作为交错帧写入目标文件。

对于大多数MP3文件都可以完美地工作，但是当输入WAV或OGG时，生成的MP4中的音频会稍微失真，并且经常以错误的速度播放(高达许多倍的快或慢)。

我看过无数使用转换函数(swr_convert)的例子，但我似乎无法摆脱导出音频中的噪音。

以下是我如何将音频流添加到MP4 (outContext是输出文件的AVFormatContext):

audioCodec = avcodec_find_encoder(outContext->oformat->audio_codec);
if (!audioCodec)
    die("Could not find audio encoder!");

// Start stream
audioStream = avformat_new_stream(outContext, audioCodec);
if (!audioStream)
    die("Could not allocate audio stream!");
audioCodecContext = audioStream->codec;
audioStream->id = 1;

// Setup
audioCodecContext->sample_fmt = AV_SAMPLE_FMT_S16;
audioCodecContext->bit_rate = 128000;
audioCodecContext->sample_rate = 44100;
audioCodecContext->channels = 2;
audioCodecContext->channel_layout = AV_CH_LAYOUT_STEREO;

// Open the codec
if (avcodec_open2(audioCodecContext, audioCodec, NULL) < 0)
    die("Could not open audio codec");

从MP3/WAV/OGG(从文件名变量)打开声音文件…

// Create contex
formatContext = avformat_alloc_context();
if (avformat_open_input(&formatContext, filename, NULL, NULL)<0)
    die("Could not open file");

// Find info
if (avformat_find_stream_info(formatContext, 0)<0)
    die("Could not find file info");
av_dump_format(formatContext, 0, filename, false);

// Find audio stream
streamId = av_find_best_stream(formatContext, AVMEDIA_TYPE_AUDIO, -1, -1, NULL, 0);
if (streamId < 0)
    die("Could not find Audio Stream");
codecContext = formatContext->streams[streamId]->codec;

// Find decoder
codec = avcodec_find_decoder(codecContext->codec_id);
if (codec == NULL)
    die("cannot find codec!");

// Open codec
if (avcodec_open2(codecContext, codec, 0)<0)
    die("Codec cannot be found");

// Set up resample context
swrContext = swr_alloc();
if (!swrContext)
    die("Failed to alloc swr context");
av_opt_set_int(swrContext, "in_channel_count", codecContext->channels, 0);
av_opt_set_int(swrContext, "in_channel_layout", codecContext->channel_layout, 0);
av_opt_set_int(swrContext, "in_sample_rate", codecContext->sample_rate, 0);
av_opt_set_sample_fmt(swrContext, "in_sample_fmt", codecContext->sample_fmt, 0);
av_opt_set_int(swrContext, "out_channel_count", audioCodecContext->channels, 0);
av_opt_set_int(swrContext, "out_channel_layout", audioCodecContext->channel_layout, 0);
av_opt_set_int(swrContext, "out_sample_rate", audioCodecContext->sample_rate, 0);
av_opt_set_sample_fmt(swrContext, "out_sample_fmt", audioCodecContext->sample_fmt, 0);
if (swr_init(swrContext))
    die("Failed to init swr context");

最后，解码+转换+编码…

// Allocate and init re-usable frames
audioFrameDecoded = av_frame_alloc();
if (!audioFrameDecoded)
        die("Could not allocate audio frame");
audioFrameDecoded->format = fileCodecContext->sample_fmt;
audioFrameDecoded->channel_layout = fileCodecContext->channel_layout;
audioFrameDecoded->channels = fileCodecContext->channels;
audioFrameDecoded->sample_rate = fileCodecContext->sample_rate;
audioFrameConverted = av_frame_alloc();
if (!audioFrameConverted)
        die("Could not allocate audio frame");
audioFrameConverted->nb_samples = audioCodecContext->frame_size;
audioFrameConverted->format = audioCodecContext->sample_fmt;
audioFrameConverted->channel_layout = audioCodecContext->channel_layout;
audioFrameConverted->channels = audioCodecContext->channels;
audioFrameConverted->sample_rate = audioCodecContext->sample_rate;
AVPacket inPacket;
av_init_packet(&inPacket);
inPacket.data = NULL;
inPacket.size = 0;
int frameFinished = 0;
while (av_read_frame(formatContext, &inPacket) >= 0) {
        if (inPacket.stream_index == streamId) {
                int len = avcodec_decode_audio4(fileCodecContext, audioFrameDecoded, &frameFinished, &inPacket);
                if (frameFinished) {
                        // Convert
                        uint8_t *convertedData=NULL;
                        if (av_samples_alloc(&convertedData,
                                             NULL,
                                             audioCodecContext->channels,
                                             audioFrameConverted->nb_samples,
                                             audioCodecContext->sample_fmt, 0) < 0)
                                die("Could not allocate samples");
                        int outSamples = swr_convert(swrContext,
                                                     &convertedData,
                                                     audioFrameConverted->nb_samples,
                                                     (const uint8_t **)audioFrameDecoded->data,
                                                     audioFrameDecoded->nb_samples);
                        if (outSamples < 0)
                                die("Could not convert");
                        size_t buffer_size = av_samples_get_buffer_size(NULL,
                                                                        audioCodecContext->channels,
                                                                        audioFrameConverted->nb_samples,
                                                                        audioCodecContext->sample_fmt,
                                                                        0);
                        if (buffer_size < 0)
                                die("Invalid buffer size");
                        if (avcodec_fill_audio_frame(audioFrameConverted,
                                                     audioCodecContext->channels,
                                                     audioCodecContext->sample_fmt,
                                                     convertedData,
                                                     buffer_size,
                                                     0) < 0)
                                die("Could not fill frame");
                        AVPacket outPacket;
                        av_init_packet(&outPacket);
                        outPacket.data = NULL;
                        outPacket.size = 0;
                        if (avcodec_encode_audio2(audioCodecContext, &outPacket, audioFrameConverted, &frameFinished) < 0)
                                die("Error encoding audio frame");
                        if (frameFinished) {
                                outPacket.stream_index = audioStream->index;
                                if (av_interleaved_write_frame(outContext, &outPacket) != 0)
                                        die("Error while writing audio frame");
                                av_free_packet(&outPacket);
                        }
                }
        }
}
av_frame_free(&audioFrameConverted);
av_frame_free(&audioFrameDecoded);
av_free_packet(&inPacket);

我也试过为传出帧设置适当的pts值，但这似乎根本不影响音质。

我也不确定如何/如果我应该分配转换后的数据，av_samples_alloc可以用于此?avcodec_fill_audio_frame呢?我的思路对吗?

任何输入都很感激(如果有必要，我也可以发送导出的mp4，如果你想听到失真)。

if (avcodec_encode_audio2(audioCodecContext, &outPacket, audioFrameConverted, &frameFinished) < 0)
                die("Error encoding audio frame");

你似乎假设编码器会吃掉所有提交的样本-它没有。它也不会在内部缓存它们。它将吃掉特定数量的样本(AVCodecContext.frame_size)，其余的应该在下一次调用avcodec_encode_audio2()时重新提交。

[编辑]

好的，所以你编辑的代码是更好的，但还没有到那一步。您仍然假设解码器将为每次调用avcodec_decode_audioN()(在重新采样之后)输出至少frame_size的样本，但事实可能并非如此。如果发生这种情况(对于ogg确实如此)，avcodec_encode_audioN()调用将编码一个不完整的输入缓冲区(因为您说它有frame_size样本，但实际上没有)。同样，您的代码也不能处理解码器输出的数字明显大于编码器期望的frame_size(如10*frame_size)的情况，在这种情况下，您将获得溢出-基本上您的1:1解码/编码映射是您的问题的主要来源。

作为一种解决方案，将swrContext视为FIFO，在其中输入所有解码器样本，并循环它，直到剩余的样本少于frame_size。我会留给你学习如何处理end-of-stream,因为你需要刷新缓存的样本的解码器(通过调用avcodec_decode_audioN()与AVPacket . data = NULL和.size = 0),冲swrContext(通过调用swr_context(),直到它返回0)以及冲洗编码器(零AVFrames喂养它,直到它返回AVPacket .size = 0),现在你可能会得到一个输出文件,最后有点截断。这应该不难理解。

此代码适用于我的m4a/ogg/mp3到m4a/aac转换:

#include "libswresample/swresample.h"
#include "libavcodec/avcodec.h"
#include "libavformat/avformat.h"
#include "libavutil/opt.h"
#include <stdio.h>
#include <stdlib.h>
static void die(char *str) {
    fprintf(stderr, "%sn", str);
    exit(1);
}
static AVStream *add_audio_stream(AVFormatContext *oc, enum AVCodecID codec_id)
{
    AVCodecContext *c;
    AVCodec *encoder = avcodec_find_encoder(codec_id);
    AVStream *st = avformat_new_stream(oc, encoder);
    if (!st) die("av_new_stream");
    c = st->codec;
    c->codec_id = codec_id;
    c->codec_type = AVMEDIA_TYPE_AUDIO;
    /* put sample parameters */
    c->bit_rate = 64000;
    c->sample_rate = 44100;
    c->channels = 2;
    c->sample_fmt = encoder->sample_fmts[0];
    c->channel_layout = AV_CH_LAYOUT_STEREO;
    // some formats want stream headers to be separate
    if(oc->oformat->flags & AVFMT_GLOBALHEADER)
        c->flags |= CODEC_FLAG_GLOBAL_HEADER;
    return st;
}
static void open_audio(AVFormatContext *oc, AVStream *st)
{
    AVCodecContext *c = st->codec;
    AVCodec *codec;
    /* find the audio encoder */
    codec = avcodec_find_encoder(c->codec_id);
    if (!codec) die("avcodec_find_encoder");
    /* open it */
    AVDictionary *dict = NULL;
    av_dict_set(&dict, "strict", "+experimental", 0);
    int res = avcodec_open2(c, codec, &dict);
    if (res < 0) die("avcodec_open");
}
int main(int argc, char *argv[]) {
    av_register_all();
    if (argc != 3) {
        fprintf(stderr, "%s <in> <out>n", argv[0]);
        exit(1);
    }
    // Allocate and init re-usable frames
    AVCodecContext *fileCodecContext, *audioCodecContext;
    AVFormatContext *formatContext, *outContext;
    AVStream *audioStream;
    SwrContext *swrContext;
    int streamId;
    // input file
    const char *file = argv[1];
    int res = avformat_open_input(&formatContext, file, NULL, NULL);
    if (res != 0) die("avformat_open_input");
    res = avformat_find_stream_info(formatContext, NULL);
    if (res < 0) die("avformat_find_stream_info");
    AVCodec *codec;
    res = av_find_best_stream(formatContext, AVMEDIA_TYPE_AUDIO, -1, -1, &codec, 0);
    if (res < 0) die("av_find_best_stream");
    streamId = res;
    fileCodecContext = avcodec_alloc_context3(codec);
    avcodec_copy_context(fileCodecContext, formatContext->streams[streamId]->codec);
    res = avcodec_open2(fileCodecContext, codec, NULL);
    if (res < 0) die("avcodec_open2");
    // output file
    const char *outfile = argv[2];
    AVOutputFormat *fmt = fmt = av_guess_format(NULL, outfile, NULL);
    if (!fmt) die("av_guess_format");
    outContext = avformat_alloc_context();
    outContext->oformat = fmt;
    audioStream = add_audio_stream(outContext, fmt->audio_codec);
    open_audio(outContext, audioStream);
    res = avio_open2(&outContext->pb, outfile, AVIO_FLAG_WRITE, NULL, NULL);
    if (res < 0) die("url_fopen");
    avformat_write_header(outContext, NULL);
    audioCodecContext = audioStream->codec;
    // resampling
    swrContext = swr_alloc();
    av_opt_set_channel_layout(swrContext, "in_channel_layout",  fileCodecContext->channel_layout, 0);
    av_opt_set_channel_layout(swrContext, "out_channel_layout", audioCodecContext->channel_layout, 0);
    av_opt_set_int(swrContext, "in_sample_rate", fileCodecContext->sample_rate, 0);
    av_opt_set_int(swrContext, "out_sample_rate", audioCodecContext->sample_rate, 0);
    av_opt_set_sample_fmt(swrContext, "in_sample_fmt", fileCodecContext->sample_fmt, 0);
    av_opt_set_sample_fmt(swrContext, "out_sample_fmt", audioCodecContext->sample_fmt, 0);
    res = swr_init(swrContext);
    if (res < 0) die("swr_init");
    AVFrame *audioFrameDecoded = av_frame_alloc();
    if (!audioFrameDecoded)
        die("Could not allocate audio frame");
    audioFrameDecoded->format = fileCodecContext->sample_fmt;
    audioFrameDecoded->channel_layout = fileCodecContext->channel_layout;
    audioFrameDecoded->channels = fileCodecContext->channels;
    audioFrameDecoded->sample_rate = fileCodecContext->sample_rate;
    AVFrame *audioFrameConverted = av_frame_alloc();
    if (!audioFrameConverted) die("Could not allocate audio frame");
    audioFrameConverted->nb_samples = audioCodecContext->frame_size;
    audioFrameConverted->format = audioCodecContext->sample_fmt;
    audioFrameConverted->channel_layout = audioCodecContext->channel_layout;
    audioFrameConverted->channels = audioCodecContext->channels;
    audioFrameConverted->sample_rate = audioCodecContext->sample_rate;
    AVPacket inPacket;
    av_init_packet(&inPacket);
    inPacket.data = NULL;
    inPacket.size = 0;
    int frameFinished = 0;
    while (av_read_frame(formatContext, &inPacket) >= 0) {
        if (inPacket.stream_index == streamId) {
            int len = avcodec_decode_audio4(fileCodecContext, audioFrameDecoded, &frameFinished, &inPacket);
            if (frameFinished) {
                // Convert
                uint8_t *convertedData=NULL;
                if (av_samples_alloc(&convertedData,
                             NULL,
                             audioCodecContext->channels,
                             audioFrameConverted->nb_samples,
                             audioCodecContext->sample_fmt, 0) < 0)
                    die("Could not allocate samples");
                int outSamples = swr_convert(swrContext, NULL, 0,
                             //&convertedData,
                             //audioFrameConverted->nb_samples,
                             (const uint8_t **)audioFrameDecoded->data,
                             audioFrameDecoded->nb_samples);
                if (outSamples < 0) die("Could not convert");
                for (;;) {
                     outSamples = swr_get_out_samples(swrContext, 0);
                     if (outSamples < audioCodecContext->frame_size * audioCodecContext->channels) break; // see comments, thanks to @dajuric for fixing this
                     outSamples = swr_convert(swrContext,
                                              &convertedData,
                                              audioFrameConverted->nb_samples, NULL, 0);
                     size_t buffer_size = av_samples_get_buffer_size(NULL,
                                    audioCodecContext->channels,
                                    audioFrameConverted->nb_samples,
                                    audioCodecContext->sample_fmt,
                                    0);
                    if (buffer_size < 0) die("Invalid buffer size");
                    if (avcodec_fill_audio_frame(audioFrameConverted,
                             audioCodecContext->channels,
                             audioCodecContext->sample_fmt,
                             convertedData,
                             buffer_size,
                             0) < 0)
                        die("Could not fill frame");
                    AVPacket outPacket;
                    av_init_packet(&outPacket);
                    outPacket.data = NULL;
                    outPacket.size = 0;
                    if (avcodec_encode_audio2(audioCodecContext, &outPacket, audioFrameConverted, &frameFinished) < 0)
                        die("Error encoding audio frame");
                    if (frameFinished) {
                        outPacket.stream_index = audioStream->index;
                        if (av_interleaved_write_frame(outContext, &outPacket) != 0)
                            die("Error while writing audio frame");
                        av_free_packet(&outPacket);
                    }
                }
            }
        }
    }
    swr_close(swrContext);
    swr_free(&swrContext);
    av_frame_free(&audioFrameConverted);
    av_frame_free(&audioFrameDecoded);
    av_free_packet(&inPacket);
    av_write_trailer(outContext);
    avio_close(outContext->pb);
    avcodec_close(fileCodecContext);
    avcodec_free_context(&fileCodecContext);
    avformat_close_input(&formatContext);
    return 0;
}

我想包含我在处理上述代码时发现的一些东西。我有一个文件陷入了无限循环。原因是文件的采样率为48000，代码将其更改为44100。这导致它总是有额外的输出样本。swr_convert,不会抓住他们。所以我最终改变了add_audio_stream来匹配输入流的采样率。

        c->sample_rate = fileCodecContext->sample_rate;

我还必须产生wav文件作为我的输出。它的帧大小是0。所以我只是选择了一个数字，经过几次测试，我选择了32。我注意到如果我设置得太大(例如128)，我就会出现音频故障。

 if (audioFrameConverted->nb_samples <= 0) audioFrameConverted->nb_samples = 32; //wav files have a 0

改变if语句在跳出循环时检查nb_samples是否frame_size为0。

                            if ((outSamples < audioCodecContext->frame_size * audioCodecContext->channels) || audioCodecContext->frame_size==0 && (outSamples < audioFrameConverted->nb_samples * audioCodecContext->channels)) break; // see comments, thanks to @dajuric for fixing this

当我测试输出到ogg文件时，时间戳数据丢失了，所以文件不能在vlc中正确播放。我加了几行来帮助解决这个问题。

        out_audioStream->time_base = in_audioStream->time_base; // entered before avio_open.
                        outPacket.dts = audioFrameDecoded->pkt_dts;//rest after avcodec_encode_audio2
                        outPacket.pts = audioFrameDecoded->pkt_pts;
                        av_packet_rescale_ts(&outPacket, in_audioStream->time_base, out_audioStream->time_base);

变量可能有点不同，我将代码转换为c#。我想这也许能帮到别人

实际上swr_convert不起作用，请尝试使用swr_convert_frame

相关内容

最新更新


热门标签：

javascript python java c# php android html jquery c++ css ios sql mysql arrays asp.net json python-3.x ruby-on-rails .net sql-server django objective-c excel regex ruby linux ajax iphone xml vba spring asp.net-mvc database wordpress string postgresql wpf windows xcode bash git oracle list vb.net multithreading eclipse algorithm macos powershell visual-studio image forms numpy scala function api selenium