如何从OpenCV c++代码流帧到Video4Linux或ffmpeg?

Question

我正在尝试使用OpenCV来处理视频流中的帧。目标是从流中获取一个帧，处理它，然后将处理后的帧放到一个新的/新鲜的流中。

我已经能够使用OpenCV视频捕获功能成功读取流。但是不知道如何用处理过的帧创建输出流。

为了进行一些基本的测试，我使用ffmpeg从本地视频文件创建了一个流，如下所示:

ffmpeg -i sample.mp4 -v 0 -vcodec mpeg4 -f mpegts 
"udp://@127.0.0.1:23000?overrun_nonfatal=1&fifo_size=50000000"

在我的c++代码中使用OpenCV库的videoccapture功能，我能够捕获上面创建的流。我试图实现的基本布局如下附件:

cv::VideoCapture capture("udp://@127.0.0.1:23000?overrun_nonfatal=1&fifo_size=50000000", cv::CAP_FFMPEG);
cv::Mat frame;
while (true) 
{
// use the above stream to capture a frame
capture >> frame;

// process the frame (not relevant here)
...
// finally, after all the processing is done I 
// want to put this frame on a new stream say at
// udp://@127.0.0.1:25000, I don't know how to do
// this, ideally would like to use Video4Linux but
// solutions with ffmpeg are appreciated as well
}

正如你可以从上面代码的注释中看到的，我不知道我应该如何开始处理这个问题，我试着搜索类似的问题，但我所能找到的是如何使用流做videoccapture，与输出到流无关。

我对这方面比较陌生，这对你们中的许多人来说可能是一个非常基本的问题，请原谅我。

Answer 1

我们可以使用与下面的Python代码示例相同的技术。

• 将FFmpeg作为子进程执行，打开stdin管道写入

  FILE *pipeout = popen(ffmpeg_cmd.c_str(), "w")
  

• 将frame.data写入FFmpeg子进程的stdin管道(在一个循环中)

  fwrite(frame.data, 1, width*height*3, pipeout);
  

• 关闭末端管道(它将关闭子进程)

  pclose(pipeout);
  

---

下面的示例是一个通用示例-构建编号帧，并将编码的视频写入MKV输出文件。

ffmpeg -y -f rawvideo -r 10 -video_size 320x240 -pixel_format bgr24 -i pipe: -vcodec libx264 -crf 24 -pix_fmt yuv420p output.mkv

您可以根据您的具体要求调整参数(将output.mkv替换为udp://@127.0.0.1:25000)。

将编号帧替换为capture >> frame，并调整大小和帧率

---

代码示例:

#include <stdio.h>
#include <chrono>
#include <thread>
#include "opencv2/opencv.hpp"
int main()
{
int width = 320;
int height = 240;
int n_frames = 100;
int fps = 10;
//Use a "generic" example (write the output video in output.mkv video file).
//ffmpeg -y -f rawvideo -r 10 -video_size 320x240 -pixel_format bgr24 -i pipe: -vcodec libx264 -crf 24 -pix_fmt yuv420p output.mkv
std::string ffmpeg_cmd = std::string("ffmpeg -y -f rawvideo -r ") + std::to_string(fps) +
" -video_size " + std::to_string(width) + "x" + std::to_string(height) +
" -pixel_format bgr24 -i pipe: -vcodec libx264 -crf 24 -pix_fmt yuv420p output.mkv";
//Execute FFmpeg as sub-process, open stdin pipe (of FFmpeg sub-process) for writing.
//In Windows we need to use _popen and in Linux popen
#ifdef _MSC_VER
FILE *pipeout = _popen(ffmpeg_cmd.c_str(), "wb");   //Windows (ffmpeg.exe must be in the execution path)
#else
//https://batchloaf.wordpress.com/2017/02/12/a-simple-way-to-read-and-write-audio-and-video-files-in-c-using-ffmpeg-part-2-video/
FILE *pipeout = popen(ffmpeg_cmd.c_str(), "w");     //Linux (assume ffmpeg exist in /usr/bin/ffmpeg (and in path).
#endif
for (int i = 0; i < n_frames; i++)
{
cv::Mat frame = cv::Mat(height, width, CV_8UC3);
frame = cv::Scalar(60, 60, 60); //Fill background with dark gray
cv::putText(frame, std::to_string(i+1), cv::Point(width/2-50*(int)(std::to_string(i+1).length()), height/2+50), cv::FONT_HERSHEY_DUPLEX, 5, cv::Scalar(30, 255, 30), 10);  // Draw a green number
cv::imshow("frame", frame);cv::waitKey(1); //Show the frame for testing
//Write width*height*3 bytes to stdin pipe of FFmpeg sub-process (assume frame data is continuous in the RAM).
fwrite(frame.data, 1, width*height*3, pipeout);
}
// Flush and close input and output pipes
fflush(pipeout);
#ifdef _MSC_VER
_pclose(pipeout);   //Windows
#else
pclose(pipeout);    //Linux
#endif
//It looks like we need to wait one more second at the end. //https://stackoverflow.com/a/62804585/4926757
std::this_thread::sleep_for(std::chrono::milliseconds(1000)); // sleep for 1 second
return 0;
}