无法使用OpenCV打开两个摄像头-多线程摄像头读取

Question

我正试图使用OpenCV通过单独的线程连续流式传输来自两台摄像机的视频。以下代码显示Segmentation fault (core dumped)

出现这种情况的原因是什么？我该如何解决此问题？

main.cpp

#include <iostream>
#include <pthread.h>
#include <opencv2/core.hpp>
#include <opencv2/highgui.hpp>
#include <opencv2/videoio.hpp>
using namespace std;
struct thread_data {
string path;
int  thread_id;
};
void *capture(void *threadarg)
{
struct thread_data *data;
data = (struct thread_data *) threadarg;
cv::VideoCapture cap(data->path);
if( !cap.isOpened())
{
std::cout<<"Not good, open camera failed"<<std::endl;
}
std::cout<< "Opened IP camera successfully!"<<std::endl;
cv::Mat frame;
string ext = ".jpg";
string result;
while (true) {
cap >> frame;
cv::imshow("Frame",frame);
cv::waitKey(1);
}
pthread_exit(NULL);
}
int main(void) {
pthread_t threads[2];
struct thread_data td[2];
int rc=0;
for( int i = 0; i < 2; i++ ) {
cout <<"main() : creating thread, " << i << endl;
td[i].thread_id = i;
td[0].path = "rtsp://admin:opencv123@192.168.1.23:554/Streaming/Channels/101/";
td[1].path = "rtsp://admin:opencv123@192.168.1.24:554/Streaming/Channels/101/";
rc = pthread_create(&threads[i], NULL, capture, (void *)&td[i]);
if (rc) {
cout << "Error:unable to create thread," << rc << endl;
exit(-1);
}
}
pthread_exit(NULL);
return 0;
}

日志：

main() : creating thread, 0    main() : creating thread, 1    
Segmentation fault (core dumped)

当我试着多次运行它时，我只能打开一个摄像头，而这个摄像头也不能连续播放。它在几秒钟内开始和停止。

有时我会收到一个错误，上面写着

OpenCV Error: Insufficient memory (Failed to allocate 140703464366800 bytes) in OutOfMemoryError

我经历了各种各样的问答；在StackOverflow上为"，但没有任何帮助。

Answer 1

这里的问题是代码面临竞争条件。我能够在我的系统上复制这个问题，并确定了以下问题：

1. OpenCV窗口标题不是唯一的
2. 有螺纹的螺纹未连接
3. 打开视频流时的种族状况

让我们详细了解这些问题。

1

OpenCV窗口通过其标题进行唯一标识。在当前代码中，标题是一个硬编码字符串"Frame"。因此，基本上，两个线程都在以未知的顺序创建/更新/销毁同一个窗口。这是一个竞争条件，可以通过向struct thread_data添加一个字符串字段来修复，该字段将用作唯一窗口标识符。

2

在主线程中，子线程是异步创建的，因此for循环将在创建线程后立即退出，程序将提前退出，而无需等待派生线程完成执行。这个问题可以通过添加函数调用来解决，以便在程序退出之前等待线程。这个过程被称为加入，可以通过为每个派生线程调用pthread_join来实现。

3

这个问题有点棘手。由于某些原因，OpenCV使用的用于视频流捕获的后端库没有以线程安全的方式运行。看起来，视频捕获打开过程容易出现竞争情况，需要同步锁定。通过在打开VideoCapture对象之前和之后调用函数pthread_mutex_lock和pthread_mutex_unlock，可以容易地实现锁定。

以下是修改后的代码，演示了上述所有问题的解决方案

#include <iostream>
#include <pthread.h>
#include <opencv2/core.hpp>
#include <opencv2/highgui.hpp>
#include <opencv2/videoio.hpp>
using namespace std;
//Mutex for thread synchronization
static pthread_mutex_t foo_mutex = PTHREAD_MUTEX_INITIALIZER;

struct thread_data 
{
string path;
int  thread_id;
string window_title; //Unique window title for each thread
};
void *capture(void *threadarg)
{
struct thread_data *data;
data = (struct thread_data *) threadarg;
cv::VideoCapture cap;

//Safely open video stream
pthread_mutex_lock(&foo_mutex);
cap.open(data->path);
pthread_mutex_unlock(&foo_mutex);
if( !cap.isOpened())
{
std::cout<<"Not good, open camera failed"<<std::endl;
}
std::cout<< "Opened IP camera successfully!"<<std::endl;
cv::Mat frame;
string ext = ".jpg";
string result;
//Create window with unique title
cv::namedWindow(data->window_title);
while (true) 
{
cap >> frame;
cv::imshow(data->window_title,frame);
cv::waitKey(10);
}
//Release VideoCapture object
cap.release();
//Destroy previously created window
cv::destroyWindow(data->window_title);
//Exit thread
pthread_exit(NULL);
}
int main(void)
{
const int thread_count = 2;
pthread_t threads[thread_count];
struct thread_data td[thread_count];

//Initialize thread data beforehand
td[0].path = "rtsp://admin:opencv123@192.168.1.23:554/Streaming/Channels/101/";
td[0].window_title = "First Window";
td[1].path = "rtsp://admin:opencv123@192.168.1.24:554/Streaming/Channels/101/";
td[1].window_title = "Second Window";

int rc=0;
for( int i = 0; i < thread_count; i++ ) 
{
cout <<"main() : creating thread, " << i << endl;
td[i].thread_id = i;
rc = pthread_create(&(threads[i]), NULL, capture, (void *)& (td[i]) );
if (rc) 
{
cout << "Error:unable to create thread," << rc << endl;
exit(-1);
}
}
//Wait for the previously spawned threads to complete execution
for( int i = 0; i < thread_count; i++ )
pthread_join(threads[i], NULL);
pthread_exit(NULL);
return 0;
}

Answer 2

void writeFrametoDisk(const cv::Mat *frame, std::string path, int frameNum, std::string windowName)
{
cv::imwrite(path.append(std::to_string(frameNum)).append(".png"), *frame);
return;
}
void openCameraStream(int deviceId, std::string dirName)
{
cv::VideoCapture cap;
cap.open(deviceId);
if(!cap.isOpened()){std::cerr << "Unable to open the camera " << std::endl;}
std::cout << cap.get(cv::CAP_PROP_FRAME_WIDTH) << " " << cap.get(cv::CAP_PROP_FRAME_HEIGHT) << std::endl;
std::string windowName = deviceId == 0 ? "Cam 0" : "Cam 1";
std::string outputDir = dirName;
bool frameStFg = false;
while(!frameStFg)
{
cv::Mat frame;
cap.read(frame);
if(frame.empty()){std::cerr << "frame buffer empty " << std::endl;}
else
{
frameStFg = true;
}
}
cv::TickMeter timer;
timer.start();
int frameCount = 0;
while(1)
{
cv::Mat frame;
cap.read(frame);
if(frame.empty()){std::cerr << "frame buffer empty " << std::endl;}
frameCount++;
std::thread th(writeFrametoDisk, &frame, outputDir, frameCount, windowName);
th.join();
//// a simple wayto exit the loop
if(frameCount > 500)
{
break;
}
}
timer.stop();
std::cout << "Device id " << deviceId << " Capture ran for " << timer.getTimeSec() << " seconds" << std::endl;
std::cout << "Device id " << deviceId << " Number of frames to be capture should be " << timer.getTimeSec() * 30 << std::endl;
std::cout << "Device id " << deviceId << " Number of frames captured " << frameCount << std::endl;
cap.release();
}
int main(int argc, char * argv[])
{
std::string outputDir1 = "";
std::string outputDir2 = "";
std::thread camera1Thread(openCameraStream, 0, outputDir1);
std::thread camera2Thread(openCameraStream, 1, outputDir2);
camera1Thread.join();
camera2Thread.join();
}

正如评论所提到的，没有任何图像流的imshow()的流式传输可以很好地解决任何问题。我的设置包括，用两个USB相机运行两个线程，这两个线程还分别启动一个新线程来保存从相机读取的图像。我没有观察到任何帧丢失或错误，并且能够以大约30帧/秒的速度进行捕获和写入。

经过一点调试，建议只在主线程中使用任何imshow()，即main()函数。希望这能帮助到任何人。