相机校准和坐标转换(OpenCV)

Question

我正在尝试将图像中的 2D 像素坐标映射到相对于固定网络摄像头的真实世界 3D 坐标。OpenCV 页面上的校准教程 （http://docs.opencv.org/doc/tutorials/calib3d/camera_calibration/camera_calibration.html）给了我以下XML文件：

<?xml version="1.0"?>
<opencv_storage>
<calibration_Time>"Wed Jun 17 12:02:01 2015"</calibration_Time>
<nrOfFrames>25</nrOfFrames>
<image_Width>640</image_Width>
<image_Height>480</image_Height>
<board_Width>8</board_Width>
<board_Height>6</board_Height>
<square_Size>20.</square_Size>
<FixAspectRatio>1.</FixAspectRatio>
<!-- flags:  +fix_aspectRatio +fix_principal_point +zero_tangent_dist -->
<flagValue>14</flagValue>
<Camera_Matrix type_id="opencv-matrix">
  <rows>3</rows>
  <cols>3</cols>
  <dt>d</dt>
  <data>
    6.4246640336041310e+02 0. 3.1950000000000000e+02 0.
    6.4246640336041310e+02 2.3950000000000000e+02 0. 0. 1.</data></Camera_Matrix>
<Distortion_Coefficients type_id="opencv-matrix">
  <rows>5</rows>
  <cols>1</cols>
  <dt>d</dt>
  <data>
    1.4184643251446313e-01 -8.7120838107875664e-01 0. 0.
    2.0360310039624929e+00</data></Distortion_Coefficients>
<Avg_Reprojection_Error>6.5243676409192319e-01</Avg_Reprojection_Error>
<Per_View_Reprojection_Errors type_id="opencv-matrix">
  <rows>25</rows>
  <cols>1</cols>
  <dt>f</dt>
  <data>
    1.23084724e+00 1.32115483e+00 5.55041075e-01 4.16657448e-01
    1.49720192e+00 5.03276110e-01 6.16328895e-01 4.94440526e-01
    4.66414422e-01 8.33932340e-01 9.33699965e-01 3.97324741e-01
    3.23639482e-01 4.10246909e-01 2.44877860e-01 2.70577997e-01
    2.78500348e-01 4.29472983e-01 4.55302685e-01 3.92662942e-01
    5.15928566e-01 5.14224648e-01 3.21243197e-01 2.64228135e-01
    3.23284745e-01</data></Per_View_Reprojection_Errors>
<!-- a set of 6-tuples (rotation vector + translation vector) for each view -->
<Extrinsic_Parameters type_id="opencv-matrix">
  <rows>25</rows>
  <cols>6</cols>
  <dt>d</dt>
  <data>
    3.1133142550749011e-02 -5.8257187373227007e-02
    1.0014202062100359e-01 -4.5770045507937944e+01
    -1.1401352227456540e+02 3.2654046608278867e+02
    6.0848078467615876e-02 -7.6884180246540271e-02
    1.0884720432626112e-01 -4.8424152148142376e+01
    -6.9706198906307435e+01 3.2155661235686972e+02
    1.3078176637099181e-01 -5.9774166568104962e-02
    1.2075445989322871e-01 -5.2213335264575136e+01
    -1.5658397119925459e+01 3.2344722541209376e+02
    1.3010533681681599e-01 -7.2350906922120220e-02
    1.2763854815023157e-01 -5.2635626593433386e+01
    -2.3806343728041508e+01 3.2705738581249113e+02
    1.3912962778204069e-01 -1.1049859842338137e-02
    2.6568761637858501e-01 -3.9609919391102117e+01
    -6.5313490580652399e+01 3.4371586975397855e+02
    4.7185924170123673e+02 2.6547080711327864e-01
    -1.9783229706065680e-01 -3.0714880692083590e-01
    -7.9458715714529916e+01 -6.3795768398766498e+01
    4.7883243059022925e+02 2.9029044731335096e-01
    -1.9211746563166354e-01 -5.8925983063271670e-01
    -8.3939542784626539e+01 -2.8903912158339107e+01
    4.8159128458553204e+02</data></Extrinsic_Parameters>
<Image_points type_id="opencv-matrix">
  <rows>25</rows>
  <cols>48</cols>
  <dt>"2f"</dt>
  <data>
    2.29625870e+02 1.39961987e+01 2.67603271e+02 1.79401379e+01
    3.08121796e+02 2.33054199e+01 3.45650085e+02 2.78233128e+01
    3.85591187e+02 3.26711960e+01 4.22817383e+02 3.64260292e+01
    4.61843079e+02 4.17984467e+01 4.99163055e+02 4.46697083e+01
    2.24534485e+02 5.22526741e+01 2.65134949e+02 5.82546883e+01
    3.03018738e+02 6.22039948e+01 3.43023987e+02 6.74814377e+01
    3.80778931e+02 6.92338257e+01 4.19888397e+02 7.64811630e+01
    4.56726318e+02 7.93488922e+01 4.96191772e+02 8.33257065e+01
    2.21918411e+02 9.32228699e+01 2.60520325e+02 9.65068970e+01
    3.00293610e+02 1.03282265e+02 3.38202759e+02 1.04744865e+02
    3.77362061e+02 1.09695145e+02 4.14984894e+02 1.13752464e+02
    4.53334625e+02 1.16593300e+02 4.90682739e+02 1.22847855e+02
    2.18192841e+02 1.31075851e+02 2.57387268e+02 1.35427399e+02
    2.95809937e+02 1.40996185e+02 3.34671997e+02 1.44297516e+02
    3.72804840e+02 1.48442688e+02 4.11339203e+02 1.53404419e+02
    4.48546783e+02 1.58333084e+02 4.87274445e+02 1.61437836e+02
    2.15059067e+02 1.71062363e+02 2.53465714e+02 1.76600189e+02
    2.92494446e+02 1.81009354e+02 3.30455017e+02 1.85346848e+02
    3.69074799e+02 1.89727127e+02 4.06779175e+02 1.93523773e+02
    4.44997833e+02 1.97280396e+02 4.82324921e+02 2.02022141e+02
    2.11744431e+02 2.11941452e+02 2.50135788e+02 2.14884964e+02
    2.88372894e+02 2.19861420e+02 3.26681396e+02 2.23228668e+02
    3.64834045e+02 2.27242767e+02 4.03112518e+02 2.32115265e+02
    4.40870544e+02 2.36501205e+02 4.78603394e+02 2.39769150e+02
    2.23281158e+02 9.84474030e+01 2.62629608e+02 1.03374336e+02
    3.02005402e+02 1.07965027e+02 3.41092072e+02 1.12782112e+02
    3.80057037e+02 1.17424217e+02 4.18419128e+02 1.21913788e+02
    4.56700836e+02 1.26250870e+02 4.94675476e+02 1.30832565e+02
    2.18958359e+02 1.39547729e+02 2.58642120e+02 1.43881851e+02
    2.97799347e+02 1.48740479e+02 3.36681335e+02 1.53159988e+02
    3.75197388e+02 1.57572586e+02 4.13595093e+02 1.61976059e+02
    4.51498779e+02 1.66460098e+02 4.89762268e+02 1.70545746e+02
    2.15278198e+02 1.79598145e+02 2.54589981e+02 1.84568054e+02
    2.93620117e+02 1.88593597e+02 3.32224091e+02 1.93084717e+02
    3.70647705e+02 1.97427017e+02 4.08592590e+02 2.01596893e+02
    4.46772217e+02 2.05726089e+02 4.84447693e+02 2.10178162e+02
    2.11591599e+02 2.19893600e+02 2.50586578e+02 2.24072342e+02
    2.89431732e+02 2.28405090e+02 3.27965729e+02 2.32436920e+02
    3.65994720e+02 2.36633270e+02 4.04072449e+02 2.40559113e+02
    4.41895966e+02 2.45062546e+02 4.79643768e+02 2.48762146e+02
    2.08430328e+02 2.58684784e+02 2.46600525e+02 2.63532990e+02
    2.85255615e+02 2.67623260e+02 3.23492676e+02 2.71995331e+02
   </data>
  </Image_points>
</opencv_storage>

如何使用此信息获取旋转平移矩阵？如何从 2D 像素坐标转换为物体相对于相机的 3D 坐标？

Answer 1

相机校准过程估计相机固有参数：相机矩阵（通常表示为K）和镜头畸变系数D。（注意：相机相对于图案的旋转平移矩阵也针对用于校准的每个图像计算，请参阅"Extrinsic_Parameters"，但通常不使用它们，因为它们没有在有用的坐标框中表示）。

如果要计算连接到相机的规范坐标系中的3D点（即原点的光学中心，图像中的X和Y轴以及沿观看方向的Z轴），只需使用[R|t] = [Id|0]即可。假设没有镜头畸变，您可以将均匀的 2D 坐标m映射到均匀的 3D 坐标M如下所示：m = K . [R|t] . M .对于镜头畸变，方程会更加复杂，并使用非均匀坐标。它们在本页开头的 OpenCV 文档中有明确的描述。

如果要使用另一个坐标框，并且知道此坐标系中表示的几个 3D 点与其在图像中的投影之间的对应关系，则可以使用函数cv::solvePnP（请参阅文档）来估计关联的[R|t]矩阵。