You're viewing an old version of the documentation. Click here to see the latest release.

通过ArUco标记实现的ROI框过滤器

This tutorial demonstrates how to filter the point cloud transformed to the ArUco marker coordinate system based on a ROI box given relative to the marker. This results in filtering out all the points that are outside of the bin.

小技巧

Zivid calibration board 包含一个 ArUco 标记。

This tutorial uses the point cloud of a ArUco marker displayed in the image below.

带有ArUco标记的料箱的点云

We can open the original point cloud in Zivid Studio and inspect it.

备注

原始点云在 Sample Data(示例数据) 中。

To skip the part about transforming the point cloud, which is also covered in 基于ArUco标记的坐标转换, go to 基于ROI框的过滤.

转换点云

首先,我们加载一个 ArUco 标记的点云。

const auto arucoMarkerFile = std::string(ZIVID_SAMPLE_DATA_DIR) + "/BinWithCalibrationBoard.zdf";
std::cout << "Reading ZDF frame from file: " << arucoMarkerFile << std::endl;
const auto frame = Zivid::Frame(arucoMarkerFile);
auto pointCloud = frame.pointCloud();

由于我们使用的是OpenCV,因此需要将2D图像从点云转换为OpenCV图像格式。

std::cout << "Converting to OpenCV image format" << std::endl;
const auto grayImage = pointCloudToGray(pointCloud);

然后我们将配置ArUco标记。

std::cout << "Configuring ArUco marker" << std::endl;
const auto markerDictionary = cv::aruco::getPredefinedDictionary(cv::aruco::DICT_4X4_100);
std::vector<int> markerIds;
std::vector<std::vector<cv::Point2f>> markerCorners;
cv::Ptr<cv::aruco::DetectorParameters> detectorParameters = cv::aruco::DetectorParameters::create();
detectorParameters->cornerRefinementMethod = cv::aruco::CORNER_REFINE_SUBPIX;

接下来在2D图像中检测ArUco标记。

std::cout << "Detecting ArUco Marker" << std::endl;
cv::aruco::detectMarkers(grayImage, markerDictionary, markerCorners, markerIds, detectorParameters);

然后估计ArUco标记的位姿。

std::cout << "Estimating pose of detected ArUco marker" << std::endl;
const auto transformMarkerToCamera = estimateArUcoMarkerPose(pointCloud, markerCorners[0]);
const auto transformCameraToMarker = transformMarkerToCamera.inverse();

将点云转换到ArUco标记坐标系。

std::cout << "Transforming point cloud from camera frame to ArUco marker frame" << std::endl;
pointCloud.transform(transformCameraToMarker);

提示

Learn more about 位置、方向和坐标变换.

基于ROI框的过滤

现在我们将定义ROI框的左下角相对于ArUco标记的位置,以及ROI框的大小。

相对于ArUco标记的ROI框大小和左下角
std::cout << "Bottom-Left ROI Box corner:" << std::endl;
const int roiBoxBottomLeftCornerX = 60; // Positive is "South"
const int roiBoxBottomLeftCornerY = 90; // Positive is "West"
const int roiBoxBottomLeftCornerZ = 15; // Positive is "Down"
std::cout << "X: " << roiBoxBottomLeftCornerX << std::endl
          << "Y: " << roiBoxBottomLeftCornerY << std::endl
          << "Z: " << roiBoxBottomLeftCornerZ << std::endl;
std::cout << "ROI Box size:" << std::endl;
const int roiBoxLength = 600;
const int roiBoxWidth = 400;
const int roiBoxHeight = 80;
std::cout << "Length: " << roiBoxLength << std::endl
          << "Width: " << roiBoxWidth << std::endl
          << "Height: " << roiBoxHeight << std::endl;

现在我们根据ROI框的大小和位置过滤点云。这是通过使 用 PCL 库实现的。

std::cout << "Filtering the point cloud based on ROI Box" << std::endl;
const boost::shared_ptr<pcl::PointCloud<pcl::PointXYZRGB>> roiPointCloudPCL = roiBoxPointCloud(
    pointCloud,
    roiBoxBottomLeftCornerX,
    roiBoxBottomLeftCornerY,
    roiBoxBottomLeftCornerZ,
    roiBoxLength,
    roiBoxWidth,
    roiBoxHeight);

最后可视化经过转换和过滤的点云。

std::cout << "Displaying transformed point cloud after ROI Box filtering" << std::endl;
visualizePointCloud(roiPointCloudPCL);
带有ArUco标记的料箱的ROI过滤和转换的点云

Lastly, we display the depth map of the transformed and filtered point cloud.

std::cout << "Displaying depth map of the transformed point cloud after ROI Box filtering" << std::endl;
visualizeDepthMap(*roiPointCloudPCL);
Depth map of the filtered and transformed point cloud of a bin with checkerboard