通过ArUco标记定义的ROI盒

本教程演示了如何使用ArUco标记定义 ROI 盒的参数以及如何使用它进行过滤。我们在这里使用了 Zivid calibration board 上的ArUco标记，假设标定板放置在料箱的右下角。料箱大小也假定为已知，该数据用于设置ROI盒的尺寸。这样你就可以自动找到相机坐标系中的ROI参数了。

备注

本教程使用了一个档案相机（file camera）对下图中的场景进行演示。

小技巧

ROI 框过滤可以减少捕获时间。

档案相机可以从 Sample Data（示例数据）下载。

首先，我们需要捕获ArUco标记的点云。

C++

const auto fileCamera = std::string(ZIVID_SAMPLE_DATA_DIR) + "/BinWithCalibrationBoard.zfc";

std::cout << "Creating virtual camera using file: " << fileCamera << std::endl;
auto camera = zivid.createFileCamera(fileCamera);

auto settings = Zivid::Settings{ Zivid::Settings::Acquisitions{ Zivid::Settings::Acquisition{} } };

const auto originalFrame = camera.capture(settings);
auto pointCloud = originalFrame.pointCloud();

C＃

string fileCamera = Environment.GetFolderPath(Environment.SpecialFolder.CommonApplicationData) + "/Zivid/BinWithCalibrationBoard.zfc";

Console.WriteLine("Creating virtual camera using file: " + fileCamera);
var camera = zivid.CreateFileCamera(fileCamera);

var settings = new Zivid.NET.Settings
{
    Acquisitions = { new Zivid.NET.Settings.Acquisition { } }
};

var originalFrame = camera.Capture(settings);
var pointCloud = originalFrame.PointCloud;

Python

file_camera = get_sample_data_path() / "BinWithCalibrationBoard.zfc"

print(f"Creating virtual camera using file: {file_camera}")
camera = app.create_file_camera(file_camera)

settings = zivid.Settings([zivid.Settings.Acquisition()])

original_frame = camera.capture(settings)
point_cloud = original_frame.point_cloud()

ArUco标记坐标系的原点位于ArUco标记的中心。

我们定义了ROI盒右下角相对于ArUco标记坐标系的位置，以及ROI盒尺寸。然后我们从长度和宽度中减去箱边缘的宽度以移除箱壁。

C++

const float roiBoxLength = 545.F;
const float roiBoxWidth = 345.F;
const float roiBoxHeight = 150.F;
// Coordinates are relative to the ArUco marker origin which lies in the center of the ArUco marker.
// Positive x-axis is "East", y-axis is "South" and z-axis is "Down".
const Zivid::PointXYZ roiBoxLowerRightCornerInArUcoFrame{ 240.F, 30.F, 5.F };
const Zivid::PointXYZ roiBoxUpperRightCornerInArUcoFrame{ roiBoxLowerRightCornerInArUcoFrame.x,
                                                          roiBoxLowerRightCornerInArUcoFrame.y - roiBoxWidth,
                                                          roiBoxLowerRightCornerInArUcoFrame.z };
const Zivid::PointXYZ roiBoxLowerLeftCornerInArUcoFrame{ roiBoxLowerRightCornerInArUcoFrame.x - roiBoxLength,
                                                         roiBoxLowerRightCornerInArUcoFrame.y,
                                                         roiBoxLowerRightCornerInArUcoFrame.z };

C＃

float roiBoxLength = 545F;
float roiBoxWidth = 345F;
float roiBoxHeight = 150F;
// Coordinates are relative to the checkerboard origin which lies in the intersection between the four checkers
// in the top-left corner of the checkerboard: Positive x-axis is "East", y-axis is "South" and z-axis is "Down"
var roiBoxLowerRightCornerInCheckerboardFrame = new Zivid.NET.PointXYZ
{
    x = 240F,
    y = 30F,
    z = 5F
};
var roiBoxUpperRightCornerInCheckerboardFrame = new Zivid.NET.PointXYZ
{
    x = roiBoxLowerRightCornerInCheckerboardFrame.x,
    y = roiBoxLowerRightCornerInCheckerboardFrame.y - roiBoxWidth,
    z = roiBoxLowerRightCornerInCheckerboardFrame.z
};
var roiBoxLowerLeftCornerInCheckerboardFrame = new Zivid.NET.PointXYZ
{
    x = roiBoxLowerRightCornerInCheckerboardFrame.x - roiBoxLength,
    y = roiBoxLowerRightCornerInCheckerboardFrame.y,
    z = roiBoxLowerRightCornerInCheckerboardFrame.z

};

Python

roi_box_length = 545
roi_box_width = 345
roi_box_height = 150
roi_box_lower_right_corner_in_aruco_frame = np.array([240, 30, 5])
roi_box_upper_right_corner_in_aruco_frame = np.array(
    [
        roi_box_lower_right_corner_in_aruco_frame[0],
        roi_box_lower_right_corner_in_aruco_frame[1] - roi_box_width,
        roi_box_lower_right_corner_in_aruco_frame[2],
    ]
)
roi_box_lower_left_corner_in_aruco_frame = np.array(
    [
        roi_box_lower_right_corner_in_aruco_frame[0] - roi_box_length,
        roi_box_lower_right_corner_in_aruco_frame[1],
        roi_box_lower_right_corner_in_aruco_frame[2],
    ]
)

然后可以设置定义ROI盒基坐标系的三个点。

C++

const Zivid::PointXYZ pointOInArUcoFrame = roiBoxLowerRightCornerInArUcoFrame;
const Zivid::PointXYZ pointAInArUcoFrame = roiBoxUpperRightCornerInArUcoFrame;
const Zivid::PointXYZ pointBInArUcoFrame = roiBoxLowerLeftCornerInArUcoFrame;

C＃

var pointOInCheckerboardFrame = roiBoxLowerRightCornerInCheckerboardFrame;
var pointAInCheckerboardFrame = roiBoxUpperRightCornerInCheckerboardFrame;
var pointBInCheckerboardFrame = roiBoxLowerLeftCornerInCheckerboardFrame;

Python

point_o_in_aruco_frame = roi_box_lower_right_corner_in_aruco_frame
point_a_in_aruco_frame = roi_box_upper_right_corner_in_aruco_frame
point_b_in_aruco_frame = roi_box_lower_left_corner_in_aruco_frame

然后我们将配置ArUco标记。

C++

std::cout << "Configuring ArUco marker" << std::endl;
const auto markerDictionary = Zivid::Calibration::MarkerDictionary::aruco4x4_50;
std::vector<int> markerId = { 1 };

C＃

Console.WriteLine("Configuring ArUco marker");
var markerDictionary = Zivid.NET.MarkerDictionary.Aruco4x4_50;
var markerId = new List<int> { 1 };

Python

print("Configuring ArUco marker")
marker_dictionary = zivid.calibration.MarkerDictionary.aruco4x4_50
marker_id = [1]

然后我们检测 ArUco 标记。

C++

std::cout << "Detecting ArUco marker" << std::endl;
const auto detectionResult = Zivid::Calibration::detectMarkers(originalFrame, markerId, markerDictionary);

C＃

Console.WriteLine("Detecting ArUco marker");
var detectionResult = Detector.DetectMarkers(originalFrame, markerId, markerDictionary);

Python

print("Detecting ArUco marker")
detection_result = zivid.calibration.detect_markers(original_frame, marker_id, marker_dictionary)

然后我们需要估计ArUco标记的位姿，将这三个点转换到相机参考系。

C++

std::cout << "Estimating pose of detected ArUco marker" << std::endl;
const auto cameraToMarkerTransform = detectionResult.detectedMarkers()[0].pose().toMatrix();
std::cout << "Transforming the ROI base frame points to the camera frame" << std::endl;
const auto roiPointsInCameraFrame = transformPoints(
    std::vector<Zivid::PointXYZ>{ pointOInArUcoFrame, pointAInArUcoFrame, pointBInArUcoFrame },
    cameraToMarkerTransform);

C＃

Console.WriteLine("Estimating pose of detected ArUco marker");
var cameraToMarkerTransform = new Zivid.NET.Matrix4x4(detectionResult.DetectedMarkers()[0].Pose().ToMatrix());
Console.WriteLine("Transforming the ROI base frame points to the camera frame");
var roiPointsInCameraFrame = TransformPoints(
    new List<Zivid.NET.PointXYZ> { pointOInCheckerboardFrame, pointAInCheckerboardFrame, pointBInCheckerboardFrame },
    cameraToMarkerTransform);

Python

print("Estimating pose of detected ArUco marker")
camera_to_marker_transform = detection_result.detected_markers()[0].pose.to_matrix()
print("Transforming the ROI base frame points to the camera frame")
roi_points_in_camera_frame = _transform_points(
    [point_o_in_aruco_frame, point_a_in_aruco_frame, point_b_in_aruco_frame],
    camera_to_marker_transform,
)

提示

详细了解位置、方向和坐标变换来了解其工作原理。

现在我们可以根据ROI盒的大小和位置过滤点云。将第一个范围（extent）设置为一个较小的负数值来避免过滤掉底板，第二个范围（extent）设置为所需的盒子高度。

C++

settings.set(Zivid::Settings::RegionOfInterest{
    Zivid::Settings::RegionOfInterest::Box::Enabled::yes,
    Zivid::Settings::RegionOfInterest::Box::PointO{ roiPointsInCameraFrame[0] },
    Zivid::Settings::RegionOfInterest::Box::PointA{ roiPointsInCameraFrame[1] },
    Zivid::Settings::RegionOfInterest::Box::PointB{ roiPointsInCameraFrame[2] },
    Zivid::Settings::RegionOfInterest::Box::Extents{ -10, roiBoxHeight } });

C＃

settings.RegionOfInterest.Box.Enabled = true;
settings.RegionOfInterest.Box.PointO = roiPointsInCameraFrame[0];
settings.RegionOfInterest.Box.PointA = roiPointsInCameraFrame[1];
settings.RegionOfInterest.Box.PointB = roiPointsInCameraFrame[2];
settings.RegionOfInterest.Box.Extents = new Zivid.NET.Range<double>(-10, roiBoxHeight);

Python

settings.region_of_interest.box.enabled = True
settings.region_of_interest.box.point_o = roi_points_in_camera_frame[0]
settings.region_of_interest.box.point_a = roi_points_in_camera_frame[1]
settings.region_of_interest.box.point_b = roi_points_in_camera_frame[2]
settings.region_of_interest.box.extents = (-10, roi_box_height)

最后，我们进行新的捕获并可视化过滤后的点云。

C++

const auto roiFrame = camera.capture(settings);

std::cout << "Displaying the ROI-filtered point cloud" << std::endl;
visualizeZividPointCloud(roiFrame);

C＃

var roiFrame = camera.Capture(settings);

Console.WriteLine("Displaying the ROI-filtered point cloud");
VisualizeZividPointCloud(roiFrame);

Python

roi_point_cloud = camera.capture(settings).point_cloud()

print("Displaying the ROI-filtered point cloud")
display_pointcloud(roi_point_cloud.copy_data("xyz"), roi_point_cloud.copy_data("rgba")[:, :, :3])