ROI box filter via Checkerboard

This tutorial demonstrates how to filter the point cloud transformed to the checkerboard coordinate system based on a ROI box given relative to the board. This results in filtering out all the points that are outside of the bin.

Tip

If you are using a Zivid One+ Small camera, the checkerboard might be too large for this application. For that case we have ROI box filter via ArUco marker tutorial.

This tutorial uses the point cloud of a checkerboard displayed in the image below.

We can open the original point cloud in Zivid Studio and inspect it.

Note

The original point cloud is also in Sample Data.

To skip the part about transforming the point cloud, which is also covered in Transform via Checkerboard, go to Filtering based on ROI box.

Transforming the point cloud

First, we load a point cloud of a checkerboard.

C++

source

const auto fileData = std::string(ZIVID_SAMPLE_DATA_DIR) + "/BinWithCalibrationBoard.zdf";
std::cout << "Reading ZDF frame from file: " << fileData << std::endl;
const auto frame = Zivid::Frame(fileData);
auto pointCloud = frame.pointCloud();

Python

source

user_options = _options()
data_file = user_options.zdf_path
print(f"Reading ZDF frame from file: {data_file}")
frame = zivid.Frame(data_file)
point_cloud = frame.point_cloud()

Then we estimate the pose of the checkerboard and we invert it because we need the pose of the camera relative to the checkerboard.

C++

source

std::cout << "Detecting and estimating pose of the Zivid checkerboard in the camera frame" << std::endl;
const auto detectionResult = Zivid::Calibration::detectFeaturePoints(pointCloud);
const auto transformCameraToCheckerboard = detectionResult.pose().toMatrix();
std::cout << "Camera pose in checkerboard frame:" << std::endl;
const auto transformCheckerboardToCamera = transformCameraToCheckerboard.inverse();
std::cout << transformCheckerboardToCamera << std::endl;

Python

source

print("Detecting and estimating pose of the Zivid checkerboard in the camera frame")
detection_result = zivid.calibration.detect_feature_points(point_cloud)
transform_camera_to_checkerboard = detection_result.pose().to_matrix()
print("Camera pose in checkerboard frame:")
transform_checkerboard_to_camera = np.linalg.inv(transform_camera_to_checkerboard)
print(transform_checkerboard_to_camera)

We then transform the point cloud to the checkerboard coordinate system.

C++

source

std::cout << "Transforming point cloud from camera frame to Checkerboard frame" << std::endl;
pointCloud.transform(transformCheckerboardToCamera);

Python

source

print("Transforming point cloud from camera frame to Checkerboard frame")
point_cloud.transform(transform_checkerboard_to_camera)

Hint

Learn more about Position, Orientation and Coordinate Transformations.

Filtering based on ROI box

Now we define the position of the bottom-left corner of the ROI box relative to the checkerboard frame, and the ROI box size.

C++

source

std::cout << "Bottom-Left ROI Box corner:" << std::endl;
const float roiBoxBottomLeftCornerX = -80.F; // Positive is "East"
const float roiBoxBottomLeftCornerY = 280.F; // Positive is "South"
const float roiBoxBottomLeftCornerZ = 5.F;   // Positive is "Down"
std::cout << "X: " << roiBoxBottomLeftCornerX << std::endl
          << "Y: " << roiBoxBottomLeftCornerY << std::endl
          << "Z: " << roiBoxBottomLeftCornerZ << std::endl;
std::cout << "ROI Box size:" << std::endl;
const float roiBoxLength = 600.F;
const float roiBoxWidth = 400.F;
const float roiBoxHeight = 80.F;
std::cout << "Length: " << roiBoxLength << std::endl
          << "Width: " << roiBoxWidth << std::endl
          << "Height: " << roiBoxHeight << std::endl;

Python

source

print("Bottom-Left ROI Box corner:")
roi_box_bottom_left_corner_x = user_options.roi_box_bottom_left_corner_x  # Positive is "East"
roi_box_bottom_left_corner_y = user_options.roi_box_bottom_left_corner_y  # Positive is "South"
roi_box_bottom_left_corner_z = user_options.roi_box_bottom_left_corner_z  # Positive is "Down"
print(f"X: {roi_box_bottom_left_corner_x}")
print(f"Y: {roi_box_bottom_left_corner_y}")
print(f"Z: {roi_box_bottom_left_corner_z}")
print("ROI Box size:")
roi_box_length = user_options.box_dimension_in_axis_x
roi_box_width = user_options.box_dimension_in_axis_y
roi_box_height = user_options.box_dimension_in_axis_z
print(f"Length: {roi_box_length}")
print(f"Width: {roi_box_width}")
print(f"Height: {roi_box_height}")

Now we filter the point cloud based on the size and position of the ROI box. The implementation is done using PCL library.

C++

source

std::cout << "Filtering the point cloud based on ROI Box" << std::endl;
const auto roiPointCloudPCL = roiBoxPointCloud(
    pointCloud,
    roiBoxBottomLeftCornerX,
    roiBoxBottomLeftCornerY,
    roiBoxBottomLeftCornerZ,
    roiBoxLength,
    roiBoxWidth,
    roiBoxHeight);

Python

source

print("Filtering the point cloud based on ROI Box")
filtered_xyz, filtered_rgba = roi_box_point_cloud(
    point_cloud,
    roi_box_bottom_left_corner_x,
    roi_box_bottom_left_corner_y,
    roi_box_bottom_left_corner_z,
    roi_box_length,
    roi_box_width,
    roi_box_height,
)

Finally, we visualize the transformed and filtered point cloud.

C++

source

std::cout << "Displaying transformed point cloud after ROI Box filtering" << std::endl;
visualizePointCloud(roiPointCloudPCL);

Python

source

print("Displaying transformed point cloud after ROI Box filtering")
display_pointcloud(filtered_xyz, filtered_rgba[:, :, 0:3])

Lastly, we display the depth map of the transformed and filtered point cloud.

C++

source

std::cout << "Displaying depth map of the transformed point cloud after ROI Box filtering" << std::endl;
visualizeDepthMap(*roiPointCloudPCL);

Python

source

print("Displaying depth map of the transformed point cloud after ROI Box filtering")
display_depthmap(filtered_xyz)

To filter the point cloud based on a ROI box, you can run our code sample.

C++

Sample: ROIBoxViaCheckerboard.cpp

ROIBoxViaCheckerboard.exe

Python

Sample: roi_box_via_checkerboard.py

python roi_box_via_checkerboard.py

The point cloud used the sample uses by default is the one we show in this tutorial. To filter an arbitrary point cloud based on a user-defined ROI box, run the same sample with the following input arguments:

Python

Sample: roi_box_via_checkerboard.py

python roi_box_via_checkerboard.py --zdf-path /path/to/file.zdf --roi-box-bottom-left-corner-x -80 --roi-box-bottom-left-corner-y 280 --roi-box-bottom-left-corner-z 5 --box-dimension-in-axis-x 600 --box-dimension-in-axis-y 400 --box-dimension-in-axis-z 80 --downsample by2x2