转换拼接教程

本教程介绍如何使用多台相机相互校准的结果。多相机转换的结果是一个变换矩阵，可用于将一个点云转换到另一个点云的坐标系中。这是拼接点云的第一步。

先决条件

在学习本教程之前，您应该已经完成了 多相机标定教程 。

加载相关变换矩阵并映射到点云

To apply the correct transformation matrix, we must map it to its corresponding frame. We use the camera or ZDF file's serial number to find the YAML file with a matching name.

When capturing directly from the camera, we read its serial number and search for a YAML file with that name, then map the transformation matrix to the camera.

C++

source

const auto transformsMappedToCameras =
    getTransformationMatricesFromYAML(transformationMatricesfileList, connectedCameras);

Python

source

transforms_mapped_to_cameras = get_transformation_matrices_from_yaml(args.yaml_files, connected_cameras)

On the other hand, when loading from ZDF files, we need to find a ZDF file from the list of files and extract its serial number from the frame. And then, we search for a YAML file on the same file list that uses that serial number as its name. Then we map a transformation matrix with a Frame. In getTransformedPointClouds we apply the transformation and extend the unorganized point cloud, right away.

C++

source

Zivid::UnorganizedPointCloud getTransformedPointClouds(
    Zivid::UnorganizedPointCloud stitchedPointCloud;
                Zivid::Matrix4x4 transformationMatrixZivid(yamlFileName);
                Zivid::UnorganizedPointCloud currentPointCloud = frame.pointCloud().toUnorganizedPointCloud();
                stitchedPointCloud.extend(currentPointCloud.transform(transformationMatrixZivid));

Python

source

def get_transformed_point_clouds(
    stitched_point_cloud = zivid.UnorganizedPointCloud()
        transformation_matrix = zivid.Matrix4x4(yaml_file)
        current_point_cloud = frame.point_cloud().to_unorganized_point_cloud()
        stitched_point_cloud.extend(current_point_cloud.transform(transformation_matrix))

Apply transformation matrix and stitch transformed point cloud with previous point clouds

我们已经用变换矩阵映射了捕获内容。现在，我们可以进行变换和拼接了。

我们可以从 transform 中捕获并提取校正 transformsMappedToCameras 。每次捕获时，我们都会转换并扩展无序点云。

C++

source

for(auto &camera : connectedCameras)
{
    if(pathNotProvided)
    {
        settingsPath = std::string(ZIVID_SAMPLE_DATA_DIR) + "/Settings/" + sanitizedModelName(camera)
                       + "_ManufacturingSpecular.yml";
    }

    std::cout << "Imaging from camera: " << camera.info().serialNumber() << std::endl;
    const auto frame = camera.capture2D3D(Zivid::Settings(settingsPath));
    const auto unorganizedPointCloud = frame.pointCloud().toUnorganizedPointCloud();
    const auto transformationMatrix = transformsMappedToCameras.at(camera.info().serialNumber().toString());
    const auto transformedUnorganizedPointCloud = unorganizedPointCloud.transformed(transformationMatrix);
    stitchedPointCloud.extend(transformedUnorganizedPointCloud);
}

Python

source

for camera in connected_cameras:
    if args.settings_path is not None:
        settings_path = args.settings_path
    else:
        settings_path = (
            get_sample_data_path() / "Settings" / f"{sanitized_model_name(camera)}_ManufacturingSpecular.yml"
        )
    print(f"Imaging from camera: {camera.info.serial_number}")
    frame = camera.capture(zivid.Settings.load(settings_path))
    unorganized_point_cloud = frame.point_cloud().to_unorganized_point_cloud()
    transformation_matrix = transforms_mapped_to_cameras[camera.info.serial_number]
    transformed_unorganized_point_cloud = unorganized_point_cloud.transformed(transformation_matrix)
    stitched_point_cloud.extend(transformed_unorganized_point_cloud)

Optional: refine alignment with local point cloud registration

The transformation matrices from multi-camera calibration are rarely perfect and usually leave small residual alignment errors between the stitched point clouds. These residuals can be reduced by applying Experimental::Toolbox::localPointCloudRegistration() to refine the alignment between overlapping point clouds after the transform-based pre-alignment.

For an explanation of this refinement step applied on top of the pre-alignment based on multi-camera calibration, see 拼接工作流程.

体素下采样

在可视化之前清理拼接的点云：

C++

source

const auto finalPointCloud = stitchedPointCloud.voxelDownsampled(0.5, 1);

Python

source

final_point_cloud = stitched_point_cloud.voxel_downsampled(0.5, 1)

Here we're asking for voxel size of 0.5 mm. The second parameter is useful if more than 2 cameras are involved. Then it can be used to have a majority vote within a pixel. For more information see 体素下采样.

可视化

We use the Visualization::Visualizer to visualize the stitched point cloud.

C++

source

void visualizePointCloud(const Zivid::UnorganizedPointCloud &unorganizedPointCloud)
{
    Zivid::Visualization::Visualizer visualizer;

    visualizer.showMaximized();
    visualizer.show(unorganizedPointCloud);
    visualizer.resetToFit();

    std::cout << "Running visualizer. Blocking until window closes." << std::endl;
    visualizer.run();
}

保存拼接后的点云

最后我们可以选择保存点云：

C++

source

const std::string &fileName = stitchedPointCloudFileName;
std::cout << "Saving " << finalPointCloud.size() << " data points to " << fileName << std::endl;

using PLY = Zivid::Experimental::PointCloudExport::FileFormat::PLY;
const auto colorSpace = Zivid::Experimental::PointCloudExport::ColorSpace::sRGB;
Zivid::Experimental::PointCloudExport::exportUnorganizedPointCloud(
    finalPointCloud, PLY{ fileName, PLY::Layout::unordered, colorSpace });

Python

source

print(f"Saving {final_point_cloud.size} data points to {args.output_file}")
export_unorganized_point_cloud(
    final_point_cloud, PLY(str(args.output_file), layout=PLY.Layout.unordered, color_space=ColorSpace.srgb)
)