Stitch by Transform Tutorial

이 튜토리얼에서는 여러 대의 카메라를 서로 보정한 결과를 사용하는 방법을 설명합니다. 다중 카메라 변환 결과는 한 포인트 클라우드를 다른 포인트 클라우드의 좌표계로 변환하는 데 사용할 수 있는 변환 행렬입니다. 이는 포인트 클라우드를 연결하는 데 매우 유용한 첫 단계입니다.

Prerequisites

이 튜토리얼을 진행하기 전에 Multi-Camera Calibration Tutorial 를 완료해야 합니다.

연관된 변환 행렬을 로드하고 포인트 클라우드에 매핑합니다.

To apply the correct transformation matrix, we must map it to its corresponding frame. We use the camera or ZDF file’s serial number to find the YAML file with a matching name.

When capturing directly from the camera, we read its serial number and search for a YAML file with that name, then map the transformation matrix to the camera.

C++

source

const auto transformsMappedToCameras =
    getTransformationMatricesFromYAML(transformationMatricesfileList, connectedCameras);

Python

source

transforms_mapped_to_cameras = get_transformation_matrices_from_yaml(args.yaml_files, connected_cameras)

On the other hand, when loading from ZDF files, we need to find a ZDF file from the list of files and extract its serial number from the frame. And then, we search for a YAML file on the same file list that uses that serial number as its name. Then we map a transformation matrix with a Frame. In getTransformedPointClouds we apply the transformation and extend the unorganized point cloud, right away.

C++

source

Zivid::UnorganizedPointCloud getTransformedPointClouds(
    Zivid::UnorganizedPointCloud stitchedPointCloud;
                Zivid::Matrix4x4 transformationMatrixZivid(yamlFileName);
                Zivid::UnorganizedPointCloud currentPointCloud = frame.pointCloud().toUnorganizedPointCloud();
                stitchedPointCloud.extend(currentPointCloud.transform(transformationMatrixZivid));

Python

source

def get_transformed_point_clouds(
    stitched_point_cloud = zivid.UnorganizedPointCloud()
        transformation_matrix = zivid.Matrix4x4(yaml_file)
        current_point_cloud = frame.point_cloud().to_unorganized_point_cloud()
        stitched_point_cloud.extend(current_point_cloud.transform(transformation_matrix))

Apply transformation matrix and stitch transformed point cloud with previous point clouds

변환 행렬을 사용하여 캡처를 매핑했습니다. 이제 변환하고 스티칭할 준비가 되었습니다.

캡처한 뒤 transformsMappedToCameras 에서 transform 을 추출할 수 있습니다. 각 캡처마다 정리되지 않은 포인트 클라우드를 변환하고 확장합니다.

C++

source

for(auto &camera : connectedCameras)
{
    if(pathNotProvided)
    {
        settingsPath = std::string(ZIVID_SAMPLE_DATA_DIR) + "/Settings/" + sanitizedModelName(camera)
                       + "_ManufacturingSpecular.yml";
    }

    std::cout << "Imaging from camera: " << camera.info().serialNumber() << std::endl;
    const auto frame = camera.capture2D3D(Zivid::Settings(settingsPath));
    const auto unorganizedPointCloud = frame.pointCloud().toUnorganizedPointCloud();
    const auto transformationMatrix = transformsMappedToCameras.at(camera.info().serialNumber().toString());
    const auto transformedUnorganizedPointCloud = unorganizedPointCloud.transformed(transformationMatrix);
    stitchedPointCloud.extend(transformedUnorganizedPointCloud);
}

Python

source

for camera in connected_cameras:
    if args.settings_path is not None:
        settings_path = args.settings_path
    else:
        settings_path = (
            get_sample_data_path() / "Settings" / f"{sanitized_model_name(camera)}_ManufacturingSpecular.yml"
        )
    print(f"Imaging from camera: {camera.info.serial_number}")
    frame = camera.capture(zivid.Settings.load(settings_path))
    unorganized_point_cloud = frame.point_cloud().to_unorganized_point_cloud()
    transformation_matrix = transforms_mapped_to_cameras[camera.info.serial_number]
    transformed_unorganized_point_cloud = unorganized_point_cloud.transformed(transformation_matrix)
    stitched_point_cloud.extend(transformed_unorganized_point_cloud)

Optional: refine alignment with local point cloud registration

The transformation matrices from multi-camera calibration are rarely perfect and usually leave small residual alignment errors between the stitched point clouds. These residuals can be reduced by applying Experimental::Toolbox::localPointCloudRegistration() to refine the alignment between overlapping point clouds after the transform-based pre-alignment.

For an explanation of this refinement step applied on top of the pre-alignment based on multi-camera calibration, see Stitching Workflow.

Voxel downsample

시각화하기 전에 스티칭된 포인트 클라우드를 정리합니다.

C++

source

const auto finalPointCloud = stitchedPointCloud.voxelDownsampled(0.5, 1);

Python

source

final_point_cloud = stitched_point_cloud.voxel_downsampled(0.5, 1)

Here we’re asking for voxel size of 0.5 mm. The second parameter is useful if more than 2 cameras are involved. Then it can be used to have a majority vote within a pixel. For more information see Voxel downsample.

Visualize

We use the Visualization::Visualizer to visualize the stitched point cloud.

C++

source

void visualizePointCloud(const Zivid::UnorganizedPointCloud &unorganizedPointCloud)
{
    Zivid::Visualization::Visualizer visualizer;

    visualizer.showMaximized();
    visualizer.show(unorganizedPointCloud);
    visualizer.resetToFit();

    std::cout << "Running visualizer. Blocking until window closes." << std::endl;
    visualizer.run();
}

Save stitched point cloud

마지막으로 선택적으로 포인트 클라우드를 저장할 수 있습니다.

C++

source

const std::string &fileName = stitchedPointCloudFileName;
std::cout << "Saving " << finalPointCloud.size() << " data points to " << fileName << std::endl;

using PLY = Zivid::Experimental::PointCloudExport::FileFormat::PLY;
const auto colorSpace = Zivid::Experimental::PointCloudExport::ColorSpace::sRGB;
Zivid::Experimental::PointCloudExport::exportUnorganizedPointCloud(
    finalPointCloud, PLY{ fileName, PLY::Layout::unordered, colorSpace });

Python

source

print(f"Saving {final_point_cloud.size} data points to {args.output_file}")
export_unorganized_point_cloud(
    final_point_cloud, PLY(str(args.output_file), layout=PLY.Layout.unordered, color_space=ColorSpace.srgb)
)