Resampling(重采样)

Zivid 的一个关键优势是固有的 2D 到 3D 映射。点云以 有序数组 形式返回,并且 RGB 值与其各自的点一一对应。

However, the required resolution of the 3D data is not always the same as the required resolution of the 2D data. For example, one often requires higher resolution 2D than 3D. When you capture a separate 2D image and a 3D point cloud, the 2D image may have a higher resolution than the 3D point cloud. This can be done with the Settings::Sampling::Pixel settings, see Pixel and Pixel(像素).

To maintain the 1-to-1 correspondence between the 2D and 3D data, the 3D data can be resampled to match the resolution of the 2D data. This can be done with Settings::Processing::Resampling.

重采样可以是上采样或下采样。以下两节将分别讨论这两者。

上采样

通常用于匹配 2D 分辨率,当 3D 分辨率低于 2D 分辨率时使用。

例子

以下示例展示了如何捕获具有不同分辨率的 2D 和 3D 数据,同时保持 2D 和 3D 数据之间的一一对应关系。

2x2 子采样 3D 和全分辨率 2D

以下图片是使用全分辨率 2D 设置和子采样 3D 设置捕获的。3D 数据进一步进行上采样以匹配 2D 图像的分辨率。

auto settings2D = Zivid::Settings2D{
    Zivid::Settings2D::Acquisitions{ Zivid::Settings2D::Acquisition{} },
    Zivid::Settings2D::Sampling::Pixel::all,
};
auto settings = Zivid::Settings{
    Zivid::Settings::Engine::stripe,
    Zivid::Settings::Acquisitions{ Zivid::Settings::Acquisition{} },
    Zivid::Settings::Sampling::Pixel::blueSubsample2x2,
    Zivid::Settings::Sampling::Color::disabled,
    Zivid::Settings::Processing::Resampling::Mode::upsample2x2,
};
settings_2d = zivid.Settings2D()
settings_2d.acquisitions.append(zivid.Settings2D.Acquisition())
settings_2d.sampling.pixel = zivid.Settings2D.Sampling.Pixel.all
settings = zivid.Settings()
settings.engine = "stripe"
settings.acquisitions.append(zivid.Settings.Acquisition())
settings.sampling.pixel = zivid.Settings.Sampling.Pixel.blueSubsample2x2
settings.sampling.color = zivid.Settings.Sampling.Color.disabled
settings.processing.resampling.mode = zivid.Settings.Processing.Resampling.Mode.upsample2x2

全分辨率 2D

3D 深度图像

3D 点云

子采样 2x2 然后上采样 2x2,点云

(旋转视角)

放大区域以更好地展示 2D -> 3D 对应关系
点云比较

全分辨率

子采样2x2

子采样2x2 + 上采样2x2

小心

上采样中的伪影

全分辨率

子采样2x2

子采样2x2 + 上采样2x2

注意箱子底部和棋盘格之间的点串。这是由于将3D数据进行上采样以匹配2D数据分辨率而产生的伪影。

4x4 子采样 3D 和 2x2 子采样 2D

以下图片是使用 2x2 子采样 2D 设置捕获的。3D 设置是 4x4 子采样,然后进行 2x2 上采样以匹配 2D 图像的分辨率。

跳转到源码

source

auto settings = Zivid::Settings{ Zivid::Settings::Acquisitions{ Zivid::Settings::Acquisition{} },
                                 Zivid::Settings::Color{ Zivid::Settings2D{
                                     Zivid::Settings2D::Acquisitions{ Zivid::Settings2D::Acquisition{} } } } };

auto model = camera.info().model();
switch(model.value())
{
    case Zivid::CameraInfo::Model::ValueType::zividTwo:
    case Zivid::CameraInfo::Model::ValueType::zividTwoL100:
    {
        settings.set(Zivid::Settings::Sampling::Pixel::blueSubsample2x2);
        settings.set(Zivid::Settings::Processing::Resampling::Mode::upsample2x2);
        settings.color().value().set(Zivid::Settings2D::Sampling::Pixel::all);
        break;
    }
    case Zivid::CameraInfo::Model::ValueType::zivid2PlusM130:
    case Zivid::CameraInfo::Model::ValueType::zivid2PlusM60:
    case Zivid::CameraInfo::Model::ValueType::zivid2PlusL110:
    {
        settings.set(Zivid::Settings::Sampling::Pixel::blueSubsample4x4);
        settings.set(Zivid::Settings::Processing::Resampling::Mode::upsample2x2);
        settings.color().value().set(Zivid::Settings2D::Sampling::Pixel::blueSubsample2x2);
        break;
    }
    case Zivid::CameraInfo::Model::ValueType::zivid2PlusMR130:
    case Zivid::CameraInfo::Model::ValueType::zivid2PlusMR60:
    case Zivid::CameraInfo::Model::ValueType::zivid2PlusLR110:
    case Zivid::CameraInfo::Model::ValueType::zivid3XL250:
    {
        settings.set(Zivid::Settings::Sampling::Pixel::by4x4);
        settings.set(Zivid::Settings::Processing::Resampling::Mode::upsample2x2);
        settings.color().value().set(Zivid::Settings2D::Sampling::Pixel::by2x2);
        break;
    }
    case Zivid::CameraInfo::Model::ValueType::zividOnePlusSmall:
    case Zivid::CameraInfo::Model::ValueType::zividOnePlusMedium:
    case Zivid::CameraInfo::Model::ValueType::zividOnePlusLarge:
    {
        throw std::runtime_error("Unsupported camera model '" + model.toString() + "'");
    }
    default: throw std::runtime_error("Unhandled enum value '" + model.toString() + "'");
}
跳转到源码

source

settings = zivid.Settings(
    acquisitions=[zivid.Settings.Acquisition()],
    color=zivid.Settings2D(acquisitions=[zivid.Settings2D.Acquisition()]),
)

model = camera.info.model
if model in {zivid.CameraInfo.Model.zividTwo, zivid.CameraInfo.Model.zividTwoL100}:
    settings.sampling.pixel = zivid.Settings.Sampling.Pixel.blueSubsample2x2
    settings.processing.resampling.mode = zivid.Settings.Processing.Resampling.Mode.upsample2x2
    settings.color.sampling.pixel = zivid.Settings2D.Sampling.Pixel.all
elif model in {
    zivid.CameraInfo.Model.zivid2PlusM130,
    zivid.CameraInfo.Model.zivid2PlusM60,
    zivid.CameraInfo.Model.zivid2PlusL110,
}:
    settings.sampling.pixel = zivid.Settings.Sampling.Pixel.blueSubsample4x4
    settings.processing.resampling.mode = zivid.Settings.Processing.Resampling.Mode.upsample2x2
    settings.color.sampling.pixel = zivid.Settings2D.Sampling.Pixel.blueSubsample2x2
elif model in {
    zivid.CameraInfo.Model.zivid2PlusMR130,
    zivid.CameraInfo.Model.zivid2PlusMR60,
    zivid.CameraInfo.Model.zivid2PlusLR110,
    zivid.CameraInfo.Model.zivid3XL250,
}:
    settings.sampling.pixel = zivid.Settings.Sampling.Pixel.by4x4
    settings.processing.resampling.mode = zivid.Settings.Processing.Resampling.Mode.upsample2x2
    settings.color.sampling.pixel = zivid.Settings2D.Sampling.Pixel.by2x2
else:
    raise ValueError(f"Unsupported camera model '{model}'")

2x2 子采样 2D

3D 深度图像

3D 点云

下采样

通常应用于点云来获得较低分辨率的点云,并且噪声水平较低。更多相关信息,请参阅 下采样

使用 Zivid SDK 对点云进行下采样有两种方法:

  1. Via the setting Settings::Processing::Resampling, which means it's controlled via the capture settings.

  2. Via the API PointCloud::downsample.

通过 Zivid::Settings::Processing::Resampling 进行下采样

当通过设置调用下采样时,由 frame.pointCloud() 返回的点云已经是下采样后的结果。

auto settings = Zivid::Settings{
    Zivid::Settings::Engine::stripe,
    Zivid::Settings::Acquisitions{ Zivid::Settings::Acquisition{} },
    Zivid::Settings::Sampling::Pixel::all,
    Zivid::Settings::Processing::Resampling::Mode::downsample2x2,
};

const auto cameraModel = camera.info().model();
if(cameraModel == Zivid::CameraInfo::Model::zivid2PlusM130
   || cameraModel == Zivid::CameraInfo::Model::zivid2PlusM60
   || cameraModel == Zivid::CameraInfo::Model::zivid2PlusL110)
{
    // For 2+, we must lower Brightness from the default 2.5 to 2.2, when using `all` mode.
    // This code can be removed by changing the Config.yml option 'Camera/Power/Limit'.
    for(auto &a : settings.acquisitions())
    {
        a.set(Zivid::Settings::Acquisition::Brightness{ 2.2 });
    }
}

std::cout << "Capturing frame" << std::endl;
const auto frame = camera.capture(settings);
const auto pointCloud = frame.pointCloud();
std::cout << "Getting BGRA image" << std::endl;
const auto image = pointCloud.copyColorsRGBA_SRGB();
const cv::Mat bgra(
    image.height(),
    image.width(),
    CV_8UC4, // NOLINTNEXTLINE(cppcoreguidelines-pro-type-const-cast)
    const_cast<void *>(static_cast<const void *>(image.data())));

std::cout << "Visualizing point cloud" << std::endl;
displayPointCloud(pointCloud, bgra);
settings = zivid.Settings()
settings.engine = "stripe"
settings.acquisitions.append(zivid.Settings.Acquisition())
settings.sampling.pixel = zivid.Settings.Sampling.Pixel.all
settings.processing.resampling.mode = zivid.Settings.Processing.Resampling.Mode.downsample2x2

model = camera.info.model
if model in (
    zivid.CameraInfo.Model.zivid2PlusM130,
    zivid.CameraInfo.Model.zivid2PlusM60,
    zivid.CameraInfo.Model.zivid2PlusL110,
):
    # For 2+, we must lower Brightness from the default 2.5 to 2.2, when using `all` mode.
    # This code can be removed by changing the Config.yml option 'Camera/Power/Limit'.
    for acquisition in settings.acquisitions:
        acquisition.brightness = 2.2

print("Capturing frame")
with camera.capture(settings) as frame:
    point_cloud = frame.point_cloud()
    xyz = point_cloud.copy_data("xyz")
    rgba = point_cloud.copy_data("rgba_srgb")

    print("Visualizing point cloud")
    display_pointcloud(xyz, rgba)

通过 Zivid::PointCloud::downsample 进行下采样

The following is identical to the previous example, but the downsampling is invoked on the PointCloud instance returned by frame.pointCloud().

auto settings = Zivid::Settings{
    Zivid::Settings::Engine::stripe,
    Zivid::Settings::Acquisitions{ Zivid::Settings::Acquisition{} },
    Zivid::Settings::Sampling::Pixel{ Zivid::Settings::Sampling::Pixel::all },
};

const auto cameraModel = camera.info().model();
if(cameraModel == Zivid::CameraInfo::Model::zivid2PlusM130
   || cameraModel == Zivid::CameraInfo::Model::zivid2PlusM60
   || cameraModel == Zivid::CameraInfo::Model::zivid2PlusL110)
{
    // For 2+, we must lower Brightness from the default 2.5 to 2.2, when using `all` mode.
    // This code can be removed by changing the Config.yml option 'Camera/Power/Limit'.
    for(auto &a : settings.acquisitions())
    {
        a.set(Zivid::Settings::Acquisition::Brightness{ 2.2 });
    }
}

std::cout << "Capturing frame" << std::endl;
const auto frame = camera.capture(settings);
auto pointCloud = frame.pointCloud();
pointCloud.downsample(Zivid::PointCloud::Downsampling::by2x2);
std::cout << "Getting BGRA image" << std::endl;
const auto image = pointCloud.copyColorsRGBA_SRGB();
const cv::Mat bgra(
    image.height(),
    image.width(),
    CV_8UC4, // NOLINTNEXTLINE(cppcoreguidelines-pro-type-const-cast)
    const_cast<void *>(static_cast<const void *>(image.data())));

std::cout << "Visualizing point cloud" << std::endl;
displayPointCloud(pointCloud, bgra);
settings = zivid.Settings()
settings.engine = "stripe"
settings.acquisitions.append(zivid.Settings.Acquisition())
settings.sampling.pixel = zivid.Settings.Sampling.Pixel.all

model = camera.info.model
if model in (
    zivid.CameraInfo.Model.zivid2PlusM130,
    zivid.CameraInfo.Model.zivid2PlusM60,
    zivid.CameraInfo.Model.zivid2PlusL110,
):
    # For 2+, we must lower Brightness from the default 2.5 to 2.2, when using `all` mode.
    # This code can be removed by changing the Config.yml option 'Camera/Power/Limit'.
    for acquisition in settings.acquisitions:
        acquisition.brightness = 2.2

print("Capturing frame")
with camera.capture(settings) as frame:
    point_cloud = frame.point_cloud()
    point_cloud.downsample(zivid.PointCloud.Downsampling.by2x2)
    xyz = point_cloud.copy_data("xyz")
    rgba = point_cloud.copy_data("rgba_srgb")

    print("Visualizing point cloud")
    display_pointcloud(xyz, rgba)

下采样可以就地完成,即直接修改当前点云。

跳转到源码

source

pointCloud.downsample(Zivid::PointCloud::Downsampling::by2x2);
跳转到源码

source

pointCloud.Downsample(Zivid.NET.PointCloud.Downsampling.By2x2);
跳转到源码

source

point_cloud.downsample(zivid.PointCloud.Downsampling.by2x2)

还可以将下采样的点云作为新的点云实例,这不会改变现有的点云。

跳转到源码

source

auto downsampledPointCloud = pointCloud.downsampled(Zivid::PointCloud::Downsampling::by2x2);
跳转到源码

source

var downsampledPointCloud = pointCloud.Downsampled(Zivid.NET.PointCloud.Downsampling.By2x2);
跳转到源码

source

downsampled_point_cloud = point_cloud.downsampled(zivid.PointCloud.Downsampling.by2x2)

2D 下采样示例

全分辨率

子采样

下采样