Resampling(重采样)

Zivid 的一个关键优势是固有的 2D 到 3D 映射。点云以 有序数组 形式返回,并且 RGB 值与其各自的点一一对应。

但是,3D 数据所需的分辨率并不总是与 2D 数据所需的分辨率相同。例如,人们通常需要比 3D 更高的 2D 分辨率。当您捕获单独的 2D 图像和 3D 点云时,2D 图像的分辨率可能高于 3D 点云。这可以通过 Settings::Sampling::Pixel 设置来完成,请参阅 Pixel(像素)

为了保持 2D 和 3D 数据之间的一一对应关系,可以对 3D 数据进行重新采样,以匹配 2D 数据的分辨率。这可以通过 Settings::Processing::Resampling 来完成。

重采样可以是上采样或降采样。以下两节将分别讨论这两者。

上采样

通常用于匹配 2D 分辨率,当 3D 分辨率低于 2D 分辨率时使用。

例子

以下示例展示了如何捕获具有不同分辨率的 2D 和 3D 数据,同时保持 2D 和 3D 数据之间的一一对应关系。

2x2 子采样 3D 和全分辨率 2D

以下图片是使用全分辨率 2D 设置和子采样 3D 设置捕获的。3D 数据进一步进行上采样以匹配 2D 图像的分辨率。

auto settings2D = Zivid::Settings2D{
    Zivid::Settings2D::Acquisitions{ Zivid::Settings2D::Acquisition{} },
    Zivid::Settings2D::Sampling::Pixel::all,
};
auto settings = Zivid::Settings{
    Zivid::Settings::Engine::phase,
    Zivid::Settings::Acquisitions{ Zivid::Settings::Acquisition{} },
    Zivid::Settings::Sampling::Pixel::blueSubsample2x2,
    Zivid::Settings::Sampling::Color::disabled,
    Zivid::Settings::Processing::Resampling::Mode::upsample2x2,
};
settings_2d = zivid.Settings2D()
settings_2d.acquisitions.append(zivid.Settings2D.Acquisition())
settings_2d.sampling.pixel = zivid.Settings2D.Sampling.Pixel.all
settings = zivid.Settings()
settings.engine = "phase"
settings.acquisitions.append(zivid.Settings.Acquisition())
settings.sampling.pixel = zivid.Settings.Sampling.Pixel.blueSubsample2x2
settings.sampling.color = zivid.Settings.Sampling.Color.disabled
settings.processing.resampling.mode = zivid.Settings.Processing.Resampling.Mode.upsample2x2

全分辨率 2D

全分辨率

3D 深度图像

先进行 2x2 子采样,再进行 2x2 上采样,得到深度图像

3D 点云

子采样 2x2 然后上采样 2x2,点云

(旋转视角)

放大区域以更好地展示 2D -> 3D 对应关系
全分辨率
先进行 2x2 子采样,再进行 2x2 上采样,得到深度图像
子采样 2x2 然后上采样 2x2,点云
点云比较

全分辨率

全分辨率

子采样2x2

子采样 2x2

子采样2x2 + 上采样2x2

子采样 2x2 + 上采样 2x2
全分辨率
子采样 2x2
先进行 2x2 子采样,然后进行 2x2 上采样

小心

上采样中的伪影

全分辨率

全分辨率下,上采样点云中存在伪影

子采样2x2

2x2 子采样,其中上采样点云中存在伪影

子采样2x2 + 上采样2x2

可见上采样伪影

注意箱子底部和棋盘格之间的点串。这是由于将3D数据进行上采样以匹配2D数据分辨率而产生的伪影。

4x4 子采样 3D 和 2x2 子采样 2D

以下图片是使用 2x2 子采样 2D 设置捕获的。3D 设置是 4x4 子采样,然后进行 2x2 上采样以匹配 2D 图像的分辨率。

转至源代码

源码

const auto settings2D = Zivid::Settings2D{
    Zivid::Settings2D::Acquisitions{ Zivid::Settings2D::Acquisition{} },
    Zivid::Settings2D::Sampling::Pixel::blueSubsample2x2,
};
auto settings = Zivid::Settings{
    Zivid::Settings::Engine::phase,
    Zivid::Settings::Acquisitions{ Zivid::Settings::Acquisition{} },
    Zivid::Settings::Sampling::Pixel::blueSubsample4x4,
    Zivid::Settings::Sampling::Color::disabled,
    Zivid::Settings::Processing::Resampling::Mode::upsample2x2,
};
转至源代码

源码

settings_2d = zivid.Settings2D()
settings_2d.acquisitions.append(zivid.Settings2D.Acquisition())
settings_2d.sampling.pixel = zivid.Settings2D.Sampling.Pixel.blueSubsample2x2
settings = zivid.Settings()
settings.engine = zivid.Settings.Engine.phase
settings.acquisitions.append(zivid.Settings.Acquisition())
settings.sampling.pixel = zivid.Settings.Sampling.Pixel.blueSubsample4x4
settings.sampling.color = zivid.Settings.Sampling.Color.disabled
settings.processing.resampling.mode = zivid.Settings.Processing.Resampling.Mode.upsample2x2

2x2 子采样 2D

2x2 子采样 2D

3D 深度图像

先进行 4x4 子采样,再进行 2x2 上采样得到的深度图像

3D 点云

先进行 4x4 子采样,再进行 2x2 上采样得到的点云

降采样

通常用于点云,以获取较低分辨率且噪声更少的点云。更多相关信息,请参阅 降采样理论降采样

使用 Zivid SDK 对点云进行降采样有两种方法:

  1. 通过设置 Settings::Processing::Resampling 来控制该功能,这意味着它是通过捕获设置进行控制的。

  2. 通过 API Zivid::PointCloud::downsample 实现。

通过 Zivid::Settings::Processing::Resampling 进行降采样

当通过设置调用降采样时,由 frame.pointCloud() 返回的点云已经是降采样后的结果。

auto settings = Zivid::Settings{
    Zivid::Settings::Engine::phase,
    Zivid::Settings::Acquisitions{ Zivid::Settings::Acquisition{} },
    Zivid::Settings::Sampling::Pixel::all,
    Zivid::Settings::Processing::Resampling::Mode::downsample2x2,
};

const auto cameraModel = camera.info().model();
if(cameraModel == Zivid::CameraInfo::Model::zivid2PlusM130
   || cameraModel == Zivid::CameraInfo::Model::zivid2PlusM60
   || cameraModel == Zivid::CameraInfo::Model::zivid2PlusL110)
{
    // For 2+, we must lower Brightness from the default 2.5 to 2.2, when using `all` mode.
    // This code can be removed by changing the Config.yml option 'Camera/Power/Limit'.
    for(auto &a : settings.acquisitions())
    {
        a.set(Zivid::Settings::Acquisition::Brightness{ 2.2 });
    }
}

std::cout << "Capturing frame" << std::endl;
const auto frame = camera.capture(settings);
const auto pointCloud = frame.pointCloud();
std::cout << "Getting BGRA image" << std::endl;
const auto image = pointCloud.copyColorsRGBA();
const cv::Mat bgra(
    image.height(),
    image.width(),
    CV_8UC4, // NOLINTNEXTLINE(cppcoreguidelines-pro-type-const-cast)
    const_cast<void *>(static_cast<const void *>(image.data())));

std::cout << "Visualizing point cloud" << std::endl;
displayPointCloud(pointCloud, bgra);
settings = zivid.Settings()
settings.engine = "phase"
settings.acquisitions.append(zivid.Settings.Acquisition())
settings.sampling.pixel = zivid.Settings.Sampling.Pixel.all
settings.processing.resampling.mode = zivid.Settings.Processing.Resampling.Mode.downsample2x2

model = camera.info.model
if model in (
    zivid.CameraInfo.Model.zivid2PlusM130,
    zivid.CameraInfo.Model.zivid2PlusM60,
    zivid.CameraInfo.Model.zivid2PlusL110,
):
    # For 2+, we must lower Brightness from the default 2.5 to 2.2, when using `all` mode.
    # This code can be removed by changing the Config.yml option 'Camera/Power/Limit'.
    for acquisition in settings.acquisitions:
        acquisition.brightness = 2.2

print("Capturing frame")
with camera.capture(settings) as frame:
    point_cloud = frame.point_cloud()
    xyz = point_cloud.copy_data("xyz")
    rgba = point_cloud.copy_data("rgba")

    print("Visualizing point cloud")
    display_pointcloud(xyz, rgba)

通过 Zivid::PointCloud::downsample 进行降采样

下面的示例与前面的示例相同,只不过降采样操作是在由 frame.pointCloud() 返回的 Zivid::PointCloud 实例上调用的。

auto settings = Zivid::Settings{
    Zivid::Settings::Engine::phase,
    Zivid::Settings::Acquisitions{ Zivid::Settings::Acquisition{} },
    Zivid::Settings::Sampling::Pixel{ Zivid::Settings::Sampling::Pixel::all },
};

const auto cameraModel = camera.info().model();
if(cameraModel == Zivid::CameraInfo::Model::zivid2PlusM130
   || cameraModel == Zivid::CameraInfo::Model::zivid2PlusM60
   || cameraModel == Zivid::CameraInfo::Model::zivid2PlusL110)
{
    // For 2+, we must lower Brightness from the default 2.5 to 2.2, when using `all` mode.
    // This code can be removed by changing the Config.yml option 'Camera/Power/Limit'.
    for(auto &a : settings.acquisitions())
    {
        a.set(Zivid::Settings::Acquisition::Brightness{ 2.2 });
    }
}

std::cout << "Capturing frame" << std::endl;
const auto frame = camera.capture(settings);
auto pointCloud = frame.pointCloud();
pointCloud.downsample(Zivid::PointCloud::Downsampling::by2x2);
std::cout << "Getting BGRA image" << std::endl;
const auto image = pointCloud.copyColorsRGBA();
const cv::Mat bgra(
    image.height(),
    image.width(),
    CV_8UC4, // NOLINTNEXTLINE(cppcoreguidelines-pro-type-const-cast)
    const_cast<void *>(static_cast<const void *>(image.data())));

std::cout << "Visualizing point cloud" << std::endl;
displayPointCloud(pointCloud, bgra);
settings = zivid.Settings()
settings.engine = "phase"
settings.acquisitions.append(zivid.Settings.Acquisition())
settings.sampling.pixel = zivid.Settings.Sampling.Pixel.all

model = camera.info.model
if model in (
    zivid.CameraInfo.Model.zivid2PlusM130,
    zivid.CameraInfo.Model.zivid2PlusM60,
    zivid.CameraInfo.Model.zivid2PlusL110,
):
    # For 2+, we must lower Brightness from the default 2.5 to 2.2, when using `all` mode.
    # This code can be removed by changing the Config.yml option 'Camera/Power/Limit'.
    for acquisition in settings.acquisitions:
        acquisition.brightness = 2.2

print("Capturing frame")
with camera.capture(settings) as frame:
    point_cloud = frame.point_cloud()
    point_cloud.downsample(zivid.PointCloud.Downsampling.by2x2)
    xyz = point_cloud.copy_data("xyz")
    rgba = point_cloud.copy_data("rgba")

    print("Visualizing point cloud")
    display_pointcloud(xyz, rgba)

降采样可以就地完成,即直接修改当前点云。

转至源代码

源码

pointCloud.downsample(Zivid::PointCloud::Downsampling::by2x2);
转至源代码

源码

pointCloud.Downsample(Zivid.NET.PointCloud.Downsampling.By2x2);
转至源代码

源码

point_cloud.downsample(zivid.PointCloud.Downsampling.by2x2)

还可以将降采样的点云作为新的点云实例,这不会改变现有的点云。

转至源代码

源码

auto downsampledPointCloud = pointCloud.downsampled(Zivid::PointCloud::Downsampling::by2x2);
转至源代码

源码

var downsampledPointCloud = pointCloud.Downsampled(Zivid.NET.PointCloud.Downsampling.By2x2);
转至源代码

源码

downsampled_point_cloud = point_cloud.downsampled(zivid.PointCloud.Downsampling.by2x2)

2D 降采样示例

全分辨率

全分辨率

子采样

四分之一分辨率子采样,放大

降采样

四分之一分辨率降采样,放大
全分辨率
四分之一分辨率子采样,再次放大
四分之一分辨率降采样,再次放大