Resampling(重采样)

Zivid 的一个关键优势是固有的 2D 到 3D 映射。点云以 有序数组 形式返回,并且 RGB 值与其各自的点一一对应。

However, the required resolution of the 3D data is not always the same as the required resolution of the 2D data. For example, one often requires higher resolution 2D than 3D. When you capture a separate 2D image and a 3D point cloud, the 2D image may have a higher resolution than the 3D point cloud. This can be done with the Sampling::Pixel settings, see Pixel and Pixel(像素).

为了保持 2D 和 3D 数据之间的一一对应关系,可以对 3D 数据进行重新采样,以匹配 2D 数据的分辨率。这可以通过 Settings::Processing::Resampling 来完成。

重采样可以是上采样或降采样。以下两节将分别讨论这两者。

上采样

通常用于匹配 2D 分辨率,当 3D 分辨率低于 2D 分辨率时使用。

例子

以下示例展示了如何捕获具有不同分辨率的 2D 和 3D 数据,同时保持 2D 和 3D 数据之间的一一对应关系。

2x2 子采样 3D 和全分辨率 2D

以下图片是使用全分辨率 2D 设置和子采样 3D 设置捕获的。3D 数据进一步进行上采样以匹配 2D 图像的分辨率。

auto settings2D = Zivid::Settings2D{
    Zivid::Settings2D::Acquisitions{ Zivid::Settings2D::Acquisition{} },
    Zivid::Settings2D::Sampling::Pixel::all,
};
auto settings = Zivid::Settings{
    Zivid::Settings::Engine::phase,
    Zivid::Settings::Acquisitions{ Zivid::Settings::Acquisition{} },
    Zivid::Settings::Sampling::Pixel::blueSubsample2x2,
    Zivid::Settings::Sampling::Color::disabled,
    Zivid::Settings::Processing::Resampling::Mode::upsample2x2,
};
settings_2d = zivid.Settings2D()
settings_2d.acquisitions.append(zivid.Settings2D.Acquisition())
settings_2d.sampling.pixel = zivid.Settings2D.Sampling.Pixel.all
settings = zivid.Settings()
settings.engine = "phase"
settings.acquisitions.append(zivid.Settings.Acquisition())
settings.sampling.pixel = zivid.Settings.Sampling.Pixel.blueSubsample2x2
settings.sampling.color = zivid.Settings.Sampling.Color.disabled
settings.processing.resampling.mode = zivid.Settings.Processing.Resampling.Mode.upsample2x2

全分辨率 2D

全分辨率

3D 深度图像

先进行 2x2 子采样,再进行 2x2 上采样,得到深度图像

3D 点云

子采样 2x2 然后上采样 2x2,点云

(旋转视角)

放大区域以更好地展示 2D -> 3D 对应关系
全分辨率
先进行 2x2 子采样,再进行 2x2 上采样,得到深度图像
子采样 2x2 然后上采样 2x2,点云
点云比较

全分辨率

全分辨率

子采样2x2

子采样 2x2

子采样2x2 + 上采样2x2

子采样 2x2 + 上采样 2x2
全分辨率
子采样 2x2
先进行 2x2 子采样,然后进行 2x2 上采样

小心

上采样中的伪影

全分辨率

全分辨率下,上采样点云中存在伪影

子采样2x2

2x2 子采样,其中上采样点云中存在伪影

子采样2x2 + 上采样2x2

可见上采样伪影

注意箱子底部和棋盘格之间的点串。这是由于将3D数据进行上采样以匹配2D数据分辨率而产生的伪影。

4x4 子采样 3D 和 2x2 子采样 2D

以下图片是使用 2x2 子采样 2D 设置捕获的。3D 设置是 4x4 子采样,然后进行 2x2 上采样以匹配 2D 图像的分辨率。

转至源代码

source

auto settings2D = Zivid::Settings2D{ Zivid::Settings2D::Acquisitions{ Zivid::Settings2D::Acquisition{} } };
auto settings = Zivid::Settings{ Zivid::Settings::Acquisitions{ Zivid::Settings::Acquisition{} } };

auto model = camera.info().model();
switch(model.value())
{
    case Zivid::CameraInfo::Model::ValueType::zividTwo:
    case Zivid::CameraInfo::Model::ValueType::zividTwoL100:
    {
        settings2D.set(Zivid::Settings2D::Sampling::Pixel::all);
        settings.set(Zivid::Settings::Sampling::Pixel::blueSubsample2x2);
        settings.set(Zivid::Settings::Processing::Resampling::Mode::upsample2x2);
        break;
    }
    case Zivid::CameraInfo::Model::ValueType::zivid2PlusM130:
    case Zivid::CameraInfo::Model::ValueType::zivid2PlusM60:
    case Zivid::CameraInfo::Model::ValueType::zivid2PlusL110:
    {
        settings2D.set(Zivid::Settings2D::Sampling::Pixel::blueSubsample2x2);
        settings.set(Zivid::Settings::Sampling::Pixel::blueSubsample4x4);
        settings.set(Zivid::Settings::Processing::Resampling::Mode::upsample2x2);
        break;
    }
    case Zivid::CameraInfo::Model::ValueType::zivid2PlusMR130:
    case Zivid::CameraInfo::Model::ValueType::zivid2PlusMR60:
    case Zivid::CameraInfo::Model::ValueType::zivid2PlusLR110:
    {
        settings2D.set(Zivid::Settings2D::Sampling::Pixel::by2x2);
        settings.set(Zivid::Settings::Sampling::Pixel::by4x4);
        settings.set(Zivid::Settings::Processing::Resampling::Mode::upsample2x2);
        break;
    }
    case Zivid::CameraInfo::Model::ValueType::zividOnePlusSmall:
    case Zivid::CameraInfo::Model::ValueType::zividOnePlusMedium:
    case Zivid::CameraInfo::Model::ValueType::zividOnePlusLarge:
    {
        throw std::runtime_error("Unsupported camera model '" + model.toString() + "'");
    }
    default: throw std::runtime_error("Unhandled enum value '" + model.toString() + "'");
}
转至源代码

source

settings_2d = zivid.Settings2D(acquisitions=[zivid.Settings2D.Acquisition()])
settings = zivid.Settings(acquisitions=[zivid.Settings.Acquisition()])

model = camera.info.model
if model in [zivid.CameraInfo.Model.zividTwo, zivid.CameraInfo.Model.zividTwoL100]:
    settings_2d.sampling.pixel = zivid.Settings2D.Sampling.Pixel.all
    settings.sampling.pixel = zivid.Settings.Sampling.Pixel.blueSubsample2x2
    settings.processing.resampling.mode = zivid.Settings.Processing.Resampling.Mode.upsample2x2
elif model in [
    zivid.CameraInfo.Model.zivid2PlusM130,
    zivid.CameraInfo.Model.zivid2PlusM60,
    zivid.CameraInfo.Model.zivid2PlusL110,
]:
    settings_2d.sampling.pixel = zivid.Settings2D.Sampling.Pixel.blueSubsample2x2
    settings.sampling.pixel = zivid.Settings.Sampling.Pixel.blueSubsample4x4
    settings.processing.resampling.mode = zivid.Settings.Processing.Resampling.Mode.upsample2x2
elif model in [
    zivid.CameraInfo.Model.zivid2PlusMR130,
    zivid.CameraInfo.Model.zivid2PlusMR60,
    zivid.CameraInfo.Model.zivid2PlusLR110,
]:
    settings_2d.sampling.pixel = zivid.Settings2D.Sampling.Pixel.by2x2
    settings.sampling.pixel = zivid.Settings.Sampling.Pixel.by4x4
    settings.processing.resampling.mode = zivid.Settings.Processing.Resampling.Mode.upsample2x2
else:
    raise ValueError(f"Unsupported camera model '{model}'")

2x2 子采样 2D

2x2 子采样 2D

3D 深度图像

先进行 4x4 子采样,再进行 2x2 上采样得到的深度图像

3D 点云

先进行 4x4 子采样,再进行 2x2 上采样得到的点云

降采样

Typically used on a point cloud to get a lower resolution point cloud, with less noise. For more information see 降采样.

使用 Zivid SDK 对点云进行降采样有两种方法:

  1. 通过设置 Settings::Processing::Resampling 来控制该功能,这意味着它是通过捕获设置进行控制的。

  2. 通过 API Zivid::PointCloud::downsample 实现。

通过 Zivid::Settings::Processing::Resampling 进行降采样

当通过设置调用降采样时,由 frame.pointCloud() 返回的点云已经是降采样后的结果。

auto settings = Zivid::Settings{
    Zivid::Settings::Engine::phase,
    Zivid::Settings::Acquisitions{ Zivid::Settings::Acquisition{} },
    Zivid::Settings::Sampling::Pixel::all,
    Zivid::Settings::Processing::Resampling::Mode::downsample2x2,
};

const auto cameraModel = camera.info().model();
if(cameraModel == Zivid::CameraInfo::Model::zivid2PlusM130
   || cameraModel == Zivid::CameraInfo::Model::zivid2PlusM60
   || cameraModel == Zivid::CameraInfo::Model::zivid2PlusL110)
{
    // For 2+, we must lower Brightness from the default 2.5 to 2.2, when using `all` mode.
    // This code can be removed by changing the Config.yml option 'Camera/Power/Limit'.
    for(auto &a : settings.acquisitions())
    {
        a.set(Zivid::Settings::Acquisition::Brightness{ 2.2 });
    }
}

std::cout << "Capturing frame" << std::endl;
const auto frame = camera.capture(settings);
const auto pointCloud = frame.pointCloud();
std::cout << "Getting BGRA image" << std::endl;
const auto image = pointCloud.copyColorsRGBA();
const cv::Mat bgra(
    image.height(),
    image.width(),
    CV_8UC4, // NOLINTNEXTLINE(cppcoreguidelines-pro-type-const-cast)
    const_cast<void *>(static_cast<const void *>(image.data())));

std::cout << "Visualizing point cloud" << std::endl;
displayPointCloud(pointCloud, bgra);
settings = zivid.Settings()
settings.engine = "phase"
settings.acquisitions.append(zivid.Settings.Acquisition())
settings.sampling.pixel = zivid.Settings.Sampling.Pixel.all
settings.processing.resampling.mode = zivid.Settings.Processing.Resampling.Mode.downsample2x2

model = camera.info.model
if model in (
    zivid.CameraInfo.Model.zivid2PlusM130,
    zivid.CameraInfo.Model.zivid2PlusM60,
    zivid.CameraInfo.Model.zivid2PlusL110,
):
    # For 2+, we must lower Brightness from the default 2.5 to 2.2, when using `all` mode.
    # This code can be removed by changing the Config.yml option 'Camera/Power/Limit'.
    for acquisition in settings.acquisitions:
        acquisition.brightness = 2.2

print("Capturing frame")
with camera.capture(settings) as frame:
    point_cloud = frame.point_cloud()
    xyz = point_cloud.copy_data("xyz")
    rgba = point_cloud.copy_data("rgba")

    print("Visualizing point cloud")
    display_pointcloud(xyz, rgba)

通过 Zivid::PointCloud::downsample 进行降采样

下面的示例与前面的示例相同,只不过降采样操作是在由 frame.pointCloud() 返回的 Zivid::PointCloud 实例上调用的。

auto settings = Zivid::Settings{
    Zivid::Settings::Engine::phase,
    Zivid::Settings::Acquisitions{ Zivid::Settings::Acquisition{} },
    Zivid::Settings::Sampling::Pixel{ Zivid::Settings::Sampling::Pixel::all },
};

const auto cameraModel = camera.info().model();
if(cameraModel == Zivid::CameraInfo::Model::zivid2PlusM130
   || cameraModel == Zivid::CameraInfo::Model::zivid2PlusM60
   || cameraModel == Zivid::CameraInfo::Model::zivid2PlusL110)
{
    // For 2+, we must lower Brightness from the default 2.5 to 2.2, when using `all` mode.
    // This code can be removed by changing the Config.yml option 'Camera/Power/Limit'.
    for(auto &a : settings.acquisitions())
    {
        a.set(Zivid::Settings::Acquisition::Brightness{ 2.2 });
    }
}

std::cout << "Capturing frame" << std::endl;
const auto frame = camera.capture(settings);
auto pointCloud = frame.pointCloud();
pointCloud.downsample(Zivid::PointCloud::Downsampling::by2x2);
std::cout << "Getting BGRA image" << std::endl;
const auto image = pointCloud.copyColorsRGBA();
const cv::Mat bgra(
    image.height(),
    image.width(),
    CV_8UC4, // NOLINTNEXTLINE(cppcoreguidelines-pro-type-const-cast)
    const_cast<void *>(static_cast<const void *>(image.data())));

std::cout << "Visualizing point cloud" << std::endl;
displayPointCloud(pointCloud, bgra);
settings = zivid.Settings()
settings.engine = "phase"
settings.acquisitions.append(zivid.Settings.Acquisition())
settings.sampling.pixel = zivid.Settings.Sampling.Pixel.all

model = camera.info.model
if model in (
    zivid.CameraInfo.Model.zivid2PlusM130,
    zivid.CameraInfo.Model.zivid2PlusM60,
    zivid.CameraInfo.Model.zivid2PlusL110,
):
    # For 2+, we must lower Brightness from the default 2.5 to 2.2, when using `all` mode.
    # This code can be removed by changing the Config.yml option 'Camera/Power/Limit'.
    for acquisition in settings.acquisitions:
        acquisition.brightness = 2.2

print("Capturing frame")
with camera.capture(settings) as frame:
    point_cloud = frame.point_cloud()
    point_cloud.downsample(zivid.PointCloud.Downsampling.by2x2)
    xyz = point_cloud.copy_data("xyz")
    rgba = point_cloud.copy_data("rgba")

    print("Visualizing point cloud")
    display_pointcloud(xyz, rgba)

降采样可以就地完成,即直接修改当前点云。

转至源代码

源码

pointCloud.downsample(Zivid::PointCloud::Downsampling::by2x2);
转至源代码

源码

pointCloud.Downsample(Zivid.NET.PointCloud.Downsampling.By2x2);
转至源代码

源码

point_cloud.downsample(zivid.PointCloud.Downsampling.by2x2)

还可以将降采样的点云作为新的点云实例,这不会改变现有的点云。

转至源代码

源码

auto downsampledPointCloud = pointCloud.downsampled(Zivid::PointCloud::Downsampling::by2x2);
转至源代码

源码

var downsampledPointCloud = pointCloud.Downsampled(Zivid.NET.PointCloud.Downsampling.By2x2);
转至源代码

源码

downsampled_point_cloud = point_cloud.downsampled(zivid.PointCloud.Downsampling.by2x2)

2D 降采样示例

全分辨率

全分辨率

子采样

四分之一分辨率子采样,放大

降采样

四分之一分辨率降采样,放大
全分辨率
四分之一分辨率子采样,再次放大
四分之一分辨率降采样,再次放大