Resampling(重采样)
Zivid 的一个关键优势是固有的 2D 到 3D 映射。点云以 有序数组 形式返回,并且 RGB 值与其各自的点一一对应。
However, the required resolution of the 3D data is not always the same as the required resolution of the 2D data.
For example, one often requires higher resolution 2D than 3D.
When you capture a separate 2D image and a 3D point cloud, the 2D image may have a higher resolution than the 3D point cloud.
This can be done with the Settings::Sampling::Pixel settings, see Pixel and Pixel(像素).
To maintain the 1-to-1 correspondence between the 2D and 3D data, the 3D data can be resampled to match the resolution of the 2D data.
This can be done with Settings::Processing::Resampling.
重采样可以是上采样或下采样。以下两节将分别讨论这两者。
上采样
通常用于匹配 2D 分辨率,当 3D 分辨率低于 2D 分辨率时使用。
例子
以下示例展示了如何捕获具有不同分辨率的 2D 和 3D 数据,同时保持 2D 和 3D 数据之间的一一对应关系。
2x2 子采样 3D 和全分辨率 2D
以下图片是使用全分辨率 2D 设置和子采样 3D 设置捕获的。3D 数据进一步进行上采样以匹配 2D 图像的分辨率。
auto settings2D = Zivid::Settings2D{
Zivid::Settings2D::Acquisitions{ Zivid::Settings2D::Acquisition{} },
Zivid::Settings2D::Sampling::Pixel::all,
};
auto settings = Zivid::Settings{
Zivid::Settings::Engine::stripe,
Zivid::Settings::Acquisitions{ Zivid::Settings::Acquisition{} },
Zivid::Settings::Sampling::Pixel::blueSubsample2x2,
Zivid::Settings::Sampling::Color::disabled,
Zivid::Settings::Processing::Resampling::Mode::upsample2x2,
};
settings_2d = zivid.Settings2D()
settings_2d.acquisitions.append(zivid.Settings2D.Acquisition())
settings_2d.sampling.pixel = zivid.Settings2D.Sampling.Pixel.all
settings = zivid.Settings()
settings.engine = "stripe"
settings.acquisitions.append(zivid.Settings.Acquisition())
settings.sampling.pixel = zivid.Settings.Sampling.Pixel.blueSubsample2x2
settings.sampling.color = zivid.Settings.Sampling.Color.disabled
settings.processing.resampling.mode = zivid.Settings.Processing.Resampling.Mode.upsample2x2
4x4 子采样 3D 和 2x2 子采样 2D
以下图片是使用 2x2 子采样 2D 设置捕获的。3D 设置是 4x4 子采样,然后进行 2x2 上采样以匹配 2D 图像的分辨率。
auto settings = Zivid::Settings{ Zivid::Settings::Acquisitions{ Zivid::Settings::Acquisition{} },
Zivid::Settings::Color{ Zivid::Settings2D{
Zivid::Settings2D::Acquisitions{ Zivid::Settings2D::Acquisition{} } } } };
auto model = camera.info().model();
switch(model.value())
{
case Zivid::CameraInfo::Model::ValueType::zividTwo:
case Zivid::CameraInfo::Model::ValueType::zividTwoL100:
{
settings.set(Zivid::Settings::Sampling::Pixel::blueSubsample2x2);
settings.set(Zivid::Settings::Processing::Resampling::Mode::upsample2x2);
settings.color().value().set(Zivid::Settings2D::Sampling::Pixel::all);
break;
}
case Zivid::CameraInfo::Model::ValueType::zivid2PlusM130:
case Zivid::CameraInfo::Model::ValueType::zivid2PlusM60:
case Zivid::CameraInfo::Model::ValueType::zivid2PlusL110:
{
settings.set(Zivid::Settings::Sampling::Pixel::blueSubsample4x4);
settings.set(Zivid::Settings::Processing::Resampling::Mode::upsample2x2);
settings.color().value().set(Zivid::Settings2D::Sampling::Pixel::blueSubsample2x2);
break;
}
case Zivid::CameraInfo::Model::ValueType::zivid2PlusMR130:
case Zivid::CameraInfo::Model::ValueType::zivid2PlusMR60:
case Zivid::CameraInfo::Model::ValueType::zivid2PlusLR110:
case Zivid::CameraInfo::Model::ValueType::zivid3XL250:
{
settings.set(Zivid::Settings::Sampling::Pixel::by4x4);
settings.set(Zivid::Settings::Processing::Resampling::Mode::upsample2x2);
settings.color().value().set(Zivid::Settings2D::Sampling::Pixel::by2x2);
break;
}
case Zivid::CameraInfo::Model::ValueType::zividOnePlusSmall:
case Zivid::CameraInfo::Model::ValueType::zividOnePlusMedium:
case Zivid::CameraInfo::Model::ValueType::zividOnePlusLarge:
{
throw std::runtime_error("Unsupported camera model '" + model.toString() + "'");
}
default: throw std::runtime_error("Unhandled enum value '" + model.toString() + "'");
}
settings = zivid.Settings(
acquisitions=[zivid.Settings.Acquisition()],
color=zivid.Settings2D(acquisitions=[zivid.Settings2D.Acquisition()]),
)
model = camera.info.model
if model in {zivid.CameraInfo.Model.zividTwo, zivid.CameraInfo.Model.zividTwoL100}:
settings.sampling.pixel = zivid.Settings.Sampling.Pixel.blueSubsample2x2
settings.processing.resampling.mode = zivid.Settings.Processing.Resampling.Mode.upsample2x2
settings.color.sampling.pixel = zivid.Settings2D.Sampling.Pixel.all
elif model in {
zivid.CameraInfo.Model.zivid2PlusM130,
zivid.CameraInfo.Model.zivid2PlusM60,
zivid.CameraInfo.Model.zivid2PlusL110,
}:
settings.sampling.pixel = zivid.Settings.Sampling.Pixel.blueSubsample4x4
settings.processing.resampling.mode = zivid.Settings.Processing.Resampling.Mode.upsample2x2
settings.color.sampling.pixel = zivid.Settings2D.Sampling.Pixel.blueSubsample2x2
elif model in {
zivid.CameraInfo.Model.zivid2PlusMR130,
zivid.CameraInfo.Model.zivid2PlusMR60,
zivid.CameraInfo.Model.zivid2PlusLR110,
zivid.CameraInfo.Model.zivid3XL250,
}:
settings.sampling.pixel = zivid.Settings.Sampling.Pixel.by4x4
settings.processing.resampling.mode = zivid.Settings.Processing.Resampling.Mode.upsample2x2
settings.color.sampling.pixel = zivid.Settings2D.Sampling.Pixel.by2x2
else:
raise ValueError(f"Unsupported camera model '{model}'")
下采样
通常应用于点云来获得较低分辨率的点云,并且噪声水平较低。更多相关信息,请参阅 下采样 。
使用 Zivid SDK 对点云进行下采样有两种方法:
Via the setting
Settings::Processing::Resampling, which means it's controlled via the capture settings.Via the API
PointCloud::downsample.
通过 Zivid::Settings::Processing::Resampling 进行下采样
当通过设置调用下采样时,由 frame.pointCloud() 返回的点云已经是下采样后的结果。
auto settings = Zivid::Settings{
Zivid::Settings::Engine::stripe,
Zivid::Settings::Acquisitions{ Zivid::Settings::Acquisition{} },
Zivid::Settings::Sampling::Pixel::all,
Zivid::Settings::Processing::Resampling::Mode::downsample2x2,
};
const auto cameraModel = camera.info().model();
if(cameraModel == Zivid::CameraInfo::Model::zivid2PlusM130
|| cameraModel == Zivid::CameraInfo::Model::zivid2PlusM60
|| cameraModel == Zivid::CameraInfo::Model::zivid2PlusL110)
{
// For 2+, we must lower Brightness from the default 2.5 to 2.2, when using `all` mode.
// This code can be removed by changing the Config.yml option 'Camera/Power/Limit'.
for(auto &a : settings.acquisitions())
{
a.set(Zivid::Settings::Acquisition::Brightness{ 2.2 });
}
}
std::cout << "Capturing frame" << std::endl;
const auto frame = camera.capture(settings);
const auto pointCloud = frame.pointCloud();
std::cout << "Getting BGRA image" << std::endl;
const auto image = pointCloud.copyColorsRGBA_SRGB();
const cv::Mat bgra(
image.height(),
image.width(),
CV_8UC4, // NOLINTNEXTLINE(cppcoreguidelines-pro-type-const-cast)
const_cast<void *>(static_cast<const void *>(image.data())));
std::cout << "Visualizing point cloud" << std::endl;
displayPointCloud(pointCloud, bgra);
settings = zivid.Settings()
settings.engine = "stripe"
settings.acquisitions.append(zivid.Settings.Acquisition())
settings.sampling.pixel = zivid.Settings.Sampling.Pixel.all
settings.processing.resampling.mode = zivid.Settings.Processing.Resampling.Mode.downsample2x2
model = camera.info.model
if model in (
zivid.CameraInfo.Model.zivid2PlusM130,
zivid.CameraInfo.Model.zivid2PlusM60,
zivid.CameraInfo.Model.zivid2PlusL110,
):
# For 2+, we must lower Brightness from the default 2.5 to 2.2, when using `all` mode.
# This code can be removed by changing the Config.yml option 'Camera/Power/Limit'.
for acquisition in settings.acquisitions:
acquisition.brightness = 2.2
print("Capturing frame")
with camera.capture(settings) as frame:
point_cloud = frame.point_cloud()
xyz = point_cloud.copy_data("xyz")
rgba = point_cloud.copy_data("rgba_srgb")
print("Visualizing point cloud")
display_pointcloud(xyz, rgba)
通过 Zivid::PointCloud::downsample 进行下采样
The following is identical to the previous example, but the downsampling is invoked on the PointCloud instance returned by frame.pointCloud().
auto settings = Zivid::Settings{
Zivid::Settings::Engine::stripe,
Zivid::Settings::Acquisitions{ Zivid::Settings::Acquisition{} },
Zivid::Settings::Sampling::Pixel{ Zivid::Settings::Sampling::Pixel::all },
};
const auto cameraModel = camera.info().model();
if(cameraModel == Zivid::CameraInfo::Model::zivid2PlusM130
|| cameraModel == Zivid::CameraInfo::Model::zivid2PlusM60
|| cameraModel == Zivid::CameraInfo::Model::zivid2PlusL110)
{
// For 2+, we must lower Brightness from the default 2.5 to 2.2, when using `all` mode.
// This code can be removed by changing the Config.yml option 'Camera/Power/Limit'.
for(auto &a : settings.acquisitions())
{
a.set(Zivid::Settings::Acquisition::Brightness{ 2.2 });
}
}
std::cout << "Capturing frame" << std::endl;
const auto frame = camera.capture(settings);
auto pointCloud = frame.pointCloud();
pointCloud.downsample(Zivid::PointCloud::Downsampling::by2x2);
std::cout << "Getting BGRA image" << std::endl;
const auto image = pointCloud.copyColorsRGBA_SRGB();
const cv::Mat bgra(
image.height(),
image.width(),
CV_8UC4, // NOLINTNEXTLINE(cppcoreguidelines-pro-type-const-cast)
const_cast<void *>(static_cast<const void *>(image.data())));
std::cout << "Visualizing point cloud" << std::endl;
displayPointCloud(pointCloud, bgra);
settings = zivid.Settings()
settings.engine = "stripe"
settings.acquisitions.append(zivid.Settings.Acquisition())
settings.sampling.pixel = zivid.Settings.Sampling.Pixel.all
model = camera.info.model
if model in (
zivid.CameraInfo.Model.zivid2PlusM130,
zivid.CameraInfo.Model.zivid2PlusM60,
zivid.CameraInfo.Model.zivid2PlusL110,
):
# For 2+, we must lower Brightness from the default 2.5 to 2.2, when using `all` mode.
# This code can be removed by changing the Config.yml option 'Camera/Power/Limit'.
for acquisition in settings.acquisitions:
acquisition.brightness = 2.2
print("Capturing frame")
with camera.capture(settings) as frame:
point_cloud = frame.point_cloud()
point_cloud.downsample(zivid.PointCloud.Downsampling.by2x2)
xyz = point_cloud.copy_data("xyz")
rgba = point_cloud.copy_data("rgba_srgb")
print("Visualizing point cloud")
display_pointcloud(xyz, rgba)
下采样可以就地完成,即直接修改当前点云。
还可以将下采样的点云作为新的点云实例,这不会改变现有的点云。