Resampling
One of the key advantages of Zivid is the inherent 2D to 3D mapping. The point clouds are returned in an organized array, and the RGB values correspond 1-to-1 with their respective points.
However, the required resolution of the 3D data is not always the same as the required resolution of the 2D data.
For example, one often requires higher resolution 2D than 3D.
When you capture a separate 2D image and a 3D point cloud, the 2D image may have a higher resolution than the 3D point cloud.
This can be done with the Settings::Sampling::Pixel
settings, see Pixel.
To maintain the 1-to-1 correspondence between the 2D and 3D data, the 3D data can be resampled to match the resolution of the 2D data.
This can be done with Settings::Processing::Resampling
.
Resample can refer to either upsampling or downsampling. Both will be discussed separately in the following two sections.
Upsampling
Typically used to match 2D resolution, if 3D resolution is lower than 2D resolution.
Examples
The following examples show how to capture 2D and 3D data with different resolutions while maintaining the 1-to-1 correspondence between the 2D and 3D data.
2x2 subsampled 3D and full resolution 2D
The following is captured with full-resolution 2D settings, and subsampled 3D settings. The 3D data is further upsampled to match the resolution of the 2D image.
auto settings2D = Zivid::Settings2D{
Zivid::Settings2D::Acquisitions{ Zivid::Settings2D::Acquisition{} },
Zivid::Settings2D::Sampling::Pixel::all,
};
auto settings = Zivid::Settings{
Zivid::Settings::Engine::phase,
Zivid::Settings::Acquisitions{ Zivid::Settings::Acquisition{} },
Zivid::Settings::Sampling::Pixel::blueSubsample2x2,
Zivid::Settings::Sampling::Color::disabled,
Zivid::Settings::Processing::Resampling::Mode::upsample2x2,
};
settings_2d = zivid.Settings2D()
settings_2d.acquisitions.append(zivid.Settings2D.Acquisition())
settings_2d.sampling.pixel = zivid.Settings2D.Sampling.Pixel.all
settings = zivid.Settings()
settings.engine = "phase"
settings.acquisitions.append(zivid.Settings.Acquisition())
settings.sampling.pixel = zivid.Settings.Sampling.Pixel.blueSubsample2x2
settings.sampling.color = zivid.Settings.Sampling.Color.disabled
settings.processing.resampling.mode = zivid.Settings.Processing.Resampling.Mode.upsample2x2
4x4 subsampled 3D and 2x2 subsampled 2D
The following is captured with 2x2 subsampled 2D settings. The 3D settings are 4x4 subsampled and then 2x2 upsampled to match the resolution of the 2D image.
const auto settings2D = Zivid::Settings2D{
Zivid::Settings2D::Acquisitions{ Zivid::Settings2D::Acquisition{} },
Zivid::Settings2D::Sampling::Pixel::blueSubsample2x2,
};
auto settings = Zivid::Settings{
Zivid::Settings::Engine::phase,
Zivid::Settings::Acquisitions{ Zivid::Settings::Acquisition{} },
Zivid::Settings::Sampling::Pixel::blueSubsample4x4,
Zivid::Settings::Sampling::Color::disabled,
Zivid::Settings::Processing::Resampling::Mode::upsample2x2,
};
settings_2d = zivid.Settings2D()
settings_2d.acquisitions.append(zivid.Settings2D.Acquisition())
settings_2d.sampling.pixel = zivid.Settings2D.Sampling.Pixel.blueSubsample2x2
settings = zivid.Settings()
settings.engine = zivid.Settings.Engine.phase
settings.acquisitions.append(zivid.Settings.Acquisition())
settings.sampling.pixel = zivid.Settings.Sampling.Pixel.blueSubsample4x4
settings.sampling.color = zivid.Settings.Sampling.Color.disabled
settings.processing.resampling.mode = zivid.Settings.Processing.Resampling.Mode.upsample2x2
Downsampling
Typically used on a point cloud to get a lower resolution point cloud, with less noise. For more information see Downsampling Theory and Downsample.
There are two ways to downsample a point cloud with Zivid SDK:
Via the setting
Settings::Processing::Resampling
, which means it’s controlled via the capture settings.Via the API
Zivid::PointCloud::downsample
.
Downsample via Zivid::Settings::Processing::Resampling
When downsampling is invoked via settings the point cloud returned by frame.pointCloud()
is already downsampled.
auto settings = Zivid::Settings{
Zivid::Settings::Engine::phase,
Zivid::Settings::Acquisitions{ Zivid::Settings::Acquisition{} },
Zivid::Settings::Sampling::Pixel::all,
Zivid::Settings::Processing::Resampling::Mode::downsample2x2,
};
const auto cameraModel = camera.info().model();
if(cameraModel == Zivid::CameraInfo::Model::zivid2PlusM130
|| cameraModel == Zivid::CameraInfo::Model::zivid2PlusM60
|| cameraModel == Zivid::CameraInfo::Model::zivid2PlusL110)
{
// For 2+, we must lower Brightness from the default 2.5 to 2.2, when using `all` mode.
// This code can be removed by changing the Config.yml option 'Camera/Power/Limit'.
for(auto &a : settings.acquisitions())
{
a.set(Zivid::Settings::Acquisition::Brightness{ 2.2 });
}
}
std::cout << "Capturing frame" << std::endl;
const auto frame = camera.capture(settings);
const auto pointCloud = frame.pointCloud();
std::cout << "Getting BGRA image" << std::endl;
const auto image = pointCloud.copyColorsRGBA();
const cv::Mat bgra(
image.height(),
image.width(),
CV_8UC4, // NOLINTNEXTLINE(cppcoreguidelines-pro-type-const-cast)
const_cast<void *>(static_cast<const void *>(image.data())));
std::cout << "Visualizing point cloud" << std::endl;
displayPointCloud(pointCloud, bgra);
settings = zivid.Settings()
settings.engine = "phase"
settings.acquisitions.append(zivid.Settings.Acquisition())
settings.sampling.pixel = zivid.Settings.Sampling.Pixel.all
settings.processing.resampling.mode = zivid.Settings.Processing.Resampling.Mode.downsample2x2
model = camera.info.model
if model in (
zivid.CameraInfo.Model.zivid2PlusM130,
zivid.CameraInfo.Model.zivid2PlusM60,
zivid.CameraInfo.Model.zivid2PlusL110,
):
# For 2+, we must lower Brightness from the default 2.5 to 2.2, when using `all` mode.
# This code can be removed by changing the Config.yml option 'Camera/Power/Limit'.
for acquisition in settings.acquisitions:
acquisition.brightness = 2.2
print("Capturing frame")
with camera.capture(settings) as frame:
point_cloud = frame.point_cloud()
xyz = point_cloud.copy_data("xyz")
rgba = point_cloud.copy_data("rgba")
print("Visualizing point cloud")
display_pointcloud(xyz, rgba)
Downsample via Zivid::PointCloud::downsample
The following is identical to the previous example, but the downsampling is invoked on the Zivid::PointCloud
instance returned by frame.pointCloud()
.
auto settings = Zivid::Settings{
Zivid::Settings::Engine::phase,
Zivid::Settings::Acquisitions{ Zivid::Settings::Acquisition{} },
Zivid::Settings::Sampling::Pixel{ Zivid::Settings::Sampling::Pixel::all },
};
const auto cameraModel = camera.info().model();
if(cameraModel == Zivid::CameraInfo::Model::zivid2PlusM130
|| cameraModel == Zivid::CameraInfo::Model::zivid2PlusM60
|| cameraModel == Zivid::CameraInfo::Model::zivid2PlusL110)
{
// For 2+, we must lower Brightness from the default 2.5 to 2.2, when using `all` mode.
// This code can be removed by changing the Config.yml option 'Camera/Power/Limit'.
for(auto &a : settings.acquisitions())
{
a.set(Zivid::Settings::Acquisition::Brightness{ 2.2 });
}
}
std::cout << "Capturing frame" << std::endl;
const auto frame = camera.capture(settings);
auto pointCloud = frame.pointCloud();
pointCloud.downsample(Zivid::PointCloud::Downsampling::by2x2);
std::cout << "Getting BGRA image" << std::endl;
const auto image = pointCloud.copyColorsRGBA();
const cv::Mat bgra(
image.height(),
image.width(),
CV_8UC4, // NOLINTNEXTLINE(cppcoreguidelines-pro-type-const-cast)
const_cast<void *>(static_cast<const void *>(image.data())));
std::cout << "Visualizing point cloud" << std::endl;
displayPointCloud(pointCloud, bgra);
settings = zivid.Settings()
settings.engine = "phase"
settings.acquisitions.append(zivid.Settings.Acquisition())
settings.sampling.pixel = zivid.Settings.Sampling.Pixel.all
model = camera.info.model
if model in (
zivid.CameraInfo.Model.zivid2PlusM130,
zivid.CameraInfo.Model.zivid2PlusM60,
zivid.CameraInfo.Model.zivid2PlusL110,
):
# For 2+, we must lower Brightness from the default 2.5 to 2.2, when using `all` mode.
# This code can be removed by changing the Config.yml option 'Camera/Power/Limit'.
for acquisition in settings.acquisitions:
acquisition.brightness = 2.2
print("Capturing frame")
with camera.capture(settings) as frame:
point_cloud = frame.point_cloud()
point_cloud.downsample(zivid.PointCloud.Downsampling.by2x2)
xyz = point_cloud.copy_data("xyz")
rgba = point_cloud.copy_data("rgba")
print("Visualizing point cloud")
display_pointcloud(xyz, rgba)
Downsampling can be done in-place, which modifies the current point cloud.
pointCloud.downsample(Zivid::PointCloud::Downsampling::by2x2);
pointCloud.Downsample(Zivid.NET.PointCloud.Downsampling.By2x2);
point_cloud.downsample(zivid.PointCloud.Downsampling.by2x2)
It is also possible to get the downsampled point cloud as a new point cloud instance, which does not alter the existing point cloud.
auto downsampledPointCloud = pointCloud.downsampled(Zivid::PointCloud::Downsampling::by2x2);
var downsampledPointCloud = pointCloud.Downsampled(Zivid.NET.PointCloud.Downsampling.By2x2);
downsampled_point_cloud = point_cloud.downsampled(zivid.PointCloud.Downsampling.by2x2)