Porting Guide for C++ SDK

Introduction 

This guide focus on changes to the API. The flow follows that of a typical application based on the Zivid SDK. First, an example of the code from 1.8.1 will be shown and then the 2.0 example.

For example:

1.8.1	2.0
You will find code snippets from SDK 1.8.1 on this side,	and code snippets from SDK 2.0 on this side.

Initialize 

No change to how the application is initialized.

Zivid::Application zivid;

Connect 

No change to the default connection.

auto camera = zivid.connectCamera();

Connect - Specific Camera & List Cameras

The serial number is now organized under Zivid::CameraInfo. This is a member of Zivid::Camera which is accessed via Zivid:Camera::info().

1.8.1

2.0

auto camera = zivid.connectCamera(Zivid::SerialNumber{ "2020C0DE" });

auto camera = zivid.connectCamera(Zivid::CameraInfo::SerialNumber("2020C0DE"));

auto cameras = zivid.cameras();

for(auto cam : cameras)
{
      std::cout << "Detected camera: "
          <<cam.serialNumber() << std::endl;
}

auto cameras = zivid.cameras();

for(auto cam : cameras)
{
    std::cout << "Detected camera: "
        << cam.info().serialNumber() << std.endl;
}

Connect - File Camera

1.8.1	2.0
auto zdfFile = "MiscObjects.zdf"; auto camera = zivid.createFileCamera(zdfFile);	const auto cameraFile = "FileCameraZividOne.zfc"; auto camera = zivid.createFileCamera(cameraFile);

The change here is that the File Camera has received its own file type. A Zivid File Camera (ZFC) contains more information than a normal Zivid Data File (ZDF).

Configure 

Settings 3D - Capture Assistant

1.8.1

2.0

const auto suggestSettingsParameters = Zivid::CaptureAssistant::SuggestSettingsParameters{
    std::chrono::milliseconds{ 900 },
    Zivid::CaptureAssistant::AmbientLightFrequency::none
};
const auto settingsVector{ Zivid::CaptureAssistant::suggestSettings(camera, suggestSettingsParameters) };

const auto suggestSettingsParameters = Zivid::CaptureAssistant::SuggestSettingsParameters{
    Zivid::CaptureAssistant::SuggestSettingsParameters::AmbientLightFrequency::none,
    Zivid::CaptureAssistant::SuggestSettingsParameters::MaxCaptureTime{ std::chrono::milliseconds{ 900 } }
};
const auto settings = Zivid::CaptureAssistant::suggestSettings(camera, suggestSettingsParameters);

We use explicit types in the constructor, and all descendants of Zivid::SuggestSettingsParameters are accepted.

Settings 3D - Manual Configuration

Zivid::Settings

Zivid SDK 2.0 introduces major changes to the capture settings. The most significant change is that a single Zivid::Settings object now contains complete settings for any capture task including multi-acquisition capture (HDR). This is in contrast to SDK 1.x, in which an HDR capture was either represented by:

a list of Settings objects, or
a list of Frames that could be merged at a later time

In SDK 2.0, a Settings object is divided into two main parts:

Zivid::Settings::Acquisitions: A list of Zivid::Settings::Acquisition, which specify the parameters of image-acquisition on the Zivid camera itself. Each Acquisition includes settings such as camera aperture (f-number) and projector brightness.
Zivid::Settings::Processing: Specifies the parameters of the point-cloud processing occurring on the PC GPU, based on the images acquired on the camera. This includes filtering settings and color settings.

Secondly, the Settings is no longer given to the camera before capture as a separate operation. In SDK 2.0, settings are passed as an argument to Zivid::Camera::capture(Zivid::Settings). The user may specify as few or as many parts of Zivid::Settings as desired. The remaining camera-model appropriate defaults will be applied at the time of calling capture().

Lastly, 2D-capture is performed by passing a Zivid::Settings2D to the same capture function, Zivid::Camera::capture(Settings2D). The new Settings2D is structured similarly to the new Settings.

Zivid::Settings:Acquisition(s)

In SDK 1, a HDR capture was specified by constructing a list of Settings. In SDK 2.0, everything is contained in a single Settings, and a HDR capture is specified by letting it contain multiple Zivid::Settings::Acquisition. These may be created at the time of constructing the Settings, or added later with e.g. settings.acquisitions().emplaceBack(acquisition). To see an example, see CaptureHDRCompleteSettings sample.

The following is a summary of the relation between members of old settings and the members of the new Zivid::Settings::Acquisition:

`Zivid::Settings::ExposureTime`	→	`Zivid::Settings::Acquisition::ExposureTime`
`Zivid::Settings::Gain`	→	`Zivid::Settings::Acquisition::Gain`
`Zivid::Settings::Brightness`	→	`Zivid::Settings::Acquisition::Brightness`
`Zivid::Settings::Iris`	→	`Zivid::Settings::Acquisition::Aperture` [1]
`Zivid::Settings::Bidirectional`	→	This setting has been removed

Zivid::Settings::Processing

Filters

A new filter Contrast Distortion is introduced, Zivid::Settings::Processing::Experimental::ContrastDistortion.

Note

Contrast Distortion filter does not have anything to do with the old Contrast value.

Filters are applied after acquisitions and are organized under Zivid::Settings::Processing::Filters. The following shows how filters have moved, and in some cases changed.

`Zivid::Settings::Filters::Saturated`	→	This filter is removed. It is always enabled.
`Zivid::Settings::Filter::Gaussian`	→	`Zivid::Settings::Processing::Filters::Smoothing::Gaussian`
`Zivid::Settings::Filter::Outlier`	→	`Zivid::Settings::Processing::Filters::Outlier::Removal`
`Zivid::Settings::Filter::Reflection`	→	`Zivid::Settings::Processing::Filters::Reflection::Removal`
`Zivid::Settings::Contrast`	→	This filter is replaced with a new filter - `Zivid::Settings::Processing::Filters::Noise::Removal` [2]

Color Balance

Color balance is improved with SDK 2.0. Color balance settings now only affects the colors of the point cloud, and has no effect on the computed XYZ point coordinates.

`Zivid::Settings::BlueBalance`	→	`Zivid::Settings::Processing::Color::Balance::Blue`
`Zivid::Settings::RedBalance`	→	`Zivid::Settings::Processing::Color::Balance::Red`
N/A	→	`Zivid::Settings::Processing::Color::Balance::Green` (New)

Example: Single Frame → Single Acquisition in Settings

1.8.1

2.0

camera << Zivid::Settings::Iris{ 20 }
       << Zivid::Settings::ExposureTime{ std::chrono::microseconds{ 10000 } }
       << Zivid::Settings::Brightness{ 1 }
       << Zivid::Settings::Gain{ 1 }
       << Zivid::Settings::Filters::Contrast::Enabled::yes
       << Zivid::Settings::Filters::Contrast::Threshold{ 5 };
const auto frame = camera.capture();

const auto settings = Zivid::Settings{
    Zivid::Settings::Acquisitions{
        Zivid::Settings::Acquisition{ Zivid::Settings::Acquisition::Aperture{ 5.66 },
                                Zivid::Settings::Acquisition::ExposureTime{ std::chrono::microseconds{ 10000 } },
                                Zivid::Settings::Acquisition::Brightness{ 1.0 },
                                Zivid::Settings::Acquisition::Gain{ 1.0 } } },
    Zivid::Settings::Processing::Filters::Noise::Removal::Enabled::yes,
    Zivid::Settings::Processing::Filters::Noise::Removal::Threshold{ 7.0 }
};
const auto frame = camera.capture(settings);

Note how Settings is now provided as an argument to the capture function, not given to the camera beforehand.

Example: HDR Frame → Multiple Acquisitions in Settings

Zivid::Settings now contains a vector to hold settings for multiple acquisitions.

1.8.1

2.0

std::vector<Zivid::Settings> settingsVector;
for(const size_t iris : { 14U, 21U, 35U })
{
    std::cout << "Add settings for frame with iris = " << iris << std::endl;
    auto settings = Zivid::Settings::Settings();
    settings.set(Zivid::Settings::Iris{ iris });
    settingsVector.emplace_back(settings);
}

const auto settings = Zivid::Settings();
for(const auto aperture : { 10.90, 5.80, 2.83 })
{
     const auto acquisitionSettings = Zivid::Settings::Acquisition{
        Zivid::Settings::Acquisitions::Aperture{ aperture },
     }
     settings.acquisitions().emplaceBack(acquisitionSettings);
}

Example: From YML File

Since Zivid::Camera does not hold settings in the new SDK, loading settings from file is done on Zivid::Settings.

1.8.1	2.0
camera.setSettings(Zivid::Settings("Frame01.yml"));	const auto settings = Zivid::Settings("Settings.yml");

Alternatively, while connecting to the camera:

1.8.1	2.0
auto camera = zivid.connectCamera(Zivid::Settings("Frame01.yml"));	N/A

The main change here is that the input file contains settings for all acquisitions. Previously we had one file per Frame. Now we have one file with everything. Note also the name change from Frame to Acquisition in this context.

Complete comparison of old and new .yaml for Zivid::Settings

1.8.1

2.0

__version__: 3
Settings:
    Bidirectional: no
    BlueBalance: 1.081000
    Brightness: 1.000000
    ExposureTime: 10000
    Filters:
        Contrast:
            Enabled: yes
            Threshold: 3.000000
        Gaussian:
            Enabled: yes
            Sigma: 1.500000
        Outlier:
            Enabled: yes
            Threshold: 20.000000
        Reflection:
            Enabled: yes
        Saturated:
            Enabled: yes
    Gain: 1.000000
    Iris: 17
    RedBalance: 1.709000

__version__: 3
Settings:
    Bidirectional: no
    BlueBalance: 1.081000
    Brightness: 1.000000
    ExposureTime: 10000
    Filters:
        Contrast:
            Enabled: yes
            Threshold: 3.000000
        Gaussian:
            Enabled: yes
            Sigma: 1.500000
        Outlier:
            Enabled: yes
            Threshold: 20.000000
        Reflection:
            Enabled: yes
        Saturated:
            Enabled: yes
    Gain: 1.000000
    Iris: 27
    RedBalance: 1.709000

__version__: 3
Settings:
    Bidirectional: no
    BlueBalance: 1.081000
    Brightness: 1.000000
    ExposureTime: 10000
    Filters:
        Contrast:
            Enabled: yes
            Threshold: 3.000000
        Gaussian:
            Enabled: yes
            Sigma: 1.500000
        Outlier:
            Enabled: yes
            Threshold: 20.000000
        Reflection:
            Enabled: yes
        Saturated:
            Enabled: yes
    Gain: 4.000000
    Iris: 35
    RedBalance: 1.709000

__version__:
    serializer: 1
    data: 4
Settings:
    Acquisitions:
        - Acquisition:
              Aperture: 7.98
              Brightness: 1.8
              ExposureTime: 10000
              Gain: 1
        - Acquisition:
              Aperture: 4.02
              Brightness: 1.8
              ExposureTime: 10000
              Gain: 1
        - Acquisition:
              Aperture: 2.81
              Brightness: 1.8
              ExposureTime: 10000
              Gain: 4
    Processing:
        Color:
            Balance:
                Blue: 1.081
                Green: 1
                Red: 1.709
        Filters:
            Experimental:
                ContrastDistortion:
                    Correction:
                        Enabled: no
                        Strength: 0.4
                    Removal:
                        Enabled: no
                        Threshold: 0.5
            Noise:
                Removal:
                    Enabled: yes
                    Threshold: 7
            Outlier:
                Removal:
                    Enabled: yes
                    Threshold: 5
            Reflection:
                Removal:
                    Enabled: yes
            Smoothing:
                Gaussian:
                    Enabled: yes
                    Sigma: 1.5

Settings 2D

The general approach to configure Zivid::Settings2D is equivalent to configuring Zivid::Settings. See Settings 3D - Manual Configuration

Zivid::Settings2D::Acquisition (Acquisition settings - former Frame settings)

The changes made on Zivid::Settings2D::Acquisition are equivalent to the changes made on Zivid::Settings::Acquisition, see Acquisition Settings (3D). Note that 2D settings allow only one acquisition (multi acquisition 2D HDR is not possible).

Zivid::Settings2D::Processing

Note that we don’t support filters for 2D settings.

Color Balance

Color balance is now added for 2D settings. Configuring Zivid::Settings2D::Processing::Color::Balance is the same as configuring Zivid::Settings::Processing::Color::Balance, see Color Balance (3D).

Capture 

3D - Single Acquisition

The new Zivid::Camera::capture() API always takes settings as input.

1.8.1	2.0
auto frame = camera.capture();	auto frame = camera.capture(settings);

3D - Multi Acquisition HDR

Whether the capture is a single acquisition or multi-acquisition (HDR) is given by the input parameter settings. The call signature is always Zivid::Camera::capture(Zivid::Settings), regardless of whether it is single acquisition or HDR.

2D Acquisition

The API to capture 2D image doesn’t have 2D in capture, i.e. it is Zivid::Camera::capture() instead of Zivid::Camera::capture2D(). Whether Zivid::Camera::capture() produces 2D or 3D is given by the input parameter, Zivid::Settings2D or Zivid::Settings.

1.8.1	2.0
auto frame2D = camera.capture2D(settings2D); auto image = frame2D.image<Zivid::RGBA8>();	auto frame2D = camera.capture(settings2D); auto image = frame2D.imageRGBA();

The captured image can be read via frame2D.imageRGBA() as opposed to frame2D.image<Zivid::RGBA8>(). The image is always a two-dimensional array of RGBA pixels with 8-bit red, green, blue and alpha channels.

Point Cloud

Before SDK 2.0 the point cloud was accessed via Zivid::Frame::getPointCloud(). This API copied all data from GPU to system memory in a 1200x1920x7 matrix, see Point Cloud. In SDK 2.0 you first get a handle to the point cloud data on the GPU through Zivid::Frame::pointCloud(). This call does not perform any copying from GPU memory. Then you can selectively copy data based on what is required. You can even copy directly into your own pre-allocated memory.

A new point-cloud quality value has also been introduced: Signal-to-Noise-Ratio (SNR). This replaces the old Contrast value, and is always a non-negative non-NaN number.

Complete list of new output data-formats and how to copy them from the GPU:

Return type	Functions for copying from GPU	Data per pixel	Total data copied
`Zivid::Array2D<Zivid::PointXYZ>`	`PointCloud::copyPointsXYZ()` (or `PointCloud::copyData<Zivid::PointXYZ>()`)	12 bytes	28 MB
`Zivid::Array2D<Zivid::PointXYZW>`	`PointCloud::copyPointsXYZW()` (or `PointCloud::copyData<Zivid::PointXYZW>()`)	16 bytes	37 MB
`Zivid::Array2D<Zivid::PointZ>`	`PointCloud::copyPointsZ()` (or `PointCloud::copyData<Zivid::PointZ>()`)	4 bytes	9 MB
`Zivid::Array2D<Zivid::ColorRGBA>`	`PointCloud::copyColorsRGBA()` (or `PointCloud::copyData<Zivid::ColorRGBA>()`)	4 bytes	9 MB
`Zivid::Array2D<Zivid::SNR>`	`PointCloud::copySNRs()` (or `PointCloud::copyData<Zivid::SNR>()`)	4 bytes	9 MB
`Zivid::Array2D<Zivid::PointXYZColorRGBA>`	`PointCloud::copyData<PointXYZColorRGBA>()`	16 bytes	37 MB
`Zivid::Array2D<Zivid::PointXYZColorBGRA>`	`PointCloud::copyPointsXYZColorsBGRA()` (or `PointCloud::copyData<PointXYZColorBGRA>()`)	16 bytes	37 MB
`Zivid::Image<Zivid::ColorRGBA>`	`PointCloud::copyImageRGBA()`	4 bytes	9 MB

Copy selected data from GPU to system memory (Zivid-allocated)

If a user was only interested in the XYZ coordinates of the point cloud they would previously need to call Zivid::Frame::getPointCloud(). This function would then copy all data to the system memory. In SDK 2.0 this may be achieved faster by only calling PointCloud::copyPointsXYZ(). Similarly, consider a use case where we only need the RGB colors from the point cloud:

1.8.1

2.0

const auto pointCloud = frame.getPointCloud();
cv::Mat rgb(pointCloud.height(), pointCloud.width(),
            CV_8UC3, cv::Scalar(0, 0, 0));

const auto height = pointCloud.height();
const auto width = pointCloud.width();

for(size_t i = 0; i < height; i++)
{
    for(size_t j = 0; j < width; j++)
    {
        auto &color = bgr.at<cv::Vec3b>(i, j);
        color[0] = pointCloud(i, j).red();
        color[1] = pointCloud(i, j).green();
        color[2] = pointCloud(i, j).blue();
    }
}

Line 1: Copy XYZ + RGBA + Contrast data from GPU to system memory.
Line 3: Allocate OpenCV matrix with an appropriate size to hold just the RGB data.
Line 8-17: Selectively copy RGB data from one part of system memory to another.

auto rgba = frame.pointCloud().copyColorsRGBA();
auto *dataPtr = const_cast<void *>(static_cast<const void *>(image.data()));
cv::Mat rgba(rgba.height(), rgba.width(), CV_8UC4, dataPtr);

Line 1: Colors are copied from the GPU and into a Zivid::Array2D<Zivid::ColorRGBA>,

which takes ownership of the data.

Line 2: Cast the data pointer as a void*, since this is what the OpenCV

matrix constructor requires.

Line 3: We wrap this block of data in an OpenCV matrix.

This is possible since the layout of Zivid::ColorRGBA exactly matches

the layout of CV_8UC4. No copying occurs in this step.

Copy selected data from GPU to system memory (user-allocated)

In the above example, ownership of the data was held by the returned Zivid::Array2D<> objects. Alternatively, the user may provide a pre-allocated memory buffer to Zivid::PointCloud::copyData(dataPtr). The type of dataPtr defines what shall be copied (PointXYZ*, ColorRGBA*, etc).

Now let us look at the exact same use case as above. However, this time we let OpenCV allocate the necessary storage, and then we ask the Zivid API to copy data directly from the GPU into this memory location:

1.8.1

2.0

const auto pointCloud = frame.getPointCloud();
cv::Mat rgb(pointCloud.height(), pointCloud.width(),
             CV_8UC3, cv::Scalar(0, 0, 0));

const auto height = pointCloud.height();
const auto width = pointCloud.width();

for(size_t i = 0; i < height; i++)
{
    for(size_t j = 0; j < width; j++)
    {
        auto &color = bgr.at<cv::Vec3b>(i, j);
        color[0] = pointCloud(i, j).red();
        color[1] = pointCloud(i, j).green();
        color[2] = pointCloud(i, j).blue();
    }
}

Line 1: Copy XYZ + RGBA + Contrast data from GPU to system memory.
Line 3: Allocate OpenCV matrix with an appropriate size.
Line 8-17: Selectively copy RGB data from one part of system memory to another.

const auto pointCloud = frame.pointCloud();
auto rgba = cv::Mat(pointCloud.height(), pointCloud.width(), CV_8UC4);
auto *dataPtr = reinterpret_cast<Zivid::ColorRGBA *>(rgba.data);
pointCloud.copyData(dataPtr);

Line 1: Get a handle to the full point cloud on the GPU.

Line 2: Allocate an OpenCV matrix with an appropriate size.

Line 3: Cast the OpenCV data pointer to ColorRGBA* so the Zivid API can

understand which data to copy.

Line 4: Copy RGBA data straight into the OpenCV memory buffer.

Visualize 

The Zivid visualization module is renamed and moved.

`Vis3D` library	→	`Visualization` library
`Zivid::CloudVisualizer`	→	`Zivid::Visualization::Visualizer`
`Zivid::CloudVisualizer::EnableColors`	→	`Zivid::Visualization::Visualizer::ColorsEnabled`
`Zivid::CloudVisualizer::EnableMeshing`	→	`Zivid::Visualization::Visualizer::MeshingEnabled`

1.8.1

2.0

Zivid::CloudVisualizer vis;
zivid.setDefaultComputeDevice(vis.computeDevice());
...
vis.showMaximized();
vis.show(frame);
vis.resetToFit();
vis.run();

Zivid::Visualization::Visualizer visualizer;
...
visualizer.showMaximized();
visualizer.show(frame);
visualizer.resetToFit();
visualizer.run();

Multiple compute devices are not supported simultaneously. Thus the call to setDefaultComputeDevice is redundant.

Save 

3D

No change to how 3D data is saved.

frame.save("Frame");

2D

1.8.1	2.0
frame2D.image<Zivid::RGBA8>().save("Image.png");	frame2D.imageRGBA().save("Image.png");

Misc 

Intrinsics

The intrinsics() function is taken out of the Camera class and placed in the Experimental::Calibration namespace. It now takes Camera as an argument.

Zivid::Camera::intrinsics()

→

Zivid::Experimental::Calibration::intrinsics(camera)

1.8.1	2.0
auto intrinsics = camera.intrinsics();	auto intrinsics = Zivid::Experimental::Calibration::intrinsics(camera);

Info

The member functions on Camera that used to get firmware version, model name, etc. have been removed. Information about the camera is now organized under Zivid::CameraInfo. This is a member of Zivid::Camera, accessed via Zivid::Camera::info().

1.8.1	2.0
auto firmwareVersion = camera.firmwareVersion();	auto firmwareVersion = camera.info().firmwareVersion();
auto modelName = camera.modelName();	auto modelName = camera.info().modelName();
auto majorRevision = camera.revision().majorRevision(); auto minorRevision = camera.revision().minorRevision();	auto majorRevision = camera.info().revision().major(); auto minorRevision = camera.info().revision().minor();
auto serialNumber = camera.serialNumber();	auto serialNumber = camera.info().serialNumber();
auto maxDataSize = camera.userDataMaxSizeBytes();	auto maxDataSize = camera.info().userData().maxSizeBytes().value();