Capture Tutorial

Introduction

This tutorial describes how to use the Zivid SDK to capture point clouds and 2D images.

For MATLAB see Zivid Capture Tutorial for MATLAB.

Tip

If you prefer watching a video, our webinar Making 3D captures easy - A tour of Zivid Studio and Zivid SDK covers the same content as the Capture Tutorial.

Prerequisites

• Install Zivid Software.

• For Python: install zivid-python

Initialize

Calling any of the APIs in the Zivid SDK requires initializing the Zivid application and keeping it alive while the program runs.

Note

Zivid::Application must be kept alive while operating the Zivid Camera. This is essentially the Zivid driver.

C++

source

Zivid::Application zivid;

C#

source

var zivid = new Zivid.NET.Application();

Python

source

app = zivid.Application()

Connect

Now we can connect to the camera.

C++

source

auto camera = zivid.connectCamera();

C#

source

var camera = zivid.ConnectCamera();

Python

source

camera = app.connect_camera()

Specific Camera

Sometimes multiple cameras are connected to the same computer, but it might be necessary to work with a specific camera in the code. This can be done by providing the serial number of the wanted camera.

C++

auto camera = zivid.connectCamera(Zivid::CameraInfo::SerialNumber{ "2020C0DE" });

C#

var camera = zivid.ConnectCamera(new Zivid.NET.CameraInfo.SerialNumber("2020C0DE"));

Python

camera = app.connect_camera(serial_number="2020C0DE")

Note

The serial number of your camera is shown in the Zivid Studio.

You may also list all cameras connected to the computer, and view their serial numbers through

C++

source

auto cameras = zivid.cameras();
std::cout << "Found " << cameras.size() << " cameras" << std::endl;
for(auto &camera : cameras)
{
    std::cout << "Camera Info: " << camera.info() << std::endl;
}

C#

source

var cameras = zivid.Cameras;
Console.WriteLine("Number of cameras found: {0}", cameras.Count);
foreach(var camera in cameras)
{
    Console.WriteLine("Camera Info: {0}", camera.Info);
}

Python

source

cameras = app.cameras()
for camera in cameras:
    print(f"Camera Info:  {camera}")

File Camera

You may want to experiment with the SDK, without access to a physical camera. Minor changes are required to keep the sample working.

C++

source

const auto fileCamera = std::string(ZIVID_SAMPLE_DATA_DIR) + "/FileCameraZividOne.zfc";
auto camera = zivid.createFileCamera(fileCamera);

C#

source

var fileCamera = Environment.GetFolderPath(Environment.SpecialFolder.CommonApplicationData)
                 + "/Zivid/FileCameraZividOne.zfc";
var camera = zivid.CreateFileCamera(fileCamera);

Python

source

file_camera = get_sample_data_path() / "FileCameraZividOne.zfc"
camera = app.create_file_camera(file_camera)

Note

The quality of the point cloud you get from FileCameraZividOne.zfc is not representative of the Zivid 3D cameras.

Configure

As with all cameras there are settings that can be configured. These may be set manually, or you use our Capture Assistant.

Capture Assistant

It can be difficult to know what settings to configure. Luckily we have the Capture Assistant. This is available in the Zivid SDK to help configure camera settings.

C++

source

const auto suggestSettingsParameters = Zivid::CaptureAssistant::SuggestSettingsParameters{
    Zivid::CaptureAssistant::SuggestSettingsParameters::AmbientLightFrequency::none,
    Zivid::CaptureAssistant::SuggestSettingsParameters::MaxCaptureTime{ std::chrono::milliseconds{ 1200 } }
};

std::cout << "Running Capture Assistant with parameters:\n" << suggestSettingsParameters << std::endl;
auto settings = Zivid::CaptureAssistant::suggestSettings(camera, suggestSettingsParameters);

C#

source

var suggestSettingsParameters = new Zivid.NET.CaptureAssistant.SuggestSettingsParameters {
    AmbientLightFrequency =
        Zivid.NET.CaptureAssistant.SuggestSettingsParameters.AmbientLightFrequencyOption.none,
    MaxCaptureTime = Duration.FromMilliseconds(1200)
};

Console.WriteLine("Running Capture Assistant with parameters:\n{0}", suggestSettingsParameters);
var settings = Zivid.NET.CaptureAssistant.Assistant.SuggestSettings(camera, suggestSettingsParameters);

Python

source

suggest_settings_parameters = zivid.capture_assistant.SuggestSettingsParameters(
    max_capture_time=datetime.timedelta(milliseconds=1200),
    ambient_light_frequency=zivid.capture_assistant.SuggestSettingsParameters.AmbientLightFrequency.none,
)

print(f"Running Capture Assistant with parameters: {suggest_settings_parameters}")
settings = zivid.capture_assistant.suggest_settings(camera, suggest_settings_parameters)

There are only two parameters to configure with Capture Assistant:

1. Maximum Capture Time in number of milliseconds.

   1. Minimum capture time is 200 ms. This allows only one acquisition.

   2. The algorithm will combine multiple acquisitions if the budget allows.

   3. The algorithm will attempt to cover as much of the dynamic range in the scene as possible.

   4. A maximum capture time of more than 1 second will get good coverage in most scenarios.

2. Ambient light compensation

   1. May restrict capture assistant to exposure periods that are multiples of the ambient light period.

   2. 60Hz is found in Japan, Americas, Taiwan, South Korea and Philippines.

   3. 50Hz is common in the rest of the world.

Manual configuration

Another option is to configure settings manually. For more information about what each settings does, please see Camera Settings. Note that Zivid Two has a set of standard settings.

Single Acquisition

We can create settings for a single capture.

C++

source

const auto settings =
    Zivid::Settings{ Zivid::Settings::Experimental::Engine::phase,
                     Zivid::Settings::Acquisitions{ Zivid::Settings::Acquisition{
                         Zivid::Settings::Acquisition::Aperture{ 5.66 },
                         Zivid::Settings::Acquisition::ExposureTime{ std::chrono::microseconds{ 6500 } } } },
                     Zivid::Settings::Processing::Filters::Outlier::Removal::Enabled::yes,
                     Zivid::Settings::Processing::Filters::Outlier::Removal::Threshold{ 5.0 } };

C#

source

var settings = new Zivid.NET.Settings {
    Acquisitions = { new Zivid.NET.Settings.Acquisition { Aperture = 5.66,
                                                          ExposureTime =
                                                              Duration.FromMicroseconds(6500) } },
    Processing = { Filters = { Outlier = { Removal = { Enabled = true, Threshold = 5.0 } } } }
};

Python

source

settings = zivid.Settings()
settings.experimental.engine = "phase"
settings.acquisitions.append(zivid.Settings.Acquisition())
settings.acquisitions[0].aperture = 5.66
settings.acquisitions[0].exposure_time = datetime.timedelta(microseconds=6500)
settings.processing.filters.outlier.removal.enabled = True
settings.processing.filters.outlier.removal.threshold = 5.0

Multi Acquisition HDR

We may also create settings to be used in an HDR capture.

C++

source

Zivid::Settings settings;
for(const auto aperture : { 11.31, 5.66, 2.83 })
{
    std::cout << "Adding acquisition with aperture = " << aperture << std::endl;
    const auto acquisitionSettings = Zivid::Settings::Acquisition{
        Zivid::Settings::Acquisition::Aperture{ aperture },
    };
    settings.acquisitions().emplaceBack(acquisitionSettings);
}

C#

source

var settings = new Zivid.NET.Settings();
foreach(var aperture in new double[] { 9.57, 4.76, 2.59 })
{
    Console.WriteLine("Adding acquisition with aperture = " + aperture);
    var acquisitionSettings = new Zivid.NET.Settings.Acquisition { Aperture = aperture };
    settings.Acquisitions.Add(acquisitionSettings);
}

Python

source

settings = zivid.Settings(acquisitions=[zivid.Settings.Acquisition(aperture=fnum) for fnum in (11.31, 5.66, 2.83)])

Fully configured settings are demonstrated below.

C++

source

std::cout << "Configuring processing settings for capture:" << std::endl;
Zivid::Settings settings{
    Zivid::Settings::Experimental::Engine::phase,
    Zivid::Settings::Processing::Filters::Smoothing::Gaussian::Enabled::yes,
    Zivid::Settings::Processing::Filters::Smoothing::Gaussian::Sigma{ 1.5 },
    Zivid::Settings::Processing::Filters::Noise::Removal::Enabled::yes,
    Zivid::Settings::Processing::Filters::Noise::Removal::Threshold{ 7.0 },
    Zivid::Settings::Processing::Filters::Outlier::Removal::Enabled::yes,
    Zivid::Settings::Processing::Filters::Outlier::Removal::Threshold{ 5.0 },
    Zivid::Settings::Processing::Filters::Reflection::Removal::Enabled::yes,
    Zivid::Settings::Processing::Filters::Reflection::Removal::Experimental::Mode::global,
    Zivid::Settings::Processing::Filters::Experimental::ContrastDistortion::Correction::Enabled::yes,
    Zivid::Settings::Processing::Filters::Experimental::ContrastDistortion::Correction::Strength{ 0.4 },
    Zivid::Settings::Processing::Filters::Experimental::ContrastDistortion::Removal::Enabled::no,
    Zivid::Settings::Processing::Filters::Experimental::ContrastDistortion::Removal::Threshold{ 0.5 },
    Zivid::Settings::Processing::Color::Balance::Red{ 1.0 },
    Zivid::Settings::Processing::Color::Balance::Green{ 1.0 },
    Zivid::Settings::Processing::Color::Balance::Blue{ 1.0 },
    Zivid::Settings::Processing::Color::Gamma{ 1.0 },
    Zivid::Settings::Processing::Color::Experimental::Mode::automatic
};
std::cout << settings.processing() << std::endl;

std::cout << "Configuring base acquisition with settings same for all HDR acquisition:" << std::endl;
const auto baseAcquisition = Zivid::Settings::Acquisition{ Zivid::Settings::Acquisition::Brightness{ 1.8 } };
std::cout << baseAcquisition << std::endl;

std::cout << "Configuring acquisition settings different for all HDR acquisitions" << std::endl;
auto exposureValues = getExposureValues(camera);
const std::vector<double> aperture = std::get<0>(exposureValues);
const std::vector<double> gain = std::get<1>(exposureValues);
const std::vector<size_t> exposureTime = std::get<2>(exposureValues);
for(size_t i = 0; i < aperture.size(); ++i)
{
    std::cout << "Acquisition " << i + 1 << ":" << std::endl;
    std::cout << "  Exposure Time: " << exposureTime.at(i) << std::endl;
    std::cout << "  Aperture: " << aperture.at(i) << std::endl;
    std::cout << "  Gain: " << gain.at(i) << std::endl;
    const auto acquisitionSettings = baseAcquisition.copyWith(
        Zivid::Settings::Acquisition::Aperture{ aperture.at(i) },
        Zivid::Settings::Acquisition::Gain{ gain.at(i) },
        Zivid::Settings::Acquisition::ExposureTime{ std::chrono::microseconds{ exposureTime.at(i) } });
    settings.acquisitions().emplaceBack(acquisitionSettings);
}

C#

source

Console.WriteLine("Configuring processing settings for capture:");
var settings = new Zivid.NET.Settings() {
    Experimental = { Engine = Zivid.NET.Settings.ExperimentalGroup.EngineOption.Phase },
    Processing = { Filters = { Smoothing = { Gaussian = { Enabled = true, Sigma = 1.5 } },
                               Noise = { Removal = { Enabled = true, Threshold = 7.0 } },
                               Outlier = { Removal = { Enabled = true, Threshold = 5.0 } },
                               Reflection = { Removal = { Enabled = true, Experimental = { Mode = ReflectionFilterModeOption.Global} } },
                               Experimental = { ContrastDistortion = { Correction = { Enabled = true,
                                                                                      Strength = 0.4 },
                                                                       Removal = { Enabled = true,
                                                                                   Threshold = 0.5 } } } },
                   Color = { Balance = { Red = 1.0, Green = 1.0, Blue = 1.0 },
                             Gamma = 1.0,
                             Experimental = { Mode = ColorModeOption.Automatic } } }
};
Console.WriteLine(settings.Processing);

Console.WriteLine("Configuring base acquisition with settings same for all HDR acquisitions:");
var baseAcquisition = new Zivid.NET.Settings.Acquisition { Brightness = 1.8 };
Console.WriteLine(baseAcquisition);

Console.WriteLine("Configuring acquisition settings different for all HDR acquisitions:");
Tuple<double[], int[], double[]> exposureValues = GetExposureValues(camera);
double[] aperture = exposureValues.Item1;
int[] exposureTime = exposureValues.Item2;
double[] gain = exposureValues.Item3;
for(int i = 0; i < aperture.Length; i++)
{
    Console.WriteLine("Acquisition {0}:", i + 1);
    Console.WriteLine("  Exposure Time: {0}", exposureTime[i]);
    Console.WriteLine("  Aperture: {0}", aperture[i]);
    Console.WriteLine("  Gain: {0}", gain[i]);
    var acquisitionSettings = baseAcquisition.CopyWith(s =>
                                                       {
                                                           s.Aperture = aperture[i];
                                                           s.ExposureTime =
                                                               Duration.FromMicroseconds(exposureTime[i]);
                                                           s.Gain = gain[i];
                                                       });
    settings.Acquisitions.Add(acquisitionSettings);
}

Python

source

print("Configuring processing settings for capture:")
settings = zivid.Settings()
settings.experimental.engine = "phase"
filters = settings.processing.filters
filters.smoothing.gaussian.enabled = True
filters.smoothing.gaussian.sigma = 1.5
filters.noise.removal.enabled = True
filters.noise.removal.threshold = 7.0
filters.outlier.removal.enabled = True
filters.outlier.removal.threshold = 5.0
filters.reflection.removal.enabled = True
filters.reflection.removal.experimental.mode = "global"
filters.experimental.contrast_distortion.correction.enabled = True
filters.experimental.contrast_distortion.correction.strength = 0.4
filters.experimental.contrast_distortion.removal.enabled = False
filters.experimental.contrast_distortion.removal.threshold = 0.5
color = settings.processing.color
color.balance.red = 1.0
color.balance.blue = 1.0
color.balance.green = 1.0
color.gamma = 1.0
settings.processing.color.experimental.mode = "automatic"
print(settings.processing)

print("Configuring acquisition settings different for all HDR acquisitions")
exposure_values = _get_exposure_values(camera)
for (aperture, gain, exposure_time) in exposure_values:
    settings.acquisitions.append(
        zivid.Settings.Acquisition(
            aperture=aperture,
            exposure_time=datetime.timedelta(microseconds=exposure_time),
            brightness=1.8,
            gain=gain,
        )
    )

2D Settings

It is possible to only capture a 2D image. This is faster than a 3D capture. 2D settings are configured as follows.

C++

source

const auto settings2D =
    Zivid::Settings2D{ Zivid::Settings2D::Acquisitions{ Zivid::Settings2D::Acquisition{
                           Zivid::Settings2D::Acquisition::ExposureTime{ std::chrono::microseconds{ 30000 } },
                           Zivid::Settings2D::Acquisition::Aperture{ 11.31 },
                           Zivid::Settings2D::Acquisition::Brightness{ 1.80 },
                           Zivid::Settings2D::Acquisition::Gain{ 2.0 } } },
                       Zivid::Settings2D::Processing::Color::Balance::Red{ 1 },
                       Zivid::Settings2D::Processing::Color::Balance::Green{ 1 },
                       Zivid::Settings2D::Processing::Color::Balance::Blue{ 1 } };

C#

source

var settings2D = new Zivid.NET.Settings2D {
    Acquisitions = { new Zivid.NET.Settings2D.Acquisition {
        Aperture = 11.31, ExposureTime = Duration.FromMicroseconds(30000), Gain = 2.0, Brightness = 1.80
    } },
    Processing = { Color = { Balance = { Red = 1.0, Blue = 1.0, Green = 1.0 } } }
};

Python

source

settings_2d = zivid.Settings2D()
settings_2d.acquisitions.append(zivid.Settings2D.Acquisition())
settings_2d.acquisitions[0].exposure_time = datetime.timedelta(microseconds=30000)
settings_2d.acquisitions[0].aperture = 11.31
settings_2d.acquisitions[0].brightness = 1.80
settings_2d.acquisitions[0].gain = 2.0
settings_2d.processing.color.balance.red = 1.0
settings_2d.processing.color.balance.green = 1.0
settings_2d.processing.color.balance.blue = 1.0
settings_2d.processing.color.gamma = 1.0

Load

Zivid Studio can store the current settings to .yml files. These can be read and applied in the API. You may find it easier to modify the settings in these (human-readable) yaml-files in your preferred editor.

C++

source

const auto settingsFile = "Settings.yml";
std::cout << "Loading settings from file: " << settingsFile << std::endl;
const auto settingsFromFile = Zivid::Settings(settingsFile);

C#

source

var settingsFile = "Settings.yml";
Console.WriteLine("Loading settings from file: " + settingsFile);
var settingsFromFile = new Zivid.NET.Settings(settingsFile);

Python

source

settings_file = "Settings.yml"
print(f"Loading settings from file: {settings_file}")
settings_from_file = zivid.Settings.load(settings_file)

Save

You can also save settings to .yml file.

C++

source

const auto settingsFile = "Settings.yml";
std::cout << "Saving settings to file: " << settingsFile << std::endl;
settings.save(settingsFile);

C#

source

var settingsFile = "Settings.yml";
Console.WriteLine("Saving settings to file: " + settingsFile);
settings.Save(settingsFile);

Python

source

settings_file = "Settings.yml"
print(f"Saving settings to file: {settings_file}")
settings.save(settings_file)

Caution

Zivid settings files must use .yml file extension ( not .yaml).

Capture

Now we can capture a 3D image. Whether there is a single acquisition or multiple acquisitions (HDR) is given by the number of acquisitions in settings.

C++

source

const auto frame = camera.capture(settings);

C#

source

using(var frame = camera.Capture(settings))

Python

source

with camera.capture(settings) as frame:

C++

The Zivid::Frame contains the point cloud and color image (stored on compute device memory) and the capture and camera information.

C#

The Zivid.NET.Frame contains the point cloud and color image (stored on compute device memory) and the capture and camera information.

Python

The zivid.Frame contains the point cloud and color image (stored on compute device memory) and the capture and camera information.

Load

Once saved, the frame can be loaded from a ZDF file.

C++

source

const auto dataFile = std::string(ZIVID_SAMPLE_DATA_DIR) + "/Zivid3D.zdf";
std::cout << "Reading ZDF frame from file: " << dataFile << std::endl;
const auto frame = Zivid::Frame(dataFile);

C#

source

var dataFile =
    Environment.GetFolderPath(Environment.SpecialFolder.CommonApplicationData) + "/Zivid/Zivid3D.zdf";
Console.WriteLine("Reading ZDF frame from file: " + dataFile);
var frame = new Zivid.NET.Frame(dataFile);

Python

source

data_file = get_sample_data_path() / "Zivid3D.zdf"
print(f"Reading point cloud from file: {data_file}")
frame = zivid.Frame(data_file)

Saving to a ZDF file is addressed later in the tutorial.

Capture 2D

If we only want to capture a 2D image, which is faster than 3D, we can do so via the 2D API.

C++

source

const auto frame2D = camera.capture(settings2D);

C#

source

using(var frame2D = camera.Capture(settings2D))

Python

source

with camera.capture(settings_2d) as frame_2d:

Caution

Zivid One+ camera has a time penalty when changing the capture mode (2D and 3D) if the 2D capture settings use brightness > 0. You can read more about it in 2D and 3D switching limitation.

Save

We can now save our results.

C++

source

const auto dataFile = "Frame.zdf";
frame.save(dataFile);

C#

source

var dataFile = "Frame.zdf";
frame.Save(dataFile);

Python

source

data_file = "Frame.zdf"
frame.save(data_file)

Tip

You can open and view Frame.zdf file in Zivid Studio.

Export

The API detects which format to use. See Point Cloud for a list of supported formats. For example, we can export the point cloud to .ply format.

C++

source

const auto dataFilePLY = "PointCloud.ply";
frame.save(dataFilePLY);

C#

source

var dataFilePLY = "PointCloud.ply";
frame.Save(dataFilePLY);

Python

source

data_file_ply = "PointCloud.ply"
frame.save(data_file_ply)

Save 2D

We can get 2D color image from a 3D capture.

C++

source

const auto image = frame.pointCloud().copyImageRGBA();

2D captures also produce 2D color images.

C++

source

const auto image = frame2D.imageRGBA();

C#

source

var image = frame2D.ImageRGBA();

Python

source

image = frame_2d.image_rgba()

Then, we can save the 2D image.

C++

source

const auto imageFile = "Image.png";
std::cout << "Saving 2D color image to file: " << imageFile << std::endl;
image.save(imageFile);

C#

source

var imageFile = "Image.png";
Console.WriteLine("Saving 2D color image to file: {0}", imageFile);
image.Save(imageFile);

Python

source

image_file = "Image.png"
print(f"Saving 2D color image to file: {image_file}")
image.save(image_file)

Conclusion

This tutorial shows how to use the Zivid SDK to connect to, configure, capture, and save from the Zivid camera.