Capture Tutorial
Introduction
This tutorial describes how to use the Zivid SDK to capture point clouds and 2D images.
For MATLAB see Zivid Capture Tutorial for MATLAB.
Tip
If you prefer watching a video, our webinar Making 3D captures easy - A tour of Zivid Studio and Zivid SDK covers the same content as the Capture Tutorial.
Prerequisites
Install Zivid Software.
For Python: install zivid-python
Initialize
Calling any of the APIs in the Zivid SDK requires initializing the Zivid application and keeping it alive while the program runs.
Note
Zivid::Application
must be kept alive while operating the Zivid Camera.
This is essentially the Zivid driver.
Zivid::Application zivid;
var zivid = new Zivid.NET.Application();
app = zivid.Application()
Connect
Now we can connect to the camera.
auto camera = zivid.connectCamera();
var camera = zivid.ConnectCamera();
camera = app.connect_camera()
Specific Camera
Sometimes multiple cameras are connected to the same computer, but it might be necessary to work with a specific camera in the code. This can be done by providing the serial number of the wanted camera.
auto camera = zivid.connectCamera(Zivid::CameraInfo::SerialNumber{ "2020C0DE" });
var camera = zivid.ConnectCamera(new Zivid.NET.CameraInfo.SerialNumber("2020C0DE"));
camera = app.connect_camera(serial_number="2020C0DE")
Note
The serial number of your camera is shown in the Zivid Studio.
You may also list all cameras connected to the computer, and view their serial numbers through
auto cameras = zivid.cameras();
std::cout << "Found " << cameras.size() << " cameras" << std::endl;
for(auto &camera : cameras)
{
std::cout << "Camera Info: " << camera.info() << std::endl;
}
var cameras = zivid.Cameras;
Console.WriteLine("Number of cameras found: {0}", cameras.Count);
foreach (var camera in cameras)
{
Console.WriteLine("Camera Info: {0}", camera.Info);
}
cameras = app.cameras()
for camera in cameras:
print(f"Camera Info: {camera}")
File Camera
The file camera option allows you to experiment with the SDK without access to a physical camera. The file cameras can be found in Sample Data where there are multiple file cameras to choose from. Each file camera demonstrates a use case within one of the main applications of the respective camera model. The example below shows how to create a file camera using the Zivid Two M70 file camera from Sample Data.
const auto fileCamera =
userInput ? fileCameraPath : std::string(ZIVID_SAMPLE_DATA_DIR) + "/FileCameraZividTwoM70.zfc";
fileCamera = Environment.GetFolderPath(Environment.SpecialFolder.CommonApplicationData) + "/Zivid/FileCameraZividTwoM70.zfc";
default=get_sample_data_path() / "FileCameraZividTwoM70.zfc",
auto camera = zivid.createFileCamera(fileCamera);
var camera = zivid.CreateFileCamera(fileCamera);
camera = app.create_file_camera(file_camera)
The acquisition settings should be initialized like shown below, but you are free to alter the processing settings.
const auto settings =
Zivid::Settings{ Zivid::Settings::Acquisitions{ Zivid::Settings::Acquisition{} },
Zivid::Settings::Processing::Filters::Smoothing::Gaussian::Enabled::yes,
Zivid::Settings::Processing::Filters::Smoothing::Gaussian::Sigma{ 1.5 },
Zivid::Settings::Processing::Filters::Reflection::Removal::Enabled::yes,
Zivid::Settings::Processing::Filters::Reflection::Removal::Experimental::Mode::global,
Zivid::Settings::Processing::Color::Balance::Red{ 1 },
Zivid::Settings::Processing::Color::Balance::Green{ 1 },
Zivid::Settings::Processing::Color::Balance::Blue{ 1 } };
var settings = new Zivid.NET.Settings
{
Acquisitions = { new Zivid.NET.Settings.Acquisition { } },
Processing = { Filters = { Smoothing = { Gaussian = { Enabled = true, Sigma = 1.5 } },
Reflection = { Removal = { Enabled = true, Experimental = { Mode = ReflectionFilterModeOption.Global} } } },
Color = { Balance = { Red = 1.0, Green = 1.0, Blue = 1.0 } } }
};
settings = zivid.Settings()
settings.acquisitions.append(zivid.Settings.Acquisition())
settings.processing.filters.smoothing.gaussian.enabled = True
settings.processing.filters.smoothing.gaussian.sigma = 1
settings.processing.filters.reflection.removal.enabled = True
settings.processing.filters.reflection.removal.experimental.mode = "global"
settings.processing.color.balance.red = 1.0
settings.processing.color.balance.green = 1.0
settings.processing.color.balance.blue = 1.0
You can read more about the file camera option in File Camera.
Configure
As with all cameras there are settings that can be configured. These may be set manually, or you use our Capture Assistant.
Capture Assistant
It can be difficult to know what settings to configure. Luckily we have the Capture Assistant. This is available in the Zivid SDK to help configure camera settings.
const auto suggestSettingsParameters = Zivid::CaptureAssistant::SuggestSettingsParameters{
Zivid::CaptureAssistant::SuggestSettingsParameters::AmbientLightFrequency::none,
Zivid::CaptureAssistant::SuggestSettingsParameters::MaxCaptureTime{ std::chrono::milliseconds{ 1200 } }
};
std::cout << "Running Capture Assistant with parameters:\n" << suggestSettingsParameters << std::endl;
auto settings = Zivid::CaptureAssistant::suggestSettings(camera, suggestSettingsParameters);
var suggestSettingsParameters = new Zivid.NET.CaptureAssistant.SuggestSettingsParameters
{
AmbientLightFrequency =
Zivid.NET.CaptureAssistant.SuggestSettingsParameters.AmbientLightFrequencyOption.none,
MaxCaptureTime = Duration.FromMilliseconds(1200)
};
Console.WriteLine("Running Capture Assistant with parameters:\n{0}", suggestSettingsParameters);
var settings = Zivid.NET.CaptureAssistant.Assistant.SuggestSettings(camera, suggestSettingsParameters);
suggest_settings_parameters = zivid.capture_assistant.SuggestSettingsParameters(
max_capture_time=datetime.timedelta(milliseconds=1200),
ambient_light_frequency=zivid.capture_assistant.SuggestSettingsParameters.AmbientLightFrequency.none,
)
print(f"Running Capture Assistant with parameters: {suggest_settings_parameters}")
settings = zivid.capture_assistant.suggest_settings(camera, suggest_settings_parameters)
There are only two parameters to configure with Capture Assistant:
Maximum Capture Time in number of milliseconds.
Minimum capture time is 200 ms. This allows only one acquisition.
The algorithm will combine multiple acquisitions if the budget allows.
The algorithm will attempt to cover as much of the dynamic range in the scene as possible.
A maximum capture time of more than 1 second will get good coverage in most scenarios.
Ambient light compensation
May restrict capture assistant to exposure periods that are multiples of the ambient light period.
60Hz is found in Japan, Americas, Taiwan, South Korea and Philippines.
50Hz is common in the rest of the world.
Manual configuration
Another option is to configure settings manually. For more information about what each settings does, please see Camera Settings. Note that Zivid Two has a set of standard settings.
Single Acquisition
We can create settings for a single acquisition capture.
const auto settings =
Zivid::Settings{ Zivid::Settings::Experimental::Engine::phase,
Zivid::Settings::Acquisitions{ Zivid::Settings::Acquisition{
Zivid::Settings::Acquisition::Aperture{ 5.66 },
Zivid::Settings::Acquisition::ExposureTime{ std::chrono::microseconds{ 6500 } } } },
Zivid::Settings::Processing::Filters::Outlier::Removal::Enabled::yes,
Zivid::Settings::Processing::Filters::Outlier::Removal::Threshold{ 5.0 } };
var settings = new Zivid.NET.Settings
{
Acquisitions = { new Zivid.NET.Settings.Acquisition { Aperture = 5.66,
ExposureTime =
Duration.FromMicroseconds(6500) } },
Processing = { Filters = { Outlier = { Removal = { Enabled = true, Threshold = 5.0 } } } }
};
settings = zivid.Settings()
settings.experimental.engine = "phase"
settings.acquisitions.append(zivid.Settings.Acquisition())
settings.acquisitions[0].aperture = 5.66
settings.acquisitions[0].exposure_time = datetime.timedelta(microseconds=6500)
settings.processing.filters.outlier.removal.enabled = True
settings.processing.filters.outlier.removal.threshold = 5.0
Multi Acquisition HDR
We may also create settings to be used in a multi-acquisition HDR capture.
Zivid::Settings settings;
for(const auto aperture : { 11.31, 5.66, 2.83 })
{
std::cout << "Adding acquisition with aperture = " << aperture << std::endl;
const auto acquisitionSettings = Zivid::Settings::Acquisition{
Zivid::Settings::Acquisition::Aperture{ aperture },
};
settings.acquisitions().emplaceBack(acquisitionSettings);
}
var settings = new Zivid.NET.Settings();
foreach (var aperture in new double[] { 9.57, 4.76, 2.59 })
{
Console.WriteLine("Adding acquisition with aperture = " + aperture);
var acquisitionSettings = new Zivid.NET.Settings.Acquisition { Aperture = aperture };
settings.Acquisitions.Add(acquisitionSettings);
}
settings = zivid.Settings(acquisitions=[zivid.Settings.Acquisition(aperture=fnum) for fnum in (11.31, 5.66, 2.83)])
Fully configured settings are demonstrated below.
std::cout << "Configuring settings for capture:" << std::endl;
Zivid::Settings settings{
Zivid::Settings::Experimental::Engine::phase,
Zivid::Settings::RegionOfInterest::Box::Enabled::yes,
Zivid::Settings::RegionOfInterest::Box::PointO{ 1000, 1000, 1000 },
Zivid::Settings::RegionOfInterest::Box::PointA{ 1000, -1000, 1000 },
Zivid::Settings::RegionOfInterest::Box::PointB{ -1000, 1000, 1000 },
Zivid::Settings::RegionOfInterest::Box::Extents{ -1000, 1000 },
Zivid::Settings::RegionOfInterest::Depth::Enabled::yes,
Zivid::Settings::RegionOfInterest::Depth::Range{ 200, 2000 },
Zivid::Settings::Processing::Filters::Smoothing::Gaussian::Enabled::yes,
Zivid::Settings::Processing::Filters::Smoothing::Gaussian::Sigma{ 1.5 },
Zivid::Settings::Processing::Filters::Noise::Removal::Enabled::yes,
Zivid::Settings::Processing::Filters::Noise::Removal::Threshold{ 7.0 },
Zivid::Settings::Processing::Filters::Outlier::Removal::Enabled::yes,
Zivid::Settings::Processing::Filters::Outlier::Removal::Threshold{ 5.0 },
Zivid::Settings::Processing::Filters::Reflection::Removal::Enabled::yes,
Zivid::Settings::Processing::Filters::Reflection::Removal::Experimental::Mode::global,
Zivid::Settings::Processing::Filters::Cluster::Removal::Enabled::yes,
Zivid::Settings::Processing::Filters::Cluster::Removal::MaxNeighborDistance{ 10 },
Zivid::Settings::Processing::Filters::Cluster::Removal::MinArea{ 100 },
Zivid::Settings::Processing::Filters::Experimental::ContrastDistortion::Correction::Enabled::yes,
Zivid::Settings::Processing::Filters::Experimental::ContrastDistortion::Correction::Strength{ 0.4 },
Zivid::Settings::Processing::Filters::Experimental::ContrastDistortion::Removal::Enabled::no,
Zivid::Settings::Processing::Filters::Experimental::ContrastDistortion::Removal::Threshold{ 0.5 },
Zivid::Settings::Processing::Filters::Experimental::HoleFilling::Enabled::yes,
Zivid::Settings::Processing::Filters::Experimental::HoleFilling::HoleSize{ 0.2 },
Zivid::Settings::Processing::Filters::Experimental::HoleFilling::Strictness{ 1 },
Zivid::Settings::Processing::Color::Balance::Red{ 1.0 },
Zivid::Settings::Processing::Color::Balance::Green{ 1.0 },
Zivid::Settings::Processing::Color::Balance::Blue{ 1.0 },
Zivid::Settings::Processing::Color::Gamma{ 1.0 },
Zivid::Settings::Processing::Color::Experimental::Mode::automatic
};
std::cout << settings << std::endl;
std::cout << "Configuring base acquisition with settings same for all HDR acquisition:" << std::endl;
const auto baseAcquisition = Zivid::Settings::Acquisition{ Zivid::Settings::Acquisition::Brightness{ 1.8 } };
std::cout << baseAcquisition << std::endl;
std::cout << "Configuring acquisition settings different for all HDR acquisitions" << std::endl;
auto exposureValues = getExposureValues(camera);
const std::vector<double> aperture = std::get<0>(exposureValues);
const std::vector<double> gain = std::get<1>(exposureValues);
const std::vector<size_t> exposureTime = std::get<2>(exposureValues);
for(size_t i = 0; i < aperture.size(); ++i)
{
std::cout << "Acquisition " << i + 1 << ":" << std::endl;
std::cout << " Exposure Time: " << exposureTime.at(i) << std::endl;
std::cout << " Aperture: " << aperture.at(i) << std::endl;
std::cout << " Gain: " << gain.at(i) << std::endl;
const auto acquisitionSettings = baseAcquisition.copyWith(
Zivid::Settings::Acquisition::Aperture{ aperture.at(i) },
Zivid::Settings::Acquisition::Gain{ gain.at(i) },
Zivid::Settings::Acquisition::ExposureTime{ std::chrono::microseconds{ exposureTime.at(i) } });
settings.acquisitions().emplaceBack(acquisitionSettings);
}
Console.WriteLine("Configuring settings for capture:");
var settings = new Zivid.NET.Settings()
{
Experimental = { Engine = Zivid.NET.Settings.ExperimentalGroup.EngineOption.Phase },
RegionOfInterest = { Box = {
Enabled = true,
PointO = new Zivid.NET.PointXYZ{ x = 1000, y = 1000, z = 1000 },
PointA = new Zivid.NET.PointXYZ{ x = 1000, y = -1000, z = 1000 },
PointB = new Zivid.NET.PointXYZ{ x = -1000, y = 1000, z = 1000 },
Extents = new Zivid.NET.Range<double>(-1000, 1000),
},
Depth =
{
Enabled = true,
Range = new Zivid.NET.Range<double>(200, 2000),
}
},
Processing = { Filters = { Smoothing = { Gaussian = { Enabled = true, Sigma = 1.5 } },
Noise = { Removal = { Enabled = true, Threshold = 7.0 } },
Outlier = { Removal = { Enabled = true, Threshold = 5.0 } },
Reflection = { Removal = { Enabled = true, Experimental = { Mode = ReflectionFilterModeOption.Global} } },
Cluster = { Removal = { Enabled = true, MaxNeighborDistance = 10, MinArea = 100} },
Experimental = { ContrastDistortion = { Correction = { Enabled = true,
Strength = 0.4 },
Removal = { Enabled = true,
Threshold = 0.5 } },
HoleFilling = { Enabled = true,
HoleSize = 0.2,
Strictness = 1 } } },
Color = { Balance = { Red = 1.0, Green = 1.0, Blue = 1.0 },
Gamma = 1.0,
Experimental = { Mode = ColorModeOption.Automatic } } }
};
Console.WriteLine(settings);
Console.WriteLine("Configuring base acquisition with settings same for all HDR acquisitions:");
var baseAcquisition = new Zivid.NET.Settings.Acquisition { Brightness = 1.8 };
Console.WriteLine(baseAcquisition);
Console.WriteLine("Configuring acquisition settings different for all HDR acquisitions:");
Tuple<double[], int[], double[]> exposureValues = GetExposureValues(camera);
double[] aperture = exposureValues.Item1;
int[] exposureTime = exposureValues.Item2;
double[] gain = exposureValues.Item3;
for (int i = 0; i < aperture.Length; i++)
{
Console.WriteLine("Acquisition {0}:", i + 1);
Console.WriteLine(" Exposure Time: {0}", exposureTime[i]);
Console.WriteLine(" Aperture: {0}", aperture[i]);
Console.WriteLine(" Gain: {0}", gain[i]);
var acquisitionSettings = baseAcquisition.CopyWith(s =>
{
s.Aperture = aperture[i];
s.ExposureTime =
Duration.FromMicroseconds(exposureTime[i]);
s.Gain = gain[i];
});
settings.Acquisitions.Add(acquisitionSettings);
}
print("Configuring settings for capture:")
settings = zivid.Settings()
settings.experimental.engine = "phase"
settings.region_of_interest.box.enabled = True
settings.region_of_interest.box.point_o = [1000, 1000, 1000]
settings.region_of_interest.box.point_a = [1000, -1000, 1000]
settings.region_of_interest.box.point_b = [-1000, 1000, 1000]
settings.region_of_interest.box.extents = [-1000, 1000]
settings.region_of_interest.depth.enabled = True
settings.region_of_interest.depth.range = [200, 2000]
filters = settings.processing.filters
filters.smoothing.gaussian.enabled = True
filters.smoothing.gaussian.sigma = 1.5
filters.noise.removal.enabled = True
filters.noise.removal.threshold = 7.0
filters.outlier.removal.enabled = True
filters.outlier.removal.threshold = 5.0
filters.reflection.removal.enabled = True
filters.reflection.removal.experimental.mode = "global"
filters.cluster.removal.enabled = True
filters.cluster.removal.max_neighbor_distance = 10
filters.cluster.removal.min_area = 100
filters.experimental.contrast_distortion.correction.enabled = True
filters.experimental.contrast_distortion.correction.strength = 0.4
filters.experimental.contrast_distortion.removal.enabled = False
filters.experimental.contrast_distortion.removal.threshold = 0.5
filters.experimental.hole_filling.enabled = True
filters.experimental.hole_filling.hole_size = 0.2
filters.experimental.hole_filling.strictness = 1
color = settings.processing.color
color.balance.red = 1.0
color.balance.blue = 1.0
color.balance.green = 1.0
color.gamma = 1.0
settings.processing.color.experimental.mode = "automatic"
print(settings)
print("Configuring acquisition settings different for all HDR acquisitions")
exposure_values = _get_exposure_values(camera)
for (aperture, gain, exposure_time) in exposure_values:
settings.acquisitions.append(
zivid.Settings.Acquisition(
aperture=aperture,
exposure_time=datetime.timedelta(microseconds=exposure_time),
brightness=1.8,
gain=gain,
)
)
2D Settings
It is possible to only capture a 2D image. This is faster than a 3D capture. 2D settings are configured as follows.
const auto settings2D =
Zivid::Settings2D{ Zivid::Settings2D::Acquisitions{ Zivid::Settings2D::Acquisition{
Zivid::Settings2D::Acquisition::ExposureTime{ std::chrono::microseconds{ 30000 } },
Zivid::Settings2D::Acquisition::Aperture{ 11.31 },
Zivid::Settings2D::Acquisition::Brightness{ 1.80 },
Zivid::Settings2D::Acquisition::Gain{ 2.0 } } },
Zivid::Settings2D::Processing::Color::Balance::Red{ 1 },
Zivid::Settings2D::Processing::Color::Balance::Green{ 1 },
Zivid::Settings2D::Processing::Color::Balance::Blue{ 1 } };
var settings2D = new Zivid.NET.Settings2D
{
Acquisitions = { new Zivid.NET.Settings2D.Acquisition {
Aperture = 11.31, ExposureTime = Duration.FromMicroseconds(30000), Gain = 2.0, Brightness = 1.80
} },
Processing = { Color = { Balance = { Red = 1.0, Blue = 1.0, Green = 1.0 } } }
};
settings_2d = zivid.Settings2D()
settings_2d.acquisitions.append(zivid.Settings2D.Acquisition())
settings_2d.acquisitions[0].exposure_time = datetime.timedelta(microseconds=30000)
settings_2d.acquisitions[0].aperture = 11.31
settings_2d.acquisitions[0].brightness = 1.80
settings_2d.acquisitions[0].gain = 2.0
settings_2d.processing.color.balance.red = 1.0
settings_2d.processing.color.balance.green = 1.0
settings_2d.processing.color.balance.blue = 1.0
settings_2d.processing.color.gamma = 1.0
Load
Zivid Studio can store the current settings to .yml files. These can be read and applied in the API. You may find it easier to modify the settings in these (human-readable) yaml-files in your preferred editor.
const auto settingsFile = "Settings.yml";
std::cout << "Loading settings from file: " << settingsFile << std::endl;
const auto settingsFromFile = Zivid::Settings(settingsFile);
var settingsFile = "Settings.yml";
Console.WriteLine("Loading settings from file: " + settingsFile);
var settingsFromFile = new Zivid.NET.Settings(settingsFile);
settings_file = "Settings.yml"
print(f"Loading settings from file: {settings_file}")
settings_from_file = zivid.Settings.load(settings_file)
Save
You can also save settings to .yml file.
const auto settingsFile = "Settings.yml";
std::cout << "Saving settings to file: " << settingsFile << std::endl;
settings.save(settingsFile);
var settingsFile = "Settings.yml";
Console.WriteLine("Saving settings to file: " + settingsFile);
settings.Save(settingsFile);
settings_file = "Settings.yml"
print(f"Saving settings to file: {settings_file}")
settings.save(settings_file)
Caution
Zivid settings files must use .yml file extension ( not .yaml).
Capture
Now we can capture a 3D image.
Whether there is a single acquisition or multiple acquisitions (HDR) is given by the number of acquisitions
in settings
.
const auto frame = camera.capture(settings);
using (var frame = camera.Capture(settings))
with camera.capture(settings) as frame:
The Zivid::Frame
contains the point cloud and color image (stored on compute device memory) and the capture and camera information.
The Zivid.NET.Frame
contains the point cloud and color image (stored on compute device memory) and the capture and camera information.
The zivid.Frame
contains the point cloud and color image (stored on compute device memory) and the capture and camera information.
Load
Once saved, the frame can be loaded from a ZDF file.
const auto dataFile = std::string(ZIVID_SAMPLE_DATA_DIR) + "/Zivid3D.zdf";
std::cout << "Reading ZDF frame from file: " << dataFile << std::endl;
const auto frame = Zivid::Frame(dataFile);
var dataFile =
Environment.GetFolderPath(Environment.SpecialFolder.CommonApplicationData) + "/Zivid/Zivid3D.zdf";
Console.WriteLine("Reading ZDF frame from file: " + dataFile);
var frame = new Zivid.NET.Frame(dataFile);
data_file = get_sample_data_path() / "Zivid3D.zdf"
print(f"Reading point cloud from file: {data_file}")
frame = zivid.Frame(data_file)
Saving to a ZDF file is addressed later in the tutorial.
Capture 2D
If we only want to capture a 2D image, which is faster than 3D, we can do so via the 2D API.
const auto frame2D = camera.capture(settings2D);
using (var frame2D = camera.Capture(settings2D))
with camera.capture(settings_2d) as frame_2d:
Caution
Zivid One+ camera has a time penalty when changing the capture mode (2D and 3D) if the 2D capture settings use brightness > 0. You can read more about it in 2D and 3D switching limitation.
Save
We can now save our results.
const auto dataFile = "Frame.zdf";
frame.save(dataFile);
var dataFile = "Frame.zdf";
frame.Save(dataFile);
data_file = "Frame.zdf"
frame.save(data_file)
Tip
You can open and view Frame.zdf
file in Zivid Studio.
Export
The API detects which format to use. See Point Cloud for a list of supported formats. For example, we can export the point cloud to .ply format.
const auto dataFilePLY = "PointCloud.ply";
frame.save(dataFilePLY);
var dataFilePLY = "PointCloud.ply";
frame.Save(dataFilePLY);
data_file_ply = "PointCloud.ply"
frame.save(data_file_ply)
Save 2D
We can get 2D color image from a 3D capture.
const auto image = pointCloud.copyImageRGBA();
2D captures also produce 2D color images.
const auto image = frame2D.imageRGBA();
var image = frame2D.ImageRGBA();
image = frame_2d.image_rgba()
Then, we can save the 2D image.
const auto imageFile = "Image.png";
std::cout << "Saving 2D color image to file: " << imageFile << std::endl;
image.save(imageFile);
var imageFile = "Image.png";
Console.WriteLine("Saving 2D color image to file: {0}", imageFile);
image.Save(imageFile);
image_file = "Image.png"
print(f"Saving 2D color image to file: {image_file}")
image.save(image_file)
Multithreading
Operations on camera objects are thread-safe, but other operations like listing cameras and connecting to cameras should be executed in sequence. Find out more in Multithreading.
Conclusion
This tutorial shows how to use the Zivid SDK to connect to, configure, capture, and save from the Zivid camera.