Optimizing Color Image

Introduction 

This tutorial aims to help you optimize the color image quality of the captures from your Zivid camera. We first cover using the essential tools for getting good color information: tuning acquisition and color settings. Then we address the most common challenges (blooming/over-saturation and color inconsistency from HDR) and provide our recommendations on how to overcome them.

Tip

Zivid 2+ cameras have higher resolution and utilize a better demosaicing algorithm than Zivid 2 cameras; thus, Zivid 2+ outputs higher quality 2D images.

Adjusting Acquisition Settings 

The quality of the color image you get from the Zivid camera depends on the acquisition settings. This tutorial does not cover finding a combination of exposure settings (exposure time, aperture, brightness, and gain) to get the desired dynamic range; Getting the Right Exposure for Good Point Clouds covers that. This tutorial focuses on tuning the specific acquisition settings that affect the color image quality.

Exposure time and projector brightness do not impact the color image quality. On the other hand, higher aperture values are key to avoiding blurry color images; see the figure below. A good rule-of-thumb is 5.66 or higher f-number values (smaller apertures). For a more detailed explanation and guidance, see Depth of Focus and Depth of Focus Calculator.


Single acquisition capture with aperture 10.37	Single acquisition capture with aperture 3.67

Note

The images above are captured outside of optimal imaging distance to emphasize the out-of-focus effect for large aperture values; images are less blurry when captured in the optimal range.

We advise using low gain values, e.g., 1-2, because high gain values increase image noise (granularity) levels, thus decreasing the 2D image quality; see the figure below.


Single acquisition capture with gain 1	Single acquisition capture with gain 16

We often adjust the acquisition settings to reach the required dynamic range and capture time, focusing on point cloud quality first. With such an approach, the color image quality is a function of the settings we optimized for the 3D quality with a given capture time. The downside of this approach is that it does not always provide good color image quality. However, the fact is that high-quality color images (low blur, low noise, and balanced illumination) result in high-quality point clouds. With this in mind, it becomes clear that following the above fine-tuning guidelines to improve the color image quality also enhances the 3D quality. The key to achieving this is compensating for low gain and apertures (high f-numbers) with increased exposure time and projector brightness values.

Adjusting Color Settings 

In addition to aperture and gain, the color image quality depends on Gamma, Color Balance, and Color Mode settings. This section aims to give insights into optimizing these settings to get desired color quality in the color image. More information about the color settings can be found here: Filters.

Gamma 

Cameras encode luminance differently than the human eye. While the human eye has an emphasis on the darker end of the spectrum, a camera encodes luminance on a linear scale. To compensate for this effect, gamma correction is applied to either darken or brighten the image and make it closer to human perception.

Note

The Gamma correction value ranges between 0.25 and 1.5 for Zivid cameras.

The lower the Gamma value, the lighter the scene will appear. If the Gamma value is increased, then the scene will appear darker.


Image captured with Gamma set to 0.6	Image captured with Gamma set to 1.3

It is questionable whether gamma correction is required for optimizing image quality for machine vision algorithms. Still, it helps us humans evaluate certain aspects of the color image quality, such as focus and grain/noise levels.

For an implementation example, check out Gamma Correction. This tutorial demonstrates how to capture a 2D image with a configurable gamma correction.

Color Balance 

The color temperature of ambient light affects the appearance of the color image. You can adjust the digital gain to red, green, and blue color channels to make the color image look natural. Below, you can see an image before and after balancing the color. If you want to find the color balance for your settings automatically, follow the Color Balance tutorial.

Note

The color balance gains range between 1.0 and 8.0 for Zivid cameras.

Two images of a box of sweets. The green color is too strong in the left image.

Performing color balance can be beneficial in strong and varying ambient light conditions. Color balance is necessary when using captures without a projector or with low projector brightness values. In other words, when ambient light is a significant part of the light seen by the camera. There are two default color balance settings, with and without the projector. When projector brightness is set to 0 or off, the color balance is calibrated to 4500 K, typical in industrial environments. For projector brightness values above 0, the color balance is calibrated for the color temperature of the projector light.

For an implementation example, see Adjusting Color Balance tutorial. This tutorial shows how to balance the color of a 2D image by taking images of a white surface (a piece of paper, wall, or similar) in a loop.

The Zivid 2+ MR130, M60, and LR110 cameras have built-in color balance, so intense or non-white ambient light has little impact on the RGB values in the color image. This eliminates the need to run additional color balance algorithm on these camera models.

The image below shows details from the 2D color image taken with Zivid 2+ MR130 in different ambient light conditions.

	300 LUX	1000 LUX	2000 LUX
3200K
5000K
6500K

Color Mode 

The Color Mode setting controls how the color image is computed. Color Mode setting has the following possible values:

ToneMapping
Automatic

ToneMapping uses all the acquisitions to create one merged and normalized color image. For multi-acquisition HDR captures the dynamic range of the captured images is usually higher than the 8-bit color image range. Tone mapping will map the HDR color data to the 8-bit color output range by applying a scaling factor. Tone mapping can also be used for single-acquisition captures to normalize the captured color image to the full 8-bit output. When using ToneMapping the color values can be inconsistent over repeated captures if you move, add or remove objects in the scene.

Automatic is the default option.
Automatic is identical to ToneMapping for multi-acquisition HDR captures with differing acquisition settings.

For single-acquisition captures, tone mapping can be used to brighten dark images.


Single acquisition capture with Color Mode set to `Automatic` (no tone mapping)	Single acquisition capture with Color Mode set to `ToneMapping`

For multi-acquisition HDR tone mapping can be used to map high-dynamic-range colors to the more limited dynamic output range.


Single acquisition capture of the first of three HDR acquisitions with Color Mode set to `Automatic` (no tone mapping)	Single acquisition capture of the second of three HDR acquisitions with Color Mode set to `Automatic` (no tone mapping)	Single acquisition capture of the third of three HDR acquisitions with Color Mode set to `Automatic` (no tone mapping)

HDR with three acquisitions with Color Mode set to `ToneMapping` or `Automatic`

Check out how to set processing settings with Zivid SDK, including Gamma, Color Balance, and Color Mode:

C++

std::cout << "Configuring settings for capture:" << std::endl;
Zivid::Settings2D settings2D{
    Zivid::Settings2D::Sampling::Color::rgb,
    Zivid::Settings2D::Sampling::Pixel::all,

    Zivid::Settings2D::Processing::Color::Balance::Blue{ 1.0 },
    Zivid::Settings2D::Processing::Color::Balance::Green{ 1.0 },
    Zivid::Settings2D::Processing::Color::Balance::Red{ 1.0 },
    Zivid::Settings2D::Processing::Color::Gamma{ 1.0 },

    Zivid::Settings2D::Processing::Color::Experimental::Mode::automatic,
};

Zivid::Settings settings{
    Zivid::Settings::Color{ settings2D },

    Zivid::Settings::Engine::phase,

    Zivid::Settings::RegionOfInterest::Box::Enabled::yes,
    Zivid::Settings::RegionOfInterest::Box::PointO{ 1000, 1000, 1000 },
    Zivid::Settings::RegionOfInterest::Box::PointA{ 1000, -1000, 1000 },
    Zivid::Settings::RegionOfInterest::Box::PointB{ -1000, 1000, 1000 },
    Zivid::Settings::RegionOfInterest::Box::Extents{ -1000, 1000 },

    Zivid::Settings::RegionOfInterest::Depth::Enabled::yes,
    Zivid::Settings::RegionOfInterest::Depth::Range{ 200, 2000 },

    Zivid::Settings::Processing::Filters::Cluster::Removal::Enabled::yes,
    Zivid::Settings::Processing::Filters::Cluster::Removal::MaxNeighborDistance{ 10 },
    Zivid::Settings::Processing::Filters::Cluster::Removal::MinArea{ 100 },

    Zivid::Settings::Processing::Filters::Hole::Repair::Enabled::yes,
    Zivid::Settings::Processing::Filters::Hole::Repair::HoleSize{ 0.2 },
    Zivid::Settings::Processing::Filters::Hole::Repair::Strictness{ 1 },

    Zivid::Settings::Processing::Filters::Noise::Removal::Enabled::yes,
    Zivid::Settings::Processing::Filters::Noise::Removal::Threshold{ 7.0 },

    Zivid::Settings::Processing::Filters::Noise::Suppression::Enabled::yes,
    Zivid::Settings::Processing::Filters::Noise::Repair::Enabled::yes,

    Zivid::Settings::Processing::Filters::Outlier::Removal::Enabled::yes,
    Zivid::Settings::Processing::Filters::Outlier::Removal::Threshold{ 5.0 },

    Zivid::Settings::Processing::Filters::Reflection::Removal::Enabled::yes,
    Zivid::Settings::Processing::Filters::Reflection::Removal::Mode::global,

    Zivid::Settings::Processing::Filters::Smoothing::Gaussian::Enabled::yes,
    Zivid::Settings::Processing::Filters::Smoothing::Gaussian::Sigma{ 1.5 },

    Zivid::Settings::Processing::Filters::Experimental::ContrastDistortion::Correction::Enabled::yes,
    Zivid::Settings::Processing::Filters::Experimental::ContrastDistortion::Correction::Strength{ 0.4 },

    Zivid::Settings::Processing::Filters::Experimental::ContrastDistortion::Removal::Enabled::no,
    Zivid::Settings::Processing::Filters::Experimental::ContrastDistortion::Removal::Threshold{ 0.5 },

    Zivid::Settings::Processing::Resampling::Mode::upsample2x2,

    Zivid::Settings::Diagnostics::Enabled::no,
};

setSamplingPixel(settings, camera);
std::cout << settings << std::endl;

C#

Console.WriteLine("Configuring settings for capture:");
var settings2D = new Zivid.NET.Settings2D()
{
    Sampling =
    {
        Color = Zivid.NET.Settings2D.SamplingGroup.ColorOption.Rgb,
        Pixel = Zivid.NET.Settings2D.SamplingGroup.PixelOption.All,
    },
    Processing =
    {
        Color =
        {
            Balance =
            {
                Blue = 1.0,
                Green = 1.0,
                Red = 1.0,
            },
            Gamma = 1.0,
            Experimental = { Mode = Zivid.NET.Settings2D.ProcessingGroup.ColorGroup.ExperimentalGroup.ModeOption.Automatic },
        },
    },
};
var settings = new Zivid.NET.Settings()
{
    Engine = Zivid.NET.Settings.EngineOption.Phase,

    RegionOfInterest =
    {
        Box = {
            Enabled = true,
            PointO = new Zivid.NET.PointXYZ{ x = 1000, y = 1000, z = 1000 },
            PointA = new Zivid.NET.PointXYZ{ x = 1000, y = -1000, z = 1000 },
            PointB = new Zivid.NET.PointXYZ{ x = -1000, y = 1000, z = 1000 },
            Extents = new Zivid.NET.Range<double>(-1000, 1000),
        },
        Depth =
        {
            Enabled = true,
            Range = new Zivid.NET.Range<double>(200, 2000),
        },
    },
    Processing =
    {
        Filters =
        {
            Cluster =
            {
                Removal = { Enabled = true, MaxNeighborDistance = 10, MinArea = 100}
            },
            Hole =
            {
                Repair = { Enabled = true, HoleSize = 0.2, Strictness = 1 },
            },
            Noise =
            {
                Removal = { Enabled = true, Threshold = 7.0 },
                Suppression = { Enabled = true },
                Repair = { Enabled = true },
            },
            Outlier =
            {
                Removal = { Enabled = true, Threshold = 5.0 },
            },
            Reflection =
            {
                Removal = { Enabled = true, Mode = ReflectionFilterModeOption.Global },
            },
            Smoothing =
            {
                Gaussian = { Enabled = true, Sigma = 1.5 },
            },
            Experimental =
            {
                ContrastDistortion =
                {
                    Correction = { Enabled = true, Strength = 0.4 },
                    Removal = { Enabled = true, Threshold = 0.5 },
                },
            },
        },
        Resampling = { Mode = Zivid.NET.Settings.ProcessingGroup.ResamplingGroup.ModeOption.Upsample2x2 },
    },
    Diagnostics = { Enabled = false },
};

settings.Color = settings2D;

SetSamplingPixel(ref settings, camera);
Console.WriteLine(settings);

Python

print("Configuring settings for capture:")
settings_2d = zivid.Settings2D()

settings_2d.sampling.color = zivid.Settings2D.Sampling.Color.rgb
settings_2d.sampling.pixel = zivid.Settings2D.Sampling.Pixel.all

settings_2d.processing.color.balance.red = 1.0
settings_2d.processing.color.balance.blue = 1.0
settings_2d.processing.color.balance.green = 1.0
settings_2d.processing.color.gamma = 1.0

settings_2d.processing.color.experimental.mode = zivid.Settings2D.Processing.Color.Experimental.Mode.automatic

settings = zivid.Settings()
settings.engine = zivid.Settings.Engine.phase

settings.region_of_interest.box.enabled = True
settings.region_of_interest.box.point_o = [1000, 1000, 1000]
settings.region_of_interest.box.point_a = [1000, -1000, 1000]
settings.region_of_interest.box.point_b = [-1000, 1000, 1000]
settings.region_of_interest.box.extents = [-1000, 1000]

settings.region_of_interest.depth.enabled = True
settings.region_of_interest.depth.range = [200, 2000]

settings.processing.filters.cluster.removal.enabled = True
settings.processing.filters.cluster.removal.max_neighbor_distance = 10
settings.processing.filters.cluster.removal.min_area = 100

settings.processing.filters.hole.repair.enabled = True
settings.processing.filters.hole.repair.hole_size = 0.2
settings.processing.filters.hole.repair.strictness = 1

settings.processing.filters.noise.removal.enabled = True
settings.processing.filters.noise.removal.threshold = 7.0

settings.processing.filters.noise.suppression.enabled = True
settings.processing.filters.noise.repair.enabled = True

settings.processing.filters.outlier.removal.enabled = True
settings.processing.filters.outlier.removal.threshold = 5.0

settings.processing.filters.reflection.removal.enabled = True
settings.processing.filters.reflection.removal.mode = (
    zivid.Settings.Processing.Filters.Reflection.Removal.Mode.global_
)

settings.processing.filters.smoothing.gaussian.enabled = True
settings.processing.filters.smoothing.gaussian.sigma = 1.5

settings.processing.filters.experimental.contrast_distortion.correction.enabled = True
settings.processing.filters.experimental.contrast_distortion.correction.strength = 0.4

settings.processing.filters.experimental.contrast_distortion.removal.enabled = False
settings.processing.filters.experimental.contrast_distortion.removal.threshold = 0.5

settings.processing.resampling.mode = zivid.Settings.Processing.Resampling.Mode.upsample2x2

settings.diagnostics.enabled = False

settings.color = settings_2d

_set_sampling_pixel(settings, camera)
print(settings)

Troubleshooting 

If you encounter any issues with your color image, check Vision troubleshooting articles.

Optimizing Color Image

Introduction

Adjusting Acquisition Settings

Adjusting Color Settings

Gamma

Color Balance

Color Mode

Troubleshooting

Introduction 

Adjusting Acquisition Settings 

Adjusting Color Settings 

Gamma 

Color Balance 

Color Mode 

Troubleshooting 