Color Inconsistency from HDR

Problem

The color image from the multi-acquisition HDR capture is a result of tone mapping if Color Mode is set to ToneMapping or Automatic (default). While tone mapping solves the challenging problem of optimizing the color for that particular capture, it has a downside. Since it is a function of the scene, the tone mapping technique introduces color inconsistency with changes in the scene. The following example explains this phenomenon.

Let us say we have a relatively dark scene (pears on a black surface). We find acquisition settings that cover a wide enough dynamic range and perform a multi-acquisition HDR capture (figure on the left). We then add a bright object (banana) to the scene and capture it again with the same settings (figure on the right).

HDR capture of a dark scene (Color Mode: Automatic or ToneMapping)

HDR capture with the same settings of the same scene but with an additional bright object added

HDR capture of a dark scene (Color Mode: Automatic or ToneMapping)

HDR capture with the same settings of the same scene but with an additional bright object added

Let us look at the output color image (figure on the right) and specifically at the dark objects initially in the scene (pears or black surface). We notice that the RGB values of these objects are different before and after adding the bright object (banana) to the scene.

The change of RGB values can be a problem for some applications, e.g., ones using algorithms that classify objects based on color information. The reason is that these algorithms will expect the RGB values to remain the same (consistent) for repeated captures.

Solutions

Configure 2D capture settings independently

If you encounter issues with color inconsistency from HDR, it is likely because you have not configured Settings::Color (added in SDK 2.14). Under these conditions, the Zivid software generates both the point cloud and the color image from the same multi-acquisition HDR capture.

To resolve color inconsistencies from HDR, consider the following.

First, configure 2D acquisition settings using Settings::Color. We recommend starting with the 2D Presets. If further adjustments are needed, check Optimizing Color Image.

Tip

You can load your old YML file in Zivid Studio, configure 2D settings, and export the settings to a YML file.

Separate 2D and 3D captures

Use capture2D() for 2D captures.

Use capture3D() for 3D captures.

Tip

You can use the same YML file / settings object for these two capture APIs.

Combined 2D and 3D capture

Use capture2D3D() to perform a single operation that combines 2D and 3D captures.

Independently configured 2D settings of a dark scene

Same settings with an additional bright object added to the scene

Independently configured 2D settings of a dark scene

Same settings with an additional bright object added to the scene

If you must use the deprecated camera.capture() API, ensure that 2D acquisition settings are explicitly configured using Settings::Color. This ensures the color image is generated based on the specified 2D acquisition settings.

These solutions require Zivid SDK 2.14 or later. If you must use an older SDK version, check the next solution.

HDR capture with UseFirstAcquisition Color Mode

To overcome the color inconsistency from HDR, identify which of the acquisitions from your HDR capture gives the best color. We recommend the acquisition optimized for the brightest object in the scene to avoid saturation. Then, set that acquisition to be the first in the acquisition settings. Finally, capture your HDR with Color Mode set to UseFirstAcquisition.

We will walk you through the process with an example. Let us assume that we have an HDR with two acquisitions. The first acquisition is optimized for dark objects (pears), and the second is optimized for bright objects (bananas). HDR capture with UseFirstAcquisition for Color Mode yields the following results.

HDR capture of a dark scene (Color Mode: UseFirstAcquisition)

HDR capture with the same settings of the same scene but with an additional bright object added

HDR capture of a dark scene (Color Mode: UseFirstAcquisition)

HDR capture with the same settings of the same scene but with an additional bright object added

The color of the dark objects (pears) is the same in both images. The color consistency is preserved.

However, because the first acquisition is optimized for dark objects, the brightest object in the scene (the banana) is saturated. Saturation will likely cause issues if we want to classify objects based on color. To overcome this problem, we can rearrange the acquisition settings. For the first acquisition, we select the one optimized for the brightest object in the scene (banana). The second is optimized for the dark objects (pears). We see that the color consistency is preserved with bright and dark objects captured together and separately. In addition, the brightest object (banana) is not saturated.

HDR capture of a dark scene (Color Mode: UseFirstAcquisition)

HDR capture with the same settings of the same scene but with an additional bright object added

HDR capture with the same settings of the same scene but only with the bright object

HDR capture of a dark scene (Color Mode: UseFirstAcquisition)

HDR capture with the same settings of the same scene but with an additional bright object added

HDR capture with the same settings of the same scene but only with the bright object

Note

The acquisition that provides the best color is an excellent acquisition. It is optimized for the brightest objects in the scene and thus also provides very good SNR for those objects. To deal with color inconsistency in HDR, an additional acquisition is not needed; that acquisition is likely already part of your HDR acquisition settings.

If the color image is too dark, it can be fixed with the Gamma setting.

Caution

The first of the acquisitions the Capture Assistant returns is likely not the most suitable one for the color image. Therefore, you might need to rearrange your acquisitions if you use the Capture Assistant and UseFirstAcquisition for Color Mode.

Hint

Make acquisition first in the sequence by clicking … → Move to top in Zivid Studio.

Move acquisition to top, making it first in the sequence

Additional Capture

Note

This solution should be used only if you have to use Automatic or ToneMapping Color Mode for your HDR capture.

An alternative solution to overcome the color inconsistency from HDR is to take a separate capture in addition to the main capture. This approach assumes using the main capture to get the point cloud data and the additional capture to get the color image. The additional capture can be a 2D or 3D capture. If you use 3D capture, it must be without tone mapping (the Color Mode setting must be set to UseFirstAcquisition).

Single acquisition capture of a dark scene with Color Mode set to UseFirstAcquisition or Automatic

Single acquisition capture with the same settings of the same scene but with an additional bright object added (Color Mode set to UseFirstAcquisition or Automatic)

Single acquisition capture of a dark scene with Color Mode set to UseFirstAcquisition or Automatic

Single acquisition capture with the same settings of the same scene but with an additional bright object added (Color Mode set to UseFirstAcquisition or Automatic)