How to deal with Color Inconsistency from HDR

Introduction

In this article we discuss how to overcome color inconsistency from HDR.

Problem

The color image from the multi-acquisition HDR capture is a result of tone mapping if Color Mode is set to ToneMapping or Automatic (default). While tone mapping solves the challenging problem of optimizing the color for that particular capture, it has a downside. Since it is a function of the scene, the tone mapping technique introduces color inconsistency with changes in the scene. The following example explains this phenomenon.

Let us say we have a relatively dark scene (pears on a black surface). We find acquisition settings that cover a wide enough dynamic range and perform a multi-acquisition HDR capture (figure on the left). We then add a bright object (banana) to the scene and capture it again with the same settings (figure on the right).

HDR capture of a dark scene (Color Mode: Automatic or ToneMapping)	HDR capture with the same settings of the same scene but with an additional bright object added

Let us look at the output color image (figure on the right) and specifically at the dark objects initially in the scene (pears or black surface). We notice that the RGB values of these objects are different before and after adding the bright object (banana) to the scene.

The change of RGB values can be a problem for some applications, e.g., ones using algorithms that classify objects based on color information. The reason is that these algorithms will expect the RGB values to remain the same (consistent) for repeated captures.

Solutions

HDR capture with UseFirstAcquisition Color Mode

Note

This solution can be used only with SDK 2.7 and higher. Change the KB to an older version in the top left corner to see a solution for SDK 2.6 or lower.

To overcome the color inconsistency from HDR, identify which of the acquisitions from your HDR capture gives the best color. We recommend the acquisition optimized for the brightest object in the scene to avoid saturation. Then, set that acquisition to be the first in the acquisition settings. Finally, capture your HDR with Color Mode set to UseFirstAcquisition.

We will walk you through the process with an example. Let us assume that we have an HDR with two acquisitions. The first of the acquisitions is optimized for the dark objects (pears). The second is optimized for bright objects (banana). HDR capture with UseFirstAcquisition for Color Mode yields the following results.

HDR capture of a dark scene (Color Mode: UseFirstAcquisition)	HDR capture with the same settings of the same scene but with an additional bright object added

The color of the dark objects (pears) is the same in both images. The color consistency is preserved.

However, because the first acquisition is optimized for the dark objects, the brightest object in the scene (banana) is saturated. Saturation will likely cause issues, e.g., if we want to classify objects based on color. To overcome this problem, we can rearrange the acquisition settings. For the first acquisition we select the one optimized for the brightest object in the scene (banana). The second is optimized for the dark objects (pears). Now, we see that the color consistency is preserved with bright and dark objects captured together and separately. In addition, the brightest object (banana) is not saturated.

HDR capture of a dark scene (Color Mode: UseFirstAcquisition)	HDR capture with the same settings of the same scene but with an additional bright object added	HDR capture with the same settings of the same scene but only with the bright object

Note

The acquisition that provides the best color is an excellent acquisition. It is optimized for the brightest objects in the scene and thus also provides very good SNR for those objects. An additional acquisition is not needed to deal with color inconsistency in HDR; that acquisition is likely already part of your HDR acquisition settings.

If the color image is too dark, it can be fixed with the Gamma setting.

Caution

The first of the acquisitions the Capture Assistant returns is likely not the most suitable one for the color image. Therefore, if using the Capture Assistant and UseFirstAcquisition for Color Mode, you might need to rearrange your acquisitions.

Hint

Make acquisition first in the sequence by clicking … → Move to top in Zivid Studio.

Additional Capture

Note

This solution should be used only if you have to use Automatic or ToneMapping Color Mode for your HDR capture.

An alternative solution to overcome the color inconsistency from HDR is to take a separate capture in addition to the main capture. This approach assumes using the main capture for getting the point cloud data and the additional capture for getting the color image. The additional capture can be a 2D or 3D capture, with or without a projector. If you use 3D capture, it must be without tone mapping (Color Mode setting set to UseFirstAcquisition).

Single acquisition capture of a dark scene with Color Mode set to UseFirstAcquisition or Automatic	Single acquisition capture with the same settings of the same scene but with an additional bright object added (Color Mode set to UseFirstAcquisition or Automatic)