Getting the Right Exposure for Good Point Clouds

Introduction

Zivid 3D cameras have four acquisition settings that affect the exposure:

In this tutorial, we will use the Histogram in Zivid Studio to assess the quality of the point cloud as we find values for our acquisition settings. While we tune the settings, we will explore considerations to take when using different values for exposure variables.

The goal is to expose as many pixels as possible with sufficiently high SNR so that the point cloud gets low temporal noise. Check out Signal-to-Noise-Ration in the Available Views in Zivid Studio. An SNR value of 7 and above is considered good. Such SNR values typically relate to the RGB channels being exposed well, with a value between 32 and 255. The method in this article aims to cover as many pixels as possible within this region.

Tip

When using the Omni Engine, an SNR value of 2 and above is considered good.

The method described in this section is aimed at capturing a complete point cloud of the entire scene. In many real applications, it is only necessary to get a good exposure of certain objects or regions within the scene. The same method can be applied in those scenarios, but regions that are not of interest do not need to be exposed.

This tutorial assumes that the scene includes both very shiny materials, such as metal cylinders, and dark materials, such as black plastics. It also assumes that imaging occurs in an environment exposed to ambient light powered by power lines of 50 Hz frequency. An example scene is shown in the image below.

sample scene with high dynamic range

Such scenes may contain a significantly wider dynamic range than the Zivid 3D cameras can capture with a single acquisition, especially if using Phase Engine. Hence it is often required to use Zivid’s HDR function with multiple acquisitions for different regions of the intensity spectrum, as illustrated in the image below.

Note

Single acquisitions using Omni Engine covers a wider dynamic range than Stripe Engine, which again covers a wider range than Phase Engine. The Stripe and Omni engines provide more benefits than just covering a wider dynamic range such as dealing with reflections.

illustrating light intensity covered by different acquisitions

The histogram

The histogram in Zivid Studio is a powerful tool for evaluating point cloud quality. It counts how many pixels have a distinct intensity value within an image and displays them in bins from 0 to 255. The histogram method that we use to evaluate point cloud quality is based on a relationship between the pixel intensity and its 3D quality. The higher the pixel intensity (unless saturated), the higher the SNR, and thus the higher the 3D quality. Each bin represents one exposure stop when displaying the histogram with a logarithmic x-axis. This representation makes it very convenient to estimate which exposure values are needed to expose certain regions of the scene well.

explanation of the light intensity histogram

The image above shows an example of a histogram where the scene is well exposed. We can tell this because most of the pixels are in the upper right half of the logarithmic graph. The following is also easy to see from the histogram. Doubling the intensity, or adding one exposure stop, will “move” the hump formed by pixels with intensities of 64-128 to 128-255 (the right-most area of the histogram). We can achieve this, for example, by doubling the exposure time.

Caution

The default Color Mode is Automatic, which is identical to ToneMapping for multi-acquisition HDR captures with differing acquisition settings. Tone mapping modifies pixel values and thus affects the histogram, making it impossible to use the histogram to evaluate exposure quality. Therefore, when using the histogram, you must evaluate one acquisition at a time and set the Color Mode to UseFirstAcquisition or Automatic.

Caution

Gamma correction and Color Balance setting modifies pixel values and thus making it impossible to use the histogram to evaluate exposure quality. Therefore, you must set Gamma and Color Balance to 1.0 when using the histogram.

Introduction to the Stops Table

A second powerful tool that we will use is the Stops Table (see below). The table shows the span of the Zivid 3D cameras’ stops, where each row shows the available range for each of the exposure parameters in Zivid. Each cell represents a stop position for one of the four exposure variables in the Zivid camera: aperture, exposure time, brightness and gain. To increase exposure with one stop, we move one cell to the right for a specific variable. To decrease exposure with one stop, we move one cell to the left for a specific variable. Example: the current exposure time is 10 000 μs. We want to increase the exposure with one stop. We increase the exposure time to 20 000 μs.

The color coding of the cells shows how “good” the values are:

  • Strong green: Great

  • Light green: Good

  • Yellow: OK

  • Red: Avoid unless necessary

Notice that we have two tables per camera. Use the correct one to compensate for 50 Hz or 60 Hz power-line frequency due to the behavior described in the Exposure Time.

Tip

Try to build your exposure values with combinations of green cells from the stops table as much as possible.

Note

The values of certain exposure settings can affect the quality of the color image. Considerations to take for good color image quality are covered later in Adjusting Acquisition Settings section in Optimizing Color Image.

Stops Table for the acquisition settings
Stops Table for the acquisition settings
Stops Table for the acquisition settings
Stops Table for the acquisition settings

Note

Different camera models have different ranges for aperture, exposure time, and brightness.

Preparation

Relaxing ROI & filters

To evaluate as many good points as possible, we start by relaxing the Region of Interest and all filters. After finding all exposure values, we set the filters as the last step.

  • Region Of Interest: Disabled

  • Cluster: Disabled

  • Hole Repair: Disabled

  • Noise Removal: 1.0

  • Noise Repair: Disabled

  • Noise Suppression: Disabled

  • Outlier: 50

  • Reflection: Disabled (if using Phase Engine)

  • Gaussian: Disabled

  • Contrast Distortion: Disabled (if using Phase Engine)

Setting color settings

To be able to use the histogram, set the color settings (see Cautions in Histogram for explanation) to the following values:

  • Color Balance - Red: 1.0

  • Color Balance - Green: 1.0

  • Color Balance - Blue: 1.0

  • Gamma: 1.0

  • Color Mode: UseFirstAcquisition

Acquisition #1 - exposing for highlights

We want to start by exposing the highlights and then gradually increase our exposure to include mid-tones and eventually lowlights (dark regions). If the following procedure is insufficient to achieve good data on especially shiny and bright parts, we recommend that you follow the steps in Dealing with Highlights and Shiny Objects as your first acquisition.

Step 1 - set low exposure values

We begin by turning off all but one acquisition, so we only have a single acquisition capture. Then, we set our exposure very low so that no pixels in the image (or the histogram) have an RGB value of 255. The following starting condition should work on most occasions.

  • Aperture (\(f\)-number): 32

  • Exposure time: 10 000 (8 333 in regions with 60 Hz power line frequency)

  • Brightness: max

  • Gain 1.0

After the capture, turn on the Histogram and set it to the Logarithmic mode.

Histogram showing overexposed image Overexposed image

If there are still regions in the image that are overexposed, as illustrated in the figure above, try to reduce the exposure time further and increase the \(f\)-number. The figure below shows a capture with no overexposed pixels. If there are no overexposed regions, go to Step 2.

Histogram showing underexposed image Underexposed image

Step 2 - get good exposure on brightest pixels

Decrease the \(f\)-number by moving one stop in the Stops Table at a time to the right. Carry this out until the brightest pixels have an intensity close to 255. Verify that the highlights in your scene have data by observing the point cloud or the RGB image.

In our example, we would need to increase by two stops. Our first acquisition would, therefore, consist of the following values:

  • Aperture (\(f\)-number): 16

  • Exposure time: 10 000 (8 333 in regions with 60 Hz power line frequency)

  • Brightness: max

  • Gain 1.0

In general, it is recommended to use the maximum Brightness value available for your camera model.

Histogram after finding first acquisition Color image after finding first acquisition

Acquisition #2 - exposing for mid-tones

After covering the highlights, we want to move on to exposing more of the image. By looking at the histogram, we identify the next portion of the image exposed with an intensity below 32. We want to move this region up in the upper half of the image towards 255, typically done by adding three-four stops.

Histogram illustrating effect of an additional acquisition

Step 1 - new acquisition

Add another acquisition and disable the first acquisition.

Add another acquisition and disable the first acquisition

Step 2 - increase exposure

Move a total of three or four more stops to the right. The first stop should be maximizing brightness, and the other stops should typically come from the aperture.

The example, acquisition #1 from above has an aperture of 16. Therefore, move three aperture stops to the right (count aperture columns in the Stops Table), which results in aperture 5.6.

Acquisition #2 will then have the following values:

  • Aperture (\(f\)-number): 5.6

  • Exposure time: 10 000 (8 333 in regions with 60 Hz power line frequency)

  • Brightness: max

  • Gain: 1.0

Histogram after finding second acquisition Color image of the second acquisition

Acquisition #N - keep increasing exposure toward lowlights

Repeat the procedure for acquisition #2. Locate the next region of pixels that needs to be exposed higher and count the number of stops required to bring the intensity of those pixels close to 255. Remember to use the histogram only with one acquisition enabled.

If our next region has an intensity of around 16, we want to bring those up to about 255, which is four additional stops. We could get these stops by adding aperture, exposure time, and gain:

Acquisition #3:

  • Aperture (\(f\)-number): 2.8

  • Exposure time: 20 000 (16 667 in regions with 60 Hz power line frequency)

  • Brightness: max

  • Gain: 2.0

Histogram after finding third acquisition Color image of the third acquisition

We might need very high exposure to extract data from the very darkest regions. This may require that the final acquisition uses high exposure time and gain.

Keep adding acquisitions until almost all pixels are on the right-hand side of the logarithmic histogram, as shown in the image below.

Histogram of the last acquisition when having full coverage Color image of the last acquisition

Enjoy the HDR

Enable all acquisitions and capture an HDR to confirm you have covered the increased dynamic range.

Color image of the last acquisition

Pro tip: further improving noise at the expense of time

If the time budget allows it, duplicate acquisitions to perform additional SNR boosting. Grabbing multiple identical acquisitions is the equivalent of performing over-sampling and will, therefore, suppress noise by up to \(\sqrt{N}\), where \(N\) is the number of acquisitions. Duplicating acquisitions is an excellent way of maximizing point cloud quality at the expense of time. Duplicate the acquisitions exposed for the regions that require higher accuracy.

Key acquisitions duplicated

Be aware of your trade-offs

Exposure Variable

Pros

Cons

Exposure Time

  • Minimum impact on noise

  • Increases acquisition time

  • Wrong use can cause interference with ambient light (typically 50 Hz/60 Hz), causing point cloud artifacts in form of ripples/waves

Aperture

  • Fast adjustment (fast to get high dynamic range in HDR)

  • Highest number of exposure stops

  • Change in DOF/Focus can increase noise and contrast distortion

  • Lower f-number values, blurry 2D images

Projector Brightness

  • Higher projector brightness, better SNR

  • Lower projector brightness, lower signal, more noise

  • High duty cycle can activate thermal throttling

Gain

  • Very fast adjustment (fastest way to get high dynamic range in HDR)

  • Higher gain, more noise

  • Higher values, grainy 2D images

Note

For Zivid cameras, changing Exposure Time is faster than changing Aperture.

Further reading

After the acquisitions have been exposed right it is time to adjust the filters as described in Adjusting Filters

Version History

SDK

Changes

2.12.0

Zivid One+ has reached its End-of-Life and is no longer supported.