Getting the Right Exposure for Good Point Clouds
Introduction
Zivid 3D cameras have four acquisition settings that affect the exposure:
In this tutorial, we will use the Histogram in Zivid Studio to assess the quality of the point cloud as we find values for our acquisition settings. While we tune the settings, we will explore considerations to take when using different values for exposure variables.
The goal is to expose as many pixels as possible with sufficiently high SNR so that the point cloud gets low temporal noise. Check out Signal-to-Noise-Ration in the Available Views in Zivid Studio. An SNR value of 7 and above is considered good. Such SNR values typically relate to the RGB channels being exposed well, with a value between 32 and 255. The method in this article aims to cover as many pixels as possible within this region.
Tip
When using the Omni Engine, an SNR value of 2 and above is considered good.
The method described in this section is aimed at capturing a complete point cloud of the entire scene. In many real applications, it is only necessary to get a good exposure of certain objects or regions within the scene. The same method can be applied in those scenarios, but regions that are not of interest do not need to be exposed.
This tutorial assumes that the scene includes both very shiny materials, such as metal cylinders, and dark materials, such as black plastics. It also assumes that imaging occurs in an environment exposed to ambient light powered by power lines of 50 Hz frequency. An example scene is shown in the image below.
Such scenes may contain a significantly wider dynamic range than the Zivid 3D cameras can capture with a single acquisition, especially if using Phase Engine. Hence it is often required to use Zivid’s HDR function with multiple acquisitions for different regions of the intensity spectrum, as illustrated in the image below.
Note
Single acquisitions using Omni Engine covers a wider dynamic range than Stripe Engine, which again covers a wider range than Phase Engine. The Stripe and Omni engines provide more benefits than just covering a wider dynamic range such as dealing with reflections.
The histogram
The histogram in Zivid Studio is a powerful tool for evaluating point cloud quality. It counts how many pixels have a distinct intensity value within an image and displays them in bins from 0 to 255. The histogram method that we use to evaluate point cloud quality is based on a relationship between the pixel intensity and its 3D quality. The higher the pixel intensity (unless saturated), the higher the SNR, and thus the higher the 3D quality. Each bin represents one exposure stop when displaying the histogram with a logarithmic x-axis. This representation makes it very convenient to estimate which exposure values are needed to expose certain regions of the scene well.
The image above shows an example of a histogram where the scene is well exposed. We can tell this because most of the pixels are in the upper right half of the logarithmic graph. The following is also easy to see from the histogram. Doubling the intensity, or adding one exposure stop, will “move” the hump formed by pixels with intensities of 64-128 to 128-255 (the right-most area of the histogram). We can achieve this, for example, by doubling the exposure time.
Caution
The default Color Mode is Automatic, which is identical to ToneMapping for multi-acquisition HDR captures with differing acquisition settings. Tone mapping modifies pixel values and thus affects the histogram, making it impossible to use the histogram to evaluate exposure quality. Therefore, when using the histogram, you must evaluate one acquisition at a time and set the Color Mode to UseFirstAcquisition or Automatic.
Caution
Gamma correction and Color Balance setting modifies pixel values and thus making it impossible to use the histogram to evaluate exposure quality. Therefore, you must set Gamma and Color Balance to 1.0 when using the histogram.
Introduction to the Stops Table
A second powerful tool that we will use is the Stops Table (see below). The table shows the span of the Zivid 3D cameras’ stops, where each row shows the available range for each of the exposure parameters in Zivid. Each cell represents a stop position for one of the four exposure variables in the Zivid camera: aperture, exposure time, brightness and gain. To increase exposure with one stop, we move one cell to the right for a specific variable. To decrease exposure with one stop, we move one cell to the left for a specific variable. Example: the current exposure time is 10 000 μs. We want to increase the exposure with one stop. We increase the exposure time to 20 000 μs.
The color coding of the cells shows how “good” the values are:
Strong green: Great
Light green: Good
Yellow: OK
Red: Avoid unless necessary
Notice that we have two tables per camera. Use the correct one to compensate for 50 Hz or 60 Hz power-line frequency due to the behavior described in the Exposure Time.
Tip
Try to build your exposure values with combinations of green cells from the stops table as much as possible.
Note
The values of certain exposure settings can affect the quality of the color image. Considerations to take for good color image quality are covered later in Adjusting Acquisition Settings section in Optimizing Color Image.
Note
Different camera models have different ranges for aperture, exposure time, and brightness.
Preparation
Relaxing ROI & filters
To evaluate as many good points as possible, we start by relaxing the Region of Interest and all filters. After finding all exposure values, we set the filters as the last step.
Region Of Interest: Disabled
Cluster: Disabled
Hole Repair: Disabled
Noise Removal: 1.0
Noise Repair: Disabled
Noise Suppression: Disabled
Outlier: 50
Reflection: Disabled (if using Phase Engine)
Gaussian: Disabled
Contrast Distortion: Disabled (if using Phase Engine)
Setting color settings
To be able to use the histogram, set the color settings (see Cautions in Histogram for explanation) to the following values:
Color Balance - Red: 1.0
Color Balance - Green: 1.0
Color Balance - Blue: 1.0
Gamma: 1.0
Color Mode: UseFirstAcquisition
Acquisition #1 - exposing for highlights
We want to start by exposing the highlights and then gradually increase our exposure to include mid-tones and eventually lowlights (dark regions). If the following procedure is insufficient to achieve good data on especially shiny and bright parts, we recommend that you follow the steps in Dealing with Highlights and Shiny Objects as your first acquisition.
Step 1 - set low exposure values
We begin by turning off all but one acquisition, so we only have a single acquisition capture. Then, we set our exposure very low so that no pixels in the image (or the histogram) have an RGB value of 255. The following starting condition should work on most occasions.
Aperture (\(f\)-number): 32
Exposure time: 10 000 (8 333 in regions with 60 Hz power line frequency)
Brightness: max
Gain 1.0
After the capture, turn on the Histogram and set it to the Logarithmic mode.
If there are still regions in the image that are overexposed, as illustrated in the figure above, try to reduce the exposure time further and increase the \(f\)-number. The figure below shows a capture with no overexposed pixels. If there are no overexposed regions, go to Step 2.
Step 2 - get good exposure on brightest pixels
Decrease the \(f\)-number by moving one stop in the Stops Table at a time to the right. Carry this out until the brightest pixels have an intensity close to 255. Verify that the highlights in your scene have data by observing the point cloud or the RGB image.
In our example, we would need to increase by two stops. Our first acquisition would, therefore, consist of the following values:
Aperture (\(f\)-number): 16
Exposure time: 10 000 (8 333 in regions with 60 Hz power line frequency)
Brightness: max
Gain 1.0
In general, it is recommended to use the maximum Brightness value available for your camera model.
Acquisition #2 - exposing for mid-tones
After covering the highlights, we want to move on to exposing more of the image. By looking at the histogram, we identify the next portion of the image exposed with an intensity below 32. We want to move this region up in the upper half of the image towards 255, typically done by adding three-four stops.
Step 1 - new acquisition
Add another acquisition and disable the first acquisition.
Step 2 - increase exposure
Move a total of three or four more stops to the right. The first stop should be maximizing brightness, and the other stops should typically come from the aperture.
The example, acquisition #1 from above has an aperture of 16. Therefore, move three aperture stops to the right (count aperture columns in the Stops Table), which results in aperture 5.6.
Acquisition #2 will then have the following values:
Aperture (\(f\)-number): 5.6
Exposure time: 10 000 (8 333 in regions with 60 Hz power line frequency)
Brightness: max
Gain: 1.0
Acquisition #N - keep increasing exposure toward lowlights
Repeat the procedure for acquisition #2. Locate the next region of pixels that needs to be exposed higher and count the number of stops required to bring the intensity of those pixels close to 255. Remember to use the histogram only with one acquisition enabled.
If our next region has an intensity of around 16, we want to bring those up to about 255, which is four additional stops. We could get these stops by adding aperture, exposure time, and gain:
Acquisition #3:
Aperture (\(f\)-number): 2.8
Exposure time: 20 000 (16 667 in regions with 60 Hz power line frequency)
Brightness: max
Gain: 2.0
We might need very high exposure to extract data from the very darkest regions. This may require that the final acquisition uses high exposure time and gain.
Keep adding acquisitions until almost all pixels are on the right-hand side of the logarithmic histogram, as shown in the image below.
Enjoy the HDR
Enable all acquisitions and capture an HDR to confirm you have covered the increased dynamic range.
Pro tip: further improving noise at the expense of time
If the time budget allows it, duplicate acquisitions to perform additional SNR boosting. Grabbing multiple identical acquisitions is the equivalent of performing over-sampling and will, therefore, suppress noise by up to \(\sqrt{N}\), where \(N\) is the number of acquisitions. Duplicating acquisitions is an excellent way of maximizing point cloud quality at the expense of time. Duplicate the acquisitions exposed for the regions that require higher accuracy.
Be aware of your trade-offs
Exposure Variable |
Pros |
Cons |
Exposure Time |
|
|
Aperture |
|
|
Projector Brightness |
|
|
Gain |
|
|
Note
For Zivid cameras, changing Exposure Time is faster than changing Aperture.
Further reading
After the acquisitions have been exposed right it is time to adjust the filters as described in Adjusting Filters