Getting the Right Exposure for Good Point Clouds
Introduction
The method described in this section is aimed at capturing a complete point cloud of the entire scene. In many real applications, it is only necessary to get a good exposure of certain objects or regions within the scene. In those scenarios the same method can be applied but regions that are not of interest do not need to be exposed.
In this tutorial, we will be using the histogram function in Zivid Studio to help us assess the quality of the point cloud as we find our exposure values. The goal is to expose as many pixels as possible with sufficiently high SNR so that the point cloud gets low temporal noise.
In Zivid Studio, the SNR can be seen in the 2D RGB or depth map image. Hover the mouse over a certain pixel and the SNR for that pixel will be visible in the bottom left of the screen, as shown here. An SNR value of 7 and above is considered good, and this typically relates to the RGB channels being exposed with a value between 32 and 254. The method in this article aims to end up with as many pixels as possible within this region.
For this tutorial, we will assume that the scene includes both very shiny materials, such as metal cylinders, and dark materials, such as black plastics. We will also assume that we work in an environment that is exposed to ambient light with 50 Hz power-line frequency. An example scene is shown in the image below.
Such scenes may contain a much wider dynamic range than the Zivid 3D cameras can capture within a single acquisition. Hence it is often required to use Zivid’s HDR function with multiple acquisitions for different regions of the intensity spectrum as illustrated in the image below.
Note
The Zivid Two has higher dynamic range than the Zivid One+ cameras. Hence fewer acquisitions are usually needed for Zivid Two.
The histogram
The histogram is a powerful tool for evaluating the point cloud quality. It counts how many pixels have a certain intensity value within an image and displays them in bins from 0 to 255. When displaying the histogram with a logarithmic x-axis, each bin represents one stop. This representation makes it very convenient to estimate which exposure values are needed to expose certain regions of the scene well.
The image above shows an example of a histogram where the scene is well exposed. This we can tell because most of the pixels are in the upper right half of the logarithmic graph. It is also easy to see that doubling the intensity, or adding one exposure stop, will “move” the hump of pixels around intensity 64 up to 128. This can, for example, be achieved by doubling the exposure time.
Caution
To keep 2D colours similar when using different exposure settings, the camera executes tone mapping when capturing HDR. The tone mapping might effect the histogram values making it not suitable to evaluate exposure quality. Hence when using the histogram to evaluate the exposure quality, evaluate one acquisition at the time and remember to set the ToneMapping setting in Zivid Studio to HdrOnly.
Introduction to the stops table
A second powerful tool that we will be using is the stops table. The table shows the span of the Zivid 3D cameras’ stops, where each row shows the available range for each of the exposure parameters in Zivid. Each cell represents a stop position for one of the four exposure variables in the Zivid camera; aperture, exposure time, brightness and gain. To increase exposure with one stop we move one cell to the right for a specific variable. To decrease exposure with one stop, we move one cell to the left for a specific variable. Example: current exposure time is 10 000 μs. We want to increase exposure with one stop. We increase exposure time to 20 000 μs.
Note that the table shows two rows for exposure time. Use one of the rows to compensate for 50 Hz power-line frequency or for 60 Hz power-line frequency as described in the Exposure Time. This table was created for Zivid One+. Values might vary for Zivid Two.
Note
The color coding of the cells shows how “good” the values are. The strong green color is considered a highly recommended value to use. Light green cells are also considered good, while yellow cells are not considered very good to use. Red columns should be avoided unless necessary. Thus, exposure values should be composed of parameter values in green cells as much as possible.
Tip
Try to build your exposure values consisting of combinations of green cells from the stops table.
Note
The values of certain exposure settings can affect the quality of the color image. Considerations to take for good color image quality are covered later in Adjusting Acquisition Settings section in Optimizing Color Image.
Preparation - relaxing filters
To evaluate as many good points as possible, we start by relaxing all filters. The filters will be set as the last step, after all exposure values have been found. We relax the filters by setting the following filter values:
Noise: 1.0
Gaussian: Disabled
Reflection: Disabled
Outlier: 50
Contrast distortion: Disabled
Acquisition #1 - exposing for highlights
We want to start by exposing the highlights and then gradually increase our exposure to include mid-tones and eventually lowlights (dark regions). If the following procedure is insufficient in achieving good data on especially shiny and bright parts, we recommend that you follow the steps in Dealing with Highlights and Shiny Objects as your first acquisition.
Step 1 - set low exposure values
We begin by turning off all but one acquisition, so we only have single capture. Then setting our exposure very low so that no pixels in the image (or the histogram) has an RGB value of 255. The following starting condition should work on most occasions.
Aperture (\(f\)-number): 32
Exposure time: 10 000 (8 333 in regions with 60 Hz power line frequency)
Brightness: 1.0
Gain 1.0
If there are still regions in the image that are overexposed as illustrated in the figure above, try to further reduce exposure time and increase \(f\)-number. The figure below illustrates a capture with no overexposed pixels.
Step 2 - get good exposure on brightest pixels
Decrease the \(f\)-number by moving one stop to the right at a time in the stops table until the brightest pixels have an intensity close to 255. Once this is achieved, verify by looking at the point cloud or the RGB image that the highlights have data. The highlight regions are typically on edges or on top of spheres and cylinders of shiny objects.
In our example, we would need to increase by 2 stops. Our first acquisition would, therefore, consist of the following values:
Aperture (\(f\)-number): 16
Exposure time: 10 000 (8 333 in regions with 60 Hz power line frequency)
Brightness: 1.0
Gain 1.0
Acquisition #2 - exposing for mid-tones
After the highlights have been covered, we want to move on to exposing more of the image. By looking at the histogram, we identify the next portion of the image that is exposed with an intensity below 32. We want to move this region up in the upper half of the image towards 255, which is typically done by adding 3-4 stops. The first added stop should typically be achieved by maximizing brightness because it gives the highest possible signal to noise ratio. Then we want to add 3 more stops by increasing the aperture so that we use aperture values that reside in green cells.
Step 1 - new acquisition
Add another acquisition and disable the first acquisition.
Step 2 - increase exposure
Move a total of 3 or 4 more stops to the right. The first stop should be maximizing brightness and the other stops should normally come from the aperture.
The example acquisition #1 from above had an aperture of 16. Therefore, move three aperture stops to the right which results in aperture 5.6, count aperture columns in the stops table. Finally we increase brightness from 1.0 to 1.8.
Acquisition #2 will then have the following values:
Aperture (\(f\)-number): 5.6
Exposure time: 10 000 (8 333 in regions with 60 Hz power line frequency)
Brightness: 1.8
Gain: 1.0
Step 3 - enjoy the HDR
Capture an HDR frame with the new settings from acquisition numbers 1 and 2.
Acquisition #N - keep increasing exposure towards lowlights
Repeat the procedure for acquisition #2. Locate the next region of pixels that needs to be exposed higher and count the number of stops needed to bring the intensity of those pixels close to 255.
If our next region has an intensity around 16, we want to bring those up to about 255, which is an additional 4 stops. We could get these stops by adding gain, aperture and exposure time:
Acquisition #3:
Aperture (\(f\)-number): 2.8
Exposure time: 20 000 (16 667 in regions with 60 Hz power line frequency)
Brightness: 1.8
Gain: 2.0
In order to extract data from the very darkest regions, we might need very high exposure. This may require that the final acquisition uses high exposure time and gain.
Keep adding acquisitions until almost all pixels are in the right-hand side of the logarithmic histogram as shown in the image below.
Pro tip: further improving noise at the expense of time
If the time-budget allows for it, it is possible to duplicate acquisitions to perform additional SNR boosting. Grabbing multiple identical acquisitions is the equivalent of performing over-sampling and will, therefore, suppress noise by up to \(\sqrt{N}\), where \(N\) is the number of acquisitions. This is a great way of maximizing point cloud quality at the expense of time. Duplicate the acquisitions that are exposed for the regions that require higher accuracy.
Be aware of your trade-offs
Exposure Variable |
Pros |
Cons |
Exposure Time |
|
|
Aperture |
|
|
Projector Brightness |
|
|
Gain |
|
|
Note
For Zivid One+, changing Aperture is faster than changing Exposure Time. For Zivid One+, changing Aperture is faster than changing Exposure Time.
Further reading
After the acquisitions have been exposed right it is time to adjust the filters as described in Adjusting Filters