Adjusting Filters

Introduction

After exposure values and frames have been found as described in Getting the Right Exposure for Good Point Clouds, it is time to adjust filters. More information about the filters can be found here: Processing Settings.

Noise filter

Noise filter removes pixels that have an SNR value below a certain limit. This can be used to remove noisy points (e.g. due to ambient light).

Value	Condition
7	“Almost always”
5-6	It may be necessary to keep the Noise filter value low in order to get points in the following scenes: dark parts strong ambient light large distance

Note

SNR filter adds close to no additional processing time.

Outlier filter

The Outlier filter removes pixels that it deems as outliers within a small local region of neighboring pixels. The outliers are measured by how far away they are from their closest neighbors. If the distance exceeds the threshold of the Outlier filter, the pixel is identified as an outlier and thus removed from the final point cloud. The unit of the Outlier filter threshold is in millimeters, and the smaller the threshold, the more pixels are removed. It is recommended to start with a relaxed Outlier filter and then tighten the threshold until you are satisfied with the density/noise level.

Value(mm)	Condition
2-10	Typical for Zivid Two
1-3	Typical for Zivid One+ Small
3-10	Typical for Zivid One+ Medium
10-30	Typical for Zivid One+ Large
50-100	Can be considered if using the camera in one of the following conditions: The image is very defocused (see 2D image in studio) Object is very dark Ambient light is strong relative to projector Scene is far away (>2m)
OFF	If you have extreme noise, but just want whatever points you can get you may turn the filter off. This could be in an extremely dark scene, very far away (>2m) or outdoors.

Note

The unit for the Outlier filter is in millimeters

Note

The outlier filter adds close to no additional processing time.

Reflection filter

The Reflection filter removes points that Zivid deem as impacted by reflection and thus erroneous. These are typically seen in the point cloud as “ghost planes” - small regions of points floating mid-air that shouldn’t be there.

Note

Enabling the Reflection filter adds certain amount of processing time per acquisition; see below.

The following values should be used:

Value	Condition
Enabled::no	If you know that your scene is not reflective, and you want to optimize acquisition time.
Enabled::yes	Turn on if you have issues detecting parts that are reflective or are inside a reflective bin.

The Reflection filter has two modes: Global (default) and Local.

Mode	Pros
Mode::local	Removing reflection artifacts such as ghost planes and highlights (see image below) Preserving 3D data on thin and pointy objects (see image below, or for more details How to preserve Thin and Pointy Objects in the Point Cloud) Processing time (up to three times faster, see processing time for global below)
Mode::global	Removing outlier points in the point cloud (see image below) Preserving the edges of objects

Reflection Filter disabled (ghost plane)	Reflection Filter enabled - Global mode (smaller ghost plane)	Reflection Filter enabled - Local mode (no ghost plane)

Reflection Filter - Global mode (no data on thin objects)	Reflection Filter - Local mode (data preserved on thin objects)

Reflection Filter - Global mode (no data on pointy objects)	Reflection Filter - Local mode (data preserved on pointy objects)

Reflection Filter - Global mode (less outliers)	Reflection Filter - Local mode (more outliers)

Hint

The local mode of the reflection filter may leave isolated floating points. This is what the Cluster Filter can compensate for.

Note

Reflection filter (Global mode) adds the following processing time per 3D capture:

Expected median (±stddev) in ms

Filter	Zivid One+			Zivid Two
	Intel UHD 750	Intel UHD G1	NVIDIA 3070	Intel UHD 750	Intel UHD G1	NVIDIA 3070
	High-end [1]	Low-end [2]	High-end [3]	High-end [1]	Low-end [2]	High-end [3]
Reflection Global	12 (±2) ms	33 (±2) ms	15 (±0.6) ms	21 (±1) ms	29 (±414) ms	14 (±0.9) ms
Reflection Local	20 (±3) ms	26 (±2) ms	4 (±0.6) ms	21 (±3) ms	25 (±474) ms	3 (±0.9) ms

[1] (1,2)

High-end machine with GPU: Intel UHD Graphics 750 (ID:0x4C8A) and CPU: 11th Gen Intel(R) Core(TM) i9-11900K @ 3.50GHz, 10GbE

[2] (1,2)

Low-end machine with GPU: Intel UHD Graphics G1 (ID:0x8A56) and CPU: Intel(R) Core(TM) i3-1005G1 CPU @ 1.20GHz, 1GbE

[3] (1,2)

High-end machine with GPU: NVIDIA GeForce RTX 3070 and CPU: 11th Gen Intel(R) Core(TM) i9-11900K @ 3.50GHz, 10GbE

Cluster filter

The Cluster Filter removes floating points and isolated clusters from the point cloud.

Tip

This filter is especially useful if you use collision avoidance while picking objects. Let’s say there is a cluster of points above an object, but that cluster doesn’t exist in the reality. The robot can interpret such floating clusters as real obstacles instead of something it can physically collide with. That can lead to undesirable picking poses of the object or even the inability to pick it.

How to control the Cluster Filter

Enabled

Turn the filter On/Off

MaxNeighborDistance

Maximum normalized distance between neighboring points that are still classified as belonging to the same cluster. The filter removes clusters whose neighboring points are further away than the MaxNeighborDistance (clusters less dense than this threshold). The lower the value, the more aggressive the filtering.

MinArea

Area of the cluster, in square millimeters. The filter removes clusters with an area below this threshold. Increasing this parameter results in removing larger clusters.

For a point to be removed, it must belong to a cluster that meets the MaxNeighborDistance and the MinArea requirements.

Parameter	Value	Condition
Enabled	no	If you do not have floating points, isolated clusters, or if you cannot afford the added processing time
	yes	If you want to remove floating points or isolated clusters
MaxNeighborDistance	10 - 8.5	Sparse clusters, works for most scenes
	8.5 - 5.5	Dense clusters
	5.5 - 4	Very dense clusters, normally originated by reflections
MinArea	< 50	For clusters that the Outlier Filter failed to remove
	50 - 400	For clusters of typical size
	> 400	For removing ghost planes, typically caused by reflections

Removing floating clusters

The Noise Filter and the Outlier Filter are basic removal filters that are almost always used. These filters together with the Reflection Filter (especially the local mode) may not be able to remove all the floating points and clusters without removing too many good points. This is typically the case for challenging scenes. The Cluster filter can remove the remaining erroneous points without removing good data from the rest of the scene.

Complementing the Outlier Filter

In the following image, the Cluster Filter is turned off and we can see collections of floating points in different sizes and densities.

In the bottom left image, we see how the Outlier filter behaves when it’s the most strict possible, keeping some floating points while removing good points. In the bottom right image, we see how the Cluster filter (with default values in this case) removes these floating points while keeping the good points intact.

Strict Outlier Filter settings (Threshold set to 1 mm); Cluster Filter turned off - Clusters not removed	Cluster Filter (with default values) - Clusters removed

Complementing the Reflection Filter - Local mode

The Reflection Filter, especially with the Local mode, can leave more groups of points floating; take a look at the bottom left image. In the bottom right image, we can see that these floating points are removed using the Cluster Filter (with default values in this case).

Reflection Filter with Local mode and Cluster Filter turned off - Clusters are not removed	Reflection Filter with Local mode and Cluster Filter turned on (default values) - Clusters removed

Removing ghost planes

Ghost planes are false point cloud data that do not exist in reality and usually result from interreflections between the objects or interreflections with the bin wall. One end of these clusters of floating points is typically connected with a point cloud surface comprised of good data. Removing ghost planes is sometimes possible by fine-tuning the Cluster Filter. These artifacts are usually dense and have a large area. Therefore, the filter must be tuned aggressively, which can result in removing some good data, including parts of objects or whole objects.

Warning

When using the cluster filter to remove ghost planes, carefully observe whether any critical data is removed.

In the following example scene, a ghost plane above the cylinder appears due to the reflection between the top of the cylinder and the bin wall. Notice that in the color image, we can see the reflection of the cylinder on the top bin wall.

This ghost plane (bottom left image) may be interpreted as an obstacle for the robot to pick the object. Such obstacles can result in getting an undesirable picking pose or even no picking pose at all. In the bottom right image, we can see that the ghost plane is removed by tuning the Cluster Filter aggressively. In this example, the MaxNeighborDistance is set to 5.00 and the MinArea to 1400.00. As a trade-off, the bottom part of the cylinder is removed as well.

Cluster Filter off - Ghost planes	Cluster Filter on - No ghost planes

Tuning the cluster filter to remove ghost planes is easier if the bin contains a single type of object. This is because it is possible to determine the area of the smallest object feature in the point cloud that needs to be kept. Then, the MinArea can be set accordingly.

Warning

When using Intel or AMD GPU, the Cluster Filter can significantly increase the processing time.

Note

Cluster Filter adds the following processing time per 3D capture:

Expected median (±stddev) in ms

Filter	Zivid One+			Zivid Two
	Intel UHD 750	Intel UHD G1	NVIDIA 3070	Intel UHD 750	Intel UHD G1	NVIDIA 3070
	High-end [4]	Low-end [5]	High-end [6]	High-end [4]	Low-end [5]	High-end [6]
Cluster	88 (±2) ms	218 (±2) ms	8 (±0.8) ms	94 (±1) ms	222 (±400) ms	8 (±0.9) ms

[4] (1,2)

High-end machine with GPU: Intel UHD Graphics 750 (ID:0x4C8A) and CPU: 11th Gen Intel(R) Core(TM) i9-11900K @ 3.50GHz, 10GbE

[5] (1,2)

Low-end machine with GPU: Intel UHD Graphics G1 (ID:0x8A56) and CPU: Intel(R) Core(TM) i3-1005G1 CPU @ 1.20GHz, 1GbE

[6] (1,2)

High-end machine with GPU: NVIDIA GeForce RTX 3070 and CPU: 11th Gen Intel(R) Core(TM) i9-11900K @ 3.50GHz, 10GbE

Gaussian smoothing

The Gaussian filter performs smoothing of points within a small local region based on 3D distances. This can be used to suppress sparse noise, correct outliers, and align points to a grid. Sigma parameter defines the aggressiveness of the filter. The higher the sigma, the more aggressive smoothing. Note that the Gaussian filter can, in many cases, correct outliers, but high sigma will also smooth out edges.

Value	Condition
<1.0	If you want to preserve as many details and features as possible. Preferred with small objects (sub-cm features).
1.5	For most conditions and medium sized objects.
2.5-5	If you care about smoother and larger surfaces such as flatness of a plane.

Gaussian filter OFF
Gaussian filter with sigma=1.5
Gaussian filter with sigma=4.0

Note

Gaussian filter adds the following processing time per 3D capture:

Expected median (±stddev) in ms

Filter	Zivid One+			Zivid Two
	Intel UHD 750	Intel UHD G1	NVIDIA 3070	Intel UHD 750	Intel UHD G1	NVIDIA 3070
	High-end [7]	Low-end [8]	High-end [9]	High-end [7]	Low-end [8]	High-end [9]
Smoothing	7 (±2) ms	14 (±2) ms	2 (±0.8) ms	6 (±2) ms	13 (±445) ms	2 (±0.8) ms

[7] (1,2)

High-end machine with GPU: Intel UHD Graphics 750 (ID:0x4C8A) and CPU: 11th Gen Intel(R) Core(TM) i9-11900K @ 3.50GHz, 10GbE

[8] (1,2)

Low-end machine with GPU: Intel UHD Graphics G1 (ID:0x8A56) and CPU: Intel(R) Core(TM) i3-1005G1 CPU @ 1.20GHz, 1GbE

[9] (1,2)

High-end machine with GPU: NVIDIA GeForce RTX 3070 and CPU: 11th Gen Intel(R) Core(TM) i9-11900K @ 3.50GHz, 10GbE

Contrast Distortion filter

The Contrast Distortion filter corrects and/or removes points that are affected by blurring in the camera lens. In the point cloud, this typically occurs in regions with high contrast (strong specular reflection), and in regions with large texture gradients. The artifact is explained in detail in Contrast Distortion Artifact.

The filter parameter strength states how much a point is corrected in 3D space, and threshold indicates the maximum distance before a point is removed (if removal mode is on). As strength and removal depend on camera focus and orientation (to camera baseline) of the object, its recommended to first tune exposure settings before tuning Contrast Distortion Filter parameters.

Warning

If the strength value is very high, it may overcompensate correction.

Value	Condition
0.1 - 0.2	If you have small contrast distortion effects and no-to-little correction is needed.
0.3 - 0.5	“Almost always”, if your camera has good focus, but contrast distortion is present. Some correction is needed.
0.6 - 0.7	“Common”, if your point cloud is affected by a large contrast distortion. Your camera may be a bit out of focus.
0.8 - 1.0	“Uncommon”, if you have a very high effect of contrast distortion. This often indicates that the camera is out of focus!

Note

Contrast Distortion filter adds the following processing time per 3D capture:

Expected median (±stddev) in ms

Filter	Zivid One+			Zivid Two
	Intel UHD 750	Intel UHD G1	NVIDIA 3070	Intel UHD 750	Intel UHD G1	NVIDIA 3070
	High-end [10]	Low-end [11]	High-end [12]	High-end [10]	Low-end [11]	High-end [12]
Contrast Distortion Correction	57 (±2) ms	97 (±1) ms	7 (±0.7) ms	58 (±2) ms	98 (±420) ms	7 (±0.9) ms
Contrast Distortion Removal	40 (±2) ms	61 (±1) ms	4 (±0.7) ms	41 (±1) ms	60 (±435) ms	4 (±0.9) ms

[10] (1,2)

High-end machine with GPU: Intel UHD Graphics 750 (ID:0x4C8A) and CPU: 11th Gen Intel(R) Core(TM) i9-11900K @ 3.50GHz, 10GbE

[11] (1,2)

Low-end machine with GPU: Intel UHD Graphics G1 (ID:0x8A56) and CPU: Intel(R) Core(TM) i3-1005G1 CPU @ 1.20GHz, 1GbE

[12] (1,2)

High-end machine with GPU: NVIDIA GeForce RTX 3070 and CPU: 11th Gen Intel(R) Core(TM) i9-11900K @ 3.50GHz, 10GbE

Tip

If Contrast Distortion Correction filter is already enabled, then Contrast Distortion Removal adds negligible processing time.

Tuning Aperture before Contrast Distortion filter

Tuning the Contrast Distortion filter is sometimes not sufficient to remove contrast distortion artifacts. The artifacts get worse with blurry color images. Therefore, it is important to ensure the color image is in focus by tuning the Aperture before tuning the filter.

We will demonstrate the process of removing contrast distortion artifacts by tuning the Aperture with an example.

The figure below shows a color image out of focus and a resulting point cloud with visible contrast distortion artifacts. The color image is blurry because a low f-number (2.38) is used for the Aperture. The Contrast Distortion strength used is 0.35.

Color image out of focus

Resulting point cloud with contrast distortion

The Aperture is changed from 2.0 to 8.0 to remove the contrast distortion artifact, and this change is compensated with a longer Exposure Time. The result is a color image in focus and a point cloud without any contrast distortion artifacts, which can be seen in the figure below.

Color image in focus

Resulting point cloud without contrast distortion

The point clouds used for this example are available for download for further inspection:

• BadFocusBadContrastDistortion.zdf

• GoodFocusNoContrastDistortion.zdf

Hole Filling

The Hole Filling filter fills missing points in the point cloud. It interpolates remaining points in a circular neighborhood around the missing points in the point cloud. The missing points may have been removed by other filters, or not reconstructed in the first place.

Note

It is recommended to apply Removal Filters before adding Hole Filling filter. This prevents interpolation of noisy points.

Enabled

Turn the filter On/Off

HoleSize

Relative diameter of holes to fill. Increasing this will fill more points, but require more computation time. The maximum allowed hole size scales with distance, so that we allow filling larger holes at greater distances, measured in mm.

Strictness

Level of strictness when considering if a point should be filled. A higher level of strictness requires a missing point to be surrounded by valid points on more sides in order to be filled. Increasing this will fill fewer points, but it will be less likely to fill gaps that are not circular, for example between two edges.

The following values should be used:

Parameter	Value	Condition
Enabled	no	If you do not accept inferred data, or if you cannot afford the added processing time.
	yes	If your algorithm requires higher point cloud density, and you currently have an issue with sparse point clouds.
HoleSize	0.01	Only fill the very smallest holes (fast)
	0.2	(Default) Fill holes with an approximate radius of ~8 mm at 1000 mm distance [13]
	0.7	Fill large holes with an approximate radius of ~25 mm at 1000 mm distance [13] (slow)
Strictness	1	Fill holes even when there are few valid points in the circular neighborhood
	4	Only fill holes that are surrounded by valid points in the circular neighborhood

[13] (1,2)

To better understand how HoleSize is a function of distance check out the visualization.

Consider the following scene with a bin filled with consumer goods:

With the fast preset for Consumer Goods you get the result on the left below. Now, compare that with default and strong Hole Filling. This may be what makes it possible to calculate a good grasp pose on all items.

		Hole Filling filter disabled
		Hole Filling filter 0.2 (default)
		Hole Filling 0.7 (strong)

Note

Hole Filling filter adds the following processing time per 3D capture:

Expected median (±stddev) in ms

Filter	Zivid One+			Zivid Two
	Intel UHD 750	Intel UHD G1	NVIDIA 3070	Intel UHD 750	Intel UHD G1	NVIDIA 3070
	High-end [14]	Low-end [15]	High-end [16]	High-end [14]	Low-end [15]	High-end [16]
Hole Filling	264 (±2) ms	403 (±2) ms	13 (±0.9) ms	139 (±1) ms	210 (±382) ms	10 (±1.0) ms

[14] (1,2)

High-end machine with GPU: Intel UHD Graphics 750 (ID:0x4C8A) and CPU: 11th Gen Intel(R) Core(TM) i9-11900K @ 3.50GHz, 10GbE

[15] (1,2)

Low-end machine with GPU: Intel UHD Graphics G1 (ID:0x8A56) and CPU: Intel(R) Core(TM) i3-1005G1 CPU @ 1.20GHz, 1GbE

[16] (1,2)

High-end machine with GPU: NVIDIA GeForce RTX 3070 and CPU: 11th Gen Intel(R) Core(TM) i9-11900K @ 3.50GHz, 10GbE

Further reading

Continue to Optimizing Color Image.

Version History

SDK	Changes
2.9.0	Added sections for Cluster and Hole Filling filters, and updated the Gaussian section based on the new behavior introduced in 2.9.0.