Hole Filling Filter

The Hole Filling filter fills missing points in the point cloud. It interpolates remaining points in a circular neighborhood around the missing points in the point cloud. The missing points may have been removed by other filters, or not reconstructed in the first place.

Note

It is recommended to apply Removal Filters before adding Hole Filling filter. This prevents interpolation of noisy points.

Why do you need the Hole Filling filter?

If your algorithm requires a certain point cloud density, then this filter can help fill gaps where density is low.

Consider the following scene with a bin filled with consumer goods:

With the fast preset for Consumer Goods you get the result on the left below. Now, compare that with default and strong Hole Filling. This may be what makes it possible to calculate a good grasp pose on all items.

		Hole Filling filter disabled
		Hole Filling filter 0.2 (default)
		Hole Filling 0.7 (strong)

How to control the Hole Filling filter

Enabled

Turn the filter On/Off

HoleSize

Relative diameter of holes to fill. Increasing this will fill more points, but require more computation time. The maximum allowed hole size scales with distance, so that we allow filling larger holes at greater distances, measured in mm.

Strictness

Level of strictness when considering if a point should be filled. A higher level of strictness requires a missing point to be surrounded by valid points on more sides in order to be filled. Increasing this will fill fewer points, but it will be less likely to fill gaps that are not circular, for example between two edges.

The following values should be used:

Parameter	Value	Condition
Enabled	no	If you do not accept inferred data, or if you cannot afford the added processing time.
	yes	If your algorithm requires higher point cloud density, and you currently have an issue with sparse point clouds.
HoleSize	0.01	Only fill the very smallest holes (fast)
	0.2	(Default) Fill holes with an approximate radius of ~8 mm at 1000 mm distance [1]
	0.7	Fill large holes with an approximate radius of ~25 mm at 1000 mm distance [1] (slow)
Strictness	1	Fill holes even when there are few valid points in the circular neighborhood
	4	Only fill holes that are surrounded by valid points in the circular neighborhood

[1] (1,2)

To better understand how HoleSize is a function of distance check out the visualization.

Visualize Hole Size

In the visualization below the hole size is illustrated as a yellow circle. It is drawn, to scale, on top of the Zivid Calibration board. The axes show the FOV of the camera at the given distance. As such you can actually see what fraction of the FOV that the checkerboard occupies.

Processing time

Due to how the filter operates, processing time scales exponentially with the hole size parameter. This is generally not an issue, as long as you have a powerful PC, e.g. 11th Gen Intel(R) Core(TM) i9-11900K @ 3.50GHz with NVIDIA GeForce RTX 3070. For less expensive hardware, and especially integrated GPUs, one should be careful with larger hole size. It may take several seconds for the highest setting on low-end hardware.

Note

Hole Filling filter adds the following processing time per 3D capture:

Expected median (±stddev) in ms

Filter	Zivid One+			Zivid Two
	Intel UHD 750	Intel UHD G1	NVIDIA 3070	Intel UHD 750	Intel UHD G1	NVIDIA 3070
	High-end [2]	Low-end [3]	High-end [4]	High-end [2]	Low-end [3]	High-end [4]
Hole Filling	264 (±2) ms	403 (±2) ms	13 (±0.9) ms	139 (±1) ms	210 (±382) ms	10 (±1.0) ms

[2] (1,2)

High-end machine with GPU: Intel UHD Graphics 750 (ID:0x4C8A) and CPU: 11th Gen Intel(R) Core(TM) i9-11900K @ 3.50GHz, 10GbE

[3] (1,2)

Low-end machine with GPU: Intel UHD Graphics G1 (ID:0x8A56) and CPU: Intel(R) Core(TM) i3-1005G1 CPU @ 1.20GHz, 1GbE

[4] (1,2)

High-end machine with GPU: NVIDIA GeForce RTX 3070 and CPU: 11th Gen Intel(R) Core(TM) i9-11900K @ 3.50GHz, 10GbE

Special considerations

Hole filling adds points to the point cloud, but it does not actually add information. The added points are interpolated from information that remains after other filters have been applied. Hole filling is good for applications where there is some compliance in other parts of the system. For example, picking with suction cups whether it’s single SKU or multiple SKUs. These applications may perform better with dense point clouds, because there is a higher chance of providing a good grasp pose.

There are applications which depend on the absolute accuracy of each individual point. Examples include high resolution inspection and/or 3D model construction. These applications may be less suited for Hole Filling.

Version History

SDK	Changes
2.9.0	Hole Filling filter added