Hole Repair Filter

The Hole Repair filter fills missing points in the point cloud. It interpolates remaining points in a circular neighborhood around the missing points in the point cloud. The missing points may have been removed by other filters, or not reconstructed in the first place.

Note

It is recommended to apply Removal Filters before adding Hole Repair filter. This prevents interpolation of noisy points.

Why do you need the Hole Repair filter?

If your algorithm requires a certain point cloud density, then this filter can help fill gaps where density is low.

Consider the following scene with a bin filled with consumer goods:

With the fast preset for Consumer Goods you get the result on the left below. Now, compare that with default and strong Hole Repair. This may be what makes it possible to calculate a good grasp pose on all items.

		Hole Repair filter disabled
		Hole Repair filter 0.2 (default)
		Hole Repair 0.7 (strong)

How to control the Hole Repair filter

Enabled

Turn the filter On/Off

HoleSize

Relative diameter of holes to fill. Increasing this will fill more points, but require more computation time. The maximum allowed hole size scales with distance, so that we allow filling larger holes at greater distances, measured in mm.

Strictness

Level of strictness when considering if a point should be filled. A higher level of strictness requires a missing point to be surrounded by valid points on more sides in order to be filled. Increasing this will fill fewer points, but it will be less likely to fill gaps that are not circular, for example between two edges.

The following values should be used:

Parameter	Value	Condition
Enabled	no	If you do not accept inferred data, or if you cannot afford the added processing time.
	yes	If your algorithm requires higher point cloud density, and you currently have an issue with sparse point clouds.
HoleSize	0.01	Only fill the very smallest holes (fast)
	0.2	(Default) Fill holes with an approximate diameter of ~8 mm at 1000 mm distance [1]
	0.7	Fill large holes with an approximate diameter of ~25 mm at 1000 mm distance [1] (slow)
Strictness	1	Fill holes even when there are few valid points in the circular neighborhood
	4	Only fill holes that are surrounded by valid points in the circular neighborhood

[1] (1,2)

To better understand how HoleSize is a function of distance check out the visualization.

Visualize Hole Size

In the visualization below the hole size is illustrated as a yellow circle. It is drawn, to scale, on top of the Zivid Calibration board. The axes show the FOV of the camera at the given distance. As such you can actually see what fraction of the FOV that the checkerboard occupies.

Processing time

Due to how the filter operates, processing time scales exponentially with the hole size parameter. This is generally not an issue, as long as you have a powerful PC, e.g. 11th Gen Intel(R) Core(TM) i9-11900K @ 3.50GHz with NVIDIA GeForce RTX 3070. For less expensive hardware, and especially integrated GPUs, one should be careful with larger hole size. It may take several seconds for the highest setting on low-end hardware.

Hole Repair filter adds the following processing time per 3D capture:

Special considerations

Hole Repair adds points to the point cloud, but it does not actually add information. The added points are interpolated from information that remains after other filters have been applied. Hole Repair is good for applications where there is some compliance in other parts of the system. For example, picking with suction cups whether it’s single SKU or multiple SKUs. These applications may perform better with dense point clouds, because there is a higher chance of providing a good grasp pose.

There are applications which depend on the absolute accuracy of each individual point. Examples include high resolution inspection and/or 3D model construction. These applications may be less suited for Hole Repair.

Version History

SDK	Changes
2.12.0	Hole Filling removed from experimental and renamed to Hole Repair.
2.9.0	Hole Filling filter added