Region of Interest Tutorial

The Region of Interest (ROI) is used to remove points outside a user-defined region of interest. The ROI can either be a box in 3D, a range of z-values from the camera, or both.

A ROI is helpful if the application only requires a portion of the field of view and not the entire scene. If you, for example, want to detect parts in a bin, your detection algorithms may benefit from the reduced search space inside the bin rather than the whole scene.

Note

The ROI does not reduce the capture time. It is applied post-process and will increase capture time by a couple of milliseconds. It is, however, applied directly on the GPU and is likely faster than a third-party implementation.

../../_images/roi.png

ROI as a Box

Three points define the base plane of the box and two extents define the height:

  • The three points are given in the camera frame of reference in 3D and define the base plane of the box. A fourth point is automatically found to bind the base plane into a frame and complete the rectangle. The three points constitute two vectors in order:

    • Point O is the origin of the vectors.

    • Point A spans the first vector from the origin.

    • Point B spans the second vector from the origin.

  • The two extents extrude the base frame into a box. The cross-product of vectors OA and OB defined by Point O, Point A and Point B gives the direction of the extents. A negative extent will therefore extrude in the opposite direction of the cross-product.

../../_images/roi_explanation.png

Illustration of the ROI box: Three points (O, A, and B) are given to define the box’s bottom plane, and a fourth point is automatically chosen to complete the rectangle. The bounded plane defined by the four points can then be extruded both upwards (+E2) and downwards (-E1) to complete the box.

Note

The base frame of the box is not constrained to a rectangle with perpendicular corners. It is therefore possible to make a parallelogram as the base, and a parallelepiped as the box.

Tip

Use the following steps when selecting the three points as a rule of thumb:

  1. Select Point O in an arbitrary corner.

  2. Select Point A such that Point B lies in a position counter-clockwise to Point A.

  3. Select Point B in a position counter-clockwise to Point A.

This way the extents will have a positive direction towards the camera.

ROI as a Depth Range

The ROI can also be defined as a range of z-values from the camera, where points within a minimum depth threshold and a maximum depth threshold are kept. This is useful if you have points in the foreground or background of the scene that you want to filter out. Note that the z-values are given in the camera frame of reference and will therefore filter perpendicularly to the camera. This will thus work best if the camera is mounted perpendicularly to the objects you want to capture.

../../_images/roi_depth.png

ROI API

The ROI is a part of the Camera Settings and is set under the Zivid SDK main settings object. It is therefore applied when you capture, and not on the point cloud object afterward.

Go to source

source

std::cout << "Configuring settings for capture:" << std::endl;
Zivid::Settings settings{
    Zivid::Settings::Experimental::Engine::phase,
    Zivid::Settings::Sampling::Color::rgb,
    Zivid::Settings::Sampling::Pixel::all,
    Zivid::Settings::RegionOfInterest::Box::Enabled::yes,
    Zivid::Settings::RegionOfInterest::Box::PointO{ 1000, 1000, 1000 },
    Zivid::Settings::RegionOfInterest::Box::PointA{ 1000, -1000, 1000 },
    Zivid::Settings::RegionOfInterest::Box::PointB{ -1000, 1000, 1000 },
    Zivid::Settings::RegionOfInterest::Box::Extents{ -1000, 1000 },
    Zivid::Settings::RegionOfInterest::Depth::Enabled::yes,
    Zivid::Settings::RegionOfInterest::Depth::Range{ 200, 2000 },
    Zivid::Settings::Processing::Filters::Smoothing::Gaussian::Enabled::yes,
    Zivid::Settings::Processing::Filters::Smoothing::Gaussian::Sigma{ 1.5 },
    Zivid::Settings::Processing::Filters::Noise::Removal::Enabled::yes,
    Zivid::Settings::Processing::Filters::Noise::Removal::Threshold{ 7.0 },
    Zivid::Settings::Processing::Filters::Noise::Suppression::Enabled::yes,
    Zivid::Settings::Processing::Filters::Noise::Repair::Enabled::yes,
    Zivid::Settings::Processing::Filters::Outlier::Removal::Enabled::yes,
    Zivid::Settings::Processing::Filters::Outlier::Removal::Threshold{ 5.0 },
    Zivid::Settings::Processing::Filters::Reflection::Removal::Enabled::yes,
    Zivid::Settings::Processing::Filters::Reflection::Removal::Experimental::Mode::global,
    Zivid::Settings::Processing::Filters::Cluster::Removal::Enabled::yes,
    Zivid::Settings::Processing::Filters::Cluster::Removal::MaxNeighborDistance{ 10 },
    Zivid::Settings::Processing::Filters::Cluster::Removal::MinArea{ 100 },
    Zivid::Settings::Processing::Filters::Experimental::ContrastDistortion::Correction::Enabled::yes,
    Zivid::Settings::Processing::Filters::Experimental::ContrastDistortion::Correction::Strength{ 0.4 },
    Zivid::Settings::Processing::Filters::Experimental::ContrastDistortion::Removal::Enabled::no,
    Zivid::Settings::Processing::Filters::Experimental::ContrastDistortion::Removal::Threshold{ 0.5 },
    Zivid::Settings::Processing::Filters::Experimental::HoleFilling::Enabled::yes,
    Zivid::Settings::Processing::Filters::Experimental::HoleFilling::HoleSize{ 0.2 },
    Zivid::Settings::Processing::Filters::Experimental::HoleFilling::Strictness{ 1 },
    Zivid::Settings::Processing::Color::Balance::Red{ 1.0 },
    Zivid::Settings::Processing::Color::Balance::Green{ 1.0 },
    Zivid::Settings::Processing::Color::Balance::Blue{ 1.0 },
    Zivid::Settings::Processing::Color::Gamma{ 1.0 },
    Zivid::Settings::Processing::Color::Experimental::Mode::automatic
};
std::cout << settings << std::endl;
Go to source

source

Console.WriteLine("Configuring settings for capture:");
var settings = new Zivid.NET.Settings()
{
    Experimental = { Engine = Zivid.NET.Settings.ExperimentalGroup.EngineOption.Phase },
    Sampling = { Color = Zivid.NET.Settings.SamplingGroup.ColorOption.Rgb, Pixel = Zivid.NET.Settings.SamplingGroup.PixelOption.All },
    RegionOfInterest = { Box = {
                            Enabled = true,
                            PointO = new Zivid.NET.PointXYZ{ x = 1000, y = 1000, z = 1000 },
                            PointA = new Zivid.NET.PointXYZ{ x = 1000, y = -1000, z = 1000 },
                            PointB = new Zivid.NET.PointXYZ{ x = -1000, y = 1000, z = 1000 },
                            Extents = new Zivid.NET.Range<double>(-1000, 1000),
                        },
                        Depth =
                        {
                            Enabled = true,
                            Range = new Zivid.NET.Range<double>(200, 2000),
                        }
    },
    Processing = { Filters = { Smoothing = { Gaussian = { Enabled = true, Sigma = 1.5 } },
                               Noise = { Removal = { Enabled = true, Threshold = 7.0 },
                                         Suppression = { Enabled = true },
                                         Repair ={ Enabled = true } },
                               Outlier = { Removal = { Enabled = true, Threshold = 5.0 } },
                               Reflection = { Removal = { Enabled = true, Experimental = { Mode = ReflectionFilterModeOption.Global} } },
                               Cluster = { Removal = { Enabled = true, MaxNeighborDistance = 10, MinArea = 100} },
                               Experimental = { ContrastDistortion = { Correction = { Enabled = true,
                                                                                      Strength = 0.4 },
                                                                       Removal = { Enabled = true,
                                                                                   Threshold = 0.5 } },
                                                HoleFilling = { Enabled = true,
                                                                HoleSize = 0.2,
                                                                Strictness = 1 } } },
                   Color = { Balance = { Red = 1.0, Green = 1.0, Blue = 1.0 },
                             Gamma = 1.0,
                             Experimental = { Mode = ColorModeOption.Automatic } } }
};
Console.WriteLine(settings);
Go to source

source

print("Configuring settings for capture:")
settings = zivid.Settings()
settings.experimental.engine = "phase"
settings.sampling.color = "rgb"
settings.sampling.pixel = "all"
settings.region_of_interest.box.enabled = True
settings.region_of_interest.box.point_o = [1000, 1000, 1000]
settings.region_of_interest.box.point_a = [1000, -1000, 1000]
settings.region_of_interest.box.point_b = [-1000, 1000, 1000]
settings.region_of_interest.box.extents = [-1000, 1000]
settings.region_of_interest.depth.enabled = True
settings.region_of_interest.depth.range = [200, 2000]
filters = settings.processing.filters
filters.smoothing.gaussian.enabled = True
filters.smoothing.gaussian.sigma = 1.5
filters.noise.removal.enabled = True
filters.noise.removal.threshold = 7.0
filters.noise.suppression.enabled = True
filters.noise.repair.enabled = True
filters.outlier.removal.enabled = True
filters.outlier.removal.threshold = 5.0
filters.reflection.removal.enabled = True
filters.reflection.removal.experimental.mode = "global"
filters.cluster.removal.enabled = True
filters.cluster.removal.max_neighbor_distance = 10
filters.cluster.removal.min_area = 100
filters.experimental.contrast_distortion.correction.enabled = True
filters.experimental.contrast_distortion.correction.strength = 0.4
filters.experimental.contrast_distortion.removal.enabled = False
filters.experimental.contrast_distortion.removal.threshold = 0.5
filters.experimental.hole_filling.enabled = True
filters.experimental.hole_filling.hole_size = 0.2
filters.experimental.hole_filling.strictness = 1
color = settings.processing.color
color.balance.red = 1.0
color.balance.blue = 1.0
color.balance.green = 1.0
color.gamma = 1.0
settings.processing.color.experimental.mode = "automatic"
print(settings)

Tutorials

Check out the below tutorials for examples of how to use the ROI API.

Version History

SDK

Changes

2.9.0

ROI API is added.