Region of Interest Tutorial
The Region of Interest (ROI) removes points outside a user-defined region of interest and can reduce capture time. The ROI can either be a box in 3D, a range of z-values from the camera, or both.
ROI는 애플리케이션이 전체 장면이 아닌 시야의 일부만 요구하는 경우에 유용합니다. 예를 들어 빈에서 부품을 감지하려는 경우 감지 알고리즘은 전체 장면이 아닌 빈 내부 로만 검색 공간이 줄어들면 이점을 얻을 수 있습니다.
ROI as a Box
ROI box filtering benefits
ROI box filtering provides one of the following benefits:
- Reduced capture time (same point cloud quality)
There is less data to transfer, copy, and process; significant speed-ups are realized with one or both:
Weak GPU (Intel/Jetson)
Heavy point cloud processing (Omni/Stripe, 5MP resolution, multi-acquisition HDR)
- Better point cloud quality (same capture time)
Different settings can be used; for example:
Stripe/Omni engine instead of Phase
Higher resolution with Sampling::Pixel
Adding more acquisitions in HDR
- Cheaper GPU (same capture time)
With less data to process, the processing can be slower, allowing you to use, for example:
Intel/Jetson instead of dedicated Nvidia
- Cheaper Network Card (same capture time)
With less data to transfer you may not saturate the network, allowing you to use, for example:
1G instead of 2.5G/10G
2.5G instead of 10G
The parameter RegionOfInterest::Box enables using ROI as a box.
Three points define the base plane of the box and two extents define the height:
The three points (
Box::PointO,Box::PointA,Box::PointB) are given in the camera frame of reference in 3D and define the base plane of the box. A fourth point is automatically found to bind the base plane into a frame and complete the rectangle. The three points constitute two vectors in order:포인트 O는 벡터의 원점입니다.
포인트 A는 원점에서 첫 번째 벡터를 구성합니다.
포인트 B는 원점에서 두 번째 벡터를 구성합니다.
The two extents (
Box::Extents) extrude the base frame into a box. The cross-product of vectors OA and OB defined by Point O, Point A and Point B gives the direction of the extents. A negative extent will therefore extrude in the opposite direction of the cross-product.
ROI 상자 그림: 상자의 바닥 평면을 정의하기 위해 세 개의 포인트(O, A 및 B)이 지정되고 네 번째 점이 자동으로 선택되어 사각형을 완성합니다. 그런 다음 네 개의 포인트으로 정의된 경계 평면을 위쪽(+E2)과 아래쪽(-E1)으로 돌출시켜 상자를 완성할 수 있습니다.
참고
상자의 기본 프레임은 수직 모서리가 있는 직사각형으로 제한되지 않습니다. 따라서 평행사변형을 밑변으로 만들고 평행육면체를 상자로 만드는 것이 가능합니다.
팁
몇가지 팁을 소개합니다. 세 점을 선택할 때 다음 단계를 따라해 보십시오.
임의의 모서리에서 포인트 O를 선택합니다.
포인트 B가 포인트 A에 대해 반시계 방향 위치에 있도록 포인트 A를 선택합니다.
포인트 A에 대해 반시계 방향 위치에서 포인트 B를 선택합니다.
이렇게 하면 익스텐트가 카메라를 향한 양의 방향을 갖게 됩니다.
Configure ROI box in Zivid Studio
참고
When using the point picker you cannot select a pixel that is a NaN in the point cloud. You can therefore only select pixels that correspond to actual points in the point cloud.
Once you have created the ROI box in Zivid Studio you can export the settings YAML and load it back into your application.
const auto settingsFile = "Settings.yml";
std::cout << "Loading settings from file: " << settingsFile << std::endl;
const auto settingsFromFile = Zivid::Settings(settingsFile);
var settingsFile = "Settings.yml";
Console.WriteLine("Loading settings from file: " + settingsFile);
var settingsFromFile = new Zivid.NET.Settings(settingsFile);
ROI box examples and Performance
Robot Picking example
Here is an example of heavy point cloud processing using a low-end PC (Nvidia MX 250 laptop GPU and a 1G network card) where ROI box filtering is very beneficial.
Imagine a robot picking from a 600 x 400 x 300 mm bin with a stationary-mounted Zivid 2+ M130 camera. For good robot clearance, the camera is mounted at 1700 mm distance from the bin top, providing a FOV of approximately 1000 x 800 mm. With the camera mounted at this distance, a lot of the FOV is outside of the ROI, allowing one to crop ~20/25% of pixel columns/rows from each side. As can be seen from the table below, the capture time can be significantly reduced with ROI box filtering.
Presets |
Acquisition Time |
Capture Time |
||
|---|---|---|---|---|
No ROI |
ROI |
No ROI |
ROI |
|
Manufacturing Specular |
0.64 s |
0.64 s |
2.1 s |
1.0 s |
Consumer Goods Quality |
0.90 s |
0.88 s |
5.0 s |
3.0 s |
Robot Guidance / Assembly example
Here is an example of heavy point cloud processing using a low-end PC (Nvidia MX 250 laptop GPU and a 1G network card) where ROI box filtering is very beneficial.
In some applications in robot guidance (e.g., drilling, welding, gluing) or assembly (e.g., peg-in-hole), the ROI can be very small compared to the FOV of the camera at imaging distance. In such scenarios, with ROIs often comprising only 5-10% of the points in the point cloud, the capture time can drastically be reduced utilizing ROI box filtering (see table below).
Presets |
Acquisition Time |
Capture Time |
||
|---|---|---|---|---|
No ROI |
ROI |
No ROI |
ROI |
|
Manufacturing Specular |
0.65 s |
0.63 s |
1.3 s |
0.7 s |
Manufacturing Small Features |
0.70 s |
0.65 s |
5.0 s |
0.8 s |
ROI as a Depth Range
The parameter RegionOfInterest::Depth enables to use ROI as a range of z-values from the camera, where points are kept within the following thresholds:
minimum depth threshold (
RegionOfInterest::Depth::minValue)maximum depth threshold (
RegionOfInterest::Depth::maxValue)
This is useful if you have points in the foreground or background of the scene that you want to filter out. Note that the z-values are given in the camera frame of reference and will therefore filter perpendicularly to the camera. This will thus work best if the camera is mounted perpendicularly to the objects you want to capture.
ROI API
ROI는 Camera Settings 의 일부이며 Zivid SDK 기본 설정 개체 아래에 설정됩니다. 따라서 캡처할 때 적용되며 나중에 포인트 클라우드 개체에는 적용되지 않습니다.
팁
ROI as a Box reduces capture time; ROI as a Depth Range does not.
std::cout << "Configuring settings for capture:" << std::endl;
Zivid::Settings settings{
Zivid::Settings::Engine::phase,
Zivid::Settings::Sampling::Color::rgb,
Zivid::Settings::Sampling::Pixel::blueSubsample2x2,
Zivid::Settings::RegionOfInterest::Box::Enabled::yes,
Zivid::Settings::RegionOfInterest::Box::PointO{ 1000, 1000, 1000 },
Zivid::Settings::RegionOfInterest::Box::PointA{ 1000, -1000, 1000 },
Zivid::Settings::RegionOfInterest::Box::PointB{ -1000, 1000, 1000 },
Zivid::Settings::RegionOfInterest::Box::Extents{ -1000, 1000 },
Zivid::Settings::RegionOfInterest::Depth::Enabled::yes,
Zivid::Settings::RegionOfInterest::Depth::Range{ 200, 2000 },
Zivid::Settings::Processing::Filters::Smoothing::Gaussian::Enabled::yes,
Zivid::Settings::Processing::Filters::Smoothing::Gaussian::Sigma{ 1.5 },
Zivid::Settings::Processing::Filters::Noise::Removal::Enabled::yes,
Zivid::Settings::Processing::Filters::Noise::Removal::Threshold{ 7.0 },
Zivid::Settings::Processing::Filters::Noise::Suppression::Enabled::yes,
Zivid::Settings::Processing::Filters::Noise::Repair::Enabled::yes,
Zivid::Settings::Processing::Filters::Outlier::Removal::Enabled::yes,
Zivid::Settings::Processing::Filters::Outlier::Removal::Threshold{ 5.0 },
Zivid::Settings::Processing::Filters::Reflection::Removal::Enabled::yes,
Zivid::Settings::Processing::Filters::Reflection::Removal::Mode::global,
Zivid::Settings::Processing::Filters::Cluster::Removal::Enabled::yes,
Zivid::Settings::Processing::Filters::Cluster::Removal::MaxNeighborDistance{ 10 },
Zivid::Settings::Processing::Filters::Cluster::Removal::MinArea{ 100 },
Zivid::Settings::Processing::Filters::Experimental::ContrastDistortion::Correction::Enabled::yes,
Zivid::Settings::Processing::Filters::Experimental::ContrastDistortion::Correction::Strength{ 0.4 },
Zivid::Settings::Processing::Filters::Experimental::ContrastDistortion::Removal::Enabled::no,
Zivid::Settings::Processing::Filters::Experimental::ContrastDistortion::Removal::Threshold{ 0.5 },
Zivid::Settings::Processing::Filters::Hole::Repair::Enabled::yes,
Zivid::Settings::Processing::Filters::Hole::Repair::HoleSize{ 0.2 },
Zivid::Settings::Processing::Filters::Hole::Repair::Strictness{ 1 },
Zivid::Settings::Processing::Resampling::Mode::upsample2x2,
Zivid::Settings::Processing::Color::Balance::Red{ 1.0 },
Zivid::Settings::Processing::Color::Balance::Green{ 1.0 },
Zivid::Settings::Processing::Color::Balance::Blue{ 1.0 },
Zivid::Settings::Processing::Color::Gamma{ 1.0 },
Zivid::Settings::Processing::Color::Experimental::Mode::automatic
};
std::cout << settings << std::endl;
Console.WriteLine("Configuring settings for capture:");
var settings = new Zivid.NET.Settings()
{
Engine = Zivid.NET.Settings.EngineOption.Phase,
Sampling = { Color = Zivid.NET.Settings.SamplingGroup.ColorOption.Rgb, Pixel = Zivid.NET.Settings.SamplingGroup.PixelOption.BlueSubsample2x2 },
RegionOfInterest = { Box = {
Enabled = true,
PointO = new Zivid.NET.PointXYZ{ x = 1000, y = 1000, z = 1000 },
PointA = new Zivid.NET.PointXYZ{ x = 1000, y = -1000, z = 1000 },
PointB = new Zivid.NET.PointXYZ{ x = -1000, y = 1000, z = 1000 },
Extents = new Zivid.NET.Range<double>(-1000, 1000),
},
Depth =
{
Enabled = true,
Range = new Zivid.NET.Range<double>(200, 2000),
}
},
Processing = { Filters = { Smoothing = { Gaussian = { Enabled = true, Sigma = 1.5 } },
Noise = { Removal = { Enabled = true, Threshold = 7.0 },
Suppression = { Enabled = true },
Repair ={ Enabled = true } },
Outlier = { Removal = { Enabled = true, Threshold = 5.0 } },
Reflection = { Removal = { Enabled = true, Mode = ReflectionFilterModeOption.Global} },
Cluster = { Removal = { Enabled = true, MaxNeighborDistance = 10, MinArea = 100} },
Hole = { Repair = { Enabled = true, HoleSize = 0.2, Strictness = 1 } },
Experimental = { ContrastDistortion = { Correction = { Enabled = true,
Strength = 0.4 },
Removal = { Enabled = true,
Threshold = 0.5 } } } },
Resampling = { Mode = Zivid.NET.Settings.ProcessingGroup.ResamplingGroup.ModeOption.Upsample2x2},
Color = { Balance = { Red = 1.0, Green = 1.0, Blue = 1.0 },
Gamma = 1.0,
Experimental = { Mode = ColorModeOption.Automatic } } }
};
Console.WriteLine(settings);
print("Configuring settings for capture:")
settings = zivid.Settings()
settings.engine = zivid.Settings.Engine.phase
settings.sampling.color = zivid.Settings.Sampling.Color.rgb
settings.sampling.pixel = zivid.Settings.Sampling.Pixel.blueSubsample2x2
settings.region_of_interest.box.enabled = True
settings.region_of_interest.box.point_o = [1000, 1000, 1000]
settings.region_of_interest.box.point_a = [1000, -1000, 1000]
settings.region_of_interest.box.point_b = [-1000, 1000, 1000]
settings.region_of_interest.box.extents = [-1000, 1000]
settings.region_of_interest.depth.enabled = True
settings.region_of_interest.depth.range = [200, 2000]
filters = settings.processing.filters
filters.smoothing.gaussian.enabled = True
filters.smoothing.gaussian.sigma = 1.5
filters.noise.removal.enabled = True
filters.noise.removal.threshold = 7.0
filters.noise.suppression.enabled = True
filters.noise.repair.enabled = True
filters.outlier.removal.enabled = True
filters.outlier.removal.threshold = 5.0
filters.reflection.removal.enabled = True
filters.reflection.removal.mode = zivid.Settings.Processing.Filters.Reflection.Removal.Mode.global_
filters.cluster.removal.enabled = True
filters.cluster.removal.max_neighbor_distance = 10
filters.cluster.removal.min_area = 100
filters.experimental.contrast_distortion.correction.enabled = True
filters.experimental.contrast_distortion.correction.strength = 0.4
filters.experimental.contrast_distortion.removal.enabled = False
filters.experimental.contrast_distortion.removal.threshold = 0.5
filters.hole.repair.enabled = True
filters.hole.repair.hole_size = 0.2
filters.hole.repair.strictness = 1
resampling = settings.processing.resampling
resampling.mode = zivid.Settings.Processing.Resampling.Mode.upsample2x2
color = settings.processing.color
color.balance.red = 1.0
color.balance.blue = 1.0
color.balance.green = 1.0
color.gamma = 1.0
settings.processing.color.experimental.mode = zivid.Settings.Processing.Color.Experimental.Mode.automatic
print(settings)
Tutorials
ROI API를 사용하는 방법에 대한 예제는 아래 튜토리얼을 확인하십시오.
For performance information, check out Region of Interest.
Version History
SDK |
Changes |
|---|---|
2.12.0 |
ROI box filtering reduces capture time. |
2.9.0 |
ROI API가 추가 되었습니다. |