Normals
Tip
Our webinar Getting your point cloud ready for your application partly covers Normals.
Introduction
This article explains surface normals, how to compute them from a point cloud in Zivid SDK, and why normals are a valuable feature. Lastly, it provides a tutorial with code examples.
A surface normal is an important property of a geometric surface. It represents a unit vector perpendicular to the tangent plane of the surface at a specific point in the point cloud. Since it is a vector, the surface normal comprises three vector elements (x, y, z) describing the 3D coordinates where each element ranges from -1 to 1. All surface normals of a plane are parallel, while the surface normals of a sphere are pointing in all possible directions.
The normal describes the surface orientation, and the rate of change in the normal describes the curvature of a surface.
Normals in Zivid Studio
The normal map provides a 2D representation of the surface normals of a scene where RGB color components describe the normal vectors. The relationship between the normal vectors and the color components are well visualized in the normal map by the bin walls and floor in the image below. This is because all the points on each surface have the normal vectors pointing more or less in the same direction.
The color convention of the normal map is negatively aligned with the camera coordinate system. Hence, normal vectors represented by blue pixels point perpendicular towards the camera. Looking at the normal map, the blue pixels can be seen on the bin floor. These normals point in the negative direction of the Z-axis of the camera coordinate system. The status bar in the bottom left corner is then close to [0, 0, -1].
Furthermore, the right bin wall is dominated by red pixels. These normals are negatively aligned with the X-axis of the camera coordinate system. This means that the surface has normal vectors pointing towards the left in the scene. Likewise, the bottom bin wall is colored green since the normals are pointing upwards in the scene. Hence, the status bar would respectively read close to [-1, 0, 0] for a red pixel and close to [0, -1, 0] for a green pixel.
As a result, the normal map is useful for seeing planes in the point cloud. The normal map is also helpful in inspecting surface curvatures, surface textures, and transitions between objects as the color gradients describe these well.
Normals API
std::cout << "Computing normals and copying them to CPU memory" << std::endl;
const auto normals = pointCloud.copyData<Zivid::NormalXYZ>();
Console.WriteLine("Computing normals and copying them to CPU memory");
var normals = pointCloud.CopyNormalsXYZ();
The Normals API computes the normal at each point in the point cloud and copies normals from the GPU memory to the CPU memory. The result is a matrix of normal vectors, one for each point in the input point cloud. The size of normals is equal to the size of the input point cloud.
Note
The Normals API in Zivid SDK is fast because the computation is done in parallel on the GPU while the point cloud data is still in GPU memory. Calculating normals with third-party libraries is likely more time-consuming: CPU computations are much slower in general, and GPU computations require another copy. See Point Cloud Capture Process for more info.
If you already have a buffer, then you can copy data directly from GPU memory to your destination buffer using copyData with the template parameter NormalXYZ.
Tip
Use Normals, Downsample, and Transform APIs through the Zivid SDK for performance reasons because the GPU performs all these computations while the point cloud data is still in GPU memory.
For the highest performance, first downsample the point cloud, then compute normals.
The normals are computed on the point cloud when it is requested to copy them. Compute normals after transforming the point cloud to have the normals in the same coordinate system as the point cloud.
Transforming normals requires transforming the point cloud first, then computing the normals from the transformed point cloud. It is not possible to only transform the normals.
Normals in applications
Different applications for various reasons require computing normals from the point cloud. A normal describes the surface orientation, and the rate of change in the normal describes the curvature of a surface.
Some examples of applications that utilize normals are CAD-based and surface-based matching, edge detection, and segmentation algorithms based on the difference of normals. Normals are also utilized by meshing algorithms and filtering algorithms to remove point cloud artifacts or outliers. For instance, normals that are almost perpendicular to the line from the camera origin to a point can indicate an artifact and be filtered out.
Compute Normals and Visualize them using PCL
This tutorial demonstrates how to compute normals from Zivid point cloud, convert point cloud and normals to PCL format, and visualize them in 3D as normal map.
First, we connect to the camera and capture a point cloud.
std::cout << "Connecting to camera" << std::endl;
auto camera = zivid.connectCamera();
std::cout << "Configuring settings" << std::endl;
Zivid::Settings settings;
for(const auto aperture : { 9.57, 4.76, 2.59 })
{
std::cout << "Adding acquisition with aperture = " << aperture << std::endl;
const auto acquisitionSettings = Zivid::Settings::Acquisition{
Zivid::Settings::Acquisition::Aperture{ aperture },
};
settings.acquisitions().emplaceBack(acquisitionSettings);
}
std::cout << "Capturing frame (HDR)" << std::endl;
const auto frame = camera.capture(settings);
const auto pointCloud = frame.pointCloud();
Then we compute the normals for the point cloud.
We then convert point cloud and normals to PCL format.
std::cout << "Creating PCL point cloud structure" << std::endl;
const auto data = pointCloud.copyData<Zivid::PointXYZColorRGBA>();
const auto pointCloudPCL = convertToPCLPointCloud(data);
std::cout << "Creating PCL normals structure suited for visualization" << std::endl;
const auto pointCloudWithNormalsPCL = convertToPCLVisualizationNormals(data, normals);
Finally, we visualize the point cloud and normals in 3D.
Version History
SDK |
Changes |
|---|---|
2.8.0 |
The normal map feature is added to Zivid Studio. |
2.5.0 |
Normals API is added. |