应用要求

Piece picking involves precisely picking and placing individual items from bins and totes. The millions of different items, often called SKUs, vary in shapes, sizes, and materials. A vision system must be capable of providing high-quality point cloud data for all these SKUs. Simultaneously, the robot system must ensure a low cycle time and a high throughput rate. Finding the line where the data quality is satisfactory, and the solution is time efficient is vital for a successful application.

The main objective for successfully picking an object is to compute an accurate picking pose for the robot. Both 2D and 3D data are helpful for finding the picking pose. In 2D, object segmentation or recognition is typically done with, e.g., template matching or machine learning networks. For utilizing 2D data, it is especially important to minimize defocus, blooming, and saturation. In 3D, object segmentation, detection, or pose estimation is typically done with, e.g., CAD model matching, machine learning networks, or geometry matching. For utilizing 3D data, it is vital to minimize false and missing points. For a more comprehensive understanding, we’ve classified the application requirements into the following sections:

Cycle times

In piece picking, robot cycles generally range between 3 to 4 seconds, reaching an hourly pick rate of over 1000 items per robot. While the robot places an item, the vision system should capture, process and compute the next picking pose before the robot is back. This makes time budgets strict. The camera time budget typically ranges from 350 ms to 700 ms.

Object shape and surface

如今，单品拣选已应用于物流行业中的各种用例。因此，需要处理的产品种类繁多。现在，我们将重点关注其中的一些对象和场景。这里展示的场景类别是经过精心挑选的，因为它们各自构成了不同类型的挑战。因此，我们列出了各种场景中尤为重要的应予保留的对象特征。单品拣选中的典型挑战性场景有：

包含纹理的场景

Features:

• Gaps

• Depth differences

• Transitions

• Shape

对于这些场景，最重要的是保留2D和3D图像的边缘。锐利的边缘使人们能够清楚地看到，例如，紧密堆叠的物体之间的间隙，以及重叠摆放的薄物体的深度差异。这意味着很容易看出一个对象在哪里结束，另一个对象从哪里开始。这对于2D和3D中的检测和分割算法都很重要。保持物体的3D形状也很重要，因为它能够使用3D数据进行检测和位姿估计算法的平面拟合。对于2D检测，还应保留其纹理，即最小化饱和度和光晕。

Scene: Closely stacked objects	Scene: Thin, overlapping objects
Features: Gaps	Features: Depth differences

Scene: Wrinkled, deformable objects, and very dark objects	Scene: Tiny objects
Features: Transitions	Features: Shape

反光场景

Features:

• Shape

• Surface coverage

对于这些场景，保持物体的3D形状是非常重要的，表面覆盖也应尽可能连续，这对于3D检测算法尤为重要。对于2D检测和对象分割算法，边缘应该清晰可见。此外，为确保能够在2D中进行对象检测，还应保留其纹理，即最小化饱和度和光晕。

Scene: Objects in poly bags	Scene: Highly reflective objects
Features: Surface coverage	Features: Shape and surface coverage

Scene: Objects in transparent packing	Scene: Objects in bubble wrap
Features: Surface coverage	Features: Surface coverage

Transparent scenes:

Features:

• Surface coverage

• Shape

Transparent and semi-transparent objects are among the most challenging objects to identify in piece-picking applications. These objects often lack well-defined edges and surface features, and the data may be noisy and patchy. This makes it difficult to represent their true shape and, for many object detection and pose estimation algorithms, to correctly identify them. Having good surface coverage is crucial for successfully detecting and picking these objects. Lighting conditions, background, and capture angle are other factors that should be considered to ensure a successful pick.

Gripper compliance

The quality of the point cloud is often a determining factor for the type of gripper employed. For example, if the point cloud data is highly accurate, a mechanical gripper with narrow tolerances can be utilized. Otherwise, a suction cup, which has more compliance, might be necessary. Because of the huge variety of objects, shape, sizes and material in piece picking applications, suction cup is commonly used. This extra compliance minimizes the chance of not reaching or crashing into objects and destroying them or the gripper. Dimension trueness, point precision, and planarity are other factors determining the level of compliance one needs in the gripper.

Motion planning and collision avoidance

An additional element to consider in piece picking is motion planning and collision avoidance. Motion planning is used to optimize the robot’s trajectories while picking, thus, saving cycle time. It is often paired with collision avoidance to avoid crashing into obstacles like bin walls, objects not currently being picked and other environmental restrictions. The obstacles seen by the vision system are then avoided by the robot. In an ideal world, the vision system would have an exact overlapping representation of the environment as it is. However, artifacts do arise. These artifacts comprise false or missing data that do not align with the real world. False data are, for instance, seen as ghost planes or floating blobs that do not exist in reality, whereas missing data are seen as holes in the point cloud. The latter is a result of incomplete surface coverage and comprises data that should have existed in the point cloud. Due to artifacts, collision avoidance may hinder the robot from reaching its destination. Hence, motion planning needs to define which obstacles are safe to disregard and which are not. With the increased quality of 3D data from the camera, the complexity of gripper compliance and motion planning with collision avoidance can be reduced.

本章节讨论了单品拣选应用的要求。现在，下一步是根据场景立体空间的要求 选择合适的Zivid相机 。