Application Requirements

For a more comprehensive understanding, we’ve classified the application requirements into the following sections:

Simple vs Complex Scenarios

In 3PL (3rd Party Logistics) the pallets are often loaded with a single SKU. The items are often cardboard boxes with printed labels. From a vision perspective, this is a relatively simple scenario. There are still challenges however, such as the long distance when the pallet is almost empty, and the reflective surfaces of the boxes. There can be artwork on the boxes, or tape sealing the boxes, which can be shiny and reflective.

In, for example, grocery logistics there are more challenges, and the scenario is more complex. Pallets in mixed SKU scenarios contain items of various shapes, sizes, and materials — such as boxes, bags, bottles, cans, and jars. Materials may include cardboard, plastic, glass, or metal. Items are often stacked in different configurations, ranging from tightly packed to loosely arranged.

In the following image you can see both a simple scenario on the left, and a more complex scenario on the right.

Depth

For best utilization of pallets and to maximize the number of items per pallet, the items are often stacked as high as possible. This is typically only limited by where the pallets are stored, including during transport, and the stability of the stack. The typical maximum height of a stack is between 2.0 to 2.5 meters. For a stationary camera setup, this means that the camera must be able to acquire data from as close as 1.5 meters to as far as 4.5 meters. It must be possible to get high quality 3D data across this entire range.

Another complication is the layer depth. By this we mean the distance from the top of the highest item to the top of the lowest item, visible at any time during picking or placing. The difference here is both a challenge for quality 3D data, as well as occlusion. An item on the top layer may occlude items on lower layers. The larger the baseline of the camera, the more problematic occlusion becomes.

Palletization vs Depalletization

Items will either be added to a pallet (palletization) or removed from a pallet (depalletization). The two different scenarios have different challenges.

In palletization it is important that the point cloud is accurate and complete such that an available location for placement can be accurately determined. There should be no artifacts that may be misinterpreted as an inaccessible area. It also means that occlusion should be minimized, such that the maximum number of available locations can be identified.

In depalletization it is more important to find a safe grasp point. By safe grasp point we mean a point that represents a real surface, that will allow a suction cup to achieve vacuum without leakage. It also means that it represents a point that is near the center of mass of item. To avoid duplicate picks it is also important that the grasp point avoids edges of items. This is all a combination of well segmented objects, and complete and accurate point cloud.

Often segmentation is performed on color images. This means that the color image must also be of high quality and easily map to the point cloud.

Cycle times

In (de-)palletization, the robot is typically larger to have the reach to pick from and place onto pallets. This means that the robot is often slower than in piece-picking applications. However, the cycle times are still important to maximize throughput. Robot cycles generally range between 4 to 10 seconds, reaching an hourly pick rate of around 1000 items per robot. While the robot places an item, the vision system should capture, process and compute the next picking pose before the robot is back. This makes time budgets less strict than .e.g. piece-picking, but still important. The camera time budget typically ranges from 500 ms to 1500 ms.

Gripper compliance

The quality of the point cloud is often a determining factor for the type of gripper employed. For example, if the point cloud data is highly accurate, a gripper with low compliance can be used, which is typically faster and more precise. A complete point cloud allows the vision algorithm to compute better grasping points, with enough surface area for the gripper to hold onto.

Because of the huge variety of objects, shape, sizes and material in (de-)palletization applications, suction cup is commonly used. This extra compliance minimizes the chance of not reaching or crashing into objects and destroying them or the gripper. Dimension trueness, point precision, and planarity are other factors determining the level of compliance one needs in the gripper.

Items in (de-)palletization are often heavy which means that the gripper is often a hybrid gripper with both suction cups and a mechanical option (e.g. clamp or tray).

Motion planning and collision avoidance

An additional element to consider in (de-)palletization is motion planning and collision avoidance. Motion planning is used to optimize the robot’s trajectories while picking and placing, thus, saving cycle time. It is often paired with collision avoidance to avoid crashing into obstacles like cage walls, objects not currently being picked and other environmental restrictions, such as other objects on a pallet in a palletizing application. The obstacles seen by the vision system are then avoided by the robot. In an ideal world, the vision system would have an exact overlapping representation of the environment as it is. However, artifacts do arise. These artifacts comprise false or missing data that do not align with the real world. False data are, for instance, seen as ghost planes or floating blobs that do not exist in reality, whereas missing data are seen as holes in the point cloud. The latter is a result of incomplete surface coverage and comprises data that should have existed in the point cloud. Due to artifacts, collision avoidance may hinder the robot from reaching its destination. Hence, motion planning needs to define which obstacles are safe to disregard and which are not. With the increased quality of 3D data from the camera, the complexity of gripper compliance and motion planning with collision avoidance can be reduced.

In summary, higher quality 3D data allows for faster and safer robot operation, which in turn increases throughput.

This section has reviewed the requirements for (de-)palletization. Now, the next step is to select the correct Zivid camera based on your scene volume.