Unsatisfactory Hand-Eye Calibration Results (Camera Outside Working Distance)

Problem

The hand-eye calibration results are marginal, with unsatisfactory touch test results (typically marginal errors in all axes). The calibration results look like one of the following combinations:

Hand-Eye Transform	Hand-Eye Residuals	Projection Verification	Touch Test
Reasonable	Marginal	Poor	Marginal
Reasonable	Good	Good	Marginal

Example of marginal hand-eye calibration results

The Hand-Eye Transformation Matrix appears reasonable and is consistent with what one would expect from rough manual measurements (see table and image below).

Hand-Eye Transformation Matrix

0.999	-0.006	0.043	-59.008
0.005	1.000	0.026	-88.747
-0.044	-0.025	0.999	129.722
0	0	0	1

The calibration residuals are within an acceptable range, indicating that the optimization has converged successfully.

Residuals

N (Poses)	Rot. avg (°)	Rot. max (°)	Trans. avg (mm)	Trans. max (mm)
14	0.099	0.259	0.344	0.823

For smaller robots, we expect the residuals to be in the sub-millimeter range, while for larger robots, the residuals can be in the millimeter range. As for the rotational component of the residuals, we expect them to be in the sub-degree range.

Complete Residuals

Residuals

Pose	Rotation (°)	Translation (mm)
1	0.080	0.229
2	0.109	0.157
3	0.043	0.142
4	0.095	0.148
5	0.122	0.553
6	0.118	0.383
7	0.028	0.610
8	0.069	0.423
9	0.125	0.277
10	0.146	0.823
11	0.260	0.229
12	0.046	0.083
13	0.057	0.628
14	0.091	0.130

Verification by Projection succeeds.

The touch test result is unsatisfactory. For example, the tip of the touch test tool misses the center of the ArUco marker (x-axis and y-axis errors) and pushes on the ArUco marker (z-axis error).

Potential Cause

The dataset was captured with the calibration object outside the camera’s recommended working distance.

Zivid cameras are calibrated to provide best dimension trueness within their operating working distance. Outside the working distance, point clouds can still look good, but dimension trueness gradually degrades as distance increases. When dimension trueness becomes too poor, hand-eye calibration can produce marginal results and the touch test can fail.

Potential Solution

1. Re-acquire the hand-eye calibration dataset within the camera’s recommended working distance. If possible, try to be within the optimal working distance for best results. See working distance for your camera model.

2. Repeat the hand-eye calibration using the new dataset.

3. Verify the hand-eye calibration results (transform, residuals, projection).

4. Re-run the touch test (if the hand-eye calibration results are good).

Reproducing the issue (hand-eye calibration outside working distance)

All hand-eye calibrations covered in this section were performed using the Zivid 2+ MR60 camera, which has the following operating distance:

• Recommended working distance: 300 mm - 1100 mm

• Optimal working distance: 350 mm - 900 mm

Good Dataset (inside working distance)

When acquiring the hand-eye calibration dataset, the calibration object was positioned at approximately 900 mm from the camera in the z-axis. This is within the recommended working distance of 300-1100 mm and at the edge of the optimal working distance of 350-900 mm for the Zivid 2+ MR60 camera.

Hand-Eye Transformation Matrix

0.999	-0.004	0.043	-56.943
0.003	1.000	0.027	-90.317
-0.043	-0.027	0.999	128.422
0	0	0	1

Residuals

N (Poses)	Rot. avg (°)	Rot. max (°)	Trans. avg (mm)	Trans. max (mm)
14	0.056	0.115	0.270	0.383

Complete Residuals

Residuals

Pose	Rotation (°)	Translation (mm)
1	0.070	0.184
2	0.051	0.295
3	0.076	0.321
4	0.049	0.383
5	0.055	0.128
6	0.030	0.328
7	0.052	0.211
8	0.020	0.337
9	0.018	0.256
10	0.088	0.345
11	0.022	0.368
12	0.111	0.093
13	0.027	0.264
14	0.115	0.344

Verification by Projection technically succeeds, but the resulting error is still too large to be acceptable. One should be very critical of projection verification, as the projection error may seem acceptable even if the hand-eye calibration is incorrect.

The Hand-Eye calibration was then verified with a touch test.

The touch test was successful, which confirms that the hand-eye calibration is correct and accurate.

Bad Dataset (outside working distance)

When acquiring the hand-eye calibration dataset, the calibration object was positioned at approximately 1300 mm from the camera in the z-axis. This exceeds the recommended working distance of 300-1100 mm for the Zivid 2+ MR60 camera.

The calibration resulted in the following results:

Hand-Eye Transformation Matrix

0.999	-0.006	0.043	-59.008
0.005	1.000	0.026	-88.747
-0.044	-0.025	0.999	129.722
0	0	0	1

Residuals

N (Poses)	Rot. avg (°)	Rot. max (°)	Trans. avg (mm)	Trans. max (mm)
14	0.099	0.259	0.344	0.823

Complete Residuals

Residuals

Pose	Rotation (°)	Translation (mm)
1	0.080	0.229
2	0.109	0.157
3	0.043	0.142
4	0.095	0.148
5	0.122	0.553
6	0.118	0.383
7	0.028	0.610
8	0.069	0.423
9	0.125	0.277
10	0.146	0.823
11	0.260	0.229
12	0.046	0.083
13	0.057	0.628
14	0.091	0.130

Verification by Projection succeeded.

The touch test result was unsatisfactory as the tip of the touch test tool missed the center of the ArUco marker (x-axis and y-axis errors) and pushed on the ArUco marker (z-axis error).

Comparisons

The results between hand-eye calibration matrices (translation values) and hand-eye calibration residuals are compared in the tables below.

Translation Values in Hand-Eye Transformation Matrix

Dataset	X (mm)	Y (mm)	Z (mm)
Good (inside working distance)	-56.943	-90.317	128.422
Bad (outside working distance)	-59.008	-88.747	129.722
Δ (Good - Bad)	2.065	-1.570	-1.300

Residuals

Dataset	N	Rot. avg (°)	Rot. max (°)	Trans. avg (mm)	Trans. max (mm)
Good (inside working distance)	14	0.056	0.115	0.270	0.383
Bad (outside working distance)	14	0.099	0.259	0.344	0.823