Unsatisfactory Hand-Eye Calibration Results (Loose or Sagging On-Arm Mount)

Problem

The hand-eye calibration results are unsatisfactory (bad or marginal), looking like something between these combinations:

Hand-Eye Transform	Hand-Eye Residuals	Projection Verification	Touch Test
Reasonable	Bad	Bad	Bad
Reasonable	Marginal	Poor	Bad

Example of marginal hand-eye calibration results

The Hand-Eye Transformation Matrix appears reasonable and is consistent with what one would expect from rough manual measurements (see table and image below).

Hand-Eye Transformation Matrix

0.999	0.004	0.047	-54.326
-0.004	1.000	-0.008	-86.359
-0.047	0.007	0.999	125.236
0	0	0	1

The calibration residuals are also noticeably larger than expected, roughly an order of magnitude higher.

Residuals

N (Poses)	Rot. avg (°)	Rot. max (°)	Trans. avg (mm)	Trans. max (mm)
14	0.756	1.545	3.162	5.944

For smaller robots, we expect the residuals to in the sub-millimeter range, while for larger robots, the residuals can be in the millimeter range. As for the rotational component of the residuals, we expect them to be in the sub-degree range.

Complete Residuals

Residuals

Pose	Rotation (°)	Translation (mm)
1	0.449	2.486
2	0.935	2.737
3	0.854	4.049
4	0.753	3.035
5	0.589	4.105
6	0.535	2.682
7	0.593	2.973
8	1.004	1.442
9	0.618	3.502
10	0.614	3.648
11	1.545	3.464
12	0.283	1.775
13	0.773	5.944
14	1.044	2.430

Verification by Projection technically succeeds, but the resulting error is still too large to be acceptable. One should be very critical of projection verification, as the projection error may seem acceptable even if the hand-eye calibration is incorrect.

The touch test result is unsatisfactory. For example, the tip of the touch test tool does not reach the ArUco marker.

Potential Cause

A common cause of high residuals is flex or movement in the on-arm mounting structure that connects the robot flange to the camera. This can occur when:

• The bracket is 3D-printed, thin, or otherwise insufficiently rigid

• Screws or fasteners are not properly tightened

• The mount sags, bends, or shifts as the robot changes orientation

• The camera’s weight or additional tooling applies torque that the mount cannot withstand

Any mechanical movement during dataset capture results in the calibration board appearing in slightly different poses, which directly increases residuals.

Potential Solution

1. Ensure that the mounting interface between the robot flange and the camera is completely rigid:

   • Use a solid metal bracket or another high-stiffness material

   • Avoid 3D-printed, thin, or long cantilevered designs

   • Verify that all screws, adapters, and extension plates are fully tightened

   • Confirm that the structure does not deflect under the weight of the camera or end-of-arm-tooling

   • If necessary, shorten the lever arm or reinforce the mount

2. Repeat the hand-eye calibration using the same dataset.

3. Verify the hand-eye calibration results (transform, residuals, projection).

4. Run the touch test (if the hand-eye calibration results are good).

Reproducing the issue (loose on-arm mount)

Good Dataset

A hand-eye calibration was successfully performed using the dataset shown in the images below.

The calibration resulted in the following results:

Hand-Eye Transformation Matrix

0.999	-0.001	0.043	-56.355
-0.000	1.000	0.020	-89.248
-0.043	-0.020	0.999	129.899
0	0	0	1

Residuals

N (Poses)	Rot. avg (°)	Rot. max (°)	Trans. avg (mm)	Trans. max (mm)
14	0.135	0.180	0.578	1.341

Complete Residuals

Residuals

Pose	Rotation (°)	Translation (mm)
1	0.180	0.465
2	0.122	0.449
3	0.100	0.545
4	0.096	0.331
5	0.126	0.356
6	0.180	0.179
7	0.175	1.341
8	0.136	0.621
9	0.102	0.996
10	0.140	0.375
11	0.135	0.763
12	0.104	0.636
13	0.156	0.448
14	0.141	0.588

Verification by Projection was successful.

The Hand-Eye calibration was then verified with a touch test.

The touch test was successful, which confirms that the hand-eye calibration is correct and accurate.

Bad Dataset (loose on-arm mount)

The same dataset was used for hand-eye calibration. However, the screws of the camera on-arm mount were intentionally left loose to simulate the issue and demonstrate the effect on hand-eye calibration results.

The calibration resulted in the following results:

Hand-Eye Transformation Matrix

0.999	0.004	0.047	-54.326
-0.004	1.000	-0.008	-86.359
-0.047	0.007	0.999	125.236
0	0	0	1

Residuals

N (Poses)	Rot. avg (°)	Rot. max (°)	Trans. avg (mm)	Trans. max (mm)
14	0.756	1.545	3.162	5.944

Complete Residuals

Residuals

Pose	Rotation (°)	Translation (mm)
1	0.449	2.486
2	0.935	2.737
3	0.854	4.049
4	0.753	3.035
5	0.589	4.105
6	0.535	2.682
7	0.593	2.973
8	1.004	1.442
9	0.618	3.502
10	0.614	3.648
11	1.545	3.464
12	0.283	1.775
13	0.773	5.944
14	1.044	2.430

Verification by Projection technically succeeded, but the resulting error was still too large to be acceptable. One should be very critical of projection verification, as the projection error may seem acceptable even if the hand-eye calibration is incorrect.

The touch test result was unsatisfactory as the tip of the touch test tool did not reach the ArUco marker.

Comparisons

The results between hand-eye calibration matrices (translation values) and hand-eye calibration residuals are compared in the tables below.

Translation Values in Hand-Eye Transformation Matrix

Dataset	X (mm)	Y (mm)	Z (mm)
Good (Tightened)	-56.355	-89.248	129.899
Bad (Loose)	-54.326	-86.359	125.236
Δ (Good - Bad)	-2.029	-2.890	4.663

Residuals

Dataset	N	Rot. avg (°)	Rot. max (°)	Trans. avg (mm)	Trans. max (mm)
Good (Tightened)	14	0.135	0.180	0.578	1.341
Bad (Loose)	14	0.756	1.545	3.162	5.944