Touch Test Unsafe to perform due to Bad Hand-Eye Calibration (Wrong Hand-Eye Configuration)

Problem

The hand-eye calibration results are bad, and it is unsafe to perform the touch test.

Hand-Eye Transform	Hand-Eye Residuals	Projection Verification	Touch Test
Bad	Bad	Bad	Unsafe

Example of bad hand-eye calibration results

The Hand-Eye Transformation Matrix does not look correct because several of its values differ more than expected from the physical measurements. The differences are not extremely large, but they are beyond what would be considered reasonable for this setup.

Hand-Eye Transformation Matrix

0.626	0.780	0.008	116.163
-0.672	0.545	-0.502	1183.242
-0.396	0.309	0.865	-286.632
0.000	0.000	0.000	1.000

The value -286.632 should be approximately 130 mm, and the value 1183.24 should be roughly 90 mm. At this stage we ignore the sign (+/-), because it depends on the coordinate-system definitions; we focus only on the distances.

The calibration residuals are also noticeably larger than expected, roughly an order of magnitude higher.

Residuals

N (Poses)	Rot. avg (°)	Rot. max (°)	Trans. avg (mm)	Trans. max (mm)
14	7.847	22.552	26.112	74.373

For smaller robots, we expect the residuals to in the sub-millimeter range, while for larger robots, the residuals can be in the millimeter range. As for the rotational component of the residuals, we expect them to be in the sub-degree range.

Complete Residuals

Residuals

Pose	Rotation (°)	Translation (mm)
1	0.690	6.318
2	9.634	18.759
3	12.329	18.773
4	6.403	17.827
5	6.900	44.173
6	0.662	3.098
7	7.149	17.514
8	8.731	28.463
9	22.552	55.483
10	12.946	24.909
11	0.666	9.978
12	9.530	17.392
13	7.362	74.373
14	4.309	28.505

Verification by Projection technically succeeds, but the resulting error is still too large to be acceptable. One should be very critical of projection verification, as the projection error may seem acceptable even if the hand-eye calibration is incorrect.

Such results strongly suggest that the hand-eye calibration is not correct, and that the touch test will fail if attempted.

Potential Cause

Hand-Eye calibration was performed with a wrong hand-eye configuration (Eye-in-Hand or Eye-to-Hand), meaning that the solver was given incorrect information about the physical setup.

As a result, you may see high residuals, poor projection accuracy, and a hand-eye transformation matrix whose distances differ from real geometry. In severe cases, the resulting transform can make touch test validation unsafe due to risk of collision.

Potential Solution

1. Ensure you run hand-eye calibration with the correct hand-eye configuration that matches your physical setup:

   • Eye-to-Hand: Camera is fixed in the environment and the robot moves the calibration object (i.e., Object-in-Hand or Stationary/Static Camera).

   • Eye-in-Hand: Camera is mounted on the robot and the calibration board is fixed in the environment (i.e., Object-to-Hand or On-Arm/Moving Camera).

2. Repeat the hand-eye calibration using the same dataset.

3. Verify the hand-eye calibration results (transform, residuals, projection).

4. Run the touch test (if the hand-eye calibration results are good).

Reproducing the issue (wrong hand-eye configuration)

Good Dataset

A hand-eye calibration was successfully performed using the dataset shown in the images below.

The calibration resulted in the following results.

Hand-Eye Transformation Matrix

0.999	-0.004	0.043	-55.936
0.003	0.999	0.013	-91.919
-0.043	-0.013	0.998	128.086
0	0	0	1

Residuals

N (Poses)	Rot. avg (°)	Rot. max (°)	Trans. avg (mm)	Trans. max (mm)
14	0.022	0.034	0.180	0.338

Complete Residuals

Residuals

Pose	Rotation (°)	Translation (mm)
1	0.017	0.163
2	0.008	0.135
3	0.028	0.172
4	0.009	0.053
5	0.031	0.236
6	0.006	0.060
7	0.009	0.154
8	0.015	0.102
9	0.031	0.147
10	0.009	0.145
11	0.030	0.249
12	0.023	0.152
13	0.034	0.255
14	0.031	0.169

The Hand-Eye calibration was verified with a touch test.

The touch test was successful, which confirms that the hand-eye calibration is correct and accurate.

Bad Dataset (calibration board moved on purpose)

To simulate the problem, the hand-eye calibration was performed with the wrong hand-eye configuration (Eye-to-Hand instead of Eye-in-Hand). The calibration resulted in the following results.

Hand-Eye Transformation Matrix

0.626	0.780	0.008	116.163
-0.672	0.545	-0.502	1183.242
-0.396	0.309	0.865	-286.632
0.000	0.000	0.000	1.000

Residuals

N (Poses)	Rot. avg (°)	Rot. max (°)	Trans. avg (mm)	Trans. max (mm)
14	7.847	22.552	26.112	74.373

Complete Residuals

Residuals

Pose	Rotation (°)	Translation (mm)
1	0.690	6.318
2	9.634	18.759
3	12.329	18.773
4	6.403	17.827
5	6.900	44.173
6	0.662	3.098
7	7.149	17.514
8	8.731	28.463
9	22.552	55.483
10	12.946	24.909
11	0.666	9.978
12	9.530	17.392
13	7.362	74.373
14	4.309	28.505

Verification by Projection technically succeeded, but the resulting error was still too large to be acceptable. One should be very critical of projection verification, as the projection error may seem acceptable even if the hand-eye calibration is incorrect.

Such results strongly suggest that the hand-eye calibration is not correct, and that the touch test will fail if attempted, so it was skipped for safety reasons.

Comparisons

The results between hand-eye transformation matrices (translation values) and hand-eye calibration residuals are compared in the tables below.

Translation Values in Hand-Eye Transformation Matrix

Dataset	X (mm)	Y (mm)	Z (mm)
Good	-55.936	-91.919	128.086
Bad (Wrong Hand-Eye Configuration)	116.163	1183.242	-286.632
Δ (Good - Bad)	-172.099	-1275.161	414.718

Residuals

Dataset	N	Rot. avg (°)	Rot. max (°)	Trans. avg (mm)	Trans. max (mm)
Good	14	0.022	0.034	0.180	0.338
Bad (Wrong Hand-Eye Configuration)	14	7.847	22.552	26.112	74.373