Unsatisfactory Hand-Eye Calibration Results (Manual Robot Pose Typos)

Problem

The hand-eye calibration results are unsatisfactory (bad or marginal), looking like one of the following combinations:

Hand-Eye Transform

Hand-Eye Residuals

Projection Verification

Touch Test

Bad

Bad

Bad

Unsafe

Reasonable

Bad

Bad

Bad

Reasonable

Marginal

Bad

Bad

Reasonable

Good

Poor

Marginal
Example of bad hand-eye calibration results

The Hand-Eye Transformation Matrix does not look correct because several of its values differ from the physical measurements by extremely large amounts that are not reasonable for this setup.

Hand-Eye Transformation Matrix

0.609

0.177

0.773

-467.320

-0.109

0.984

-0.140

17.581

-0.785

0.001

0.619

794.894

0

0

0

1

The value 794.894 should be approximately 130 mm, and the value 17.581 should be roughly 90 mm. At this stage we ignore the sign (+/-), because it depends on the coordinate-system definitions; we focus only on the distances.

../../_images/eye_in_hand_approximate_hand_eye_matrix.png

The calibration residuals are also significantly larger than expected, a couple of orders of magnitude higher.

Residuals

N (Poses)

Rot. avg (°)

Rot. max (°)

Trans. avg (mm)

Trans. max (mm)

11

2.855

10.677

102.330

405.720

For smaller robots, we expect the residuals to in the sub-millimeter range, while for larger robots, the residuals can be in the millimeter range. As for the rotational component of the residuals, we expect them to be in the sub-degree range.

Complete Residuals
Residuals

Pose

Rotation (°)

Translation (mm)

1

1.685

405.720

2

1.816

90.325

3

0.215

74.578

4

1.901

143.061

5

2.296

21.398

6

10.678

94.157

7

5.541

102.133

8

0.235

40.173

9

3.979

48.811

10

1.894

76.962

11

1.177

28.309

Verification by Projection fails, which suggest that the hand-eye calibration is not correct, and that the touch test will fail badly.

../../_images/projection_verification_bad.png
Example of marginal hand-eye calibration results

The Hand-Eye Transformation Matrix appears reasonable and is consistent with what one would expect from rough manual measurements (see table and image below).

Hand-Eye Transformation Matrix

0.999

-0.003

0.040

-54.871

0.002

0.999

0.029

-90.920

-0.040

-0.029

0.999

128.709

0

0

0

1
../../_images/eye_in_hand_approximate_hand_eye_matrix.png

The calibration residuals are within an acceptable range, indicating that the optimization has converged successfully.

Residuals

N (Poses)

Rot. avg (°)

Rot. max (°)

Trans. avg (mm)

Trans. max (mm)

11

0.027

0.059

0.160

0.388

For smaller robots, we expect the residuals to in the sub-millimeter range, while for larger robots, the residuals can be in the millimeter range. As for the rotational component of the residuals, we expect them to be in the sub-degree range.

Complete Residuals
Residuals

Pose

Rotation (°)

Translation (mm)

1

0.025

0.388

2

0.016

0.270

3

0.021

0.098

4

0.028

0.197

5

0.014

0.057

6

0.056

0.154

7

0.018

0.086

8

0.019

0.230

9

0.031

0.074

10

0.059

0.167

11

0.015

0.043

Verification by Projection technically succeeds, but the resulting error is still too large to be acceptable. One should be very critical of projection verification, as the projection error may seem acceptable even if the hand-eye calibration is incorrect.

../../_images/projection_verification_poor.png

The touch test result is unsatisfactory (bad or marginal).

The tip of the touch test tool pushes on the ArUco marker too much.

The tip of the touch test tool pushes on the ArUco marker a bit.

Potential Cause

Human error (typo) when copying robot poses from the robot controller to the PC.

Typos in robot poses cause the pose-image pairs to become inconsistent, which breaks the geometric assumptions required for hand-eye calibration. Even a small incorrect digit can distort one pose enough to pull the entire optimization toward a wrong solution, resulting in poor residuals, unrealistic transformations, and overall unsatisfactory calibration results.

Potential Solution

  1. If you are manually copying poses from the robot controller to the PC, double-check the values to ensure no typos were made.

  2. Ensure the correct rotation convention is used when converting robot poses from the robot controller into the transformation matrices for Zivid hand-eye calibration.

  3. Repeat the hand-eye calibration using the same dataset.

  4. Verify the hand-eye calibration results (transform, residuals, projection).

  5. Run the touch test (if the hand-eye calibration results are good).

../../_images/touching-test-result-aruco.jpg
Reproducing the issue (bad hand-eye calibration)

Good Dataset

A hand-eye calibration was successfully performed using the dataset shown in the images below.

../../_images/hand-eye-calibration-small-movements-dataset-only-11.png

The calibration resulted in the following results:

Hand-Eye Transformation Matrix

0.999

-0.002

0.041

-55.598

0.001

0.999

0.028

-90.654

-0.041

-0.028

0.999

129.044

0

0

0

1
Residuals

N (Poses)

Rot. avg (°)

Rot. max (°)

Trans. avg (mm)

Trans. max (mm)

11

0.025

0.054

0.109

0.192
Complete Residuals
Residuals

Pose

Rotation (°)

Translation (mm)

1

0.025

0.388

2

0.016

0.270

3

0.021

0.098

4

0.028

0.197

5

0.014

0.057

6

0.056

0.154

7

0.018

0.086

8

0.019

0.230

9

0.031

0.074

10

0.059

0.167

11

0.015

0.043

The Hand-Eye calibration was verified with a touch test.

../../_images/touch_test_correct.png

The touch test was successful, which confirms that the hand-eye calibration is correct and accurate.

Bad Dataset (Large Translation Error)

To simulate a human typo during a manual robot pose entry, a large translation error was introduced for one pose in the good dataset. The X value of the translation vector was changed from -742.58 to -142.58 mm (first integer digit: 7 → 1).

The calibration resulted in the following results:

Hand-Eye Transformation Matrix

0.609

0.177

0.773

-467.320

-0.109

0.984

-0.140

17.581

-0.785

0.001

0.619

794.894

0

0

0

1
Residuals

N (Poses)

Rot. avg (°)

Rot. max (°)

Trans. avg (mm)

Trans. max (mm)

11

2.855

10.677

102.330

405.720
Complete Residuals
Residuals

Pose

Rotation (°)

Translation (mm)

1

1.685

405.720

2

1.816

90.325

3

0.215

74.578

4

1.901

143.061

5

2.296

21.398

6

10.678

94.157

7

5.541

102.133

8

0.235

40.173

9

3.979

48.811

10

1.894

76.962

11

1.177

28.309

Verification by Projection failed, which suggests that the hand-eye calibration is not correct, and that the touch test would fail badly. Therefore, the touch test was skipped for safety reasons.

../../_images/projection_verification_bad.png

Comparisons

The results between hand-eye calibration matrices (translation values) and hand-eye calibration residuals are compared in the tables below.

Translation Values in Hand-Eye Transformation Matrix

Dataset

X (mm)

Y (mm)

Z (mm)

Good

-55.598

-90.654

129.044

Bad (large transl. error)

-467.320

17.581

794.894

Δ (Good - Bad)

411.722

-108.235

-665.850
Residuals

Dataset

N

Rot. avg (°)

Rot. max (°)

Trans. avg (mm)

Trans. max (mm)

Good

11

0.025

0.054

0.109

0.192

Bad (large transl. error)

11

2.855

10.677

102.330

405.720
Reproducing the issue (marginal hand-eye calibration)

Good Dataset

A hand-eye calibration was successfully performed using the dataset shown in the images below.

../../_images/hand-eye-calibration-small-movements-dataset-only-11.png

The calibration resulted in the following results:

Hand-Eye Transformation Matrix

0.999

-0.002

0.041

-55.598

0.001

0.999

0.028

-90.654

-0.041

-0.028

0.999

129.044

0

0

0

1
Residuals

N (Poses)

Rot. avg (°)

Rot. max (°)

Trans. avg (mm)

Trans. max (mm)

11

0.025

0.054

0.109

0.192
Complete Residuals
Residuals

Pose

Rotation (°)

Translation (mm)

1

0.025

0.388

2

0.016

0.270

3

0.021

0.098

4

0.028

0.197

5

0.014

0.057

6

0.056

0.154

7

0.018

0.086

8

0.019

0.230

9

0.031

0.074

10

0.059

0.167

11

0.015

0.043

The Hand-Eye calibration was verified with a touch test.

../../_images/touch_test_correct.png

The touch test was successful, which confirms that the hand-eye calibration is correct and accurate.

Bad Datasets

To simulate a human typo during a manual robot pose entry, the good dataset was modified by introducing an error in three different ways, resulting in three new datasets:

A small angular error (first digit after the “.”) was made for one pose for one of the Roll-Pitch-Yaw angles. The Yaw value was changed from 6.82° to 6.22° (first digit after “.” 8 → 2).

A large angular error (first digit) was made for one pose for one of the Roll-Pitch-Yaw angles. The Yaw value was changed from 6.93° to 3.93° (first integer digit 6 → 3).

A small translational error (first digit after the “.”) was made for one pose for one of translation vector values. The X value was changed from -48.81 to -48.21 (first digit after “.” 8 → 2).

The calibration resulted in the following hand-eye matrices and residuals:

Hand-Eye Transformation Matrix

0.999

-0.002

0.044

-57.032

0.001

1.000

0.029

-90.899

-0.044

-0.029

0.999

129.580

0

0

0

1
Hand-Eye Transformation Matrix

0.999

0.005

0.047

-58.558

-0.006

1.000

0.029

-87.455

-0.047

-0.029

0.998

131.117

0

0

0

1
Hand-Eye Transformation Matrix

0.999

-0.003

0.040

-54.871

0.002

1.000

0.029

-90.920

-0.040

-0.029

0.999

128.709

0

0

0

1

Residuals

Residuals

Dataset

N

Rot. avg (°)

Rot. max (°)

Trans. avg (mm)

Trans. max (mm)

Small Angle Error

11

0.104

0.536

0.256

0.877
Complete Residuals
Residuals

Pose

Rotation (°)

Translation (mm)

1

0.536334

0.877

2

0.055345

0.431

3

0.057495

0.189

4

0.064677

0.301

5

0.054738

0.115

6

0.059730

0.029

7

0.070040

0.238

8

0.062262

0.119

9

0.060767

0.069

10

0.060585

0.288

11

0.062108

0.160
Residuals

Dataset

N

Rot. avg (°)

Rot. max (°)

Trans. avg (mm)

Trans. max (mm)

Large Angle Error

11

0.505

2.742

0.966

4.393
Complete Residuals
Residuals

Pose

Rotation (°)

Translation (mm)

1

2.742018

4.393

2

0.280878

1.050

3

0.269892

0.750

4

0.288396

1.053

5

0.291592

0.435

6

0.247856

0.691

7

0.315325

0.886

8

0.278548

0.410

9

0.297165

0.382

10

0.262904

0.314

11

0.285421

0.262
Residuals

Dataset

N

Rot. avg (°)

Rot. max (°)

Trans. avg (mm)

Trans. max (mm)

Small Translation Error

11

0.027

0.059

0.160

0.388
Complete Residuals
Residuals

Pose

Rotation (°)

Translation (mm)

1

0.025433

0.388

2

0.016149

0.270

3

0.021316

0.098

4

0.027773

0.197

5

0.013938

0.057

6

0.056120

0.154

7

0.018387

0.086

8

0.019042

0.230

9

0.031110

0.074

10

0.059334

0.167

11

0.014671

0.043

Comparisons

The results between hand-eye calibration matrices (translation values) and hand-eye calibration residuals are compared in the tables below.

Translation Values in Hand-Eye Transformation Matrix

Dataset

X (mm)

Y (mm)

Z (mm)

Good

-55.598

-90.654

129.044

Bad (Small Angle Error)

-57.032

-90.899

129.580

Δ (Good - Bad)

1.434

0.244

-0.536
Translation Values in Hand-Eye Transformation Matrix

Dataset

X (mm)

Y (mm)

Z (mm)

Good

-55.598

-90.654

129.044

Bad (Large Angle Error)

-58.558

-87.455

131.117

Δ (Good - Bad)

2.960

-3.199

-2.073
Translation Values in Hand-Eye Transformation Matrix

Dataset

X (mm)

Y (mm)

Z (mm)

Good

-55.598

-90.654

129.044

Bad (Small Transl. Error)

-54.871

-90.920

128.709

Δ (Good - Bad)

-0.727

0.265

0.335

Residuals

Residuals

Dataset

N

Rot. avg (°)

Rot. max (°)

Trans. avg (mm)

Trans. max (mm)

Good

11

0.025

0.054

0.109

0.192

Bad (Small Angle Error)

11

0.104

0.536

0.256

0.877
Residuals

Dataset

N

Rot. avg (°)

Rot. max (°)

Trans. avg (mm)

Trans. max (mm)

Good

11

0.025

0.054

0.109

0.192

Bad (Large Angle Error)

11

0.505

2.742

0.966

4.393
Residuals

Dataset

N

Rot. avg (°)

Rot. max (°)

Trans. avg (mm)

Trans. max (mm)

Good

11

0.025

0.054

0.109

0.192

Bad (Small Transl. Error)

11

0.027

0.059

0.160

0.388

Touch Tests

The tip of the touch test tool does not reach the ArUco marker.

The tip of the touch test tool pushes on the ArUco marker too much.

The tip of the touch test tool lands at the center of the ArUco marker.