Unsatisfactory Hand-Eye Calibration Results (Bad Dimension Trueness)

Problem

The hand-eye calibration results are marginal, with unsatisfactory touch test results (typically z-axis error), looking like one of the following combinations:

Hand-Eye Transform

Hand-Eye Residuals

Projection Verification

Touch Test

Reasonable

Marginal

Poor

Marginal

Reasonable

Good

Good

Marginal
Example of marginal hand-eye calibration results

The Hand-Eye Transformation Matrix appears reasonable and is consistent with what one would expect from rough manual measurements (see table and image below).

Hand-Eye Transformation Matrix

0.999

-0.004

0.043

-56.879

0.003

1.000

0.028

-90.051

-0.043

-0.027

0.999

130.259

0

0

0

1
../../_images/eye_in_hand_approximate_hand_eye_matrix.png

The calibration residuals are within an acceptable range, indicating that the optimization has converged successfully.

Residuals

N (Poses)

Rot. avg (°)

Rot. max (°)

Trans. avg (mm)

Trans. max (mm)

14

0.050

0.102

0.328

0.718

For smaller robots, we expect the residuals to be in the sub-millimeter range, while for larger robots, the residuals can be in the millimeter range. As for the rotational component of the residuals, we expect them to be in the sub-degree range.

Complete Residuals
Residuals

Pose

Rotation (°)

Translation (mm)

1

0.018

0.070

2

0.102

0.187

3

0.036

0.118

4

0.028

0.275

5

0.048

0.426

6

0.056

0.374

7

0.058

0.537

8

0.043

0.458

9

0.057

0.099

10

0.064

0.569

11

0.034

0.203

12

0.053

0.334

13

0.025

0.220

14

0.082

0.718

Verification by Projection succeeds.

../../_images/projection_verification_good.png

The touch test result is unsatisfactory. For example, the tip of the touch test tool pushes on the ArUco marker (z-axis error).

Potential Cause

The camera’s dimension trueness is not sufficiently good.

Zivid cameras are made to withstand industrial working environments and continue to provide quality point clouds. However, there are situations that can affect the camera’s dimension trueness and lead to unsatisfactory hand-eye calibration results. In these situations, running the Infield Correction can bring the dimension trueness to an acceptable level and improve hand-eye calibration accuracy.

When should Infield Correction be performed?

  • The camera experienced a mild collision.

  • The camera was mishandled (for example, dropped).

  • The camera experienced substantial environmental changes.

  • The application has high or very high accuracy requirements.

Potential Solution

  1. Perform Infield Correction.

  2. Repeat the hand-eye calibration using the same dataset.

  3. Verify the hand-eye calibration results (transform, residuals, projection).

  4. Re-run the touch test (if the hand-eye calibration results are good).

../../_images/touching-test-result-aruco.jpg
Reproducing the issue (marginal dimension trueness)

Bad Dataset (before infield correction)

A camera with unsatisfactory dimension trueness was used for hand-eye calibration. The calibration resulted in the following results:

Hand-Eye Transformation Matrix

0.999

-0.004

0.043

-56.879

0.003

1.000

0.028

-90.051

-0.043

-0.027

0.999

130.259

0

0

0

1
Residuals

N (Poses)

Rot. avg (°)

Rot. max (°)

Trans. avg (mm)

Trans. max (mm)

14

0.050

0.102

0.328

0.718
Complete Residuals
Residuals

Pose

Rotation (°)

Translation (mm)

1

0.018

0.070

2

0.102

0.187

3

0.036

0.118

4

0.028

0.275

5

0.048

0.426

6

0.056

0.374

7

0.058

0.537

8

0.043

0.458

9

0.057

0.099

10

0.064

0.569

11

0.034

0.203

12

0.053

0.334

13

0.025

0.220

14

0.082

0.718

Verification by Projection succeeded.

../../_images/projection_verification_poor.png

The touch test result was unsatisfactory as the tip of the touch test tool pushed on the ArUco marker (z-axis error).

Good Dataset (after infield correction)

Infield correction was performed to improve the camera’s dimension trueness and the hand-eye calibration was repeated. The calibration resulted in the following results:

Hand-Eye Transformation Matrix

0.999

-0.004

0.043

-56.870

0.003

1.000

0.027

-90.409

-0.043

-0.027

0.999

128.381

0

0

0

1
Residuals

N (Poses)

Rot. avg (°)

Rot. max (°)

Trans. avg (mm)

Trans. max (mm)

14

0.056

0.115

0.276

0.382
Complete Residuals
Residuals

Pose

Rotation (°)

Translation (mm)

1

0.070

0.194

2

0.050

0.280

3

0.075

0.341

4

0.050

0.382

5

0.055

0.127

6

0.030

0.310

7

0.052

0.204

8

0.018

0.333

9

0.019

0.251

10

0.089

0.355

11

0.023

0.377

12

0.111

0.098

13

0.027

0.265

14

0.115

0.346

Verification by Projection was successful.

../../_images/projection_verification_good.png

The Hand-Eye calibration was then verified with a touch test.

The touch test was successful, which confirms that the hand-eye calibration is correct and accurate.

Comparisons

The results between hand-eye calibration matrices (translation values) and hand-eye calibration residuals are compared in the tables below.

Translation Values in Hand-Eye Transformation Matrix

Dataset

X (mm)

Y (mm)

Z (mm)

Bad (before infield correction)

-56.879

-90.051

130.259

Good (after infield correction)

-56.870

-90.409

128.381

Δ (Good - Bad)

0.009

-0.358

-1.878
Residuals

Dataset

N

Rot. avg (°)

Rot. max (°)

Trans. avg (mm)

Trans. max (mm)

Bad (before infield correction)

14

0.050

0.102

0.328

0.718

Good (after infield correction)

14

0.056

0.115

0.276

0.382