A Camera Self-Calibration Method Based on Plane Lattice and Orthogonality

The calibration using orthogonal line is one of the basic approaches of camera calibration, but it requires the orthogonal line be accurately detected, which makes results of error increases. This paper propose a novel camera self-calibration technique using plane lattices and virtual orthogonal line. The rigorous analytical relations among the feature point coordinates of the plane lattice, the corresponding image point coordinate, intrinsic parameters, relative pose are induced according to homography matrix of the central projection. Let a slope of non-parallel and non-orthogonal virtual line in the lattice plane, and the slope of its orthonormal line can be calculated. In at least three photographs taken, vanishing points can be solved in two groups of orthogonal directions by using the homography matrix, so the camera intrinsic parameters are linearly figured out. This method has both simple principle and convenient pattern manufacture, and does not involve image matching, besides having no requirement concerning camera motion. Simulation experiments and real data show that this algorithm is feasible, and provides a higher accuracy and robustness.


Introduction
In computer vision, it has an important significance to the investigation of camera calibration, which is the premise and foundation for obtaining three-dimensional (3D) information, and an important part of the binocular visual field.Accurate Calibration of the camera intrinsic parameters can not only directly improve measurement precision but establishes nice base for further realizing stereo image matching and 3D reconstruction.Meanwhile, real-time of calibration can meet the needs of industry machine vision such as spatial query, navigation etc.
It is necessary for a traditional calibration method to design a geometric object (calibration pattern) with high precision, and then match accurate data of the calibration pattern and its imaging to solve the camera intrinsic parameters.In 1992, Hartley [1] and Faugeras [2] first proposed the idea of camera self-calibration.So called camera self-calibration, it refers to calibrating the camera intrinsic parameters only by the image sequences acquired from camera motion, without understanding geometric data of an 3D-object in scene.Camera self-calibration  [3][4][5][6][7][8][9][10][11].However, the calibration methods which not use geometric data in scene are nonlinear, complicated and poor robustness.Zhang [12] presented a simple and flexible self-calibration method, which utilize a pinpoint lattice template instead of a traditional calibration object.According to the mutual movement between the template and a camera, photographs of the template are taken from more than three different orientations, so the camera intrinsic parameters can be linearly solved by computing the homography matrix between the template and its image.On the basis of Zhang's method [12], Meng et al. [13] put forward a kind of self-calibration method which used a circle and several lines passing through the center of the circle as a planar pattern to determine the camera intrinsic parameters according to the image of circular points.The self-calibration method based on circular points is first put forward.From then on, a number of calibration methods are proposed on the basis of Zhang [12] and Meng et al. [13].Wu FC et al. [14] proposed a linear method to determine the camera intrinsic parameters by rectangle.With the deficiency of accurate location for lattice pattern in Zhang's method [12], Li XJ et al. [15] proposed a camera self-calibration method based on planar similar figure.Wang GH et al. [16] proposed a self-calibration method based on Kruppa equation of checkerboard.In addition, in recent years, some self-calibration methods based on circular points or vanishing point have emerged [17][18][19][20].Meanwhile, many calibration methods of panoramic camera have appeared by using checkerboard pattern [21][22].
Considering that line belongs to one of the basic geometric elements, which is common and easy to detect, so a camera can be calibrated by using virtual orthogonal lines after homography matrix.Taking into account the straight line is one of the basic geometric, common and easy to detect.So in this work, we use virtual orthogonal lines to calibrate after obtain the homography.In contrast with other calibration methods, this approach only needs to establish the homography matrix between world coordinate system (WCS) and image coordinate system.Based on the estimation of the homography matrix, vanishing points can be gotten in two mutually perpendicular directions, so the camera intrinsic parameters can be solved from three images.

Camera Model
In this study, we use the pinhole camera model which is the simplified model of geometry relation in optical imaging system (see Figure 1).
, respectively, where t is the homogeneous term, and usually 1 t  .When   , it corresponds to the element on a plane at infinity.The Euclid space which is supplemented with the infinite far element is called an extended Euclid space [8].
. Based on the principle of perspective projection transformation, their relationship is where  is an arbitrary scale factor, P is called the projection matrix which can be further broken down into with the camera extrinsic parameters   , R t .Here, R is a 3  3 rotation matrix and t is a 3  1 translation vector, which determine the position of WCS with respect to camera coordinate system (CCS), and K is called the camera intrinsic matrix, which is given by , with the coordinates 0 0 ( , ) u v of the principal point, the scale factors u f , v f in image u , v axes, and the parameter s describing the skewness of the two image axes.

Estimating the Homography
Without loss of generality, we assume a lattice plane lies on plane XOY ( 0 Z  ) of WCS.Any point ( , ,0,1) X Y  on the lattice plane is transformed from the WCS into the CCS.
Then, from (1), it is projected by: where H is the homography matrix between the lattice plane and its image, i r is the th i column of the rotation matrix R .

Let
where is the four-dimensional(4D) vector of nonhomogeneous coordinates including a space point and its corresponding image point, called the measuring spacious point.
In fact, the third component of 3D vector M HM  is a linear combination of the first two components, so The homography matrix H can be regarded as the intersection set of two quadric surfaces in 4D space, so it is given by: can be gotten according to more pairs of measurement points.Using a measurement matrix from the set, the homography matrix H can be obtained by solving the following minimization problem:

Solving Camera Intrinsic Parameters
Proposition 1: Assume there is a pair of orthogonal lines on the lattice plane, their points at infinity are denoted as 1 2 , V V   , and their image point are denoted as 1 2 , v v , so have , where . Proof: According camera imaging principle, it can be represented as Rearranging the above equation, we have Two lines through the infinity points 1 2 , V V   are mutually, so the equation is given by Let , therefore, the above equation may be written as Image coordinates of arbitrary point can be gotten on a pattern by homography matrix.The point at infinity corresponding to arbitrary line on a plane lies on the plane.If known any point coordinates at infinity, its image coordinates can be obtain, namely the vanishing point coordinates.
Proposition 2: If known arbitrary two either non-parallel or non-orthogonal virtual linear slopes, the five camera intrinsic parameters can be determined with at least three images.
Proof: If let a slope of virtual line on the planar pattern be k , the homogeneous coordinates of one point at infinity can be expressed as   1, , 0 k in the line direction, and the homogeneous coordinates of another point at infinity can be expressed   1, -1 , 0 k in its orthogonal direction.When given slops of arbitrary two neither parallel nor orthogonal lines, the two lines and the points at infinity of their orthogonal lines can be gotten.The vanishing points corresponding to the two lines can be obtained by the perspective projection transformation.That is , we can obtain becomes 0 Af  .A linear solution of f is gotten by using the least square method to linearly solve, and  can be obtained.The five camera parameters are determined by Cholesky decomposition of  .
We use the established propositions to derive the following algorithm.
Step 1: Make a lattice plane as a template, and take at least three pictures at different azimuths.
Step 2: Use Harris corner detection to get the coordinates of each point on the lattice pattern, and estimate the homography matrix H between the template and its image.
Step 3: Suppose the slopes 1 2 , k k of two neither parallel nor orthogonal virtual lines on lattice plane, and then calculate coordinates of the vanishing point in two orthogonal directions according to (10).
Step 4: Establish the constraint equation according to (9) to obtain  through solving (11).
Step 5: Determine the five camera intrinsic parameters by Cholesky decomposition of  .

Experiments 5.1. Simulation results
In simulation experiments, the planar chessboard as a lattice template, the camera intrinsic parameters were set at In addition, suppose slopes of two either non-parallel or non-orthogonal virtual lines in lattices plane are 1 0.25 k   and 2 0.125 k  , respectively.The chessboard and its images are as follows (see Figure 2).
The experimental results show that the algorithm provided in this paper is real and feasible.In order to further test the robustness of the algorithm, we add Gaussian noise with 0 mean and  standard deviation to the projected image points, and vary the noise level from 0.1 pixels to 2.0 pixels.For each noise level, we perform 100 independent trials and calculate their mean values, and the results shown are the average.As we can see from Fig 3, errors increase linearly with the noise level.Furthermore, we compare with Zhang's calibration approach [12], where the algorithm provided in this paper is expressed as v-point, Zhang's calibration approach is expressed as zhang (see Figure 3).

Real data
The camera to be calibrated is a CCD camera.The image resolution is 320×240.The planar pattern contains 9×7=63 corner points.It was printed with a high-quality printer.Three images of the pattern under different orientations were taken, as shown in Figure 4.
We applied our and Zhang's calibration algorithm to the three images.The results are shown in Table 1 (the skew factor 0 s  in Zhang's method).

Conclusion
In this paper, we presented a novel approach of camera calibration.First, we estimated the homography between the planar pattern and its image.Then, supposed the slopes of two either non-parallel or non-orthogonal virtual lines in lattice plane, used the homography matrix to compute the vanishing points in the directions of two groups of orthogonal lines, and solved the camera intrinsic parameters by vanishing points.the results of simulation and real data showed that this algorithm was feasible and available, and had a certain precision and robustness.

Figure 2 .
Figure 2. (a) the planar chessboard in space.(b), (c), (d) three images under different orientations Title of manuscript is short and clear, implies research results (First Author) 773

Figure 3 .
Figure 3.The experimental results of proposed method's compare with Zhang's method.(a), (b), (c), (d) the error comparison of the camera intrinsic parameters 0 0 , , , u vf f u v of this paper's method and Zhang's method.(e) Since let the skew factor 0 s  in Zhang's method, the error of the camera intrinsic parameter s is shown by this paper's method.

Table 1 .
Calibrating camera intrinsic parameters with two different methods