Wide Baseline Matching Using Support Vector Regression

In this paper, we newly solve wide baseline matching using support vector regression (SVR). High correct ratio initial matches are used to train SVR relationships, obtained by matching large-scale SIFT features and discarding some mismatches by our improved topological filtering scheme; and new matches are searched near the prediction given by trained SVR relationships. Both indoor and outdoor environments image pairs under wide baseline condition are tested, experiment results show that our algorithm automatically gain large numbers of accurate point correspondences.


Introduction
Finding correspondences in a set of images of a scene is a fundamental part of most computer vision tasks, such as camera self-calibration, registration, structure and motion recovery, object recognition and so on [1].
Most of current wide baseline matching approaches obey this general scheme: 1. detect interest points or distinguish regions and extract their local features; 2.initial features matching; 3.discarding mismatches and expanding correct matches iteratively or not;4.gettingfinal matches and geometry relations.The third step:Discarding mismatches and extending correct matches is the most challenging work in the current study about wide baseline matching approaches.RANdom SAmple Consensus(RANSAC) was applied to estimate global geometry relationship--the fundamental matrix or homography between image pairs [2], outliers are eliminated, and the inliers are used to refine estimation, the refine estimation derived from inliers can guide matching so as to extend matches.Some works developing new robust estimators were reported [7].However, the initial matches are possibly sparse and with high ratio false matches, too early to get the global geometry relationship estimation or even bring on a bad result.Ferrari et al. [3]refined each initial match by looking for the local affine transformation that optimized the similarity, and the matches with refined similarity above a similarity threshold were kept.New matches were searched guided by local affine transformations, and a sidedness topological constraint for triple matches was used to remove bad matches.Tuytelaars and Gool gave two local constraints [5], and the matches with sufficient other matches obeying the same local constraints were reserved.Steele and Egbert [9] expanded the matches guided by local similarity transformation estimated previously, and constrained the matches by epipolar geometry with RANSAC estimation using a high inlier error tolerance.Using global geometric relationships-the basic matrix [2] or local geometric relationslocal affine transformation to guide the match [2,9].However,the fundamental matrix just provides a bilinear constraint, if there is no adequate match, then it can not be a better estimate.Local geometric relations can provide prediction image, but it is very sensitive.As support vector regression (SVR) can construct good classification rules in high dimensional space, this paper proposes a Wide baseline matching algorithm based on SVR , and using an improved topology filter to weed out false match,by the experimental simulation, trying to solve a wide baseline image pairs matching problem.
We present a novel wide baseline matching algorithm using support vector regression (SVR) in Section 2 and Section 3. As a part of this algorithm, an improved topological filter has been invented for discarding mismatches, called Random Planes Topological Filter (RPTF) in Section 4. Experiment results of indoor and outdoor image pairs are shown in Section 5, and the paper is concluded in Section 6 finally.

Support Vector Regression (SVR)
The support vector machine (SVM) invented by Vapnik [12]and his research groups is a popular classification approach.It minimizes the upper bound of the generalization error.Furthermore, with the introduction of Vapnik's   insensitive loss function, the SVM is extended to solve nonlinear regression estimation problem, called support vector regression (SVR).Here we describe the standard SVR algorithm: . In this new space, we consider a linear function, which corresponds to a nonlinear one in the original space: where ,   is the inner product, and The final solution is:

Wide baseline matching using SVR
We change the problem of expanding accurate matches to a prediction problem: based on the available matches with noise, for unmatched points in one image, we want to get correspondence pixel coordinate predictions in the other image [10], and then search matches near the prediction.This prediction can be derived by SVR as following description: are pixel coordinates of the image pairs, and their correspondence relationship can be represented as: Suppose initial matches are:   Because   insensitive loss function is used, we should search for the new match in a neighborhood of the prediction in the next image.This algorithm is summarized as follows: Step 1: Detect interest points or distinguish regions and extract their local features in both image pair.Step 2: Get initial feature matches by nearest neighbor based matching strategy.Step 3: Execute (a)-(c): (a) Discard some mismatches, (b) Use current matches for SVR to get relationship (11), (c) Search new matches near the prediction given by relationship (11).Step 4: Estimate the fundamental matrix or homography between the image pair by the normalized eight-point method .

Discarding mismatches
The key to use SVR to expand correct matches is that we should obtain high correct ratio initial matches as training data.Ferrari et al. [3] invented a topological filter for removing mismatches.

Considering three points  
in one image and their correspondence points in the other image, the function X on the right side of the directed line from 2 X to 3 X , or value 1 if it's on the left side.The sidedness constraint (13) holds for all the coplanar points.

A triple match (such as
) including one, or more mismatches has higher chances to violate equation (13).Based on this assumption, Ferrari et al. [3] design a voting scheme called topological filter for removing mismatches: for each match  Actually, all of the detected interesting points or distinguishing regions can't be coplanar; otherwise the fundamental matrix estimation is a degenerate problem [12].The threshold topo t is actually determined by both camera motion and scene structure, so setting an appropriate value for it is a very hard problem.However, there is no need to use this topological filter on the whole match set, if dividing the match set into subsets consisting of coplanar matches at first, we can use this filter simply with 0  topo t on each subset [11].When the camera has translation between image pair, coplanar points and their correspondence points obey the homography relationship [12].Considering these, we iteratively estimated some homographies by four randomly sampled matches and take their inliers as coplanar match subset.It's a bit different from the RANSAC method introduced in [12]: at each iterative step, we only searched the homography that has adequate number (more than six for example) of inliers rather than the homography that has largest number of inliers as RANSAC, so the method is efficient.We called this improved topological filter "Random Planes Topological Filter" (RPTF).

Experiments
We tested our algorithm on images acquired from a variety of indoor and outdoor environments, such as: Lab-desk (Fig. 1), Corridor (Fig. 2), Gloriette (Fig. 3), House (Fig. 4), and Park (Fig. 5).The camera underwent significant translation or ration for each image pair, and illumination changed for some pairs.Repeating things like books or leaves made matching more difficult.For every image pair, we extracted SIFT features [4],There were about 5000 to 8000 from an image, showed in Table 1.However, we just used large-scale features extracted from the six coarsest layers to get initial matches for the accuracy and the efficiency.We set similarity threshold as 0.1 and the ratio threshold 0.7.After discarding some mismatches by RPTF, we trained the SVR to get the relationship (2).Radial basis functions (RBF) was selected as the kernel function for its local property, and parameters were . The obtained SVR relationship (2) was used to guild the searching for new matches,The search widow radius is3ε, and the similarity threshold is 0.8.After the expansion step, the fundament matrix was estimated automatically using RANSAC and inliers were reserved as the final matches.
To measure the accuracy of the matches,we handpicked some matches in each image pair to compute an accurate fundamental matrix F (manual F in Table 2), and computed the average match distances (  d in Table 2) from their respective epipolar lines against this manual F for both handpicked match sets and final match sets automatically acquired by our algorithm.We also computed the average match distances from their respective epipolar lines against the automatically estimated F (auto F in Table 2) for final matches.From Table 1 and Table 2, we can see that final matches were expanded about 7 to 10 times of initial matches, while of final matches against manual F were less than 2 times of handpicked matches.The time performance of the proposed approach was presented in Table 3

Conclusion
The paper demonstrated how to use SVR to accomplish wide baseline matching.As a part of this novel algorithm, we invented an improved topological filter for discarding mismatches.And Experiment results showed that our algorithm can automatically obtain large numbers of accurate matches for indoor or outdoor image pairs under wide baseline condition.
Essentially, when using SVR, the selection of kernel function is crucial.Finding or designing some kernel functions, which have better learning and generalization capability for the relationship, is a valuable work as further research.


ISSN: 1693-6930 TELKOMNIKA Vol.11, No. 3, September 2013: 597 -602 598 the kernel function.The goal is to find empirical risk function(2) in feature space where  denotes the regularization constant and   L   represents the   insensitiveloss function, and is defined as Lagrange multiplier method minimizing(2)yields the following dualoptimization problem; minimize

Table 1 .
Features extracted from image pairs and for initial matching

Table 2 .
. The algorithm was matlab 7.9 platform with Intel CPU 2.4GHz.From Table3, one can see that, the training time of SVR is a small part of the running time of the whole algorithm.Matching expansion result and evaluation

Table 3 .
The time performance of the proposed approach