REMOTE SENSING IMAGE REGISTRATION ALGORITHM BASED ON PARTIAL COVARIANCE MATRIX Peng-bo Wang1, Yue-shan Liu1, Jin-cheng Li1, Xue-yun Wei2 1School of Electronics and Information Engineering, Beihang University, Beijing 100191, China 2School of Electronic Information, Jiangsu University of Science and Technology, Zhenjiang, 212003, China
[email protected] Keywords: Image Processing, Image Registration, Remote Sensing, Covariance Matrix. Abstract Image registration is a prerequisite for accomplishing high level tasks such as image fusion, surface reconstruction, change detection, and object recognition. To achieve multi- sensor remote sensing image registration, a remote sensing image registration algorithm based on partial covariance matrix is presented in this paper. The partial contour covariance matrix and adaptive optimal window selection technology are introduced into the registration processing to reduce the impact of rotation factor and scale factor, which can enhance the robustness of the registration algorithm. Experiment results demonstrate the efficiency and the accuracy of the proposed algorithm for multi-sensor remote sensing image registration. 1 Introduction Remote sensing system has been proven to be a powerful tool for monitoring the earth surface and atmosphere at global, regional, and local scale. It provides important coverage, mapping and classification of land cover features such as vegetation, soil, water and forest. The degree of accuracy achieved in classification depends on the quality of the images and knowledge possessed by the researcher. In many applications, information provided by single sensor is incomplete, inconsistent, and imprecise. Fusion of different information can make an improved understanding of observation area by decreasing the uncertainties of the single source. Image registration is the premise of image fusion. The goal of image registration is to establish the correspondence between two images and determine the geometric transformation that aligns one image with the other [1]. Currently, image registration algorithms can be divided into three categories: the algorithms based on the phase correlation technology [2], the algorithms based on global grayscale statistical information [3], and the algorithms based on image features [1, 4]. Because the first two categories mainly rely on the gradation characteristic of the image, which restricts their applications in the process of remote sensing image registration. Therefore, the algorithm based on image features is commonly used in the remote sensing image registration. The conventional algorithms use statistical parameters such as correlation coefficients or interactive information to establish the mapping relationship between the input image features. However, with the increase of rotation factor and scale factor, the similarity between images drops rapidly. So the conventional algorithms cannot get good effects. In this paper, a remote sensing image registration algorithm based on partial covariance matrix is presented. The partial contour covariance matrix is introduced into the registration processing to extract the rotation factor and scale factor. Adaptive optimal window selection technology is used to reduce the impact of rotation factor and scale factor, which can enhance the robustness of the registration processing. The remainder of this paper is organized as follows. Section II introduces the principle of registration algorithm. A registration experiment is given in Section III, which shows the validity of the algorithm. Finally, the conclusions and discussions are presented in Section IV. 2 Registration algorithm based on partial contour covariance matrix The image registration algorithm based on image features is generally divided into three steps: feature extraction, feature association and image conversion. The key part of algorithm is to operate association process on the image features and to extract the mapping relationship between those input images. However, the similarity of the input images to be registered will gradually weaken along with the increasing of rotation factor and scale factor between the input images. That will influence the association process between local features and even make it impossible to complete the registration process. In this paper, a remote-sensing image registration algorithm based on partial covariance matrix is proposed. First, the Harris corner detector is used to detect corner features, and partial contour feature is extracted after local segmentation; then, the partial contour covariance matrix is used to extract the rotation factor and scale factor, and adaptive optimal window selection technology is used to reduce the impact of rotation factor and scale factor; at last, the registered image- pair is required by image resampling. Figure 1 shows the flow chart of the remote sense image registration algorithm based on partial contour covariance matrix. Input images Start End Selection and extraction of image features Extraction of rotation factor and scale factor Determine the optimal size of the analysis window Image conversion and resampling Extraction of the mapping relationship Figure 1. The flow chart of the remote-sense image registration algorithm based on partial covariance matrix. 2.1 Selection and extraction of image features It is all right for optical image to achieve image registration processing utilizing edge contour. But when it comes to radar images, the speckle noise is so strong that it is hard to extract the complete edge contour. Under this circumstance, it cannot get good results if the edge features are used directly. To solve this problem effectively, the combination of the corner feature and edge contour feature is selected as local features in this paper. Taking into account the corner feature usually located in the grayscale mutation, it is easy to extract partial contour feature. In the paper, the Harris corner detector is used to detect corner features [4]. This algorithm determines whether there is the corner feature by analysis of the gradient changes in local analysis window. According to the definition and the extraction method, the corner feature is usually located in the grayscale mutation. Therefore, there is obvious change of the greyscale value in local area of corner. Hence, the partial image segmentation can be realized by traditional image segmentation algorithm, and partial contour feature can be easily extracted. In this paper, the ostu threshold is used to segment local image. 2.2 Extraction of rotation factor and scale factor With the rotation factor and scale factor between the input images increasing gradually, the local similarity between them decreases sharply, and features association becomes more difficult. The partial contour covariance matrix between the input images can extract the rotation factor and scale factor effectively [6, 7, 8, 9], which can decrease the influence of rotation factor and scale factor on feature association and improve the robustness of registration. Assuming that iI and jT are corner of the reference image and the image to be registered respectively, then there is affine transformation f between 0I ---the local neighbourhood of iI , and 0T ----the local neighbourhood of 0T , making the equation established. 0 0 0T f I AI (1) Where A is an 2 2 affine transformation matrix, 11 12 21 22 a a A a a Before extracting the rotation factor and scale factor, the ostu threshold is used to segment local image. The value of points, the gray value of which is above the threshold, is set to 1 while those under the threshold are set to 0. Then the feature images which can be recorded as I and T come into being. Even considering the large grayscale difference in multi- mode remote sensing, the basic structural features of the image are consistent. Hence, there is still an affine transformation between feature images. Then the partial contour covariance matrix is used to extract the rotation factor and scale factor. Calculating covariance matrix of the reference image and the image to be registered, then the equation can be established: ' T IA A (2) Where I is the covariance matrixes of reference image, ' 1 1 N ij ijI j r r N T is the covariance matrixes of the image to be registered, ' 1 1 N tj tjT j r r N ; N is the number of the nonzero pixels; j x r y is the position vector for each point, superscript ' denotes the transpose. Because both covariance matrixes I and T are normal matrixes, there are orthogonal matrixes 1U 2U and diagonal matrixes 1 2 meeting the following equation: ' ' 1 1 1 1 ' ' 2 2 2 1 1 1 N ij ijI j N tj tjT j r r U U N r r U U N (3) Where 1 11 1 1 cos sin sin cos U 2 22 2 2 cos sin sin cos U 11 1 12 0 0 11 12 21 2 22 0 0 . Submitting the above equation into equation (2): ' ' ' 2 2 2 1 1 1U U AU U A (4) If 0.52 2 2B U 0.5 1 1 1B U , then 1 2 1A B B (5) Then, we can automatic extract the rotation factor and scale factor referring to the local structural distribution of the reference image and the image to be registered. 2 1 2 1/i i i (6) 2.3 Determine the optimal size of the analysis window With the increasing of rotation factor and scale factor, the performance of the traditional similarity criterion such as the correlation coefficient, the cross-correlation coefficient and the interactive information etc. decrease sharply. This is mainly caused by the following two aspects: 1. The rotation factor and scale factor; 2. The difference of the image content in local analysis window influenced by the scale factor. Therefore, it is necessary to extract the rotation factor and scale factor and adjust the size of window adaptively. The basic principle of determining the optimal window size can be listed as follows: when the size of the window is optimal, the local analysis windows have the same structural distribution. The ratio of the covariance matrix eigenvalues reflects the scale factor between the input images. The ratio of the analysis windows size also reflects the scale factor. Theoretically, when it is the optimal window size, the two ratios are consistent. Therefore, the optimal window size can be required by contrasting the ratio of the local analysis window size with scale factor extracted from the partial contour covariance matrix. If the ratio of the local analysis window size is the same as scale factor extracted from the partial contour covariance matrix, this means that the image texture features within the local analysis window in the reference image and the image to be registered are the same, and the optimal window size is required. On the contrary, if the ratio of the local analysis window size and scale factor extracted from the partial contour covariance matrix is not the same, this means that the image texture features within the local analysis window are different. Adjusting the size of the local analysis window until the ratio is the same as scale factor extracted from the partial contour covariance matrix, or the image size reaches the threshold. 2.4 image conversion and resampling After extracting the rotation factor, the scale factor and the size of the local analysis window, the partial image correction can be realized by using the above-mentioned prior knowledge. The homologue points are extracted by using correlation coefficient and affine invariant. Then, affine invariance is used to estimate the affine parameters between the input images. 11 12 11 21 22 12 21 22 1 0 0 0 0 1 0 0 wi ri ri ri wi ri ri ri x y x x x ya a ax y y x ya a ay a a (7) At last, image resampling according to established affine parameters is used to obtain the registered remote sensing image-pair. 3 The result of the registration experiment In order to verify the effectiveness of registration algorithm presented in the paper, an experiment is operated on real remote sensing images. The input images are shown in Figure 2. The pair of SAR images is obtained from airborne platform with 5 meters resolution. (a) (b) Figure 2. Airborne SAR images pair. The result of homologue pointâs extraction is shown in Figure 3, and the result of registration is shown in Figure 4. Table 1 shows the result of the registration experiment, in which X1 represents the X coordinate of the homologue points in the reference image, Y1 represents the Y coordinate of the homologue points in the reference image, X2 represents the X coordinate of the homologue points in the image to be registered, Y2 represents the Y coordinate of the homologue points in the image to be registered, Xâ represents the X coordinate of the homologue points in the resampling image, Yâ represents the Y coordinate of the homologue points in the resampling image. Besides, the position root mean square error represents the distance between the position of the homologue point resampling and those in the reference image. From the result of experiment, we can find that the proposed algorithm can achieve accurate registration of remote sensing images. (a) (b) Figure 3. The result of homologue points extracted. Figure 4. The result of registration. sequence number X1 Y1 X2 Y2 X â Yâ RMSE 0 202 135 162 206 202.91 134.82 0.92 1 234 70 193 127 233.87 70.40 0.43 2 204 150 163 224 204.07 149.33 0.67 3 118 246 78 341 118.37 245.51 0.48 4 206 198 164 284 205.58 197.76 0.48 5 244 71 203 261 244.07 71.01 0.07 6 255 178 213 261 255.35 178.21 0.40 7 275 59 233 114 274.54 59.11 0.47 8 250 32 209 80 249.79 32.12 0.24 9 94 301 53 410 93.45 301.72 0.91 10 202 135 162 206 202.91 134.82 0.92 Table 1. The result of registration processing. 4 Conclusions Remote sensing image registration is an important image processing technology in remote sense field. Aimed at the challenge of remote sensing image registration, a remote sensing image registration algorithm based on partial contour covariance matrix is presented in this paper. The partial contour covariance matrix is used to extract the rotation factor and scale factor. Adaptive optimal window size selected is used to reduce the impact of scale factor, which can enhance the robustness of the registration processing. The registered image-pair is required by image resampling. The experiment of SAR images shows the validity of the algorithm. References [1] Hui L, Manjunath B S, Mitra S K. A contour-based approach to multisensor image registration [J]. IEEE Transactions on Image Processing, 1994, 4(3):320-334. [2] Qin-sheng Chen, Defrise M, Deconinck F. Symmetric phase-only matched filtering of Fourier-Mellin Transforms for image registration and recognition[J]. Pattern Analysis and Machine Intelligence, 1994, 16(12):1156-1168. [3] Maes F, Collignon A, Vandermeulen D, Marchal G., Suetens P. Multimodality image registration by maximization of mutual information[J], IEEE Transactions on medical imaging, 1997, 16(2):187-198. [4] Harris C G, Stephens M J. A combined corner and edge detector[A]. Proceedings Fourth Alvey Vision Conference, 1988:147-151. [5] Ton J, Jain A K. Registering Landsat images by point matching[J]. IEEE Transactions on Geoscience and Remote Sensing, 1989:642-651. [6] Nagao K, Grimson W. Affine matching of planar sets[J]. Computer Vision and Image Understanding, 1998, 70(1):1-22. [7] Xin Kang, Chonghao Han, Yi Yang. Structure-driven SAR Image Registration[J]. Journal of System Simulation, 2006, 18(5):1307-1310. [8] Tao Sun, Xiukun Wang, Gang Shao, Lin Feng. Affine Registration of 2D Point Pattern Image[J]. Journal of Computer Aided Design & Computer Graphics, 2005, 17(7):1497-1503. [9] Zhiguo Tan. Point Pattern Matching and its Application[D]. National university of Defence Technology, 2008.