Method for building panoramas based on feature matching

Yu Tong,Zixing Cai

2004-01-01

Abstract:In this paper a new method is presented to build panorama based on feature matching. It is permitted in this method that the image has some perspective distortion, small overlap, brightness difference and small concentric errors. An efficient image registration algorithm on feature matching scheme is also proposed in this paper.

What problem does this paper attempt to address?

A Feature-Based Approach to Panorama Generation

郭立,郑迈,王丙勤

2008-01-01

Abstract:Panorama has various application prospects such as teleconference,virtual reality,etc.However,in the case of large misregistration or parallax,many existing algorithms perform poorly or even fail to work through.To tackle this challenging situation,a robust approach was proposed to generating panorama.This method first estimated the perspective motion model to align the different views by repeatable features.And then it ordered the images into different layers based on salient features before blending.The experimental results show that the proposed method can effectively synthesize different views with misregistration or parallax.
Multi-Viewpoint Panorama Construction with Wide-Baseline Images

Guofeng Zhang,Yi He,Weifeng Chen,Jiaya Jia,Hujun Bao

DOI: https://doi.org/10.1109/tip.2016.2535225

IF: 10.6

2016-01-01

IEEE Transactions on Image Processing

Abstract:We present a novel image stitching approach, which can produce visually plausible panoramic images with input taken from different viewpoints. Unlike previous methods, our approach allows wide baselines between images and non-planar scene structures. Instead of 3D reconstruction, we design a mesh-based framework to optimize alignment and regularity in 2D. By solving a global objective function consisting of alignment and a set of prior constraints, we construct panoramic images, which are locally as perspective as possible and yet nearly orthogonal in the global view. We improve composition and achieve good performance on misaligned areas. Experimental results on challenging data demonstrate the effectiveness of the proposed method.
Omni-directional Image Unwarping Based on Taylor Model

LEI Jie,DU Xin,ZHU Yunfang,LIU Jilin

2010-01-01

Journal of Image and Graphics

Abstract:Omni-directional camera has attracted more attentions in computer vision community due to its quick wide-angle imaging.But the great distortion makes the post-processing complex,which limits the algorithm applications of classical perspective camera.This paper deduces an unwarping algorithm from arbitrary ring omni-directional image to cylinder panorama based on the Taylor model of omni-directional imaging system.And we also present an unwarping method to produce a novel kind of panorama which is composed of N perspective planer images.Both two reprojection unwarpings do not depend on a particular model or accurate parameters of omni-directional system.In the unwarped panoramas,each pixel is assigned with a light ray in the 3D space,which makes the applications of stereo rectification and 3D reconstruction simple.Finally,the experimental results with the simulations and real data illustrate the validity of the proposed method.
A Simple and Efficient Approach for Creating Cylindrical Panorama

WH An,BF Zhou

DOI: https://doi.org/10.1117/12.561118

2004-01-01

Abstract:The methods of image registration and fusion are very important basis of cylindrical panorama creating. There are many relevant algorithms been developed, but many of them usually have limitations on their qualities and efficiency. In this paper, an automatic approach for creating cylindrical panorama is introduced. Compared with many commonly used algorithms(7,10), this algorithm is simple and effective. In our method, we adopted an FFT-based phase-correlation algorithm to register images, and we also employed a multi-resolution pyramid algorithm to perform image fusion. By preprocessing the input images using such methods as histogram equalization, we further increased the accuracy and robustness of image registration. Our experiments show that the method is simple, efficient and effective.
Six-to-one: Cubemap-guided Feature Calibration for Panorama Object Detection.

Jingbo Miao,Yanwei Liu,Kan Wang,Jinxia Liu,Antonios Argyriou,Yanni Han,Zhen Xu

DOI: https://doi.org/10.1109/ictai56018.2022.00200

2022-01-01

Abstract:Object detection methods for perspective images have proven increasingly efficient, but the techniques for equirectangular projection (ERP) panoramas from inherently spherical imaging cannot still achieve satisfactory performance. Due to the various degrees of distortion at different pixel locations, current algorithms cannot adapt to the changes in shape and contour caused by stretching, which results in performance degradation when migrating them from perspective images to spherical ones. In this paper, we improve the network for panorama object detection and introduce the cube-domain information with discontinuity but low distortion to correct the panorama features. Unlike previous works, we consider the impact of semantic discontinuity from all tangent planes instead of overlaying features when needed. Considering the six facets as unified, i.e., six-to-one for extraction, the proposed Facet-Link module enhances the long-range sensing capability at the facet level in the frequency domain. Moreover, the position alignment packs different facets, i.e., six-to-one for calibration, to preserve more global signals during the correction stage, which establishes semantic pathways for feature interactions between panorama and cubemap in the two dimensions, facet-facet and cube-pano, respectively. Extensive experiments on synthetic and real-world datasets verify the effectiveness and robustness of our proposed method.
A Fast Epipolar Line Matching Method Based on 3D Spherical Panorama

Shuai Liu,Jun Chen,Min Sun,Lingli Zhao,Xiang Wei

DOI: https://doi.org/10.1109/ACCESS.2019.2961184

IF: 3.9

2020-01-01

IEEE Access

Abstract:A fast epipolar line matching method based on 3D spherical panorama is proposed in this paper in order to overcome the low automation degree of feature matching in applying the 3D panoramic model measurement. First, the search scope of panoramic image matching was restricted from 2D to 1D strip-shaped buffer zone through the epipolar constraints. The search scope was further refined using color invariant correlation coefficient. Finally, a matching calculation model was constructed based on features to complete the panoramic feature matching. The experiment verified the effectiveness and feasibility of the proposed method in reducing the mismatching of spherical panoramic features. The proposed method also effectively solved the automatic search for corresponding points and resolved geometric information. The findings help lay a solid foundation for the resampling and measurement of the panoramic measurement model and the generation of depth maps.
Construction and Global Alignment of Panoramic Mosaics

WANG Hai-ying,QIN Kai-huai

DOI: https://doi.org/10.16182/j.cnki.joss.2010.08.030

2010-01-01

Abstract:A new method of construction and refinement of panoramic mosaics from sequences of images was proposed.First,the initial registration was taken with the phase correlation based on Fourier-Mellin transforms that enhanced the robustness of registration.Second,the registration precision was improved by calculating the parameters of the projection transform with an iterative optimization that includes additional illumination model.Lastly,to satisfy the head-tail consistence of panorama,a new global alignment method was proposed that reduced the accumulated error to create 360° full-view panoramic mosaics.Compared with other algorithms,this method can improve the success rate of the automatic registration and mosaic,especially when the hand-held camera had an irregular movement or adjacent images had difference of brightness.The experimental results showed that it had higher accuracy and fairly good practical value.
Efficient panorama mosaicing based on enhanced-FAST and graph cuts

Xun Wang,Jie Sun,Haoyu Peng

DOI: https://doi.org/10.1007/978-3-642-25792-6-115

2012-01-01

Abstract:This paper presents an efficient and accurate method for creating full view panoramas. A new feature points detection algorithm called Enhanced- FAST is proposed to accurately align images and a graph cuts algorithm is used to merge two adjacent images seamlessly. Based on the FAST algorithm, the Enhanced-FAST algorithm smoothes and extends the sampling area making the feature points detection more insensitive to noise. Our graph cuts algorithm uses image Laplacian to compute the edge weights which can find an optimized seam even under different lighting. Experiments and comparisons show that our method is efficient and robust to image noise and lighting changing. © 2012 Springer-Verlag Berlin Heidelberg.
A Feature Selecting and Matching Method Based on Unwrapped Panoramic Images

高明,曹洋,方帅

DOI: https://doi.org/10.3969/j.issn.1003-5060.2009.04.001

2009-01-01

Abstract:In order to improve the stability of feature selecting and matching,a method is proposed to abstract features on the unwrapped panorama's multi-scale space.At first,the panoramic image is unwrapped and rectified based on the virtual cylinder.Then multi-scale space is built for the unwrapped panoramic image.Finally the candidate features are selected by Harris dictator and by Laplace function that decides local maximum features in multi-scale space.Those features have the characteristics of scale invariance,rotating invariance as well as illuminating invariance,and will be stable under the conditions of limited changes of visual angles.Experiment proves that this method is more stable than abstracting features on wrapped panorama's multi-scale space and unwrapped panorama's single-scale space.
Spherical Panoramic Mosaics With The Image Division And Warping Methods

Weihua An,Bingfeng Zhou

2006-01-01

Abstract:Some limitations exist in previous approaches for spherical panoramic mosaics. They usually employ optimizing methods for image registration, and need particular imaging equipments. In this paper we present a new approach to automatically create seamless spherical panoramas from image sequences. These images are acquired from an approximately fixed point by using off-the-shelf cameras and tripods. This approach employs the FFT-based phase-correlation algorithm to approximately compute the transformation relationships between images. To avoid the accumulation of registration errors, we mosaic the images acquired along each vertical line into a vertical sub-image, and then generate a spherical panorama from all the vertical sub-images by employing the image division and warping methods. As demonstrated by the experiment results, the new approach can generate high-quality mosaics while requiring simpler manual operations and less time consumption.
Object panorama construction using large-parallax images

Kyunghwa Jung,Ho-Gun Ha,In-Ho Jeon,Jaesung Hong

DOI: https://doi.org/10.1007/s11042-022-13134-1

IF: 2.577

2022-04-27

Multimedia Tools and Applications

Abstract:Conventional panorama techniques create a wide-angle image by stitching images taken from the same viewpoint. In contrast, the method proposed in this work produces an unwrapped surface image of a three-dimensional spherical object. Traditionally, in order to construct a panoramic image including multiple faces of an object, consecutive video frames must be captured around the object so that images with small parallax can be stitched together to avoid ghost artifacts. In this study, however, we use only two input images taken from different viewpoints to construct the panoramic surface image of a spherical object. This kind of constraint can occur when the cameras have limitation on changing their poses. The acquired two input images have a larger parallax than the video frames. Therefore, in order to align the overlapping regions of the large-parallax images, an image-morphing method with a curved interpolation line is proposed. The interpolation curve is designed for a spherical target object and it reduces dent distortion. As image morphing is highly vulnerable to feature mismatches, the corresponding features in the parallax images are paired by active feature matching using a structured light. During image composition, the seam boundary that minimizes ghost effects at the transition between images is determined based on image similarity. The experimental results for large-parallax images with an angle difference of 60° demonstrate the effectiveness of the proposed method.

computer science, information systems, theory & methods,engineering, electrical & electronic, software engineering
An Image Sorting Algorithm Based on Sift Feature Matching

Zhou Ying,Zhang Jun

DOI: https://doi.org/10.1109/iccacs.2015.7361365

2015-01-01

Abstract:The panoramic image mosaic technology means images with partial overlapping area are composited to a 360-degree circular image which describes the information of a certain scene. A hotspot of this technology is how to determine the image order accurately and fast when the number of images is large. In this paper, we present a novel image sorting algorithm based on SIFT(Scale-Invariant Features Transform). In the proposed method, matching feature is firstly characterized by calculating the number of matching points between images to find the nearest image of a certain one. After calculating the matching points one by one, an accurate picture order could be finally obtained. The results show the effectiveness of the proposed algorithm.
DeepPanoContext: Panoramic 3D Scene Understanding with Holistic Scene Context Graph and Relation-based Optimization

Cheng Zhang,Zhaopeng Cui,Cai Chen,Shuaicheng Liu,Bing Zeng,Hujun Bao,Yinda Zhang

DOI: https://doi.org/10.1109/iccv48922.2021.01240

2021-01-01

Abstract:Panorama images have a much larger field-of-view thus naturally encode enriched scene context information compared to standard perspective images, which however is not well exploited in the previous scene understanding methods. In this paper, we propose a novel method for panoramic 3D scene understanding which recovers the 3D room layout and the shape, pose, position, and semantic category for each object from a single full-view panorama image. In order to fully utilize the rich context information, we design a novel graph neural network based context model to predict the relationship among objects and room layout, and a differentiable relationship-based optimization module to optimize object arrangement with well-designed objective functions on-the-fly. Realizing the existing data are either with incomplete ground truth or overly-simplified scene, we present a new synthetic dataset with good diversity in room layout and furniture placement, and realistic image quality for total panoramic 3D scene understanding. Experiments demonstrate that our method outperforms existing methods on panoramic scene understanding in terms of both geometry accuracy and object arrangement. Code is available at https://chengzhag.github.io/publication/dpc.
Distance Measurement In Panorama

Zhongding Jiang,Nan Jiang,Yijie Wang,Binyu Zang

DOI: https://doi.org/10.1109/ICIP.2007.4379604

2007-01-01

Abstract:Computing world distances of scene features from the captured images is a common task in image analysis and scene understanding. Previous projective geometry based methods focus on measuring distance from one single image. Hence, the scope of measurable scene is limited by the field-of-view (FOV) of one single camera. In this paper, we propose one method of measuring distances of line segments in real world scene using panorama representation. With a full view panorama, the scope of measurable scene is increased and can fully cover the sphere of 360 x 360 FOV. With panorama representation, distance of long-range features, which can not be fully captured by a single image, can be measured from the panoramic image. A prototype system called PanoMeasure is developed for enabling user to interactively measure the distances of line segments. Experiments with simulated data and real measurement results verify that the method offers high accuracy.
Panoramic image mosaic method based on image segmentation and Improved SIFT algorithm

Lei Zhuang,Jiyan Yu,Yang Song

DOI: https://doi.org/10.1088/1742-6596/2113/1/012066

2021-11-01

Journal of Physics: Conference Series

Abstract:Abstract Aiming at the problem of large amount of calculation in extracting image feature points in panoramic image mosaic by SIFT algorithm, a panoramic image mosaic algorithm based on image segmentation and Improved SIFT is proposed in this paper. The algorithm fully considers the characteristics of panoramic image stitching. Firstly, the stitched image is divided into blocks, and the maximum overlapping block of image pairs is extracted by using mutual information. The SIFT key points are extracted by SIFT algorithm, and the dog is filtered before the spatial extreme value detection of SIFT algorithm to eliminate the feature points with small intensity value; When establishing the feature descriptor, the 128 dimension of the original algorithm is reduced to 64 dimensions to reduce the amount of calculation. In the feature point registration process, the feature descriptor is reduced to 32 dimensions, the feature point pairs are roughly extracted by the optimal node first BBF algorithm, and the feature point pairs are registered and screened by RANSAC; Finally, the image transformation matrix is obtained to realize panoramic image mosaic. The experimental results show that the proposed algorithm not only ensures the panoramic mosaic effect, but also extracts the feature points in 11% of the time of the traditional SIFT algorithm, and the feature point registration speed is 27.17% of the traditional SIFT algorithm.
A Global Optimization Approach for Construction of Panoramic Mosaics

Haiying Wang,Kaihuai Qin

DOI: https://doi.org/10.1117/12.833213

2009-01-01

Abstract:A global optimization approach for construction of a 360° full-view panoramic mosaic is presented with global consistency in this paper. There are three primary steps in this approach. The first is a local registration using SIFT. In the second step, an efficient algorithm for finding a global consistent registration of all images is developed. The local registration matrices with the boundary condition are used for computing the initial parameters of the iterative optimization for the global consistent registration, to adjust/reduce the alignment error accumulated by the local registering and stitching of adjacent two images. Lastly, the overlapping areas of the images are stitched by the graph cut algorithm and blended with a multi-resolution method. It is proven by the experiment results in this paper that the presented approach can effectively construct panoramic mosaics with global end-to-end consistency.
Measurable Panorama Construction Based on Binocular Spherical Projective Geometry

Shuai Liu,Jun Chen,Min Sun,Lingli Zhao

2015-01-01

Journal of Computer-Aided Design & Computer Graphics

Abstract:Stereo vision reconstruction is an unresolved and hot research topic in the fields of computer vision and photogrammetry. The conventional techniques are mainly based on a limited field of view (FOV), which re-sults in the problems of blind areas and insufficient overlapping degrees. Though panorama is able to show a 360-degree field of view, it performs poorly in precision. In this paper, we present an innovative measurement vi-sion system on the strength of binocular spherical panoramas. We proposed a geometric model according to the principles of spherical panorama projection and multi-view geometry. In this model, we adopted the homography transformation and the essential transformation to obtain the parameters of the rotation matrix and the translation vector, which are used to establish binocular spherical projection geometry. The experiment we carried out vali-dated the feasibility of our model, which could be applied to the 3D measurement process.
An improved adaptive triangular mesh-based image warping method

Wei Tang,Fangxiu Jia,Xiaoming Wang

DOI: https://doi.org/10.3389/fnbot.2022.1042429

IF: 3.493

2023-01-23

Frontiers in Neurorobotics

Abstract:It is of vital importance to stitch the two images into a panorama in many computer vision applications of motion detection and tracking and virtual reality, panoramic photography, and virtual tours. To preserve more local details and with few artifacts in panoramas, this article presents an improved mesh-based joint optimization image stitching model. Since the uniform vertices are usually used in mesh-based warps, we consider the matched feature points and uniform points as grid vertices to strengthen constraints on deformed vertices. Simultaneously, we define an improved energy function and add a color similarity term to perform the alignment. In addition to good alignment and minimal local distortion, a regularization parameter strategy of combining our method with an as-projective-as-possible (APAP) warp is introduced. Then, controlling the proportion of each part by calculating the distance between the vertex and the nearest matched feature point to the vertex. This ensures a more natural stitching effect in non-overlapping areas. A comprehensive evaluation shows that the proposed method achieves more accurate image stitching, with significantly reduced ghosting effects in the overlapping regions and more natural results in the other areas. The comparative experiments demonstrate that the proposed method outperforms the state-of-the-art image stitching warps and achieves higher precision panorama stitching and less distortion in the overlapping. The proposed algorithm illustrates great application potential in image stitching, which can achieve higher precision panoramic image stitching.

robotics,computer science, artificial intelligence,neurosciences
Automatic Registration of Panoramic Images and Point Clouds in Urban Large Scenes Based on Line Features

Panke Zhang,Hao Ma,Liuzhao Wang,Ruofei Zhong,Mengbing Xu,Siyun Chen

DOI: https://doi.org/10.3390/rs16234450

IF: 5

2024-11-29

Remote Sensing

Abstract:As the combination of panoramic images and laser point clouds becomes more and more widely used as a technique, the accurate determination of external parameters has become essential. However, due to the relative position change of the sensor and the time synchronization error, the automatic and accurate matching of the panoramic image and the point cloud is very challenging. In order to solve this problem, this paper proposes an automatic and accurate registration method for panoramic images and point clouds of urban large scenes based on line features. Firstly, the multi-modal point cloud line feature extraction algorithm is used to extract the edge of the point cloud. Based on the point cloud intensity orthoimage (an orthogonal image based on the point cloud's intensity values), the edge of the road markings is extracted, and the geometric feature edge is extracted by the 3D voxel method. Using the established virtual projection correspondence for the panoramic image, the panoramic image is projected onto the virtual plane for edge extraction. Secondly, the accurate matching relationship is constructed by using the feature constraint of the direction vector, and the edge features from both sensors are refined and aligned to realize the accurate calculation of the registration parameters. The experimental results show that the proposed method shows excellent registration results in challenging urban scenes. The average registration error is better than 3 pixels, and the root mean square error (RMSE) is less than 1.4 pixels. Compared with the mainstream methods, it has advantages and can promote the further research and application of panoramic images and laser point clouds.

environmental sciences,imaging science & photographic technology,remote sensing,geosciences, multidisciplinary
Semantic Aware Stitching for Panorama

Yuan Jia,Zhongyao Li,Lei Zhang,Bin Song,Rui Song

DOI: https://doi.org/10.3390/s24113512

IF: 3.9

2024-05-31

Sensors

Abstract:The most critical aspect of panorama generation is maintaining local semantic consistency. Objects may be projected from different depths in the captured image. When warping the image to a unified canvas, pixels at the semantic boundaries of the different views are significantly misaligned. We propose two lightweight strategies to address this challenge efficiently. First, the original image is segmented as superpixels rather than regular grids to preserve the structure of each cell. We propose effective cost functions to generate the warp matrix for each superpixel. The warp matrix varies progressively for smooth projection, which contributes to a more faithful reconstruction of object structures. Second, to deal with artifacts introduced by stitching, we use a seam line method tailored to superpixels. The algorithm takes into account the feature similarity of neighborhood superpixels, including color difference, structure and entropy. We also consider the semantic information to avoid semantic misalignment. The optimal solution constrained by the cost functions is obtained under a graph model. The resulting stitched images exhibit improved naturalness. Extensive testing on common panorama stitching datasets is performed on the algorithm. Experimental results show that the proposed algorithm effectively mitigates artifacts, preserves the completeness of semantics and produces panoramic images with a subjective quality that is superior to that of alternative methods.

engineering, electrical & electronic,instruments & instrumentation,chemistry, analytical

Method for building panoramas based on feature matching

A Feature-Based Approach to Panorama Generation

Multi-Viewpoint Panorama Construction with Wide-Baseline Images

Omni-directional Image Unwarping Based on Taylor Model

A Simple and Efficient Approach for Creating Cylindrical Panorama

Six-to-one: Cubemap-guided Feature Calibration for Panorama Object Detection.

A Fast Epipolar Line Matching Method Based on 3D Spherical Panorama

Construction and Global Alignment of Panoramic Mosaics

Efficient panorama mosaicing based on enhanced-FAST and graph cuts

A Feature Selecting and Matching Method Based on Unwrapped Panoramic Images

Spherical Panoramic Mosaics With The Image Division And Warping Methods

Object panorama construction using large-parallax images

An Image Sorting Algorithm Based on Sift Feature Matching

DeepPanoContext: Panoramic 3D Scene Understanding with Holistic Scene Context Graph and Relation-based Optimization

Distance Measurement In Panorama

Panoramic image mosaic method based on image segmentation and Improved SIFT algorithm

A Global Optimization Approach for Construction of Panoramic Mosaics

Measurable Panorama Construction Based on Binocular Spherical Projective Geometry

An improved adaptive triangular mesh-based image warping method

Automatic Registration of Panoramic Images and Point Clouds in Urban Large Scenes Based on Line Features

Semantic Aware Stitching for Panorama