Technology
Given two images, that may or may not overlap, i2Align™
has two
inter-related jobs: to decide if the images do indeed overlap and, if
so, to compute an accurate mapping between the two
images. i2Align™ achieves this using a three-step
process: (1)
generating initial estimates of the inter-image transformation, (2)
refining the most promising initial estimates, and (3) deciding if any
of the resulting estimates align the same structures from the two
images. The earlier Dual-Bootstrap algorithm (GDB-ICP), IEEE
Trans. on Pattern Analysis and Machine Intelligence Nov 2007,
which we developed with colleagues and fellow students at Rensselaer
Polytechnic Institute, used matching of SIFT features to generate
initial estimates, a region growth and model selection algorithm to
refine the estimates, and a combination of stability, accuracy and
orientation-consistency measures to make the final decision. Each of
these components has been replaced with a proprietary new
technique in i2Align™, which is now more intelligent
and faster
than the Dual-Bootstrap algorithm.
The power of the algorithm allows it to produce inter-image alignments such as shown in the following gif animation.
Transformation Models
i2k Align™ includes similarity, affine, homography, planar (homography plus radial-lens distortion), cylindrical (also with radial-lens distortion) and quadratic transformation models.
See
These are accurate enough to produce well-aligned images in a wide
variety of situations. When local distortions and parallax are
present in the images, the images will still usually be aligned, with
the distortions and parallax appearing as inter-image shifts.
The animation below shows three image aligned iPhone images taken at a resort in Hawaii. The misalignments due to a small change in viewpoint (and individuals walking in the field of view) may be seen in the trees and, especially, in the bushes on the front right.
Even in this case, the resulting correspondences are generally
quite
accurate, and for montaging, our seam-selection algorithm can remove
most of the motion artifacts, as seen in the following montage
Geometric Distortion Correction
When multiple images are aligned, there are extra degrees of freedom
in the transformations, adding flexibility to the mapping of the
images. i2k Align™ uses this flexibility to undo as much of the
mapping distortion as it can, substantially improving the overall
appearance of the montage. The following two montages show the
difference between not using
distortion correction (top) and using distortion correction (bottom).
Particular attention should be paid to the horizon.
Automatic Masking
Due to the physical set-up of the image acquisition process, some non-photographic images contain an outer region that is outside the true image content. Automatically cropping this area is needed for clean montages, and it aids in the align process, especially for low overlap images. When this exterior region is all black or all white, the masking process is trivial, but it can become considerably harder for poor quality images (e.g. images scanned from slides). i2k Align™ offers the option to automatically compute a convex mask for images labeled "Other Modalities".
Uneven Illumination and Vignetting Correction
Some photographic lenses and and most retinal fundus cameras produce
images that tend to be darker on the image periphery than near the
image center. For photographic lenses this is due to "vignetting",
while for fundus images it is caused by uneven illumination. In
either case, good quality montages and better overall appearance may
be obtained if this can be automatically corrected.
DualAlign™ has developed new algorithms for automatically correcting vignetting and uneven illumination both in a single image and across multiple images. These are applied in the photographic montaging process of i2k Align™.
Camera Parameter Estimation and Illumination Correction
Different settings on the same camera can produce different recorded
colors and overall image intensities. Even with the same aperture and
shutter-speed settings, internal correction algorithms inside digital
cameras can produce different color values for the same object in two
different images. This problem is particularly difficult for
cell-phone cameras, which include essentially no user control.
DualAlign™ has developed algorithms for camera parameter estimation,
brightness correction and, to some degree, white balancing (color
correction) that eliminate many of these problems.
As an example, the montage below combines four images taken of the
Christian Science Center using the camera on a Blackberry Curve. Notice
the color transitions in the montage, especially between the bottom
and top of the building on the right and in the sky.
Now look at what happens after the i2k Align™ camera estimation,
brightness correction and white-balancing algorithms are applied:
Seam Selection
Photographers are limited in the panoramas they can build if they need to use a tripod and require a stationary scene. On the other hand, using a hand-held camera and photographing a dynamic scene requires software that eliminates artifacts due to parallax and corrects for movement in the scene. i2k Align™ includes algorithms that select seams and blending regions between images based on a detailed model of the image misalignments and color differences that remain after illumination correction.We have already seen an example above where the misalignments in three images taken at a resort in Hawaii were removed to form a seamless montage. This is not limited to photographic images. The montage of six thermal images below has had the image parallax removed through seam selection.
Six Input Images
The Resulting Montage
|
