Optimal spectral image fusion for detection of shoreline targets

Spectral-spatial sharpening processes can be used to identify spectral features or anomalies in high-spatial-resolution multispectral (MS) data and in high-spectral-resolution imagery.^1–3 Such HS and MS imagery can be used to process pushbroom images that are obtained with the use of inertial motion units and augmented global positioning from airborne or shipborne platforms. These image analysis techniques thus hold much promise for enhancing situational awareness during environmental surveillance and monitoring activities.

In a previously developed technique for the fusion of hyperspectral (HS) and MS signatures, line targets are numerically embedded in the imagery.⁴ To minimize differences between these high-spatial-resolution HS synthetic signatures and the actual observed HS signatures, however, requires co-registration of the images. This co-registration can be achieved with the use of nearest neighborhood resampling methods, where randomly sampled areas and pixels within the images are selected from the co-registered data. In addition, singular value decomposition⁵ can be performed on random pairs of pixels in the MS channels and on each HS channel. This mathematical process allows synthetic HS signatures to be ‘predicted.’ Images are thus created for each HS channel from the MS channels, and synthetic HS images with high spatial resolution are produced. To insure that optimal spectral sharpening is achieved, the synthetic HS imagery is processed using low-frequency and high-frequency filtering, i.e., with a 2D discrete cosine and inverse cosine transform⁶ and a 2D Butterworth filter.⁷ The use of these (or similar) filters allows spatial features found in high-spatial-resolution MS imagery to be accurately transferred or embedded into the final synthetic HS images (and with higher spatial resolution). In addition, the high-spectral-resolution features can be optimally transferred from the observed low-spatial-resolution HS imagery to the synthetic high-spatial-resolution HS data cubes. A key step in the optimization of this frequency filtering is changing the cutoff and order coefficients of the 2D Butterworth filter. This can be accomplished with the use of interval-halving techniques, in which each band and processed pixel signature is randomly selected so that the original and final sharpened signatures can be compared. The nonparametric Kolmogorov-Smirnov (KS) test⁵ value can be used to make this comparison. For this test, it is necessary to create ‘cumulative signatures’ of synthetic corrected reflectance,⁸ radiance, or digital counts versus wavelength (using random samples of pixels and areas). These steps allow the differences between the observed and predicted signatures to be minimized (see Figure 1).

Figure 1. Illustration of how a cumulative spectral signature (right) can be created from a scaled spectral signature (left), for use in the nonparametric Kolmogorov-Smirnov test. This statistic can be used to compare observed and optimized synthetic spectral signatures. The test is also used in singular value decomposition and synthetic hyperspectral image model building processes, as well as in optimizing the order and cutoff frequency during 2D Butterworth filtering (used for spectral-spatial sharpening of multispectral and hyperspectral images).

In this work a technique for numerically embedding targets in high-spatial-resolution MS images is demonstrated for the purpose of investigating littoral zone features within images acquired from the shorelines of the Gulf of Mexico.⁹ In addition, an optimal fusion of HS and MS imagery is performed to obtain new higher spatial and spectral resolution signatures of targets that are obscured by shoreline vegetation. With this approach, it is possible to achieve better detection of difficult targets such as these (and those in the submerged littoral zone) through the use of combined data from different sensor types, image fusion, and simulation techniques.

There are several ways to test the optimization of a spectral-spatial sharpening process. One method, used in this work, is to use line targets or features that are mathematically embedded within the observed high-spatial resolution MS imagery and the associated spectral anomalies in the HS images. In this technique, the HS and MS original channels or bands are spatially and spectrally ‘spiked’ (i.e., embedded) with an artificial feature or target. The MS channels from an area can be modified by embedding a line target (changing the pixel values). In the same way, a spectral anomaly is embedded into the corresponding area of the HS scene. The spectral-spatial sharpening process is then run to determine the optimal filter cutoff, as well as the order coefficients that result from the optimization and the KS signature tests. It is therefore possible to observe and quantify the influence of the optimization procedures by recovering the line target in the new spectrally and spatially sharpened imagery. A single realization of the process—in which the results of embedding imagery for testing optimized spectral-spatial resolution enhancement protocols are demonstrated—is illustrated in Figure 2.

Figure 2. Realization of the spectral-spatial sharpening process. The original multispectral (MS) image—of the numerically embedded line target on the shore—is shown on the left. The upper left of this image shows the hyperspectral (HS) anomaly, at a lower spatial resolution, embedded in the HS channels at the location of the line target shown. The resulting synthetic spatial-spectral sharpened HS image (with the line target on the shoreline) is shown in the middle. An example image of the embedded line target is shown in the upper left of this middle image and a zoomed-in version of this optimized spectral-spatial sharpened line target is given in the image at the top right. The lower right image is a three-band red-green-blue (RGB) display of the sharpened HS cube of the shoreline.

These spectral-spatial sharpening techniques have been applied to images that were collected from aircraft and sea vessels.^10–12 The HS and MS images that were collected simultaneously are shown in Figure 2. These images were obtained from a vessel that was close to a shoreline with known anomalies (i.e., weathered oil features) at the water–land interface. After the sharpening process on these images was complete, the suspected anomalies could be analyzed with the use of optimized higher-order derivative spectroscopy¹³ and optimal multi-wavelength contrast algorithms. Optimal translating and dilating wavelet analysis¹¹ of the synthetic HS signatures was also conducted.

Other examples in which this approach is demonstrated are shown in Figures 3 and 4. For both these cases the spectral-spatial sharpening operations, for feature detection of weathered oil on shorelines, were used. In Figure 3, the HS imagery (middle) has a ground sampling distance (GSD) of about 5cm. With the use of the MS digital image shown on the left (with a GSD of about 5mm), the middle image was sharpened. This spectral-spatial sharpening of the HS image cube's spectra enhanced the feature detections (i.e., of the weathered oil residue). As such, this methodology has been successfully demonstrated as an effective alternative to visual inspections and surveillance. Airborne and vessel HS pushbroom sensor data can also be corrected,¹⁴ with the use of an inertial motion unit.^15–17 An optimized airborne three-channel MS-HS sharpened image is shown in Figure 4. The sharpened image illustrates the presence of submerged weathered oil features along Florida's Gulf of Mexico ‘panhandle’ shoreline. This data was acquired with a GSD of about 10cm, from an altitude of about 1500m.

Figure 3. Images of Jimmy Bay, LA, acquired from a small vessel. The HS image in the middle has a ground sampling distance (GSD) of about 5cm, and is sharpened using the MS digital image (left) that has a GSD of about 5mm. The spectral-spatial sharpened HS image cube spectra (right) allow enhanced feature detection of weathered oil residue along the shoreline.

Figure 4. An optimized three-channel MS-HS spectral-spatial sharpened shoreline image that shows subsurface weathered oil features along the Gulf of Mexico and Florida's ‘panhandle’ coastline. The image was acquired from an airborne platform at an altitude of 1500m and with a GSD of about 10cm.

In summary, a technique for achieving an optimal fusion of hyperspectral and multispectral imagery, in which line targets are numerically embedded into acquired images, has been developed. The differences between high-spatial-resolution synthetic and observed spectral signatures are also minimized. Images that were obtained from airborne and shipborne platforms have been used to demonstrate the power of this technique for target detection along shorelines. An area of ongoing work involves the use of the optimized HS-MS image fusion approach for studying in situ colloidal aggregates and flocs within lutoclines and moving fluid mud layers.