Pansharpening using The Gram-Schmidt method
Who doesn’t love high-resolution satellite data? Unfortunately, we don’t always have the luxury of high-quality sensors and we often find ourselves dealing with low-quality multispectral imagery. Well, we still need to find a way to work with them. Is there any way to increase the resolution of multi-spectral satellite imagery without changing the sensor which captured it itself? There is, and it’s called Pansharpening.
Pansharpening is the process of combining the panchromatic band with a lower-resolution multispectral band to produce a higher-resolution multispectral band. The panchromatic band, commonly supplied by satellite imagery providers, combines all visible bands into one high-resolution grayscale image. This band captures a lot more energy due to it capturing a larger portion of the EM spectrum. This allows it to sample much smaller areas than normal bands, leading to its high resolution. For example, Landsat-9’s RGB band resolution is 30m, while its pan band resolution is 15m.
The pansharpening process is used almost everywhere. Map providers particularly love it — Google Maps allows you to zoom into satellite pictures with crystal clear quality. While upscaling multi-spectral imagery can sometimes provide great data, it also creates a very difficult-to-address issue: the upscaled data must possess all the spectral characteristics of the original data. Multiple methods of pansharpening exist, each of which has a novel take on keeping the number of spatial distortions to a minimum.
In this article, we will look at one of the most interesting pansharpening methods, the Gram-Schmidt transformation. This method was first published in 1998 and was shortly after patented by Kodak in 2000. Since then, it has become one of the most widely used methods of upscaling in imaging. It adapts the Gram-Schmidt vector orthogonalization process.
Understanding Gram-Schmidt orthogonalization
Orthogonalization is the process of discovering two vectors that are at 90 degrees to each other. When two vectors are orthogonalized, they will become orthonormal to each other; they will be both unit vectors and orthogonal to each other. The Gram-Schmidt method uses projection, a type of vector transformation that preserves addition and multiplication in the vector space. Applying a projection transformation to a vector more than once will give the same result as if it was applied only once.
The projection operator can be defined as followed:
u and v are two vectors. <u, v> is the inner product of the two vectors, showing the component of u in the direction of v. When <u, v> = 0, the two vectors are orthogonal (at right angles with each other). This operation causes a projection transformation of the vector v in the line on which u is situated.
The Gram-Schmidt process uses this operator to compute two sequences, u₁…uᵢ and n₁…nᵢ:
The sequence u₁…uᵢ is the set of orthogonal vectors. To calculate uᵢ, this process essentially transforms vi using the projection transformation onto the linear subspace created by u₁…uᵢ. uᵢ is therefore the difference between vi and this subspace. Calculating the sequence u₁…uᵢ is Gram-Schmidt orthogonalization, giving us the required set of orthogonal vectors while calculating the vector set n₁…nᵢ is Gram-Schmidt orthonormalization.
Pansharpening using Gram-Schmidt
The first step in pansharpening using the Gram-Schmidt method is to create a lower-resolution panchromatic band that matches the resolution of the multispectral bands. This is done using a pre-defined sensor-specific multispectral weighted average of the multispectral bands. Next, the multispectral bands are decorrelated. Each one is treated as one multi-dimensional vector, with each pixel being one dimension. The greater the number of pixels, the greater the number of dimensions. This decorrelation is done using the Gram-Schmidt process.
The low-resolution panchromatic band which we computed is taken as the first vector, u1. The multispectral bands are correlated and orthogonalized to this band. Next, the low-rez pan band is replaced by the high-rez one. The multi-spectral bands transform in the same way as the lower-resolution pan band. This produces high-resolution versions of themself.
