New framework enhances remote sensing image fusion with frequency-independent feature learning

November 5, 2024 by Chen Na, Chinese Academy of Sciences

Collected at: https://techxplore.com/news/2024-11-framework-remote-image-fusion-frequency.html

A research team led by Prof. Xie Chengjun and Zhang Jie from the Hefei Institutes of Physical Science of the Chinese Academy of Sciences developed a frequency domain-independent feature learning framework that allows for better representation and fusion of different types of remote sensing images.

This work was recently published in IEEE Transactions on Circuits and Systems for Video Technology.

Pan-sharpening, a critical technology in remote sensing image processing, combines high-resolution panchromatic images with low-resolution multispectral images to produce detailed high-resolution multispectral images. This technology is vital for enhancing the balance between spatial and spectral resolution in optical remote sensing satellites.

Current pan-sharpening methods assume identical data distributions in training and testing datasets and often falter when dealing with out-of-distribution data.

The research team introduced a frequency decoupled domain-independent feature learning framework. This approach analyzes domain-independent information distribution in image amplitude and phase components, utilizing frequency information separation modules and learnable high-frequency filters to decouple image information.

The processed information goes through two dedicated sub-networks, and gets dynamical feature channels adjustment to improve image fusion and quality.

Cross-scenario tests on multiple public datasets showed that the framework has a strong generalization performance, and can effectively handle diverse data distributions. It maintained excellent performance on the training set of WorldView-III and outperformed other methods on generalization datasets.

Visual comparisons confirmed that this framework effectively extracts and learns information, and will ensure consistent performance even with varying data distributions.

This framework marks a significant step forward for applications requiring high-fidelity image data across a wide array of satellite imaging scenarios, according to the team.

More information: Jie Zhang et al, Frequency decoupled domain-irrelevant feature learning for Pan-sharpening, IEEE Transactions on Circuits and Systems for Video Technology (2024). DOI: 10.1109/TCSVT.2024.3480950