Ultra-high resolution image understanding

Parameter-Efficient Local-Context Cooperation via Vision Foundation Models for UHR Remote Sensing Image Segmentation

Qi Li, Chunxiao Chen, Jiaxin Cai, Junjie Huang, Yansheng Li, Wenxi Liu

2026 IEEE Transactions on Geoscience and Remote Sensing (TGRS) Paper

Abstract: Ultrahigh resolution (UHR) remote sensing image segmentation aims to achieve a fine-grained understanding of complex ground scenes. In recent years, vision foundation models (VFMs) have shown strong capability in learning generic structural priors from large-scale visual data, indicating great potential for such fine-grained scene understanding. However, their application to UHR remote sensing images remains limited, as the massive parameter scales of VFMs are difficult to train under UHR remote sensing images. Motivated by the success of the parameter-efficient fine-tuning paradigm on VFMs, we propose a novel parameter-efficient local-context cooperation (PEACE) framework, which significantly reduces trainable parameter overhead while improving segmentation accuracy. In particular, PEACE leverages a shared VFM with minimal trainable parameters to collaboratively process local and corresponding contextual patches partitioned from the UHR remote sensing image. A multireceptive local adapter (MRLA) and a multireceptive context adapter (MRCA) are designed to capture spatial features of local and contextual inputs across multiple receptive fields. Finally, contextual semantics are integrated into local representations. Furthermore, a context-sensitive assistance strategy (CSAS) leverages the correct prediction of the context to effectively overcome the primary limitations of the patch-based training paradigm. Experimental results demonstrate that PEACE effectively exploits VFMs and remote sensing foundation models (RSFMs) with minimal parameter increments and achieves versatility and superior performance across several UHR remote sensing image benchmarks.

Playing to the Strengths of High-and Low-Resolution Cues for Ultra-high Resolution Image Segmentation

Qi Li, Jiexin Luo, Chunxiao Chen, Jiaxin Cai, Wenjie Yang, Yuanlong Yu, Shengfeng He, Wenxi Liu

2025 IEEE Robotics and Automation Letters (RA-L) Paper Code

Abstract: In ultra-high resolution image segmentation task for robotic platforms like AAVs and autonomous vehicles, existing paradigms process a downsampled input image through a deep network and the original high-resolution image through a shallow network, then fusing their features for final segmentation. Although these features are designed to be complementary, they often contain redundant or even conflicting semantic information, which leads to blurred edge contours, particularly for small objects. This is especially detrimental to robotics applications requiring precise spatial awareness. To address this challenge, we propose a novel paradigm that disentangles the task into two independent subtasks concerning high- and low-resolution inputs, leveraging high-resolution features exclusively to capture low-level structured details and low-resolution features for extracting semantics. Specifically, for the high-resolution input, we propose a region-pixel association experts scheme that partitions the image into multiple regions. For the low-resolution input, we assign compact semantic tokens to the partitioned regions. Additionally, we incorporate a high-resolution local perception scheme with an efficient field-enriched local context module to enhance small object recognition in case of incorrect semantic assignment. Extensive experiments demonstrate the state-of-the-art performance of our method and validate the effectiveness of each designed component.

Memory-constrained semantic segmentation for ultra-high resolution UAV imagery

Qi Li, Jiaxin Cai, Jiexin Luo, Yuanlong Yu, Jason Gu, Jia Pan, Wenxi Liu

2024 IEEE Robotics and Automation Letters (RA-L) Paper Code

Abstract: Ultra-high resolution image segmentation poses a formidable challenge for UAVs with limited computation resources. Moreover, with multiple deployed tasks (e.g., mapping, localization, and decision making), the demand for a memory efficient model becomes more urgent. This letter delves into the intricate problem of achieving efficient and effective segmentation of ultra-high resolution UAV imagery, while operating under stringent GPU memory limitation. To address this problem, we propose a GPU memory-efficient and effective framework. Specifically, we introduce a novel and efficient spatial-guided high-resolution query module, which enables our model to effectively infer pixel-wise segmentation results by querying nearest latent embeddings from low-resolution features. Additionally, we present a memory-based interaction scheme with linear complexity to rectify semantic bias beneath the high-resolution spatial guidance via associating cross-image contextual semantics. For evaluation, we perform comprehensive experiments over public benchmarks under both conditions of small and large GPU memory usage limitations. Notably, our model gains around 3% advantage against SOTA in mIoU using comparable memory. Furthermore, we show that our model can be deployed on the embedded platform with less than 8 G memory like Jetson TX2.

Ultra-high Resolution Image Segmentation via Locality-aware Context Fusion and Alternating Local Enhancement

Wenxi Liu, Qi Li, Xindai Lin, Weixiang Yang, Shengfeng He, Yuanlong Yu

2024 International Journal of Computer Vision (IJCV) Paper

Abstract: Ultra-high resolution image segmentation has raised increasing interests in recent years due to its realistic applications. In this paper, we innovate the widely used high-resolution image segmentation pipeline, in which an ultra-high resolution image is partitioned into regular patches for local segmentation and then the local results are merged into a high-resolution semantic mask. In particular, we introduce a novel locality-aware context fusion based segmentation model to process local patches, where the relevance between local patch and its various contexts are jointly and complementarily utilized to handle the semantic regions with large variations. Additionally, we present the alternating local enhancement module that restricts the negative impact of redundant information introduced from the contexts, and thus is endowed with the ability of fixing the locality-aware features to produce refined results. Furthermore, in comprehensive experiments, we demonstrate that our model outperforms other state-of-the-art methods in public benchmarks and verify the effectiveness of the proposed modules.

From contexts to locality: Ultra-high resolution image segmentation via locality-aware contextual correlation

Qi Li, Weixiang Yang, Wenxi Liu, Yuanlong Yu, Shengfeng He

2021 ICCV Paper Code

Abstract: Ultra-high resolution image segmentation has raised increasing interests in recent years due to its realistic applications. In this paper, we innovate the widely used high-resolution image segmentation pipeline, in which an ultra-high resolution image is partitioned into regular patches for local segmentation and then the local results are merged into a high-resolution semantic mask. In particular, we introduce a novel locality-aware contextual correlation based segmentation model to process local patches, where the relevance between local patch and its various contexts are jointly and complementarily utilized to handle the semantic regions with large variations. Additionally, we present a contextual semantics refinement network that associates the local segmentation result with its contextual semantics, and thus is endowed with the ability of reducing boundary artifacts and refining mask contours during the generation of final high-resolution mask. Furthermore, in comprehensive experiments, we demonstrate that our model outperforms other state-of-the-art methods in public benchmarks.