Abstract:Stable diffusion has demonstrated strong image synthesis ability to given text descriptions, suggesting it to contain strong semantic clue for grouping objects. The researchers have explored employing stable diffusion for training-free segmentation. Most existing approaches refine cross-attention map by self-attention map once, demonstrating that self-attention map contains useful semantic information to improve segmentation. To fully utilize self-attention map, we present a deep experimental analysis on iteratively refining cross-attention map with self-attention map, and propose an effective iterative refinement framework for training-free segmentation, named iSeg. The proposed iSeg introduces an entropy-reduced self-attention module that utilizes a gradient descent scheme to reduce the entropy of self-attention map, thereby suppressing the weak responses corresponding to irrelevant global information. Leveraging the entropy-reduced self-attention module, our iSeg stably improves refined cross-attention map with iterative refinement. Further, we design a category-enhanced cross-attention module to generate accurate cross-attention map, providing a better initial input for iterative refinement. Extensive experiments across different datasets and diverse segmentation tasks reveal the merits of proposed contributions, leading to promising performance on diverse segmentation tasks. For unsupervised semantic segmentation on Cityscapes, our iSeg achieves an absolute gain of 3.8% in terms of mIoU compared to the best existing training-free approach in literature. Moreover, our proposed iSeg can support segmentation with different kinds of images and interactions. The project is available at <a class="link-external link-https" href="https://linsun449.github.io/iSeg" rel="external noopener nofollow">this https URL</a>.

TinySeg: Model Optimizing Framework for Image Segmentation on Tiny Embedded Systems

Troubleshooting image segmentation models with human-in-the-loop

Edge Segmentation: Empowering Mobile Telemedicine with Compressed Cellular Neural Networks

Computation and Memory Efficient Image Segmentation

T-RECX: Tiny-Resource Efficient Convolutional neural networks with early-eXit

JetSeg: Efficient Real-Time Semantic Segmentation Model for Low-Power GPU-Embedded Systems

TinySAM: Pushing the Envelope for Efficient Segment Anything Model

Memory-Constrained Semantic Segmentation for Ultra-High Resolution UAV Imagery

Embedded Deployment of Semantic Segmentation in Medicine through Low-Resolution Inputs

HiRISE: High-Resolution Image Scaling for Edge ML via In-Sensor Compression and Selective ROI

Tiny Machine Learning: Progress and Futures

Memory-based Semantic Segmentation for Off-road Unstructured Natural Environments

TinyAD: Memory-efficient anomaly detection for time series data in Industrial IoT

An Efficient Data Analytics and Optimized Algorithm for Enhancing the Performance of Image Segmentation Using Deep Learning Model

iSeg: An Iterative Refinement-based Framework for Training-free Segmentation

CAD: Memory Efficient Convolutional Adapter for Segment Anything

Design of a Novel Neural Network Compression Method for Tiny Machine Learning

Fibonet: A Light-weight and Efficient Neural Network for Image Segmentation

Faster Segment Anything: Towards Lightweight SAM for Mobile Applications

Efficient Neural Networks for Tiny Machine Learning: A Comprehensive Review

LightEyes: A Lightweight Fundus Segmentation Network for Mobile Edge Computing