Abstract:Camouflaged Object Segmentation (COS) faces significant challenges due to the scarcity of annotated data, where meticulous pixel-level annotation is both labor-intensive and costly, primarily due to the intricate object-background boundaries. Addressing the core question, "Can COS be effectively achieved in a zero-shot manner without manual annotations for any camouflaged object?" we affirmatively respond and introduce a robust zero-shot COS framework. This framework leverages the inherent local pattern bias of COS and employs a broad semantic feature space derived from salient object segmentation (SOS) for efficient zero-shot transfer. We incorporate an Masked Image Modeling (MIM) based image encoder optimized for Parameter-Efficient Fine-Tuning (PEFT), a Multimodal Large Language Model (M-LLM), and a Multi-scale Fine-grained Alignment (MFA) mechanism. The MIM pre-trained image encoder focuses on capturing essential low-level features, while the M-LLM generates caption embeddings processed alongside these visual cues. These embeddings are precisely aligned using MFA, enabling our framework to accurately interpret and navigate complex semantic contexts. To optimize operational efficiency, we introduce a learnable codebook that represents the M-LLM during inference, significantly reducing computational overhead. Our framework demonstrates its versatility and efficacy through rigorous experimentation, achieving state-of-the-art performance in zero-shot COS with $F_{\beta}^w$ scores of 72.9\% on CAMO and 71.7\% on COD10K. By removing the M-LLM during inference, we achieve an inference speed comparable to that of traditional end-to-end models, reaching 18.1 FPS. Code: <a class="link-external link-https" href="https://github.com/R-LEI360725/ZSCOS-CaMF" rel="external noopener nofollow">this https URL</a>

Delve into Cosine: Few-shot Object Detection Via Adaptive Norm-cutting Scale

Bilaterally Normalized Scale-Consistent Sinkhorn Distance for Few-Shot Image Classification

Towards Stabilized Few-Shot Object Detection with Less Forgetting via Sample Normalization

Diagnosing and Remedying Shot Sensitivity with Cosine Few-Shot Learners

Boosting Few-Shot Object Detection with Discriminative Representation and Class Margin

Enhancing Few-Shot Object Detection with Modified Faster R-CNN and Transfer Learning

NegCosIC: Negative Cosine Similarity-Invariance- Covariance Regularization for Few-Shot Learning

Beyond Max-Margin: Class Margin Equilibrium for Few-shot Object Detection

Scale adaption-guided human face detection

Understanding Negative Proposals in Generic Few-Shot Object Detection

Mutually Reinforcing Structure with Proposal Contrastive Consistency for Few-Shot Object Detection.

Few-Shot Object Detection Based on Adaptive Attention Mechanism and Large-Margin Softmax

In defense of local descriptor-based few-shot object detection

Cross attention redistribution with contrastive learning for few shot object detection

Multi-Scale Positive Sample Refinement for Few-Shot Object Detection

Few-Shot Object Detection by Second-Order Pooling

Adapting Object Detectors with Conditional Domain Normalization

Cos R-CNN for Online Few-shot Object Detection

Few-shot X-ray Prohibited-item Detection Based on Multi-scale Feature Fusion and Sample Balancing

Decoupling Classifier for Boosting Few-shot Object Detection and Instance Segmentation

Towards Real Zero-Shot Camouflaged Object Segmentation without Camouflaged Annotations