Abstract:The deployment of multimodal large language models (MLLMs) has demonstrated remarkable success in engaging in conversations involving visual inputs, thanks to the superior power of large language models (LLMs). Those MLLMs are typically built based on the LLMs, with an image encoder to process images into the token embedding space of the LLMs. However, the integration of visual modality has introduced a unique vulnerability: the MLLM becomes susceptible to malicious visual inputs and prone to generating sensitive or harmful responses, even though the LLM has been trained on textual dataset to align with human value. In this paper, we first raise the question: ``Do the MLLMs possess safety-awareness against malicious image inputs?". We find that after adding a principle that specifies the safety requirement into the input of the MLLM, the model's safety awareness becomes boosted. This phenomenon verifies the existence of MLLM's safety-awareness against image inputs, it is only weakened by the modality gap. We then introduce a simple yet effective technique termed CoCA, which amplifies the safety-awareness of the MLLM by calibrating its output distribution. Our proposed strategy helps the model reclaim its original safety awareness without losing its original capabilities. We verify the effectiveness of our approach on both multimodal safety and understanding benchmarks.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the security awareness of multimodal large - language models (MLLMs) when dealing with malicious image inputs. Although the security of these models on text data has been well - trained through techniques such as supervised fine - tuning (SFT) or reinforcement learning from human feedback (RLHF), after the introduction of the visual modality, MLLMs become sensitive to malicious image inputs and are prone to generating harmful responses. The key question raised in the paper is: "Do MLLMs have security awareness against malicious image inputs?" The research found that MLLMs do have this security awareness, but it is weakened due to modal differences (i.e., the gap between the continuous representation of images and the discrete representation of text). To solve this problem, the author proposes a simple and effective method - Constitutional Calibration (CoCA). CoCA enables the model to regain its original security awareness without losing its original capabilities by magnifying the influence of security prompts when calibrating the model output distribution. Specifically, CoCA calculates the difference between the predicted logits with and without security principles during the decoding stage and adds this difference multiplied by a scaling factor to the original logit. This method not only enhances the model's security awareness but also retains the model's visual understanding and reasoning abilities. The main contributions of the paper include: 1. Verifying that MLLMs do have security awareness against malicious image inputs, but it is weakened due to modal differences. 2. Exploring a simple and effective training - free method - Constitutional Calibration (CoCA), which effectively magnifies the influence of security prompts and enables the model to regain its original security awareness. 3. Through extensive experiments, it is verified that CoCA not only enhances security awareness but also retains the model's original visual understanding and reasoning abilities.

CoCA: Regaining Safety-awareness of Multimodal Large Language Models with Constitutional Calibration

MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Models

Eyes Closed, Safety On: Protecting Multimodal LLMs via Image-to-Text Transformation

MLLM-Protector: Ensuring MLLM's Safety without Hurting Performance

Safety of Multimodal Large Language Models on Images and Text.

SafeBench: A Safety Evaluation Framework for Multimodal Large Language Models

Safety of Multimodal Large Language Models on Images and Texts

Unbridled Icarus: A Survey of the Potential Perils of Image Inputs in Multimodal Large Language Model Security

Multimodal Situational Safety

CompCap: Improving Multimodal Large Language Models with Composite Captions

MLLMGuard: A Multi-dimensional Safety Evaluation Suite for Multimodal Large Language Models

Immune: Improving Safety Against Jailbreaks in Multi-modal LLMs via Inference-Time Alignment

Mitigating Modality Prior-Induced Hallucinations in Multimodal Large Language Models via Deciphering Attention Causality

A Comprehensive Survey of Multimodal Large Language Models: Concept, Application and Safety

Coherence-Driven Multimodal Safety Dialogue with Active Learning for Embodied Agents

Safety Alignment for Vision Language Models

VLSBench: Unveiling Visual Leakage in Multimodal Safety

MLLM-as-a-Judge for Image Safety without Human Labeling

Cross-modality Information Check for Detecting Jailbreaking in Multimodal Large Language Models

Incorporating Visual Experts to Resolve the Information Loss in Multimodal Large Language Models

Unveiling Uncertainty: A Deep Dive into Calibration and Performance of Multimodal Large Language Models