Abstract:Neural networks have proven to be very successful in automatically capturing the composition of language and different structures across a range of multi-modal tasks. Thus, an important question to investigate is how neural networks learn and organise such structures. Numerous studies have examined the knowledge captured by language models (LSTMs, transformers) and vision architectures (CNNs, vision transformers) for respective uni-modal tasks. However, very few have explored what structures are acquired by multi-modal transformers where linguistic and visual features are combined. It is critical to understand the representations learned by each modality, their respective interplay, and the task's effect on these representations in large-scale architectures. In this paper, we take a multi-modal transformer trained for image captioning and examine the structure of the self-attention patterns extracted from the visual stream. Our results indicate that the information about different relations between objects in the visual stream is hierarchical and varies from local to a global object-level understanding of the image. In particular, while visual representations in the first layers encode the knowledge of relations between semantically similar object detections, often constituting neighbouring objects, deeper layers expand their attention across more distant objects and learn global relations between them. We also show that globally attended objects in deeper layers can be linked with entities described in image descriptions, indicating a critical finding - the indirect effect of language on visual representations. In addition, we highlight how object-based input representations affect the structure of learned visual knowledge and guide the model towards more accurate image descriptions. A parallel question that we investigate is whether the insights from cognitive science echo the structure of representations that the current neural architecture learns. The proposed analysis of the inner workings of multi-modal transformers can be used to better understand and improve on such problems as pre-training of large-scale multi-modal architectures, multi-modal information fusion and probing of attention weights. In general, we contribute to the explainable multi-modal natural language processing and currently shallow understanding of how the input representations and the structure of the multi-modal transformer affect visual representations.

Analyzing The Language of Visual Tokens

Visualizing and Understanding Neural Models in NLP

Understanding the Effect of using Semantically Meaningful Tokens for Visual Representation Learning

Language Model Beats Diffusion -- Tokenizer is Key to Visual Generation

Towards Interpreting Visual Information Processing in Vision-Language Models

Principles of Visual Tokens for Efficient Video Understanding

Tokens, the oft-overlooked appetizer: Large language models, the distributional hypothesis, and meaning

How structured are the representations in transformer-based vision encoders? An analysis of multi-object representations in vision-language models

From Pixels to Tokens: Byte-Pair Encoding on Quantized Visual Modalities

What Does a Language-And-Vision Transformer See: The Impact of Semantic Information on Visual Representations

What Makes for Good Visual Tokenizers for Large Language Models?

Break the Visual Perception: Adversarial Attacks Targeting Encoded Visual Tokens of Large Vision-Language Models

Vision Transformers with Natural Language Semantics

Beyond Text: Frozen Large Language Models in Visual Signal Comprehension

Elucidating the design space of language models for image generation

Not All Images Are Worth 16x16 Words: Dynamic Vision Transformers with Adaptive Sequence Length

A Spitting Image: Modular Superpixel Tokenization in Vision Transformers

TokenLearner: What Can 8 Learned Tokens Do for Images and Videos?

An Image is Worth More Than 16x16 Patches: Exploring Transformers on Individual Pixels

Pursuing Atomic Video Words By Information Projection

OmniTokenizer: A Joint Image-Video Tokenizer for Visual Generation