Pharmacological characterization of phenytoin-resistant amygdala-kindled rats, a new model of drug-resistant partial epilepsy

W. Löscher,C. Rundfeldt,D. Hönack

DOI: https://doi.org/10.1016/0920-1211(93)90058-F

IF: 2.991

1993-07-01

Epilepsy Research

Abstract:

What problem does this paper attempt to address?

Leveraging Superfluous Information in Contrastive Representation Learning

Xuechu Yu

2024-08-20

Abstract:Contrastive representation learning, which aims to learnthe shared information between different views of unlabeled data by maximizing the mutual information between them, has shown its powerful competence in self-supervised learning for downstream tasks. However, recent works have demonstrated that more estimated mutual information does not guarantee better performance in different downstream tasks. Such works inspire us to conjecture that the learned representations not only maintain task-relevant information from unlabeled data but also carry task-irrelevant information which is superfluous for downstream tasks, thus leading to performance degeneration. In this paper we show that superfluous information does exist during the conventional contrastive learning framework, and further design a new objective, namely SuperInfo, to learn robust representations by a linear combination of both predictive and superfluous information. Besides, we notice that it is feasible to tune the coefficients of introduced losses to discard task-irrelevant information, while keeping partial non-shared task-relevant information according to our SuperInfo loss.We demonstrate that learning with our loss can often outperform the traditional contrastive learning approaches on image classification, object detection and instance segmentation tasks with significant improvements.

Machine Learning,Artificial Intelligence
Exploiting Pseudo Image Captions for Multimodal Summarization.

Chaoya Jiang,Rui Xie,Wei Ye,Jinan Sun,Shikun Zhang

DOI: https://doi.org/10.18653/v1/2023.findings-acl.12

2023-01-01

Abstract:Cross-modal contrastive learning in vision language pretraining (VLP) faces the challenge of (partial) false negatives. In this paper, we study this problem from the perspective of Mutual Information (MI) optimization. It is common sense that InfoNCE loss used in contrastive learning will maximize the lower bound of MI between anchors and their positives, while we theoretically prove that MI involving negatives also matters when noises commonly exist. Guided by a more general lower bound form for optimization, we propose a contrastive learning strategy regulated by progressively refined cross-modal similarity, to more accurately optimize MI between an image/text anchor and its negative texts/images instead of improperly minimizing it. Our method performs competitively on four downstream cross-modal tasks and systematically balances the beneficial and harmful effects of (partial) false negative samples under theoretical guidance.
A Mutual Information Perspective on Federated Contrastive Learning

Christos Louizos,Matthias Reisser,Denis Korzhenkov

2024-05-03

Abstract:We investigate contrastive learning in the federated setting through the lens of SimCLR and multi-view mutual information maximization. In doing so, we uncover a connection between contrastive representation learning and user verification; by adding a user verification loss to each client's local SimCLR loss we recover a lower bound to the global multi-view mutual information. To accommodate for the case of when some labelled data are available at the clients, we extend our SimCLR variant to the federated semi-supervised setting. We see that a supervised SimCLR objective can be obtained with two changes: a) the contrastive loss is computed between datapoints that share the same label and b) we require an additional auxiliary head that predicts the correct labels from either of the two views. Along with the proposed SimCLR extensions, we also study how different sources of non-i.i.d.-ness can impact the performance of federated unsupervised learning through global mutual information maximization; we find that a global objective is beneficial for some sources of non-i.i.d.-ness but can be detrimental for others. We empirically evaluate our proposed extensions in various tasks to validate our claims and furthermore demonstrate that our proposed modifications generalize to other pretraining methods.

Machine Learning
Long-Tailed Recognition by Mutual Information Maximization between Latent Features and Ground-Truth Labels

Min-Kook Suh,Seung-Woo Seo

2023-08-08

Abstract:Although contrastive learning methods have shown prevailing performance on a variety of representation learning tasks, they encounter difficulty when the training dataset is long-tailed. Many researchers have combined contrastive learning and a logit adjustment technique to address this problem, but the combinations are done ad-hoc and a theoretical background has not yet been provided. The goal of this paper is to provide the background and further improve the performance. First, we show that the fundamental reason contrastive learning methods struggle with long-tailed tasks is that they try to maximize the mutual information maximization between latent features and input data. As ground-truth labels are not considered in the maximization, they are not able to address imbalances between class labels. Rather, we interpret the long-tailed recognition task as a mutual information maximization between latent features and ground-truth labels. This approach integrates contrastive learning and logit adjustment seamlessly to derive a loss function that shows state-of-the-art performance on long-tailed recognition benchmarks. It also demonstrates its efficacy in image segmentation tasks, verifying its versatility beyond image classification.

Machine Learning,Computer Vision and Pattern Recognition
Robust Contrastive Learning against Noisy Views

Ching-Yao Chuang,R Devon Hjelm,Xin Wang,Vibhav Vineet,Neel Joshi,Antonio Torralba,Stefanie Jegelka,Yale Song

DOI: https://doi.org/10.48550/arXiv.2201.04309

2022-01-12

Abstract:Contrastive learning relies on an assumption that positive pairs contain related views, e.g., patches of an image or co-occurring multimodal signals of a video, that share certain underlying information about an instance. But what if this assumption is violated? The literature suggests that contrastive learning produces suboptimal representations in the presence of noisy views, e.g., false positive pairs with no apparent shared information. In this work, we propose a new contrastive loss function that is robust against noisy views. We provide rigorous theoretical justifications by showing connections to robust symmetric losses for noisy binary classification and by establishing a new contrastive bound for mutual information maximization based on the Wasserstein distance measure. The proposed loss is completely modality-agnostic and a simple drop-in replacement for the InfoNCE loss, which makes it easy to apply to existing contrastive frameworks. We show that our approach provides consistent improvements over the state-of-the-art on image, video, and graph contrastive learning benchmarks that exhibit a variety of real-world noise patterns.

Computer Vision and Pattern Recognition,Machine Learning
Separating common from salient patterns with Contrastive Representation Learning

Robin Louiset,Edouard Duchesnay,Antoine Grigis,Pietro Gori

2024-02-19

Abstract:Contrastive Analysis is a sub-field of Representation Learning that aims at separating common factors of variation between two datasets, a background (i.e., healthy subjects) and a target (i.e., diseased subjects), from the salient factors of variation, only present in the target dataset. Despite their relevance, current models based on Variational Auto-Encoders have shown poor performance in learning semantically-expressive representations. On the other hand, Contrastive Representation Learning has shown tremendous performance leaps in various applications (classification, clustering, etc.). In this work, we propose to leverage the ability of Contrastive Learning to learn semantically expressive representations well adapted for Contrastive Analysis. We reformulate it under the lens of the InfoMax Principle and identify two Mutual Information terms to maximize and one to minimize. We decompose the first two terms into an Alignment and a Uniformity term, as commonly done in Contrastive Learning. Then, we motivate a novel Mutual Information minimization strategy to prevent information leakage between common and salient distributions. We validate our method, called SepCLR, on three visual datasets and three medical datasets, specifically conceived to assess the pattern separation capability in Contrastive Analysis. Code available at https://github.com/neurospin-projects/2024_rlouiset_sep_clr.

Computer Vision and Pattern Recognition
Vision Language Pre-training by Contrastive Learning with Cross-Modal Similarity Regulation

Chaoya Jiang,Wei Ye,Haiyang Xu,Miang yan,Shikun Zhang,Jie Zhang,Fei Huang

2023-06-22

Abstract:Cross-modal contrastive learning in vision language pretraining (VLP) faces the challenge of (partial) false negatives. In this paper, we study this problem from the perspective of Mutual Information (MI) optimization. It is common sense that InfoNCE loss used in contrastive learning will maximize the lower bound of MI between anchors and their positives, while we theoretically prove that MI involving negatives also matters when noises commonly exist. Guided by a more general lower bound form for optimization, we propose a contrastive learning strategy regulated by progressively refined cross-modal similarity, to more accurately optimize MI between an image/text anchor and its negative texts/images instead of improperly minimizing it. Our method performs competitively on four downstream cross-modal tasks and systematically balances the beneficial and harmful effects of (partial) false negative samples under theoretical guidance.

Computer Vision and Pattern Recognition,Artificial Intelligence
$f$-MICL: Understanding and Generalizing InfoNCE-based Contrastive Learning

Yiwei Lu,Guojun Zhang,Sun Sun,Hongyu Guo,Yaoliang Yu

2024-02-16

Abstract:In self-supervised contrastive learning, a widely-adopted objective function is InfoNCE, which uses the heuristic cosine similarity for the representation comparison, and is closely related to maximizing the Kullback-Leibler (KL)-based mutual information. In this paper, we aim at answering two intriguing questions: (1) Can we go beyond the KL-based objective? (2) Besides the popular cosine similarity, can we design a better similarity function? We provide answers to both questions by generalizing the KL-based mutual information to the $f$-Mutual Information in Contrastive Learning ($f$-MICL) using the $f$-divergences. To answer the first question, we provide a wide range of $f$-MICL objectives which share the nice properties of InfoNCE (e.g., alignment and uniformity), and meanwhile result in similar or even superior performance. For the second question, assuming that the joint feature distribution is proportional to the Gaussian kernel, we derive an $f$-Gaussian similarity with better interpretability and empirical performance. Finally, we identify close relationships between the $f$-MICL objective and several popular InfoNCE-based objectives. Using benchmark tasks from both vision and natural language, we empirically evaluate $f$-MICL with different $f$-divergences on various architectures (SimCLR, MoCo, and MoCo v3) and datasets. We observe that $f$-MICL generally outperforms the benchmarks and the best-performing $f$-divergence is task and dataset dependent.

Machine Learning
Rethinking Minimal Sufficient Representation in Contrastive Learning

Haoqing Wang,Xun Guo,Zhi-Hong Deng,Yan Lu

DOI: https://doi.org/10.1109/cvpr52688.2022.01557

2022-01-01

Abstract:Contrastive learning between different views of the data achieves outstanding success in the field of self-supervised representation learning and the learned representations are useful in broad downstream tasks. Since all supervision information for one view comes from the other view, contrastive learning approximately obtains the minimal sufficient representation which contains the shared information and eliminates the non-shared information between views. Considering the diversity of the downstream tasks, it cannot be guaranteed that all task-relevant information is shared between views. Therefore, we assume the non-shared task-relevant information cannot be ignored and theoretically prove that the minimal sufficient representation in contrastive learning is not sufficient for the downstream tasks, which causes performance degradation. This reveals a new problem that the contrastive learning models have the risk of overfitting to the shared information between views. To alleviate this problem, we propose to increase the mutual information between the representation and input as regularization to approximately introduce more task-relevant information, since we cannot utilize any downstream task information during training. Extensive experiments verify the rationality of our analysis and the effectiveness of our method. It significantly improves the performance of several classic contrastive learning models in downstream tasks. Our code is available at https://github.com/Haoqing-Wang/InfoCL.
A Unified Framework for Contrastive Learning from a Perspective of Affinity Matrix

Wenbin Li,Meihao Kong,Xuesong Yang,Lei Wang,Jing Huo,Yang Gao,Jiebo Luo

DOI: https://doi.org/10.48550/arxiv.2211.14516

2022-01-01

Abstract: In recent years, a variety of contrastive learning based unsupervised visual representation learning methods have been designed and achieved great success in many visual tasks. Generally, these methods can be roughly classified into four categories: (1) standard contrastive methods with an InfoNCE like loss, such as MoCo and SimCLR; (2) non-contrastive methods with only positive pairs, such as BYOL and SimSiam; (3) whitening regularization based methods, such as W-MSE and VICReg; and (4) consistency regularization based methods, such as CO2. In this study, we present a new unified contrastive learning representation framework (named UniCLR) suitable for all the above four kinds of methods from a novel perspective of basic affinity matrix. Moreover, three variants, i.e., SimAffinity, SimWhitening and SimTrace, are presented based on UniCLR. In addition, a simple symmetric loss, as a new consistency regularization term, is proposed based on this framework. By symmetrizing the affinity matrix, we can effectively accelerate the convergence of the training process. Extensive experiments have been conducted to show that (1) the proposed UniCLR framework can achieve superior results on par with and even be better than the state of the art, (2) the proposed symmetric loss can significantly accelerate the convergence of models, and (3) SimTrace can avoid the mode collapse problem by maximizing the trace of a whitened affinity matrix without relying on asymmetry designs or stop-gradients.
Graph contrastive learning with min-max mutual information

Yuhua Xu,Junli Wang,Mingjian Guang,Chungang Yan,Changjun Jiang

DOI: https://doi.org/10.1016/j.ins.2024.120378

IF: 8.1

2024-03-03

Information Sciences

Abstract:Graph contrastive learning has achieved rapid development in learning representations from graph-structured data, which aims to maximize the mutual information between two representations learned from different augmented views of a graph. However, maximizing the mutual information between different views without any constraints may cause encoders to capture information irrelevant to downstream tasks, limiting the efficiency of graph contrastive learning methods. To tackle these issues, we propose a Graph Contrastive Learning method with Min-max mutual Information (GCLMI). Specifically, we conduct theoretical analysis to present our learning objective. It designs a min-max principle to constrain the mutual information among multiple views, including between a graph and each of its augmented views, as well as between different augmented views. Based on the learning objective, we further construct two augmented views by separating the feature and topology information of a graph to preserve different semantic information from the graph. Subsequently, we maximize the mutual information between each augmented view and the graph while minimizing the mutual information between two augmented views, to learn informative and diverse representations. Extensive experiments are conducted on a variety of graph datasets, and experimental results show that GCLMI achieves better or competitive performance compared with state-of-the-art methods.

computer science, information systems
Contrastive Multimodal Fusion with TupleInfoNCE

Yunze Liu,Qingnan Fan,Shanghang Zhang,Hao Dong,Thomas Funkhouser,Li Yi

DOI: https://doi.org/10.1109/iccv48922.2021.00079

2021-01-01

Abstract:This paper proposes a method for representation learning of multimodal data using contrastive losses. A traditional approach is to contrast different modalities to learn the information shared among them. However, that approach could fail to learn the complementary synergies between modalities that might be useful for downstream tasks. Another approach is to concatenate all the modalities into a tuple and then contrast positive and negative tuple correspondences. However, that approach could consider only the stronger modalities while ignoring the weaker ones. To address these issues, we propose a novel contrastive learning objective, TupleInfoNCE. It contrasts tuples based not only on positive and negative correspondences, but also by composing new negative tuples using modalities describing different scenes. Training with these additional negatives encourages the learning model to examine the correspondences among modalities in the same tuple, ensuring that weak modalities are not ignored. We provide a theoretical justification based on mutual-information for why this approach works, and we propose a sample optimization algorithm to generate positive and negative samples to maximize training efficacy. We find that TupleInfoNCE significantly outperforms previous state of the arts on three different downstream tasks.
On the Importance of Contrastive Loss in Multimodal Learning

Yunwei Ren,Yuanzhi Li

2023-04-08

Abstract:Recently, contrastive learning approaches (e.g., CLIP (Radford et al., 2021)) have received huge success in multimodal learning, where the model tries to minimize the distance between the representations of different views (e.g., image and its caption) of the same data point while keeping the representations of different data points away from each other. However, from a theoretical perspective, it is unclear how contrastive learning can learn the representations from different views efficiently, especially when the data is not isotropic. In this work, we analyze the training dynamics of a simple multimodal contrastive learning model and show that contrastive pairs are important for the model to efficiently balance the learned representations. In particular, we show that the positive pairs will drive the model to align the representations at the cost of increasing the condition number, while the negative pairs will reduce the condition number, keeping the learned representations balanced.

Machine Learning,Computation and Language,Computer Vision and Pattern Recognition
Model-Aware Contrastive Learning: Towards Escaping the Dilemmas

Zizheng Huang,Haoxing Chen,Ziqi Wen,Chao Zhang,Huaxiong Li,Bo Wang,Chunlin Chen

DOI: https://doi.org/10.48550/arXiv.2207.07874

2023-06-11

Abstract:Contrastive learning (CL) continuously achieves significant breakthroughs across multiple domains. However, the most common InfoNCE-based methods suffer from some dilemmas, such as \textit{uniformity-tolerance dilemma} (UTD) and \textit{gradient reduction}, both of which are related to a $\mathcal{P}_{ij}$ term. It has been identified that UTD can lead to unexpected performance degradation. We argue that the fixity of temperature is to blame for UTD. To tackle this challenge, we enrich the CL loss family by presenting a Model-Aware Contrastive Learning (MACL) strategy, whose temperature is adaptive to the magnitude of alignment that reflects the basic confidence of the instance discrimination task, then enables CL loss to adjust the penalty strength for hard negatives adaptively. Regarding another dilemma, the gradient reduction issue, we derive the limits of an involved gradient scaling factor, which allows us to explain from a unified perspective why some recent approaches are effective with fewer negative samples, and summarily present a gradient reweighting to escape this dilemma. Extensive remarkable empirical results in vision, sentence, and graph modality validate our approach's general improvement for representation learning and downstream tasks.

Machine Learning
Learning the Unlearned: Mitigating Feature Suppression in Contrastive Learning

Jihai Zhang,Xiang Lan,Xiaoye Qu,Yu Cheng,Mengling Feng,Bryan Hooi

2024-07-15

Abstract:Self-Supervised Contrastive Learning has proven effective in deriving high-quality representations from unlabeled data. However, a major challenge that hinders both unimodal and multimodal contrastive learning is feature suppression, a phenomenon where the trained model captures only a limited portion of the information from the input data while overlooking other potentially valuable content. This issue often leads to indistinguishable representations for visually similar but semantically different inputs, adversely affecting downstream task performance, particularly those requiring rigorous semantic comprehension. To address this challenge, we propose a novel model-agnostic Multistage Contrastive Learning (MCL) framework. Unlike standard contrastive learning which inherently captures one single biased feature distribution, MCL progressively learns previously unlearned features through feature-aware negative sampling at each stage, where the negative samples of an anchor are exclusively selected from the cluster it was assigned to in preceding stages. Meanwhile, MCL preserves the previously well-learned features by cross-stage representation integration, integrating features across all stages to form final representations. Our comprehensive evaluation demonstrates MCL's effectiveness and superiority across both unimodal and multimodal contrastive learning, spanning a range of model architectures from ResNet to Vision Transformers (ViT). Remarkably, in tasks where the original CLIP model has shown limitations, MCL dramatically enhances performance, with improvements up to threefold on specific attributes in the recently proposed MMVP benchmark.

Computer Vision and Pattern Recognition,Machine Learning
Adversarial Contrastive Learning via Asymmetric InfoNCE.

Qiying Yu,Jieming Lou,Xianyuan Zhan,Qizhang Li,Wangmeng Zuo,Yang Liu,Jingjing Liu

DOI: https://doi.org/10.1007/978-3-031-20065-6_4

2022-01-01

Abstract:Contrastive learning (CL) has recently been applied to adversarial learning tasks. Such practice considers adversarial samples as additional positive views of an instance, and by maximizing their agreements with each other, yields better adversarial robustness. However, this mechanism can be potentially flawed, since adversarial perturbations may cause instance-level identity confusion, which can impede CL performance by pulling together different instances with separate identities. To address this issue, we propose to treat adversarial samples unequally when contrasted, with an asymmetric InfoNCE objective (A-InfoNCE) that allows discriminating considerations of adversarial samples. Specifically, adversaries are viewed as inferior positives that induce weaker learning signals, or as hard negatives exhibiting higher contrast to other negative samples. In the asymmetric fashion, the adverse impacts of conflicting objectives between CL and adversarial learning can be effectively mitigated. Experiments show that our approach consistently outperforms existing Adversarial CL methods across different finetuning schemes without additional computational cost. The proposed A-InfoNCE is also a generic form that can be readily extended to other CL methods. Code is available at https://github.com/yqy2001/A-InfoNCE.
Multi-label Contrastive Predictive Coding

Jiaming Song,Stefano Ermon

DOI: https://doi.org/10.48550/arXiv.2007.09852

2020-12-03

Abstract:Variational mutual information (MI) estimators are widely used in unsupervised representation learning methods such as contrastive predictive coding (CPC). A lower bound on MI can be obtained from a multi-class classification problem, where a critic attempts to distinguish a positive sample drawn from the underlying joint distribution from $(m-1)$ negative samples drawn from a suitable proposal distribution. Using this approach, MI estimates are bounded above by $\log m$, and could thus severely underestimate unless $m$ is very large. To overcome this limitation, we introduce a novel estimator based on a multi-label classification problem, where the critic needs to jointly identify multiple positive samples at the same time. We show that using the same amount of negative samples, multi-label CPC is able to exceed the $\log m$ bound, while still being a valid lower bound of mutual information. We demonstrate that the proposed approach is able to lead to better mutual information estimation, gain empirical improvements in unsupervised representation learning, and beat a current state-of-the-art knowledge distillation method over 10 out of 13 tasks.

Machine Learning
What makes for good views for contrastive learning

Ben Poole,Phillip Isola,Yonglong Tian,Chen Sun,Dilip Krishnan,C. Schmid

2020-05-20

Abstract:Contrastive learning between multiple views of the data has recently achieved state of the art performance in the field of self-supervised representation learning. Despite its success, the influence of different view choices has been less studied. In this paper, we use empirical analysis to better understand the importance of view selection, and argue that we should reduce the mutual information (MI) between views while keeping task-relevant information intact. To verify this hypothesis, we devise unsupervised and semi-supervised frameworks that learn effective views by aiming to reduce their MI. We also consider data augmentation as a way to reduce MI, and show that increasing data augmentation indeed leads to decreasing MI and improves downstream classification accuracy. As a by-product, we also achieve a new state-of-the-art accuracy on unsupervised pre-training for ImageNet classification ($73\%$ top-1 linear readoff with a ResNet-50). In addition, transferring our models to PASCAL VOC object detection and COCO instance segmentation consistently outperforms supervised pre-training. Code:this http URL

Computer Science
Towards Understanding the Mechanism of Contrastive Learning via Similarity Structure: A Theoretical Analysis

Hiroki Waida,Yuichiro Wada,Léo Andéol,Takumi Nakagawa,Yuhui Zhang,Takafumi Kanamori

2023-04-02

Abstract:Contrastive learning is an efficient approach to self-supervised representation learning. Although recent studies have made progress in the theoretical understanding of contrastive learning, the investigation of how to characterize the clusters of the learned representations is still limited. In this paper, we aim to elucidate the characterization from theoretical perspectives. To this end, we consider a kernel-based contrastive learning framework termed Kernel Contrastive Learning (KCL), where kernel functions play an important role when applying our theoretical results to other frameworks. We introduce a formulation of the similarity structure of learned representations by utilizing a statistical dependency viewpoint. We investigate the theoretical properties of the kernel-based contrastive loss via this formulation. We first prove that the formulation characterizes the structure of representations learned with the kernel-based contrastive learning framework. We show a new upper bound of the classification error of a downstream task, which explains that our theory is consistent with the empirical success of contrastive learning. We also establish a generalization error bound of KCL. Finally, we show a guarantee for the generalization ability of KCL to the downstream classification task via a surrogate bound.

Machine Learning
Mutual Information Driven Equivariant Contrastive Learning for 3D Action Representation Learning

Lilang Lin,Jiahang Zhang,Jiaying Liu

DOI: https://doi.org/10.1109/tip.2024.3372451

IF: 10.6

2024-03-15

IEEE Transactions on Image Processing

Abstract:Self-supervised contrastive learning has proven to be successful for skeleton-based action recognition. For contrastive learning, data transformations are found to fundamentally affect the learned representation quality. However, traditional invariant contrastive learning is detrimental to the performance on the downstream task if the transformation carries important information for the task. In this sense, it limits the application of many data transformations in the current contrastive learning pipeline. To address these issues, we propose to utilize equivariant contrastive learning, which extends invariant contrastive learning and preserves important information. By integrating equivariant and invariant contrastive learning into a hybrid approach, the model can better leverage the motion patterns exposed by data transformations and obtain a more discriminative representation space. Specifically, a self-distillation loss is first proposed for transformed data of different intensities to fully utilize invariant transformations, especially strong invariant transformations. For equivariant transformations, we explore the potential of skeleton mixing and temporal shuffling for equivariant contrastive learning. Meanwhile, we analyze the impacts of different data transformations on the feature space in terms of two novel metrics proposed in this paper, namely, consistency and diversity. In particular, we demonstrate that equivariant learning boosts performance by alleviating the dimensional collapse problem. Experimental results on several benchmarks indicate that our method outperforms existing state-of-the-art methods.

computer science, artificial intelligence,engineering, electrical & electronic

Pharmacological characterization of phenytoin-resistant amygdala-kindled rats, a new model of drug-resistant partial epilepsy

Leveraging Superfluous Information in Contrastive Representation Learning

Exploiting Pseudo Image Captions for Multimodal Summarization.

A Mutual Information Perspective on Federated Contrastive Learning

Long-Tailed Recognition by Mutual Information Maximization between Latent Features and Ground-Truth Labels

Robust Contrastive Learning against Noisy Views

Separating common from salient patterns with Contrastive Representation Learning

Vision Language Pre-training by Contrastive Learning with Cross-Modal Similarity Regulation

$f$-MICL: Understanding and Generalizing InfoNCE-based Contrastive Learning

Rethinking Minimal Sufficient Representation in Contrastive Learning

A Unified Framework for Contrastive Learning from a Perspective of Affinity Matrix

Graph contrastive learning with min-max mutual information

Contrastive Multimodal Fusion with TupleInfoNCE

On the Importance of Contrastive Loss in Multimodal Learning

Model-Aware Contrastive Learning: Towards Escaping the Dilemmas

Learning the Unlearned: Mitigating Feature Suppression in Contrastive Learning

Adversarial Contrastive Learning via Asymmetric InfoNCE.

Multi-label Contrastive Predictive Coding

What makes for good views for contrastive learning

Towards Understanding the Mechanism of Contrastive Learning via Similarity Structure: A Theoretical Analysis

Mutual Information Driven Equivariant Contrastive Learning for 3D Action Representation Learning