Abstract:Person Re-identification (Re-ID) aims to match person images across non-overlapping cameras. The existing approaches formulate this task as fine-grained representation learning with deep neural networks, which involves extracting image features using a deep convolutional network, followed by mapping the features into a discriminative space through another smaller network, in order to make full use of all possible cues. However, recent Re-ID methods that strive to capture every cue and make the space more discriminative have resulted in longer features, ranging from 1024 to 14336, leading to higher time (distance computation) and space (feature storage) complexities. There are two potential solutions: reduction-after-training methods (such as Principal Component Analysis and Linear Discriminant Analysis) and reduction-during-training methods (such as 1 × 1 Convolution). The former utilizes a statistical approach aiming for a global optimum but lacking end-to-end optimization of large data and deep neural networks. The latter lacks theoretical guarantees and may be vulnerable to training noise such as dataset noise or initialization seed. To address these limitations, we propose a method called Euclidean-Distance-Preserving Feature Reduction (EDPFR) that combines the strengths of both reduction-after-training and reduction-during-training methods. EDPFR first formulates the feature reduction process as a matrix decomposition and derives a condition to preserve the Euclidean distance between features, thus ensuring accuracy in theory. Furthermore, the method integrates the matrix decomposition process into a deep neural network to enable end-to-end optimization and batch training, while maintaining the theoretical guarantee. The result of the EDPFR is a reduction of the feature dimensions from fa and fb to fa' and fb', while preserving their Euclidean distance, i.e.L2(fa,fb)=L2(fa',fb'). In addition to its Euclidean-Distance-Preserving capability, EDPFR also features a novel feature-level distillation loss. One of the main challenges in knowledge distillation is dimension mismatch. While previous distillation losses, usually project the mismatched features to matched class-level, spatial-level, or similarity-level spaces, this can result in a loss of information and decrease the flexibility and efficiency of distillation. Our proposed feature-level distillation leverages the benefits of the Euclidean-Distance-Preserving property and performs distillation directly in the feature space, resulting in a more flexible and efficient approach. Extensive on three Re-ID datasets, Market-1501, DukeMTMC-reID and MSMT demonstrate the effectiveness of our proposed Euclidean-Distance-Preserving Feature Reduction.

Partial Feature Decorrelation for Non-I.I.D Image classification

Out-of-distribution Generalization via Partial Feature Decorrelation

On Feature Decorrelation in Self-Supervised Learning

Decompose Learning: Combine Feature Extraction and Classification

FDN: Feature Decoupling Network for Head Pose Estimation.

FediOS: Decoupling Orthogonal Subspaces for Personalization in Feature-skew Federated Learning

DECOR: Dynamic Decoupling and Multi-Objective Optimization for Long-tailed Remote Sensing Image Classification

DECOR: Dynamic Decoupling and Multiobjective Optimization for Long-Tailed Remote Sensing Image Classification

Residual Deep PCA-based Feature Extraction for Hyperspectral Image Classification

Calibrated Feature Decomposition for Generalizable Person Re-Identification

Tolerant Self-Distillation for Image Classification

Interesting Receptive Region and Feature Excitation for Partial Person Re-identification

Long-Range Feature Dependencies Capturing for Low-Resolution Image Classification

Disentangling the intrinsic feature from the related feature in image classification using knowledge distillation and object replacement

Partial Knowledge Distillation for Alleviating the Inherent Inter-Class Discrepancy in Federated Learning

Euclidean-Distance-Preserved Feature Reduction for efficient person re-identification

DKD-pFed: A novel framework for personalized federated learning via decoupling knowledge distillation and feature decorrelation

Learning clustering-friendly representations via partial information discrimination and cross-level interaction

Deep Intrinsic Decomposition With Adversarial Learning for Hyperspectral Image Classification

A novel regularization method for decorrelation learning of non-parallel hyperplanes

DGPF-RENet: A Low Data Dependence Network with Low Training Iterations for Hyperspectral Image Classification