Abstract:Establishing the relationships among hierarchical visual attributes of objects in the visual world is crucial for human cognition. The classic convolution neural network (CNN) can successfully extract hierarchical features but ignore the relationships among features, resulting in shortcomings compared to humans in areas like interpretability and domain generalization. Recently, algorithms have introduced feature relationships by external prior knowledge and special auxiliary modules, which have been proven to bring multiple improvements in many computer vision tasks. However, prior knowledge is often difficult to obtain, and auxiliary modules bring additional consumption of computing and storage resources, which limits the flexibility and practicality of the algorithm. In this paper, we aim to drive the CNN model to learn the relationships among hierarchical deep features without prior knowledge and consumption increasing, while enhancing the fundamental performance of some aspects. Firstly, the task of learning the relationships among hierarchical features in CNN is defined and three key problems related to this task are pointed out, including the quantitative metric of connection intensity, the threshold of useless connections, and the updating strategy of relation graph. Secondly, Relational Embedding Convolution (RE-Conv) layer is proposed for the representation of feature relationships in convolution layer, followed by a scheme called use & disuse strategy which aims to address the three problems of feature relation learning. Finally, the improvements brought by the proposed feature relation learning scheme have been demonstrated through numerous experiments, including interpretability, domain generalization, noise robustness, and inference efficiency. In particular, the proposed scheme outperforms many state-of-the-art methods in the domain generalization community and can be seamlessly integrated with existing methods for further improvement. Meanwhile, it maintains comparable precision to the original CNN model while reducing floating point operations (FLOPs) by approximately 50%.

Evaluating the progress of deep learning for visual relational concepts

Visual relationship detection with a deep convolutional relationship network

Visual Reasoning in Object-Centric Deep Neural Networks: A Comparative Cognition Approach

Evaluating (and Improving) the Correspondence Between Deep Neural Networks and Human Representations

Seeing eye-to-eye? A comparison of object recognition performance in humans and deep convolutional neural networks under image manipulation

Deep Learning for Content-Based Image Retrieval: A Comprehensive Study

Detecting Visual Relationships with Deep Relational Networks

Research on image classification leveraging deep convolutional neural networks and visual cognition

Relational inductive biases, deep learning, and graph networks

Towards Human-Like Machine Comprehension: Few-Shot Relational Learning in Visually-Rich Documents

Spatial relation categorization in infants and deep neural networks

RelNN: A Deep Neural Model for Relational Learning

Learning Hierarchical Relational Representations through Relational Convolutions

Improved Relation Classification by Deep Recurrent Neural Networks with Data Augmentation

Relational Deep Learning: Graph Representation Learning on Relational Databases

Deep Neural Networks Can Learn Generalizable Same-Different Visual Relations

Understanding the computational demands underlying visual reasoning

Relational recurrent neural networks

Computer Vision : History , the Rise of Deep Networks , and Future Vistas Panel on Perception and Cognition , MORS Meeting on Artificial Intelligence and Autonomy

Relational Self-Attention: What's Missing in Attention for Video Understanding

Learning feature relationships in CNN model via relational embedding convolution layer