Abstract:Perceptual learning enables humans to recognize and represent stimuli invariant to various transformations and build a consistent representation of the self and physical world. Such representations preserve the invariant physical relations among the multiple perceived sensory cues. This work is an attempt to exploit these principles in an engineered system. We design a novel neural network architecture capable of learning, in an unsupervised manner, relations among multiple sensory cues. The system combines computational principles, such as competition, cooperation, and correlation, in a neurally plausible computational substrate. It achieves that through a parallel and distributed processing architecture in which the relations among the multiple sensory quantities are extracted from time-sequenced data. We describe the core system functionality when learning arbitrary non-linear relations in low-dimensional sensory data. Here, an initial benefit rises from the fact that such a network can be engineered in a relatively straightforward way without prior information about the sensors and their interactions. Moreover, alleviating the need for tedious modelling and parametrization, the network converges to a consistent description of any arbitrary high-dimensional multisensory setup. We demonstrate this through a real-world learning problem, where, from standard RGB camera frames, the network learns the relations between physical quantities such as light intensity, spatial gradient, and optical flow, describing a visual scene. Overall, the benefits of such a framework lie in the capability to learn non-linear pairwise relations among sensory streams in an architecture that is stable under noise and missing sensor input.

A Perception-Inspired Deep Learning Framework for Predicting Perceptual Texture Similarity

Perception Driven Texture Generation

Perception-Aware Texture Similarity Prediction

Human Visual Perception Based Image Quality Assessment for Video Prediction

A Depth Estimation Framework Based on Unsupervised Learning and Cross-Modal Translation

Deep Multiple-Attribute-Perceived Network For Real-World Texture Recognition

A sketch-texture retrieval framework using perceptual similarity

A high-throughput approach for the efficient prediction of perceived similarity of natural objects

Transfer of View-manifold Learning to Similarity Perception of Novel Objects

Learning to Generate Images with Perceptual Similarity Metrics

Modelling Spatial Correlations by Using Deep CNN and LSTM for Texture Image Classification

Deep Learning for Content-Based Image Retrieval: A Comprehensive Study

Engineering Deep Representations for Modeling Aesthetic Perception

A Comparison of Statistical Learning of Naturalistic Textures Between DCNNs and the Human Visual Hierarchy

A Human-Like Siamese-Based Visual-Tactile Fusion Model for Object Recognition

Deep Unsupervised Learning of Visual Similarities

Color Patterns And Enhanced Texture Learning For Detecting Computer-Generated Images

A Framework for Learning Invariant Physical Relations in Multimodal Sensory Processing

Enhancing texture representation with deep tracing pattern encoding

Texture-guided Coding for Deep Features

DeepDC: Deep Distance Correlation as a Perceptual Image Quality Evaluator