Abstract:Perception is a complex process that involves brain activities at different levels. The availability of models for the representation and interpretation of the sensory information opens up new research avenues that cut across neuroscience, imaging, information engineering, and modern robotics. The goal of the multidisciplinary field of perceptual signal processing is to identify the features of the stimuli that determine their “perception,” namely “a single unified awareness derived from sensory processes while a stimulus is present,” and to establish associated computational models that can be generalized and exploited for designing a human-centered approach to imaging. In the case of vision, the stimuli go through a complex analysis chain of visual pathways, starting with the encoding by the photoreceptors in the retina (low-level processing) and ending with cognitive mechanisms (high-level processes) that depend on the task being performed. Accordingly, low-level models are concerned with image representation and aim at emulating the way that the visual stimulus is encoded by the early stages of the visual system, as well as at capturing the varying sensitivity to the features of the input stimuli, whereas high-level models are related to image interpretation and allow to predict the performance of a human observer in a predefined task. A global model that accounts for both such bottom-up and top-down approaches would enable an automatic interpretation of the visual stimuli based on both low-level features and semantic contents. In image processing, methods that take advantage of such models include feature extraction, content-based image description and retrieval, model-based coding, and the emergent domain of medical image perception. This special issue gives a flavor of the scope and potential of perception-based image and video processing by providing an overview of the way that visual mechanisms at different levels can be modeled and exploited. In particular, the eleven selected papers span the following fields:

Introduction to the Issue on Perception-Driven 360° Video Processing

State-of-the-Art in 360° Video/Image Processing: Perception, Assessment and Compression

A Survey on 360° Images and Videos in Mixed Reality: Algorithms and Applications

360-Degree Video Super Resolution and Quality Enhancement Challenge: Methods and Results

A Survey on 360° Video Streaming

A Survey on 360° Video Streaming: Acquisition, Transmission, and Display

360-Degree Video Streaming: A Survey of the State of the Art

Editorial: image perception

Introduction to the Issue on Emerging Technologies for Video Compression

Visual Distortions in 360-degree Videos

Parallax 360 : Stereoscopic 360 ° Scene

Perceptual Analysis of Perspective Projection for Viewport Rendering in 360° Images

Parallax360: Stereoscopic 360° Scene Representation for Head-Motion Parallax.

Cognitive Analysis of 360 degree Surround Photos

Image-Based Scene Representations for Head-Motion Parallax in 360° Panoramas

VMP360

From Capture to Display: A Survey on Volumetric Video

Editorial IEEE Transactions on Multimedia Special Section on Video Analytics: Challenges, Algorithms, and Applications.

Quality Assessment of Stereoscopic 360-degree Images from Multi-viewports

Probabilistic Viewport Adaptive Streaming for 360-Degree Videos