Abstract:Based on the psychological and physiological fact that humans employ a visual attention mechanism to connect perception and action by selecting the relevant parts of the environment in an unconscious or conscious way and using the relevant parts to produce an appropriate action, this thesis presents a cognitive visual perception paradigm that determines how visual inputs reach awareness and guide actions. Based on the idea that a general way of organizing the visual scene is to parcel it into discrete objects, object-based visual attention theory is employed in the proposed paradigm. This proposed paradigm models robotic visual perception as a three-stage process: pre-attentive processing, attentional selection and post-attentive perception. It indicates that robotic visual perception starts from a low-level cognitive attentional selection procedure that guides attention to the relevant object of the scene, followed by a high-level post-attentive analysis procedure that analyzes the attended object and formulates it into an internal mental representation used for further cognitive behaviors. The pre-attentive processing stage extracts pre-attentive features and divides the input scene into uniform proto-objects by using an irregular pyramid based segmentation method. The attentional selection stage guides attention to one proto-object of interest by means of unconscious bottom-up competition and conscious top-down biasing. The bottom-up competition is modeled by estimating the saliency of each proto-object. The top-down biasing is modeled by using integrated competition hypothesis: by directing attention to a task-relevant feature of an object, a competitive advantage over the whole object is produced. Furthermore, this thesis asserts that the task-relevant feature can be autonomously deduced from the internal representation of the task-relevant object that is specified by or inferred from the current task.Once a proto-object is selected by attention, it proceeds to the post-attentive perception stage, which includes perceptual completion processing, extraction of post-attentive features, object recognition, and development of the internal representation of the attended object in long-term memory. The internal representation is autonomously organized and learned under the framework of probabilistic neural networks in the sense that an object is modeled as a hierarchical cluster. Thus, each instance in the cluster can be abstracted as a mental state that can be used for high-level cognitive behaviors, such as attentional prediction and action determination. This proposed cognitive visual perception paradigm is applied into distinct robotic tasks, including detection of salient objects, detection of task-relevant objects and target tracking. Experimental results under different conditions are shown to validate this paradigm.

Sensorimotor Visual Perception on Embodied System Using Free Energy Principle

Sensorimotor learning for artificial body perception

Modelling Human Visual Motion Processing with Trainable Motion Energy Sensing and a Self-attention Network

Active Vision for Robot Manipulators Using the Free Energy Principle

The free energy principle for action and perception: A mathematical review

Investigating the impact of free energy based behavior on human in human-agent interaction

Embodied Visual Recognition

Bayesian mechanics of perceptual inference and motor control in the brain

Adaptive robot body learning and estimation through predictive coding

Vision-Motor Abstraction toward Robot Cognition

Active inference body perception and action for humanoid robots

Artificial Visual Perception Nervous System Based on Low-Dimensional Material Photoelectric Memristors

Enacted Visual Perception: A Computational Model based on Piaget Equilibrium

A System View of the Recognition and Interpretation of Observed Human Shape, Pose and Action

Cognitive Visual Perception Mechanism for Robots Using Object-Based Visual Attention

Cognitive vision: The case for embodied perception

Free Energy Projective Simulation (FEPS): Active inference with interpretability

End-to-End Pixel-Based Deep Active Inference for Body Perception and Action

Vision-sensorimotor Abstraction and Imagination Towards Exploring Robot’s Inner World

Intrinsically Motivated Learning of Visual Motion Perception and Smooth Pursuit