Abstract:In the last few years we have seen a growing interest in machine learning approaches to computer vision and, especially, to semantic labeling. Nowadays state of the art systems use deep learning on millions of labeled images with very successful results on benchmarks, though it is unlikely to expect similar results in unrestricted visual environments. Most learning schemes essentially ignore the inherent sequential structure of videos: this might be a critical issue, since any visual recognition process is remarkably more complex when shuffling video frames. Based on this remark, we propose a re-foundation of the communication protocol between visual agents and the environment, which is referred to as learning to see like children. Like for human interaction, visual concepts are acquired by the agents solely by processing their own visual stream along with human supervisions on selected pixels. We give a proof of concept that remarkable semantic labeling can emerge within this protocol by using only a few supervised examples. This is made possible by exploiting a constraint of motion coherent labeling that virtually offers tons of supervisions. Additional visual constraints, including those associated with object supervisions, are used within the context of learning from constraints. The framework is extended in the direction of lifelong learning, so as our visual agents live in their own visual environment without distinguishing learning and test set. Learning takes place in deep architectures under a progressive developmental scheme. In order to evaluate our Developmental Visual Agents (DVAs), in addition to classic benchmarks, we open the doors of our lab, allowing people to evaluate DVAs by crowd-sourcing. Such assessment mechanism might result in a paradigm shift in methodologies and algorithms for computer vision, encouraging truly novel solutions within the proposed framework.

Predicting Word Learning in Children from the Performance of Computer Vision Systems

A Computational Model of Early Word Learning from the Infant's Point of View

Fast and robust visual object recognition in young children

Visually grounded learning of keyword prediction from untranscribed speech

A model of early word acquisition based on realistic-scale audiovisual naming events

Machine Learning Accurately Classifies Age of Toddlers Based on Eye Tracking

Embodied vision for learning object representations

Learning to see like children: proof of concept

Analyzing the Roles of Language and Vision in Learning from Limited Data

Human-like Visual Learning and Reasoning

Learning high-level visual representations from a child's perspective without strong inductive biases

Signatures of cross-modal alignment in children's early concepts

Write a Classifier: Predicting Visual Classifiers from Unstructured Text

Impact of early visual experience on later usage of color cues

Learning the meanings of function words from grounded language using a visual question answering model

Deep Predictive Learning in Neocortex and Pulvinar

Quantifying the Roles of Visual, Linguistic, and Visual-Linguistic Complexity in Verb Acquisition

Grounded language acquisition through the eyes and ears of a single child

Computer vision analysis for children's social play classification in peer-play scenarios.