Abstract:How do humans learn language, and can the first language be learned at all? These fundamental questions are still hotly debated. In contemporary linguistics, there are two major schools of thought that give completely opposite answers. According to Chomsky's theory of universal grammar, language cannot be learned because children are not exposed to sufficient data in their linguistic environment. In contrast, usage-based models of language assume a profound relationship between language structure and language use. In particular, contextual mental processing and mental representations are assumed to have the cognitive capacity to capture the complexity of actual language use at all levels. The prime example is syntax, i.e., the rules by which words are assembled into larger units such as sentences. Typically, syntactic rules are expressed as sequences of word classes. However, it remains unclear whether word classes are innate, as implied by universal grammar, or whether they emerge during language acquisition, as suggested by usage-based approaches. Here, we address this issue from a machine learning and natural language processing perspective. In particular, we trained an artificial deep neural network on predicting the next word, provided sequences of consecutive words as input. Subsequently, we analyzed the emerging activation patterns in the hidden layers of the neural network. Strikingly, we find that the internal representations of nine-word input sequences cluster according to the word class of the tenth word to be predicted as output, even though the neural network did not receive any explicit information about syntactic rules or word classes during training. This surprising result suggests, that also in the human brain, abstract representational categories such as word classes may naturally emerge as a consequence of predictive coding and processing during language acquisition.

Word Acquisition in Neural Language Models

The Influence of Linguistic Information on Cortical Tracking of Words

Cortical Tracking of Constituent Structure in Language Acquisition

From Babbling to Fluency: Evaluating the Evolution of Language Models in Terms of Human Language Acquisition

Language acquisition: do children and language models follow similar learning stages?

Prosodic Features from Large Corpora of Child-Directed Speech as Predictors of the Age of Acquisition of Words

Evaluating Neural Language Models as Cognitive Models of Language Acquisition

Towards a theory of how the structure of language is acquired by deep neural networks

What Artificial Neural Networks Can Tell Us About Human Language Acquisition

Word learning and the acquisition of syntactic--semantic overhypotheses

Word class representations spontaneously emerge in a deep neural network trained on next word prediction

Is Child-Directed Speech Effective Training Data for Language Models?

Second Language Acquisition of Neural Language Models

Human Inspired Progressive Alignment and Comparative Learning for Grounded Word Acquisition

A Language-agnostic Model of Child Language Acquisition

Advances in the computational study of language acquisition

A model of early word acquisition based on realistic-scale audiovisual naming events

Investigating Critical Period Effects in Language Acquisition through Neural Language Models

A computational model of early language acquisition from audiovisual experiences of young infants

A Neural Network Model of Lexical Competition during Infant Spoken Word Recognition

Finding Structure in One Child's Linguistic Experience