Abstract:Linear sequences of words are implicitly represented in our brains by hierarchical structures that organize the composition of words in sentences. Linguists formalize different frameworks to model this hierarchy; two of the most common syntactic frameworks are Constituency and Dependency. Constituency represents sentences as nested groups of phrases, while dependency represents a sentence by assigning relations between its words. Recently, the pursuit of intelligent machines has produced Language Models (LMs) capable of solving many language tasks with a human-level performance. Many studies now question whether LMs implicitly represent syntactic hierarchies. This thesis focuses on producing constituency and dependency structures from LMs in an unsupervised setting. I review the critical methods in this field and highlight a line of work that utilizes a numerical representation for binary constituency trees (Syntactic Distance). I present a detailed study on StructFormer (SF) (Shen et al., 2021), which retrofits a transformer encoder architecture with a parser network to produce constituency and dependency structures. I present six experiments to analyze and address this field's challenges; experiments include investigating the effect of repositioning the parser network within the SF architecture, evaluating subword-based induced trees, and benchmarking the models developed in the thesis experiments on linguistic tasks. Models benchmarking is performed by participating in the BabyLM challenge, published at CoNLL 2023 (Momen et al., 2023). The results of this thesis encourage further development in the direction of retrofitting transformer-based models to induce syntactic structures, supported by the acceptable performance of SF in different experimental settings and the observed limitations that require innovative solutions to advance the state of syntactic structure induction.

Dependency networks: A discussion note about Lamb’s theory of linguistic structure

How Do Local Syntactic Structures Influence Global Properties in Language Networks?

Dependency Distance: A New Perspective on Syntactic Patterns in Natural Languages.

Bare Phrase Structure, Label-Less Trees, and Specifier-Less Syntax. is Minimalism Becoming a Dependency Grammar?

The Dependency Status of Function Words: Auxiliaries.

Syntactic Networks of Interlanguage Across L2 Modalities and Proficiency Levels.

Coordination and Referential Dependencies: a Dependency Grammar Account in Terms of Predicate-Valent Structures

The Status of Function Words in Dependency Grammar: A Critique of Universal Dependencies (UD)

DDM at Work: Reply to comments on “Dependency distance: A new perspective on syntactic patterns in natural languages”

Linguistic Networks:Metaphor or Tool?

Linguistic Structure Induction from Language Models

What Role Does Syntax Play in a Language Network?

Diagnostics for Constituents: Dependency, Constituency, and the Status of Function Words.

The Role of Working Memory in Shaping Syntactic Dependency Structures

Beyond the Constituent - A Dependency Grammar Analysis of Chains

Dependency direction as a means of word-order typology: A method based on dependency treebanks

The Complexity of Chinese Syntactic Dependency Networks

Can syntactic networks indicate morphological complexity of a language?

A Historical Overview of the Status of Function Words in Dependency Grammar.

Dependency Network Approach to the Oral Production of English- and Chinese-speaking Healthy Older Adults

Dependency Distance As A Metric of Language Comprehension Difficulty