Abstract:This technical report is an appendix to Eisner (1996): it gives superior experimental results that were reported only in the talk version of that paper. Eisner (1996) trained three probability models on a small set of about 4,000 conjunction-free, dependency-grammar parses derived from the Wall Street Journal section of the Penn Treebank, and then evaluated the models on a held-out test set, using a novel O(n^3) parsing algorithm. The present paper describes some details of the experiments and repeats them with a larger training set of 25,000 sentences. As reported at the talk, the more extensive training yields greatly improved performance. Nearly half the sentences are parsed with no misattachments; two-thirds are parsed with at most one misattachment. Of the models described in the original written paper, the best score is still obtained with the generative (top-down) "model C." However, slightly better models are also explored, in particular, two variants on the comprehension (bottom-up) "model B." The better of these has an attachment accuracy of 90%, and (unlike model C) tags words more accurately than the comparable trigram tagger. Differences are statistically significant. If tags are roughly known in advance, search error is all but eliminated and the new model attains an attachment accuracy of 93%. We find that the parser of Collins (1996), when combined with a highly-trained tagger, also achieves 93% when trained and tested on the same sentences. Similarities and differences are discussed.

Exploiting auxiliary distributions in stochastic unification-based grammars

Estimators for Stochastic ``Unification-Based'' Grammars

On quantum coding for ensembles of mixed states

The Status of Function Words in Dependency Grammar: A Critique of Universal Dependencies (UD)

Monte Carlo Syntax Marginals for Exploring and Using Dependency Parses

Evaluation of an Algorithmic‐Level Left‐Corner Parsing Account of Surprisal Effects

Learning Grammar With Explicit Annotations For Subordinating Conjunctions

Exploiting limited data for parsing

Improving Unsupervised Constituency Parsing via Maximizing Semantic Information

An Empirical Comparison of Probability Models for Dependency Grammar

Combining Generative and Discriminative Approaches to Unsupervised Dependency Parsing via Dual Decomposition

Structural Optimization Ambiguity and Simplicity Bias in Unsupervised Neural Grammar Induction

Learning of Structurally Unambiguous Probabilistic Grammars

On Eliciting Syntax from Language Models via Hashing

A Distributional Analogue to the Successor Representation

Unused information in token probability distribution of generative LLM: improving LLM reading comprehension through calculation of expected values

Probabilistic Programs with Stochastic Conditioning

Unsupervised Structure Learning of Stochastic And-Or Grammars

Expectation-based syntactic comprehension

Dynamic Syntax Mapping: A New Approach to Unsupervised Syntax Parsing

Stochastic And-Or Grammars: A Unified Framework and Logic Perspective