Primary liposarcoma of bone. Case report.

A. Addison,S. Payne

DOI: https://doi.org/10.2106/00004623-198264020-00029

1982-02-01

Abstract:

What problem does this paper attempt to address?

Rule-Based and Word-Level Statistics-Based Processing of Language: Insights from Neuroscience

Nai Ding,Lucia Melloni,Xing Tian,David Poeppel

DOI: https://doi.org/10.1080/23273798.2016.1215477

2016-01-01

Abstract:To flexibly convey meaning, the human language faculty iteratively combines smaller units such as words into larger structures such as phrases based on grammatical principles. During comprehension, however, it remains unclear how the brain encodes the relationship between words and combines them into phrases. One hypothesis is that internal grammatical principles governing language generation are also used to parse the hierarchical syntactic structure of spoken language. An alternative hypothesis suggests, in contrast, that decoding language during comprehension solely relies on statistical relationships between words or strings of words, that is, the N-gram statistics, and no hierarchical linguistic structures are constructed. Here, we briefly review distinctions between rule-based hierarchical models and statistics-based linear string models for comprehension. Recent neurolinguistic studies show that tracking of probabilistic relationships between words is not sufficient to explain cortical encoding of linguistic constituent structure and support the involvement of rule-based processing during language comprehension.
Learning to Predict Novel Noun-Noun Compounds

Prajit Dhar,Lonneke van der Plas

DOI: https://doi.org/10.48550/arXiv.1906.03634

2019-09-25

Abstract:We introduce temporally and contextually-aware models for the novel task of predicting unseen but plausible concepts, as conveyed by noun-noun compounds in a time-stamped corpus. We train compositional models on observed compounds, more specifically the composed distributed representations of their constituents across a time-stamped corpus, while giving it corrupted instances (where head or modifier are replaced by a random constituent) as negative evidence. The model captures generalisations over this data and learns what combinations give rise to plausible compounds and which ones do not. After training, we query the model for the plausibility of automatically generated novel combinations and verify whether the classifications are accurate. For our best model, we find that in around 85% of the cases, the novel compounds generated are attested in previously unseen data. An additional estimated 5% are plausible despite not being attested in the recent corpus, based on judgments from independent human raters.

Computation and Language
Music for the birds: effects of auditory enrichment on captive bird species.

Lindsey Robbins,Susan W. Margulis

DOI: https://doi.org/10.1002/zoo.21260

Zoo Biology

Abstract:With the increase of mixed species exhibits in zoos, targeting enrichment for individual species may be problematic. Often, mammals may be the primary targets of enrichment, yet other species that share their environment (such as birds) will unavoidably be exposed to the enrichment as well. The purpose of this study was to determine if (1) auditory stimuli designed for enrichment of primates influenced the behavior of captive birds in the zoo setting, and (2) if the specific type of auditory enrichment impacted bird behavior. Three different African bird species were observed at the Buffalo Zoo during exposure to natural sounds, classical music and rock music. The results revealed that the average frequency of flying in all three bird species increased with naturalistic sounds and decreased with rock music (F = 7.63, df = 3,6, P = 0.018); vocalizations for two of the three species (Superb Starlings and Mousebirds) increased (F = 18.61, df = 2,6, P = 0.0027) in response to all auditory stimuli, however one species (Lady Ross's Turacos) increased frequency of duetting only in response to rock music (X(2) = 18.5, df = 2, P < 0.0001). Auditory enrichment implemented for large mammals may influence behavior in non-target species as well, in this case leading to increased activity by birds.
Building Probabilistic Models for Natural Language

Stanley F. Chen

DOI: https://doi.org/10.48550/arXiv.cmp-lg/9606014

1996-06-11

Computation and Language

Abstract:In this thesis, we investigate three problems involving the probabilistic modeling of language: smoothing n-gram models, statistical grammar induction, and bilingual sentence alignment. These three problems employ models at three different levels of language; they involve word-based, constituent-based, and sentence-based models, respectively. We describe techniques for improving the modeling of language at each of these levels, and surpass the performance of existing algorithms for each problem. We approach the three problems using three different frameworks. We relate each of these frameworks to the Bayesian paradigm, and show why each framework used was appropriate for the given problem. Finally, we show how our research addresses two central issues in probabilistic modeling: the sparse data problem and the problem of inducing hidden structure.
Syntactic Complexity Development in the Writings of EFL Learners: Insights from a Dependency Syntactically-Annotated Corpus

Jingyang Jiang,Peng Bi,Haitao Liu

DOI: https://doi.org/10.1016/j.jslw.2019.100666

IF: 5.448

2019-01-01

Journal of Second Language Writing

Abstract:This study investigates the syntactic complexity, as measured by both large-grained and fine-grained measures, of 410 narrative writings across four writing proficiency levels written by beginner and intermediate L2 English learners. By exploring the differences in syntactic complexity between writings at different proficiency levels, the study is purposed to find out the measures that can best discriminate and predict writing proficiency. The L2 Syntactic Complexity Analyzer and the dependency syntactically-annotated corpus are used respectively to collect the data for large-grained and fine-grained measures. With regard to large-grained measures, it is found that students with higher writing proficiency tend to produce longer language units, more subordinate clauses, more coordinate clauses, and more noun phrases in their writings; mean length of T-unit, mean length of sentence, and dependent clauses per clause can better predict writing proficiency than other traditional large-grained measures. As for fine-grained measures, it is found that three types of subordinate clauses, that is, adverbial clauses, complement clauses and relative clauses, and two types of noun modifiers, that is, prepositional phrases and adjectival relative clauses, occur more frequently in the writings of more proficient learners; the frequency of compound nouns correlates negatively with writing proficiency.
Botulinum toxin a improves muscle spasms and rigidity in stiff‐person syndrome

R. Liguori,C. Cordivari,E. Lugaresi,P. Montagna

DOI: https://doi.org/10.1002/mds.870120636

IF: 9.698

1997-11-01

Movement Disorders

Abstract:We studied the effect of botulinum toxin A (BTA) on painful muscular spasms and rigidity in two bedridden patients with clinical, electrophysiologic, and immunologic evidence of stiff‐person syndrome. We injected BTA of saline solution into several limb muscles with both the rater and patient blinded to the order of the injections. A physician, unaware of the treatment order, used an objective rating scale for rigidity and spasm frequency scale and independently assessed the treatment results. BTA administration significantly reduced rigidity and stopped the spasms in all limbs. Following BTA injection on one side, the spasm frequency decreased bilaterally possibly because of the spread of hematogenous toxin.
Evaluating statistical language models as pragmatic reasoners

Benjamin Lipkin,Lionel Wong,Gabriel Grand,Joshua B Tenenbaum

2023-05-02

Abstract:The relationship between communicated language and intended meaning is often probabilistic and sensitive to context. Numerous strategies attempt to estimate such a mapping, often leveraging recursive Bayesian models of communication. In parallel, large language models (LLMs) have been increasingly applied to semantic parsing applications, tasked with inferring logical representations from natural language. While existing LLM explorations have been largely restricted to literal language use, in this work, we evaluate the capacity of LLMs to infer the meanings of pragmatic utterances. Specifically, we explore the case of threshold estimation on the gradable adjective ``strong'', contextually conditioned on a strength prior, then extended to composition with qualification, negation, polarity inversion, and class comparison. We find that LLMs can derive context-grounded, human-like distributions over the interpretations of several complex pragmatic utterances, yet struggle composing with negation. These results inform the inferential capacity of statistical language models, and their use in pragmatic and semantic parsing applications. All corresponding code is made publicly available (<a class="link-external link-https" href="https://github.com/benlipkin/probsem/tree/CogSci2023" rel="external noopener nofollow">this https URL</a>).

Computation and Language,Artificial Intelligence
Real-time implementation of synthetic aperture vector flow imaging on a consumer-level tablet

Tommaso Di Ianni,T. Kjeldsen,C. V. Hoyos,Jesper Mosegaard,J. Jensen

DOI: https://doi.org/10.1109/ultsym.2017.8092359

IUS

Abstract:In this work, a 2-D vector flow imaging (VFI) method based on synthetic aperture sequential beamforming (SASB) and directional transverse oscillation is implemented on a commercially available tablet. The SASB technique divides the beamforming process in two parts, whereby the required data rate between the probe and back-end can be reduced by a factor of 64 compared to conventional delay-and-sum focusing. The lowered data rate enables real-time wireless transfer for both B-mode and VFI data. In the present setup, element data were acquired from a straight vessel with the SARUS research scanner and processed by a first-stage beamformer in a fixed focus. The data were subsequently transferred to an HTC Nexus 9 tablet through an ASUS RT-AC68U Wi-Fi router to simulate a wireless probe. The second-stage beamforming of the B-mode and flow data and the velocity estimation were implemented on the tablet's built-in GPU (Nvidia Tegra K1) through the OpenGL ES 3.1 API. Real-time performance was achieved with rates up to 26 VFI frames per second (38 ms/frame) for concurrent processing and Wi-Fi transmission.
Word learning and the acquisition of syntactic--semantic overhypotheses

Jon Gauthier,Roger Levy,Joshua B. Tenenbaum

DOI: https://doi.org/10.48550/arXiv.1805.04988

2018-05-14

Computation and Language

Abstract:Children learning their first language face multiple problems of induction: how to learn the meanings of words, and how to build meaningful phrases from those words according to syntactic rules. We consider how children might solve these problems efficiently by solving them jointly, via a computational model that learns the syntax and semantics of multi-word utterances in a grounded reference game. We select a well-studied empirical case in which children are aware of patterns linking the syntactic and semantic properties of words --- that the properties picked out by base nouns tend to be related to shape, while prenominal adjectives tend to refer to other properties such as color. We show that children applying such inductive biases are accurately reflecting the statistics of child-directed speech, and that inducing similar biases in our computational model captures children's behavior in a classic adjective learning experiment. Our model incorporating such biases also demonstrates a clear data efficiency in learning, relative to a baseline model that learns without forming syntax-sensitive overhypotheses of word meaning. Thus solving a more complex joint inference problem may make the full problem of language acquisition easier, not harder.
Language Acquisition in Computers

Megan Belzner,Sean Colin-Ellerin,Jorge H. Roman

DOI: https://doi.org/10.48550/arXiv.1206.0042

2012-05-31

Computation and Language

Abstract:This project explores the nature of language acquisition in computers, guided by techniques similar to those used in children. While existing natural language processing methods are limited in scope and understanding, our system aims to gain an understanding of language from first principles and hence minimal initial input. The first portion of our system was implemented in Java and is focused on understanding the morphology of language using bigrams. We use frequency distributions and differences between them to define and distinguish languages. English and French texts were analyzed to determine a difference threshold of 55 before the texts are considered to be in different languages, and this threshold was verified using Spanish texts. The second portion of our system focuses on gaining an understanding of the syntax of a language using a recursive method. The program uses one of two possible methods to analyze given sentences based on either sentence patterns or surrounding words. Both methods have been implemented in C++. The program is able to understand the structure of simple sentences and learn new words. In addition, we have provided some suggestions regarding future work and potential extensions of the existing program.
Modelling constructivist language acquisition through syntactico-semantic pattern finding

Jonas Doumen,Katrien Beuls,Paul Van Eecke

DOI: https://doi.org/10.1098/rsos.231998

IF: 3.5

2024-07-26

Royal Society Open Science

Abstract:The constructivist acquisition of language by children has been elaborately documented by researchers in psycholinguistics and cognitive science. However, despite the centrality of human-like communication in the field of artificial intelligence, no faithful computational operationalizations of the mechanisms through which children learn language exist to date. In this article, we fill part of this void by introducing a mechanistic model of the constructivist acquisition of language through syntactico-semantic pattern finding. Concretely, we present a methodology for learning grammars based on similarities and differences in the form and meaning of linguistic observations alone. The resulting grammars consist of form-meaning mappings of variable extent and degree of abstraction, called constructions, which facilitate both language comprehension and production. Applying our methodology to the CLEVR benchmark dataset, we provide a proof of concept that demonstrates the online, incremental, data-efficient, transparent and effective learning of item-based construction grammars from utterance–meaning pairs.

multidisciplinary sciences
Linguistic Structure Induction from Language Models

Omar Momen

2024-03-12

Abstract:Linear sequences of words are implicitly represented in our brains by hierarchical structures that organize the composition of words in sentences. Linguists formalize different frameworks to model this hierarchy; two of the most common syntactic frameworks are Constituency and Dependency. Constituency represents sentences as nested groups of phrases, while dependency represents a sentence by assigning relations between its words. Recently, the pursuit of intelligent machines has produced Language Models (LMs) capable of solving many language tasks with a human-level performance. Many studies now question whether LMs implicitly represent syntactic hierarchies. This thesis focuses on producing constituency and dependency structures from LMs in an unsupervised setting. I review the critical methods in this field and highlight a line of work that utilizes a numerical representation for binary constituency trees (Syntactic Distance). I present a detailed study on StructFormer (SF) (Shen et al., 2021), which retrofits a transformer encoder architecture with a parser network to produce constituency and dependency structures. I present six experiments to analyze and address this field's challenges; experiments include investigating the effect of repositioning the parser network within the SF architecture, evaluating subword-based induced trees, and benchmarking the models developed in the thesis experiments on linguistic tasks. Models benchmarking is performed by participating in the BabyLM challenge, published at CoNLL 2023 (Momen et al., 2023). The results of this thesis encourage further development in the direction of retrofitting transformer-based models to induce syntactic structures, supported by the acceptable performance of SF in different experimental settings and the observed limitations that require innovative solutions to advance the state of syntactic structure induction.

Computation and Language,Artificial Intelligence
Explaining Large Language Model-Based Neural Semantic Parsers (Student Abstract)

Daking Rai,Yilun Zhou,Bailin Wang,Ziyu Yao

DOI: https://doi.org/10.48550/arXiv.2301.13820

2023-01-26

Abstract:While large language models (LLMs) have demonstrated strong capability in structured prediction tasks such as semantic parsing, few amounts of research have explored the underlying mechanisms of their success. Our work studies different methods for explaining an LLM-based semantic parser and qualitatively discusses the explained model behaviors, hoping to inspire future research toward better understanding them.

Computation and Language,Artificial Intelligence
Large Language Models as Neurolinguistic Subjects: Identifying Internal Representations for Form and Meaning

Linyang He,Ercong Nie,Helmut Schmid,Hinrich Schütze,Nima Mesgarani,Jonathan Brennan

2024-11-12

Abstract:This study investigates the linguistic understanding of Large Language Models (LLMs) regarding signifier (form) and signified (meaning) by distinguishing two LLM evaluation paradigms: psycholinguistic and neurolinguistic. Traditional psycholinguistic evaluations often reflect statistical biases that may misrepresent LLMs' true linguistic capabilities. We introduce a neurolinguistic approach, utilizing a novel method that combines minimal pair and diagnostic probing to analyze activation patterns across model layers. This method allows for a detailed examination of how LLMs represent form and meaning, and whether these representations are consistent across languages. Our contributions are three-fold: (1) We compare neurolinguistic and psycholinguistic methods, revealing distinct patterns in LLM assessment; (2) We demonstrate that LLMs exhibit higher competence in form compared to meaning, with the latter largely correlated to the former; (3) We present new conceptual minimal pair datasets for Chinese (COMPS-ZH) and German (COMPS-DE), complementing existing English datasets.

Computation and Language
Subspace Chronicles: How Linguistic Information Emerges, Shifts and Interacts during Language Model Training

Max Müller-Eberstein,Rob van der Goot,Barbara Plank,Ivan Titov

DOI: https://doi.org/10.48550/arXiv.2310.16484

2023-10-25

Abstract:Representational spaces learned via language modeling are fundamental to Natural Language Processing (NLP), however there has been limited understanding regarding how and when during training various types of linguistic information emerge and interact. Leveraging a novel information theoretic probing suite, which enables direct comparisons of not just task performance, but their representational subspaces, we analyze nine tasks covering syntax, semantics and reasoning, across 2M pre-training steps and five seeds. We identify critical learning phases across tasks and time, during which subspaces emerge, share information, and later disentangle to specialize. Across these phases, syntactic knowledge is acquired rapidly after 0.5% of full training. Continued performance improvements primarily stem from the acquisition of open-domain knowledge, while semantics and reasoning tasks benefit from later boosts to long-range contextualization and higher specialization. Measuring cross-task similarity further reveals that linguistically related tasks share information throughout training, and do so more during the critical phase of learning than before or after. Our findings have implications for model interpretability, multi-task learning, and learning from limited data.

Computation and Language
The probabilistic analysis of language acquisition: Theoretical, computational, and experimental analysis

Anne S. Hsu,Nick Chater,Paul M.B. Vitányi

DOI: https://doi.org/10.1016/j.cognition.2011.02.013

IF: 4.011

2011-09-01

Cognition

Abstract:There is much debate over the degree to which language learning is governed by innate language-specific biases, or acquired through cognition-general principles. Here we examine the probabilistic language acquisition hypothesis on three levels: We outline a novel theoretical result showing that it is possible to learn the exact generative model underlying a wide class of languages, purely from observing samples of the language. We then describe a recently proposed practical framework, which quantifies natural language learnability, allowing specific learnability predictions to be made for the first time. In previous work, this framework was used to make learnability predictions for a wide variety of linguistic constructions, for which learnability has been much debated. Here, we present a new experiment which tests these learnability predictions. We find that our experimental results support the possibility that these linguistic constructions are acquired probabilistically from cognition-general principles.

psychology, experimental
Augmented linguistic analysis skills: Machine translation and generative AI as pedagogical aids for analyzing complex English compounds

Rudy Loock,Benjamin Holt

DOI: https://doi.org/10.29140/tltl.v6n3.1489

2024-07-25

Technology in Language Teaching & Learning

Abstract:In this article, our aim is to assess the efficacy of machine translation tools and state-of-the-art generative AI platforms such as ChatGPT in fostering enhanced linguistic analysis skills among students in order to improve their understanding of the language. Our case study deals with the analysis of complex English compounds—a known challenge for French learners of English as a second language. Our investigation centers on students’ ability to identify the head noun within complex noun phrases, both in full sentences and in press titles. The study involved two distinct cohorts: students in their third and final year of an undergraduate program in applied foreign languages, and students in their second and final year of a master's program in professional translation. We evaluated the participants’ ability to identify head nouns—a necessary skill for comprehending complex noun phrases—under two conditions: (i) without the aid of any linguistic tools and (ii) with the assistance of machine translation outputs by a generic online translator. Subsequently, we explored the capabilities of advanced generative AI tools—in this instance ChatGPT—of correctly identifying head nouns. Our results show that students may benefit from the presence of machine translation outputs, albeit with varying degrees of success. Our experimentation with ChatGPT shows that, given appropriate prompts, the tool is capable of identifying head nouns, suggesting that generative AI tools may be a more effective tool in helping students analyze and understand complex noun phrases in English.
Grammar-based Game Description Generation using Large Language Models

Tsunehiko Tanaka,Edgar Simo-Serra

2024-07-25

Abstract:To lower the barriers to game design development, automated game design, which generates game designs through computational processes, has been explored. In automated game design, machine learning-based techniques such as evolutionary algorithms have achieved success. Benefiting from the remarkable advancements in deep learning, applications in computer vision and natural language processing have progressed in level generation. However, due to the limited amount of data in game design, the application of deep learning has been insufficient for tasks such as game description generation. To pioneer a new approach for handling limited data in automated game design, we focus on the in-context learning of large language models (LLMs). LLMs can capture the features of a task from a few demonstration examples and apply the capabilities acquired during pre-training. We introduce the grammar of game descriptions, which effectively structures the game design space, into the LLMs' reasoning process. Grammar helps LLMs capture the characteristics of the complex task of game description generation. Furthermore, we propose a decoding method that iteratively improves the generated output by leveraging the grammar. Our experiments demonstrate that this approach performs well in generating game descriptions.

Artificial Intelligence
Learning Language Structures through Grounding

Freda Shi

2024-10-22

Abstract:Language is highly structured, with syntactic and semantic structures, to some extent, agreed upon by speakers of the same language. With implicit or explicit awareness of such structures, humans can learn and use language efficiently and generalize to sentences that contain unseen words. Motivated by human language learning, in this dissertation, we consider a family of machine learning tasks that aim to learn language structures through grounding. We seek distant supervision from other data sources (i.e., grounds), including but not limited to other modalities (e.g., vision), execution results of programs, and other languages. We demonstrate the potential of this task formulation and advocate for its adoption through three schemes. In Part I, we consider learning syntactic parses through visual grounding. We propose the task of visually grounded grammar induction, present the first models to induce syntactic structures from visually grounded text and speech, and find that the visual grounding signals can help improve the parsing quality over language-only models. As a side contribution, we propose a novel evaluation metric that enables the evaluation of speech parsing without text or automatic speech recognition systems involved. In Part II, we propose two execution-aware methods to map sentences into corresponding semantic structures (i.e., programs), significantly improving compositional generalization and few-shot program synthesis. In Part III, we propose methods that learn language structures from annotations in other languages. Specifically, we propose a method that sets a new state of the art on cross-lingual word alignment. We then leverage the learned word alignments to improve the performance of zero-shot cross-lingual dependency parsing, by proposing a novel substructure-based projection method that preserves structural knowledge learned from the source language.

Computation and Language,Artificial Intelligence,Computer Vision and Pattern Recognition
Evaluating Neural Language Models as Cognitive Models of Language Acquisition

Héctor Javier Vázquez Martínez,Annika Lea Heuser,Charles Yang,Jordan Kodner

2023-10-31

Abstract:The success of neural language models (LMs) on many technological tasks has brought about their potential relevance as scientific theories of language despite some clear differences between LM training and child language acquisition. In this paper we argue that some of the most prominent benchmarks for evaluating the syntactic capacities of LMs may not be sufficiently rigorous. In particular, we show that the template-based benchmarks lack the structural diversity commonly found in the theoretical and psychological studies of language. When trained on small-scale data modeling child language acquisition, the LMs can be readily matched by simple baseline models. We advocate for the use of the readily available, carefully curated datasets that have been evaluated for gradient acceptability by large pools of native speakers and are designed to probe the structural basis of grammar specifically. On one such dataset, the LI-Adger dataset, LMs evaluate sentences in a way inconsistent with human language users. We conclude with suggestions for better connecting LMs with the empirical study of child language acquisition.

Computation and Language,Artificial Intelligence

Primary liposarcoma of bone. Case report.

Rule-Based and Word-Level Statistics-Based Processing of Language: Insights from Neuroscience

Learning to Predict Novel Noun-Noun Compounds

Music for the birds: effects of auditory enrichment on captive bird species.

Building Probabilistic Models for Natural Language

Syntactic Complexity Development in the Writings of EFL Learners: Insights from a Dependency Syntactically-Annotated Corpus

Botulinum toxin a improves muscle spasms and rigidity in stiff‐person syndrome

Evaluating statistical language models as pragmatic reasoners

Real-time implementation of synthetic aperture vector flow imaging on a consumer-level tablet

Word learning and the acquisition of syntactic--semantic overhypotheses

Language Acquisition in Computers

Modelling constructivist language acquisition through syntactico-semantic pattern finding

Linguistic Structure Induction from Language Models

Explaining Large Language Model-Based Neural Semantic Parsers (Student Abstract)

Large Language Models as Neurolinguistic Subjects: Identifying Internal Representations for Form and Meaning

Subspace Chronicles: How Linguistic Information Emerges, Shifts and Interacts during Language Model Training

The probabilistic analysis of language acquisition: Theoretical, computational, and experimental analysis

Augmented linguistic analysis skills: Machine translation and generative AI as pedagogical aids for analyzing complex English compounds

Grammar-based Game Description Generation using Large Language Models

Learning Language Structures through Grounding

Evaluating Neural Language Models as Cognitive Models of Language Acquisition