What do Transformers Know about Government?

Jue Hou,Anisia Katinskaia,Lari Kotilainen,Sathianpong Trangcasanchai,Anh-Duc Vu,Roman Yangarber

2024-04-22

Abstract:This paper investigates what insights about linguistic features and what knowledge about the structure of natural language can be obtained from the encodings in transformer language

Computation and Language,Machine Learning

What problem does this paper attempt to address?

What Formal Languages Can Transformers Express? A Survey

Lena Strobl,William Merrill,Gail Weiss,David Chiang,Dana Angluin

DOI: https://doi.org/10.1162/tacl_a_00663

2024-09-04

Abstract:As transformers have gained prominence in natural language processing, some researchers have investigated theoretically what problems they can and cannot solve, by treating problems as formal languages. Exploring such questions can help clarify the power of transformers relative to other models of computation, their fundamental capabilities and limits, and the impact of architectural choices. Work in this subarea has made considerable progress in recent years. Here, we undertake a comprehensive survey of this work, documenting the diverse assumptions that underlie different results and providing a unified framework for harmonizing seemingly contradictory findings.

Machine Learning,Computation and Language,Formal Languages and Automata Theory,Logic in Computer Science
Why transformers are obviously good models of language

Felix Hill

2024-08-07

Abstract:Nobody knows how language works, but many theories abound. Transformers are a class of neural networks that process language automatically with more success than alternatives, both those based on neural computations and those that rely on other (e.g. more symbolic) mechanisms. Here, I highlight direct connections between the transformer architecture and certain theoretical perspectives on language. The empirical success of transformers relative to alternative models provides circumstantial evidence that the linguistic approaches that transformers embody should be, at least, evaluated with greater scrutiny by the linguistics community and, at best, considered to be the currently best available theories.

Computation and Language
On the Ability and Limitations of Transformers to Recognize Formal Languages

Satwik Bhattamishra,Kabir Ahuja,Navin Goyal

DOI: https://doi.org/10.48550/arXiv.2009.11264

2020-10-08

Abstract:Transformers have supplanted recurrent models in a large number of NLP tasks. However, the differences in their abilities to model different syntactic properties remain largely unknown. Past works suggest that LSTMs generalize very well on regular languages and have close connections with counter languages. In this work, we systematically study the ability of Transformers to model such languages as well as the role of its individual components in doing so. We first provide a construction of Transformers for a subclass of counter languages, including well-studied languages such as n-ary Boolean Expressions, Dyck-1, and its generalizations. In experiments, we find that Transformers do well on this subclass, and their learned mechanism strongly correlates with our construction. Perhaps surprisingly, in contrast to LSTMs, Transformers do well only on a subset of regular languages with degrading performance as we make languages more complex according to a well-known measure of complexity. Our analysis also provides insights on the role of self-attention mechanism in modeling certain behaviors and the influence of positional encoding schemes on the learning and generalization abilities of the model.

Computation and Language,Machine Learning
Transformers, Contextualism, and Polysemy

Jumbly Grindrod

2024-09-26

Abstract:The transformer architecture, introduced by Vaswani et al. (2017), is at the heart of the remarkable recent progress in the development of language models, including widely-used chatbots such as Chat-GPT and Claude. In this paper, I argue that we can extract from the way the transformer architecture works a theory of the relationship between context and meaning. I call this the transformer theory, and I argue that it is novel with regard to two related philosophical debates: the contextualism debate regarding the extent of context-sensitivity across natural language, and the polysemy debate regarding how polysemy should be captured within an account of word meaning.

Computation and Language,Artificial Intelligence
Transformers in Speech Processing: A Survey

Siddique Latif,Aun Zaidi,Heriberto Cuayahuitl,Fahad Shamshad,Moazzam Shoukat,Junaid Qadir

2023-03-21

Abstract:The remarkable success of transformers in the field of natural language processing has sparked the interest of the speech-processing community, leading to an exploration of their potential for modeling long-range dependencies within speech sequences. Recently, transformers have gained prominence across various speech-related domains, including automatic speech recognition, speech synthesis, speech translation, speech para-linguistics, speech enhancement, spoken dialogue systems, and numerous multimodal applications. In this paper, we present a comprehensive survey that aims to bridge research studies from diverse subfields within speech technology. By consolidating findings from across the speech technology landscape, we provide a valuable resource for researchers interested in harnessing the power of transformers to advance the field. We identify the challenges encountered by transformers in speech processing while also offering insights into potential solutions to address these issues.

Computation and Language,Sound,Audio and Speech Processing
Transformers in Natural Language Processing: A Comprehensive Review

Mohd Arsalan

DOI: https://doi.org/10.22214/ijraset.2024.62863

2024-05-31

International Journal for Research in Applied Science and Engineering Technology

Abstract:Abstract: This research paper provides a comprehensive review of transformers, a groundbreaking architecture in Natural Language Processing (NLP), and their impact on the field. We delve into the unique ability of transformers, with their attention mechanisms, to capture complex linguistic dependencies and process sequences effectively. This enables them to grasp contextual nuances, syntactic structures, and semantic intricacies inherent in human language, outperforming traditional models. This review explores the transformer architecture, various pre-trained language models based on it, and their diverse applications in multilingual NLP. We further discuss fine-tuning and transfer learning techniques for adapting transformers to specific tasks, recent developments, challenges, open problems, and future directions. This comprehensive analysis aims to contribute to a deeper understanding of how transformers have revolutionized NLP, offering insights into their potential for shaping the future of language understanding and generation
A mathematical perspective on Transformers

Borjan Geshkovski,Cyril Letrouit,Yury Polyanskiy,Philippe Rigollet

2024-08-12

Abstract:Transformers play a central role in the inner workings of large language models. We develop a mathematical framework for analyzing Transformers based on their interpretation as interacting particle systems, which reveals that clusters emerge in long time. Our study explores the underlying theory and offers new perspectives for mathematicians as well as computer scientists.

Machine Learning,Analysis of PDEs,Dynamical Systems
Introduction to Transformers: an NLP Perspective

Tong Xiao,Jingbo Zhu

2023-01-01

Abstract:Transformers have dominated empirical machine learning models of natural language processing. In this paper, we introduce basic concepts of Transformers and present key techniques that form the recent advances of these models. This includes a description of the standard Transformer architecture, a series of model refinements, and common applications. Given that Transformers and related deep learning techniques might be evolving in ways we have never seen, we cannot dive into all the model details or cover all the technical areas. Instead, we focus on just those concepts that are helpful for gaining a good understanding of Transformers and their variants. We also summarize the key ideas that impact this field, thereby yielding some insights into the strengths and limitations of these models.
Do Transformers Encode a Foundational Ontology? Probing Abstract Classes in Natural Language

Mael Jullien,Marco Valentino,Andre Freitas

DOI: https://doi.org/10.48550/arXiv.2201.10262

2022-01-25

Abstract:With the methodological support of probing (or diagnostic classification), recent studies have demonstrated that Transformers encode syntactic and semantic information to some extent. Following this line of research, this paper aims at taking semantic probing to an abstraction extreme with the goal of answering the following research question: can contemporary Transformer-based models reflect an underlying Foundational Ontology? To this end, we present a systematic Foundational Ontology (FO) probing methodology to investigate whether Transformers-based models encode abstract semantic information. Following different pre-training and fine-tuning regimes, we present an extensive evaluation of a diverse set of large-scale language models over three distinct and complementary FO tagging experiments. Specifically, we present and discuss the following conclusions: (1) The probing results indicate that Transformer-based models incidentally encode information related to Foundational Ontologies during the pre-training pro-cess; (2) Robust FO taggers (accuracy of 90 percent)can be efficiently built leveraging on this knowledge.

Computation and Language
Knowledge-Infused Self Attention Transformers

Kaushik Roy,Yuxin Zi,Vignesh Narayanan,Manas Gaur,Amit Sheth

2023-06-23

Abstract:Transformer-based language models have achieved impressive success in various natural language processing tasks due to their ability to capture complex dependencies and contextual information using self-attention mechanisms. However, they are not without limitations. These limitations include hallucinations, where they produce incorrect outputs with high confidence, and alignment issues, where they generate unhelpful and unsafe outputs for human users. These limitations stem from the absence of implicit and missing context in the data alone. To address this, researchers have explored augmenting these models with external knowledge from knowledge graphs to provide the necessary additional context. However, the ad-hoc nature of existing methods makes it difficult to properly analyze the effects of knowledge infusion on the many moving parts or components of a transformer. This paper introduces a systematic method for infusing knowledge into different components of a transformer-based model. A modular framework is proposed to identify specific components within the transformer architecture, such as the self-attention mechanism, encoder layers, or the input embedding layer, where knowledge infusion can be applied. Additionally, extensive experiments are conducted on the General Language Understanding Evaluation (GLUE) benchmark tasks, and the findings are reported. This systematic approach aims to facilitate more principled approaches to incorporating knowledge into language model architectures.

Computation and Language
Transformers are Expressive, But Are They Expressive Enough for Regression?

Swaroop Nath,Harshad Khadilkar,Pushpak Bhattacharyya

2024-08-30

Abstract:Transformers have become pivotal in Natural Language Processing, demonstrating remarkable success in applications like Machine Translation and Summarization. Given their widespread adoption, several works have attempted to analyze the expressivity of Transformers. Expressivity of a neural network is the class of functions it can approximate. A neural network is fully expressive if it can act as a universal function approximator. We attempt to analyze the same for Transformers. Contrary to existing claims, our findings reveal that Transformers struggle to reliably approximate smooth functions, relying on piecewise constant approximations with sizable intervals. The central question emerges as: ''Are Transformers truly Universal Function Approximators?'' To address this, we conduct a thorough investigation, providing theoretical insights and supporting evidence through experiments. Theoretically, we prove that Transformer Encoders cannot approximate smooth functions. Experimentally, we complement our theory and show that the full Transformer architecture cannot approximate smooth functions. By shedding light on these challenges, we advocate a refined understanding of Transformers' capabilities. Code Link: <a class="link-external link-https" href="https://github.com/swaroop-nath/transformer-expressivity" rel="external noopener nofollow">this https URL</a>.

Machine Learning
How Do Transformers Learn Topic Structure: Towards a Mechanistic Understanding

Yuchen Li,Yuanzhi Li,Andrej Risteski

2023-07-25

Abstract:While the successes of transformers across many domains are indisputable, accurate understanding of the learning mechanics is still largely lacking. Their capabilities have been probed on benchmarks which include a variety of structured and reasoning tasks -- but mathematical understanding is lagging substantially behind. Recent lines of work have begun studying representational aspects of this question: that is, the size/depth/complexity of attention-based networks to perform certain tasks. However, there is no guarantee the learning dynamics will converge to the constructions proposed. In our paper, we provide fine-grained mechanistic understanding of how transformers learn "semantic structure", understood as capturing co-occurrence structure of words. Precisely, we show, through a combination of mathematical analysis and experiments on Wikipedia data and synthetic data modeled by Latent Dirichlet Allocation (LDA), that the embedding layer and the self-attention layer encode the topical structure. In the former case, this manifests as higher average inner product of embeddings between same-topic words. In the latter, it manifests as higher average pairwise attention between same-topic words. The mathematical results involve several assumptions to make the analysis tractable, which we verify on data, and might be of independent interest as well.

Machine Learning,Computation and Language
Understanding Transformers via N-gram Statistics

Timothy Nguyen

2024-07-01

Abstract:Transformer based large-language models (LLMs) display extreme proficiency with language yet a precise understanding of how they work remains elusive. One way of demystifying transformer predictions would be to describe how they depend on their context in terms of simple template functions. This paper takes a first step in this direction by considering families of functions (i.e. rules) formed out of simple N-gram based statistics of the training data. By studying how well these rulesets approximate transformer predictions, we obtain a variety of novel discoveries: a simple method to detect overfitting during training without using a holdout set, a quantitative measure of how transformers progress from learning simple to more complex statistical rules over the course of training, a model-variance criterion governing when transformer predictions tend to be described by N-gram rules, and insights into how well transformers can be approximated by N-gram rulesets in the limit where these rulesets become increasingly complex. In this latter direction, we find that for 78% of LLM next-token distributions on TinyStories, their top-1 predictions agree with those provided by our N-gram rulesets.

Computation and Language,Artificial Intelligence,Machine Learning
Do Transformers know symbolic rules, and would we know if they did?

Tommi Gröndahl,Yujia Guo,N. Asokan

DOI: https://doi.org/10.48550/arXiv.2203.00162

2023-03-01

Abstract:To improve the explainability of leading Transformer networks used in NLP, it is important to tease apart genuine symbolic rules from merely associative input-output patterns. However, we identify several inconsistencies in how ``symbolicity'' has been construed in recent NLP literature. To mitigate this problem, we propose two criteria to be the most relevant, one pertaining to a system's internal architecture and the other to the dissociation between abstract rules and specific input identities. From this perspective, we critically examine prior work on the symbolic capacities of Transformers, and deem the results to be fundamentally inconclusive for reasons inherent in experiment design. We further maintain that there is no simple fix to this problem, since it arises -- to an extent -- in all end-to-end settings. Nonetheless, we emphasize the need for more robust evaluation of whether non-symbolic explanations exist for success in seemingly symbolic tasks. To facilitate this, we experiment on four sequence modelling tasks on the T5 Transformer in two experiment settings: zero-shot generalization, and generalization across class-specific vocabularies flipped between the training and test set. We observe that T5's generalization is markedly stronger in sequence-to-sequence tasks than in comparable classification tasks. Based on this, we propose a thus far overlooked analysis, where the Transformer itself does not need to be symbolic to be part of a symbolic architecture as the processor, operating on the input and output as external memory components.

Machine Learning,Artificial Intelligence,Computation and Language
HuggingFace's Transformers: State-of-the-art Natural Language Processing

Joe Davison,Yacine Jernite,Clara Ma,Lysandre Debut,J. Plu,Sam Shleifer,Julien Chaumond,Thomas Wolf,Clement Delangue,Victor Sanh,Pierric Cistac,Teven Le Scao,Alexander M. Rush,Sylvain Gugger,Canwen Xu,Tim Rault,Quentin Lhoest,Mariama Drame,Patrick von Platen,Anthony Moi,Morgan Funtowicz,Rémi Louf

2019-10-09

Abstract:Recent progress in natural language processing has been driven by advances in both model architecture and model pretraining. Transformer architectures have facilitated building higher-capacity models and pretraining has made it possible to effectively utilize this capacity for a wide variety of tasks. \textit{Transformers} is an open-source library with the goal of opening up these advances to the wider machine learning community. The library consists of carefully engineered state-of-the art Transformer architectures under a unified API. Backing this library is a curated collection of pretrained models made by and available for the community. \textit{Transformers} is designed to be extensible by researchers, simple for practitioners, and fast and robust in industrial deployments. The library is available at \url{this https URL}.

Computer Science
Transformers Can Represent $n$-gram Language Models

Anej Svete,Ryan Cotterell

2024-06-20

Abstract:Existing work has analyzed the representational capacity of the transformer architecture by means of formal models of computation. However, the focus so far has been on analyzing the architecture in terms of language \emph{acceptance}. We contend that this is an ill-suited problem in the study of \emph{language models} (LMs), which are definitionally \emph{probability distributions} over strings. In this paper, we focus on the relationship between transformer LMs and $n$-gram LMs, a simple and historically relevant class of language models. We show that transformer LMs using the hard or sparse attention mechanisms can exactly represent any $n$-gram LM, giving us a concrete lower bound on their probabilistic representational capacity. This provides a first step towards understanding the mechanisms that transformer LMs can use to represent probability distributions over strings.

Computation and Language,Artificial Intelligence,Computational Complexity,Formal Languages and Automata Theory,Machine Learning
Speech Recognition Transformers: Topological-lingualism Perspective

Shruti Singh,Muskaan Singh,Virender Kadyan

2024-08-27

Abstract:Transformers have evolved with great success in various artificial intelligence tasks. Thanks to our recent prevalence of self-attention mechanisms, which capture long-term dependency, phenomenal outcomes in speech processing and recognition tasks have been produced. The paper presents a comprehensive survey of transformer techniques oriented in speech modality. The main contents of this survey include (1) background of traditional ASR, end-to-end transformer ecosystem, and speech transformers (2) foundational models in a speech via lingualism paradigm, i.e., monolingual, bilingual, multilingual, and cross-lingual (3) dataset and languages, acoustic features, architecture, decoding, and evaluation metric from a specific topological lingualism perspective (4) popular speech transformer toolkit for building end-to-end ASR systems. Finally, highlight the discussion of open challenges and potential research directions for the community to conduct further research in this domain.

Computation and Language,Sound,Audio and Speech Processing
Knowledge Circuits in Pretrained Transformers

Yunzhi Yao,Ningyu Zhang,Zekun Xi,Mengru Wang,Ziwen Xu,Shumin Deng,Huajun Chen

2024-10-16

Abstract:The remarkable capabilities of modern large language models are rooted in their vast repositories of knowledge encoded within their parameters, enabling them to perceive the world and engage in reasoning. The inner workings of how these models store knowledge have long been a subject of intense interest and investigation among researchers. To date, most studies have concentrated on isolated components within these models, such as the Multilayer Perceptrons and attention head. In this paper, we delve into the computation graph of the language model to uncover the knowledge circuits that are instrumental in articulating specific knowledge. The experiments, conducted with GPT2 and TinyLLAMA, have allowed us to observe how certain information heads, relation heads, and Multilayer Perceptrons collaboratively encode knowledge within the model. Moreover, we evaluate the impact of current knowledge editing techniques on these knowledge circuits, providing deeper insights into the functioning and constraints of these editing methodologies. Finally, we utilize knowledge circuits to analyze and interpret language model behaviors such as hallucinations and in-context learning. We believe the knowledge circuits hold potential for advancing our understanding of Transformers and guiding the improved design of knowledge editing. Code and data are available in <a class="link-external link-https" href="https://github.com/zjunlp/KnowledgeCircuits" rel="external noopener nofollow">this https URL</a>.

Computation and Language,Artificial Intelligence,Computer Vision and Pattern Recognition,Information Retrieval,Machine Learning
The Antecedents of Transformer Models

Simon Dennis,Kevin Shabahang,Hyungwook Yim

DOI: https://doi.org/10.1177/09637214241279504

IF: 7.867

2024-11-24

Current Directions in Psychological Science

Abstract:Current Directions in Psychological Science, Ahead of Print. Transformer models of language represent a step change in our ability to account for cognitive phenomena. Although the specific architecture that has garnered recent interest is quite young, many of its components have antecedents in the cognitive science literature. In this article, we start by providing an introduction to large language models aimed at a general psychological audience. We then highlight some of the antecedents, including the importance of scale, instance-based memory models, paradigmatic association and systematicity, positional encodings of serial order, and the learning of control processes. This article offers an exploration of the relationship between transformer models and their precursors, showing how they can be understood as a next phase in our understanding of cognitive processes.

psychology, multidisciplinary
Transformers as Soft Reasoners over Language

Peter Clark,Oyvind Tafjord,Kyle Richardson

DOI: https://doi.org/10.48550/arXiv.2002.05867

2020-05-06

Abstract:Beginning with McCarthy's Advice Taker (1959), AI has pursued the goal of providing a system with explicit, general knowledge and having the system reason over that knowledge. However, expressing the knowledge in a formal (logical or probabilistic) representation has been a major obstacle to this research. This paper investigates a modern approach to this problem where the facts and rules are provided as natural language sentences, thus bypassing a formal representation. We train transformers to reason (or emulate reasoning) over these sentences using synthetically generated data. Our models, that we call RuleTakers, provide the first empirical demonstration that this kind of soft reasoning over language is learnable, can achieve high (99%) accuracy, and generalizes to test data requiring substantially deeper chaining than seen during training (95%+ scores). We also demonstrate that the models transfer well to two hand-authored rulebases, and to rulebases paraphrased into more natural language. These findings are significant as it suggests a new role for transformers, namely as limited "soft theorem provers" operating over explicit theories in language. This in turn suggests new possibilities for explainability, correctability, and counterfactual reasoning in question-answering.

Computation and Language,Artificial Intelligence

What do Transformers Know about Government?

What Formal Languages Can Transformers Express? A Survey

Why transformers are obviously good models of language

On the Ability and Limitations of Transformers to Recognize Formal Languages

Transformers, Contextualism, and Polysemy

Transformers in Speech Processing: A Survey

Transformers in Natural Language Processing: A Comprehensive Review

A mathematical perspective on Transformers

Introduction to Transformers: an NLP Perspective

Do Transformers Encode a Foundational Ontology? Probing Abstract Classes in Natural Language

Knowledge-Infused Self Attention Transformers

Transformers are Expressive, But Are They Expressive Enough for Regression?

How Do Transformers Learn Topic Structure: Towards a Mechanistic Understanding

Understanding Transformers via N-gram Statistics

Do Transformers know symbolic rules, and would we know if they did?

HuggingFace's Transformers: State-of-the-art Natural Language Processing

Transformers Can Represent $n$-gram Language Models

Speech Recognition Transformers: Topological-lingualism Perspective

Knowledge Circuits in Pretrained Transformers

The Antecedents of Transformer Models

Transformers as Soft Reasoners over Language