Abstract:In the recent past, more than 5 years or so, Deep Learning (DL) especially the large language models (LLMs) has generated extensive studies out of a distinctly average downturn field of knowledge made up of a traditional society of researchers. As a result, DL is now so pervasive that its use is widespread across the body of research related to machine learning computing. The rapid emergence and apparent dominance of DL architectures over traditional machine learning techniques on a variety of tasks have been truly astonishing to witness. DL models outperformed in a variety of areas, including natural language processing (NLP), image analysis, language understanding, machine translation, computer vision, speech processing, audio recognition, style imitation, and computational biology. In this study, the aim is to explain the rudiments of DL, such as neural networks, convolutional neural networks, deep belief networks, and various variants of DL. The study will explore how these models have been applied to NLP and delve into the underlying mathematics behind them. Additionally, the study will investigate the latest advancements in DL and NLP, while acknowledging the key challenges and emerging trends in the field. Furthermore, it will discuss the core component of DL, namely embeddings, from a taxonomic perspective. Moreover, a literature review will be provided focusing on the application of DL models for six popular pattern recognition tasks: speech recognition, question answering, part of speech tagging, named entity recognition, text classification, and machine translation. Finally, the study will demystify state-of-the-art DL libraries/frameworks and available resources. The outcome and implication of this study reveal that LLMs face challenges in dealing with pragmatic aspects of language due to their reliance on statistical learning techniques and lack of genuine understanding of context, presupposition, implicature, and social norms. Furthermore, this study provides a comprehensive analysis of the current state-of-the-art advancements and highlights significant obstacles and emerging developments. The article has the potential to enhance readers’ understanding of the subject matter.

Symbolic, Distributed and Distributional Representations for Natural Language Processing in the Era of Deep Learning: a Survey

Visualizing and Understanding Neural Models in NLP

Synergizing Machine Learning & Symbolic Methods: A Survey on Hybrid Approaches to Natural Language Processing

Discrete and continuous representations and processing in deep learning: Looking forward

Deep Learning for Natural Language Processing: A Survey

Emergence of machine language: towards symbolic intelligence with neural networks

A Survey on Symbolic Knowledge Distillation of Large Language Models

Interpreting Deep Learning Models in Natural Language Processing: A Review

Distributed Representation of Semantics in the Human Brain: Evidence from Studies Using Natural Language Processing Techniques

Deep Symbolic Learning: Discovering Symbols and Rules from Perceptions

Exploring the frontiers of deep learning and natural language processing: A comprehensive overview of key challenges and emerging trends

Deep Learning in Natural Language Processing: A State-of-the-Art Survey

Symbolic Techniques for Deep Learning: Challenges and Opportunities

A Neural-Symbolic Approach to Natural Language Tasks

Semantics, Representations and Grammars for Deep Learning

Local vs distributed representations: What is the right basis for interpretability?

Neural Encoding and Decoding With Distributed Sentence Representations

A Survey of Text Representation Methods and Their Genealogy

A Comprehensive Survey of Sentence Representations: From the BERT Epoch to the ChatGPT Era and Beyond

Learning distributed representations of knowledge that preserve deductive reasoning