Abstract:Recent advancements in deep learning (DL) have posed a significant challenge for automatic speech recognition (ASR). ASR relies on extensive training datasets, including confidential ones, and demands substantial computational and storage resources. Enabling adaptive systems improves ASR performance in dynamic environments. DL techniques assume training and testing data originate from the same domain, which is not always true. Advanced DL techniques like deep transfer learning (DTL), federated learning (FL), and reinforcement learning (RL) address these issues. DTL allows high-performance models using small yet related datasets, FL enables training on confidential data without dataset possession, and RL optimizes decision-making in dynamic environments, reducing computation costs. This survey offers a comprehensive review of DTL, FL, and RL-based ASR frameworks, aiming to provide insights into the latest developments and aid researchers and professionals in understanding the current challenges. Additionally, transformers, which are advanced DL techniques heavily used in proposed ASR frameworks, are considered in this survey for their ability to capture extensive dependencies in the input ASR sequence. The paper starts by presenting the background of DTL, FL, RL, and Transformers and then adopts a well-designed taxonomy to outline the state-of-the-art approaches. Subsequently, a critical analysis is conducted to identify the strengths and weaknesses of each framework. Additionally, a comparative study is presented to highlight the existing challenges, paving the way for future research opportunities.

Document Automation Architectures: Updated Survey in Light of Large Language Models

Document Automation Architectures and Technologies: A Survey

AutoSurvey: Large Language Models Can Automatically Write Surveys

Rethinking Legal Compliance Automation: Opportunities with Large Language Models

A Survey of Deep Learning Approaches for OCR and Document Understanding

Large Language Models for Data Annotation and Synthesis: A Survey

Large Language Models in Law: A Survey

A Survey on Large Language Model based Autonomous Agents

Automatic Speech Recognition using Advanced Deep Learning Approaches: A survey

Large Language Model-Based Agents for Software Engineering: A Survey

Document AI: A Comparative Study of Transformer-Based, Graph-Based Models, and Convolutional Neural Networks For Document Layout Analysis

Natural Language Processing for the Legal Domain: A Survey of Tasks, Datasets, Models, and Challenges

Document Automation Using Artificial Intelligence

Large Language Models for Data Annotation: A Survey

Large Language Models for Constructing and Optimizing Machine Learning Workflows: A Survey

Real-world Machine Learning Systems: A survey from a Data-Oriented Architecture Perspective

Large language models for automated Q&A involving legal documents: a survey on algorithms, frameworks and applications

A Survey on Neural Architecture Search

Large Language Models for Human-like Autonomous Driving: A Survey

Transformers and Language Models in Form Understanding: A Comprehensive Review of Scanned Document Analysis