Abstract:Recent breakthroughs in artificial intelligence have driven a paradigm shift, where large language models (LLMs) with billions or trillions of parameters are trained on vast datasets, achieving unprecedented success across a series of language tasks. However, despite these successes, LLMs still rely on probabilistic modeling, which often captures spurious correlations rooted in linguistic patterns and social stereotypes, rather than the true causal relationships between entities and events. This limitation renders LLMs vulnerable to issues such as demographic biases, social stereotypes, and LLM hallucinations. These challenges highlight the urgent need to integrate causality into LLMs, moving beyond correlation-driven paradigms to build more reliable and ethically aligned AI systems. While many existing surveys and studies focus on utilizing prompt engineering to activate LLMs for causal knowledge or developing benchmarks to assess their causal reasoning abilities, most of these efforts rely on human intervention to activate pre-trained models. How to embed causality into the training process of LLMs and build more general and intelligent models remains unexplored. Recent research highlights that LLMs function as causal parrots, capable of reciting causal knowledge without truly understanding or applying it. These prompt-based methods are still limited to human interventional improvements. This survey aims to address this gap by exploring how causality can enhance LLMs at every stage of their lifecycle-from token embedding learning and foundation model training to fine-tuning, alignment, inference, and evaluation-paving the way for more interpretable, reliable, and causally-informed models. Additionally, we further outline six promising future directions to advance LLM development, enhance their causal reasoning capabilities, and address the current limitations these models face.

Explainable Fault Diagnosis of Control Systems Using Large Language Models

Integrating LLMs for Explainable Fault Diagnosis in Complex Systems

An Explainable AI Approach to Large Language Model Assisted Causal Model Auditing and Development

Fault Diagnosis in Power Grids with Large Language Model

Towards Uncovering How Large Language Model Works: An Explainability Perspective

Cause and Effect: Can Large Language Models Truly Understand Causality?

Explainability for Large Language Models: A Survey

Explainable Artificial Intelligence for Fault Diagnosis of Industrial Processes

Real-Time Anomaly Detection and Reactive Planning with Large Language Models

Integrating Automated Knowledge Extraction with Large Language Models for Explainable Medical Decision-Making

Consultation on Industrial Machine Faults with Large language Models

Research on Fault Diagnosis Method Based on Structural Causal Model in Tennessee Eastman Process

A Fault Diagnosis Model Based on Language-Valued Reasoning

Towards Effectively Detecting and Explaining Vulnerabilities Using Large Language Models

Large Language Models for Explainable Decisions in Dynamic Digital Twins

Explaining Agent Behavior with Large Language Models

A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validation

Improving Causal Reasoning in Large Language Models: A Survey

Causality for Large Language Models

Explainable Fault Diagnosis of Oil-Immersed Transformers: A Glass-Box Model

Explainability: Relevance based dynamic deep learning algorithm for fault detection and diagnosis in chemical processes