Abstract:This paper presents LogiCode, a novel framework that leverages Large Language Models (LLMs) for identifying logical anomalies in industrial settings, moving beyond traditional focus on structural inconsistencies. By harnessing LLMs for logical reasoning, LogiCode autonomously generates Python codes to pinpoint anomalies such as incorrect component quantities or missing elements, marking a significant leap forward in anomaly detection technologies. A custom dataset "LOCO-Annotations" and a benchmark "LogiBench" are introduced to evaluate the LogiCode's performance across various metrics including binary classification accuracy, code generation success rate, and precision in reasoning. Findings demonstrate LogiCode's enhanced interpretability, significantly improving the accuracy of logical anomaly detection and offering detailed explanations for identified anomalies. This represents a notable shift towards more intelligent, LLM-driven approaches in industrial anomaly detection, promising substantial impacts on industry-specific applications.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to detect logical anomalies in an industrial environment. Traditional methods mainly focus on the detection of structural defects, such as dents or scratches, which can be identified by vision - based techniques. However, these methods are not effective in dealing with logical anomalies. Logical anomalies refer to high - level semantic logic errors that occur when combining normal components, such as an incorrect number of components or incorrect component positions. These problems are common in industrial assembly and packaging and can lead to serious functional consequences. Therefore, a new method is needed to effectively detect and interpret these logical anomalies. The paper proposes a new framework named LogiCode, which uses large - language models (LLMs) for logical reasoning and automatically generates Python code to detect logical anomalies. LogiCode not only improves detection accuracy but also provides detailed explanations, enhancing the interpretability of the results. In addition, the paper also introduces the LOCO - Annotations dataset and the LogiBench benchmarking tool for evaluating the performance of LogiCode. ### Key contributions: 1. **Proposed a new logical - anomaly - detection method**: The LogiCode framework utilizes the logical - reasoning capabilities of LLMs to generate executable Python code, simulating the human reasoning process, and can accurately detect and interpret logical anomalies in industrial scenarios. 2. **Introduced the LogiBench benchmarking tool**: This tool combines LLMs automatic evaluation and expert manual analysis to comprehensively evaluate the accuracy and effectiveness of logical - anomaly - detection systems. 3. **Contributed the LOCO - Annotations dataset**: This carefully - designed dataset covers a wide range of logical - anomaly - detection scenarios, providing rich resources to support automatic and manual evaluation. The dataset has been publicly released to promote future research. Through these innovations, LogiCode not only fills the gaps in existing logical - anomaly - detection methods but also improves the interpretability and adaptability of detection, providing a new solution for industrial quality control.

LogiCode: an LLM-Driven Framework for Logical Anomaly Detection

LogLLM: Log-based Anomaly Detection Using Large Language Models

Large Language Models for Code Analysis: Do LLMs Really Do Their Job?

LogAnomaly: Unsupervised Detection of Sequential and Quantitative Anomalies in Unstructured Logs

Improving Natural Language Capability of Code Large Language Model

Adapting Large Language Models to Log Analysis with Interpretable Domain Knowledge

LMs: Understanding Code Syntax and Semantics for Code Analysis

Leveraging Log Instructions in Log-based Anomaly Detection

Uncovering LLM-Generated Code: A Zero-Shot Synthetic Code Detector via Code Rewriting

Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning

LLMeLog: an Approach for Anomaly Detection Based on LLM-enriched Log Events

AutoLog: A Log Sequence Synthesis Framework for Anomaly Detection

CodeHalu: Investigating Code Hallucinations in LLMs via Execution-based Verification

MP-CodeCheck: Evolving Logical Expression Code Anomaly Learning with Iterative Self-Supervision

If LLM Is the Wizard, Then Code Is the Wand: A Survey on How Code Empowers Large Language Models to Serve as Intelligent Agents

A Robust Wide & Deep Learning Framework for Log-Based Anomaly Detection

LogicAL: Towards logical anomaly synthesis for unsupervised anomaly localization

DALD: Improving Logits-based Detector without Logits from Black-box LLMs

Improving Logits-based Detector without Logits from Black-box LLMs

Showing LLM-Generated Code Selectively Based on Confidence of LLMs