Entity Extraction from High-Level Corruption Schemes via Large Language Models

Panagiotis Koletsis,Panagiotis-Konstantinos Gemos,Christos Chronis,Iraklis Varlamis,Vasilis Efthymiou,Georgios Th. Papadopoulos

2024-09-05

Abstract:The rise of financial crime that has been observed in recent years has created an increasing concern around the topic and many people, organizations and governments are more and more frequently trying to combat it. Despite the increase of interest in this area, there is a lack of specialized datasets that can be used to train and evaluate works that try to tackle those problems. This article proposes a new micro-benchmark dataset for algorithms and models that identify individuals and organizations, and their multiple writings, in news articles, and presents an approach that assists in its creation. Experimental efforts are also reported, using this dataset, to identify individuals and organizations in financial-crime-related articles using various low-billion parameter Large Language Models (LLMs). For these experiments, standard metrics (Accuracy, Precision, Recall, F1 Score) are reported and various prompt variants comprising the best practices of prompt engineering are tested. In addition, to address the problem of ambiguous entity mentions, a simple, yet effective LLM-based disambiguation method is proposed, ensuring that the evaluation aligns with reality. Finally, the proposed approach is compared against a widely used state-of-the-art open-source baseline, showing the superiority of the proposed method.

Computation and Language

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the challenge of entity extraction in financial crime data, especially identifying individuals and organizations from news articles and their various writing forms. Although the attention to financial crime has increased in recent years, there is a lack of specialized datasets to train and evaluate the work on solving these problems. Therefore, this paper proposes a new micro - benchmark dataset for algorithms and models to identify individuals and organizations in articles related to financial crime, and introduces a method to assist in creating this dataset. In addition, the paper also reports the efforts of experiments using different sub - billion - parameter large - language models (LLMs), as well as the evaluation results using standard metrics (accuracy, precision, recall, F1 - score). To deal with the problem of ambiguous entity mentions, the paper also proposes a simple and effective LLM - based disambiguation method to ensure that the evaluation is in line with the real situation. Finally, the proposed method is compared with widely - used open - source baseline methods, showing the advantages of the proposed method.

Entity Extraction from High-Level Corruption Schemes via Large Language Models

DetoxBench: Benchmarking Large Language Models for Multitask Fraud & Abuse Detection

Large Language Models as Financial Data Annotators: A Study on Effectiveness and Efficiency

Structured Entity Extraction Using Large Language Models

SECURE: Benchmarking Large Language Models for Cybersecurity

Large Language Models for Judicial Entity Extraction: A Comparative Study

Exposing LLM Vulnerabilities: Adversarial Scam Detection and Performance

LLMs Perform Poorly at Concept Extraction in Cyber-security Research Literature

Trustworthy AI: Securing Sensitive Data in Large Language Models

ThreatModeling-LLM: Automating Threat Modeling using Large Language Models for Banking System

Leveraging Large Language Models to Democratize Access to Costly Financial Datasets for Academic Research

On Leveraging Large Language Models for Enhancing Entity Resolution: A Cost-efficient Approach

Detecting Scams Using Large Language Models

Capture the Flag: Uncovering Data Insights with Large Language Models

A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challenges

Assessing biomedical knowledge robustness in large language models by query-efficient sampling attacks

Advancing Anomaly Detection: Non-Semantic Financial Data Encoding with LLMs

Deconstructing The Ethics of Large Language Models from Long-standing Issues to New-emerging Dilemmas: A Survey

Data-Centric Financial Large Language Models

Evaluating Large Language Models on Financial Report Summarization: An Empirical Study

GSAP-NER: A Novel Task, Corpus, and Baseline for Scholarly Entity Extraction Focused on Machine Learning Models and Datasets