POLygraph: Polish Fake News Dataset

Daniel Dzienisiewicz,Filip Graliński,Piotr Jabłoński,Marek Kubis,Paweł Skórzewski,Piotr Wierzchoń

2024-07-01

Abstract:This paper presents the POLygraph dataset, a unique resource for fake news detection in Polish. The dataset, created by an interdisciplinary team, is composed of two parts: the "fake-or-not" dataset with 11,360 pairs of news articles (identified by their URLs) and corresponding labels, and the "fake-they-say" dataset with 5,082 news articles (identified by their URLs) and tweets commenting on them. Unlike existing datasets, POLygraph encompasses a variety of approaches from source literature, providing a comprehensive resource for fake news detection. The data was collected through manual annotation by expert and non-expert annotators. The project also developed a software tool that uses advanced machine learning techniques to analyze the data and determine content authenticity. The tool and dataset are expected to benefit various entities, from public sector institutions to publishers and fact-checking organizations. Further dataset exploration will foster fake news detection and potentially stimulate the implementation of similar models in other languages. The paper focuses on the creation and composition of the dataset, so it does not include a detailed evaluation of the software tool for content authenticity analysis, which is planned at a later stage of the project.

Computation and Language

What problem does this paper attempt to address?

The paper aims to address the issue of fake news detection in the Polish online environment. Specifically, it introduces the POLygraph dataset, a unique resource for detecting fake news in Polish. The dataset consists of two parts:

1. **"True and False News" dataset**: Contains 11,360 pairs of news articles and their labels (indicating whether the news is fake), with each article identified by its URL.
2. **"They Say It's Fake" dataset**: Contains 5,082 news articles along with their URLs and tweets commenting on these articles. Each tweet is accompanied by a label expressing the commenter's view on the article's authenticity.

Unlike existing datasets, POLygraph encompasses various methods, providing a comprehensive resource for fake news detection. The dataset is manually annotated by both expert and non-expert annotators, and a software tool has been developed to analyze the data using advanced machine learning techniques to determine the authenticity of the content. This tool and dataset are expected to benefit entities in multiple fields, including public sector institutions, publishers, and fact-checking organizations. Additionally, further exploration of the dataset will promote fake news detection and may stimulate the implementation of similar models in other languages. The paper focuses on the creation and composition of the dataset and does not provide a detailed evaluation of the software tool used for content authenticity analysis, which will be conducted in the later stages of the project.