Similarity Detection Pipeline for Crawling a Topic Related Fake News Corpus

Inna Vogel,Jeong-Eun Choi,Meghana Meghana

DOI: https://doi.org/10.48550/arXiv.2009.13367

2021-03-01

Abstract:Fake news detection is a challenging task aiming to reduce human time and effort to check the truthfulness of news. Automated approaches to combat fake news, however, are limited by the lack of labeled benchmark datasets, especially in languages other than English. Moreover, many publicly available corpora have specific limitations that make them difficult to use. To address this problem, our contribution is threefold. First, we propose a new, publicly available German topic related corpus for fake news detection. To the best of our knowledge, this is the first corpus of its kind. In this regard, we developed a pipeline for crawling similar news articles. As our third contribution, we conduct different learning experiments to detect fake news. The best performance was achieved using sentence level embeddings from SBERT in combination with a Bi-LSTM (k=0.88).

Computation and Language,Information Retrieval