Abstract:We propose a novel training and inference method for detecting political bias in long text content such as newspaper opinion articles. Obtaining long text data and annotations at sufficient scale for training is difficult, but it is relatively easy to extract political polarity from tweets through their authorship; as such, we train on tweets and perform inference on articles. Universal sentence encoders and other existing methods that aim to address this domain-adaptation scenario deliver inaccurate and inconsistent predictions on articles, which we show is due to a difference in opinion concentration between tweets and articles. We propose a two-step classification scheme that utilizes a neutral detector trained on tweets to remove neutral sentences from articles in order to align opinion concentration and therefore improve accuracy on that domain. We evaluate our two-step approach using a variety of test suites, including a set of tweets and long-form articles where annotations were crowd-sourced to decrease label noise, measuring accuracy and Spearman-rho rank correlation. In practice, KnowBias achieves a high accuracy of 86 (rho = 0.65) on these tweets and 75 (rho = 0.69) on long-form articles. While we validate our method on political bias, our scheme is general and can be readily applied to other settings, where there exist such domain mismatches between source and target domains. Our implementation is available for public use at <a class="link-external link-https" href="https://knowbias.ml" rel="external noopener nofollow">this https URL</a>.

Analysis of Media Writing Style Bias through Text-Embedding Networks

No Word Embedding Model Is Perfect: Evaluating the Representation Accuracy for Social Bias in the Media

What Was Written vs. Who Read It: News Media Profiling Using Text Analysis and Social Media Context

Connecting the Dots in News Analysis: Bridging the Cross-Disciplinary Disparities in Media Bias and Framing

An Interdisciplinary Approach for the Automated Detection and Visualization of Media Bias in News Articles

How-to Present News on Social Media: A Causal Analysis of Editing News Headlines for Boosting User Engagement

Disentangling Structure and Style: Political Bias Detection in News by Inducing Document Hierarchy

Developing a Natural Language Understanding Model to Characterize Cable News Bias

In Plain Sight: Media Bias Through the Lens of Factual Reporting

DocNet: Semantic Structure in Inductive Bias Detection Models

Identification of Biased Terms in News Articles by Comparison of Outlet-specific Word Embeddings

NewsUnfold: Creating a News-Reading Application That Indicates Linguistic Media Bias and Collects Feedback

Sentence-level Media Bias Analysis with Event Relation Graph

Balancing Transparency and Accuracy: A Comparative Analysis of Rule-Based and Deep Learning Models in Political Bias Classification

Mapping the Media Landscape: Predicting Factual Reporting and Political Bias Through Web Interactions

Evaluating Metrics for Bias in Word Embeddings

The Media Bias Taxonomy: A Systematic Literature Review on the Forms and Automated Detection of Media Bias

KnowBias: A Novel AI Method to Detect Polarity in Online Content

IndiVec: An Exploration of Leveraging Large Language Models for Media Bias Detection with Fine-Grained Bias Indicators

Social Biases in Automatic Evaluation Metrics for NLG