Why forecast an election that's too close to call?

Andrew Gelman

DOI: https://doi.org/10.1038/d41586-024-03470-7

IF: 64.8

2024-10-30

Nature

Abstract:Predictive models don't make the news, but they have a crucial role in democracy. Predictive models don't make the news, but they have a crucial role in democracy.

multidisciplinary sciences

What problem does this paper attempt to address?

Modeling undecided voters to forecast elections: From bandwagon behavior and the spiral of silence perspective

Yezheng Liu,Chang Ye,Jianshan Sun,Yuanchun Jiang,Hai Wang

DOI: https://doi.org/10.1016/j.ijforecast.2020.06.011

IF: 7.022

2021-04-01

International Journal of Forecasting

Abstract:<p>Forecasting election results has been a highly attractive activity among political and social scientists. Different forecasting methods have been proposed, but those based on public opinion polls are the most common. However, there are challenges to using opinion polls, especially because they neglect undecided voters. Due to the significant number of undecided participants and their impact on voting outcomes, we analyze the potential behavior of undecided voters by considering opinion polls and sentiment based on voter expectation from the perspective of the bandwagon effect and the spiral of silence. We establish a hierarchical Bayesian forecasting model to predict voting results, and apply it to the 2016 United States presidential election and the 2016 Brexit referendum. The results of our model suggest that voting outcomes are more predictable when fully utilizing the impact of undecided voters. The results indicate that integrating aggregated polls into the hierarchical Bayesian framework is a strong predictor for forecasting outcomes, and they provide evidence for the influence of sentiment based on voter expectation in forecasting election results.</p>

economics,management
Developing Spatio-Temporal Approach to Predict Economic Dynamics Based on Online News

Yuzhou Zhang,Hua Sun,Guang Gao,Lidan Shou,Dun Wu

DOI: https://doi.org/10.1038/s41598-022-20489-w

IF: 4.6

2022-01-01

Scientific Reports

Abstract:Economic forecasting is a scientific decision-making tool, and it is one of the important basis for the government to formulate economic plans, predict the implementation of the plan, and guide the implementation of the plan. Current knowledge about the use of online news in the prediction of economic patterns in China is limited, especially considering the spatio-temporal dynamics over time. This study explored the spatio-temporal patterns of economic output values in Yinzhou, Ningbo, China between 2018 and 2021, and proposed generalized linear model (GLM) and Geographically weighted regression (GWR) model to predict the dynamics using online news data. The results indicated that there were spatio-temporal variations in the economic dynamics in the study area. The online news showed a great potential to predict economic dynamics, with better performance in the GWR model. The findings suggested online news combining with spatio-temporal approach can better forecast economic dynamics, which can be seen as a pre-requisite for developing an online news-based surveillance system The advanced spatio-temporal analysis enables governments to garner insights about the patterns of economic dynamics over time, which may enhance the ability of government to formulate economic plans and to predict the implementation of the plan. The proposed model may be extended to greater geographic area to validate such approach.
Why polls fail to predict elections

Zhenkun Zhou,Matteo Serafino,Luciano Cohan,Guido Caldarelli,Hernan A. Makse

DOI: https://doi.org/10.48550/arXiv.2101.11389

2021-01-27

Abstract:In the past decade we have witnessed the failure of traditional polls in predicting presidential election outcomes across the world. To understand the reasons behind these failures we analyze the raw data of a trusted pollster which failed to predict, along with the rest of the pollsters, the surprising 2019 presidential election in Argentina which has led to a major market collapse in that country. Analysis of the raw and re-weighted data from longitudinal surveys performed before and after the elections reveals clear biases (beyond well-known low-response rates) related to mis-representation of the population and, most importantly, to social-desirability biases, i.e., the tendency of respondents to hide their intention to vote for controversial candidates. We then propose a longitudinal opinion tracking method based on big-data analytics from social media, machine learning, and network theory that overcomes the limits of traditional polls. The model achieves accurate results in the 2019 Argentina elections predicting the overwhelming victory of the candidate Alberto Fernández over the president Mauricio Macri; a result that none of the traditional pollsters in the country was able to predict. Beyond predicting political elections, the framework we propose is more general and can be used to discover trends in society; for instance, what people think about economics, education or climate change.

Social and Information Networks
How time and pollster history affect U.S. election forecasts under a compartmental modeling approach

Ryan Branstetter,Samuel Chian,Joseph Cromp,William L He,Christopher M Lee,Mengqi Liu,Emma Mansell,Manas Paranjape,Thanmaya Pattanashetty,Alexia Rodrigues,Alexandria Volkening

2024-11-04

Abstract:In the months leading up to political elections in the United States, forecasts are widespread and take on multiple forms, including projections of what party will win the popular vote, state ratings, and predictions of vote margins at the state level. It can be challenging to evaluate how accuracy changes in the lead up to Election Day or to put probabilistic forecasts into historical context. Moreover, forecasts differ between analysts, highlighting the many choices in the forecasting process. With this as motivation, here we take a more comprehensive view and begin to unpack some of the choices involved in election forecasting. Building on a prior compartmental model of election dynamics, we present the forecasts of this model across months, years, and types of race. By gathering together monthly forecasts of presidential, senatorial, and gubernatorial races from 2004--2022, we provide a larger-scale perspective and discuss how treating polling data in different ways affects forecast accuracy. We conclude with our 2024 election forecasts (upcoming at the time of writing).

Physics and Society
Forecasting presidential elections: Accuracy of ANES voter intentions

Hyein Ko,Natalie Jackson,Tracy Osborn,Michael S. Lewis-Beck

DOI: https://doi.org/10.1016/j.ijforecast.2024.03.003

IF: 7.022

2024-04-18

International Journal of Forecasting

Abstract:Despite research on the accuracy of polls as tools for forecasting presidential elections, we lack an assessment of how accurately the ANES, arguably the most used survey in political science, measures aggregate vote intention relative to the actual election results. Our ANES 1952–2020 results indicate that the reported vote from the post-election surveys accurately measures the actual vote (e.g., it is off by 2.23 percentage points, on average). Moreover, the intended vote measure from the pre-election surveys reasonably accurately predicts the actual aggregate popular vote outcome. While outliers may exist, they do not appear to come from variations in the survey mode, sample weights, time, political party, or turnout. We conclude that political scientists can confidently use the intended vote measure, keeping in mind that forecasting the popular vote may not always reveal the actual winner.

economics,management
The forecast trap

Carl Boettiger

DOI: https://doi.org/10.1111/ele.14024

Abstract:Encouraged by decision makers' appetite for future information on topics ranging from elections to pandemics, and enabled by the explosion of data and computational methods, model-based forecasts have garnered increasing influence on a breadth of decisions in modern society. Using several classic examples from fisheries management, I demonstrate that selecting the model or models that produce the most accurate and precise forecast (measured by statistical scores) can sometimes lead to worse outcomes (measured by real-world objectives). This can create a forecast trap, in which the outcomes such as fish biomass or economic yield decline while the manager becomes increasingly convinced that these actions are consistent with the best models and data available. The forecast trap is not unique to this example, but a fundamental consequence of non-uniqueness of models. Existing practices promoting a broader set of models are the best way to avoid the trap.
Forecasting Elections with Agent-Based Modeling: Two Live Experiments.

Ming Gao,Zhongyuan Wang,Kai Wang,Chenhui Liu,Shiping Tang

DOI: https://doi.org/10.1371/journal.pone.0270194

IF: 3.7

2022-01-01

PLoS ONE

Abstract:Election forecasting has been traditionally dominated by subjective surveys and polls or methods centered upon them. We have developed a novel platform for forecasting elections based on agent-based modeling (ABM), which is entirely independent from surveys and polls. The platform uses statistical results from objective data along with simulation models to capture how voters have voted in past elections and how they are likely to vote in an upcoming election. We screen for models that can reproduce results that are very close to the actual results of historical elections and then deploy these selected models to forecast an upcoming election with simulations by combining extrapolated data from historical demographic record and more updated data on economic growth, employment, shock events, and other factors. Here, we report the results of two recent experiments of real-time election forecasting: the 2020 general election in Taiwan and six states in the 2020 general election in the United States. Our mostly objective method using ABM may transform how elections are forecasted and studied.
"I Wanted to Predict Elections with Twitter and all I got was this Lousy Paper" -- A Balanced Survey on Election Prediction using Twitter Data

Daniel Gayo-Avello

DOI: https://doi.org/10.48550/arXiv.1204.6441

2012-04-29

Abstract:Predicting X from Twitter is a popular fad within the Twitter research subculture. It seems both appealing and relatively easy. Among such kind of studies, electoral prediction is maybe the most attractive, and at this moment there is a growing body of literature on such a topic. This is not only an interesting research problem but, above all, it is extremely difficult. However, most of the authors seem to be more interested in claiming positive results than in providing sound and reproducible methods. It is also especially worrisome that many recent papers seem to only acknowledge those studies supporting the idea of Twitter predicting elections, instead of conducting a balanced literature review showing both sides of the matter. After reading many of such papers I have decided to write such a survey myself. Hence, in this paper, every study relevant to the matter of electoral prediction using social media is commented. From this review it can be concluded that the predictive power of Twitter regarding elections has been greatly exaggerated, and that hard research problems still lie ahead.

Computers and Society,Computation and Language,Social and Information Networks,Physics and Society
Predicting the 2020 US Presidential Election with Twitter

Michael Caballero

DOI: https://doi.org/10.5121/csit.2021.111006

2021-07-19

Abstract:One major sub-domain in the subject of polling public opinion with social media data is electoral prediction. Electoral prediction utilizing social media data potentially would significantly affect campaign strategies, complementing traditional polling methods and providing cheaper polling in real-time. First, this paper explores past successful methods from research for analysis and prediction of the 2020 US Presidential Election using Twitter data. Then, this research proposes a new method for electoral prediction which combines sentiment, from NLP on the text of tweets, and structural data with aggregate polling, a time series analysis, and a special focus on Twitter users critical to the election. Though this method performed worse than its baseline of polling predictions, it is inconclusive whether this is an accurate method for predicting elections due to scarcity of data. More research and more data are needed to accurately measure this method's overall effectiveness.

Social and Information Networks,Machine Learning
The Risk of Narrow, Disputable Results in the U.S. Electoral College

Michael Geruso,Dean Spears

DOI: https://doi.org/10.1162/rest_a_01418

2024-02-09

Abstract:Abstract Close elections are important for many reasons, including that consequent election disputes can weaken democratic legitimacy and risk political violence. We quantify the probability of close outcomes in US presidential races with novel applications of empirical election models from several sources. We show that razor-thin margins are very likely under the Electoral College (EC). And we establish that the EC causes this closeness: It would not occur under any plausibly comparable popular vote system. The tendency of the EC to generate close elections is found today and throughout US presidential voting history.

economics,social sciences, mathematical methods
Evolving Accuracy: A Genetic Algorithm to Improve Election Night Forecasts

Ronald Hochreiter,Christoph Waldhauser

DOI: https://doi.org/10.48550/arXiv.1401.4674

2014-01-19

Abstract:In this paper, we apply genetic algorithms to the field of electoral studies. Forecasting election results is one of the most exciting and demanding tasks in the area of market research, especially due to the fact that decisions have to be made within seconds on live television. We show that the proposed method outperforms currently applied approaches and thereby provide an argument to tighten the intersection between computer science and social science, especially political science, further. We scrutinize the performance of our algorithm's runtime behavior to evaluate its applicability in the field. Numerical results with real data from a local election in the Austrian province of Styria from 2010 substantiate the applicability of the proposed approach.

Neural and Evolutionary Computing
Democratic forecast: Small groups predict the future better than individuals and crowds.

Guillaume Dezecache,Martin Dockendorff,Dardo N. Ferreiro,Ophelia Deroy,Bahador Bahrami

DOI: https://doi.org/10.1037/xap0000424

2022-09-09

Abstract:Predictions pose unique problems. Experts regularly get them wrong, and collective solutions (such as prediction markets and super-forecaster schemes) do better but remain selective and costly. Contrary to the idea that face-to-face discussion hinders collective intelligence, social deliberation improves the resolution of general knowledge problems, with four consensually agreed answers outperforming the aggregate knowledge of 5,000 nondeliberating individuals. Could discussion help predict the future in an efficient, cheap, and inclusive way? We show that smaller groups of lay individuals, when organized, come up with better predictions than those they provide alone. Deliberation and consensus made individual predictions significantly more accurate. Aggregating as few as two consensual predictions did better than classical "wisdom of crowds" aggregation of 100 individual ones. Against the view that discussion can impair decision-making, our results demonstrate that collective intelligence of small groups and consensus-seeking improves accuracy about yet unknown facts, opening the avenue for efficient, inclusive, and inexpensive group forecasting solutions. (PsycInfo Database Record (c) 2022 APA, all rights reserved)
Persuasion, Betrayal and Regret in Election Campaigns

M. Andrecut

2024-07-31

Abstract:Elections play a fundamental role in democratic societies, however they are often characterized by unexpected results. Here we discuss an election campaign model inspired by the compartmental epidemiology, and we show that the model captures the main characteristics of an election campaign: persuasion, betrayal and regret. All of these three factors can be used together or independently to influence the campaign, and to determine the winner. We include results for both the deterministic and the stochastic versions of the model, and we show that the decision to not vote significantly increases the fluctuations in the model, amplifying the chance of controversial results, in agreement with the well known "paradox of not voting".

Physics and Society
Electoral and Public Opinion Forecasts with Social Media Data: A Meta-Analysis

Marko M. Skoric,Jing Liu,Kokil Jaidka

DOI: https://doi.org/10.3390/info11040187

2020-03-31

Information

Abstract:In recent years, many studies have used social media data to make estimates of electoral outcomes and public opinion. This paper reports the findings from a meta-analysis examining the predictive power of social media data by focusing on various sources of data and different methods of prediction; i.e., (1) sentiment analysis, and (2) analysis of structural features. Our results, based on the data from 74 published studies, show significant variance in the accuracy of predictions, which were on average behind the established benchmarks in traditional survey research. In terms of the approaches used, the study shows that machine learning-based estimates are generally superior to those derived from pre-existing lexica, and that a combination of structural features and sentiment analyses provides the most accurate predictions. Furthermore, our study shows some differences in the predictive power of social media data across different levels of political democracy and different electoral systems. We also note that since the accuracy of election and public opinion forecasts varies depending on which statistical estimates are used, the scientific community should aim to adopt a more standardized approach to analyzing and reporting social media data-derived predictions in the future.
A Hybrid Method of Sentiment Analysis and Machine Learning Algorithm for the U.S. Presidential Election Forecasting

Guocheng Feng,Huaiyu Cai,Kaihao Chen,Zhijian Li

DOI: https://doi.org/10.48550/arXiv.2312.05584

2023-12-09

Social and Information Networks

Abstract:U.S. Presidential Election forecasting has been a research interest for several decades. Currently, election prediction consists of two main approaches: traditional models that incorporate economic data and poll surveys, and models that leverage Twitter (or X) and other social media platforms due to their increasing popularity in the past decade. However, traditional approaches have predominantly focused on national-level predictions, while social media-based approaches often oversimplify the nuanced differences between online discourse and the broader voting population's political landscape. In this work, we perform a hybrid method of both the machine learning algorithm and the sentiment analysis on the state level with various independent variables including census data, economic indicators, polling averages, and the newly defined average sentiment scores from Twitter. Our prediction for the 2020 U.S. Presidential Election yielded promising results. Most of our models successfully predicted a victory for the Democratic candidate with 96% accuracy using Gradient Boosting Trees and Multi-Layer Perceptron algorithms. This novel prediction framework addresses the limitations of existing U.S. Presidential Election forecasting approaches, particularly in terms of state-level predictions. It provides a valuable foundation for future research in this field and contributes to advancing our understanding of election dynamics.
Artificial intelligence for elections: the case of 2019 Argentina primary and presidential election

Zhenkun Zhou,Hernan A. Makse

DOI: https://doi.org/10.48550/arXiv.1910.11227

2019-10-24

Abstract:We use a method based on machine learning, big-data analytics, and network theory to process millions of messages posted in Twitter to predict election outcomes. The model has achieved accurate results in the current Argentina primary presidential election on August 11, 2019 by predicting the large difference win of candidate Alberto Fernandez over president Mauricio Macri; a result that none of the traditional pollsters in that country was able to predict, and has led to a major bond market collapse. We apply the model to the upcoming Argentina presidential election on October 27, 2019 yielding the following results: Fernandez 47.5%, Macri 30.9% and third party 21.6%. Our method improves over traditional polling methods which are based on direct interactions with small number of individuals that are plagued by ever declining response rates, currently falling in the low single digits. They provide a reliable polling method that can be applied not only to predict elections but to discover any trend in society, for instance, what people think about climate change, politics or education.

Social and Information Networks,Physics and Society
Forecasting elections from partial information using a Bayesian model for a multinomial sequence of data

Soudeep Deb,Rishideep Roy,Shubhabrata Das

DOI: https://doi.org/10.1002/for.3107

2024-03-05

Journal of Forecasting

Abstract:Predicting the winner of an election is of importance to multiple stakeholders. To formulate the problem, we consider an independent sequence of categorical data with a finite number of possible outcomes in each. The data is assumed to be observed in batches, each of which is based on a large number of such trials and can be modeled via multinomial distributions. We postulate that the multinomial probabilities of the categories vary randomly depending on batches. The challenge is to predict accurately on cumulative data based on data up to a few batches as early as possible. On the theoretical front, we first derive sufficient conditions of asymptotic normality of the estimates of the multinomial cell probabilities and present corresponding suitable transformations. Then, in a Bayesian framework, we consider hierarchical priors using multivariate normal and inverse Wishart distributions and establish the posterior convergence. The desired inference is arrived at using these results and ensuing Gibbs sampling. The methodology is demonstrated with election data from two different settings—one from India and the other from the United States. Additional insights of the effectiveness of the proposed methodology are attained through a simulation study.

economics,management
Crowd prediction systems: Markets, polls, and elite forecasters

Pavel Atanasov,Jens Witkowski,Barbara Mellers,Philip Tetlock

DOI: https://doi.org/10.1016/j.ijforecast.2023.12.009

IF: 7.022

2024-01-24

International Journal of Forecasting

Abstract:What systems should we use to elicit and aggregate judgmental forecasts? Who should be asked to make such forecasts? We address these questions by assessing two widely used crowd prediction systems: prediction markets and prediction polls. Our main test compares a prediction market against team-based prediction polls, using data from a large, multi-year forecasting competition. Each of these two systems uses inputs from either a large, sub-elite or a small, elite crowd. We find that small, elite crowds outperform larger ones, whereas the two systems are statistically tied. In addition to this main research question, we examine two complementary questions. First, we compare two market structures—continuous double auction (CDA) markets and logarithmic market scoring rule (LMSR) markets—and find that the LMSR market produces more accurate forecasts than the CDA market, especially on low-activity questions. Second, given the importance of elite forecasters, we compare the talent-spotting properties of the two systems and find that markets and polls are equally effective at identifying elite forecasters. Overall, the performance benefits of "superforecasting" hold across systems. Managers should move towards identifying and deploying small, select crowds to maximize forecasting performance.

economics,management
Forecasting election results by studying brand importance in online news

A. Fronzetti Colladon

DOI: https://doi.org/10.1016/j.ijforecast.2019.05.013

2021-05-13

Abstract:This study uses the semantic brand score, a novel measure of brand importance in big textual data, to forecast elections based on online news. About 35,000 online news articles were transformed into networks of co-occurring words and analyzed by combining methods and tools from social network analysis and text mining. Forecasts made for four voting events in Italy provided consistent results across different voting systems: a general election, a referendum, and a municipal election in two rounds. This work contributes to the research on electoral forecasting by focusing on predictions based on online big data; it offers new perspectives regarding the textual analysis of online news through a methodology which is relatively fast and easy to apply. This study also suggests the existence of a link between the brand importance of political candidates and parties and electoral results.

Social and Information Networks,Computation and Language,Physics and Society
KOALA: A new paradigm for election coverage

Alexander Bauer,Andreas Bender,André Klima,Helmut Küchenhoff

DOI: https://doi.org/10.48550/arXiv.1807.09665

2018-07-15

Abstract:Common election poll reporting is often misleading as sample uncertainty is addressed insufficiently or not covered at all. Furthermore, main interest usually lies beyond the simple party shares. For a more comprehensive opinion poll and election coverage, we propose shifting the focus towards the reporting of survey-based probabilities for specific events of interest. We present such an approach for multi-party electoral systems, focusing on probabilities of coalition majorities. A Monte Carlo approach based on a Bayesian Multinomial-Dirichlet model is used for estimation. Probabilities are estimated, assuming the election was held today (''now-cast''), not accounting for potential shifts in the electorate until election day ''fore-cast''. Since our method is based on the posterior distribution of party shares, the approach can be used to answer a variety of questions related to the outcome of an election. We also introduce visualization techniques that facilitate a more adequate depiction of relevant quantities as well as respective uncertainties. The benefits of our approach are discussed by application to the German federal elections in 2013 and 2017. An open source implementation of our methods is freely available in the R package coalitions.

Applications

Why forecast an election that's too close to call?

Modeling undecided voters to forecast elections: From bandwagon behavior and the spiral of silence perspective

Developing Spatio-Temporal Approach to Predict Economic Dynamics Based on Online News

Why polls fail to predict elections

How time and pollster history affect U.S. election forecasts under a compartmental modeling approach

Forecasting presidential elections: Accuracy of ANES voter intentions

The forecast trap

Forecasting Elections with Agent-Based Modeling: Two Live Experiments.

"I Wanted to Predict Elections with Twitter and all I got was this Lousy Paper" -- A Balanced Survey on Election Prediction using Twitter Data

Predicting the 2020 US Presidential Election with Twitter

The Risk of Narrow, Disputable Results in the U.S. Electoral College

Evolving Accuracy: A Genetic Algorithm to Improve Election Night Forecasts

Democratic forecast: Small groups predict the future better than individuals and crowds.

Persuasion, Betrayal and Regret in Election Campaigns

Electoral and Public Opinion Forecasts with Social Media Data: A Meta-Analysis

A Hybrid Method of Sentiment Analysis and Machine Learning Algorithm for the U.S. Presidential Election Forecasting

Artificial intelligence for elections: the case of 2019 Argentina primary and presidential election

Forecasting elections from partial information using a Bayesian model for a multinomial sequence of data

Crowd prediction systems: Markets, polls, and elite forecasters

Forecasting election results by studying brand importance in online news

KOALA: A new paradigm for election coverage