On Tables with Numbers, with Numbers

Konstantinos Kogkalidis,Stergios Chatzikyriakidis

2024-08-28

Abstract:This paper is a critical reflection on the epistemic culture of contemporary computational linguistics, framed in the context of its growing obsession with tables with numbers. We argue against tables with numbers on the basis of their epistemic irrelevance, their environmental impact, their role in enabling and exacerbating social inequalities, and their deep ties to commercial applications and profit-driven research. We substantiate our arguments with empirical evidence drawn from a meta-analysis of computational linguistics research over the last decade.

Computation and Language

What problem does this paper attempt to address?

The paper attempts to address the issue of the over-reliance on tables with numbers in contemporary computational linguistics. The authors argue that this phenomenon presents problems in terms of scientific epistemology, environmental impact, social inequality, and close ties to commercial interests, and they support their viewpoint through a meta-analysis of computational linguistics research over the past 10 years.

Specifically, the paper focuses on the following aspects:

1. **Epistemological Premises**: Exploring how tables with numbers have become the "currency" in scientific research and what mechanisms influence the actual value of these tables.
2. **Environmental Footprint**: Discussing the environmental costs generated by large-scale experiments and comparative evaluations, especially as model sizes increase and benchmarks become more complex.
3. **Social Inequality**: Analyzing how tables with numbers exacerbate and perpetuate inequalities and harmful power structures, particularly the gap between resource-rich and resource-poor institutions.
4. **Ties to Commercial Interests**: Investigating the intrinsic connections between tables with numbers and corporate interests, profits, and the accumulation of technological capital.

Through these analyses, the authors hope to provoke reflection on current research methods in computational linguistics and propose more comprehensive and responsible research directions.