Abstract:We propose measurement modeling from the quantitative social sciences as a framework for understanding fairness in computational systems. Computational systems often involve unobservable theoretical constructs, such as socioeconomic status, teacher effectiveness, and risk of recidivism. Such constructs cannot be measured directly and must instead be inferred from measurements of observable properties (and other unobservable theoretical constructs) thought to be related to them -- i.e., operationalized via a measurement model. This process, which necessarily involves making assumptions, introduces the potential for mismatches between the theoretical understanding of the construct purported to be measured and its operationalization. We argue that many of the harms discussed in the literature on fairness in computational systems are direct results of such mismatches. We show how some of these harms could have been anticipated and, in some cases, mitigated if viewed through the lens of measurement modeling. To do this, we contribute fairness-oriented conceptualizations of construct reliability and construct validity that unite traditions from political science, education, and psychology and provide a set of tools for making explicit and testing assumptions about constructs and their operationalizations. We then turn to fairness itself, an essentially contested construct that has different theoretical understandings in different contexts. We argue that this contestedness underlies recent debates about fairness definitions: although these debates appear to be about different operationalizations, they are, in fact, debates about different theoretical understandings of fairness. We show how measurement modeling can provide a framework for getting to the core of these debates.

On the mathematical modelling of measurement

An Explicit Solution for Nonlinear Plasma Waves of Arbitrary Amplitude

A Model of Classical and Quantum Measurement

A review of the fundamental concepts of measurement

On the geometry of physical measurements: topological and algebraic aspects

What Is Mathematical Modeling?

An abstract theory of physical measurements

The Scaled Uniform Model Revisited

Measurement and Fairness

Understanding Measurement in Light of Its Origins

Mathematical Modelling by Help of Category Theory: Models and Relations between Them

Constructing solvable models of the quantum measurement process

Modelling of measuring systems -- From white box models to cognitive approaches

Data-driven Modeling in Metrology -- A Short Introduction, Current Developments and Future Perspectives

Calibrating the theory of model mediated measurement: metrological extension, dimensional analysis, and high pressure physics

On the lattice of conceptual measurements

Mathematical Modeling of Physical Reality: From Numbers to Fractals, Quantum Mechanics and the Standard Model

Implementing measurement error models with mechanistic mathematical models in a likelihood-based framework for estimation, identifiability analysis, and prediction in the life sciences

Mesoscopic Bayesian Inference by Solvable Models

On Measurements, Numbers and p-Adic Mathematical Physics

Calibrating Model-Based Inferences and Decisions