Abstract:Each year, selective American colleges sort through tens of thousands of applications to identify a first-year class that displays both academic merit and diversity. In the 2023-2024 admissions cycle, these colleges faced unprecedented challenges. First, the number of applications has been steadily growing. Second, test-optional policies that have remained in place since the COVID-19 pandemic limit access to key information historically predictive of academic success. Most recently, longstanding debates over affirmative action culminated in the Supreme Court banning race-conscious admissions. Colleges have explored machine learning (ML) models to address the issues of scale and missing test scores, often via ranking algorithms intended to focus on 'top' applicants. However, the Court's ruling will force changes to these models, which were able to consider race as a factor in ranking. There is currently a poor understanding of how these mandated changes will shape applicant ranking algorithms, and, by extension, admitted classes. We seek to address this by quantifying the impact of different admission policies on the applications prioritized for review. We show that removing race data from a developed applicant ranking algorithm reduces the diversity of the top-ranked pool without meaningfully increasing the academic merit of that pool. We contextualize this impact by showing that excluding data on applicant race has a greater impact than excluding other potentially informative variables like intended majors. Finally, we measure the impact of policy change on individuals by comparing the arbitrariness in applicant rank attributable to policy change to the arbitrariness attributable to randomness. We find that any given policy has a high degree of arbitrariness and that removing race data from the ranking algorithm increases arbitrariness in outcomes for most applicants.

Inside the Black Box: Detecting and Mitigating Algorithmic Bias across Racialized Groups in College Student-Success Prediction

Evaluation of Fairness Trade-offs in Predicting Student Success

Auditing the Imputation Effect on Fairness of Predictive Analytics in Higher Education

Bias Analysis of AI Models for Undergraduate Student Admissions

Evaluating Algorithmic Bias in Models for Predicting Academic Performance of Filipino Students

De-Biasing the Bias: Methods for Improving Disparity Assessments with Noisy Group Measurements

Algorithms for College Admissions Decision Support: Impacts of Policy Change and Inherent Variability

Evaluating Recommendation System Bias on a Public Dataset based on Demographic Information

A statistical framework for fair predictive algorithms

Why Is My Classifier Discriminatory?

Algorithmic Fairness in Education

An algorithm for removing sensitive information: application to race-independent recidivism prediction

Algorithms that "Don't See Color": Comparing Biases in Lookalike and Special Ad Audiences

Racial Bias in Clinical and Population Health Algorithms: A Critical Review of Current Debates

Comprehensive analysis of factors influencing the real-world application of machine learning for student success rate calculation and their impacts on student achievement & educational institutions

Protected Attributes Tell Us Who, Behavior Tells Us How: A Comparison of Demographic and Behavioral Oversampling for Fair Student Success Modeling

Biasly: a machine learning based platform for automatic racial discrimination detection in online texts

An AI-Guided Data Centric Strategy to Detect and Mitigate Biases in Healthcare Datasets

Algorithmic Bias in Recidivism Prediction: A Causal Perspective

An Introduction to Artificial Intelligence and Solutions to the Problems of Algorithmic Discrimination

Biased Programmers? Or Biased Data? A Field Experiment in Operationalizing AI Ethics