Abstract:Each year, selective American colleges sort through tens of thousands of applications to identify a first-year class that displays both academic merit and diversity. In the 2023-2024 admissions cycle, these colleges faced unprecedented challenges. First, the number of applications has been steadily growing. Second, test-optional policies that have remained in place since the COVID-19 pandemic limit access to key information historically predictive of academic success. Most recently, longstanding debates over affirmative action culminated in the Supreme Court banning race-conscious admissions. Colleges have explored machine learning (ML) models to address the issues of scale and missing test scores, often via ranking algorithms intended to focus on 'top' applicants. However, the Court's ruling will force changes to these models, which were able to consider race as a factor in ranking. There is currently a poor understanding of how these mandated changes will shape applicant ranking algorithms, and, by extension, admitted classes. We seek to address this by quantifying the impact of different admission policies on the applications prioritized for review. We show that removing race data from a developed applicant ranking algorithm reduces the diversity of the top-ranked pool without meaningfully increasing the academic merit of that pool. We contextualize this impact by showing that excluding data on applicant race has a greater impact than excluding other potentially informative variables like intended majors. Finally, we measure the impact of policy change on individuals by comparing the arbitrariness in applicant rank attributable to policy change to the arbitrariness attributable to randomness. We find that any given policy has a high degree of arbitrariness and that removing race data from the ranking algorithm increases arbitrariness in outcomes for most applicants.

Impact of R1 classification on universities

Putting the K in Rank: How Kindergarten Classrooms Impact Short and Long-Run Outcomes

Inter-ranking reputational effects: an analysis of the Academic Ranking of World Universities (ARWU) and the Times Higher Education World University Rankings (THE) reputational relationship

The Effects of International Mergers and Acquisitions on Innova tive Financial Performance for the Higher Education Industry

A Longitudinal Analysis of University Rankings

Algorithms for College Admissions Decision Support: Impacts of Policy Change and Inherent Variability

The impact of being named the top party school on peer rankings and the academic profile of a university

Student Characteristics, Institutional Factors, and Outcomes in Higher Education and Beyond: An Analysis of Standardized Test Scores and Other Factors at the Institutional Level with School Rankings and Salary

Analyzing the disciplinary focus of universities: Can rankings be a one-size-fits-all?

Statistical Significance and Effect Sizes of Differences among Research Universities at the Level of Nations and Worldwide based on the Leiden Rankings

An Investigation of a Multiple-Measures Teaching Evaluation System and Its Relationship With Students’ College-Going Outcomes

The role of conference externalities and other factors in determining the annual recruiting rankings of football bowl subdivision (FBS) teams

University Rankings:Either Raising Level or Optimizing System: The Review on Ellen Hazelkorn's Rankings Views

Commensuration of the globalised higher education sector: how university rankings act as a credential for world-class status in China

The Big Sort: College Reputation and Labor Market Outcomes

Isolating Peer Effects in the Returns to College Selectivity

What do university rankings by fields rank? Exploring discrepancies between the organizational structure of universities and bibliometric classifications

Assessing the Regional Economic Development Impacts of Universities: A Review of Current Approaches

The Returns to Elite College Education: A Quasi-Experimental Analysis

Strategic Ranking

University Admission: Is Achievement a Sufficient Criterion?