Abstract:With the increasing popularity and social influence of search engines in IR, various studies have raised concerns on the presence of bias in search engines and the social responsibilities of IR systems. As an essential component of search engine, ranking is a crucial mechanism in presenting the search results or recommending items in a fair fashion. In this article, we focus on the top-k diversity fairness ranking in terms of statistical parity fairness and disparate impact fairness. The former fairness definition provides a balanced overview of search results where the number of documents from different groups are equal; The latter enables a realistic overview where the proportion of documents from different groups reflect the overall proportion. Using 100 queries and top 100 results per query from Google as the data, we first demonstrate how topical diversity bias is present in the top web search results. Then, with our proposed entropy-based metrics for measuring the degree of bias, we reveal that the top search results are unbalanced and disproportionate to their overall diversity distribution. We explore several fairness ranking strategies to investigate the relationship between fairness, diversity, novelty and relevance. Our experimental results show that using a variant of fair ε-greedy strategy, we could bring more fairness and enhance diversity in search results without a cost of relevance. In fact, we can improve the relevance and diversity by introducing the diversity fairness. Additional experiments with TREC datasets containing 50 queries demonstrate the robustness of our proposed strategies and our findings on the impact of fairness. We present a series of correlation analysis on the amount of fairness and diversity, showing that statistical parity fairness highly correlates with diversity while disparate impact fairness does not. This provides clear and tangible implications for future works where one would want to balance fairness, diversity and relevance in search results.

Ranking Search Intents Underlying a Query.

Search Result Diversity Evaluation Based on Intent Hierarchies.

Evaluating Search Result Diversity Using Intent Hierarchies.

Search Result Diversification Based on Hierarchical Intents.

Mining and ranking users’ intents behind queries

Affinity rank: a new scheme for efficient web search.

Estimating Intent Types for Search Result Diversification.

Low-cost, Bottom-Up Measures for Evaluating Search Result Diversification

Evaluating diversified search results using per-intent graded relevance.

Modeling Intent Graph for Search Result Diversification

Improve Web Search Diversification with Intent Subtopic Mining

Simple Evaluation Metrics for Diversified Search Results.

Ranking with Diverse Intents and Correlated Contents.

Search Result Diversification Based on Query Facets

Enhancing Web Search with Queries of Equivalent Intents

Detecting Multilingual and Multi-Regional Query Intent in Web Search

User Session Level Diverse Reranking of Search Results

Beyond Greedy Search: Pruned Exhaustive Search for Diversified Result Ranking.

QUIDS: Query Intent Generation via Dual Space Modeling

Toward creating a fairer ranking in search engine results

Ranking Relevance in Yahoo Search