Sample Selection for Maximal Diversity

Feng Pan,Adam Roberts,Leonard McMillan,Fernando Pardo Manuel de Villena,David Threadgill,Wei Wang

DOI: https://doi.org/10.1109/icdm.2007.16

2007-01-01

Abstract:The problem of selecting a sample subset sufficient to preserve diversity arises in many applications. One example is in the design of recombinant inbred lines (RIL) for genetic association studies. In this context, genetic diversity is measured by how many alleles are retained in the resulting inbred strains. RIL panels that are derived from more than two parental strains, such as the Collaborative Cross [2, 14], present a particular challenge with regard to which of the many existing lab mouse strains should be included in the initial breeding funnel in order to maximize allele retention. A similar problem occurs in the study of customer reviews when selecting a subset of products with a maximal diversity, in reviews. Diversity, in this case implies the presence of a set of products having both positive and negative ranks for each customer In this paper we demonstrate that selecting an optimal diversity subset is an NP-complete problem via reduction to set cover This reduction is sufficiently tight that greedy approximations to the set cover problem directly apply to maximizing diversity. We then suggest a slightly modified subset selection problem in which an initial greedy diversity solution is used to effectively prune an exhaustive search for all diversity subsets bounded from below by a specified coverage threshold. Extensive experiments on real datasets are performed to demonstrate the effectiveness and efficiency of our approach.