Abstract:Crowdsourced selection asks the crowd to select entities that satisfy a query condition, e.g., selecting the photos of people wearing sunglasses from a given set of photos. Existing studies focus on a single query predicate and in this paper we study the crowdsourced selection problem on multi-attribute data, e.g., selecting the female photos with dark eyes and wearing sunglasses. A straightforward method asks the crowd to answer every entity by checking every predicate in the query. Obviously, this method involves huge monetary cost. Instead, we can select an optimized predicate order and ask the crowd to answer the entities following the order. Since if an entity does not satisfy a predicate, we can prune this entity without needing to ask other predicates and thus this method can reduce the cost. There are two challenges in finding the optimized predicate order. The first is how to detect the predicate order and the second is to capture correlation among different predicates. To address this problem, we propose predicate order based framework to reduce monetary cost. Firstly, we define an expectation tree to store selectivities on predicates and estimate the best predicate order. In each iteration, we estimate the best predicate order from the expectation tree, and then choose a predicate as a question to ask the crowd. After getting the result of the current predicate, we choose next predicate to ask until we get the result. We will update the expectation tree using the answer obtained from the crowd and continue to the next iteration. We also study the problem of answering multiple queries simultaneously, and reduce its cost using the correlation between queries. Finally, we propose a confidence based method to improve the quality. The experiment result shows that our predicate order based algorithm is effective and can reduce cost significantly compared with baseline approaches.

A Sampling-Based Framework For Crowdsourced Select Query With Multiple Predicates

Crowdsourced query optimization for selection query with mul-tiple predicates

Crowdsourced Selection on Multi-Attribute Data

Stable Job Assignment for Crowdsourcing.

Crowdsourced Top- K Queries by Pairwise Preference Judgments with Confidence and Budget Control

A LDA Topic Model Based Collection Selection Method for Distributed Information Retrieval

Distribution-Aware Crowdsourced Entity Collection

Answering Skyline Queries over Incomplete Data with Crowdsourcing

Crowd-Selection Query Processing in Crowdsourcing Databases: A Task-Driven Approach.

Leveraging Transitive Relations for Crowdsourced Joins

A Partial-Order-based Framework for Cost-Effective Crowdsourced Entity Resolution

Random Sampling over Joins Revisited

SAQP++: Bridging the Gap Between Sampling-Based Approximate Query Processing and Aggregate Precomputation.

Cross-domain-aware Worker Selection with Training for Crowdsourced Annotation

CrowdRecruiter: Selecting Participants for Piggyback Crowdsensing under Probabilistic Coverage Constraint

Random Sampling for Group-By Queries

Crowdsourcing-assisted Query Structure Interpretation

Analysis of Crowdsourced Sampling Strategies for HodgeRank with Sparse Random Graphs

Crowdsourced Query Processing on Microblogs.

Selectivity Estimation by Batch-Query Based Histogram and Parametric Method.

Adaptive Sampling For Selectivity Estimation In Spatial Database