Abstract:Crowdsourced software testing (short for crowdtesting) is a special type of crowdsourcing. It requires that crowdworkers master appropriate skill-sets and commit significant effort for completing a task. Abundant uncertainty may arise during a crowdtesting process due to imperfect information between the task requester and crowdworkers. For example, a worker frequently chooses tasks in an ad hoc manner in crowdtesting context, and an inappropriate task selection may lead to the worker's failing to detect any bugs, and significant testing effort unpaid and wasted. Recent studies have explored methods for supporting task requesters to make informed decisions on task pricing, worker recommendation, and so on. Unfortunately, very few study offers decision making support from the crowdworkers' perspectives. We motivate this study through a pilot study, revealing the large portion (74 percent) of unpaid crowdworkers' effort due to the inappropriate task choice. Drawn from our previous work on context-aware crowdworker recommendations, we advocate a more effective alternative to manual task selection would be to provide contextualized and personalized task recommendation considering the diverse distribution of worker preference and expertise, with objectives to increase their winning chances and to potentially reduce the frequency of unpaid crowd work. This paper proposes a context-aware personalized task recommendation approach PTRec, consisting of a testing context model and a learning-based task recommendation model to aid dynamic worker decision in selecting crowdtesting tasks. The testing context model is constructed in two perspectives, i.e., process context and resource context, to capture the in-process progress-oriented information and crowdworkers' characteristics respectively. Built on top of this context model, the learning-based task recommendation model extracts 60 f-atures automatically, and employs random forest learner to generate dynamic and personalized task recommendation which matches workers' expertise and interest. The evaluation is conducted on 636 crowdtesting tasks involving 2,404 crowdworkers from one of the largest crowdtesting platforms, and results show our approach can achieve an average precision of 82 percent, average recall of 84 percent, and save an estimated average of 81 percent effort originally spent on exploring, significantly outperforming four commonly-used and state-of-the-art baselines. This indicates its potential in recommending proper tasks to workers so as to improve bug detection efficiency and increase their monetary earnings.

Test Case Recommendation Based on Balanced Distance of Test Targets

Code Reviewer Recommendation in Tencent: Practice, Challenge, and Direction*

MUTCR: Test Case Recommendation via Multi-Level Signature Matching

Code Reviewer Recommendation in Tencent

Recommending Code Reviewers for Proprietary Software Projects: A Large Scale Study

Test Case Recommendations with Distributed Representation of Code Syntactic Features

Who Should Review This Change?: Putting Text and File Location Analyses Together for More Accurate Recommendations

HomoTR

Dual Analysis for Recommending Developers to Resolve Bugs

Test Case Reuse Based on Software Testing Knowledge Graph and Collaborative Filtering Recommendation Algorithm

Test Case Generation Evaluator for the Implementation of Test Case Generation Algorithms Based on Learning to Rank.

Test recommendation system based on slicing coverage filtering

A Collaborative Filtering Recommender System for Test Case Prioritization in Web Applications

RTCM: a Natural Language Based, Automated, and Practical Test Case Generation Framework

Testability-Driven Development: An Improvement to the TDD Efficiency

Quality Assessment of Crowdsourced Test Cases

Diversified and Scalable Service Recommendation with Accuracy Guarantee.

Context-aware Personalized Crowdtesting Task Recommendation

A Social Balance Theory-Based Service Recommendation Approach.

LTM: Scalable and Black-box Similarity-based Test Suite Minimization based on Language Models

TDMatcher: A Topic-Based Approach to Task-Developer Matching with Predictive Intelligence for Recommendation.