Abstract:Crowdsourced software development (CSSD) is a fast-growing field among software practitioners and researchers from the last two decades. Despite being a favorable environment, no intelligent mechanism exists to assign price to CSSD tasks. Software development effort estimation (SDEE) on the other hand is already an established field in traditional software engineering. SDEE is largely facilitated by machine learning (ML), particularly, ML-based ensemble effort estimation (EEE) which targets accurate estimate by avoiding biases of single ML model. This accuracy of EEE can be exploited for CSSD platforms to establish intelligent cost assignment mechanism. This study aims to integrate EEE with CSSD platform to provide justified costing solution for crowdsourced tasks. Effort-based cost estimation model is proposed, implementing EEE to predict task's effort along with natural language processing (NLP) analysis on task's textual description to assign effort-based cost. TopCoder is selected as targeted CSSD platform, and the proposed scheme is implemented on TopCoder QA category comprising software testing tasks. Ensemble prediction is incorporated via random forest, support vector machine and neural network as base learners. LDA topic modeling is utilized for NLP analysis on the textual aspects of CSSD task, with a specific emphasis on the testing and technology factors. Effort estimation results confirm that EEE models, particularly stacking and weighted ensemble, surpass their base learners with 50% overall increased accuracy. Moreover, R 2 , log-likelihood and topic quality measures confirm considerable LDA model significance. Findings confirmed that cost adjustment achieved from EEE and NLP defines acceptable price range, covering major testing aspects.

How Much Should I Pay? An Empirical Analysis on Monetary Prize in TopCoder

Software crowdsourcing task pricing based on topic model analysis

A Brief Survey of Data Pricing for Machine Learning

Analysis of Task Pricing Under Crowdsourcing Mode

An Empirical Investigation of Worker Communities in TopCoder

The Labor Economics of Paid Crowdsourcing

A Posted Pricing Mechanism Based on Random Forests in Crowdsourcing Market

Analysis of the Pricing Problem of Photographic Tasks Based on Multiple Linear Regression Models

A Expectation-Aware Dynamic Pricing Model for Spatial Crowdsourcing Platforms

Task Price Prediction Based on Clustering and DNN in Crowdsensing

Different Ways of Research on Pricing Done Math Model Base on Big Data

An online cost sensitive decision-making method in crowdsourcing systems.

Optimal Computing Resource Management Based on Utility Maximization in Mobile Crowdsourcing

Research on Task Pricing of Crowdsourcing Platform

Task Re-Pricing Model Based on Density-Based Spatial Clustering of Applications

Pricing Research of ‘Photo-Earning’ Task Based on Logistic Regression and BP Neural Network

Optimal Posted-Price Mechanism in Microtask Crowdsourcing

Dynamic pricing in profit-driven task assignment: a domain-of-influence based approach

On Dynamically Pricing Crowdsourcing Tasks

Cost Adjustment for Software Crowdsourcing Tasks Using Ensemble Effort Estimation and Topic Modeling

An Efficient and Truthful Pricing Mechanism for Team Formation in Crowdsourcing Markets