Abstract:Crowdsourced software development (CSSD) is a fast-growing field among software practitioners and researchers from the last two decades. Despite being a favorable environment, no intelligent mechanism exists to assign price to CSSD tasks. Software development effort estimation (SDEE) on the other hand is already an established field in traditional software engineering. SDEE is largely facilitated by machine learning (ML), particularly, ML-based ensemble effort estimation (EEE) which targets accurate estimate by avoiding biases of single ML model. This accuracy of EEE can be exploited for CSSD platforms to establish intelligent cost assignment mechanism. This study aims to integrate EEE with CSSD platform to provide justified costing solution for crowdsourced tasks. Effort-based cost estimation model is proposed, implementing EEE to predict task's effort along with natural language processing (NLP) analysis on task's textual description to assign effort-based cost. TopCoder is selected as targeted CSSD platform, and the proposed scheme is implemented on TopCoder QA category comprising software testing tasks. Ensemble prediction is incorporated via random forest, support vector machine and neural network as base learners. LDA topic modeling is utilized for NLP analysis on the textual aspects of CSSD task, with a specific emphasis on the testing and technology factors. Effort estimation results confirm that EEE models, particularly stacking and weighted ensemble, surpass their base learners with 50% overall increased accuracy. Moreover, R 2 , log-likelihood and topic quality measures confirm considerable LDA model significance. Findings confirmed that cost adjustment achieved from EEE and NLP defines acceptable price range, covering major testing aspects.

Software crowdsourcing task pricing based on topic model analysis

Task Assignment with Guaranteed Quality for Crowdsourcing Platforms.

Analysis of Task Pricing Under Crowdsourcing Mode

Dynamic pricing in profit-driven task assignment: a domain-of-influence based approach

How Much Should I Pay? An Empirical Analysis on Monetary Prize in TopCoder

A Expectation-Aware Dynamic Pricing Model for Spatial Crowdsourcing Platforms

Research on Task Pricing of Crowdsourcing Platform

On Dynamically Pricing Crowdsourcing Tasks

Profit-driven Task Assignment in Spatial Crowdsourcing

Different Ways of Research on Pricing Done Math Model Base on Big Data

Task Re-Pricing Model Based on Density-Based Spatial Clustering of Applications

An online cost sensitive decision-making method in crowdsourcing systems.

Optimal Computing Resource Management Based on Utility Maximization in Mobile Crowdsourcing

Dynamic Task Pricing in Multi-Requester Mobile Crowd Sensing with Markov Correlated Equilibrium

Analysis of the Pricing Problem of Photographic Tasks Based on Multiple Linear Regression Models

Profit Optimization in Spatial Crowdsourcing: Effectiveness and Efficiency

Explicit Assignment and Dynamic Pricing of Macro Online Tasks in Spatial Crowdsourcing

Consumption Pricing Mechanism of Scientific and Technological Resources Based on Multi-Agent Game Theory: an Interactive Analytical Model and Experimental Validation

A Posted Pricing Mechanism Based on Random Forests in Crowdsourcing Market

An Efficient and Truthful Pricing Mechanism for Team Formation in Crowdsourcing Markets

Cost Adjustment for Software Crowdsourcing Tasks Using Ensemble Effort Estimation and Topic Modeling