Crowdsourcing: A taxonomy and systematic mapping study
Mahmood Hosseini,Alimohammad Shahri,Keith Phalp,Jacqui Taylor,Raian Ali
DOI: https://doi.org/10.1016/j.cosrev.2015.05.001
IF: 8.757
2015-08-01
Computer Science Review
Abstract:Context:Crowdsourcing, or tapping into the power of the crowd for problem solving, has gained ever-increasing attraction since it was first introduced. Crowdsourcing has been used in different disciplines, and it is becoming well-accepted in the marketplace as a new business model which utilizes Human Intelligence Tasks (HITs).Objective:While both academia and industry have extensively delved into different aspects of crowdsourcing, there seems to be no common understanding of what crowdsourcing really means and what core and optional features it has. Also, we still lack information on the kinds and disciplines of studies conducted on crowdsourcing and how they defined it in the context of their application area. This paper will clarify this ambiguity by analysing the distribution and demographics of research in crowdsourcing and extracting taxonomy of the variability and commonality in the constructs defining the concept in the literature.Method:We conduct a systematic mapping study and analyse 113 papers, selected via a formal process, and report and discuss the results. The study is combined by a content analysis process to extract a taxonomy of features describing crowdsourcing.Results:We extract and describe the taxonomy of features which characterize crowdsourcing in its four constituents; the crowd, the crowdsourcer, the crowdsourced task and the crowdsourcing platform. In addition, we report on different mappings between these features and the characteristics of the studied papers. We also analyse the distribution of the research using multiple criteria and draw conclusions. For example, our results show a constantly increasing interest in the area, especially in North America and a significant interest from industry. Also, we illustrate that although crowdsourcing is shown to be useful in a variety of disciplines, the research in the field of computer science still seems to be dominant in investigating it.Conclusions:This study allows forming a clear picture of the research in crowdsourcing and understanding the different features of crowdsourcing and their popularity, what type of research was conducted, where and how and by whom. The study enables researchers and practitioners to estimate the current status of the research in this new field. Our taxonomy of extracted features provides a reference model which could be used to configure crowdsourcing and also define it precisely and make design decisions on which of its variation to adopt.
computer science, information systems, theory & methods, software engineering