Abstract:BACKGROUND: Software engineering researchers have carried out many empirical studies on open source software (OSS) projects to understand the OSS phenomenon, and to develop better software engineering techniques. Many of these studies typically use only a few successful projects as study subjects. Recently, these studies have received criticisms and challenges on their representativeness on OSS projects. AIM: First, we aim to examine to what extent data extracted from successful projects are different from data extracted from the majority. If data extracted from successful projects are quite different from data extracted from the majority, approaches that are effective on successful projects may not be effective in general. Second, we aim to examine whether successful OSS projects are representative to the whole population of OSS. If they are not, conclusions that are drawn from only successful projects may reflect the OSS phenomenon partially. METHODOLOGY: We analyzed 11, 684 OSS projects that are hosted on Source Forge. When researchers select subjects, they typically select successful projects that are attractive to both users and developers. Considering this preference, we clustered these projects into four categories based their attractiveness to users and developers. Here, we use the K-means clustering technique to produce combined result. Furthermore, we selected eight indicators that are used in many existing studies (e.g., team sizes), and compared indicators that are extracted from different categories to investigate to what degree they are different. RESULT: For the first research aim, the result shows that 66.1% projects are under developing projects, 14.7% projects are user-preference projects, 14.2% projects are developer-preference projects, and only 5.0% projects are considered successful. For the second research aim, the result shows that all the eight analyzed indicators are highly unbalanced with the gamma distribution. Furthermore, the result reveals that users and developers of Source Forge have different perceptions on the development status defined by Source Forge. CONCLUSION: We conclude that successful projects are not representative to the whole population of OSS, and data extracted from successful projects are quite different from data extracted from the majority. The result implies that conclusions drawn from only a few successful projects may be challenged. This work is important to allow researchers to refine conclusions of existing studies, and to better understand and to carefully select OSS project subjects for their future empirical experiments.

Ranking Open Source Software Based on Crowd Wisdom

Software Ranker: A New Comprehensive Software Ranking Approach

Comparative Case Studies of Open Source Software Peer Review Practices

Recommending Code Reviewers for Proprietary Software Projects: A Large Scale Study

Social Computing in Open Source Community: A Study of Software Reuse

Seeing eye to eye? An exploratory study of free open source software users' perceptions

Ossrec: An Open Source Software Recommendation System Based On Wisdom Of Crowds

OSSEAN: Mining Crowd Wisdom in Open Source Communities

Recommending Frequently Encountered Bugs.

An empirical study of code reuse between GitHub and stack overflow during software development

Software Ranking and Analysis Based on Mining Market Requirements and Characteristics

A Systematic Mapping Study of Crowd Knowledge Enhanced Software Engineering Research Using Stack Overflow

Dolphin: A Search Engine For Oss Based On Crowd Discussions Across Communities

Mining crowd sourcing repositories for open innovation in software engineering

OSS PESTO: An Open Source Software Project Evaluation and Selection TOol

Network-Based Ranking for Open Source Software Developer Prediction.

How is Software Reuse Discussed in Stack Overflow?

Multi-Feature Based Personal Recommendation Approach for Open Source Project

Assessing the Representativeness of Open Source Projects in Empirical Software Engineering Studies

Knowledge reuse in software projects: Retrieving software development Q&A posts based on project task similarity

Evaluating Bug Severity Using Crowd-based Knowledge: an Exploratory Study