Abstract:Computing devices and associated software govern everyday life, and form the backbone of safety critical systems in banking, healthcare, automotive and other fields. Increasing system complexity, quickly evolving technologies and paradigm shifts have kept software quality research at the forefront. Standards such as ISO's 25010 express it in terms of sub-characteristics such as maintainability, reliability and security. A significant body of literature attempts to link these subcharacteristics with software metric values, with the end goal of creating a metric-based model of software product quality. However, research also identifies the most important existing barriers. Among them we mention the diversity of software application types, development platforms and languages. Additionally, unified definitions to make software metrics truly language-agnostic do not exist, and would be difficult to implement given programming language levels of variety. This is compounded by the fact that many existing studies do not detail their methodology and tooling, which precludes researchers from creating surveys to enable data analysis on a larger scale. In our paper, we propose a comprehensive study of metric values in the context of three complex, open-source applications. We align our methodology and tooling with that of existing research, and present it in detail in order to facilitate comparative evaluation. We study metric values during the entire 18-year development history of our target applications, in order to capture the longitudinal view that we found lacking in existing literature. We identify metric dependencies and check their consistency across applications and their versions. At each step, we carry out comparative evaluation with existing research and present our results.

Employing Contribution and Quality Metrics for Quantifying the Software Development Process

Quality and productivity outcomes relating to continuous integration in GitHub

Metrics Dashboard: A Hosted Platform for Software Quality Metrics

Three Metrics to Explore the Openness of GitHub projects

Towards Understanding the Impact of Code Modifications on Software Quality Metrics

Who is the Real Hero? Measuring Developer Contribution via Multi-dimensional Data Integration

Mining the Contributions Along the Lifecycles of Open-Source Projects

Monitoring Student Activity in Collaborative Software Development

How do Software Engineering Researchers Use GitHub? An Empirical Study of Artifacts & Impact

Measuring software engineer's contribution in practice: An industrial experience report

Evaluation of Software Product Quality Metrics

Is this GitHub Project Maintained? Measuring the Level of Maintenance Activity of Open-Source Projects

An Empirical Study of Developer Quality

Do Design Metrics Capture Developers Perception of Quality? An Empirical Study on Self-Affirmed Refactoring Activities

VISUALISED PRODUCT QUALITY MONITORING TOOL BASED ON SOFTWARE DEVELOPMENT PROCESS DATA

Software Code Quality Measurement: Implications from Metric Distributions

Improving Code Review with GitHub Issue Tracking

Influence of Communication Among Shared Developers on the Productivity of Open Source Software Projects

Measuring the Effect of Social Communications on Individual Working Rhythms: A Case Study of Open Source Software

Fusing Industry and Academia at GitHub (Experience Report)

Evaluating Software Contribution Quality: Time-to-Modification Theory