An Explainable Automated Model for Measuring Software Engineer Contribution

Yue Li,He Zhang,Yuzhe Jin,Zhong Ren,Liming Dong,Jun Lyu,Lanxin Yang,David Lo,Dong Shao
DOI: https://doi.org/10.1145/3691620.3695071
2024-01-01
Abstract:Software engineers play an important role throughout the software development life-cycle, particularly in industry emphasizing quality assurance and timely delivery. Contribution measurement provides proper incentives to software engineers that motivate them to continuously improve the quality and efficiency of their work. However, existing research tends to ignore contribution measurement for software engineers in practice, relying heavily on peer review and lacking objectivity and transparency. Specifically, these studies still have two weaknesses. First, a few studies explore which metrics can be useful for contribution measurement in practice. Second, managers measure the contribution of software engineers based on their experience and lack of explainable automated tools to assist them. To this end, we conduct mixed-method studies to investigate contribution measurement in the enterprise, and then propose an explainable model for measuring the contribution of software engineers. First, we collect and synthesize 16 metrics for contribution measurement by interviewing 18 industrial practitioners. Second, we propose an explainable model, called Memento, integrating Multi-dimEnsional MEtrics for measuriNg conTributiOn. We conduct an industrial case study with a global enterprise to evaluate and refine Memento. Finally, we administer a survey to industrial practitioners to verify whether the explainable model is useful for contribution analysis, which resulted in 67 valid responses. Memento is used by the enterprise to measure the contribution of 5,174 employees. A total of 100 employees are identified as low contributors by Memento, and the five reasons for their low contribution are determined based on the results of our explainable model. The results indicate that Memento can effectively measure the contribution of software engineers, which provides a practical reference for researchers interested in contribution measurement.
What problem does this paper attempt to address?