Abstract:Fairness in machine learning (ML) applications is an important practice for developers in research and industry. In ML applications, unfairness is triggered due to bias in the data, curation process, erroneous assumptions, and implicit bias rendered within the algorithmic development process. As ML applications come into broader use developing fair ML applications is critical. Literature suggests multiple views on how fairness in ML is described from the users perspective and students as future developers. In particular, ML developers have not been the focus of research relating to perceived fairness. This paper reports on a pilot investigation of ML developers perception of fairness. In describing the perception of fairness, the paper performs an exploratory pilot study to assess the attributes of this construct using a systematic focus group of developers. In the focus group, we asked participants to discuss three questions- 1) What are the characteristics of fairness in ML? 2) What factors influence developers belief about the fairness of ML? and 3) What practices and tools are utilized for fairness in ML development? The findings of this exploratory work from the focus group show that to assess fairness developers generally focus on the overall ML application design and development, i.e., business-specific requirements, data collection, pre-processing, in-processing, and post-processing. Thus, we conclude that the procedural aspects of organizational justice theory can explain developers perception of fairness. The findings of this study can be utilized further to assist development teams in integrating fairness in the ML application development lifecycle. It will also motivate ML developers and organizations to develop best practices for assessing the fairness of ML-based applications.

"It's the most fair thing to do but it doesn't make any sense": Perceptions of Mathematical Fairness Notions by Hiring Professionals

Mathematical Notions vs. Human Perception of Fairness: A Descriptive Approach to Fairness for Machine Learning

How Do Fairness Definitions Fare? Examining Public Attitudes Towards Algorithmic Definitions of Fairness

Understanding Relations Between Perception of Fairness and Trust in Algorithmic Decision Making

Perceived algorithmic fairness: An empirical study of transparency and anthropomorphism in algorithmic recruiting

Fairness and Bias in Algorithmic Hiring: a Multidisciplinary Survey

It is Giving Major Satisfaction: Why Fairness Matters for Developers

The Measure and Mismeasure of Fairness

Perception of fairness in algorithmic decisions: Future developers' perspective

When Causality Meets Fairness: A Survey

Assessing Perceived Fairness from Machine Learning Developer's Perspective

Measuring Non-Expert Comprehension of Machine Learning Fairness Metrics

Fairness Preferences, Actual and Hypothetical: A Study of Crowdworker Incentives

A Human-in-the-loop Framework to Construct Context-aware Mathematical Notions of Outcome Fairness

Human Perceptions of Fairness in Algorithmic Decision Making: A Case Study of Criminal Risk Prediction

Survey on Fairness Notions and Related Tensions

Understanding Fairness in Software Engineering: Insights from Stack Exchange

Toward A Logical Theory Of Fairness and Bias

Parametric Fairness with Statistical Guarantees

Fairness definitions explained

Fairness as Equality of Opportunity: Normative Guidance from Political Philosophy