Abstract:The integration of machine learning models in various real-world applications is becoming more prevalent to assist humans in their daily decision-making tasks as a result of recent advancements in this field. However, it has been discovered that there is a tradeoff between the accuracy and fairness of these decision-making tasks. In some cases, these AI systems can be unfair by exhibiting bias or discrimination against certain social groups, which can have severe consequences in real life. Inspired by one of the most well-known human learning skills called grouping, we address this issue by proposing a novel machine learning framework where the ML model learns to group a diverse set of problems into distinct subgroups to solve each subgroup using its specific sub-model. Our proposed framework involves three stages of learning, which are formulated as a three-level optimization problem: (i) learning to group problems into different subgroups; (ii) learning group-specific sub-models for problem-solving; and (iii) updating group assignments of training examples by minimizing the validation loss. These three learning stages are performed end-to-end in a joint manner using gradient descent. To improve fairness and accuracy, we develop an efficient optimization algorithm to solve this three-level optimization problem. To further reduce the risk of overfitting in small datasets, we incorporate domain adaptation techniques in the second stage of training. We further apply our method to neural architecture search. Extensive experiments on various datasets demonstrate our method's effectiveness and performance improvements in both fairness and accuracy. Our proposed Learning by Grouping can reduce overfitting and achieve state-of-the-art performances with fixed human-designed network architectures and searchable network architectures on various datasets.

Learning Linear Programs from Optimal Decisions

Learning to Optimize for Mixed-Integer Non-linear Programming

Learning to optimize with convergence guarantees using nonlinear system theory

Maximum Optimality Margin: A Unified Approach for Contextual Linear Programming and Inverse Linear Programming

A Unified Linear Programming Framework for Offline Reward Learning from Human Demonstrations and Feedback

Optimization Learning: Perspective, Method, and Applications.

Melding the Data-Decisions Pipeline: Decision-Focused Learning for Combinatorial Optimization

Learning to Solve Optimization Problems with Hard Linear Constraints

Incorporating Linear Regression Problems into an Adaptive Framework with Feasible Optimizations.

Optimal Learning for Sequential Decision Making for Expensive Cost Functions with Stochastic Binary Feedbacks

Learning by Grouping: A Multilevel Optimization Framework for Improving Fairness in Classification without Losing Accuracy

Unleashing Linear Optimizers for Group-Fair Learning and Optimization

Score Function Gradient Estimation to Widen the Applicability of Decision-Focused Learning

A gentle introduction to gradient-based optimization and variational inequalities for machine learning

Linear programming using diagonal linear networks

Online Learning for Linear Programming with Time-Dependent Cost

Learning Algorithm Hyperparameters for Fast Parametric Convex Optimization

Greedy Learning to Optimize with Convergence Guarantees

Efficient Learning of Decision-Making Models: A Penalty Block Coordinate Descent Algorithm for Data-Driven Inverse Optimization

CaVE: A Cone-Aligned Approach for Fast Predict-then-optimize with Binary Linear Programs