Abstract:One-shot non-autoregressive neural networks, different from RL-based ones, have been actively adopted for solving combinatorial optimization (CO) problems, which can be trained by the objective score in a self-supervised manner. Such methods have shown their superiority in efficiency (e.g. by parallelization) and potential for tackling predictive CO problems for decision-making under uncertainty. While the discrete constraints often become a bottleneck for gradient-based neural solvers, as currently handled in three typical ways: 1) adding a soft penalty in the objective, where a bounded violation of the constraints cannot be guaranteed, being critical to many constraint-sensitive scenarios; 2) perturbing the input to generate an approximate gradient in a black-box manner, though the constraints are exactly obeyed while the approximate gradients can hurt the performance on the objective score; 3) a compromise by developing soft algorithms whereby the output of neural networks obeys a relaxed constraint, and there can still occur an arbitrary degree of constraint-violation. Towards the ultimate goal of establishing a general framework for neural CO solver with the ability to control an arbitrary-small degree of constraint violation, in this paper, we focus on a more achievable and common setting: the cardinality constraints, which in fact can be readily encoded by a differentiable optimal transport (OT) layer. Based on this observation, we propose OT-based cardinality constraint encoding for end-to-end CO problem learning with two variants: Sinkhorn and Gumbel-Sinkhorn, whereby their violation of the constraints can be exactly characterized and bounded by our theoretical results. On synthetic and real-world CO problem instances, our methods surpass the state-of-the-art CO network and are comparable to (if not superior to) the commercial solver Gurobi. In particular, we further showcase a case study of applying our approach to the predictive portfolio optimization task on real-world asset price data, improving the Sharpe ratio from 1.1 to 2.0 of a strong LSTM+Gurobi baseline under the classic predict-then-optimize paradigm.

An Analysis under a Unified Fomulation of Learning Algorithms with Output Constraints

Learn from Your Neighbor: Learning Multi-modal Mappings from Sparse Annotations

Evaluating Neuron Interpretation Methods of NLP Models

Accuracy Versus Simplification in an Approximate Logic Neural Model

Not Everything is All You Need: Toward Low-Redundant Optimization for Large Language Model Alignment

Neural Combinatorial Optimization: a New Player in the Field

Reflecting After Learning for Understanding

Towards a Unified Theoretical Understanding of Non-contrastive Learning via Rank Differential Mechanism

Are LLM-based Evaluators Confusing NLG Quality Criteria?

Towards One-shot Neural Combinatorial Solvers: Theoretical and Empirical Notes on the Cardinality-Constrained Case

A Pseudo-Semantic Loss for Autoregressive Models with Logical Constraints

Unveiling the Limits of Learned Local Search Heuristics: Are You the Mightiest of the Meek?

Learning Solutions of Stochastic Optimization Problems with Bayesian Neural Networks

LLM Comparative Assessment: Zero-shot NLG Evaluation through Pairwise Comparisons using Large Language Models

Learning with Logical Constraints but without Shortcut Satisfaction

Systematic Evaluation of LLM-as-a-Judge in LLM Alignment Tasks: Explainable Metrics and Diverse Prompt Templates

Learning by Grouping: A Multilevel Optimization Framework for Improving Fairness in Classification without Losing Accuracy

Artificial Neural Networks that Learn to Satisfy Logic Constraints

Robustness of different loss functions and their impact on networks learning capability

KTO: Model Alignment as Prospect Theoretic Optimization

Constrained Neural Networks for Interpretable Heuristic Creation to Optimise Computer Algebra Systems