Abstract:With the rapid development of large language models (LLMs) and ever-evolving practical requirements, finding an efficient and effective alignment method has never been more critical. However, the tension between the complexity of current alignment methods and the need for rapid iteration in deployment scenarios necessitates the development of a model-agnostic alignment approach that can operate under these constraints. In this paper, we introduce Aligner, a novel and simple alignment paradigm that learns the correctional residuals between preferred and dispreferred answers using a small model. Designed as a model-agnostic, plug-and-play module, Aligner can be directly applied to various open-source and API-based models with only one-off training, making it suitable for rapid iteration. Notably, Aligner can be applied to any powerful, large-scale upstream models. Moreover, it can even iteratively bootstrap the upstream models using corrected responses as synthetic human preference data, breaking through the model's performance ceiling. Our experiments demonstrate performance improvements by deploying the same Aligner model across 11 different LLMs, evaluated on the 3H dimensions (helpfulness, harmlessness, and honesty). Specifically, Aligner-7B has achieved an average improvement of 68.9% in helpfulness and 23.8% in harmlessness across the tested LLMs while also effectively reducing hallucination. In the Alpaca-Eval leaderboard, stacking Aligner-2B on GPT-4 Turbo improved its LC Win Rate from 55.0% to 58.3%, surpassing GPT-4 Omni's 57.5% Win Rate (community report).

An AMR Aligner Tuned by Transition-based Parser

AMR Parsing with Action-Pointer Transformer

AMR Parsing as Sequence-to-Graph Transduction

Structure-aware Fine-tuning of Sequence-to-sequence Transformers for Transition-based AMR Parsing

A Two-Stage Method for Chinese AMR Parsing

Guiding AMR Parsing with Reverse Graph Linearization

Making Better Use of Bilingual Information for Cross-Lingual AMR Parsing.

AMRs Assemble! Learning to Ensemble with Autoregressive Models for AMR Parsing

AMR Parsing Via Graph-Sequence Iterative Inference

Aligner: Efficient Alignment by Learning to Correct

ATP: AMRize then Parse! Enhancing AMR Parsing with PseudoAMRs

From text to graph: a general transition-based AMR parsing using neural network

Combining Multiple Alignments to Improve Machine Translation.

AMR Parsing using Stack-LSTMs

Towards Integrated Machine Translation Using Structural Alignment From Syntax-Augmented Synchronous Parsing

Diversify and Combine: Improving Word Alignment for Machine Translation on Low-Resource Languages.

RIGA at SemEval-2016 Task 8: Impact of Smatch Extensions and Character-Level Neural Translation on AMR Parsing Accuracy

AMR Parsing with Causal Hierarchical Attention and Pointers

Pushing the Limits of AMR Parsing with Self-Learning

AMR Parsing with Instruction Fine-tuned Pre-trained Language Models

Aligner²: Enhancing Joint Multiple Intent Detection and Slot Filling Via Adjustive and Forced Cross-Task Alignment