In-Context Learning for Few-Shot Molecular Property Prediction

Christopher Fifty,Jure Leskovec,Sebastian Thrun

2023-10-13

Abstract:In-context learning has become an important approach for few-shot learning in Large Language Models because of its ability to rapidly adapt to new tasks without fine-tuning model parameters. However, it is restricted to applications in natural language and inapplicable to other domains. In this paper, we adapt the concepts underpinning in-context learning to develop a new algorithm for few-shot molecular property prediction. Our approach learns to predict molecular properties from a context of (molecule, property measurement) pairs and rapidly adapts to new properties without fine-tuning. On the FS-Mol and BACE molecular property prediction benchmarks, we find this method surpasses the performance of recent meta-learning algorithms at small support sizes and is competitive with the best methods at large support sizes.

Machine Learning

What problem does this paper attempt to address?

The paper aims to address the issue of few-shot learning in molecular property prediction. Specifically, the authors propose a new algorithm called CAMP (Context Aware Molecule Prediction), which draws on the concept of in-context learning from the field of natural language processing and applies it to the task of molecular property prediction. The main features of CAMP include:

1. **No fine-tuning required**: Unlike traditional fine-tuning methods, CAMP does not require any updates to the model parameters to adapt to new tasks.
2. **Applicable to non-natural language data**: Unlike in-context learning methods that are limited to natural language processing, CAMP can directly handle molecular structure and property measurement data.
3. **Low latency**: CAMP has low inference time latency, making it well-suited for scenarios such as high-throughput screening.

In the experimental section, the authors compare CAMP with several other few-shot learning baseline methods, including multi-task learning (MT), model-agnostic meta-learning (MAML), and prototypical networks (ProtoNet). The results show that on the FS-Mol and BACE molecular property prediction benchmarks, CAMP outperforms other methods with smaller support set sizes and is competitive with larger support set sizes. Additionally, CAMP has lower inference time latency, making it more advantageous in practical applications.