Abstract:Foundation models have achieved remarkable success in several fields such as natural language processing, computer vision and more recently biology. DNA foundation models in particular are emerging as a promising approach for genomics. However, so far no model has delivered granular, nucleotide-level predictions across a wide range of genomic and regulatory elements, limiting their practical usefulness. In this paper, we build on our previous work on the Nucleotide Transformer (NT) to develop a segmentation model, SegmentNT, that processes input DNA sequences up to 30kb-long to predict 14 different classes of genomic elements at single nucleotide resolution. By utilizing pre-trained weights from NT, SegmentNT surpasses the performance of several ablation models, including convolution networks with one-hot encoded nucleotide sequences and models trained from scratch. SegmentNT can process multiple sequence lengths with zero-shot generalization for sequences of up to 50kb. We show improved performance on the detection of splice sites throughout the genome and demonstrate strong nucleotide-level precision. Because it evaluates all gene elements simultaneously, SegmentNT can predict the impact of sequence variants not only on splice site changes but also on exon and intron rearrangements in transcript isoforms. Finally, we show that a SegmentNT model trained on human genomic elements can generalize to elements of different human and plant species and that a trained multispecies SegmentNT model achieves stronger generalization for all genic elements on unseen species. In summary, SegmentNT demonstrates that DNA foundation models can tackle complex, granular tasks in genomics at a single-nucleotide resolution. SegmentNT can be easily extended to additional genomic elements and species, thus representing a new paradigm on how we analyze and interpret DNA. We make our SegmentNT-30kb human and multispecies models available on our in Jax and in Pytorch.

A Foundational Large Language Model for Edible Plant Genomes

Within- and Cross-Species Predictions of Plant Specialized Metabolism Genes Using Transfer Learning.

Cross-species modeling of plant genomes at single nucleotide resolution using a pre-trained DNA language model

CropGF: a Comprehensive Visual Platform for Crop Gene Family Mining and Analysis.

Robust Predictions of Specialized Metabolism Genes Through Machine Learning

Large Language Models in Plant Biology

Advancing Plant Single-Cell Genomics with Foundation Models

Advancing Plant Metabolic Research By Using Large Language Models To Expand Databases And Extract Labelled Data

SegmentNT: annotating the genome at single-nucleotide resolution with DNA foundation models

Identification, characterization, and design of plant genome sequences using deep learning

Advancing plant biology through deep learning-powered natural language processing

PlantConnectome: knowledge graph encompassing >70,000 plant articles

Deep learning for plant genomics and crop improvement

Expression-based machine learning models for predicting plant tissue identity

Welcome to the big leaves: Best practices for improving genome annotation in non-model plant genomes

The Nucleotide Transformer: Building and Evaluating Robust Foundation Models for Human Genomics

pLM-DBPs: Enhanced DNA-Binding Protein Prediction in Plants Using Embeddings From Protein Language Models

DNABERT-2: Efficient Foundation Model and Benchmark For Multi-Species Genome

The gene function prediction challenge: large language models and knowledge graphs to the rescue

Genome-Wide Prediction of Complex Traits in Two Outcrossing Plant Species Through Deep Learning and Bayesian Regularized Neural Network