Abstract:Research on long non-coding RNAs (lncRNAs) has garnered significant attention due to their critical roles in gene regulation and disease mechanisms. However, the complexity and diversity of lncRNA sequences, along with the limited knowledge of their functional mechanisms and the regulation of their expressions, pose significant challenges to lncRNA studies. Given the tremendous success of large language models (LLMs) in capturing complex dependencies in sequential data, this study aims to systematically explore the potential and limitations of LLMs in the sequence analysis related to the transcriptional regulation of lncRNA genes. Our extensive experiments demonstrated promising performance of fine-tuned genome foundation models on progressively complex tasks. Furthermore, we conducted an insightful analysis of the critical impact of task complexity, model selection, data quality, and biological interpretability for the studies of the regulation of lncRNA gene expression.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: **Challenges in the analysis of long non - coding RNA (lncRNA) gene transcriptional regulation**. Specifically, lncRNA plays a crucial role in gene regulation and disease mechanisms. However, its complexity and diversity, as well as the limited understanding of its functional mechanisms and expression regulation, have brought significant challenges to lncRNA research. These problems include: 1. **Complexity and diversity of lncRNA sequences**: lncRNA sequences are less conserved than protein - coding genes and lack obvious sequence motifs or structural features, making their identification and functional prediction very difficult. 2. **Unknown functional mechanisms**: The functions and regulatory mechanisms of most lncRNAs are still unknown. 3. **Low expression levels**: The expression levels of lncRNAs are usually much lower than those of protein - coding genes, which increases the difficulty of research. 4. **Limitations of data quality**: Existing lncRNA data sets may not be comprehensive or of high quality, affecting the effectiveness of model training. To address these challenges, the authors propose to use large - language models (LLMs) to capture complex dependencies in sequences and systematically explore the potential and limitations of LLMs in the analysis of lncRNA gene transcriptional regulation by fine - tuning pre - trained genomic foundation models (such as DNABERT, DNABERT - 2, and Nucleotide Transformer). ### Main objectives - **Evaluating the capabilities of LLMs**: Through a series of downstream tasks, evaluate the performance of LLMs in lncRNA - related tasks, including biological sequence classification, promoter sequence detection, classification of high - and low - expression gene promoter sequences, and classification of protein - coding gene and lncRNA gene promoter sequences. - **Exploring the impacts of task complexity, model selection, and data quality**: Analyze the critical impacts of these factors on the performance of LLMs, with the aim of providing guidance for future lncRNA research. - **Improving biological interpretability**: Through methods such as feature - importance analysis, enhance the biological interpretability of LLMs in lncRNA analysis and reveal potential regulatory mechanisms. Through these efforts, the authors hope to fill the current gaps in research and promote the further development of the lncRNA biology field.

Exploring the Potentials and Challenges of Using Large Language Models for the Analysis of Transcriptional Regulation of Long Non-coding RNAs

Study strategies for long non-coding RNAs and their roles in regulating gene expression.

Large-scale Study of Long Non-Coding RNA Functions Based on Structure and Expression Features

Deciphering RNA regulation with a foundation language model

Regulatory Long Non-Coding Rna and Its Functions

Enhancing Gene Set Overrepresentation Analysis with Large Language Models

A comprehensive survey on deep learning-based identification and predicting the interaction mechanism of long non-coding RNAs

An Evaluation of Large Language Models in Bioinformatics Research

Genomic Language Models: Opportunities and Challenges

The Computational Approaches of Lncrna Identification Based on Coding Potential: Status Quo and Challenges

Approaches for Modes of Action Study of Long Non-Coding RNAs: From Single Verification to Genome-Wide Determination

Long Noncoding RNAs: Insights from Biological Features and Functions to Diseases.

Long non-coding RNA-guided regulation in organisms

How do Large Language Models understand Genes and Cells

Panning for Long Noncoding RNAs

A Comprehensive Evaluation of Large Language Models in Mining Gene Interactions and Pathway Knowledge

Editorial: Machine Learning-Based Methods for RNA Data Analysis—volume II

Large language models in bioinformatics: applications and perspectives

Advances in long noncoding RNAs: identification, structure prediction and function annotation.

Evaluating the representational power of pre-trained DNA language models for regulatory genomics

Long Non-Coding RNAs and Complex Diseases: from Experimental Results to Computational Models.