Beyond Generative Artificial Intelligence: Roadmap for Natural Language Generation

María Miró Maestre,Iván Martínez-Murillo,Tania J. Martin,Borja Navarro-Colorado,Antonio Ferrández,Armando Suárez Cueto,Elena Lloret

2024-07-15

Abstract:Generative Artificial Intelligence has grown exponentially as a result of Large Language Models (LLMs). This has been possible because of the impressive performance of deep learning methods created within the field of Natural Language Processing (NLP) and its subfield Natural Language Generation (NLG), which is the focus of this paper. Within the growing LLM family are the popular GPT-4, Bard and more specifically, tools such as ChatGPT have become a benchmark for other LLMs when solving most of the tasks involved in NLG research. This scenario poses new questions about the next steps for NLG and how the field can adapt and evolve to deal with new challenges in the era of LLMs. To address this, the present paper conducts a review of a representative sample of surveys recently published in NLG. By doing so, we aim to provide the scientific community with a research roadmap to identify which NLG aspects are still not suitably addressed by LLMs, as well as suggest future lines of research that should be addressed going forward.

Computation and Language

What problem does this paper attempt to address?

The paper aims to explore the latest advancements and future research directions in the field of Natural Language Generation (NLG), particularly in the context of Large Language Models (LLMs). Specifically:

1. **Review Existing Literature**: By reviewing recently published related survey articles, identify unresolved issues in the NLG field and propose future research directions.

2. **Evaluate LLM Performance in NLG**: Analyze the strengths and weaknesses of current LLMs in handling various tasks, especially those requiring higher-level semantic understanding and reasoning.

3. **Propose a Research Roadmap**: Based on the analysis of existing literature, develop a guiding framework for NLG research aimed at advancing the field in the future, particularly in areas such as multimodal support, multilingual support, and knowledge integration.

4. **Address Existing Challenges**: Through experimental validation, highlight the limitations of the latest LLMs like GPT-4 and Bard in handling specific scenarios, such as multimodal input processing, low-resource language support, and common-sense reasoning capabilities, thereby emphasizing the importance of further research in these areas.

In summary, this paper attempts to provide NLG researchers with a clear direction guide through systematic literature review and empirical analysis to address the new challenges brought by the era of LLMs.