Abstract:Automatic text generation is the generation of natural language text by machines. Enabling machines to generate readable and coherent text is one of the most vital yet challenging tasks. Traditionally, text generation has been implemented either by using production rules of a predefined grammar or performing statistical analysis of existing human-written texts to predict sequences of words. Recently a paradigm change has emerged in text generation, induced by technological advancements, including deep learning methods and pre-trained transformers. However, many open challenges in text generation need to be addressed, including the generation of fluent, coherent, diverse, controllable, and consistent human-like text. This survey aims to provide a comprehensive overview of current advancements in automated text generation and introduce the topic to researchers by offering pointers and synthesis to pertinent studies. This paper studied the relevant twelve years of articles from 2011 onwards in the field of text generation and observed a total of 146 prime studies relevant to the objective of this survey that has been thoroughly reviewed and discussed. It covers core text generation applications, including text summarization, question–answer generation, story generation, machine translation, dialogue response generation, paraphrase generation, and image/video captioning. The most commonly used datasets for text generation and existing tools with their application domain have also been mentioned. Various text decoding and optimization methods have been provided with their strengths and weaknesses. For evaluating the effectiveness of the generated text, automatic evaluation metrices have been discussed. Finally, the article discusses the main challenges and notable future directions in the field of automated text generation for potential researchers.

Survey Paper: Automatic Title Generation for Text with RNN and Pre-trained Transformer Language Model

Automatic Title Generation for Learning Resources and Pathways with Pre-trained Transformer Models

A Systematic survey on automated text generation tools and techniques: application, evaluation, and challenges

Can pre-trained language models generate titles for research papers?

Domain Controlled Title Generation with Human Evaluation

Natural Language Generation Using Sequential Models: A Survey

Generative AI-Based Text Generation Methods Using Pre-Trained GPT-2 Model

Pre-trained Language Models for Text Generation: A Survey

A Survey of Controllable Text Generation using Transformer-based Pre-trained Language Models

A Survey of AI Text-to-Image and AI Text-to-Video Generators

Pretrained Language Models for Text Generation: A Survey

The survey: Text generation models in deep learning

Automated News Summarization Using Transformers

Transformers Go for the LOLs: Generating (Humourous) Titles from Scientific Abstracts End-to-End

Advanced NLP Techniques for Sentiment Analysis and Text Summarization Using RNNs and Transformers

A Survey of Natural Language Generation

Transformer Based Implementation for Automatic Book Summarization

Survey of transformers and towards ensemble learning using transformers for natural language processing

Recent Advances in Neural Text Generation: A Task-Agnostic Survey

A survey on text generation using generative adversarial networks