Abstract:The fixed-size context of Transformer makes GPT models incapable of generating arbitrarily long text. In this paper, we introduce RecurrentGPT, a language-based simulacrum of the recurrence mechanism in RNNs. RecurrentGPT is built upon a large language model (LLM) such as ChatGPT and uses natural language to simulate the Long Short-Term Memory mechanism in an LSTM. At each timestep, RecurrentGPT generates a paragraph of text and updates its language-based long-short term memory stored on the hard drive and the prompt, respectively. This recurrence mechanism enables RecurrentGPT to generate texts of arbitrary length without forgetting. Since human users can easily observe and edit the natural language memories, RecurrentGPT is interpretable and enables interactive generation of long text. RecurrentGPT is an initial step towards next-generation computer-assisted writing systems beyond local editing suggestions. In addition to producing AI-generated content (AIGC), we also demonstrate the possibility of using RecurrentGPT as an interactive fiction that directly interacts with consumers. We call this usage of generative models by ``AI As Contents'' (AIAC), which we believe is the next form of conventional AIGC. We further demonstrate the possibility of using RecurrentGPT to create personalized interactive fiction that directly interacts with readers instead of interacting with writers. More broadly, RecurrentGPT demonstrates the utility of borrowing ideas from popular model designs in cognitive science and deep learning for prompting LLMs. Our code is available at <a class="link-external link-https" href="https://github.com/aiwaves-cn/RecurrentGPT" rel="external noopener nofollow">this https URL</a> and an online demo is available at <a class="link-external link-https" href="https://www.aiwaves.org/recurrentgpt" rel="external noopener nofollow">this https URL</a>.

Real-time interactive sequence generation and control with Recurrent Neural Network ensembles

Residual Recurrent Neural Networks for Learning Sequential Representations.

Recurrently Controlled Recurrent Networks

A Hierarchical Neural Autoencoder for Paragraphs and Documents

Generating Sequences With Recurrent Neural Networks

A Critical Review of Recurrent Neural Networks for Sequence Learning

Recurrent Control Nets for Deep Reinforcement Learning

RecurrentGPT: Interactive Generation of (Arbitrarily) Long Text

Interactive Music Generation with Positional Constraints using Anticipation-RNNs

Recurrent Neural Network Method for Automatic Generation of Music

Dynamical system simulation with attention and recurrent neural networks

A Novel Method of Music Generation Based on Three Different Recurrent Neural Networks

Recurrent Neural Networks Learn to Store and Generate Sequences using Non-Linear Representations

A recurrent neural network framework for flexible and adaptive decision making based on sequence learning

ReGAN: RE[LAX|BAR|INFORCE] based Sequence Generation using GANs

On End-to-End Program Generation from User Intention by Deep Neural Networks

Modelling High-Dimensional Sequences with LSTM-RTRBM: Application to Polyphonic Music Generation

Text-based LSTM networks for Automatic Music Composition

Alleviate Exposure Bias in Sequence Prediction \\ with Recurrent Neural Networks

Goal-directed Sequence Generation with Simulation Feedback Method

A Character-Level Sequence-to-sequence Method for Subtitle Learning