Abstract:In this paper, we introduce a novel Artificial Intelligence (AI) system inspired by the philosophical and psychoanalytical concept of imagination as a ``Re-construction of Experiences". Our AI system is equipped with an imagination-inspired module that bridges the gap between textual inputs and other modalities, enriching the derived information based on previously learned experiences. A unique feature of our system is its ability to formulate independent perceptions of inputs. This leads to unique interpretations of a concept that may differ from human interpretations but are equally valid, a phenomenon we term as ``Interpretable Misunderstanding". We employ large-scale models, specifically a Multimodal Large Language Model (MLLM), enabling our proposed system to extract meaningful information across modalities while primarily remaining unimodal. We evaluated our system against other large language models across multiple tasks, including emotion recognition and question-answering, using a zero-shot methodology to ensure an unbiased scenario that may happen by fine-tuning. Significantly, our system outperformed the best Large Language Models (LLM) on the MELD, IEMOCAP, and CoQA datasets, achieving Weighted F1 (WF1) scores of 46.74%, 25.23%, and Overall F1 (OF1) score of 17%, respectively, compared to 22.89%, 12.28%, and 7% from the well-performing LLM. The goal is to go beyond the statistical view of language processing and tie it to human concepts such as philosophy and psychoanalysis. This work represents a significant advancement in the development of imagination-inspired AI systems, opening new possibilities for AI to generate deep and interpretable information across modalities, thereby enhancing human-AI interaction.

Brain model of text animation as a data mining strategy

Write‐An‐Animation: High‐level Text‐based Animation Editing with Character‐Scene Interaction

Dance To Music Expressively: A Brain-Inspired System Based On Audio-Semantic Model For Cognitive Development Of Robots

Toward an example-based machine translation from written text to ASL using virtual agent animation

Generating Animations from Screenplays

Assembling Actor-based Mind-Maps from Text Stream

Imaginations of WALL-E : Reconstructing Experiences with an Imagination-Inspired Module for Advanced AI Systems

A Simulated Brain Model for Products' Creative Design

SketchDreamer: Interactive Text-Augmented Creative Sketch Ideation

Co-Writing Screenplays and Theatre Scripts with Language Models: An Evaluation by Industry Professionals

A Neuro-mimetic Realization of the Common Model of Cognition via Hebbian Learning and Free Energy Minimization

Two-dimensional animation film vision innovation based on VR technology and Markov chain model

BlenderAlchemy: Editing 3D Graphics with Vision-Language Models

Artificial Intelligence-Assisted Simulation Research on Intelligent Behavior of Film and Television 3D Animation Characters

Sketchforme: Composing Sketched Scenes from Text Descriptions for Interactive Applications

Seeking the Building Blocks of Visual Imagery and Creativity in a Cognitively Inspired Neural Network

DreamHuman: Animatable 3D Avatars from Text

Mind captioning: Evolving descriptive text of mental content from human brain activity

Text-to-movie authoring of anatomy lessons

Inspiration through Observation: Demonstrating the Influence of Automatically Generated Text on Creative Writing

Reading visually embodied meaning from the brain: Visually grounded computational models decode visual-object mental imagery induced by written text