Abstract:Recent advancements in natural language processing (NLP) have catalyzed the development of models capable of generating coherent and contextually relevant responses. Such models are applied across a diverse array of applications, including but not limited to chatbots, expert systems, question-and-answer robots, and language translation systems. Large Language Models (LLMs), exemplified by OpenAI's Generative Pretrained Transformer (GPT), have significantly transformed the NLP landscape. They have introduced unparalleled abilities in generating text that is not only contextually appropriate but also semantically rich. This evolution underscores a pivotal shift towards more sophisticated and intuitive language understanding and generation capabilities within the field. Models based on GPT are developed through extensive training on vast datasets, enabling them to grasp patterns akin to human writing styles and deliver insightful responses to intricate questions. These models excel in condensing text, extending incomplete passages, crafting imaginative narratives, and emulating conversational exchanges. However, GPT LLMs are not without their challenges, including ethical dilemmas and the propensity for disseminating misinformation. Additionally, the deployment of these models at a practical scale necessitates a substantial investment in training and computational resources, leading to concerns regarding their sustainability. ChatGPT, a variant rooted in transformer-based architectures, leverages a self-attention mechanism for data sequences and a reinforcement learning-based human feedback (RLHF) system. This enables the model to grasp long-range dependencies, facilitating the generation of contextually appropriate outputs. Despite ChatGPT marking a significant leap forward in NLP technology, there remains a lack of comprehensive discourse on its architecture, efficacy, and inherent constraints. Therefore, this survey aims to elucidate the ChatGPT model, offering an in-depth exploration of its foundational structure and operational efficacy. We meticulously examine Chat-GPT's architecture and training methodology, alongside a critical analysis of its capabilities in language generation. Our investigation reveals ChatGPT's remarkable aptitude for producing text indistinguishable from human writing, whilst also acknowledging its limitations and susceptibilities to bias. This analysis is intended to provide a clearer understanding of ChatGPT, fostering a nuanced appreciation of its contributions and challenges within the broader NLP field. We also explore the ethical and societal implications of this technology, and discuss the future of NLP and AI. Our study provides valuable insights into the inner workings of ChatGPT, and helps to shed light on the potential of LLMs for shaping the future of technology and society. The approach used as Eco-GPT, with a three-level cascade (GPT-J, J1-G, GPT-4), achieves 73% and 60% cost savings in CaseHold and CoQA datasets, outperforming GPT-4.

Chattronics: using GPTs to assist in the design of data acquisition systems

GPT-Based Models Meet Simulation: How to Efficiently Use Large-Scale Pre-Trained Language Models Across Simulation Tasks

Large Language Models as Data Preprocessors

How Can Large Language Models Help Humans in Design and Manufacturing?

Quo Vadis ChatGPT? From Large Language Models to Large Knowledge Models

MechGPT, a language-based strategy for mechanics and materials modeling that connects knowledge across scales, disciplines and modalities

Geotechnical Parrot Tales (GPT): Harnessing Large Language Models in geotechnical engineering

Demystifying ChatGPT: An In-depth Survey of OpenAI's Robust Large Language Models

LLM4DS: Evaluating Large Language Models for Data Science Code Generation

GPT in Data Science: A Practical Exploration of Model Selection

Exploring a GPT-based large language model for variable autonomy in a VR-based human-robot teaming simulation

GPT-4 as an interface between researchers and computational software: improving usability and reproducibility

Synergizing Human Expertise and AI Efficiency with Language Model for Microscopy Operation and Automated Experiment Design

Assessing Large Language Models in Mechanical Engineering Education: A Study on Mechanics-Focused Conceptual Understanding

The use of GPT-4o and Other Large Language Models for the Improvement and Design of Self-Assessment Scales for Measurement of Interpersonal Communication Skills

Exploring and Characterizing Large Language Models For Embedded System Development and Debugging

ModelGPT: Unleashing LLM's Capabilities for Tailored Model Generation

Using GPT for Market Research

Exploring Large Language Models to Facilitate Variable Autonomy for Human-Robot Teaming

Large Language Models for Cultural Heritage

ChatGPT as an inventor: Eliciting the strengths and weaknesses of current large language models against humans in engineering design