The Generative AI Paradox: "What It Can Create, It May Not Understand"

Peter West,Ximing Lu,Nouha Dziri,Faeze Brahman,Linjie Li,Jena D. Hwang,Liwei Jiang,Jillian Fisher,Abhilasha Ravichander,Khyathi Chandu,Benjamin Newman,Pang Wei Koh,Allyson Ettinger,Yejin Choi

2023-11-01

Abstract:The recent wave of generative AI has sparked unprecedented global attention, with both excitement and concern over potentially superhuman levels of artificial intelligence: models now take only seconds to produce outputs that would challenge or exceed the capabilities even of expert humans. At the same time, models still show basic errors in understanding that would not be expected even in non-expert humans. This presents us with an apparent paradox: how do we reconcile seemingly superhuman capabilities with the persistence of errors that few humans would make? In this work, we posit that this tension reflects a divergence in the configuration of intelligence in today's generative models relative to intelligence in humans. Specifically, we propose and test the Generative AI Paradox hypothesis: generative models, having been trained directly to reproduce expert-like outputs, acquire generative capabilities that are not contingent upon -- and can therefore exceed -- their ability to understand those same types of outputs. This contrasts with humans, for whom basic understanding almost always precedes the ability to generate expert-level outputs. We test this hypothesis through controlled experiments analyzing generation vs. understanding in generative models, across both language and image modalities. Our results show that although models can outperform humans in generation, they consistently fall short of human capabilities in measures of understanding, as well as weaker correlation between generation and understanding performance, and more brittleness to adversarial inputs. Our findings support the hypothesis that models' generative capability may not be contingent upon understanding capability, and call for caution in interpreting artificial intelligence by analogy to human intelligence.

Artificial Intelligence,Computation and Language,Computer Vision and Pattern Recognition,Machine Learning

What problem does this paper attempt to address?

The problem that this paper attempts to solve is an obvious paradox regarding Generative AI: these models exhibit super - human capabilities in generation tasks, but at the same time, they show errors in understanding tasks that even basic humans would not make. Specifically, the paper explores why Generative AI models can produce outputs that challenge or even surpass the expert level within seconds, but their ability to understand these outputs is far inferior to that of humans. This phenomenon is the opposite of the human cognitive process, because for humans, basic understanding is usually a prerequisite for generating high - level outputs.

To explore this paradox, the paper proposes the "Generative AI Paradox" hypothesis: generative models acquire the generation ability by directly training to replicate expert - level outputs, and this ability does not depend on their ability to understand these outputs, so it can exceed the latter. In contrast, basic human understanding almost always precedes the ability to generate expert - level outputs. The paper analyzes the generation and understanding abilities in language and image modalities through controlled experiments to test this hypothesis.

The main contributions of the paper are as follows:
1. **Defining the relationship between generation and understanding**: Two evaluation methods, namely selectivity and inquisitiveness, are proposed to measure the relationship between the model's performance in generation tasks and its performance in understanding tasks.
2. **Experimental verification**: Through experiments on multiple datasets and tasks, the superior performance of generative models in generation tasks and their deficiencies in understanding tasks are verified.
3. **Discussion of possible reasons**: Potential factors that cause generative models to surpass their understanding ability in generation ability are explored, including the training objectives of the models, the size and nature of the input, etc.

Overall, the paper aims to reveal the uniqueness of Generative AI models in terms of capabilities and configurations, and emphasizes the need for caution when evaluating and understanding these models, and avoiding simply making analogies with human intelligence.