Quantifying Generalization Complexity for Large Language Models

Zhenting Qi,Hongyin Luo,Xuliang Huang,Zhuokai Zhao,Yibo Jiang,Xiangjun Fan,Himabindu Lakkaraju,James Glass

2024-10-03

Abstract:While large language models (LLMs) have shown exceptional capabilities in understanding complex queries and performing sophisticated tasks, their generalization abilities are often deeply entangled with memorization, necessitating more precise evaluation. To address this challenge, we introduce Scylla, a dynamic evaluation framework that quantitatively measures the generalization abilities of LLMs. Scylla disentangles generalization from memorization via assessing model performance on both in-distribution (ID) and out-of-distribution (OOD) data through 20 tasks across 5 levels of complexity. Through extensive experiments, we uncover a non-monotonic relationship between task complexity and the performance gap between ID and OOD data, which we term the generalization valley. Specifically, this phenomenon reveals a critical threshold - referred to as critical complexity - where reliance on non-generalizable behavior peaks, indicating the upper bound of LLMs' generalization capabilities. As model size increases, the critical complexity shifts toward higher levels of task complexity, suggesting that larger models can handle more complex reasoning tasks before over-relying on memorization. Leveraging Scylla and the concept of critical complexity, we benchmark 28LLMs including both open-sourced models such as LLaMA and Qwen families, and close-sourced models like Claude and GPT, providing a more robust evaluation and establishing a clearer understanding of LLMs' generalization capabilities.

Computation and Language

What problem does this paper attempt to address?

The paper aims to address the relationship between the generalization ability and memory capacity of large language models (LLMs) when handling complex tasks. Specifically, the authors propose a dynamic evaluation framework called SCYLLA, which is used to quantitatively measure the generalization ability of LLMs and decouple generalization ability from memory capacity. By evaluating the model's performance on in-distribution (ID) data and out-of-distribution (OOD) data, SCYLLA can reveal performance differences of the model under varying task complexities.

The study finds that at a certain level of task complexity, the model's reliance on non-generalization behavior peaks, a phenomenon referred to as the "generalization valley." As the model size increases, this peak shifts to higher levels of task complexity, indicating that larger models can handle more complex reasoning tasks without heavily relying on memory. Additionally, by benchmarking 28 LLMs (including open-source models like the LLaMA and Qwen series, as well as closed-source models like Claude and GPT), the research further clarifies the generalization capabilities of these models.

Overall, by introducing the SCYLLA framework, the paper not only provides new insights into the generalization ability of LLMs but also demonstrates how model size affects their performance on tasks of varying complexity, thereby offering a more detailed perspective on the capability boundaries of LLMs.