Abstract:We propose cognitive prompting as a novel approach to guide problem-solving in large language models (LLMs) through structured, human-like cognitive operations such as goal clarification, decomposition, filtering, abstraction, and pattern recognition. By employing systematic, step-by-step reasoning, cognitive prompting enables LLMs to efficiently tackle complex, multi-step tasks. We evaluate the effectiveness of cognitive prompting on Meta's LLaMA models, comparing performance on arithmetic reasoning tasks using the GSM8K dataset and on commonsense reasoning benchmarks. Our analysis includes comparisons between models without cognitive prompting, models with a static sequence of cognitive operations, and models using reflective cognitive prompting, where the LLM dynamically self-selects the sequence of cognitive operations. The results show that cognitive prompting, particularly when dynamically adapted, significantly improves the performance of larger models, such as LLaMA3.1 70B, and enhances their ability to handle multi-step reasoning tasks. This approach also improves interpretability and flexibility, highlighting cognitive prompting as a promising strategy for general-purpose AI reasoning.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is: how to enhance the performance of large - language models (LLMs) in multi - step reasoning tasks through cognitive prompting. Specifically, the author proposes a new method - cognitive prompting, which guides LLMs to conduct structured, human - like multi - step reasoning by simulating human cognitive operations (such as goal clarification, decomposition, filtering, abstraction, and pattern recognition, etc.). This method aims to improve the ability of LLMs to handle complex tasks, especially in mathematics and common - sense reasoning.
### Main contributions:
1. **Propose the cognitive prompting method**: Guide LLMs to solve complex problems through a series of structured human cognitive operations, such as goal clarification, task decomposition, pattern recognition, etc.
2. **Experimental verification**: Experiments were carried out on the GSM8K dataset and common - sense reasoning benchmarks using Meta's LLaMA model to verify the effectiveness of cognitive prompting.
3. **Dynamic adaptation**: Reflective cognitive prompting was introduced, enabling the model to dynamically select the most appropriate sequence of cognitive operations according to the task, further improving performance.
### Experimental results:
- **Arithmetic reasoning**: On the GSM8K dataset, the 70B model using cognitive prompting significantly outperforms the model without prompting, and in particular, reflective cognitive prompting achieves the highest score (0.91).
- **Common - sense reasoning**: In common - sense reasoning tasks, cognitive prompting also significantly improves the performance of the model, especially on the smaller 8B model. For the larger 70B model, although static cognitive prompting has an improvement, the effect of reflective cognitive prompting is slightly lower than that without prompting, possibly because too many reasoning steps lead to over - fitting.
### Conclusion:
Cognitive prompting significantly improves the performance of LLMs in multi - step reasoning tasks by simulating the human cognitive process. In particular, reflective cognitive prompting can dynamically adapt to task requirements and further enhance the reasoning ability of the model. However, for larger models, excessive reasoning steps may lead to performance degradation, and regularization methods need to be introduced for optimization. Future work will explore the application of cognitive prompting in more fields, such as legal reasoning, medical decision - making, and strategic planning.