LLMs as Potential Brainstorming Partners for Math and Science Problems

Sophia Gu
2023-10-11
Abstract:With the recent rise of widely successful deep learning models, there is emerging interest among professionals in various math and science communities to see and evaluate the state-of-the-art models' abilities to collaborate on finding or solving problems that often require creativity and thus brainstorming. While a significant chasm still exists between current human-machine intellectual collaborations and the resolution of complex math and science problems, such as the six unsolved Millennium Prize Problems, our initial investigation into this matter reveals a promising step towards bridging the divide. This is due to the recent advancements in Large Language Models (LLMs). More specifically, we conduct comprehensive case studies to explore both the capabilities and limitations of the current state-of-the-art LLM, notably GPT-4, in collective brainstorming with humans.
Computation and Language
What problem does this paper attempt to address?
This paper explores the possibility and limitations of large language models (LLMs) as partners for humans in mathematical and scientific problem-solving. The study mainly showcases the potential of LLMs in supporting advanced mathematical and scientific contexts, such as proposing new questions, improving problem definitions, suggesting innovative methods or solutions, through case analysis, especially in the performance evaluation of OpenAI's GPT-4. The paper points out that although the current human-computer collaboration cannot solve complex mathematical and scientific problems, the advancements of LLMs provide hope for narrowing this gap. The research is divided into two main objectives: firstly, to demonstrate the capabilities and limitations of GPT-4 as a partner in human brainstorming through detailed case studies and qualitative analysis, particularly in open-ended questions; secondly, to explore the abilities of GPT-4 in constructing new questions and methods, beyond the evaluation of traditional closed-ended problems. The experimental section demonstrates how GPT-4 assists in understanding complex concepts, proposing research questions, and suggesting potential methods, while also revealing its limitations in independent judgement, spontaneous questioning, and handling high-dimensional problems. Although GPT-4 currently cannot perform numerical calculations, it is capable of providing relevant statistical insights and step-by-step problem-solving strategies. Overall, the paper emphasizes how LLMs enhance the ability of professionals in problem-solving through their extensive knowledge base combined with personal training. It also highlights the potential for future development of LLMs. At the same time, it emphasizes the importance of human-guided dialogue and understanding the machine's reasoning process.