O1 Replication Journey: A Strategic Progress Report -- Part 1

Yiwei Qin,Xuefeng Li,Haoyang Zou,Yixiu Liu,Shijie Xia,Zhen Huang,Yixin Ye,Weizhe Yuan,Hector Liu,Yuanzhi Li,Pengfei Liu

2024-10-08

Abstract:This paper introduces a pioneering approach to artificial intelligence research, embodied in our O1 Replication Journey. In response to the announcement of OpenAI's groundbreaking O1 model, we embark on a transparent, real-time exploration to replicate its capabilities while reimagining the process of conducting and communicating AI research. Our methodology addresses critical challenges in modern AI research, including the insularity of prolonged team-based projects, delayed information sharing, and the lack of recognition for diverse contributions. By providing comprehensive, real-time documentation of our replication efforts, including both successes and failures, we aim to foster open science, accelerate collective advancement, and lay the groundwork for AI-driven scientific discovery. Our research progress report diverges significantly from traditional research papers, offering continuous updates, full process transparency, and active community engagement throughout the research journey. Technologically, we proposed the journey learning paradigm, which encourages models to learn not just shortcuts, but the complete exploration process, including trial and error, reflection, and backtracking. With only 327 training samples and without any additional tricks, journey learning outperformed conventional supervised learning by over 8\% on the MATH dataset, demonstrating its extremely powerful potential. We believe this to be the most crucial component of O1 technology that we have successfully decoded. We share valuable resources including technical hypotheses and insights, cognitive exploration maps, custom-developed tools, etc at <a class="link-external link-https" href="https://github.com/GAIR-NLP/O1-Journey" rel="external noopener nofollow">this https URL</a>.

Artificial Intelligence,Computation and Language

What problem does this paper attempt to address?

The key problems that this paper attempts to solve are some important challenges existing in current artificial intelligence research, especially issues such as information enclosure, isolation in teamwork, information - sharing delay, and insufficient recognition of diverse contributions in modern AI projects. Specifically:

1. **Information enclosure and transparency**: Currently, many AI research projects (especially large - scale teamwork projects) often have the problem of information enclosure, making it difficult for the outside world to obtain the progress and details of the projects. This not only hinders technological progress but also affects the principle of scientific openness.

2. **Isolation in teamwork**: Long - term teamwork projects are prone to poor internal information flow, and external researchers cannot timely understand the project progress, thus affecting the cooperation and communication of the entire community.

3. **Information - sharing delay**: Traditional scientific research papers are usually published after the project is completed, which means that many valuable intermediate results and failure experiences are not shared in a timely manner, resulting in waste of resources and duplication of work.

4. **Insufficient recognition of diverse contributions**: In large - scale teamwork, the work of individual contributors may not be fully recognized, affecting the enthusiasm and creativity of researchers.

To solve these problems, the author team launched the O1 Replication Journey, aiming to explore and replicate the capabilities of OpenAI's O1 model in a transparent and real - time manner, and re - envision how AI research is carried out and disseminated. Their methods include:

- **Providing comprehensive and real - time documentation**: Record not only successful results but also failure experiences to promote open science and accelerate collective progress.
- **Proposing the "journey - learning" paradigm**: Encourage the model to learn not only shortcuts but also the complete exploration process, including trial - and - error, reflection, and backtracking. This method performs better than traditional supervised learning on the MATH dataset, demonstrating its strong potential.

Overall, this paper hopes to promote a more open, collaborative, and responsible AI research culture through this innovative research method, while providing valuable data and experience for future AI system training.