Competition-level code generation with AlphaCode

Yujia Li,David Choi,Junyoung Chung,Nate Kushman,Julian Schrittwieser,Rémi Leblond,Tom Eccles,James Keeling,Felix Gimeno,Agustin Dal Lago,Thomas Hubert,Peter Choy,Cyprien de Masson d'Autume,Igor Babuschkin,Xinyun Chen,Po-Sen Huang,Johannes Welbl,Sven Gowal,Alexey Cherepanov,James Molloy,Esme Sutherland Robson,Pushmeet Kohli,Nando de Freitas,Koray Kavukcuoglu,Oriol Vinyals,Daniel J. Mankowitz
DOI: https://doi.org/10.1126/science.abq1158
IF: 56.9
2022-12-10
Science
Abstract:Programming is a powerful and ubiquitous problem-solving tool. Systems that can assist programmers or even generate programs themselves could make programming more productive and accessible. Recent transformer-based neural network models show impressive code generation abilities yet still perform poorly on more complex tasks requiring problem-solving skills, such as competitive programming problems. Here, we introduce AlphaCode, a system for code generation that achieved an average ranking in the top 54.3% in simulated evaluations on recent programming competitions on the Codeforces platform. AlphaCode solves problems by generating millions of diverse programs using specially trained transformer-based networks and then filtering and clustering those programs to a maximum of just 10 submissions. This result marks the first time an artificial intelligence system has performed competitively in programming competitions.
multidisciplinary sciences
What problem does this paper attempt to address?