Computing over the Reals: Where Turing Meets Newton
Lenore Blum
2004-01-01
Abstract:The classical (Turing) theory of computation has been extraordinarily successful in providing the foundations and framework for theoretical computer science. Yet its dependence on 0's and 1's is fundamentally inadequate for providing such a foundation for modern scientific computation where most algorithms --with origins in Newton, Euler, Gauss, et. al. -are real number algorithms. In 1989, Mike Shub, Steve Smale and I introduced a theory of computation and complexity over an arbitrary ring or field R [BSS89]. If R is Z2 = ({0, 1}, +, ⋅), the classical computer science theory is recovered. If R is the field of real numbers , Newton’s algorithm, the paradigm algorithm of numerical analysis, fits naturally into our model of computation. Complexity classes P, NP and the fundamental question “Does P = NP?” can be formulated naturally over an arbitrary ring R. The answer to the fundamental question depends in general on the complexity of deciding feasibility of polynomial systems over R. When R is Z2, this becomes the classical satisfiability problem of CookLevin [Cook71, Levin73]. When R is the field of complex numbers , the answer depends on the complexity of Hilbert’s Nullstellensatz. 1 This paper is based on the AWM Noether Lecture I gave at the Joint Mathematics Meetings in San Diego, January, 2001. When I started to write up my talk, I came across a paper, “La diversite des mathematiques face a un probleme de logique,” by Alain Yger based on an expository talk he gave at the journees IREM d'Aquitaine, June 20, 2001. (An English translation can be downloaded from his website: http://www.math.u-bordeaux.fr/~yger/exposes.html [Yger01]). Here Yger discusses in greater detail algebraic aspects of the P = NP? problem over . At the time it seemed reasonable to combine the two papers, but ultimately I decided to focus more on my original lecture theme of the two traditions of computing. There is overlap also with Felipe Cucker’s “Three lectures on real computation,” based on talks he gave in Kaikoura, New Zealand in January, 2000 [Cucker01] and earlier expositions of my own [Blum90, Blum91] and of Steve Smale’s [Smale90, Smale97]. 2 Supported in part by NSF grant # CCR-0122581. I am grateful to the Toyota Technological Institute at Chicago for the gift of space-time to write this paper. Computing over the Reals: Where Turing Meets Newton lblum@cs.cmu.edu 2 The notion of reduction between problems (e.g. between traveling salesman and satisfiability) has been a powerful tool in classical complexity theory. But now, in addition, the transfer of complexity results from one domain to another becomes a real possibility. For example, we can ask: Suppose we can show P = NP over (using all the mathematics that is natural here). Then, can we conclude that P = NP over another field such as the algebraic numbers, or even over Z2? (Answer: Yes and essentially yes.) In this paper, I discuss these results and indicate how basic notions from numerical analysis such as condition, round-off and approximation are being introduced into complexity theory, bringing together ideas germinating from the real calculus of Newton and the discrete computation of computer science. The canonical reference for this material is the book, Complexity and Real Computation [BCSS98]. Complexity and Real Computation and authors Mike Shub, Lenore Blum, Felipe Cucker, Steve Smale. (Photo taken by Victor Pan at Dagsthul, 1995.) 1. Two Traditions of Computation The two major traditions of the theory of computation have, for the most part, run a parallel non intersecting course. On the one hand, we have numerical analysis and scientific computation; on the other hand, we have the tradition of computation theory arising from logic and computer science. Fundamental to both traditions is the notion of algorithm. Newton’s method is the paradigm example of an algorithm cited most often in numerical analysis texts. The Turing machine is the underlying model of computation given in most computer science texts on algorithms. Yet, Newton’s method is not discussed in these computer science texts (e.g. [CLRS01]), nor are Turing machines mentioned in texts on numerical analysis (e.g. [StoerBurlirsch02]). More fundamental differences arise with the distinct underlying spaces, the mathematics employed and problems tackled by each tradition. In numerical analysis and scientific computation, algorithms are generally defined are over the reals or complex numbers and the relevant mathematics is that of the continuum. On the other hand, 0’s and 1’s are the Computing over the Reals: Where Turing Meets Newton lblum@cs.cmu.edu 3 basic bits of the theory of computation of computer science and the mathematics employed is generally discrete. The problems of numerical analysts tend to come from the classical tradition of equation solving and the calculus. Those of the computer scientist tend to have more recent combinatorial origins. The highly developed theory of computation and complexity theory of computer science in general is unnatural for analyzing problems arising in numerical analysis, yet no comparable formal theory has emanated from the latter. One aim of our work is to reconcile the dissonance between these two traditions, perhaps to unify, but most importantly to see how perspectives and tools of each can inform the other. Numerical Analysis/ Scientific Computation Logic/Computer Science • Newton’s Method, Paradigm Example in Most Numerical Analysis Texts • Turing Machine, Underlying Model in most Computer Science Texts on Algorithms • No Mention of Turing Machines* • No Mention of Newton’s Method • Real & Complex #’s • 0’s & 1’s (bits)* • Math is Continuous • Math is discrete • Problems are Classical • Problems are Newer • Major conference: FoCM (Foundation of Computational Mathematics) • Major conference: FOCS (Foundation of Computer Science) * No formal model of computation or systematic Complexity Theory * Everything coded by bits, unnatural for problems of numerical analysis. We begin with some background (section 2) and motivation (section 3), then we present our unifying model (section 4) and main complexity results (section 5) and finally, we see Turing meet Newton and fundamental links introduced (section 6).