A Personal Perspective on Numerical Analysis and Optimization

Desmond J. Higham

DOI: https://doi.org/10.48550/arXiv.2009.11369

2020-09-24

Abstract:I give a brief, non-technical, historical perspective on numerical analysis and optimization. I also touch on emerging trends and future challenges. This content is based on the short presentation that I made at the opening ceremony of \emph{The International Conference on Numerical Analysis and Optimization}, which was held at Sultan Qaboos University, Muscat, Oman, on January 6--9, 2020. Of course, the material covered here is necessarily incomplete and biased towards my own interests and comfort zones. My aim is to give a feel for how the area has developed over the past few decades and how it may continue.

Numerical Analysis

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to review and look forward to the development process and future challenges in the fields of numerical analysis and optimization. Specifically, the author Desmond J. Higham explores the following points from a personal perspective:

1. **Definition and Development History**:
- Numerical analysis: Design, analyze, and implement computational algorithms to provide approximate solutions to applied mathematics problems.
- Optimization: Design, analyze, and implement computational algorithms to find the best solutions to applied mathematics problems (when there are multiple feasible solutions).

2. **Impact of Technological Advances**:
- Since the beginning of the digital computing era in the 1950s, the rapid development of technology has promoted the progress of numerical analysis and optimization. For example, the speed of supercomputers has increased from 3 megaflops (Mflops) of CDC6600 to today's petaflops (Pflops).
- The increase in data sources, especially the progress in experimental techniques and the digitization of daily life, has further promoted the development of scientific computing.

3. **Current Trends and Future Challenges**:
- Handling or utilizing randomness, including simulating inherently random mathematical models and using randomized methods when solving deterministic problems.
- Accurately and efficiently simulating mathematical models across a wide range of spatio - temporal scales.
- Solving extremely high - dimensional problems, especially large - scale optimization and inverse problems in machine learning and imaging.
- Designing algorithms using the latest computer architectures, balancing issues such as memory bandwidth, data access, and communication, and exploring the application of low - precision special function units.

4. **Influence**:
- The article lists the top ten algorithms that had the greatest impact on the scientific development and engineering practice in the 20th century, and most of these algorithms are based on the ideas of numerical analysis and optimization.

In summary, this paper aims to show how numerical analysis and optimization have developed over the past few decades and look forward to future research directions and challenges by reviewing their history.