The Effects of Moore's Law and Slacking on Large Computations

C Gottbrath,J Bailin,C Meakin,T Thompson,J.J. Charfman

DOI: https://doi.org/10.48550/arXiv.astro-ph/9912202

1999-12-10

Abstract:We show that, in the context of Moore's Law, overall productivity can be increased for large enough computations by `slacking' or waiting for some period of time before purchasing a computer and beginning the calculation.

Astrophysics

What problem does this paper attempt to address?

Moore's Law, the Life Cycle of Scientific Computing Codes and the Diminishing Importance of Parallel Computing

Rainald Löhner,Chi Yang,Juan R. Cebral,Fernando F. Camelli,Fumiya Togashi,Joseph D. Baum,Hong Luo,Eric L. Mestreau,Orlando A. Soto

DOI: https://doi.org/10.1016/b978-044452206-1/50006-9

2006-01-01

Abstract:The chapter describes the typical life cycle of scientific computing codes. Particular relevance is placed on the number of users, their concerns, the machines on which the codes operate as they mature, as well as the relative importance of parallel computing. It is seen that parallel computing achieves the highest importance in the early phases of code development, acting as an enabling technology without which new scientific codes could not develop. Given the typical times, new applications tend to run at their inception, Moore's law itself is perhaps the biggest incentive for new scientific computing codes. Without it, computing time would not decrease in the future and the range of applications would soon be exhausted. One of the most remarkable constants in the rapidly changing world is the rate of growth for the number of transistors that are packaged onto a square inch. This rate, commonly known as Moore's law, is approximately a factor of 2 every 18 months that translates into a factor of 10 every 5 years.
Moore's Law: The potential, limits, and breakthroughs

Fulai Zhu,Jiahao Zong,Peiyu Xu

DOI: https://doi.org/10.54254/2755-2721/10/20230038

2023-09-25

Abstract:Moore's Law is a concept that notes the doubling of the number of transistors on a microchip around every two years, resulting in exponential advancement in computing power and diminishing costs. However, as transistor sizes have approached the physical limits of silicon-based technology, maintaining this pace of growth has become increasingly challenging. As a result, researchers have been exploring alternative solutions, such as System-in-package(SiP), Chiplets, Non-volatile memory, Biocomputing, quantum computing, and photonics. While these technologies are still in the early stages of development, they offer promising solutions to the challenges facing the continued growth of computing power. As we continue to explore new avenues for technological advancement, we may find that Moore's Law continues to hold true in unexpected and exciting ways.

Physics,Computer Science,Engineering
A Discussion on the Extension of Moore's Law

TF Li,ZJ Li

DOI: https://doi.org/10.1109/icsict.2004.1434998

2004-01-01

Abstract:Both theory and ITRS predict that the scaling of conventional silicon CMOS is going to approach its ultimate. Several emerging devices based on different new mechanism are proposed as possible successors. But still, as classical 2-level switches, they have common limits of quantum uncertainty and thermodynamic fluctuation. In order to further enhance and improve the ability and performance of ULSI systems, either more efficient devices or processing systems with new model and architecture are required. In this article, these topics are briefly discussed in unified view of the product of device delay time and its power dissipation.
Cambrian explosion of computing and big data in the post-moore era

Satoshi Matsuoka

DOI: https://doi.org/10.1145/3208040.3225055

2018-06-11

Abstract:The so-called "Moore's Law", by which the performance of the processors will increase exponentially by factor of 4 every 3 years or so, is slated to be ending in 10--15 year timeframe due to the lithography of VLSIs reaching its limits around that time, and combined with other physical factors. We are now embarking on a project to revolutionize the total system architectural stack in a holistic fashion in the Post-Moore era, from devices and hardware, abstracted by system software and programming models and languages, and optimized according to the device characteristics with new algorithms and applications that exploit them. Such systems will have multitudes of varieties according to the matching characteristics of applications to the underlying architecture, leading to what can be metaphorically described as Cambrian Explosion of computing systems. The diverse elements of such systems will be interconnected with next-generation terabit optics and networks, allowing metropolitan-scale computing infrastructure that would truly realize high performance parallel and distributed computing. However, which algorithms and applications would benefit the most from such future computing, given that some physical constants, e.g., communication latency, cannot be improved? We speculate on some of the scenarios that would change the nature of current Cloud-centric infrastructures towards the Post-Moore era.
How Amdahl's low restricts supercomputer applications and building ever bigger supercomputers

János Végh

DOI: https://doi.org/10.48550/arXiv.1708.01462

2017-12-29

Abstract:This paper reinterprets Amdahl's law in terms of execution time and applies this simple model to supercomputing. The systematic discussion results in practical formulas enabling to calculate expected running time using large number of processors from experimental runs using low number of processors, delivers a quantitative measure of computational efficiency of supercomputing applications. Through separating non-parallelizable contribution to fractions according to their origin, Amdahl's law enables to derive a timeline for supercomputers (quite similar to Moore's law) and describes why Amdahl's law limits the size of supercomputers. The paper validates that Amdahl's 50-years old model (with slight extension) correctly describes the performance limitations of the present supercomputers. Using some simple and reasonable assumptions, the absolute performance bound of supercomputers is concluded, furthermore that serious enhancements are still necessary to achieve the exaFLOPS dream value.

Distributed, Parallel, and Cluster Computing
Opportunities and Challenges for Next Generation Computing

Gregory D. Hager,Mark D. Hill,Katherine Yelick

DOI: https://doi.org/10.48550/arXiv.2008.00023

2020-07-31

Computers and Society

Abstract:Computing has dramatically changed nearly every aspect of our lives, from business and agriculture to communication and entertainment. As a nation, we rely on computing in the design of systems for energy, transportation and defense; and computing fuels scientific discoveries that will improve our fundamental understanding of the world and help develop solutions to major challenges in health and the environment. Computing has changed our world, in part, because our innovations can run on computers whose performance and cost-performance has improved a million-fold over the last few decades. A driving force behind this has been a repeated doubling of the transistors per chip, dubbed Moore's Law. A concomitant enabler has been Dennard Scaling that has permitted these performance doublings at roughly constant power, but, as we will see, both trends face challenges. Consider for a moment the impact of these two trends over the past 30 years. A 1980's supercomputer (e.g. a Cray 2) was rated at nearly 2 Gflops and consumed nearly 200 KW of power. At the time, it was used for high performance and national-scale applications ranging from weather forecasting to nuclear weapons research. A computer of similar performance now fits in our pocket and consumes less than 10 watts. What would be the implications of a similar computing/power reduction over the next 30 years - that is, taking a petaflop-scale machine (e.g. the Cray XK7 which requires about 500 KW for 1 Pflop (=1015 operations/sec) performance) and repeating that process? What is possible with such a computer in your pocket? How would it change the landscape of high capacity computing? In the remainder of this paper, we articulate some opportunities and challenges for dramatic performance improvements of both personal to national scale computing, and discuss some "out of the box" possibilities for achieving computing at this scale.
Is there a Moore's law for quantum computing?

Olivier Ezratty

2023-03-28

Abstract:There is a common wisdom according to which many technologies can progress according to some exponential law like the empirical Moore's law that was validated for over half a century with the growth of transistors number in chipsets. As a still in the making technology with a lot of potential promises, quantum computing is supposed to follow the pack and grow inexorably to maturity. The Holy Grail in that domain is a large quantum computer with thousands of errors corrected logical qubits made themselves of thousands, if not more, of physical qubits. These would enable molecular simulations as well as factoring 2048 RSA bit keys among other use cases taken from the intractable classical computing problems book. How far are we from this? Less than 15 years according to many predictions. We will see in this paper that Moore's empirical law cannot easily be translated to an equivalent in quantum computing. Qubits have various figures of merit that won't progress magically thanks to some new manufacturing technique capacity. However, some equivalents of Moore's law may be at play inside and outside the quantum realm like with quantum computers enabling technologies, cryogeny and control electronics. Algorithms, software tools and engineering also play a key role as enablers of quantum computing progress. While much of quantum computing future outcomes depends on qubit fidelities, it is progressing rather slowly, particularly at scale. We will finally see that other figures of merit will come into play and potentially change the landscape like the quality of computed results and the energetics of quantum computing. Although scientific and technological in nature, this inventory has broad business implications, on investment, education and cybersecurity related decision-making processes.

Quantum Physics
Limits on Fundamental Limits to Computation

Igor L. Markov

DOI: https://doi.org/10.1038/nature13570

2015-01-09

Abstract:An indispensable part of our lives, computing has also become essential to industries and governments. Steady improvements in computer hardware have been supported by periodic doubling of transistor densities in integrated circuits over the last fifty years. Such Moore scaling now requires increasingly heroic efforts, stimulating research in alternative hardware and stirring controversy. To help evaluate emerging technologies and enrich our understanding of integrated-circuit scaling, we review fundamental limits to computation: in manufacturing, energy, physical space, design and verification effort, and algorithms. To outline what is achievable in principle and in practice, we recall how some limits were circumvented, compare loose and tight limits. We also point out that engineering difficulties encountered by emerging technologies may indicate yet-unknown limits.

Emerging Technologies,Quantum Physics
A Numerical Model Oriented Large-scale Parallel I/O Optimization Method.

Lei Zhang,Tao Wu,Zheng Zhou,Dong Wang,Xi Chen

DOI: https://doi.org/10.1109/DSDE58527.2023.00015

2023-01-01

Abstract:Numerical models usually contain a large number of data reading and writing, which generally takes a long time. At present, massively parallel technology has accelerated the speed of computing in model, but due to the limitation of computer I/O ability, the acceleration effect is limited, and there is a serious load imbalance during the model operation process, resulting in a waste of computing resources. To deal with that, this paper proposes a new I/O scheme, which can dynamically optimize task allocation during program operation, make full use of computing resources, and increase the mutual coverage of computing and I/O processes, so as to effectively shorten the model running time. Finally, the test in the earth simulator shows that the new I/O scheme improves the operation speed of the regional high-precision air pollution model by 25.45%.
Towards Scalability Collapse Behavior on Multicores

Yan Cui,Yu Chen,Yuanchun Shi

DOI: https://doi.org/10.1002/cpe.2983

2014-01-01

Abstract:Multicore processor systems have become mainstream. To release the full potential of multiple cores, applications are programmed to be parallel to keep every core busy. Unfortunately, lock contention within operating systems can limit the scalability so seriously that use of more cores leads to reduced throughput (scalability collapse). To understand and characterize the collapse behavior easily, a discrete-event simulation model, which considers both the sequential execution of critical sections and the overhead of hardware resource contention, is designed and implemented. By the use of the model, we observe that the percentage of time used to wait for locks and the number of tasks requesting for a lock have a significant correlation with the occurrence of scalability collapse. On the basis of these observations, two new techniques (lock contention aware scheduler and requester-based adaptive lock) are proposed to remove the scalability collapse on multicores. The proposed methods are implemented in the Linux kernel 2.6.29.4 and evaluated on an AMD 32-core system to verify their effectiveness. By using micro-benchmarks and macro-benchmarks, we find that these methods can remove scalability collapse totally for four of five workloads exhibiting the collapse behavior. For one workload that does not suffer scalability collapse, these proposed methods only introduce negligible overhead. Copyright (c) 2012 John Wiley & Sons, Ltd.
High Performance Matrix Multiplication on Many Cores

Nan Yuan,Yongbin Zhou,Guangming Tan,Junchao Zhang,Dongrui Fan

DOI: https://doi.org/10.1007/978-3-642-03869-3_87

2009-01-01

Abstract:Moore's Law suggests that the number of processing cores on a single chip increases exponentially. The future performance increases will be mainly extracted from thread-level parallelism exploited by multi/many-core processors (MCP). Therefore, it is necessary to find out how to build the MCP hardware and how to program the parallelism on such MCP. In this work, we intend to identity the key architecture mechanisms and software optimizations to guarantee high performance for multithreaded programs. To illustrate this, we customize a dense matrix multiplication algorithm on Godson-T MCP as a case study to demonstrate the efficient synergy and interaction between hardware and software. Experiments conducted on the cycle-accurate simulator show that the optimized matrix multiplication could obtain 97.1% (124.3GFLOPS) of the peak performance of Godson-T.
Moore's Law is dead, long live Moore's Law!

Nick Zhang

2022-05-27

Abstract:Moore's Law has been used by semiconductor industry as predicative indicators of the industry and it has become a self-fulfilling prophecy. Now more people tend to agree that the original Moore's Law started to falter. This paper proposes a possible quantitative modification to Moore's Law. It can cover other derivative laws of Moore's Law as well. It intends to more accurately predict the roadmap of chip's performance and energy consumption.

General Literature
Comments on the parallelization efficiency of the Sunway TaihuLight supercomputer

János Végh

DOI: https://doi.org/10.48550/arXiv.1608.00249

2016-08-01

Abstract:In the world of supercomputers, the large number of processors requires to minimize the inefficiencies of parallelization, which appear as a sequential part of the program from the point of view of Amdahl's law. The recently suggested new figure of merit is applied to the recently presented supercomputer, and the timeline of "Top 500" supercomputers is scrutinized using the metric. It is demonstrated, that in addition to the computing performance and power consumption, the new supercomputer is also excellent in the efficiency of parallelization. Based on the suggested merit, a "Moore-law" like observation is derived for the timeline of parallelization efficacy of supercomputers.

Distributed, Parallel, and Cluster Computing
Extending Amdahl's Law in the Multicore Era

Erlin Yao,Yungang Bao,Guangming Tan,Mingyu Chen

DOI: https://doi.org/10.1145/1639562.1639571

2009-01-01

ACM SIGMETRICS Performance Evaluation Review

Abstract:research-article Share on Extending Amdahl's law in the multicore era Authors: Erlin Yao Institute of Computing Technology, Chinese Academy of Sciences Institute of Computing Technology, Chinese Academy of SciencesView Profile , Yungang Bao Institute of Computing Technology, Chinese Academy of Sciences Institute of Computing Technology, Chinese Academy of SciencesView Profile , Guangming Tan Institute of Computing Technology, Chinese Academy of Sciences Institute of Computing Technology, Chinese Academy of SciencesView Profile , Mingyu Chen Institute of Computing Technology, Chinese Academy of Sciences Institute of Computing Technology, Chinese Academy of SciencesView Profile Authors Info & Claims ACM SIGMETRICS Performance Evaluation ReviewVolume 37Issue 2September 2009pp 24–26https://doi.org/10.1145/1639562.1639571Published:16 October 2009Publication History 22citation654DownloadsMetricsTotal Citations22Total Downloads654Last 12 Months30Last 6 weeks13 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access
At the Locus of Performance: Quantifying the Effects of Copious 3D-Stacked Cache on HPC Workloads

Jens Domke,Emil Vatai,Balazs Gerofi,Yuetsu Kodama,Mohamed Wahib,Artur Podobas,Sparsh Mittal,Miquel Pericàs,Lingqi Zhang,Peng Chen,Aleksandr Drozd,Satoshi Matsuoka

2023-10-16

Abstract:Over the last three decades, innovations in the memory subsystem were primarily targeted at overcoming the data movement bottleneck. In this paper, we focus on a specific market trend in memory technology: 3D-stacked memory and caches. We investigate the impact of extending the on-chip memory capabilities in future HPC-focused processors, particularly by 3D-stacked SRAM. First, we propose a method oblivious to the memory subsystem to gauge the upper-bound in performance improvements when data movement costs are eliminated. Then, using the gem5 simulator, we model two variants of a hypothetical LARge Cache processor (LARC), fabricated in 1.5 nm and enriched with high-capacity 3D-stacked cache. With a volume of experiments involving a broad set of proxy-applications and benchmarks, we aim to reveal how HPC CPU performance will evolve, and conclude an average boost of 9.56x for cache-sensitive HPC applications, on a per-chip basis. Additionally, we exhaustively document our methodological exploration to motivate HPC centers to drive their own technological agenda through enhanced co-design.

Distributed, Parallel, and Cluster Computing
Non-clairvoyant Speed Scaling for Weighted Flow Time

Sze-Hang Chan,Tak-Wah Lam,Lap-Kei Lee

DOI: https://doi.org/10.1007/978-3-642-15775-2_3

2010-01-01

Abstract:We study online job scheduling on a processor that can vary its speed dynamically to manage its power. We attempt to extend the recent success in analyzing total unweighted flow time plus energy to total weighted flow time plus energy. We first consider the non-clairvoyant setting where the size of a job is only known when the job finishes. We show an online algorithm WLAPS that is 8α2-competitive for weighted flow time plus energy under the traditional power model, which assumes the power P(s) to run the processor at speed s to be sα for some α> 1. More interestingly, for any arbitrary power function P(s), WLAPS remains competitive when given a more energy-efficient processor; precisely, WLAPS is \documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$16(1+\frac{1}{\epsilon})^2$\end{document}-competitive when using a processor that, given the power P(s), can run at speed (1 + ε)s for some ε> 0. Without such speedup, no non-clairvoyant algorithm can be O(1)-competitive for an arbitrary power function [8]. For the clairvoyant setting (where the size of a job is known at release time), previous results on minimizing weighted flow time plus energy rely on scaling the speed continuously over time [5-7]. The analysis of WLAPS has inspired us to devise a clairvoyant algorithm LLB which can transform any continuous speed scaling algorithm to one that scales the speed at discrete times only. Under an arbitrary power function, LLB can give an \documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$4(1+\frac{1}{\epsilon})$\end{document}-competitive algorithm using a processor with (1 + ε)-speedup.
Molecular Dynamics Simulation On Commodity Shared-Memory Multiprocessor Systems With Lightweight Multithreading

John Thornley,Maria Hui,Hao Li,Tahir Cagin,William A. Goddard

1999-01-01

Abstract:Because of the time required to simulate systems with realistic numbers of particles, a great deal of research has been devoted to parallel N-body simulation. Most of this research has focused on the development of sophisticated strategies for data distribution and load balancing on message-passing systems. In this paper, we investigate the implications of the recent advent of powerful commodity shared-memory multiprocessor systems with large caches and support for lightweight multithreading. As the basis of our investigation, we have ported a large parallel molecular dynamics simulation program (MPSim) to a lightweight multithreaded implementation for Intel Pentium family multiprocessors running Windows NT. We compare the performance of our lightweight multithreaded version of MPSim running on commodity multiprocessors with that of previous versions running on expensive parallel supercomputers. In particular, we test the hypothesis that data distribution and load balancing can be significantly simplified on commodity shared-memory multiprocessors. Our experiments indicate that (i) the performance of modern commodity multiprocessors is competitive with that of expensive high-end platforms, and (ii) parallel N-body simulation algorithms can be made considerably simpler and more flexible by taking advantage of shared memory and lightweight multithreading.
Accelerating Scientific Computing in the Post-Moore’s Era

Kathleen E. Hamilton,Catherine D. Schuman,Steven R. Young,Ryan S. Bennink,Neena Imam,Travis S. Humble

DOI: https://doi.org/10.1145/3380940

2020-04-02

ACM Transactions on Parallel Computing

Abstract:Novel uses of graphical processing units for accelerated computation revolutionized the field of high-performance scientific computing by providing specialized workflows tailored to algorithmic requirements. As the era of Moore’s law draws to a close, many new non–von Neumann processors are emerging as potential computational accelerators, including those based on the principles of neuromorphic computing, tensor algebra, and quantum information. While development of these new processors is continuing to mature, the potential impact on accelerated computing is anticipated to be profound. We discuss how different processing models can advance computing in key scientific paradigms: machine learning and constraint satisfaction. Significantly, each of these new processor types utilizes a fundamentally different model of computation, and this raises questions about how to best use such processors in the design and implementation of applications. While many processors are being developed with a specific domain target, the ubiquity of spin-glass models and neural networks provides an avenue for multi-functional applications. This also hints at the infrastructure needed to integrate next-generation processing units into future high-performance computing systems.
Super-speeds with Zero-RAM: Next Generation Large-Scale Optimization in Your Laptop!

Mark Amo-Boateng

DOI: https://doi.org/10.48550/arXiv.1709.02500

2017-09-11

Abstract:This article presents the novel breakthrough general purpose algorithm for large scale optimization problems. The novel algorithm is capable of achieving breakthrough speeds for very large-scale optimization on general purpose laptops and embedded systems. Application of the algorithm to the Griewank function was possible in up to 1 billion decision variables in double precision took only 64485 seconds (~18 hours) to solve, while consuming 7,630 MB (7.6 GB) or RAM on a single threaded laptop CPU. It shows that the algorithm is computationally and memory (space) linearly efficient, and can find the optimal or near-optimal solution in a fraction of the time and memory that many conventional algorithms require. It is envisaged that this will open up new possibilities of real-time large-scale problems on personal laptops and embedded systems.

Distributed, Parallel, and Cluster Computing,Computational Complexity,Data Structures and Algorithms,Performance,Optimization and Control
The Multi-Core Era - Trends and Challenges

Peter Tröger

DOI: https://doi.org/10.48550/arXiv.0810.5439

2008-10-30

Abstract:Since the very beginning of hardware development, computer processors were invented with ever-increasing clock frequencies and sophisticated in-build optimization strategies. Due to physical limitations, this 'free lunch' of speedup has come to an end. The following article gives a summary and bibliography for recent trends and challenges in CMP architectures. It discusses how 40 years of parallel computing research need to be considered in the upcoming multi-core era. We argue that future research must be driven from two sides - a better expression of hardware structures, and a domain-specific understanding of software parallelism.

Distributed, Parallel, and Cluster Computing

The Effects of Moore's Law and Slacking on Large Computations

Moore's Law, the Life Cycle of Scientific Computing Codes and the Diminishing Importance of Parallel Computing

Moore's Law: The potential, limits, and breakthroughs

A Discussion on the Extension of Moore's Law

Cambrian explosion of computing and big data in the post-moore era

How Amdahl's low restricts supercomputer applications and building ever bigger supercomputers

Opportunities and Challenges for Next Generation Computing

Is there a Moore's law for quantum computing?

Limits on Fundamental Limits to Computation

A Numerical Model Oriented Large-scale Parallel I/O Optimization Method.

Towards Scalability Collapse Behavior on Multicores

High Performance Matrix Multiplication on Many Cores

Moore's Law is dead, long live Moore's Law!

Comments on the parallelization efficiency of the Sunway TaihuLight supercomputer

Extending Amdahl's Law in the Multicore Era

At the Locus of Performance: Quantifying the Effects of Copious 3D-Stacked Cache on HPC Workloads

Non-clairvoyant Speed Scaling for Weighted Flow Time

Molecular Dynamics Simulation On Commodity Shared-Memory Multiprocessor Systems With Lightweight Multithreading

Accelerating Scientific Computing in the Post-Moore’s Era

Super-speeds with Zero-RAM: Next Generation Large-Scale Optimization in Your Laptop!

The Multi-Core Era - Trends and Challenges