Integrating Amdahl-like Laws and Divisible Load Theory

Yang Cao,Fei Wu,Thomas Robertazzi

DOI: https://doi.org/10.48550/arXiv.1902.01952

2019-02-06

Abstract:Integrating Amdahl's and Amdahl-like laws with Divisible Load Theory promises mathematical design methodology that will make possible the efficient design of tomorrow's systems.

Distributed, Parallel, and Cluster Computing

What problem does this paper attempt to address?

A New Computing Model for C-Like Hardware Description Languages

Jimin Wang,Lingdi Ping,Jiebing Wang,Kang Sun

DOI: https://doi.org/10.1109/IMSCCS.2006.153

2006-01-01

Abstract:Traditional C-like hardware description languages are based on single dimensional computing model. They describe only behavior of designs, while cost information is leaved to be digging out by a synthesizer at late stage of design process. However, cost information is needed in high level synthesis for making trade-off among different aspects of cost to achieve global optimized design. To solve the problem, we extended traditional computing model to multi-dimensional so cost information can be included in the language, and integrated it to a popular C++ variant hardware description language - SystemC. The result language describes both behavior and cost of designs so trade-off can be made at relative high level during synthesis process and early stage of design process, thus make it easy to optimize designs for different target applications.
Amdahl's and Hill-Marty Laws Revisited for FPGA-based MPSoCs: from Theory to Practice.

Junneng Zhang,Chao Wang,Xi Li,Xuehai Zhou,Aili Wang,Gangyong Jia,Nadia Nedjah

DOI: https://doi.org/10.1504/ijhpsa.2014.061466

2014-01-01

International Journal of High Performance Systems Architecture

Abstract:In this paper we extend and analyse Amdahl's and Hill-Marty laws to general heterogeneous on-chip cluster era, without the abstract limitation of base core equivalents BCEs. To understand the relative merits between speedup and the cluster characteristics, including microprocessors amount, accelerators speedup, and task partitioning plans, we also analyse the theoretical results about how the extended Amdahl's Law is applied to leverage load balancing of a heterogeneous on-chip cluster. Simulation results demonstrate that our extended model reinforces state-of-the-art performance evaluation methods for hybrid MPSoC architectures and also provide creditable new insights to heterogeneous research communities, in particular for scalable FPGA based on-chip cluster domains.
An Improved Algorithm for Drift Diffusion Transport and Its Application on Large Scale Parallel Simulation of Resistive Random Access Memory Arrays

Da-Wei Wang,Wenchao Chen,Wen-Sheng Zhao,Guo-Dong Zhu,Zhen-Guo Zhao,Jose E. Schutt-Aine,Wen-Yan Yin

DOI: https://doi.org/10.1109/access.2019.2902406

IF: 3.9

2019-01-01

IEEE Access

Abstract:A hybrid finite volume-finite element method which can avoid overestimation of volume shared by each vertex in the meshing grid is proposed to solve the diffusive transport governed continuity equation. The simulation results demonstrate that the improved algorithm can eliminate unphysical distortions as compared to the conventional Scharfetter Gummel method. Based on the proposed algorithm, a parallel-computation simulator is developed for large-scale electrothermal simulation of resistive random access memory (RRAM) arrays, in which the domain decomposition method and J parallel adaptive unstructured mesh applications infrastructure are adopted. The validity, speedup, and scalability of the parallel simulator are investigated on the TianHe-2 supercomputer. Based on the simulated results, the electrothermal characteristics and reliability analysis of large-scale RRAM arrays are investigated in detail.
How Amdahl's low restricts supercomputer applications and building ever bigger supercomputers

János Végh

DOI: https://doi.org/10.48550/arXiv.1708.01462

2017-12-29

Abstract:This paper reinterprets Amdahl's law in terms of execution time and applies this simple model to supercomputing. The systematic discussion results in practical formulas enabling to calculate expected running time using large number of processors from experimental runs using low number of processors, delivers a quantitative measure of computational efficiency of supercomputing applications. Through separating non-parallelizable contribution to fractions according to their origin, Amdahl's law enables to derive a timeline for supercomputers (quite similar to Moore's law) and describes why Amdahl's law limits the size of supercomputers. The paper validates that Amdahl's 50-years old model (with slight extension) correctly describes the performance limitations of the present supercomputers. Using some simple and reasonable assumptions, the absolute performance bound of supercomputers is concluded, furthermore that serious enhancements are still necessary to achieve the exaFLOPS dream value.

Distributed, Parallel, and Cluster Computing
Extending Amdahl's Law in the Multicore Era

Erlin Yao,Yungang Bao,Guangming Tan,Mingyu Chen

DOI: https://doi.org/10.1145/1639562.1639571

2009-01-01

ACM SIGMETRICS Performance Evaluation Review

Abstract:research-article Share on Extending Amdahl's law in the multicore era Authors: Erlin Yao Institute of Computing Technology, Chinese Academy of Sciences Institute of Computing Technology, Chinese Academy of SciencesView Profile , Yungang Bao Institute of Computing Technology, Chinese Academy of Sciences Institute of Computing Technology, Chinese Academy of SciencesView Profile , Guangming Tan Institute of Computing Technology, Chinese Academy of Sciences Institute of Computing Technology, Chinese Academy of SciencesView Profile , Mingyu Chen Institute of Computing Technology, Chinese Academy of Sciences Institute of Computing Technology, Chinese Academy of SciencesView Profile Authors Info & Claims ACM SIGMETRICS Performance Evaluation ReviewVolume 37Issue 2September 2009pp 24–26https://doi.org/10.1145/1639562.1639571Published:16 October 2009Publication History 22citation654DownloadsMetricsTotal Citations22Total Downloads654Last 12 Months30Last 6 weeks13 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access
Divisible-Load Scheduling for Network-Based Computing Systems with Processor Startup Overheads and Release Times.

Xiaoli Wang,Yuping Wang,Junfan Lai

DOI: https://doi.org/10.1109/cis.2016.0012

2016-01-01

Abstract:As scientific applications become more data intensive, finding an efficient scheduling strategy for massive computing on network-based computing systems has drawn increasingly attention. Most existing scheduling models assume that all processors are idle at the beginning of workload assignment. In fact, in the real distributed computing environments, processors may still be occupied with any previous assigned workload when a new load arrives. The time period between busy to idle states of a processor is referred to as its release time. Based on the fact that processors have arbitrary release times, we propose a new divisible-load scheduling model with hybrid time constraints and design an effective global optimization genetic algorithm to solve it. Finally, experiment results show the effectiveness and efficiency of the proposed algorithm.
Performance Analysis and Prediction of Divisible Load Application in Reconfigurable Computing

WANG Ying,CHEN Wei-nan,ZHOU Xue-gong,PENG Cheng-lian

2010-01-01

Abstract:This paper presents a performance analysis and prediction of Divisible Load Application(DLA)based on parameterized execution model of hardware threads.The problem of divisible load scheduling is mainly discussed for a reconfigurable computing platform including multiple Reconfigurable Processing Units(RPUs)and the influence of configuration on load scheduling and optimal processing time is also analyzed.It is proved that there exist maximum RPU number and optimal RPU number when configuration does not completely overlap with communication.An optimal solution for the load scheduling and the processing time is derived for DLA design and implementation.
A Numerical Model Oriented Large-scale Parallel I/O Optimization Method.

Lei Zhang,Tao Wu,Zheng Zhou,Dong Wang,Xi Chen

DOI: https://doi.org/10.1109/DSDE58527.2023.00015

2023-01-01

Abstract:Numerical models usually contain a large number of data reading and writing, which generally takes a long time. At present, massively parallel technology has accelerated the speed of computing in model, but due to the limitation of computer I/O ability, the acceleration effect is limited, and there is a serious load imbalance during the model operation process, resulting in a waste of computing resources. To deal with that, this paper proposes a new I/O scheme, which can dynamically optimize task allocation during program operation, make full use of computing resources, and increase the mutual coverage of computing and I/O processes, so as to effectively shorten the model running time. Finally, the test in the earth simulator shows that the new I/O scheme improves the operation speed of the regional high-precision air pollution model by 25.45%.
MultiAmdahl: Optimal Resource Allocation in Heterogeneous Architectures

Leonid Yavits,Amir Morad,Uri Weiser,Ran Ginosar

DOI: https://doi.org/10.48550/arXiv.1705.06923

2017-05-19

Hardware Architecture

Abstract:Future multiprocessor chips will integrate many different units, each tailored to a specific computation. When designing such a system, the chip architect must decide how to distribute limited system resources such as area, power, and energy among the computational units. We extend MultiAmdahl, an analytical optimization technique for resource allocation in heterogeneous architectures, for energy optimality under a variety of constant system power scenarios. We conclude that reduction in constant system power should be met by reallocating resources from general-purpose computing to heterogeneous accelerator-dominated computing, to keep the overall energy consumption at a minimum. We extend this conclusion to offer an intuition regarding energy-optimal resource allocation in data center computing.
Using Amdahl's Law for Performance Analysis of Many-Core SoC Architectures Based on Functionally Asymmetric Processors

Hao Shen,Frédéric Pétrot

DOI: https://doi.org/10.1007/978-3-642-19137-4_4

2011-01-01

Abstract:Amdahl's law is a fundamental tool for understanding the evolution of performance as a function of parallelism. Following a recent trend on the timing and power analysis of general purpose many-core chips using this law, we carry out an analysis aiming at many-core SoCs integrating processors sharing the same core instruction set but each potentially having additional extensions. For SoCs targeting well defined classes of applications, higher performances can be achieved by adding application specific extensions either through the addition of instructions in the core instruction set or through coprocessors leading to architectures with functionally asymmetric processors. This kind of architectures is becoming technically viable and advocated by several groups, but the theoretical study of their properties is yet to be performed: this is precisely our goal in this paper. We use Amdahl's law to prove the performance advantage of using extensions for many-core SoCs and shows that the many-core architecture based on functionally asymmetric processors can achieve the same performance as the symmetric one but at a lower cost.
A Study of Amdahl′s Law Performance Model on Asymmetric Multicore System

冯叶,邓倩妮

DOI: https://doi.org/10.19304/j.cnki.issn1000-7180.2011.08.009

2011-01-01

Abstract:Amdahl＇s law studies the performance of parallel computing and reveals performance limitation.The asymmetric multicore architecture which is designed to break this limit need appropriate performance model to support its theoretical value.New model analyzes different speedups between two architectures and tells asymmetric architecture can gain performance advantage.Also this model gives advisable design proposals to maximize asymmetric multicore architecture＇s performance.
Analyzing Parallelization and Program Performance in Heterogeneous MPSoCs

Chao Wang,Xi Li,Junneng Zhang,Gangyong Jia,Peng Chen,Xuehai Zhou

DOI: https://doi.org/10.1109/mascots.2012.61

2012-01-01

Abstract:In this paper we extend and analyze Amdahl's law to general heterogeneous MPSoC era, to find out how the speedup is affected by the parameters, including amount and speedup for microprocessors and accelerators, as well as the task partition characteristics. We also analyze the theoretical results about how the extended Amdahl's Law is applied to leverage load balancing of a heterogeneous MPSoC without the abstract limitation of base core equivalents (BCEs). A prototype on FPGA is constructed with Microblaze processors and JPEG hardware accelerators. The experimental results demonstrate that our extended model reinforces state-of-the-art performance evaluation methods for hybrid MPSoC architectures and also provide creditable new insights on the heterogeneous research communities, in particular for scalable FPGA based reconfigurable MPSoCs.
A Communication- and Memory-Aware Model for Load Balancing Tasks

Jonathan Lifflander,Philippe P. Pebay,Nicole L. Slattengren,Pierre L. Pebay,Robert A. Pfeiffer,Joseph D. Kotulski,Sean T. McGovern

2024-04-26

Abstract:While load balancing in distributed-memory computing has been well-studied, we present an innovative approach to this problem: a unified, reduced-order model that combines three key components to describe "work" in a distributed system: computation, communication, and memory. Our model enables an optimizer to explore complex tradeoffs in task placement, such as increased parallelism at the expense of data replication, which increases memory usage. We propose a fully distributed, heuristic-based load balancing optimization algorithm, and demonstrate that it quickly finds close-to-optimal solutions. We formalize the complex optimization problem as a mixed-integer linear program, and compare it to our strategy. Finally, we show that when applied to an electromagnetics code, our approach obtains up to 2.3x speedups for the imbalanced execution.

Distributed, Parallel, and Cluster Computing
An Optimization Model for Divisible-Load Scheduling Considering Processor Time-Window

Junfan LAI,Yuping WANG,Xiaoli WANG

DOI: https://doi.org/10.7652/xjtuxb201709017

2017-01-01

Abstract:A divisible-load scheduling problem under the constraint of processor time-window for heterogeneous distributed systems is presented,and the make-span is minimized by finding the optimal load partition strategy and the optimal distribution sequence of processors.On the basis of existing research,the concept of time-window is introduced first,which makes the model more practical.Then a novel divisible-load scheduling non-blocking model is proposed considering the time-window.Meanwhile,a gen.etic algorithm is designed for the proposed model.In order to solve the model quickly and efficiently,the load partition and the distribution sequence of processors are encoded at the same time.Different crossover operators are designed to optimize the load partition and the distribution sequence of processors,and a modification operator is designed to modify the load partition scheme which does not satisfy the constraint of processors' time-window.Moreover,An efficient local search operator to is also designed speed up the convergence rate of the algorithm.Finally,simulation experiments are carried out,and the results show that under the constraint of the processor time-window,the proposed algorithm can improve the performance of the existing algorithms by at least 20 ％,which proves the correctness and effectiveness of the proposed algorithm.
A Discussion on the Extension of Moore's Law

TF Li,ZJ Li

DOI: https://doi.org/10.1109/icsict.2004.1434998

2004-01-01

Abstract:Both theory and ITRS predict that the scaling of conventional silicon CMOS is going to approach its ultimate. Several emerging devices based on different new mechanism are proposed as possible successors. But still, as classical 2-level switches, they have common limits of quantum uncertainty and thermodynamic fluctuation. In order to further enhance and improve the ability and performance of ULSI systems, either more efficient devices or processing systems with new model and architecture are required. In this article, these topics are briefly discussed in unified view of the product of device delay time and its power dissipation.
Multi-Amdahl: Optimal Resource Sharing with Multiple Program Execution Segments

Tsahee Zidenberg,Isaac Keslassy,Uri Weiser

DOI: https://doi.org/10.48550/arXiv.1105.2960

2011-05-16

Abstract:This paper presents Multi-Amdahl, a resource allocation analytical tool for heterogeneous systems. Our model includes multiple program execution segments, where each one is accelerated by a specific hardware unit. The acceleration speedup of the specific hardware unit is a function of a limited resource, such as the unit area, power, or energy. Using the Lagrange theorem we discover the optimal resource distribution between all specific units. We then illustrate this general Multi-Amdahl technique using several examples of area and power allocation among several cores and accelerators.

Hardware Architecture
Optimal Divisible Load Scheduling for Resource-Sharing Network

Fei Wu,Yang Cao,Thomas Robertazzi

DOI: https://doi.org/10.48550/arXiv.1902.01898

2019-02-06

Abstract:Scheduling is an important task allowing parallel systems to perform efficiently and reliably. For modern computation systems, divisible load is a special type of data which can be divided into arbitrary sizes and independently processed in parallel. Such loads are commonly encountered in applications which are processing a great amount of similar data units. For a multi-task processor, the processor's speed may be time-varying due to the arrival and departure of other background jobs. This paper studies an optimal divisible loads scheduling problem on a single level tree network, whose processing speeds and channel speeds are time-varying. Two recursive algorithms are provided to solve this problem when the arrival and departure times of the background jobs are known a priori and an iterative algorithm is provided to solve the case where such times are not known. Numerical tests and evaluations are performed for these three algorithms under different numbers of background jobs and processors.

Distributed, Parallel, and Cluster Computing
On the Scalability and Dynamic Load-Balancing of Optimistic Gate Level Simulation

Sina Meraji,Wei Zhang,Carl Tropper

DOI: https://doi.org/10.1109/tcad.2010.2049044

IF: 2.9

2010-01-01

IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems

Abstract:As proscribed by Moore's law, the size of integrated circuits has grown geometrically, resulting in simulation becoming the major bottleneck in the circuit design process. Parallel simulation provides us with a way to cope with this growth. In this paper, we describe an optimistic (time warp) parallel discrete event simulator which can simulate all synthesizeable Verilog circuits. We investigate its scalability and describe a machine learning based dynamic load balancing algorithm for use with the simulator. We initially developed two dynamic load balancing algorithms to balance the load and the communication, respectively, during the course of a simulation. Making use of reinforcement learning (RL), we then created an algorithm which is an amalgam of these two algorithms. To the best of our knowledge, this is the first time that RL has been used for the dynamic load-balancing of time warp. We investigated the scalability and the effectiveness of the dynamic load balancing algorithms on gate level simulations of several realistic very large scale integration (VLSI) circuits. Our experimental results showed that our simulator is indeed scalable. They also reveled a 88.6% improvement in the simulation time through the use of our RL algorithm.
Large Language Models for Energy-Efficient Code: Emerging Results and Future Directions

Huiyun Peng,Arjun Gupte,Nicholas John Eliopoulos,Chien Chou Ho,Rishi Mantri,Leo Deng,Wenxin Jiang,Yung-Hsiang Lu,Konstantin Läufer,George K. Thiruvathukal,James C. Davis

2024-10-12

Abstract:Energy-efficient software helps improve mobile device experiences and reduce the carbon footprint of data centers. However, energy goals are often de-prioritized in order to meet other requirements. We take inspiration from recent work exploring the use of large language models (LLMs) for different software engineering activities. We propose a novel application of LLMs: as code optimizers for energy efficiency. We describe and evaluate a prototype, finding that over 6 small programs our system can improve energy efficiency in 3 of them, up to 2x better than compiler optimizations alone. From our experience, we identify some of the challenges of energy-efficient LLM code optimization and propose a research agenda.

Software Engineering
Distributed Scheduling Strategies for Processing Multiple Divisible Loads with Unknown Network Resources

He Li,Guangzhong Sun,Yinlong Xu

DOI: https://doi.org/10.1109/npc.2007.134

2007-01-01

Abstract:The problem of scheduling multiple divisible loads on bus networks with unknown parameters is addressed in this paper. Most research in optimal partitioning of massive loads to be distributed among processors in a distributed system is under the assumptions of known and deterministic network resources. Lately, researchers have started addressing issues that are application oriented and go beyond those simple assumptions of load distribution. In this paper, a heterogeneous network with unknown network resources is considered, and several strategies with increasing refinements that minimize the total processing time of multiple divisible loads are designed. These strategies perform close to the real distributed systems more and more. Application comparison of these proposed strategies is carried out.

Integrating Amdahl-like Laws and Divisible Load Theory

A New Computing Model for C-Like Hardware Description Languages

Amdahl's and Hill-Marty Laws Revisited for FPGA-based MPSoCs: from Theory to Practice.

An Improved Algorithm for Drift Diffusion Transport and Its Application on Large Scale Parallel Simulation of Resistive Random Access Memory Arrays

How Amdahl's low restricts supercomputer applications and building ever bigger supercomputers

Extending Amdahl's Law in the Multicore Era

Divisible-Load Scheduling for Network-Based Computing Systems with Processor Startup Overheads and Release Times.

Performance Analysis and Prediction of Divisible Load Application in Reconfigurable Computing

A Numerical Model Oriented Large-scale Parallel I/O Optimization Method.

MultiAmdahl: Optimal Resource Allocation in Heterogeneous Architectures

Using Amdahl's Law for Performance Analysis of Many-Core SoC Architectures Based on Functionally Asymmetric Processors

A Study of Amdahl′s Law Performance Model on Asymmetric Multicore System

Analyzing Parallelization and Program Performance in Heterogeneous MPSoCs

A Communication- and Memory-Aware Model for Load Balancing Tasks

An Optimization Model for Divisible-Load Scheduling Considering Processor Time-Window

A Discussion on the Extension of Moore's Law

Multi-Amdahl: Optimal Resource Sharing with Multiple Program Execution Segments

Optimal Divisible Load Scheduling for Resource-Sharing Network

On the Scalability and Dynamic Load-Balancing of Optimistic Gate Level Simulation

Large Language Models for Energy-Efficient Code: Emerging Results and Future Directions

Distributed Scheduling Strategies for Processing Multiple Divisible Loads with Unknown Network Resources