Feasibility of Dynamic Binary Parallelization

Jing Yang,K. Skadron,M. Soffa,K. Whitehouse

2011-01-01

Abstract:This paper proposes DBP, an automatic technique that transparently parallelizes a sequential binary executable while it is running. A prototype implementation in simulation was able to increase sequential execution speeds by up to 1.96x, averaged over three benchmarks suites.

Computer Science

What problem does this paper attempt to address?

Automatic Parallelization for Binary on Multi-core Platforms

Haibo Zhao,Fei Zheng,Jian Wu,Baosong Nan,Boliang Li,Kuizhi Mei

DOI: https://doi.org/10.1145/3207677.3277982

2018-01-01

Abstract:While(1) multi-core systems are becoming popular, many serial binaries exist. This paper proposed a method to exploit parallelism in serial binaries and automatically parallelize them on multi-core systems. We extended existing compiler optimization techniques to binaries and developed extra approaches to further improve the performance. Preliminary results on Polybench benchmark and our hand gesture recognition program showed an average speedup of 1.8 and 1.4 over serial performance respectively on an armv7 machine with 2 threads.
GXBIT: Combining polyhedral model with dynamic binary translation

Kang Zhang,Fanfu Zhou,Alei Liang

2012-01-01

Abstract:During the last decade, polyhedral model has been widely used as a mathematical model for autoparallelization, and in recent years, with the development of multi-core architecture, polyhedral model has been employed to transform sequential code to parallel code that can run simultaneously on different cores. At the same time, the rapid development of GPU makes CPU/GPU architecture become increasingly popular because of GPU's powerful parallel processing capabilities. However, we have no other methods of using GPU except for CUDA and Stream SDK, which are all based on explicit programming. Apart from this, there are two constraints for explicit programming: binary incompatibility among different GPUs as well as the cost of rewriting source code. Considering these constraints, we use polyhedral model and a dynamic binary translator to build a virtual execution environment: GXBit. GXBit is composed of analysis and execution stage, and the former one is the main focus of this paper. Analysis stage uses binary instrumentation and binary analysis to probe potential parallel parts (usually nested loop) of a binary executable and then polyhedral model is employed to detect whether there is data dependence or not among all iterations of a nested loop. Execution stage is discussed briefly in this paper. Although there are performance loss in binary translation, GXBit has an 8x speedup on average to X86 compute-intensive version through the result of running two applications taken from the CUDA SDK Sample and one test case from the UIUC Parboil Benchmark Suite. © 2005 - 2012 JATIT & LLS. All rights reserved.
A Translation Framework for Executing the Sequential Binary Code on CPU/GPU Based Architectures

Erzhou Zhu,Haibing Guan,Guoxing Dong,Yindong Yang,Hongbo Yang

DOI: https://doi.org/10.4304/jsw.6.12.2331-2340

2011-01-01

Journal of Software

Abstract:The method of using DBT (dynamic binary translation) to execute the source ISAs binary code on target platforms has been perplexed by low overhead for many years. GPU as a many-core processor has tremendous computational power. Employing GPU as a coprocessor to parallel execute the hot spot of binary code hold a great promise of substantially reduce the overhead of DBT. This paper presents a novel translation framework for constructing the virtual execution environment aiming at accelerating the process of DBT on CPU/GPU based architectures. With parallelizable parts (hot spots) of binary code and their related information, the framework converts the sequential code into PTX form and executes them on GPUs. Under the framework, we need not to rewrite the source code, and the binary compatibility issues between different GPUs are also resolved properly. Experimental results on several programs from CUDA SDK Code Samples and Parboil Benchmark Suite show that the framework can significantly improve the performance, usually have 10X speedup on average compared to X86 native platforms. Especially, when the scale of input become larger, the performance becomes even better.
The Optimizations in Dynamic Binary Translation

Haibing Guan,Hongbo Yang,Zhengwei Qi,Yindong Yang

DOI: https://doi.org/10.1109/icut.2010.5677870

2010-01-01

Abstract:The dynamic binary translation(DBT) is used to support legacy binary code, provides program instrumentation, improves code performance, and so on. The dynamic nature of the translation usually incurs extra substantial execution overhead in DBT systems. As a result, DBT may be impractically slow if it do not or can not employ optimizations to offset the overhead. This paper investigates a few optimizations to alleviate the overhead in DBT. We evaluate these optimizations in CrossBit, which is a resourceable and retargetable dynamic binary translator, including block linking, condition codes optimization, register mapping optimization, static-integrated optimization, multithreaded optimization. The experimental results show that these optimizations can improve DBT performance significantly.
The Implementation of Dynamic Linking in Dynamic Binary Translation Systems

Song Yiqing,He Yuemei,Liang Alei,Guan Haibing

DOI: https://doi.org/10.1109/ETCS.2009.404

2009-01-01

Abstract:Dynamic binary translation (DBT) is a commonly used technology in virtual machines. Most DBT systems, like Dynamo, can only translate programs statically linked but not ones dynamically linked. Statically linked programs will lead to expansion of binary codes, which is the indirect reason of loss on performance. In this paper, we propose an approach to implement dynamic linking on DBT systems. During the research, we came across some challenges related with operating systems, compilation and architecture, which will be discussed later in detail. And we have developed a module and had it tested on our DBT system, CrossBit. The statistics show that CrossBit with dynamic linking reduces the number of translated blocks and execution time.
Improving Performance of Dynamic Programming Via Parallelism and Locality on Multicore Architectures

Guangming Tan,Ninghui Sun,Guang R. Gao

DOI: https://doi.org/10.1109/tpds.2008.78

IF: 5.3

2009-01-01

IEEE Transactions on Parallel and Distributed Systems

Abstract:Dynamic programming (DP) is a popular technique which is used to solve combinatorial search and optimization problems. This paper focuses on one type of DP, which is called nonserial polyadic dynamic programming (NPDP). Owing to the nonuniform data dependencies of NPDP, it is difficult to exploit either parallelism or locality. Worse still, the emerging multi/many-core architectures with small on-chip memory make these issues more challenging. In this paper, we address the challenges of exploiting the fine grain parallelism and locality of NPDP on multicore architectures. We describe a latency-tolerant model and a percolation technique for programming on multicore architectures. On an algorithmic level, both parallelism and locality do benefit from a specific data dependence transformation of NPDP. Next, we propose a parallel pipelining algorithm by decomposing computation operators and percolating data through a memory hierarchy to create just-in-time locality. In order to predict the execution time, we formulate an analytical performance model of the parallel algorithm. The parallel pipelining algorithm achieves not only high scalability on the 160-core IBM Cyclops64, but portable performance as well, across the 8-core Sun Niagara and quad-cores Intel Clovertown.
DDP-B - A Distributed Dynamic Parallel Framework for Meta-genomics Binary Similarity.

Mengxian Chi,Xu Jin,Feng Li,Hong An

DOI: https://doi.org/10.1007/978-3-030-30709-7_12

2019-01-01

Abstract:Great efforts have been made on meta-genomics in the field of new species exploration in the past decades. With the development of next-generation sequencing technology, meta-genomics datasets have been produced as large as dozens of hundreds of gigabytes or even several terabytes, which brings a severe challenge to data analysis. Besides, conventional meta-genomics comparing algorithms may not take full advantage of powerful computing capacity from parallel computing techniques due to lack of parallelism. In this paper, we propose DDP-B, a distributed dynamic parallel framework for meta-genomics binary similarity analysis, to overcome these limitations. In this framework, we introduce a binary distance algorithm for meta-genomics similarity measurement and develop different levels of parallel granularity of the algorithm utilizing MPI, OpenMP, and SIMD techniques. Moreover, we establish a dynamic scheduling method to deliver asynchronous parallel computing tasks and design a distributed cluster to deploy the dynamic parallel system, which completes 2.97K pairs of meta-genomics vectors comparison per second and achieves an 134.79x speedup versus the baseline in the optimal condition. Our framework shows stable scalability when assigned larger workloads.
Improving Performance of Dynamic Programming via Parallelism and Locality on

guangming tan,ninghui sun,guang r gao

2009-01-01

Abstract:Dynamic programming (DP) is a popular technique which is used to solve combinatorial search and optimization problems. This paper focuses on one type of DP, which is called nonserial polyadic dynamic programming (NPDP). Owing to the nonuniform data dependencies of NPDP, it is difficult to exploit either parallelism or locality. Worse still, the emerging multi/many-core architectures with small on-chip memory make these issues more challenging. In this paper, we address the challenges of exploiting the fine grain parallelism and locality of NPDP on multicore architectures. We describe a latency-tolerant model and a percolation technique for programming on multicore architectures. On an algorithmic level, both parallelism and locality do benefit from a specific data dependence transformation of NPDP. Next, we propose a parallel pipelining algorithm by decomposing computation operators and percolating data through a memory hierarchy to create just-in-time locality. In order to predict the execution time, we formulate an analytical performance model of the parallel algorithm. The parallel pipelining algorithm achieves not only high scalability on the 160-core IBM Cyclops64, but portable performance as well, across the 8-core Sun Niagara and quad-cores Intel Clovertown.
A Parallel Dynamic Programming Algorithm on a Multi-Core Architecture

Guangming Tan,Ninghui Sun,Guang R. Gao

DOI: https://doi.org/10.1145/1248377.1248399

2007-01-01

Abstract:Dynamic programming is an efficient technique to solve combinatorial search and optimization problem. There have been many parallel dynamic programming algorithms. The purpose of this paper is to study a family of dynamic programming algorithm where data dependence appear between non-consecutive stages, in other words, the data dependence is non-uniform. This kind of dynnamic programming is typically called nonserial polyadic dynamic programming . Owing to the non-uniform data dependence, it is harder to optimize this problem for parallelism and locality on parallel architectures. In this paper, we address the chanllenge of exploiting fine grain parallelism and locality of nonserial polyadic dynamic programming on a multi-core architecture. We present a programming and execution model for multi-core architectures with memory hierarchy. In the framework of the new model, the parallelism and locality benifit from a data dependence transformation. We propose a parallel pipelined algorithm for filling the dynamic programming matrix by decomposing the computation operators. The new parallel algorithm tolerates the memory access latency using multi-thread and is easily improved with tile technique. We formulate and analytically solve the optimization problem determing the tile size that minimizes the total execution time. The experiments on a simulator give a validation of the proposed model and show that the fine grain parallel algorithm achieves sub-linear speedup and that a potential high scalability on multi-core arichitecture.
Speculative Parallelization of Sequential Loops on Multicores

Chen Tian,Min Feng,Vijay Nagarajan,Rajiv Gupta

DOI: https://doi.org/10.1007/s10766-009-0111-z

2009-01-01

International Journal of Parallel Programming

Abstract:The advent of multicores presents a promising opportunity for speeding up the execution of sequential programs through their parallelization. In this paper we present a novel solution for efficiently supporting software-based speculative parallelization of sequential loops on multicore processors. The execution model we employ is based upon state separation , an approach for separately maintaining the speculative state of parallel threads and non-speculative state of the computation. If speculation is successful, the results produced by parallel threads in speculative state are committed by copying them into the computation’s non-speculative state. If misspeculation is detected, no costly state recovery mechanisms are needed as the speculative state can be simply discarded. Techniques are proposed to reduce the cost of data copying between non-speculative and speculative state and efficiently carrying out misspeculation detection. We apply the above approach to speculative parallelization of loops in several sequential programs which results in significant speedups on a Dell PowerEdge 1900 server with two Intel Xeon quad-core processors.
Speculative Parallelization Using State Separation and Multiple Value Prediction.

Chen Tian,Min Feng,Rajiv Gupta

DOI: https://doi.org/10.1145/1837855.1806663

2010-01-01

Abstract:With the availability of chip multiprocessor (CMP) and simultaneous multithreading (SMT) machines, extracting thread level parallelism from a sequential program has become crucial for improving performance. However, many sequential programs cannot be easily parallelized due to the presence of dependences. To solve this problem, different solutions have been proposed. Some of them make the optimistic assumption that such dependences rarely manifest themselves at runtime. However, when this assumption is violated, the recovery causes very large overhead. Other approaches incur large synchronization or computation overhead when resolving the dependences. Consequently, for a loop with frequently arising cross-iteration dependences, previous techniques are not able to speed up the execution. In this paper we propose a compiler technique which uses state separation and multiple value prediction to speculatively parallelize loops in sequential programs that contain frequently arising cross-iteration dependences. The key idea is to generate multiple versions of a loop iteration based on multiple predictions of values of variables involved in cross-iteration dependences (i.e., live-in variables). These speculative versions and the preceding loop iteration are executed in separate memory states simultaneously. After the execution, if one of these versions is correct (i.e., its predicted values are found to be correct), then we merge its state and the state of the preceding iteration because the dependence between the two iterations is correctly resolved. The memory states of other incorrect versions are completely discarded. Based on this idea, we further propose a runtime adaptive scheme that not only gives a good performance but also achieves better CPU utilization. We conducted experiments on 10 benchmark programs on a real machine. The results show that our technique can achieve 1.7x speedup on average across all used benchmarks.
Efficient Nonserial Polyadic Dynamic Programming on the Cell Processor

Li Liu,Mu Wang,Jinlei Jiang,Ruizhe Li,Guangwen Yang

DOI: https://doi.org/10.1109/ipdps.2011.186

2011-01-01

Abstract:Dynamic programming (DP) is an effective technique for many search and optimization problems. However, the high arithmetic complexity limits its extensive use. Although modern processor architectures with multiple cores and SIMD (single instruction multiple data) instructions provide increasingly high computing power, even the state-of-the-art fully optimized algorithm still largely underutilizes modern multi-core processors. In this paper we propose to improve one family of DP, nonserial polyadic DP (NPDP), targeting a heterogeneous multi-core architecture, the Cell Broadband Engine. We first design a new data layout which efficiently utilizes the on-chip memory system of the Cell processor. Next we devise a CellNPDP algorithm with two tiers. The first tier is a SPE (a co-processor on the Cell processor) procedure which efficiently computes a block of data that can fit into one SPE's local store. The second tier is a parallel procedure which enables all SPEs to efficiently compute all blocks of data. To evaluate CellNPDP, we use both performance modeling and experiments. The performance model reveals that the processor utilization of NPDP can be independent of the problem size. To empirically evaluate CellNPDP, we use two platforms: the IBM QS20 dual-Cell blade and a CPU platform with two latest quad-core CPUs. On both platforms, the processor utilization of CellNPDP is larger than 60%, which demonstrates that our optimizations and CellNPDP can be architecture-independent. Compared to the state-of-the-art fully optimized algorithm on the CPU platform, CellNPDP is 44-fold faster for single-precision and 28-fold faster for double-precision, which is a significant improvement to NPDP.
Partitioning the Conventional DBT System for Multiprocessors

Ru-Hui Ma,Hai-Bing Guan,Er-Zhou Zhu,Hong-Bo Yang,Yin-Dong Yang,A-Lei Liang

DOI: https://doi.org/10.1007/s11390-011-1148-1

2011-01-01

Abstract:Noticeable performance improvement via ever-increasing transistors is gradually trapped into a predicament since software cannot logically and efficiently utilize hardware resource, such as multi-core resource. This is an inevitable problem in dynamic binary translation (DBT) system as well. Though special purpose hardware as aide tool, through some interfaces, provided by DBT enables the system to achieve higher performance, the limitation of it is significant, that is, it is impossible to be used widely by another one. To overcome this drawback, we focus on building compatible software architecture to acquire higher performance without platform dependence. In this paper, we propose a novel multithreaded architecture for DBT system through partitioning distinct function module, which is to adequately utilize multiprocessors resource. This new architecture devides couples the common DBT system (DBTs) working routine into dynamic translation, optimization, and translated code execution phases, and then ramifies them into different threads to enable them concurrently executed. In this new architecture, several efficient novel methods are presented to cope with intractable work that puzzles most researchers, such as communication mechanism, cache layout, and mutual exclusion between threads. Experimental results using SPECint 2000 indicate that this new architecture for DBT system can achieve higher performance — speed up the traditional DBT system by about average 10.75%, with better CPU utilization.
Parallel and (Nearly) Work-Efficient Dynamic Programming

Xiangyun Ding,Yan Gu,Yihan Sun

DOI: https://doi.org/10.1145/3626183.3659958

2024-05-22

Abstract:The idea of dynamic programming (DP), proposed by Bellman in the 1950s, is one of the most important algorithmic techniques. However, in parallel, many fundamental and sequentially simple problems become more challenging, and open to a (nearly) work-efficient solution (i.e., the work is off by at most a polylogarithmic factor over the best sequential solution). In fact, sequential DP algorithms employ many advanced optimizations such as decision monotonicity or special data structures, and achieve better work than straightforward solutions. Many such optimizations are inherently sequential, which creates extra challenges for a parallel algorithm to achieve the same work bound.

Data Structures and Algorithms,Distributed, Parallel, and Cluster Computing
Improving Dynamically-Generated Code Performance on Dynamic Binary Translators

Wenwen Wang,Jiacheng Wu,Xiaoli Gong,Tao Li,Pen-Chung Yew

DOI: https://doi.org/10.1145/3186411.3186413

2018-01-01

ACM SIGPLAN Notices

Abstract:The recent transition in the software industry toward dynamically generated code poses a new challenge to existing dynamic binary translation (DBT) systems. A significant re-translation overhead could be introduced due to the maintenance of the consistency between the dynamically-generated guest code and the corresponding translated host code. To address this issue, this paper presents a novel approach to optimize DBT systems for guest applications with dynamically-generated code. The proposed approach can maximize the reuse of previously translated host code to mitigate the re-translation overhead. A prototype based on such an approach has been implemented on an existing DBT system HQEMU. Experimental results on a set of JavaScript applications show that it can achieve a 1.24X performance speedup on average compared to the original HQEMU.
A Language of Suggestions for Program Parallelization

Chao Zhang,Chen Ding,Kirk Kelsey,Tongxin Bai,Xiaoming Gu,Xiaobing Feng

2009-01-01

Abstract:Coarse-grained task parallelism exists in sequential code and can be leveraged to boost the use of chip multi-processors. However, large tasks may execute thousands of lines of code and are often too complex to analyze and manage statically. This report describes a programming system called suggestible parallelization. It consists of a programming in- terface and a support system. The interface is a small language with three primitives for marking possibly parallel tasks and their possible dependences. The support system is implemented in software and ensures correct parallel execution through speculative par- allelization, speculative communication and speculative memory allocation. It manages parallelism dynamically to tolerate unevenness in task size, inter-task delay and hardware speed. When evaluated using four full-size benchmark applications, suggestible parallelization obtains up to a 6 times speedup over 10 processors for sequential legacy applications up to 35 thousand lines in size. The overhead of software speculation is not excessively high compared to unprotected parallel execution.
Parallel Execution of Non-Do Loops

Hai-Bo Chen,Yun-Gui Ci

1987-01-01

Abstract:Speculative parallel execution of non-analyzable codes on Distributed Shared-Memory (DSM) multiprocessors is challenging due to the long-latency and distribution involved. However , such an approach may well be the best way of speeding up codes whose dependences can not be compiler analyzed. In previous work, we suggested executing the loop speculatively in parallel and adding extensions to the memory hierarchy hardware to detect any dependence violation. If the violation occurs, execution is interrupted, the variables are restored, and the code is re-executed serially. The scheme is targeted to loops where most of the invocations turn out to run in parallel without any dependence violation. In this paper, we present a more advanced scheme for the speculative parallel execution of loops that have a modest number of cross-iteration dependences. In this case, when a dependence violation is detected, we locally repair the state. Then, we restart parallel execution from that point on. We call the general algorithm the Sliding Commit algorithm. If the loop dependences are of the special form of reduction, we use a specialized algorithm. Simulations indicate signiicant speedups relative to sequential execution. Finally, we propose hardware for optimizing reductions and obtain very good experimental results.
PerfDBT: Efficient Performance Regression Testing of Dynamic Binary Translation

Jin Wu,Jian Dong,Ruili Fang,Wenwen Wang,Decheng Zuo

DOI: https://doi.org/10.1109/iccd50377.2020.00071

2020-01-01

Abstract:Dynamic binary translation (DBT) has been adopted in many important applications. Due to the large scale and complexity of a DBT system, a minor code change may lead to unexpected impact on the performance. Therefore, it is necessary to conduct performance regression testing for DBT systems. However, existing benchmark suites are not suitable for daily performance testing due to the extremely-long testing time. To address this challenge, we propose PerffiBT, which employs a novel approach to automatically generate test programs from existing long-running benchmarks. The execution times of the generated test programs are much shorter, which allows them to be used for daily performance regression testing of a DBT system. Experimental results demonstrate that the test programs generated by PerfDBT can achieve an average of 71X testing efficiency compared to original benchmarks. Furthermore, it can also deliver similar testing results to original benchmarks.
A Dynamic-Static Combined Code Layout Reorganization Approach for Dynamic Binary Translation.

Haibing Guan,Erzhou Zhu,Kai Chen,Ruhui Ma,Yunchao He,Haipeng Deng,Hongbo Yang

DOI: https://doi.org/10.4304/jsw.6.12.2341-2349

2011-01-01

Abstract:Dynamic binary translation (DBT) has attracted much attention as a powerful technique for the runtime adaptation of software among different ISAs. It offers unprecedented flexibility in the control and modification of a program during the runtime. However, its inherent high overhead has perplexed researchers for many years. In order to reduce the overhead of DBT, this paper presents a dynamic-static combined approach to reorganize the layout of software cache. Under this approach, we first employ an emulating execution to collect the profile information and the translated target code. Especially, the path of execution flow will be tracked. In the static phase, based on the profile information collected in the previous stage, we first use the method of code replicating to build the traces, and then reorganize the layout of the target code by putting the hottest traces at the top of the software cache. Because of exact prediction and improved locality, the execution stream will concentrate on a small area with less control transfer. This approach can greatly reduce the overhead of DBT on the condition that the program runs repeatedly. Experimental results on executing the SPEC 2000 benchmarks show that our approach can reduce more than 30% run time on average.
Improving SIMD Parallelism via Dynamic Binary Translation

Ding-Yong Hong,Yu-Ping Liu,Sheng-Yu Fu,Jan-Jan Wu,Wei-Chung Hsu

DOI: https://doi.org/10.1145/3173456

2018-06-02

ACM Transactions on Embedded Computing Systems

Abstract:Recent trends in SIMD architecture have tended toward longer vector lengths, and more enhanced SIMD features have been introduced in newer vector instruction sets. However, legacy or proprietary applications compiled with short-SIMD ISA cannot benefit from the long-SIMD architecture that supports improved parallelism and enhanced vector primitives, resulting in only a small fraction of potential peak performance. This article presents a dynamic binary translation technique that enables short-SIMD binaries to exploit benefits of new SIMD architectures by rewriting short-SIMD loop code. We propose a general approach that translates loops consisting of short-SIMD instructions to machine-independent IR, conducts SIMD loop transformation/optimization at this IR level, and finally translates to long-SIMD instructions. Two solutions are presented to enforce SIMD load/store alignment, one for the problem caused by the binary translator’s internal translation condition and one general approach using dynamic loop peeling optimization. Benchmark results show that average speedups of 1.51× and 2.48× are achieved for an ARM NEON to x86 AVX2 and x86 AVX-512 loop transformation, respectively.

computer science, software engineering, hardware & architecture

Feasibility of Dynamic Binary Parallelization

Automatic Parallelization for Binary on Multi-core Platforms

GXBIT: Combining polyhedral model with dynamic binary translation

A Translation Framework for Executing the Sequential Binary Code on CPU/GPU Based Architectures

The Optimizations in Dynamic Binary Translation

The Implementation of Dynamic Linking in Dynamic Binary Translation Systems

Improving Performance of Dynamic Programming Via Parallelism and Locality on Multicore Architectures

DDP-B - A Distributed Dynamic Parallel Framework for Meta-genomics Binary Similarity.

Improving Performance of Dynamic Programming via Parallelism and Locality on

A Parallel Dynamic Programming Algorithm on a Multi-Core Architecture

Speculative Parallelization of Sequential Loops on Multicores

Speculative Parallelization Using State Separation and Multiple Value Prediction.

Efficient Nonserial Polyadic Dynamic Programming on the Cell Processor

Partitioning the Conventional DBT System for Multiprocessors

Parallel and (Nearly) Work-Efficient Dynamic Programming

Improving Dynamically-Generated Code Performance on Dynamic Binary Translators

A Language of Suggestions for Program Parallelization

Parallel Execution of Non-Do Loops

PerfDBT: Efficient Performance Regression Testing of Dynamic Binary Translation

A Dynamic-Static Combined Code Layout Reorganization Approach for Dynamic Binary Translation.

Improving SIMD Parallelism via Dynamic Binary Translation