A Many-Ported and Shared Memory Architecture for High-Performance ADAS SoCs

Hao Luan,Yu Yao,Chang Huang

DOI: https://doi.org/10.1109/mdat.2022.3202997

2022-10-29

IEEE Design and Test

Abstract:This article presents a shared memory architecture to enable high data throughput among multiple parallel accesses native to advanced driver assistance system (ADAS) applications.

engineering, electrical & electronic,computer science, hardware & architecture

What problem does this paper attempt to address?

Evaluation and Design Trade-Offs Between Symmetric Shared-Memory and Distributed Shared-Memory

LI Lei,SHEN Hai-bin,YAN Xiao-lang

DOI: https://doi.org/10.3969/j.issn.1671-7147.2008.03.001

2008-01-01

Abstract:In order to make reference in choosing different memory mechanisms in early stage of multiprocessor design process,we integrated two kinds of memory mechanisms(shared and distributed memories) into Simulink-based MPSoC design flow to evaluate the performance for both mechanisms.In Motion-JPEG decoder case study,the developing time for Global FIFO(GFIFO),which represented for GSM,is less than DMS,which represented DSM.DMS decreased execution time and communication time compared to GFIFO.We can make accurate decisions in early multiprocessors design stages based on the results.
Combinatorics and Geometry for the Many-ported, Distributed and Shared Memory Architecture

Hao Luan,Alan Gatherer

DOI: https://doi.org/10.48550/arXiv.2010.08667

2020-10-17

Abstract:Manycore SoC architectures based on on-chip shared memory are preferred for flexible and programmable solutions in many application domains. However, the development of many ported memory is becoming increasingly challenging as we approach the end of Moore's Law while systems requirements demand larger shared memory and more access ports. Memory can no longer be designed simply to minimize single transaction access time, but must take into account the functionality on the SoC. In this paper we examine a common large memory usage in SoC, where the memory is used as storage for large buffers that are then moved for time scheduled processing. We merge two aspects of many ported memory design, combinatorial analysis of interconnect, and geometric analysis of critical paths, extending both to show that in this case the SoC performance benefits significantly from a hierarchical, distributed and staged architecture with lower-radix switches and fractal randomization of memory bank addressing, along with judicious and geometry aware application of speed up. The results presented show the new architecture supports 20% higher throughput with 20% lower latency and 30% less interconnection area at approximately the same power consumption. We demonstrate the flexibility and scalability of this architecture on silicon from a physical design perspective by taking the design through layout. The architecture enables a much easier implementation flow that works well with physically irregular port access and memory dominant layout, which is a common issue in real designs.

Hardware Architecture
DaDianNao: A Machine-Learning Supercomputer

Yunji Chen,Tao Luo,Shaoli Liu,Shijin Zhang,Liqiang He,Jia Wang,Ling Li,Tianshi Chen,Zhiwei Xu,Ninghui Sun,Olivier Temam

DOI: https://doi.org/10.1109/micro.2014.58

2014-01-01

Abstract:Many companies are deploying services, either for consumers or industry, which are largely based on machine-learning algorithms for sophisticated processing of large amounts of data. The state-of-the-art and most popular such machine-learning algorithms are Convolutional and Deep Neural Networks (CNNs and DNNs), which are known to be both computationally and memory intensive. A number of neural network accelerators have been recently proposed which can offer high computational capacity/area ratio, but which remain hampered by memory accesses. However, unlike the memory wall faced by processors on general-purpose workloads, the CNNs and DNNs memory footprint, while large, is not beyond the capability of the on-chip storage of a multi-chip system. This property, combined with the CNN/DNN algorithmic characteristics, can lead to high internal bandwidth and low external communications, which can in turn enable high-degree parallelism at a reasonable area cost. In this article, we introduce a custom multi-chip machine-learning architecture along those lines. We show that, on a subset of the largest known neural network layers, it is possible to achieve a speedup of 450.65x over a GPU, and reduce the energy by 150.31x on average for a 64-chip system. We implement the node down to the place and route at 28nm, containing a combination of custom storage and computational units, with industry-grade interconnects.
Design Of A Dynamic Memory Access Scheduler

Jun Zheng,Kang Sun,Xuezeng Pan,Lingdi Ping

DOI: https://doi.org/10.1109/ICASIC.2007.4415557

2007-01-01

Abstract:Computer systems are becoming increasingly limited by memory performance. A dynamic SDRAM access scheduler (DSAS) according to modem SDRAM technology and memory access scheduling algorithms is proposed in this paper. Based on SDR-SDRAM technology and new version AMBA AXI bus, DSAS dynamically schedules the accesses to SDRAM and reduces precharge time. A modularized configurable automatic verification platform is established to verify basic functions of DSAS. We analyze the results and conclude that the memory controller using DSAS is capable of predicting future operations, thus greatly hide the precharge time, which takes much time in SDRAM operations; AXI bus throughput rate has improved by 19%similar to 52% during frequent SDRAM accesses.
Image Processing Memory Optimization for Multi-camera Based Advanced Driver Assistance Systems

Liu Genxian,Wang Dongsheng,Wang Haixia,Liu Zhenyu

DOI: https://doi.org/10.1109/ICMTMA.2014.78

2014-01-01

Abstract:Advanced Driver Assistance Systems (ADASs) have become a hot research area in intelligent transportation systems which are important for building Wisdom City. The major challenge of ADASs is to meet the real-time image processing requirements, especially when multiple cameras are adopted into ADASs. The image processing algorithms are highly data dominated, which means it is crucial to reduce memory access latency for improving performance of image processing. However, no efficient memory optimization studies have been performed for ADASs yet. In the paper we introduced a new unit, called Image Processing Acceleration Unit (IPAU), for optimizing the memory bandwidth of real-time video processing systems. The first feature of IPAU is block mode data layout in DRAM devices. The second feature is multiscale block mode hardware support through reorder buffer. The last feature is hardware rotation accelerator. Based on evaluation results, memory with IPAU can improve memory bandwidth by 49% compared to a memory hierarchy without IPAU, and reduce the energy consumption by 30% on the average.
Configurable Multi-Port Memory Architecture for High-Speed Data Communication

Narendra Singh Dhakad,Santosh Kumar Vishvakarma

2024-07-30

Abstract:Memory management is necessary with the increasing number of multi-connected AI devices and data bandwidth issues. For this purpose, high-speed multi-port memory is used. The traditional multi-port memory solutions are hard-bounded to a fixed number of ports for read or write operations. In this work, we proposed a pseudo-quad-port memory architecture. Here, ports can be configured (1-port, 2-port, 3-port, 4-port) for all possible combinations of read/write operations for the 6T static random access memory (SRAM) memory array, which improves the speed and reduces the bandwidth for data transfer. The proposed architecture improves the bandwidth of data transfer by 4x. The proposed solution provides 1.3x and 2x area efficiency as compared to dual-port 8T and quad-port 12T SRAM. All the design and performance analyses are done using 65nm CMOS technology.

Hardware Architecture
Achieving Multi-Port Memory Performance on Single-Port Memory with Coding Techniques

Hardik Jain,Matthew Edwards,Ethan Elenberg,Ankit Singh Rawat,Sriram Vishwanath

DOI: https://doi.org/10.48550/arXiv.2001.09599

2020-01-27

Abstract:Many performance critical systems today must rely on performance enhancements, such as multi-port memories, to keep up with the increasing demand of memory-access capacity. However, the large area footprints and complexity of existing multi-port memory designs limit their applicability. This paper explores a coding theoretic framework to address this problem. In particular, this paper introduces a framework to encode data across multiple single-port memory banks in order to {\em algorithmically} realize the functionality of multi-port memory. This paper proposes three code designs with significantly less storage overhead compared to the existing replication based emulations of multi-port memories. To further improve performance, we also demonstrate a memory controller design that utilizes redundancy across coded memory banks to more efficiently schedule read and write requests sent across multiple cores. Furthermore, guided by DRAM traces, the paper explores {\em dynamic coding} techniques to improve the efficiency of the coding based memory design. We then show significant performance improvements in critical word read and write latency in the proposed coded-memory design when compared to a traditional uncoded-memory design.

Hardware Architecture
Near Data Acceleration with Concurrent Host Access

Benjamin Y. Cho,Yongkee Kwon,Sangkug Lym,Mattan Erez

DOI: https://doi.org/10.1109/isca45697.2020.00072

2020-05-01

Abstract:Near-data accelerators (NDAs) that are integrated with the main memory have the potential for significant power and performance benefits. Fully realizing these benefits requires the large available memory capacity to be shared between the host and NDAs in a way that permits both regular memory access by some applications and accelerating others with an NDA, avoids copying data, enables collaborative processing, and simultaneously offers high performance for both host and NDA. We identify and solve new challenges in this context: mitigating row-locality interference from host to NDAs, reducing read/write-turnaround overhead caused by fine-grain interleaving of host and NDA requests, architecting a memory layout that supports the locality required for NDAs and sophisticated address inter-leaving for host performance, and supporting both packetized and traditional memory interfaces. We demonstrate our approach in a simulated system that consists of a multi-core CPU and NDA-enabled DDR4 memory modules. We show that our mechanisms enable effective and efficient concurrent access using a set of microbenchmarks, then demonstrate the potential of the system for the important stochastic variance-reduced gradient (SVRG) algorithm.
Design Space Exploration of Algorithmic Multi-Port Memories in High-Performance Application-Specific Accelerators

Khushal Sethi

DOI: https://doi.org/10.48550/arXiv.2007.09363

2022-09-04

Abstract:Memory load/store instructions consume an important part in execution time and energy consumption in domain-specific accelerators. For designing highly parallel systems, available parallelism at each granularity is extracted from the workloads. The maximal use of parallelism at each granularity in these high-performance designs requires the utilization of multi-port memories. Currently, true multiport designs are less popular because there is no inherent EDA support for multiport memory beyond 2-ports, utilizing more ports requires circuit-level implementation and hence a high design time. In this work, we present a framework for Design Space Exploration of Algorithmic Multi-Port Memories (AMM) in ASICs. We study different AMM designs in the literature, discuss how we incorporate them in the Pre-RTL Aladdin Framework with different memory depth, port configurations and banking structures. From our analysis on selected applications from the MachSuite (accelerator benchmark suite), we understand and quantify the potential use of AMMs (as true multiport memories) for high performance in applications with low spatial locality in memory access patterns.

Hardware Architecture
Design of Dual-Shared Dram Controller Based on Switch

Yifeng Li,Bo Zhang,Xiaoxia Han,Gang Zhang

DOI: https://doi.org/10.1007/978-3-642-24273-1_41

2011-01-01

Abstract:According to the harsh desire to share memory of multi-processors system on chip, this paper presents a switch-based Dual-Shared DRAM Controller (DSMC). The DSMC is composed of center control module. two IP(interface to processor) module, two ID(interface to DRAM) module, CM(Clock Manager) module, and IR(initialize and refresh) module. It use two memories as shared memory and resolves conflicts which may occur when two processors access the same memory, and actualizes every control to DRAM, such as initializing, refreshing, reading and writing. At last, it makes two processors access the shared memories coinstantaneous at will.
High Throughput Memory Data-Path Design For Multi-Core Architecture

Li Jinsong,Du Gaoming,Zhang Duoli,Song Yukun,Li Li,Pan Hongbing

DOI: https://doi.org/10.1109/CAR.2010.5456686

2010-01-01

Abstract:Network on chip (NoC) has been proposed as new on-chip communication paradigm for the muti-core processing era. But the memory wall problem is a design bottleneck, especially in real-time applications. This paper proposes a high throughput memory data-path design that can guarantee realtime I/O throughput for an in-house developed multi-core system. The main contribution is as follows: Firstly, a data path DMA controller between the local processing cores and the on chip communication network is designed. Once the data access is established, it can transfer one data per cycle. Secondly, preprogrammed mechanism is used to schedule data exchange to reduce more cycles of register configuration. The design was successfully implemented on FPGA with fade-in-fade-out realtime video application. Experiment results show that 1024-bit data can be transferred in only 51 cycles and the system can run at 90MHz.
Design and Implementation of an AHB-based High-Speed Memory Access Layer

YAN Dong,TONG Dong

DOI: https://doi.org/10.3969/j.issn.1000-1220.2006.08.037

2006-01-01

Abstract:For high-speed I/O devices in System-on-a-chip (SoC), DMA support will relieve the processor from data transfer, and improve the performance of the overall system. This paper investigates the features of on-chip system bus and DMA mechanism, and design an AHB-based High-speed Memory Access Layer that supports data transfer between ehternet network interfaces and system memory. In the implementation of HMAL, the bus interface design is optimized, and an analytical method is proposed for determining the FIFO design parameter. Experimental results show that HMAL promotes the data transfer rate effectually and support network application-specified SoCs.
Asynchronous Memory Access Unit: Exploiting Massive Parallelism for Far Memory Access

Luming Wang,Xu Zhang,Songyue Wang,Zhuolun Jiang,Tianyue Lu,Mingyu Chen,Siwei Luo,Keji Huang

2024-04-17

Abstract:The growing memory demands of modern applications have driven the adoption of far memory technologies in data centers to provide cost-effective, high-capacity memory solutions. However, far memory presents new performance challenges because its access latencies are significantly longer and more variable than local DRAM. For applications to achieve acceptable performance on far memory, a high degree of memory-level parallelism (MLP) is needed to tolerate the long access latency. While modern out-of-order processors are capable of exploiting a certain degree of MLP, they are constrained by resource limitations and hardware complexity. The key obstacle is the synchronous memory access semantics of traditional load/store instructions, which occupy critical hardware resources for a long time. The longer far memory latencies exacerbate this limitation.

Hardware Architecture
A Shared Memory Module for Asynchronous Arrays of Processors

Michael J Meeuwsen,Zhiyi Yu,Bevan M Baas

DOI: https://doi.org/10.1155/2007/86273

2007-01-01

EURASIP Journal on Embedded Systems

Abstract:A shared memory module connecting multiple independently clocked processors is presented. The memory module itself is independently clocked, supports hardware address generation, mutual exclusion, and multiple addressing modes. The architecture supports independent address generation and data generation/consumption by different processors which increases efficiency and simplifies programming for many embedded and DSP tasks. Simultaneous access by different processors is arbitrated using a least-recently-serviced priority scheme. Simulations show high throughputs over a variety of memory loads. A standard cell implementation shares an 8 K-word SRAM among four processors, and can support a 64 K-word SRAM with no additional changes. It cycles at 555 MHz and occupies 1.2 mm2 in 0.18 μ m CMOS.
Agent-Based Memory Access for Many-Core CMPs

Weiwei Fu,Mingmin Yuan,Tianzhou Chen,Li Liu

DOI: https://doi.org/10.1109/ISPDC.2014.9

2014-01-01

Abstract:The trend of increasing on-chip core counts and integrating memory controllers (MCs) makes core-to-memory communication a major obstacle in scaling memory access performance for many-core CMPs. Unmanaged on-chip traffic for long-distance memory accesses, combined with information asymmetry between cores and remote MCs may lead to serious inefficiency in processing massive parallel memory accesses. In this paper we propose a novel agent-based memory access model for CMPs. We employ multiple agents inside the network to assist memory accesses to remote MCs, whose role lies in conducting memory requests from nearby cores, merging some repetitive memory requests and optimizing the scheduling under the backpressure of target MCs. We further describe a simple but effective case for agent-based memory access called Quad Agent, which deploys static agent modules in each quadrant of the network to serve requests towards MCs in other quadrants. The details of memory access merging, scheduling schemes and the architectural supports are discussed. Simulation results show that Quad Agent can reduce memory access latency by 13.3% on average and achieve 20.5% IPC speedup compared with the baseline. The performance promotion is due to increased memory access merge rate, row buffer hit rate and also prevention of traffic congestion and bank starvation.
High‐speed end‐to‐edge data caching and forwarding architecture based on field programmable gate array

Xiao Liu,Shubo Liu,Juncheng Wu,Song Song,Zhaohui Cai,Guoqing Tu

DOI: https://doi.org/10.4218/etrij.2024-0212

2024-11-11

ETRI Journal

Abstract:The application of the Internet of Things generates massive end‐device data. Traditional end‐to‐edge data caching methods lack parallelism, consume numerous resources, and lack data backup mechanisms. We propose a lightweight high‐speed data caching and forwarding architecture based on a field‐programmable gate array. In the basic scheme, a double‐data‐rate memory read/write mixed control method is proposed to efficiently cache massive amounts of data. To improve reliability, we propose address‐space virtualization to forward data to backup devices. To satisfy the demand for higher performance scenarios, we propose a real‐time data shunting mechanism to realize an optimized scheme that supports higher data rates. These two schemes support single‐link high‐speed data caching and forwarding at 10 and 40 Gbps. Compared with the existing method, the basic and optimized schemes improve the performance by 25% and four times and save 66% and 40% of the resources, respectively. Thus, we provide a lightweight and reliable high‐performance end‐to‐edge data caching solution for resource‐constrained edge devices.

telecommunications,engineering, electrical & electronic
AiDAC: A Low-Cost In-Memory Computing Architecture with All-Analog Multi-Bit Compute and Interconnect

Zihao Xuan,Song Chen,Yi Kang

2023-12-21

Abstract:Analog in-memory computing (AiMC) is an emerging technology that shows fantastic performance superiority for neural network acceleration. However, as the computational bit-width and scale increase, high-precision data conversion and long-distance data routing will result in unacceptable energy and latency overheads in the AiMC system. In this work, we focus on the potential of in-charge computing and in-time interconnection and show an innovative AiMC architecture, named AiDAC, with three key contributions: (1) AiDAC enhances multibit computing efficiency and reduces data conversion times by grouping capacitors technology; (2) AiDAC first adopts row drivers and column time accumulators to achieve large-scale AiMC arrays integration while minimizing the energy cost of data movements. (3) AiDAC is the first work to support large-scale all-analog multibit vector-matrix multiplication (VMM) operations. The evaluation shows that AiDAC maintains high-precision calculation (less than 0.79% total computing error) while also possessing excellent performance features, such as high parallelism (up to 26.2TOPS), low latency (<20ns/VMM), and high energy efficiency (123.8TOPS/W), for 8bits VMM with 1024 input channels.

Hardware Architecture,Artificial Intelligence
A Scalable Distributed Memory Architecture For Network On Chip

Zhang Yuang,Li Li,Yang Shengguang,Dong Lan,Lou Xiaoxiang,Gao Minglun

DOI: https://doi.org/10.1109/APCCAS.2008.4746256

2008-01-01

Abstract:Besides processors, DSPs and other IP blocks, Network on Chip (NoC) also integrates lots of memories. However, the research of on chip memory subsystem for NoC has not been undertaken thoroughly. In this paper, a distributed None Uniform Memory Access (NUMA) memory architecture for NoC is developed; the performance and the programming mode of this system are discussed. Two kinds of parallel algorithms are implemented, the pipelined matrix multiplication and the full-parallel FFT. Simulation results verify the proposed architecture can achieve high parallelism and provide flexible programming scheme.
A Tale of Two Paths: Toward a Hybrid Data Plane for Efficient Far-Memory Applications

Lei Chen,Shi Liu,Chenxi Wang,Haoran Ma,Yifan Qiao,Zhe Wang,Chenggang Wu,Youyou Lu,Xiaobing Feng,Huimin Cui,Shan Lu,Harry Xu

2024-06-23

Abstract:With rapid advances in network hardware, far memory has gained a great deal of traction due to its ability to break the memory capacity wall. Existing far memory systems fall into one of two data paths: one that uses the kernel's paging system to transparently access far memory at the page granularity, and a second that bypasses the kernel, fetching data at the object granularity. While it is generally believed that object fetching outperforms paging due to its fine-grained access, it requires significantly more compute resources to run object-level LRU and eviction. We built Atlas, a hybrid data plane enabled by a runtime-kernel co-design that simultaneously enables accesses via these two data paths to provide high efficiency for real-world applications. Atlas uses always-on profiling to continuously measure page locality. For workloads already with good locality, paging is used to fetch data, whereas for those without, object fetching is employed. Object fetching moves objects that are accessed close in time to contiguous local space, dynamically improving locality and making the execution increasingly amenable to paging, which is much more resource-efficient. Our evaluation shows that Atlas improves the throughput (e.g., by 1.5x and 3.2x) and reduces the tail latency (e.g., by one and two orders of magnitude) when using remote memory, compared with AIFM and Fastswap, the state-of-the-art techniques respectively in the two categories.

Distributed, Parallel, and Cluster Computing
Direct Distributed Memory Access for CMPs

Weiwei Fu,Li Liu,Tianzhou Chen

DOI: https://doi.org/10.1016/j.jpdc.2013.11.004

IF: 4.542

2013-01-01

Journal of Parallel and Distributed Computing

Abstract:On-chip distributed memory has emerged as a promising memory organization for future many-core systems, since it efficiently exploits memory level parallelism and can lighten off the load on each memory module by providing a comparable number of memory interfaces with on-chip cores. The packet-based memory access model (PDMA) has provided a scalable and flexible solution for distributed memory management, but suffers from complicated and costly on-chip network protocol translation and massive interferences among packets, which leads to unpredictable performance. In this paper we propose a direct distributed memory access (DDMA) model, in which remote memory can be directly accessed by local cores via remote-to-local virtualization, without network protocol translation. From the perspective of local cores, remote memory controllers (MC) can be directly manipulated through accessing the local agent MC, which is responsible for accessing remote memory through high-performance inter-tile communication. We further discuss some detailed architecture supports for the DDMA model, including the memory interface design, work flow and the protocols involved. Simulation results of executing PARSEC benchmarks show that our DDMA architecture outperforms PDMA in terms of both average memory access latency and IPC by 17.8% and 16.6% respectively on average. Besides, DDMA can better manage congested memory traffic, since a reduction of bandwidth in running memory-intensive SPEC2006 workloads only incurs 18.9% performance penalty, compared with 38.3% for PDMA.

A Many-Ported and Shared Memory Architecture for High-Performance ADAS SoCs

Evaluation and Design Trade-Offs Between Symmetric Shared-Memory and Distributed Shared-Memory

Combinatorics and Geometry for the Many-ported, Distributed and Shared Memory Architecture

DaDianNao: A Machine-Learning Supercomputer

Design Of A Dynamic Memory Access Scheduler

Image Processing Memory Optimization for Multi-camera Based Advanced Driver Assistance Systems

Configurable Multi-Port Memory Architecture for High-Speed Data Communication

Achieving Multi-Port Memory Performance on Single-Port Memory with Coding Techniques

Near Data Acceleration with Concurrent Host Access

Design Space Exploration of Algorithmic Multi-Port Memories in High-Performance Application-Specific Accelerators

Design of Dual-Shared Dram Controller Based on Switch

High Throughput Memory Data-Path Design For Multi-Core Architecture

Design and Implementation of an AHB-based High-Speed Memory Access Layer

Asynchronous Memory Access Unit: Exploiting Massive Parallelism for Far Memory Access

A Shared Memory Module for Asynchronous Arrays of Processors

Agent-Based Memory Access for Many-Core CMPs

High‐speed end‐to‐edge data caching and forwarding architecture based on field programmable gate array

AiDAC: A Low-Cost In-Memory Computing Architecture with All-Analog Multi-Bit Compute and Interconnect

A Scalable Distributed Memory Architecture For Network On Chip

A Tale of Two Paths: Toward a Hybrid Data Plane for Efficient Far-Memory Applications

Direct Distributed Memory Access for CMPs