Abstract:In recent years, tensor computation has become a promising tool for solving big data analysis, machine learning, medical image, and EDA problems. To ease the memory and computation intensity of tensor processing, decomposition techniques, especially tensor-train decomposition (TTD), are widely adopted to compress the extremely high-dimensional tensor data. Despite TTD’s potential to break the curse of dimensionality, researchers have not yet leveraged its full computational potential, mainly because of two reasons: 1) executing TTD itself is time- and energy-consuming due to the singular value decomposition (SVD) operation inside each of TTD’s iteration and 2) additional software/hardware optimizations are often required to process the obtained TT-format data in certain applications such as deep learning inference. In this article, we address these challenges with two approaches. First, we propose an algorithm-hardware co-design with customized architecture, namely, TTD Engine to accelerate TTD. We use MRI image compression as a demo application to illustrate the efficacy of the proposed accelerator. Second, we present a case study demonstrating the benefit of TT-format data processing and the efficacy of using TTD Engine. In the case study, we use the TT approach to realize convolution operation, which is difficult and nontrivial for TT-format data. Experimental results show that, TTD Engine achieves, on average, $14.9 \times $ – $36.9 \times $ speedup over CPU implementations and $4.1\times $ – $9.9\times $ speedup compared to the GPU baseline. The energy efficiency is also improved by at least $14.4\times $ and $5.4\times $ over CPU and GPU, respectively. Moreover, our hardware-enabled TT-format data processing further leads to more efficient implementations of complicated operations and applications.

A Carbon-Nanotube-based Tensor Processing Unit

Hardware accelerators based on nanotube transistors

DaDianNao: A Machine-Learning Supercomputer

Monolithic Three‐dimensional Integration of Aligned Carbon Nanotube Transistors for High‐performance Integrated Circuits

15.4 A 5.99-to-691.1TOPS/W Tensor-Train In-Memory-Computing Processor Using Bit-Level-Sparsity-Based Optimization and Variable-Precision Quantization

A 5.99-to-691.1TOPS/W Tensor-Train In-Memory-Computing Processor Using Bit-Level-Sparsity-Based Optimization and Variable-Precision Quantization

7.2 A 12nm Programmable Convolution-Efficient Neural-Processing-Unit Chip Achieving 825TOPS

EN-T: Optimizing Tensor Computing Engines Performance via Encoder-Based Methodology

Photonic tensor cores for machine learning

Photonic tensor processing unit with single dataflow and programmable high-precision weighting control

TT@CIM: A Tensor-Train In-Memory-Computing Processor Using Bit-Level-Sparsity Optimization and Variable Precision Quantization

Heterogeneous Integration of In-Memory Analog Computing Architectures with Tensor Processing Units

GPTPU: Accelerating Applications using Edge Tensor Processing Units

Simulation of quantum physics with Tensor Processing Units: brute-force computation of ground states and time evolution

Layer-Wise Mixed-Modes CNN Processing Architecture With Double-Stationary Dataflow and Dimension-Reshape Strategy

Hardware-Enabled Efficient Data Processing with Tensor-Train Decomposition

A 3.89-Gops/mw Scalable Recurrent Neural Network Processor with Improved Efficiency on Memory and Computation

High-Performance Carbon Nanotube Thin-Film Transistor Technology

Flex-TPU: A Flexible TPU with Runtime Reconfigurable Dataflow Architecture

TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddings