Abstract:Bare-metal instances are crucial for high-value, mission-critical applications on the cloud. Tenants exclusively use these dedicated hardware resources. Local virtualized disks are essential for bare-metal instances to provide flexible and high-performance storage resources. Traditionally tenants can choose polling-based software virtualization techniques, but they consume too many valuable host CPU cores and suffer from performance degradation. Cloud vendors are hard to deploy existing hardware-assisted local storage solutions in bare-metal instances due to no access to the host OS to install customized drivers. Moreover, cloud vendors have difficulties managing and maintaining the local storage devices in bare-metal instances because hardware resources and host operating systems are completely utilized by tenants, then it will impact the availability of storage devices.This paper presents our design and experience with BM-Store, a novel high-performance hardware-assisted virtual local storage architecture for bare-metal clouds. BM-Store is transparent to the host that tenants are unaware of the underlying hardware architecture. Therefore, it can be deployed on a large scale in cloud vendors. BM-Store consists of two components: an FPGA-based BMS-Engine and an ARM-based BMS-Controller. The BMS-Engine accelerates the I/O path to enable high-performance virtual storage independent of disk devices without consuming any CPU resource on the host. The BMS-Controller is responsible for resource management and maintenance to achieve flexible and high available local storage. The results of the extensive experiments show that BM-Store can achieve near-native performance, which only introduces about 3 µs extra latency and average 4.0% throughput overhead to native disks. Compared to SPDK vhost, BM-Store achieves an average bandwidth improvement of 15.7% in microbenchmark and a maximum throughput enhancement of 13.4% in real-world applications.

Optimizing NVMe Storage for Large-scale Deployment: Key Technologies and Strategies in Alibaba Cloud

LightPool: A NVMe-oF-based High-performance and Lightweight Storage Pool Architecture for Cloud-Native Distributed Database

BM-Store: A Transparent and High-performance Local Storage Architecture for Bare-metal Clouds Enabling Large-scale Deployment

High-performance and Scalable Software-based NVMe Virtualization Mechanism with I/O Queues Passthrough

MBSA: a Lightweight and Flexible Storage Architecture for Virtual Machines

Performance Characterization of SmartNIC NVMe-over-Fabrics Target Offloading

Towards Unaligned Writes Optimization in Cloud Storage With High-Performance SSDs

HyQ: Hybrid I/O Queue Architecture for NVMe over Fabrics to Enable High- Performance Hardware Offloading

xNVMe: Unleashing Storage Hardware-Software Co-design

When Cloud Storage Meets RDMA.

Achelous: Enabling Programmability, Elasticity, and Reliability in Hyperscale Cloud Networks.

Optimising the Deployment of Virtual Machine Image Replicas in Cloud Storage Clusters

Towards Optimizing Storage Costs on the Cloud

An empirical evaluation and analysis of the performance of NVM express solid state drive

Efficient Crash Consistency for NVMe over PCIe and RDMA

Storage and Memory Characterization of Data Intensive Workloads for Bare Metal Cloud

Designing a Secure, Scalable, and Cost-Effective Cloud Storage Solution: A Novel Approach to Data Management using NextCloud, TrueNAS, and QEMU/KVM

Design and application of new storage systems

Cost-effective, Energy-efficient, and Scalable Storage Computing for Large-scale AI Applications

Performance Evaluation and Modeling of HPC I/O on Non-Volatile Memory

Crash Consistent Non-Volatile Memory Express