Abstract:As consumers are increasingly engaged in social networking and E-commerce activities, businesses grow to rely on Big Data analytics for intelligence, and traditional IT infrastructures continue to migrate to the cloud and edge, these trends cause distributed data storage demand to rise at an unprecedented speed. Erasure coding has seen itself quickly emerged as a promising technique to reduce storage cost while providing similar reliability as replicated systems, widely adopted by companies like Facebook, Microsoft and Google. However, it also brings new challenges in characterizing and optimizing the access latency when data objects are erasure coded in distributed storage. The aim of this monograph is to provide a review of recent progress (both theoretical and practical) on systems that employ erasure codes for distributed storage.<p>In this monograph, we will first identify the key challenges and taxonomy of the research problems and then give an overview of different models and approaches that have been developed to quantify latency of erasure-coded storage. This includes recent work leveraging MDS-Reservation, Fork-Join, Probabilistic, and Delayed-Relaunch scheduling policies, as well as their applications to characterizing access latency (e.g., mean, tail, and asymptotic latency) of erasure-coded distributed storage systems. We will also extend the discussions to video streaming from erasure-coded distributed storage systems. Next, we will bridge the gap between theory and practice, and discuss lessons learned from prototype implementations. In particular, we will discuss exemplary implementations of erasure-coded storage, illuminate key design degrees of freedom and tradeoffs, and summarize remaining challenges in real-world storage systems such as in content delivery and caching. Open problems for future research are discussed at the end of each chapter.</p><h3>Suggested Citation</h3>Vaneet Aggarwal and Tian Lan (2021), "Modeling and Optimization of Latency in Erasure-coded Storage Systems", Foundations and Trends® in Communications and Information Theory: Vol. 18: No. 3, pp 380-525. http://dx.doi.org/10.1561/0100000108

Utility-driven Optimization of TTL Cache Hierarchies under Network Delays

On the Impact of Network Delays on Time-to-Live Caching

What, When and Where to Cache: A Unified Optimization Approach.

Optimizing Timer-based Policies for General Cache Networks.

Exact Analysis of TTL Cache Networks: The Case of Caching Policies driven by Stopping Times

Cache-induced Hierarchical Cooperation in Wireless Device-to-Device Caching Networks

Distributed Packet Forwarding and Caching Based on Stochastic Network Utility Maximization

Delay-Optimal Edge Cache Replacement with Non-Markovian Content Fetching

Network cache design under stationary requests: Exact analysis and Poisson approximation

Delay-Optimal Edge Caching with Imperfect Content Fetching Via Stochastic Learning

Cache Placement in Two-Tier HetNets with Limited Storage Capacity: Cache or Buffer?

Delay Analysis and Optimization in Cache-Enabled Multi-Cell Cooperative Networks.

Modeling and Optimization of Latency in Erasure-coded Storage Systems

Probabilistic Caching in Multi-Tier Heterogeneous Networks: Optimization and Tradeoff

Cache Placement Optimization In Hierarchical Networks: Analysis And Performance Evaluation

On the Convergence of the TTL Approximation for an LRU Cache under Independent Stationary Request Processes

On the Complexity of Optimal Routing and Content Caching in Heterogeneous Networks.

Adaptive Caching Networks with Optimality Guarantees

Storage, Communication, and Load Balancing Trade-off in Distributed Cache Networks

Practical Bounds on Optimal Caching with Variable Object Sizes

Order-optimal Caching in Hierarchical Networks