Redundant Data Strategy Improve Seeking Efficiency Of Optical Array

Haichuan Zhang,Duanyi Xu,Longfa Pan

DOI: https://doi.org/10.1117/12.248717

1996-01-01

Abstract:The average distance between different data will decrease through reasonable data distribution. Redundant MacroBlock data strategy is such a way to reduce seeking distance, and improve seeking efficiency, which is presented in theory and experiment.

What problem does this paper attempt to address?

Character Analysis for Optical Disk Array

HC Zhang,DY Xu,LF Pan

DOI: https://doi.org/10.1117/12.251977

1996-01-01

Abstract:Optical disk array improves performance of optical drive in many aspects. The mechanism of separate drive and disk reduce the MTTR (mean time to repaired) of single failure. So data security increases much better than other array system. Parallel structure and reasonable redundant data distribution multiply the I/O speed much more by reducing average distance between different data.
Optical array for Video-on-Demand (VOD) system

Haichuan Zhang,Longfa Pan,Duanyi Xu

DOI: https://doi.org/10.1364/isom.1996.otub.4

1996-01-01

Abstract:Although RAID (Redundant Array of Inexpensive Disk) technology has been widely used in HDD (Hard Disk Drive) array in the past several years, optical array just begins its steps using optical drive with higher data transfer rate and faster seeking operation1. Optical drive (especially CD-ROM) have improved sustain transfer rate by several times, and some special designed optical drives can reduce seek time near to HDD. These progresses make it possible that low cost optical array become outside memory of video server in Video-on-Demand1,2,3 system instead of expensive HDD array. A new kind of RAID form can reduce distance between different data segments. So this new data strategy reduce the seeking time too, which make optical array can be accessed by more users simultaneously than before.
A Novel Data Redundancy Scheme for De-Duplication Storage System

Zhou Zhengda,Zhou Jingli

DOI: https://doi.org/10.1109/kam.2010.5646152

2010-01-01

Abstract:As the amount of digital data growing explosively, data deduplication becomes an attractive technique for conserving the storage and network requirement. However, the tradeoff between redundancy for data availability and data duplicate elimination for storage utilization becomes an issue in the deduplication storage system. To address the issue, we proposed a novel data redundancy scheme to improve the data availability for the deduplication storage system. The key idea is to optimize the system-level availability by assigning an optimal replication degree for the individual data chunk based on its commonality in the data set. We present the design and implementation of the prototype of the high availability storage system and evaluate the feasibility and effectiveness of the optimization approach. The evaluation experiments demonstrated the approach is practical and effective.
Shifted Element Arrangement in Mirror Disk Arrays for High Data Availability During Reconstruction

Xianghong Luo,Jiwu Shu,Ying Zhao

DOI: https://doi.org/10.1109/icpp.2012.53

2012-01-01

Abstract:The mirror method and its variants are widely used in storage systems. However, they suffer from low data availability during on-line reconstruction because sequential reads are inevitable under the current element arrangements. In this paper, we propose a shifted element arrangement method in mirror disk arrays to provide high data availability during reconstruction, which can be used in both the mirror method and its variants. We design this novel element arrangement to minimize the maximum number of elements that are read out from one disk, thus minimize the number of read accesses during the reconstruction process. Theoretically, we prove that our shifted element arrangement improves data availability by a factor of n or (2n+1)/4 (n is the number of disks in the data disk array), respectively, in the mirror methods without or with parity, while still enjoying the theoretical optimal write efficiency. Our experimental evaluation with n equal to three to seven shows that the shifted element arrangement achieved an improvement of a factor of 1.54 to 4.55 in data availability during reconstruction, and compatible write efficiency.
Optimization for Data De-Duplication Algorithm Based on File Content

Xuejun Nie,Leihua Qin,Jingli Zhou,Ke Liu,Jianfeng Zhu,Yu Wang

DOI: https://doi.org/10.1007/s12200-010-0103-z

2010-01-01

Frontiers of Optoelectronics

Abstract:Content defined chunking (CDC) is a prevalent data de-duplication algorithm for removing redundant data segments in archival storage systems. Current researches on CDC do not consider the unique content characteristic of different file types, and they determine chunk boundaries in a random way and apply a single strategy for all file types. It has been proven that such method cannot achieve optimal performance for compound archival data. We analyze the content characteristic of different file types and propose candidate anchor histogram (CAH) to capture it. We propose an improved strategy for determining chunk boundaries based on CAH and tune some key parameters of CDC based on the data layout of underlying data de-duplication file system (TriDFS), which can efficiently store variable-sized chunks on fixed-sized physical blocks. These strategies are evaluated with representative archival data, and the result indicates that they can increase on average the compression ratio by 16.3% and write throughput by 13.7%, while only decrease the read throughput by 2.5%.
New Way to Improve the Reliability of High-Speed and Mass Data Recording Optical Disk Array

CY Wang,HB Jia,DY Xu

DOI: https://doi.org/10.1117/12.997585

1999-01-01

Abstract:Optical recording has some inborn advantages over magnetic recording. Under some circumstances where magnetic recording can not meet the specific requirements, optical recording can play an important role. In this paper, an optical disk array was designed to record successive data with high speed and mass capacity under the circumstances where air-pressure is fairly low and with high electromagnetic interference.
Optimization for Data De-duplication Algorithm Based on Storage Environment Aware

ZHOU Jing-li,NIE Xue-jun,QIN Lei-hua,LIU Ke,ZHU Jian-feng,WANG Yu

DOI: https://doi.org/10.3969/j.issn.1002-137X.2011.02.014

2011-01-01

Computer Science

Abstract:torage applications such as backup and archive are creating more and more duplication data,which has caused increasing waste in storage space and energy consumption,and how to delete duplication data has become a hot subject in research.CDC(Content-Defined Chunking) is a prevail algorithm for data de-duplication and can be applicable in various environment,however it does not take into account some characteristics which are specific to individual envi-ronment and can influence its result.We studied the CDC’s application in storage system and put up with two con-straints for determining parameters for CDC:(1)Determining parameters such as average block size based on the block organization of storage devices;(2)Determining block boundary based on candidate boundary distribution.The result in-dicates that,comparing with the separate CDC without restraint conditions,these two constraints can achieve 16.3% higher compression ratio with 4 data sets.
Hyperplane-based Block Data Distribution Strategy

Jian-cheng DU,Hao HUANG,Wei-jiang YU,Dao-xu CHEN,Li XIE

1999-01-01

Ruan Jian Xue Bao/Journal of Software

Abstract:The traditional data distribution strategy was the dimension-based distribution, which can not effectively treat data distribution under irregular data reference mode. The hyperplane-based block data distribution strategy was proposed, which is characterized by the unidirected data distribution. The determination of hyperplane and method to perform partition and allocation for data according to the hyperplane were given. The data distribution problems under uniform computational environments and the irregular data reference mode were discussed. The results of preliminary experiment have shown that the proposed data distribution strategy is simple and effective, and may be considered as a supplement for the traditional scheme.
Optimized Multi-Dimensional Optical Storage Reading Strategy

Hequn Wang,Jing Pei,Longfa Pan

DOI: https://doi.org/10.1007/s12200-014-0477-4

2014-01-01

Frontiers of Optoelectronics

Abstract:A novel multi-dimensional (MD) optical storage was presented, which was realized by utilizing the space between tracks. Based on scalar diffraction theory, the channel bits parameters of the multi-dimensional optical storage were optimized, and the linear and nonlinear signals were analyzed accurately. Therefore, the format of the multi-dimensional optical disc was obtained, which makes the detection of readout signal easier. With respect to servo, coding and readout physics parameter of channel, the multi-dimensional optical disc is compatible with traditional disc such as Blu-ray disc (BD). Also, the novel multi-dimensional optical storage is able to achieve a doubled density and a ten-fold readout data rate compared with traditional optical discs.
Data Sharing Through Optical Array in Video-On-Demand (vod) System

HC Zhang,LF Pan,DY Xu

DOI: https://doi.org/10.1117/12.248709

1996-01-01

Abstract:Optical array and special data arrangement affect data sharing feature of Video-on-Demand system significantly. Some aspects related to maximum user number have been analyzed in theory and verified in experiment.
Consistent Hash Based Flexible Redundant Data Partition Strategy

Zhiyun Zheng,Ning Wang,Dun Li,Zhimeng Du,Zhenfei Wang

DOI: https://doi.org/10.1109/cbd.2017.32

2017-01-01

Abstract:Because of the development of computer networks and the scattered locations of system, distributed data-storage system is widely deployed. Data partition and redundant strategy determine the space utilization, access performance and data reliability of the storage system. Based on the analysis of current data partition and redundant mechanisms, we propose a novel Consistent Hash Based Flexible Redundant Data Partition Strategy (CHAFRAS) for key-value data models. It partitions the hash ring dynamically according to the number of virtual nodes and manages data copies on the level of partitioned unit. Furthermore, CHAFRAS shortens the path of data access and simplifies the management of data copies on the basis of keeping the features of scalability and load balancing. Extensive experimental results offer an insight into the proposed techniques and suggest that they are efficient in terms of convergence time and access delay by comparing with the state-of-the-art storage system Dynamo.
ISRA-Based Grouping: A Disk Reorganization Approach for Disk Energy Conservation and Disk Performance Enhancement

Xue-Liang Liao,Shi Bai,Yu-Ping Wang,Shi-Min Hu

DOI: https://doi.org/10.1109/tc.2010.153

IF: 3.183

2010-01-01

IEEE Transactions on Computers

Abstract:Reducing disk energy consumption and improving disk performance in high-performance computer systems are increasingly pressing issues for reasons of disk economy and efficiency. To achieve these goals, we define the concept of Immediate Successor Relationship Amount (ISRA) to represent the successor relationship of data blocks, and propose an ISRA-based grouping algorithm for disk reorganization, based on an undirected graph. We group data blocks that experience frequent successive accesses, then sort them using a merge-sort-like algorithm to determine the position of every group as well as the new position of every block within those groups. We evaluate our approach in terms of disk seek time and disk energy consumption, using Disksim and the log energy model. The results show clearly that both disk seek time and the energy needs can be reduced by about 50 percent.
DA Placement: A Dual-Aware Data Placement in a Deduplicated and Erasure-Coded Storage System

Mingzhu Deng,Ming Zhao,Fang Liu,Zhiguang Chen,Nong Xiao

DOI: https://doi.org/10.1007/978-3-030-05051-1_25

2018-01-01

Abstract:Simultaneously incorporating deduplication as well as erasure coding is preferred for modern storage systems for the enhanced storage efficiency and economical data reliability. However, simple incorporation suffers from the "read imbalance problem", in which parallel data accesses are curbed by throttled storage nodes. This problem is due to the uneven data placement in the system, which is unaware of the employment of both deduplication and erasure coding, each of whom alters the order of data if unattended. This paper proposes a systematic design and implementation of a Dual-Aware(DA) placement in a combined storage system to achieve both deduplication-awareness and erasure-coding-awareness at the same time. DA not only records the node number of each unique data to allow for quick references with ease, but also dynamically tracks used nodes for each writes request. In this way, deduplication awareness is formed to skip inconvenient placement locations. Besides, DA serializes the placement of parity blocks with a stripe and across stripes. Such realization of erasure coding awareness ensures the separation of data and parity, as well as maintains data sequentiality at bordering stripes. Additionally, DA manages to extend with further load-balancing through an innovative use of the deduplication level, which intuitively predicts future accesses of a piece of data. In short, DA manages to boost system performance with little memory or computation cost. Extensive experiments using both real-world traces and synthesized workloads, prove DA achieves a better read performance. For example, DA respectively leads an average latency margin of 30.86% and 29.63%, over the baseline rolling placement(BA) and random placement(RA) under CAFTL traces over a default cluster of 12 nodes with RS(8,4).
A Novel Optimization Method to Improve De-duplication Storage System Performance

Chuanyi Liu,Yibo Xue,Dapeng Ju,Dongsheng Wang

DOI: https://doi.org/10.1109/ICPADS.2009.103

2009-01-01

Abstract:Data De-duplication has become a commodity component in data-intensive storage systems. But compared with other traditional storage paradigms, de-duplication system achieves elimination of data duplications or redundancies at the cost of bringing several additional layers or function components into the I/O path, and these additional components are either CPU-intensive or I/O intensive, largely hindering the overall system performance. Direct against the above potential system bottlenecks, this paper quantitatively analyzes the overhead of each main component introduced by de-duplication, and then proposes two performance optimization methods. The one is parallel calculation of content aware chunk identifiers, which fully utilizes the parallelism both inter and intra chunks by using a certain task partition and chunk content distribution algorithm. Experiments demonstrate that it can improve up to 150% of the system throughput, and at the same time much better utilize the multiprocessor resources. The other one is storage pipelining, which overlaps the CPU-bound, I/O-bound and network communication tasks. Through a dedicated five-stage storage pipeline design for file archival operations, experimental results show that the system throughput can increase up to 25% according to our workloads.
An Efficient Data Reuse Strategy for Multi-Pattern Data Access.

Wensong Li,Fan Yang,Hengliang Zhu,Xuan Zeng,Dian Zhou

DOI: https://doi.org/10.1145/3240765.3240778

2018-01-01

Abstract:Memory partitioning has been widely adopted to increase the memory bandwidth. Data reuse is a hardware-efficient way to improve data access throughput by exploiting locality in memory access patterns. We found that for many applications in image and video processing, a global data reuse scheme can be shared by multiple patterns. In this paper, we propose an efficient data reuse strategy for multi-pattern data access. Firstly, a heuristic algorithm is proposed to extract the reuse information as well as find the non-reusable data elements of each pattern. Then the non-reusable elements are partitioned into several memory banks by an efficient memory partitioning algorithm. Moreover, the reuse information is utilized to generate the global data reuse logic shared by the multi-pattern. We design a novel algorithm to minimize the number of registers required by the data reuse logic. Experimental results show that compared with the state-of-the-art approach, our proposed method can reduce the number of required BRAMs by 62.2% on average, with the average reduction of 82.1% in SLICE, 87.1% in LUTs, 71.6% in Flip-Flops, 73.1% in DSP48Es, 83.8% in SRLs, 46.7% in storage overhead, 79.1% in dynamic power consumption, and 82.6% in execution time of memory partitioning. Besides, the performance is improved by 14.4%.
Leveraging Data Deduplication to Improve the Performance of Primary Storage Systems in the Cloud

Bo Mao,Hong Jiang,Shaoen Wu,Tian Lan

DOI: https://doi.org/10.1145/2523616.2525939

2013-01-01

Abstract:Recent studies have shown that moderate to high data redundancy exists in primary storage systems, such as VM-based, enterprise and HPC storage systems, which indicates that the data deduplication technology can be used to effectively reduce the write traffic and storage space in such environments. However, our experimental studies reveal that applying data deduplication to primary storage systems will cause space contention in main memory and data fragmentation on disks. This is in part because applying data deduplication introduces significant index memory overhead to the existing system and in part because a file or block is split into multiple small data chunks that are often located in non-sequential locations on disks after deduplication. This fragmentation of data can cause a subsequent read operation to invoke many disk I/O requests, thus leading to performance degradation.
Analysis of Approaches to High Density Optical Data Storage

菅冀祁,王玉英,熊剑平,贾惠波,佘鹏

DOI: https://doi.org/10.3969/j.issn.1001-5868.2004.04.017

2004-01-01

Abstract:With the development of optical data storage technology, how to improve storage density has been paid much attention to. Using the method of channel analysis, the factors and parameters influencing optical storage density are discussed, and it is concluded that the resolution of the optical system and the storage mechanism are two most important factors. In order to realize high density storage, either the resolution is improved through shorter wavelength, larger numerical aperture, super resolution and adding one more recording dimensions, or storage mechanism by decreasing recording spots and multiplexing recording spots.
Energy-Efficient Fast Data Retrieval Strategy Based on RS Coded Placement in LEO Constellation

Zhineng Wu,Shushi Gu,Zhikai Zhang,Qinyu Zhang,Yifeng Jin,Lei Zhang,Wei Xiang

DOI: https://doi.org/10.1109/vtc2024-spring62846.2024.10683149

2024-01-01

Abstract:Low earth orbit (LEO) constellation network is a crucial component and paradigm for big data applications in the future satellite Internet. However, the characteristic of multi-hop transmission significantly increases the data retrieval delay and energy consumption depending on the inter-satellite communications. To mitigate this issue, this paper explores encoding redundancy by reed-solomon (RS) codes to generate the data and parity blocks, and store them in the satellite nodes of LEO constellation. We propose a fast data retrieval strategy, involving three stages: data collection, parity block placement, and data retrieval. Through the derivations of delay and energy consumption, we find that both are intensively related to the placement of parity blocks. To reduce energy consumption during the data retrieval process, we formulate a minimum problem under the delay constraint, which is an integer nonlinear programming problem. Then, we design an energy-efficient parity block placement based on genetic algorithm (PBP-GA), which is heuristic with fast convergence property. Simulation results show that, PBP-GA achieves a comprehensive performance improvement in average data retrieval delay and total energy consumption, compared to other placement schemes, i.e., random placement, retrieval cluster priority placement, nearby placement and non-coding. Specifically, PBP-GA can find the optimal number of parity blocks in a practical scenario of data retrieval in LEO constellation.
OPS: an optimized partial stripe write scheme to improve performance of XOR-based disk arrays tolerating triple disk failures

Xunsong Huang,Chentao Wu,Jie Li

DOI: https://doi.org/10.1145/3318265.3318274

2019-01-01

Abstract:In cloud storage and big data processing systems, RAID especially disk arrays tolerating triple disk failures (3DFTs) is a popular choice to provide high reliability with low monetary cost. For 3DFTs, a key obstacle is the low partial stripe write performance, which is caused by large amount of parity modifications based on complex erasure coding layouts. In order to solve this problem, in this paper, we propose an optimized partial stripe write (OPS) method, which reorganizes the distribution of write data blocks to share partial parities among data blocks, thereby improving overall I/O performance. The OPS method can effectively reduce the number of modified parities. To illustrate the effectiveness of our OPS method, we used Disksim to evaluate several different partial stripe write methods through simulation. The results show that OPS can reduce the average response time by up to 37.21% and decreases the number of write operations by up to 26.22% compared to the traditional partial strip writing method..
Data De-duplication on Similar File Detection

Yueguang Zhu,Xingjun Zhang,Runting Zhao,Xiaoshe Dong

DOI: https://doi.org/10.1109/IMIS.2014.9

2014-01-01

Abstract:At present, there exist many bottlenecks in block level data de-duplication on the metadata management and read/write rate. In order to achieve higher de-duplication elimination ratio, the traditional way is to expand the range of data for data de-duplication, but that would make metadata fields longer and increase the number of metadata entries. When detecting the redundant data, metadata needs to be constantly imported and exported into the memory and access bottleneck will be produced. So it is necessary to detect similar documents to classify valuable data for de-duplication. In this paper, we propose a new method of block-level data de-duplication combined with similar file detection. At the time of guaranteeing the de-duplication elimination ratio, we narrow the range of data to reduce the metadata and eliminate performance bottlenecks. We present a detailed evaluation of our method and other existing data deduplication methods, and we show that our method meets its design goals as it improves the de-duplication ratio while reducing overhead costs.

Redundant Data Strategy Improve Seeking Efficiency Of Optical Array

Character Analysis for Optical Disk Array

Optical array for Video-on-Demand (VOD) system

A Novel Data Redundancy Scheme for De-Duplication Storage System

Shifted Element Arrangement in Mirror Disk Arrays for High Data Availability During Reconstruction

Optimization for Data De-Duplication Algorithm Based on File Content

New Way to Improve the Reliability of High-Speed and Mass Data Recording Optical Disk Array

Optimization for Data De-duplication Algorithm Based on Storage Environment Aware

Hyperplane-based Block Data Distribution Strategy

Optimized Multi-Dimensional Optical Storage Reading Strategy

Data Sharing Through Optical Array in Video-On-Demand (vod) System

Consistent Hash Based Flexible Redundant Data Partition Strategy

ISRA-Based Grouping: A Disk Reorganization Approach for Disk Energy Conservation and Disk Performance Enhancement

DA Placement: A Dual-Aware Data Placement in a Deduplicated and Erasure-Coded Storage System

A Novel Optimization Method to Improve De-duplication Storage System Performance

An Efficient Data Reuse Strategy for Multi-Pattern Data Access.

Leveraging Data Deduplication to Improve the Performance of Primary Storage Systems in the Cloud

Analysis of Approaches to High Density Optical Data Storage

Energy-Efficient Fast Data Retrieval Strategy Based on RS Coded Placement in LEO Constellation

OPS: an optimized partial stripe write scheme to improve performance of XOR-based disk arrays tolerating triple disk failures

Data De-duplication on Similar File Detection