Abstract:With the rapid development of computer and network technology, to deal with explosively growing mass data correctly, distributed file systems have come into being, and they have been applied in many enterprises. However, in the packet storage process, the traditional TCP/IP transmission protocol has to pass through the operating system and application software layer. It takes up many server resources and memory bus bandwidth, which places a heavy burden on the server’s CPU and memory. In this paper, we analyze the research status of the distributed file system based on analyzing and comparing the advantages of RDMA over traditional TCP Ethernet communication architecture. We propose an optimization scheme for the network transmission performance of the original FastDFS to replace the original TCP module with an RDMA network communication module, and we design a lightweight distributed file system with higher performance than the original FastDFS, which is suitable for small file storage. Using the zero-copy technology and kernel bypass technology of soft RDMA, the network communication mechanism in FastDFS is optimized to free the server’s CPU from heavy data processing work. In this paper, the RDMA-based FastDFS is compared with the original FastDFS. The test results show that the network transmission performance of the optimized FastDFS is improved compared with the original FastDFS, which achieves the expected effect and can better meet the requirements of mass small file storage for today’s Internet applications.

The design of file transfer system with high speed and its implementation in GDSS

An Implementation of High Performance Data Transfer Client for Grid Computing

A research on the improvement of file transfer efficiency in grid community

A Novel Scalable Architecture of Cloud Storage System for Small Files Based on P2P

Design and Optimization of a Distributed File System Based on RDMA

Intelligent File Transfer Protocol for Grid Environment

Global File Store: a Massive Storage System on the Internet - Concept and Design

A web-based system for adaptive data transfer in grid

A File Distribution Mechanism based on Netty Communication Framework

Design of Secure File Transfer System Based on Embedded Linux

A Novel Parallel Transmission Strategy for Data Grid.

Design and Implementation of Scalable Grid File Access Interface

The xDotGrid Native, Cross-Platform, High-Performance xDFS File Transfer Framework

File transfer strategy of XML data warehousing in grid

A storage model and its corresponding parallel download scheduling algorithm for data grid systems

Transferring of FTP files based upon Winsock

Design and Implementation of An Across Netgap Data Transmission System

Research Of Efficient C/S Model Based File Transfer Method

Data Transfer Based on Multiple Replicas in the Grid Environment

Distributed File Streamer: A Framework for Distributed Application Data Coupling

The Design and Implementation of a Slice-based Loose-coupled Distributed File System