An Implementation of High Performance Data Transfer Client for Grid Computing

Zhu Chao-yan,ZHENG Yao,WEI Gui-yi

DOI: https://doi.org/10.3969/j.issn.1671-7147.2005.02.010

2005-01-01

Abstract:This paper discusses an implementation of such an FTP client as MFTP, which is also an important component of the MASSIVE Environment. MFTP is based on the GridFTP protocol and is built on the Globus Toolkit. It has a user-friendly and interactive GUI.

What problem does this paper attempt to address?

Data Transfer Based on Multiple Replicas in the Grid Environment

YING Yifeng,SONG Guanghua

DOI: https://doi.org/10.3969/j.issn.1000-3428.2007.01.015

2007-01-01

Abstract:Data-intensive computing applications often require the efficient transfer of terabytes or even petabytes of data in wide-area,distributed computing environments.This paper adopts a new approach to implement the striped transfer of data on the basis of GridFTP by utilizing multiple replica sources concurrently.MFTP,a FTP client implementation of the presented approach is addressed.Experiments show that MFTP improves the data transfer rate and reliability on grids.
A Public Grid Computing Framework Based on a Hierarchical Combination of Middleware

Yingjie Xia,Yao Zheng,Yudang Li

DOI: https://doi.org/10.1007/11610496_92

2006-01-01

Abstract:Grids, as one implementation of public computing, can be deployed by some middleware to integrate all kinds of resources across institutional boundaries, to tackle complex problems in scientific and engineering computation, data storage, scientific visualization, etc. In this paper, we propose a public computing grid framework built upon the hierarchical combination of two popular middleware products, Globus Toolkit (GT) and Sun Grid Engine (SGE). The communication between them can be conducted by Transfer-queue Over Globus (TOG). We discuss issues related to the deployment and the usage of this kind of hybrid grids. We examine it using serial programs and parallel program codes, and compare it with the grids deployed by only GT or SGE, respectively. The case studies imply that this hybrid grid is suitable for loosely coupled applications with high computational complexity.
The Design and Implementation of Scalable Information Services in a Grid Environment

W Wang,Y Zheng,Gh Song

DOI: https://doi.org/10.1109/scc.2005.97

2005-01-01

Abstract:This paper proposes an Extensible Monitoring and Discovery Service for grid environment MASSIVE (MEMDS). The MASSIVE project focuses on developing a multidisciplinary applications-oriented simulation and visualization environment. In the MEMDS, appropriate mechanisms make custom information services established and deployed easily, and the developed APIs simplify the access to the MDS and provide a uniform interface for programmers. Corresponding practices show that the MEMDS improves the extensibility and flexibility of the grid information service.
A fair bulk data transmission protocol in grid environments

Fanjun Su,Xuezeng Pan,Yong lv,Lingdi Ping

DOI: https://doi.org/10.1007/11428862_142

2005-01-01

Abstract:In this paper, we propose FHSTCP (Fair High-Speed TCP) as an improvement of HSTCP, which adds a fair factor to eliminate the difference of congestion window caused by different RTT and adopts block-pacing to reduce the burstiness. Simulation results show that FHSTCP can alleviate the RTT unfairness meanwhile keeping advantages of HSTCP.
An Enabling Environment for Distributed Simulation and Visualization

Y Zheng,GH Song,JF Zhang,JJ Chen

DOI: https://doi.org/10.1109/grid.2004.10

2004-01-01

Abstract:In this paper we present the MASSIVE project that aims to use Grid technology to establish an enabling environment for distributed simulation and visualization of large-scale scientific and engineering research. The project focuses on collaborative numerical simulation and visualization in grid environments. Furthermore, the project is also to develop Grid-enabled capability and products for use by the wider community. The current status of the MASSIVE project is reported and a prototype Grid utilizing resources at the Center for Engineering and Scientific Computation (CESC) in ZheJiang University (ZJU) is discussed. This Grid gets a model geometry generated by CAD systems and transfers it to an SGI Onyx 3900 supercomputer where it is meshed. The meshed geometry is then transferred to a DAWNING PC Cluster where it is used to solve computational fluid dynamics and computational solid mechanics problems and it is finally visualized showing numerical results.
MASSIVE: A Multidisciplinary Applications-Oriented Simulation and Visualization Environment

Guiyi Wei,Guanghua Song,Yao Zheng,Cuiju Luan,Chaoyan Zhu,Wei Wang

DOI: https://doi.org/10.1109/SCC.2004.1358069

2004-01-01

Abstract:The objective of the MASSIVE project is to build a Grid platform to enable large-scale and distributed scientific and engineering computations. Its focus is numerical simulation and visualization. This paper addresses a prototype Grid utilizing resources at the Center for Engineering and Scientific Computation (CESC) in Zhejiang University (ZJU). This Grid is capable of taking a model geometry generated by CAD systems, transferring it to an SGI Onyx 3900 supercomputer where it is meshed. The meshed geometry is then transferred to a Dawning PC cluster where it is used to solve computational fluid dynamics and computational solid mechanics problems and it is finally visualized showing numerical results.
An Engineering Computation Oriented Visual Grid Framework

Guiyi Wei,Yao Zheng,Jifa Zhang,Guanghua Song

DOI: https://doi.org/10.1007/978-3-540-24679-4_16

2004-01-01

Abstract:Grid computing technology is a focused field in high performance computing. This paper describes an engineering computation oriented visual grid framework VGrid, which is capable to bridge the gap between currently deployed grid services and the computational applications. Based on the Globus toolkit, and coupled with a client component, a services pool and a server component, VGrid visually performs resource discovery, task schedule, and result processing. VGrid improves the efficiency of utilization of resources by introducing a logical resource concept. VGrid applications of numerical simulations in engineering sciences are demonstrated.
Grid Service-Based Parallel Finite Element Analysis

Guiyi Wei,Yao Zheng,Jifa Zhang

DOI: https://doi.org/10.1007/978-3-540-24679-4_25

2004-01-01

Abstract:In this paper, we present our work on the integration of existing engineering applications using Grid Services. We address a visual grid framework that provides an integrated development environment for engineering computation applications. Furthermore, we demonstrate bow such services can interact with each other. These interactions enable a level of integration that assists the scientific application architect in leveraging applications running in heterogeneous runtime environments. Our framework is implemented by using the existing infrastructures and middleware, the Globus Toolkit. We test our framework for computation solid mechanics applications that require large data transferring, interactive steering, using of multiple platforms, visualization, and access via a portal with graphic interface.
A Flexible Architecture for Job Management in A Grid Environment

Luan Cui-ju,Song Guang-hua,Zheng Yao,Zhang Ji-fa

DOI: https://doi.org/10.1631/jzus.2007.a0095

2006-01-01

Journal of Zhejiang University SCIENCE A

Abstract:Job management is a key issue in computational grids, and normally involves job definition, scheduling, executing and monitoring. However, job management in the existing grid middleware needs to be improved in terms of efficiency and flexibility. This paper addresses a flexible architecture for job management with detailed design and implementation. Frameworks for job scheduling and monitoring, as two important aspects, are also presented. The proposed job management has the advantages of reusability of job definition, flexible and automatic file operation, visual steering of file transfer and job execution, and adaptive application job scheduler. A job management wizard is designed to implement each step. Therefore, what the grid user needs to do is only to define the job by constructing necessary information at runtime. In addition, the job space is adopted to ensure the security of the job management. Experimental results showed that this approach is user-friendly and system efficient.
Multi-discipline computations and collaborative visualization on the campus grid

Yingjie Xia,Guanghua Song,Yao Zheng

DOI: https://doi.org/10.3321/j.issn:1671-4512.2007.z2.029

2007-01-01

Abstract:A campus grid is developed and deployed to integrate high performance computing resources in Zhejiang University.The architecture is based on a hierarchical combination of two grid middleware products,Globus Toolkit and Sun Grid Engine,in order to make use of their advantages.The campus grid is applied for scientific computation,bioinformatics,computational fluid dynamics,etc.Some test cases using Message Passing Interface(MPI) show that the campus grid can provide powerful and stable computing cycles.The campus grid also provides two collaborative visualization services,the local rendering and the server rendering,which can be customized by the user.Finally,the collaborative visualization service facilitates users to research and analyze cooperatively.
An infrastructure for grid job monitoring

Cuiju Luan,Guanghua Song,Yao Zheng

DOI: https://doi.org/10.1007/11590354_55

2005-01-01

Abstract:As a part of an ongoing project titled MASSIVE, we have designed and implemented the MASSIVE Monitoring System (MMS) to monitor grid jobs. MMS is able to work on multiple sites and support multiple users. In addition to grid jobs, MMS is capable of monitoring and steering file operations. The Job Register Code (JRC) is adopted to map a job to its processes, and used to decide whether a user can access the monitoring information or steer the jobs. MMS supports top-down start-up of the monitoring components, which can ensure the components at their places.
Design and Implementation of Scalable Grid File Access Interface

LIU Ai-gui,CHENG Yao-dong,XU Dong,CHEN Gang

DOI: https://doi.org/10.3969/j.issn.1000-3428.2007.20.090

2008-01-01

Abstract:This paper presents the design and implementation of a grid file access interface(GFAI).GFAI implements a POSIX and scalable grid file access interface,which can screen the details of access protocols for different storage systems and solve heterogeneity problem of different file access or file transfer protocols.It provides a uniform and easy use grid I/O interface for users.
FTP-M: An FTP-like Multicast File Transfer Application

G. Varghese,M. Mysore

2001-09-11

Abstract:Research in Reliable Multicast has focused on providing scalable, semi-reliable delivery of data. Such solutions based on schemes like SRM have proved adequate for multicast delivery of multimedia data or whiteboard applications, but are inadequate for the strict delivery semantics of file transfer. This paper describes a simple application called FTP-M that provides an efficient one-to-many multicast data transfer by simply extending the FTP protocol and user interface. FTP-M relies on a {\em strictly-reliable} Reliable Multicast protocol layer beneath to send data reliably to multiple receivers. In this paper, we describe the design and implementation of FTP-M. We suggest that FTP-M has the potential to be adopted as a standard way to achieve multicast file transfer.

Computer Science
MPICH-G2: A Grid-Enabled Implementation of the Message Passing Interface

N. T. Karonis,B. Toonen,I. Foster

DOI: https://doi.org/10.48550/arXiv.cs/0206040

2002-06-26

Abstract:Application development for distributed computing "Grids" can benefit from tools that variously hide or enable application-level management of critical aspects of the heterogeneous environment. As part of an investigation of these issues, we have developed MPICH-G2, a Grid-enabled implementation of the Message Passing Interface (MPI) that allows a user to run MPI programs across multiple computers, at the same or different sites, using the same commands that would be used on a parallel computer. This library extends the Argonne MPICH implementation of MPI to use services provided by the Globus Toolkit for authentication, authorization, resource allocation, executable staging, and I/O, as well as for process creation, monitoring, and control. Various performance-critical operations, including startup and collective operations, are configured to exploit network topology information. The library also exploits MPI constructs for performance management; for example, the MPI communicator construct is used for application-level discovery of, and adaptation to, both network topology and network quality-of-service mechanisms. We describe the MPICH-G2 design and implementation, present performance results, and review application experiences, including record-setting distributed simulations.

Distributed, Parallel, and Cluster Computing
Jparss: A Java Parallel Network Package For Grid Computing

Jie Chen,Walt Akers,Ying Chen,William A. Watson III

2002-01-01

Abstract:The emergence of high speed wide area networks makes grid computing a reality. However grid applications that need reliable data transfer still have difficulties to achieve optimal TCP performance due to network tuning of TCP window size to improve bandwidth and to reduce latency on a high speed wide area network. This paper presents a Java package called JPARSS (Java Parallel Secure Stream (Socket)) that divides data into partitions that are sent over several parallel Java streams simultaneously and allows Java or Web applications to achieve optimal TCP performance in a grid environment without the necessity of tuning TCP window size. This package enables single sign-on, certificate delegation and secure or plain-text data transfer using several security components based on X.509 certificate and SSL. Several experiments will be presented to show that using Java parallel streams is more effective than tuning TCP window size. In addition a simple architecture using Web services to facilitate peer to peer and third party file transfer will be presented.
Applying Computational Grid Technology to Power System

f j luo,zhao yang dong,can wan,ying ying chen,ke meng,k p wong

DOI: https://doi.org/10.4028/www.scientific.net/AMM.58-60.1442

2011-01-01

Applied Mechanics and Materials

Abstract:This paper proposes a computational grid platform for solving the large-scale power system applications. The platform is based on Globus Toolkit middleware and Grid Way meta-scheduler. It can enable large-scale sharing of computational resources across institutional boundary. This paper first discusses the architecture and each component of the platform, and then the test bed is described. Finally, the test results of probabilistic load flow (PLF) by Monte-Carlo simulation are presented. The test results show that the computational Grid system can provide comparable performance.
Implement of FTP Client Based on C

GONG Li,MAO Xiaojiao

DOI: https://doi.org/10.3969/j.issn.1006-4052.2010.24.040

2010-01-01

Abstract:This paper discusses the content and working principle of File Transfer Protocol and designs an implement of FTP Client in C/S structure,then uses C#language in.NET platform to form Windows intrface of FTP client.Finally it uses the class library of.NET to implement the features such as uploading,deleting and altering.
Agile Data Streaming for Grid Applications

Wen Zhang,Junwei Cao,Yishen Zhong,Lianchen Liu,Cheng Wu

DOI: https://doi.org/10.1109/gcc.2008.10

2008-01-01

Abstract:The grid is proposed to be the new computing infrastructure for cross-domain resource sharing and collaboration in virtual organizations. While most current work in grid computing community focus on adaptability of grid resource management, agility is proposed in this work as another essential requirement, aiming to provide mass customization and personalization on grid computing service provision. This work is focused on data streaming aspects of grid agility. Since volumes of data streams are usually extremely high but available bandwidth and storage are often very limit in a grid environment, an enabling environment for grid data streaming applications is implemented with supports of on-demand data transfers and just-in-time data cleanups in this work. The implementation includes performance sensors, predictors and a scheduler, with real-time measurements, prediction and scheduling capabilities. The GridFTP is utilized and data transfer schedules are performed using the on-the-fly adjustable GridFTP parallelism. Experimental results show that the data streaming environment is agile enough to meet dynamically changing application data processing requirements and scale well regarding to storage and bandwidth usage.
Improving the Performance of MPI Applications over Computational Grid.

Bo Song,Guangwen Yang,Qiming Fang

DOI: https://doi.org/10.1109/gcc.2007.80

2007-01-01

Abstract:The Message Passing Interface (MPI) is a standard in parallel computing, and can also be used as a highperformance programming model for Grid application development. How to execute MPI applications efficiently over a computational Grid has become a big challenge to developers, due to the distributed nature of Grid resources and complex hierarchies of Grid links. In this paper, we present three useful techniques for improving the performance of MPI applications over a computational Grid. We introduce the multithreaded model to the implementation of MPI point-topoint operations, to overlap communication with computation and speed up point-to-point operations. To enable the porting of MPI applications to a Grid composed of multiple private-IP clusters, a crosssubnet communication mechanism based on NAT has been designed. To improve the performance of MPI collective operations over a computational Grid, we implements a kind of topology-aware collective communication algorithms based on a local communicator creation mechanism. These three techniques are adopted in an ongoing Grid-enabled MPI implementation called FiTMPI.
Fmp: A High Performance Communication Protocol For Pc Clusters Based On Myrinet

Wm Zheng,Jc Zhang

2001-01-01

Abstract:PC clusters based on high-speed LAN are emerging as a competitive alternative for large-scale parallel computing. With the fast increase of the performance of network interconnections and PCs, software solutions have lagged behind. Traditional communication protocols fall to make full use of the high performance of networks for their low efficiency.FMP is a reduced user-level message passing system for PC clusters based on Myrinet. It provides high-speed, reliable and in-order message delivery, and supports multiplexing and protective accessing of the network interface. FMP achieves one-way latency of 9us and bandwidth of 83Mbytes/s which compares favorably with other communication protocols. Moreover, an optimized version of MPI and PVM on top of FMP are both implemented. This paper analyzes the crucial issues, a high efficient message passing system, in designing introduces the design and implementation of FMP and evaluates its performance results.

An Implementation of High Performance Data Transfer Client for Grid Computing

Data Transfer Based on Multiple Replicas in the Grid Environment

A Public Grid Computing Framework Based on a Hierarchical Combination of Middleware

The Design and Implementation of Scalable Information Services in a Grid Environment

A fair bulk data transmission protocol in grid environments

An Enabling Environment for Distributed Simulation and Visualization

MASSIVE: A Multidisciplinary Applications-Oriented Simulation and Visualization Environment

An Engineering Computation Oriented Visual Grid Framework

Grid Service-Based Parallel Finite Element Analysis

A Flexible Architecture for Job Management in A Grid Environment

Multi-discipline computations and collaborative visualization on the campus grid

An infrastructure for grid job monitoring

Design and Implementation of Scalable Grid File Access Interface

FTP-M: An FTP-like Multicast File Transfer Application

MPICH-G2: A Grid-Enabled Implementation of the Message Passing Interface

Jparss: A Java Parallel Network Package For Grid Computing

Applying Computational Grid Technology to Power System

Implement of FTP Client Based on C

Agile Data Streaming for Grid Applications

Improving the Performance of MPI Applications over Computational Grid.

Fmp: A High Performance Communication Protocol For Pc Clusters Based On Myrinet