DC^2 : unified data collection for data centers
Wenfeng Xia,Yonggang Wen,Haiyong Xie,Bin Liu
DOI: https://doi.org/10.1007/s11227-014-1233-7
IF: 3.3
2014-01-01
The Journal of Supercomputing
Abstract:Modern data centers are playing an important role in a world full of information and communication technologies (ICTs). Many efforts have been paid to build a more efficient, cleaner data center for economic, social, and environmental benefits. This objective is being enabled by emerging technologies such as cloud computing and software-defined networking (SDN). However, a data center is inherently heterogeneous, consisting of servers, networking devices, cooling devices, power supply devices, etc., resulting in daunting challenges in its management and control. Previous approaches typically focus on only a single domain, for example, traditional cloud computing for server resource (e.g., computing resource and storage resource) management and SDN for network management. In a similar context of networking device heterogeneity, network function virtualization has been proposed to offer a standard abstract interface to manage all networking devices. In this research, we take the challenge of building a suit of unified middleware to monitor and control the three intrinsic subsystems in a data centre, including ICT, power, and cooling. Specifically, we present DC^2 , a unified scalable IP-based data collection system for data center management with elevated extensibility , as an initial step to offer a unified platform for data center operations. Our system consists of three main parts, i.e., data-source adapters for information collection over various subsystems in a data center, a unified message bus for data transferring, and a high-performance database for persistent data storage. We have conducted performance benchmark for the key building components, namely messaging server and database, confirming that our system is scalable for a data center with high device density and real-time management requirements. Key features, such as configuration files, dynamical module loading, and data compression, enhance our implementation with high extensibility and performance. The effectiveness of our proposed data collection system is verified by sample applications, such as, traffic flow migration for load balancing, VM migration for resource reservation, and server power management for hardware safety. This research lays out a foundation for a unified data centre management in future.