A feature extraction and time warping based neural expansion architecture for cloud resource usage forecasting

Gurjot Singh,Prajit Sengupta,Anant Mehta,Jatin Bedi
DOI: https://doi.org/10.1007/s10586-023-04224-2
2024-01-12
Cluster Computing
Abstract:Accurate resource utilization estimation is crucial for efficient resource allocation, capacity planning, and cost optimization in cloud systems. In the past, several artificial intelligence, machine learning, and deep learning-based techniques have been evolved to forecast cloud cluster workload. Despite the abundance of available techniques, most existing techniques fail to achieve the desired prediction efficiency and generalization capability. They are computationally inefficient in accurately determining resource utilization at a machine-level granularity. The current research proposes a computationally less-expensive hybrid approach combining cluster analysis and deep neural learning with transfer learning to estimate the machine-level workload. The method implements clustering to identify the similarity patterns among the non-linear usage profiles of machines present in the input dataset. Subsequently, the generalized deep neural learning models are developed considering only a sample dataset belonging to each identified cluster. Lastly, the concept of transfer learning is deployed using pre-trained generalized models to estimate the workload for all remaining machines relating to the clusters. The performance validation of the proposed approach is carried out on the real-world traces dataset of the google cluster. The comparative evaluation of the proposed approach with benchmark approaches verifies the achieved performance benefits and accuracy.
computer science, information systems, theory & methods
What problem does this paper attempt to address?