OSCAR-P and aMLLibrary: Profiling and Predicting the Performance of FaaS-based Applications in Computing Continua

Roberto Sala,Bruno Guindani,Enrico Galimberti,Federica Filippini,Hamta Sedghani,Danilo Ardagna,Sebastián Risco,Germán Moltó,Miguel Caballer

2024-11-12

Abstract:This paper proposes an automated framework for efficient application profiling and training of Machine Learning (ML) performance models, composed of two parts: OSCAR-P and aMLLibrary. OSCAR-P is an auto-profiling tool designed to automatically test serverless application workflows running on multiple hardware and node combinations in cloud and edge environments. OSCAR-P obtains relevant profiling information on the execution time of the individual application components. These data are later used by aMLLibrary to train ML-based performance models. This makes it possible to predict the performance of applications on unseen configurations. We test our framework on clusters with different architectures (x86 and arm64) and workloads, considering multi-component use-case applications. This extensive experimental campaign proves the efficiency of OSCAR-P and aMLLibrary, significantly reducing the time needed for the application profiling, data collection, and data processing. The preliminary results obtained on the ML performance models accuracy show a Mean Absolute Percentage Error lower than 30% in all the considered scenarios.

Performance

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: how to efficiently perform automated profiling and prediction of the performance of function - as - a - service (FaaS) - based applications in the computing continua. Specifically, the paper proposes an integrated framework, consisting of two parts, OSCAR - P and aMLLibrary, to address the following challenges:

1. **Performance evaluation of complex applications**: With the development of cloud computing and edge computing, applications may be distributed across different computing layers (such as cloud, edge devices, etc.), making their performance evaluation complex. Traditional manual methods are time - consuming and difficult to scale.

2. **Resource allocation and management**: In order to meet real - time processing requirements, it is necessary to accurately predict the application performance under different configurations, so as to rationally plan and manage computing resources. This includes determining which components should run on which hardware and how to dynamically adjust resources according to the load.

3. **Automation and efficiency**: Reduce human intervention, improve the efficiency of performance profiling and data collection, and reduce the time cost of initial configuration testing.

4. **Cross - platform compatibility**: Ensure that the proposed framework can adapt to multiple hardware architectures (such as x86 and arm64) and different workload scenarios.

By combining the ability of OSCAR - P to automatically generate and test application workflows and the ability of aMLLibrary to predict performance using machine - learning models, this framework can accurately predict the performance of applications in unseen configurations, significantly reducing the time and resources required for performance profiling. Preliminary experimental results show that in all considered scenarios, the mean absolute percentage error (MAPE) of this framework is less than 30%.

In terms of formulas, the paper does not involve specific mathematical formulas, but when describing performance indicators, it mentions the mean absolute percentage error (MAPE), which is defined as:
\[ \text{MAPE} = \frac{1}{n} \sum_{t = 1}^{n} \left| \frac{A_t - F_t}{A_t} \right| \times 100\% \]
where \( A_t \) is the actual value, \( F_t \) is the predicted value, and \( n \) is the number of samples.