Client Error Clustering Approaches in Content Delivery Networks (CDN)

Ermiyas Birihanu,Jiyan Mahmud,Péter Kiss,Adolf Kamuzora,Wadie Skaf,Tomáš Horváth,Tamás Jursonovics,Peter Pogrzeba,Imre Lendák
DOI: https://doi.org/10.48550/arXiv.2210.05314
2022-10-11
Abstract:Content delivery networks (CDNs) are the backbone of the Internet and are key in delivering high quality video on demand (VoD), web content and file services to billions of users. CDNs usually consist of hierarchically organized content servers positioned as close to the customers as possible. CDN operators face a significant challenge when analyzing billions of web server and proxy logs generated by their systems. The main objective of this study was to analyze the applicability of various clustering methods in CDN error log analysis. We worked with real-life CDN proxy logs, identified key features included in the logs (e.g., content type, HTTP status code, time-of-day, host) and clustered the log lines corresponding to different host types offering live TV, video on demand, file caching and web content. Our experiments were run on a dataset consisting of proxy logs collected over a 7-day period from a single, physical CDN server running multiple types of services (VoD, live TV, file). The dataset consisted of 2.2 billion log lines. Our analysis showed that CDN error clustering is a viable approach towards identifying recurring errors and improving overall quality of service.
Networking and Internet Architecture,Artificial Intelligence
What problem does this paper attempt to address?
The problem that this paper attempts to solve is to analyze error logs in the content distribution network (CDN) through clustering methods, in order to identify recurring errors and improve service quality. Specifically, researchers are concerned with how to use different clustering methods to process and analyze the massive Web server and proxy logs generated by the CDN system, especially those logs containing error information (HTTP status codes greater than or equal to 400). These error logs contain important information about the system's operating conditions and can help CDN operators discover problems in the system, thereby taking measures to improve service quality and system stability. The main objectives of the research include: 1. **Analysis of Applicability**: Evaluate the applicability and effectiveness of different clustering methods in CDN error log analysis. 2. **Feature Selection**: Select key features from a large amount of log data, such as content type, HTTP status code, time, host, etc., for more effective clustering analysis. 3. **Error Identification**: Through clustering analysis, identify different types of error patterns, especially those that occur frequently. 4. **Optimization Suggestions**: Based on the clustering results, provide CDN operators with specific suggestions for improving service quality and system performance. Through these steps, researchers hope to achieve more effective error management and system optimization in large - scale CDN systems.