Abstract:Software systems usually record important runtime information in their logs. Logs help practitioners understand system runtime behaviors and diagnose field failures. As logs are usually very large in size, automated log analysis is needed to assist practitioners in their software operation and maintenance efforts. Typically, the first step of automated log analysis is log parsing, i.e., converting unstructured raw logs into structured data. However, log parsing is challenging, because logs are produced by static templates in the source code (i.e., logging statements) yet the templates are usually inaccessible when parsing logs. Prior work proposed automated log parsing approaches that have achieved high accuracy. However, as the volume of logs grows rapidly in the era of cloud computing, efficiency becomes a major concern in log parsing. In this work, we propose an automated log parsing approach, Logram, which leverages n-gram dictionaries to achieve efficient log parsing. We evaluated Logram on 16 public log datasets and compared Logram with five state-of-the-art log parsing approaches. We found that Logram achieves a similar parsing accuracy to the best existing approaches while outperforms these approaches in efficiency (i.e., 1.8 to 5.1 times faster than the second fastest approaches). Furthermore, we deployed Logram on Spark and we found that Logram scales out efficiently with the number of Spark nodes (e.g., with near-linear scalability) without sacrificing parsing accuracy. In addition, we demonstrated that Logram can support effective online parsing of logs, achieving similar parsing results and efficiency with the offline mode.

An N-Gram Prediction Model Based on Web-Log Mining

Research on personalized Web service

Web-logMiningfor QuantitativeTemp Oral-Event Prediction

A Web Predictive Model Based on Dependency Graph

Dynamic mining for web navigation patterns based on markov model

A Vague Improved Markov Model Approach for Web Page Prediction

Microblog Users' Life Time Activity Prediction

Assessing the Feasibility of Web-Request Prediction Models on Mobile Platforms

A novel prediction model based on hierarchical characteristic of web site

Web Usage Mining of Science Platform by Frequent Generalized Subsequences Method

Mining User Interests in Web Logs of an Online News Service Based on Memory Model

Use Web Usage Mining to Assist Online E-Learning Assessment.

A Dynamic Recommendation System Based on Log Mining.

Intention Modeling for Web Navigation

Web log mining techniques to optimize Apriori association rule algorithm in sports data information management

Web Pre-fetching Using Adaptive Weight Hybrid-Order Markov Model.

Log mining to improve the performance of site search

Web User Profiling Using Data Redundancy

MAPLE: Mobile App Prediction Leveraging Large Language Model Embeddings

Mining on Web logs for recommendation

Logram: Efficient Log Parsing Using n-Gram Dictionaries