Dynamic Logging with Dylog in Networked Embedded Systems

DongWei,LuoLuyao,HuangChao

2015-01-01

Abstract:Event logging is an important technique for networked embedded systems like wireless sensor networks. It can greatly help developers to understand complex system behaviors and diagnose program bugs...

What problem does this paper attempt to address?

Dynamic Logging with Dylog for Networked Embedded Systems

Wei Dong,Chao Huang,Jiliang Wang,Chun Chen,Jiajun Bu

DOI: https://doi.org/10.1145/2807698

2015-01-01

ACM Transactions on Embedded Computing Systems

Abstract:We present Dylog, a dynamic logging facility for networked embedded systems. Dylog employs several techniques to enable lightweight and interactive logging. First, Dylog uses binary instrumentation for dynamically inserting or removing logging statements, enabling interactive debugging at the runtime. Second, Dylog incorporates an efficient storage system and log collection protocol for recording and transferring the logging messages. In particular, Dylog significantly reduces the communication cost by storing string identifiers and restoring them back to corresponding strings at the PC. Third, Dylog employs MAC layer timestamping and a linear clock model for reconstructing the synchronized time of the logging messages with a very high precision. We implement and evaluate Dylog on TinyOS 2.1.1/TelosB. Results show that Dylog incurs a reasonable overhead. Dylog can help gain great visibility into the system behaviors, and diagnose performance issues at the source code level.
Understanding Routing Dynamics in a Large-Scale Wireless Sensor Network

Tong Zhu,Wei Dong,Yuan He,Qiang Ma

DOI: https://doi.org/10.1109/MASS.2013.56

2013-01-01

Abstract:Routing dynamics are intrinsic characteristics of operational wireless sensor networks (WSNs). We present the measurement and analysis results for routing dynamics in a large-scale WSN. We seek to answer several fundamental questions: How dynamically are current routing protocols performing? What causes routing dynamics? What is the impact of routing dynamics? Answers to the above questions are critical to understanding the interactions among multiple network elements, evaluating protocol design strategies, and improving system performances. However, measurements in large-scale WSNs are challenging due to the lack of dedicated log information (be analogous to configuration files, syslog messages used in Internet). We propose an approach to identify the routing dynamics based on limited information and correlate them with system events to find out the root causes. The key findings of our study include: parent change events mainly affect local nodes, i.e. They do not cause routing instability on far-away nodes, environment dynamics and routing loops have large impact on routing, small portion of parent changes might not be necessary, while a large portion of parent changes are effective in improving network performance.
THE IMPLEMENTATION OF FAULTS DEBUGGING AND DETECTION BASED ON VXWORKS

Yun Xia,Zhu Miaoliang,Yuan Shuhong

DOI: https://doi.org/10.3969/j.issn.1000-386X.2007.01.017

2007-01-01

Abstract:In the development of embedded realtime application,the software will always collapse or reset for some unknown reasons due to the lack of experiences and skills of the programmer.There are many reasons of this such as stack overflow,memory leakage,dead loop task,invalid pointer etc.If it happens,there are many unknown errors which are always different and very difficult to debug for developer.This article will tell readers some ways to debug and detect faults for embedded realtime software development base on VxWorks.
Development of remote pseudo-dynamic test platform

FAN Yun-lei,GUO Yu-rong,XIAO Yan

DOI: https://doi.org/10.3321/j.issn:1673-2049.2005.04.005

2005-01-01

Abstract:A system for distance monitoring and a seismic experiment controlling was introduced.This system was used to develop the method of a remote pseudo-dynamic test.A network communication platform NetSLab was needed to realize the remote control of the pseudo-dynamic test.Authors used Visual Basic developing the standard open-end single layer structural remote pseudo-dynamic test platform.The pseudo-dynamic test was remotely executed using this program.The application of the program was verified well by test results.
Research on embedded single address space OS in dynamic loading

Li Peiliang,Li Zhenpeng

DOI: https://doi.org/10.3969/j.issn.1000-8519.2010.07.006

2010-01-01

Abstract:Presently some feature phone and embedded OS aren’t virtual address, they can’t update modules or load new application dynamically, it needs re-compile and link all system software, this will limit the flexibility and extension of the system. If we can implement the dynamic load of the module and application, we will improve the system flexibility and extension. This paper introduces the dynamic load solution, such as run the dynamic module, position independence code and the call of system interface, it gives the reference implantation also, and verified in one feature phone, it works as design. At last, the paper analyzes the system security and next step extension.
Observing a Moving Target -- Reliable Transmission of Debug Logs from Embedded Mobile Devices

Björn Daase,Leon Matthes,Lukas Pirl,Lukas Wenzel

DOI: https://doi.org/10.48550/arXiv.2110.01412

2021-10-05

Abstract:Mobile embedded devices of the Internet of Things (IoT) face tight resource constraints and uncertain environments, including energy scarcity and unstable connectivity. This aggravates debugging, optimization, monitoring, etc.; for which logging information must be accessible throughout all phases of development and product life cycles. This work compares approaches for transmitting logs with regard to application requirements (e.g., bandwidth), resource consumption (e.g., memory), operating constraints (e.g., power supply), and the medium (e.g., UART, WiFi). A qualitative comparison suggests that the adequacy of approaches depends on the concrete application and the phase in the life cycle. We report from our case study, where the embedded mobile device is represented by a self-driving slot car (Carrera D132). With this target device, failure logs, new firmware, and monitoring data need to be exchanged. The gathered experiences support our qualitative discussion: Wireless techniques can suit the needs of many phases in the life cycle but it is particularly evident that energy consumption is crucial. With a loaded wireless stack and while transmitting logs, the car's operating voltage drop within a \~ 20 ms power interruption is \~ 1.6 times higher. This limits communication flexibility significantly.

Distributed, Parallel, and Cluster Computing
Embedded Platform Patterns for Distributed and Secure Logging

Fikret Basic,Christian Steger,Robert Kofler

DOI: https://doi.org/10.1145/3489449.3490004

2023-11-09

Abstract:With the advent of modern embedded systems, logging as a process is becoming more and more prevalent for diagnostic and analytic services. Traditionally, storage and managing of the logged data are generally kept as a part of one entity together with the main logic components. In systems that implement network connections, this activity is usually handled over a remote device. However, enabling remote connection is still considered a limiting factor for many embedded devices due to the demanding production cost. A significant challenge is presented to vendors who need to decide how the data will be extracted and handled for an embedded platform during the design concept phase. It is generally desirable that logging memory modules are able to be addressed as separate units. These devices need to be appropriately secured and verifiable on a different system since data compromise can lead to enormous privacy and even financial losses. In this paper, we present two patterns. First, a pattern that allows flexible logging operation design in terms of module and interface responsibility separation. Second, a pattern for the design of secure logging processes during the utilization of constrained embedded devices. The introduced patterns fulfil the following conditions: (i) flexibility, design is independent of the chip vendors making the logging memory modules easily replaceable, (ii) self-sufficiency, every logging controller is maintained as a separate entity in a decentralized topology, (iii) security, through providing authenticity, confidentiality, and integrity by means of using a dedicated security module.

Cryptography and Security,Software Engineering,Systems and Control
Data Acquisition and Distributed Computing System Based on Embedded Network

Zhi-feng HAN,Zhi-huan SONG

DOI: https://doi.org/10.3969/j.issn.1004-1699.2006.03.088

2006-01-01

Abstract:Taking the advantage of network distributed openness and DSP high speed signal processing ability, the design and elements of system's hardware and software are described, as well as how it works through embedded network to make access to user station and transmit data by XML-Socket. The result shows that this system can work stably in the industrial environment with various sensors; it's able to fulfil the requirements of real-time data acquisition and distributed computing.
Post-Deployment Anomaly Detection and Diagnosis in Networked Embedded Systems by Program Profiling and Symptom Mining

Wei Dong,Luyao Luo,Chun Chen,Jiajun Bu,Xue Liu,Yunhao Liu

DOI: https://doi.org/10.1109/tpds.2016.2542815

IF: 5.3

2016-01-01

IEEE Transactions on Parallel and Distributed Systems

Abstract:Detecting and diagnosing anomalies in networked embedded systems like sensor networks is a very difficult task, due to the variable workloads and severe resource constraints. We notice that most node-level debugging tools can provide detailed program information inside the node but fail to detect when and where a problem occurs in the network. On the other hand, most network-level diagnosis tools can effectively detect a problem from the network but fail to narrow down the problem within the node because they lack detailed program information. To close the gap, we propose D2, a new anomaly detection and diagnosis method by combining program profiling and symptom mining. D2 employs binary instrumentation to perform lightweight function count profiling. Based on the statistics, D2 uses PCA (Principal Component Analysis) based approach for automatically detecting network anomalies. Compared to previous methods, D2 is able to point programmers closer to the most likely causes by a novel approach combining statistical tests and program call graph analysis. We implement our method based on TinyOS 2.1.1 and evaluate its effectiveness by case studies in the development of a working sensor network. Results show that our method is effective for detecting and diagnosing problems in real-world sensor network systems, and at the same time, incurs an acceptable overhead.
Machine Deserves Better Logging: A Log Enhancement Approach for Automatic Fault Diagnosis

Tong Jia,Ying Li,Chengbo Zhang,Wensheng Xia,Jie Jiang,Yuhong Liu

DOI: https://doi.org/10.1109/issrew.2018.00-22

2018-01-01

Abstract:When systems fail, log data is often the most important information source for fault diagnosis. However, the performance of automatic fault diagnosis is limited by the ad-hoc nature of logs. The key problem is that existing developer-written logs are designed for humans rather than machines to automatically detect system anomalies. To improve the quality of logs for fault diagnosis, we propose a novel log enhancement approach which automatically identifies logging points that reflect anomalous behavior during system fault. We evaluate our approach on three popular software systems AcmeAir, HDFS and TensorFlow. Results show that it can significantly improve fault diagnosis accuracy by 50% on average compared to the developers' manually placed logging points.
OSEK/VDX-Based Dynamic Network Management on Automotive Network

Chengjiong Wei,Min Yao,Pan Lu,Qi Hu,Nenggan Zheng

DOI: https://doi.org/10.1109/icess.2009.64

2009-01-01

Abstract:The rapid growth in complexity of the automotive ECU (Electronic Control Unit) networks has posed challenges in real-time network management, which collects network information for fault diagnostic systems. This paper proposes a dynamic network management framework based on the OSEK Network Management protocols, which can automatically adapt to the unstable network load. A method to calculate appropriate network load thresholds is also presented. Empirical study shows that this framework can satisfy the requirements of collecting necessary network information without sacrificing the performance of automotive ECU networks.
A Remote Control System for Embedded Environment

时磊,董金祥

DOI: https://doi.org/10.3969/j.issn.1000-3428.2004.22.072

2004-01-01

Abstract:This article describes a wireless remote control system based on remote log-in and dynamic binary code loading technique. The control system can perform operations remotely through wireless network, and it is capable of changing working process by loading and running binary code dynamically. Also, the remote control system has a certain security policy, which is an especial requirement of the application.
The Design of Embedded Digital Video Recorder System based on Network

CHEN Yao-hong,CHEN Yao-wu

DOI: https://doi.org/10.3969/j.issn.1002-6886.2006.04.028

2006-01-01

Abstract:An embedded digital video recorder system based on network is presented and developed.The system which combined computer、network and embedded technologies gives an effective remote control to DVR.The system design,hardware design,software design are discussed,especially the client section is described in detail.
Dynamic Features Incorporated Locally Weighted Deep Learning Model for Soft Sensor Development.

Le Yao,Zhiqiang Ge

DOI: https://doi.org/10.1109/tim.2021.3073702

IF: 5.6

2021-01-01

IEEE Transactions on Instrumentation and Measurement

Abstract:Local learning models have been widely applied for time-variant process soft sensor development, where historical samples that are similar to the online testing sample are selected for local modeling and prediction. However, those similarity measurements and local models are commonly established based on the static samples, which are seriously inadequate when process dynamics exist. To tackle this problem, a novel locally weighted deep learning algorithm that takes dynamic features to describe both the similarities and relationships is proposed in this article. First, a deep dynamic feature extracting (DFE) network is constructed based on the long short-term memory (LSTM) encoder-decoder with attention mechanism, which maps time-sequence samples to a group of hidden dynamic features. With the extracted features incorporated into the original input features, a locally weighted autoencoder regression (LWAER) network is proposed for soft sensor modeling. Meanwhile, since both networks consist of unsupervised feature extracting and supervised feature regression, a large scale of unlabeled samples can be utilized in the semisupervised learning of model parameters. Finally, the superiorities of the proposed dynamic features incorporated LWAER (DFI-LWAER) model is verified by the outstanding performances in two real industrial soft sensing cases.
Connecting the Dots: Reconstructing Network Behavior with Individual and Lossy Logs

Jiliang Wang,Xiaolong Zheng,Xufei Mao,Zhichao Cao,Daibo Liu,Yunhao Liu

DOI: https://doi.org/10.1109/icpp.2015.26

2015-01-01

Abstract:In distributed networks such as wireless ad hoc networks, local and lossy logs are often available on individual nodes. We propose REFILL, which analyzes lossy and unsynchronized logs collected from individual nodes and reconstructs the network behaviors. We design an inference engine based on protocol semantics to abstract states on each node. Further we leverage inherent and implicit event correlations in and between nodes to connect interference engines and analyze logs from different nodes. Based on unsynchronized and incomplete logs, REFILL can reconstruct network behavior, recover the network scenario and understand what has happened in the network. We show that the result of REFILL can be used to guide protocol design, network management, diagnosis, etc. We implement REFILL and apply it to a large-scale wireless sensor network project. REFILL provides a detailed per-packet tracing information based on event flows. We show that REFILL can reveal and verify fundamental issues, like locating packet loss positions and root causes. Further, we present implications and demonstrate how to leverage REFILL to enhance network performance.
Design of Wireless Logging Instrument System for Monitoring Oil Drilling Platform

Yan Yu,Siyue Huang,Jie Wang,Jinping Ou

DOI: https://doi.org/10.1109/jsen.2015.2391280

IF: 4.3

2015-01-01

IEEE Sensors Journal

Abstract:In order to achieve more effective monitoring and to solve the heavy workload issues of installation and removal on the existing wired-system logging site, thus guarantee the safety in normal operation, a wireless logging instrument system is designed to monitor the oil drilling platform. This paper introduces the overall architecture of the wireless system, and develops the hardware system - in which embedded software is designed - based on analog-digital conversion device, field-programmable gate array chip, and microprocessor. Combined with wireless sensor technology, analog measurements, and encoding quadruple frequency and phase-detection techniques, the wireless nodes in the system can detect and preprocess analog signals in the frequency range from 0.1 to 200 Hz and encoder signals of 90° phase difference, then output data through wireless transmitter module to upper monitoring center. Experimental results show that this system can realize a real-time wireless communication and monitoring for oil drilling platform stably. Moreover, it has features such as coding flexibility, high precision, good reliability, high integration, small size, and low-power consumption. Thus, the system provides a promising solution of data collection and analysis for the drilling platform.
Design Of Network-Based Distributed Industrial Monitoring System

N He,Jp Xiong,Cl Jiang,Hb Jia

2002-01-01

Abstract:An embedded system with Ethernet interface is designed to transfer the distributed acquisition data to the server, which plays the role of data storage and management. The service of delivering the raw data or data analysis statistics to the remote users is offered via network. The system hardware structure and software design are also presented in this paper. At the end of the paper, factors that affect the system performance are analyzed.
Disk logging algorithms and performance analysis based on Petri net models

Feng Zhou,Chuang Lin,Weimin Zheng

2002-01-01

Ruan Jian Xue Bao/Journal of Software

Abstract:Disk log is an important type of data structure used in databases and file systems. The synchronous nature of logging often renders it the performance bottleneck of the whole system. Using disk-scheduling algorithms optimized for logging is one way to solve this problem. A disk logging algorithm, latency-optimized logging, is proposed. A high level Petri net (HLPN) model of a disk subsystem and track-based logging and subsequently HLPN models of three logging algorithms (sector-continuous logging, track-based logging and latency-optimized logging) are built and performance of the algorithms is analyzed. The results show that both of the latter two can improve logging performance by a factor of ten and the proposed algorithm performs much better under heavy load.
Dynamic linking and loading in networked embedded systems

Wei Dong,Chun Chen,Xue Liu,Jiajun Bu,Yunhao Liu

DOI: https://doi.org/10.1109/MOBHOC.2009.5336957

2009-01-01

Abstract:We present a holistic dynamic linking and loading mechanism in networked embedded systems. Our design and implementation are guided by four requirements, which are to provide (i) minimal code size (ii) efficient execution and loading speed (iii) portable design (iv) isolated kernel/application development. First, we develop a tool to minimize the standard ELF format via many techniques in order to reduce the code dissemination cost. Second, we employ the techniques of pre-relocating and pre-linking (to kernel functions) to reduce the run-time linking overhead, thus improving the loading speed. Third, based on relocatable ELF and the modular design of the dynamic linker and loader, our approach can be easily ported to different platforms. Fourth, by maintaining a kernel jump table, we provide a clean isolation between kernel and application development. We have implemented the dynamic linking and loading mechanism on SenSpire OS, a micro sensor node operating system. The evaluation results show that our design and implementation meet our design goals: the code size of our SELF format is only 15%–30% of that of standard ELF, 38%–83% of that of CELF, a compact ELF format for the Contiki operating system; the loading speed improvement varies from 40%–50% compared to the standard mechanism; our design is portable to both MicaZ and TelosB motes, and we allow updating both application modules and kernel services in isolation without prior knowledge about the whole system information.
Finding Needles in the Haystack: Harnessing Syslogs for Data Center Management

Chen Liang,Theophilus Benson,Partha Kanuparthy,Yihua He

DOI: https://doi.org/10.48550/arXiv.1605.06150

2016-05-20

Abstract:Network device syslogs are ubiquitous and abundant in modern data centers with most large data centers producing millions of messages per day. Yet, the operational information reflected in syslogs and their implications on diagnosis or management tasks are poorly understood. Prevalent approaches to understanding syslogs focus on simple correlation and abnormality detection and are often limited to detection providing little insight towards diagnosis and resolution. Towards improving data center operations, we propose and implement Log-Prophet, a system that applies a toolbox of statistical techniques and domain-specific models to mine detailed diagnoses. Log-Prophet infers causal relationships between syslog lines and constructs succinct but valuable problem graphs, summarizing root causes and their locality, including cascading problems. We validate Log-Prophet using problem tickets and through operator interviews. To demonstrate the strength of Log-Prophet, we perform an initial longitudinal study of a large online service provider's data center. Our study demonstrates that Log-Prophet significantly reduces the number of alerts while highlighting interesting operational issues.

Networking and Internet Architecture

Dynamic Logging with Dylog in Networked Embedded Systems

Dynamic Logging with Dylog for Networked Embedded Systems

Understanding Routing Dynamics in a Large-Scale Wireless Sensor Network

THE IMPLEMENTATION OF FAULTS DEBUGGING AND DETECTION BASED ON VXWORKS

Development of remote pseudo-dynamic test platform

Research on embedded single address space OS in dynamic loading

Observing a Moving Target -- Reliable Transmission of Debug Logs from Embedded Mobile Devices

Embedded Platform Patterns for Distributed and Secure Logging

Data Acquisition and Distributed Computing System Based on Embedded Network

Post-Deployment Anomaly Detection and Diagnosis in Networked Embedded Systems by Program Profiling and Symptom Mining

Machine Deserves Better Logging: A Log Enhancement Approach for Automatic Fault Diagnosis

OSEK/VDX-Based Dynamic Network Management on Automotive Network

A Remote Control System for Embedded Environment

The Design of Embedded Digital Video Recorder System based on Network

Dynamic Features Incorporated Locally Weighted Deep Learning Model for Soft Sensor Development.

Connecting the Dots: Reconstructing Network Behavior with Individual and Lossy Logs

Design of Wireless Logging Instrument System for Monitoring Oil Drilling Platform

Design Of Network-Based Distributed Industrial Monitoring System

Disk logging algorithms and performance analysis based on Petri net models

Dynamic linking and loading in networked embedded systems

Finding Needles in the Haystack: Harnessing Syslogs for Data Center Management