Abstract:We propose a novel guidance approach for visual Exploratory Data Analysis (EDA) systems in industrial sensor data use cases. We demonstrate its feasibility by capturing expert EDA sessions, generating guided EDA notebooks, and examining the reproducibility of expert insights based on these notebooks through a user study. In recent years, digitization in the industrial sector has increased steadily. Digital data not only allows us to monitor the underlying production process using machine learning methods (anomaly detection, behaviour analysis) but also to understand the underlying production process. Insights from Exploratory Data Analysis (EDA) play an important role in building data‐driven processes because data scientists learn essential characteristics of the data in the context of the domain. Due to the complexity of production processes, it is usually difficult for data scientists to acquire this knowledge by themselves. Hence, they have to rely on continuous close collaboration with domain experts and their acquired domain expertise. However, direct communication does not promote documentation of the knowledge transfer from domain experts to data scientists. In this respect, changing team constellations, for example due to a change in personnel, result in a renewed high level of effort despite the same knowledge transfer problem. As a result, EDA is a cost‐intensive iterative process. We, therefore, investigate a system to extract information from the interactions that domain experts perform during EDA. Our approach relies on recording interactions and system states of an exploration tool and generating guided exploration sessions for domain novices. We implement our approach in a software tool and demonstrate its capabilities using two real‐world use cases from the manufacturing industry. We evaluate its feasibility in a user study to investigate whether domain novices can reproduce the most important insights from domain experts about the datasets of the use cases based on generated EDA sessions. From the results of this study, we conclude the feasibility of our system as participants are able to reproduce on average 86.5% of insights from domain experts.

Exploratory text data analysis for quality hypothesis generation

Exploratory image data analysis for quality improvement hypothesis generation

A text-based analysis approach to representing the design selection process

ETC-Based Methodology on Data Processing Validation

Goals, Process, and Challenges of Exploratory Data Analysis: An Interview Study

Qutaber: task-based exploratory data analysis with enriched context awareness

Automating Exploratory Data Analysis via Machine Learning: An Overview

Guided Exploration of Industrial Sensor Data

DETEXA: declarative extensible text exploration and analysis through SQL

Exploratory Data Analysis using Python

The artificial intelligence advantage: Supercharging exploratory data analysis

Towards High-Performance Exploratory Data Analysis (EDA) Via Stable Equilibrium Point

Using Text Mining and Data Mining Techniques for Applied Learning Assessment

Automated Question Generation on Tabular Data for Conversational Data Exploration

A Generalized Methodology for Data Analysis

MetaInsight: Automatic Discovery of Structured Knowledge for Exploratory Data Analysis

Exploratory Data Analysis for Banking and Finance: Unveiling Insights and Patterns

Quality Analytics in a Big Data supply chain: Commodity data analytics for quality engineering

An experiment exploring the theoretical and methodological challenges in developing a semi-automated approach to analysis of small-N qualitative data

Drilling data quality improvement and information extraction with case studies

Exploranative Code Quality Documents