Abstract:Context: Ensuring safety for any sophisticated system is getting more complex due to the rising number of features and functionalities. This calls for formal methods to entrust confidence in such systems. Nevertheless, using formal methods in industry is demanding because of their lack of usability and the difficulty of understanding verification results. Objective: We evaluate the acceptance of formal methods by Bosch automotive engineers, particularly whether the difficulty of understanding verification results can be reduced. Method: We perform two different exploratory studies. First, we conduct a user survey to explore challenges in identifying inconsistent specifications and using formal methods by Bosch automotive engineers. Second, we perform a one-group pretest-posttest experiment to collect impressions from Bosch engineers familiar with formal methods to evaluate whether understanding verification results is simplified by our counterexample explanation approach. Results: The results from the user survey indicate that identifying refinement inconsistencies, understanding formal notations, and interpreting verification results are challenging. Nevertheless, engineers are still interested in using formal methods in real-world development processes because it could reduce the manual effort for verification. Additionally, they also believe formal methods could make the system safer. Furthermore, the one-group pretest-posttest experiment results indicate that engineers are more comfortable understanding the counterexample explanation than the raw model checker output. Limitations: The main limitation of this study is the generalizability beyond the target group of Bosch automotive engineers.

A Preliminary Study of the Influence of Experiential Backgrounds of Respondents to A Reference Survey for Verification of Fsv

Comparison of Data with Multiple Degrees of Freedom Utilizing the Feature Selective Validation Method

Performance Improvement Of Fsv In A Special Situation

FSV: State of the Art and Current Research Fronts

Data Comparison with Many Degrees of Freedom: the FSV (feature Selective Validation) in Multiple Dimensions

An Experience-Based Approach for Test Execution Effort Estimation

Professionalism among Chinese Engineers: an Empirical Study.

Investigating Confidence Histograms and Classification in Fsv: Part Ii-Float Fsv

A Statistical Assessment of the Performance of FSV

The Role Of Fsv In Emc Characterization

Comparison of Three-Dimensional Datasets by Using the Generalized N-Dimensional ( N-D) Feature Selective Validation (FSV) Technique

Reliability of self-rated experience and confidence as predictors for students’ performance in software engineering

Applying the Analytic Hierarchy Process (AHP) to an FSV-Based Comparison of Multiple Datasets

Using Image Quality Assessment (IQA) Databases to Provide an Appraisal of the Ability of the Feature Selective Validation Method (FSV) to Compare Two-Dimensional Datasets

A Review on the Drawbacks and Enhancement Opportunities of the Feature Selective Validation

Notice of Retraction: Statistical Figures of Merit for the Feature Selective Validation Method

Notice of Retraction: Review of the Feature Selective Validation Method (FSV). Part II - Performance Analysis and Research Fronts

A User-Study Protocol for Evaluation of Formal Verification Results and their Explanation

A User Study for Evaluation of Formal Verification Results and their Explanation at Bosch

Inferencing into the void: problems with implicit populations Comments on `Empirical software engineering experts on the use of students and professionals in experiments'

Useful Statistical Methods for Human Factors Research in Software Engineering: A Discussion on Validation with Quantitative Data