Robert V. Binder
Abstract:Thirty-six years after the first edition of IEEE standard 982.1, Measures of the Software Aspects of Dependability, the third edition focuses on the measurement of in-service software dependability. This article explains how this new point of view evolved and shaped the third edition's guidance for software dependability measurement.
Software Engineering,Computational Engineering, Finance, and Science,Cryptography and Security,Performance
What problem does this paper attempt to address?
The paper focuses on the measurement of software reliability, usability, maintainability, and recoverability, which is referred to as software trustworthiness, and discusses how the third edition of IEEE Standard 982.1 focuses on the measurement of software trustworthiness during service. The new version of the standard no longer focuses on recommendations for development practices, but instead focuses on black-box measurement of software during service, introducing real-time anomaly scoring criteria to ensure visibility into faulty behavior, security anomalies, and vulnerabilities. The standard also distinguishes between software support and software maintenance, clarifying the boundaries between downtime caused by service actions and work required for upstream facilities.
The new version of Standard 982.1 (possibly referred to as 982-2024) emphasizes the following changes:
1. Emphasizes black-box measurement methods during service, excluding predictive models for software reliability growth.
2. Replaces maintainability with supportiveness to reflect software's support actions during runtime, rather than upstream debugging and programming work.
3. Defines software trustworthiness, including measurements of reliability, usability, recoverability, and supportiveness, to meet required threshold levels.
4. Introduces the In-Service Reference Model (ISRM) to structure the classification of various anomalies or failures and describe their characteristics using established reliability formulas.
5. Emphasizes explicit trustworthiness requirements, quantifying them using percentiles or confidence intervals rather than just average values.
The focus of the standard is to provide a consistent and actionable measurement framework for software-intensive systems, addressing the critical role of 21st-century software in business, security, safety, and societal risks. It emphasizes the importance of measurement to answer questions such as "Is system X more reliable (or unreliable) compared to system Y?" and "Has the trustworthiness of my system increased or decreased across successive versions?"