Quantifying the Dynamics of Failure Across Science, Startups and Security

Yian Yin,Yang Wang,James A. Evans,Dashun Wang

DOI: https://doi.org/10.1038/s41586-019-1725-y

IF: 64.8

2019-01-01

Nature

Abstract:An amendment to this paper has been published and can be accessed via a link at the top of the paper.

What problem does this paper attempt to address?

Quantifying dynamics of failure across science, startups, and security

Yian Yin,Yang Wang,James A. Evans,Dashun Wang

DOI: https://doi.org/10.1038/s41586-019-1725-y

2019-03-19

Abstract:Human achievements are often preceded by repeated attempts that initially fail, yet little is known about the mechanisms governing the dynamics of failure. Here, building on the rich literature on innovation, human dynamics and learning, we develop a simple one-parameter model that mimics how successful future attempts build on those past. Analytically solving this model reveals a phase transition that separates dynamics of failure into regions of stagnation or progression, predicting that near the critical threshold, agents who share similar characteristics and learning strategies may experience fundamentally different outcomes following failures. Below the critical point, we see those who explore disjoint opportunities without a pattern of improvement, and above it, those who exploit incremental refinements to systematically advance toward success. The model makes several empirically testable predictions, demonstrating that those who eventually succeed and those who do not may be initially similar, yet are characterized by fundamentally distinct failure dynamics in terms of the efficiency and quality of each subsequent attempt. We collected large-scale data from three disparate domains, tracing repeated attempts by (i) NIH investigators to fund their research, (ii) innovators to successfully exit their startup ventures, and (iii) terrorist organizations to post casualties in violent attacks, finding broadly consistent empirical support across all three domains. Together, our findings unveil identifiable yet previously unknown early signals that allow us to identify failure dynamics that will lead to ultimate victory or defeat. Given the ubiquitous nature of failures and the paucity of quantitative approaches to understand them, these results represent a crucial step toward deeper understanding of the complex dynamics beneath failures, the essential prerequisites for success.

Physics and Society,Digital Libraries,Social and Information Networks
Dynamics of Load Entropy During Cascading Failure Propagation in Scale-Free Networks

Z. J. Bao,Y. J. Cao,L. J. Ding,Z. X. Han,G. Z. Wang

DOI: https://doi.org/10.1016/j.physleta.2008.07.034

IF: 2.707

2008-01-01

Physics Letters A

Abstract:In this Letter, we introduce the concept of load entropy, which can be an average measure of a network's heterogeneity in the load distribution. Then we investigate the dynamics of load entropy during failure propagation using a new cascading failures load model, which can represent the node removal mechanism in many real-life complex systems. Simulation results show that in the early stage of failure propagation the load entropy for a larger cascading failure increases more sharply than that for a smaller one, and consequently the cascading failure with a larger damage can be identified at the early stage of failure propagation according to the load entropy. Particularly, load entropy can be used as an index to be optimized in cascading failures control and defense in many real-life complex networks.
A Systematic Study of Failure Proximity

Chao Liu,Xiangyu Zhang,Jiawei Han

DOI: https://doi.org/10.1109/TSE.2008.66

IF: 7.4

2008-01-01

IEEE Transactions on Software Engineering

Abstract:Software end-users are the best testers, who keep revealing bugs in software that has undergone rigorous in-house testing. In order to leverage their testing efforts, failure reporting components have been widely deployed in released software. Many utilities of the collected failure data depend on an effective failure indexing technique, which, at the optimal case, would index all failures due to the same bug together. Unfortunately, the problem of failure proximity, which underpins the effectiveness of an indexing technique, has not been systematically studied. This article presents the first systematic study of failure proximity. A failure proximity consists of two components: a fingerprinting function that extracts signatures from failures, and a distance function that calculates the likelihood of two failures being due to the same bug. By considering different instantiations of the two functions, we study an array of six failure proximities (two of them are new) in this article. These proximities range from the simplest approach that checks failure points to the most sophisticated approach that utilizes fault localization algorithms to extract failure signatures. Besides presenting technical details of each proximity, we also study the properties of each proximity and tradeoffs between proximities. These altogether deliver a systematic view of failure proximity.
Failure dynamics of the global risk network

Boleslaw K. Szymanski,Xin Lin,Andrea Asztalos,Sameet Sreenivasan

DOI: https://doi.org/10.1038/srep10998

2015-05-13

Abstract:Risks threatening modern societies form an intricately interconnected network that often underlies crisis situations. Yet, little is known about how risk materializations in distinct domains influence each other. Here we present an approach in which expert assessments of risks likelihoods and influence underlie a quantitative model of the global risk network dynamics. The modeled risks range from environmental to economic and technological and include difficult to quantify risks, such as geo-political or social. Using the maximum likelihood estimation, we find the optimal model parameters and demonstrate that the model including network effects significantly outperforms the others, uncovering full value of the expert collected data. We analyze the model dynamics and study its resilience and stability. Our findings include such risk properties as contagion potential, persistence, roles in cascades of failures and the identity of risks most detrimental to system stability. The model provides quantitative means for measuring the adverse effects of risk interdependence and the materialization of risks in the network.

Computers and Society,Social and Information Networks,Physics and Society
Quantification of Distributionally Robust Risk of Cascade of Failures in Platoon of Vehicles

Vivek Pandey,Guangyi Liu,Arash Amini,Nader Motee

2023-09-10

Abstract:Achieving safety is a critical aspect of attaining autonomy in a platoon of autonomous vehicles. In this paper, we propose a distributionally robust risk framework to investigate cascading failures in platoons. To examine the impact of network connectivity and system dynamics on the emergence of cascading failures, we consider a time-delayed network model of the platoon of vehicles as a benchmark. To study the cascading effects among pairs of vehicles in the platoon, we use the measure of conditional distributionally robust functional. We extend the risk framework to quantify cascading failures by utilizing a bi-variate normal distribution. Our work establishes closed-form risk formulas that illustrate the effects of time-delay, noise statistics, underlying communication graph, and sets of soft failures. The insights gained from our research can be applied to design safe platoons that are robust to the risk of cascading failures. We validate our results through extensive simulations.

Systems and Control
The Dynamics of Product Development in Software Startups

Narendranath Shanbhag,Eric Pardede

DOI: https://doi.org/10.4018/978-1-6684-3702-5.ch101

2022-01-01

Abstract:Software startups are increasingly under high pressure to deliver successful products to survive and thrive in the modern highly competitive technology market. Larger organizations with deep pockets can replicate the same business ideas used by startups with relative ease. So how does the average startup stand a chance at succeeding at this seemingly David vs. Goliath contest? This article looks at the available literature and identifies such factors that can affect the success of software development startups. Using causal loop constructs from the field of system dynamics, the interactions among the various identified factors are visualised to reveal the dynamics of the system. The result is as a three-dimensional view of success factors in form of time, capital and (product) differentiation. The modelled system is then simulated, and the resultant trend is reviewed and interpreted. This research acts as ground work for analysing the workings of software development startups and sets the stage for a more holistic study of the area, upon which further research can be carried out.
Quantifying the Dynamics of Innovation Abandonment Across Scientific, Technological, Commercial, and Pharmacological Domains

Binglu Wang,Ching Jin,Chaoming Song,Johannes Bjelland,Brian Uzzi,Dashun Wang

2024-12-10

Abstract:Despite the vast literature on the diffusion of innovations that impacts a broad range of disciplines, our understanding of the abandonment of innovations remains limited yet is essential for a deeper understanding of the innovation lifecycle. Here, we analyze four large-scale datasets that capture the temporal and structural patterns of innovation abandonment across scientific, technological, commercial, and pharmacological domains. The paper makes three primary contributions. First, across these diverse domains, we uncover one simple pattern of preferential abandonment, whereby the probability for individuals or organizations to abandon an innovation increases with time and correlates with the number of network neighbors who have abandoned the innovation. Second, we find that the presence of preferential abandonment fundamentally alters the way in which the underlying ecosystem breaks down, inducing a novel structural collapse in networked systems commonly perceived as robust against abandonments. Third, we derive an analytical framework to systematically understand the impact of preferential abandonment on network dynamics, pinpointing specific conditions where it may accelerate, decelerate, or have an identical effect compared to random abandonment, depending on the network topology. Together, these results deepen our quantitative understanding of the abandonment of innovation within networked social systems, with implications for the robustness and functioning of innovation communities. Overall, they demonstrate that the dynamics of innovation abandonment follow simple yet reproducible patterns, suggesting that the uncovered preferential abandonment may be a generic property of the innovation lifecycle.

Physics and Society,Data Analysis, Statistics and Probability
Rain or Shine? — Making Sense of Cloudy Reliability Data

Iyswarya Narayanan,Bikash Sharma,Di Wang,Sriram Govindan,Laura Caulfield,Anand Sivasubramaniam,Aman Kansal,Jie Liu,Badriddine Khessib,Kushagra Vaid

DOI: https://doi.org/10.1109/ICDCS.2017.103

2017-01-01

Abstract:Cloud datacenters must ensure high availability for the hosted applications and failures can be the bane of datacenter operators. Understanding the what, when and why of failures can help tremendously to mitigate their occurrence and impact. Failures can, however, depend on numerous spatial and temporal factors spanning hardware, workloads, support facilities, and even the environment. One has to rely on failure data from the field to quantify the influence of these factors on failures. Towards this goal, we collect failures data along with many parameters that might influence failures from two large production datacenters with very diverse characteristics. We show that multiple factors simultaneously affect failures, and these factors may interact in non-trivial ways. This makes conventional approaches that study aggregate characteristics or single parameter influences, rather inaccurate. Instead, we build a multi-factor analysis framework to systematically identify influencing factors, quantify their relative impact, and help in more accurate decision making for failure mitigation. We demonstrate this approach for three important decisions: spare capacity provisioning, comparing the reliability of hardware for vendor selection, and quantifying flexibility in datacenter climate control for cost-reliability trade-offs.
Success and Failure in Software Engineering: a Followup Systematic Literature Review

Damian A. Tamburri,Fabio Palomba,Rick Kazman

DOI: https://doi.org/10.48550/arXiv.2006.12086

2020-06-22

Abstract:Success and failure in software engineering are still among the least understood phenomena in the discipline. In a recent special journal issue on the topic, Mantyla et al. started discussing these topics from different angles; the authors focused their contributions on offering a general overview of both topics without deeper detail. Recognising the importance and impact of the topic, we have executed a followup, more in-depth systematic literature review with additional analyses beyond what was previously provided. These new analyses offer: (a) a grounded-theory of success and failure factors, harvesting over 500+ factors from the literature; (b) 14 manually-validated clusters of factors that provide relevant areas for success- and failure-specific measurement and risk-analysis; (c) a quality model composed of previously unmeasured organizational structure quantities which are germane to software product, process, and community quality. We show that the topics of success and failure deserve further study as well as further automated tool support, e.g., monitoring tools and metrics able to track the factors and patterns emerging from our study. This paper provides managers with risks as well as a more fine-grained analysis of the parameters that can be appraised to anticipate the risks.

Software Engineering
Analysis of Software Reliability Modeling Considering Testing Compression Factor and Failure-to-Fault Relationship.

Chin-Yu Huang,Chu-Ti Lin

DOI: https://doi.org/10.1109/tc.2009.103

IF: 3.183

2010-01-01

IEEE Transactions on Computers

Abstract:This paper is an attempt to relax and improve the assumptions regarding software reliability modeling. To approximate reality much more closely, we take into account the concepts of testing compression factor and the quantified ratio of faults to failures in the modeling. Numerical examples based on real failure data show that the proposed framework has a fairly good prediction capability. Further, we also address the optimal software release time problem and conduct a detailed sensitivity analysis through the proposed model.
Why Early-Stage Software Startups Fail: A Behavioral Framework

Carmine Giardino,Xiaofeng Wang,Pekka Abrahamsson

DOI: https://doi.org/10.48550/arXiv.1709.04749

2017-09-14

Software Engineering

Abstract:Software startups are newly created companies with little operating history and oriented towards producing cutting-edge products. As their time and resources are extremely scarce, and one failed project can put them out of business, startups need effective practices to face with those unique challenges. However, only few scientific studies attempt to address characteristics of failure, especially during the early- stage. With this study we aim to raise our understanding of the failure of early-stage software startup companies. This state-of-practice investigation was performed using a literature review followed by a multiple-case study approach. The results present how inconsistency between managerial strategies and execution can lead to failure by means of a behavioral framework. Despite strategies reveal the first need to understand the problem/solution fit, actual executions prioritize the development of the product to launch on the market as quickly as possible to verify product/market fit, neglecting the necessary learning process.
Reflections on Software Failure Analysis

Paschal C. Amusuo,Aishwarya Sharma,Siddharth R. Rao,Abbey Vincent,James C. Davis

DOI: https://doi.org/10.1145/3540250.3560879

2022-09-22

Abstract:Failure studies are important in revealing the root causes, behaviors, and life cycle of defects in software systems. These studies either focus on understanding the characteristics of defects in specific classes of systems or the characteristics of a specific type of defect in the systems it manifests in. Failure studies have influenced various software engineering research directions, especially in the area of software evolution, defect detection, and program repair. In this paper, we reflect on the conduct of failure studies in software engineering. We reviewed a sample of 52 failure study papers. We identified several recurring problems in these studies, some of which hinder the ability of the engineering community to trust or replicate the results. Based on our findings, we suggest future research directions, including identifying and analyzing failure causal chains, standardizing the conduct of failure studies, and tool support for faster defect analysis.

Software Engineering
Quantifying Success in Science: An Overview

Xiaomei Bai,Hanxiao Pan,Jie Hou,Teng Guo,Ivan Lee,Feng Xia

DOI: https://doi.org/10.48550/arXiv.2008.04649

2020-08-10

Abstract:Quantifying success in science plays a key role in guiding funding allocations, recruitment decisions, and rewards. Recently, a significant amount of progresses have been made towards quantifying success in science. This lack of detailed analysis and summary continues a practical issue. The literature reports the factors influencing scholarly impact and evaluation methods and indices aimed at overcoming this crucial weakness. We focus on categorizing and reviewing the current development on evaluation indices of scholarly impact, including paper impact, scholar impact, and journal impact. Besides, we summarize the issues of existing evaluation methods and indices, investigate the open issues and challenges, and provide possible solutions, including the pattern of collaboration impact, unified evaluation standards, implicit success factor mining, dynamic academic network embedding, and scholarly impact inflation. This paper should help the researchers obtaining a broader understanding of quantifying success in science, and identifying some potential research directions.

Digital Libraries,Physics and Society
The failure tolerance of mechatronic software systems to random and targeted attacks

Dharshana Kasthurirathna,Andy Dong,Mahendrarajah Piraveenan,Irem Y. Tumer

DOI: https://doi.org/10.48550/arXiv.1310.1050

2013-09-27

Distributed, Parallel, and Cluster Computing

Abstract:This paper describes a complex networks approach to study the failure tolerance of mechatronic software systems under various types of hardware and/or software failures. We produce synthetic system architectures based on evidence of modular and hierarchical modular product architectures and known motifs for the interconnection of physical components to software. The system architectures are then subject to various forms of attack. The attacks simulate failure of critical hardware or software. Four types of attack are investigated: degree centrality, betweenness centrality, closeness centrality and random attack. Failure tolerance of the system is measured by a 'robustness coefficient', a topological 'size' metric of the connectedness of the attacked network. We find that the betweenness centrality attack results in the most significant reduction in the robustness coefficient, confirming betweenness centrality, rather than the number of connections (i.e. degree), as the most conservative metric of component importance. A counter-intuitive finding is that "designed" system architectures, including a bus, ring, and star architecture, are not significantly more failure-tolerant than interconnections with no prescribed architecture, that is, a random architecture. Our research provides a data-driven approach to engineer the architecture of mechatronic software systems for failure tolerance.
Series System Failure Rate Modeling and the Mechanism to Yield Bathtub Curve

Liyang Xie,Wenqiang Lin,Mingchuan Zhang,Shaoze Yan

DOI: https://doi.org/10.4028/www.scientific.net/amm.121-126.4858

2011-01-01

Applied Mechanics and Materials

Abstract:Failure rate functions were usually constructed in terms of life distribution. The present paper defines “discrete failure rate”, analyzes the dependence of failure rate curve shape on the probabilistic property of load-strength interaction, and develops load-strength-interaction based system failure rate model. For the majority of mechanical equipment, operation load is normally a random process, components are subjected to a large number of stochastic load actions during its service life. Meanwhile, material property degrades continuously owing to the failure mechanisms such as fatigue. It is the ever changing load-strength relationship that yields the variation in failure rate. Based on the criterion that failure occurs when a load higher than the relevant strength appears, failure rate model is developed based on the interaction mechanism between load and residual strength, and their statistical characteristics as well. Based on the interpretation to the variation of failure rate in terms of load uncertainty and strength uncertainty, the effect of statistical risk of load and the effect of strength degradation on failure rate curve shape are highlighted.
Study on Influencing Factors of Enterprise R&D and the Failure Cause

Zhao Wenhong,Xu Yuan

DOI: https://doi.org/10.3969/j.issn.1001-7348.2011.04.016

2011-01-01

Abstract:As the market competition increases,R&D becomes an important means of remaining competitive for enterprises and their influencing factors have been the key field of academic researches.This paper focused on the empirical analysis of these influencing factors.Several results came out and we believed that creating an atmosphere that tolerating failure would help decrease the R&D failure rate.
A Simulation Method to Estimate Two Types of Time-Varying Failure Rate of Dynamic Systems

Zhonglai Wang,Xiaoqiang Zhang,Hong-Zhong Huang,Zissimos P. Mourelatos

DOI: https://doi.org/10.1115/1.4034300

2016-01-01

Journal of Mechanical Design

Abstract:The failure rate of dynamic systems with random parameters is time-varying even for linear systems excited by a stationary random input. In this paper, we propose a simulation-based method to estimate two types (type I and type II) of time-varying failure rate of dynamic systems. The input stochastic processes are discretized in time and the trajectories of the output stochastic process are calculated. The time of interest is partitioned into a series of time intervals and the saddlepoint approximation (SPA) is employed to estimate the probability of failure in each interval. Type I follows the commonly used definition of failure rate. It is estimated at discrete time intervals using SPA and the correlation information from a properly selected time-dependent copula function. Type II is a proposed new concept of time-varying failure rate. It provides a way to predict the failure rate considering a virtual "good-as-old" repair action of repairable dynamic systems. The effectiveness of the proposed method is illustrated with a vehicle vibration example.
The Time-Dependent Failure Mechanism of Rocks and Associated Application in Slope Engineering: An Explanation Based on Numerical Investigation

Honglei Liu,Lianchong Li,Shaohua Li,Weimin Yang

DOI: https://doi.org/10.1155/2020/1680265

IF: 1.43

2020-03-29

Mathematical Problems in Engineering

Abstract:In this study, a numerical model for long-term deformation and progressive failure of rock slope is presented. The model accounts for both rock heterogeneity and the initiation, activation, nucleation, and coalescence of cracks in rock slope through a stochastic local stress field and local rock degradation by using an exponential softening law. The time-dependent behaviour of rocks is taken as a macroscopic consequence of damage evolution and strength degradation in microstructure. A series of demonstrative slope cases containing preexisting joints are constructed and investigated. The slope instability occurs at a particular point in time when the rock strength is reduced to a certain value. The temporal and spatial evolution of joint linkage structures is numerically obtained, which clearly shows how the local stress field and damage evolution within the joint network contribute to the fracture pattern and the long-term instability. Then, a practical slope case in jointed and layered rock formations in Yunyang city is studied. The prevailing failure phenomena of the slope, including gradual surface scaling, sliding collapses, and block falling, are numerically reproduced, with an emphasis placed on the slope failure process and development tendency. There is a good agreement on the failure mode and instability time between the numerical simulations and the field observations.

engineering, multidisciplinary,mathematics, interdisciplinary applications
Failure Analysis and Quantification for Contemporary and Future Supercomputers

Li Tan,Nathan DeBardeleben

DOI: https://doi.org/10.48550/arXiv.1911.02118

2019-11-06

Abstract:Large-scale computing systems today are assembled by numerous computing units for massive computational capability needed to solve problems at scale, which enables failures common events in supercomputing scenarios. Considering the demanding resilience requirements of supercomputers today, we present a quantitative study on fine-grained failure modeling for contemporary and future large-scale computing systems. We integrate various types of failures from different system hierarchical levels and system components, and summarize the overall system failure rates formally. Given that nowadays system-wise failure rate needs to be capped under a threshold value for reliability and cost-efficiency purposes, we quantitatively discuss different scenarios of system resilience, and analyze the impacts of resilience to different error types on the variation of system failure rates, and the correlation of hierarchical failure rates. Moreover, we formalize and showcase the resilience efficiency of failure-bounded supercomputers today.

Distributed, Parallel, and Cluster Computing
Cascading Failure in Cyber-Physical Systems: A Review on Failure Modeling and Vulnerability Analysis

Sizhe He,Yadong Zhou,Yujie Yang,Ting Liu,Yuxun Zhou,Jie Li,Tong Wu,Xiaohong Guan

DOI: https://doi.org/10.1109/tcyb.2024.3411868

IF: 11.8

2024-01-01

IEEE Transactions on Cybernetics

Abstract:Cascading failures pose a significant security threat to networked systems, with recent global incidents underscoring their destructive potential. The security threat of cascading failures has always existed, but the evolution of cyber-physical systems (CPSs) has introduced novel dimensions to cascading failures, intensifying their threats owing to the intricate fusion of cyber and physical domains. Addressing these threats requires a nuanced understanding achieved through failure modeling and vulnerability analysis. By analyzing the historical failures in different CPSs, the cascading failure in CPSs is comprehensively defined as a complicated propagation process in coupled cyber and physical systems, initialized by natural accidents or human interference, which exhibits a progressive evolution within the networked structure and ultimately results in unexpected large-scale systemic failures. Subsequently, this study advances the development of instructions for modeling cascading failures and conducting vulnerability analyses within CPSs. The examination also delves into the core challenges inherent in these methodologies. Moreover, a comprehensive survey and classification of extant research methodologies and solutions are undertaken, accompanied by a concise evaluation of their advancements and limitations. To validate the performance of these methodologies, numerical experiments are conducted to ascertain their distinct features. In conclusion, this article advocates for future research initiatives, particularly emphasizing the exploration of uncertainty analysis, defense strategies, and verification platforms. By addressing these areas, the resilience of CPSs against cascading failures can be significantly enhanced.

Quantifying the Dynamics of Failure Across Science, Startups and Security

Quantifying dynamics of failure across science, startups, and security

Dynamics of Load Entropy During Cascading Failure Propagation in Scale-Free Networks

A Systematic Study of Failure Proximity

Failure dynamics of the global risk network

Quantification of Distributionally Robust Risk of Cascade of Failures in Platoon of Vehicles

The Dynamics of Product Development in Software Startups

Quantifying the Dynamics of Innovation Abandonment Across Scientific, Technological, Commercial, and Pharmacological Domains

Rain or Shine? — Making Sense of Cloudy Reliability Data

Success and Failure in Software Engineering: a Followup Systematic Literature Review

Analysis of Software Reliability Modeling Considering Testing Compression Factor and Failure-to-Fault Relationship.

Why Early-Stage Software Startups Fail: A Behavioral Framework

Reflections on Software Failure Analysis

Quantifying Success in Science: An Overview

The failure tolerance of mechatronic software systems to random and targeted attacks

Series System Failure Rate Modeling and the Mechanism to Yield Bathtub Curve

Study on Influencing Factors of Enterprise R&D and the Failure Cause

A Simulation Method to Estimate Two Types of Time-Varying Failure Rate of Dynamic Systems

The Time-Dependent Failure Mechanism of Rocks and Associated Application in Slope Engineering: An Explanation Based on Numerical Investigation

Failure Analysis and Quantification for Contemporary and Future Supercomputers

Cascading Failure in Cyber-Physical Systems: A Review on Failure Modeling and Vulnerability Analysis