Abstract:Metrics and frameworks to quantifiably assess security measures have arisen from needs of three distinct research communities - statistical measures from the intrusion detection and prevention literature, evaluation of cyber exercises, e.g.,red-team and capture-the-flag competitions, and economic analyses addressing cost-versus-security tradeoffs. In this paper we provide two primary contributions to the security evaluation literature - a representative survey, and a novel framework for evaluating security that is flexible, applicable to all three use cases, and readily interpretable. In our survey of the literature we identify the distinct themes from each community's evaluation procedures side by side and flesh out the drawbacks and benefits of each. The evaluation framework we propose includes comprehensively modeling the resource, labor, and attack costs in dollars incurred based on expected resource usage, accuracy metrics, and time. This framework provides a unified approach in that it incorporates the accuracy and performance metrics, which dominate intrusion detection evaluation, the time to detection and impact to data and resources of an attack, favored by educational competitions' metrics, and the monetary cost of many essential security components used in financial analysis. Moreover, it is flexible enough to accommodate each use case, easily interpretable and comparable, and comprehensive in terms of costs <a class="link-external link-http" href="http://considered.Finally" rel="external noopener nofollow">this http URL</a>, we provide two examples of the framework applied to real-world use cases. Overall, we provide a survey and a grounded, flexible framework with multiple concrete examples for evaluating security which can address the needs of three currently distinct communities.

Model-Based Quantitative Network Security Metrics: A Survey

Quantifiable Network Security Measurement - A Study Based on an Index System.

Towards Quantifying the (in)security of Networked Systems.

Composite Metrics for Network Security Analysis

A Stochastic Model for Quantitative Security Analyses of Networked Systems.

Quantifiable & Comparable Evaluations of Cyber Defensive Capabilities: A Survey & Novel, Unified Approach

Security Rating Metrics for Distributed Wireless Systems

How to Quantify the Security Level of Embedded Systems? A Taxonomy of Security Metrics

Present and Future of Network Security Monitoring

A Two-Fold Traffic Flow Model for Network Security Management

Predictive Cyber-security Analytics Framework: A non-homogenous Markov model for Security Quantification

Multiple behavior information fusion based quantitative threat evaluation

Quantitative Security Risk Evaluation using CVSS Metrics by Estimation of Frequency and Maturity of Exploit

A Quantitative Method for Evaluating Network Security Based on Attack Graph.

Model-based Joint Analysis of Safety and Security: Survey and Identification of Gaps

Stochastic Modeling and Evaluation for Network Security

Quantitative Measurement of Cyber Resilience: Modeling and Experimentation

Network Security Modeling and Cyber Attack Simulation Methodology

Centrality Metrics of Importance in Access Behaviors and Malware Detections.

Cybersecurity: A Stochastic Predictive Model to Determine Overall Network Security Risk Using Markovian Process

Study of Security Metrics of Software System for Comparative Evaluation