Abstract:Machine Learning (ML) software, used to implement an ML algorithm, is widely used in many application domains such as financial, business, and engineering domains. Faults in ML software can cause substantial losses in these application domains. Thus, it is very critical to conduct effective testing of ML software to detect and eliminate its faults. However, testing ML software is difficult, especially on producing test oracles used for checking behavior correctness (such as using expected properties or expected test outputs). To tackle the test-oracle issue, this thesis presents a novel black-box approach of multiple-implementation testing for supervised learning software. The insight underlying the approach is that there can be multiple implementations (independently written) for a supervised learning algorithm, and majority of them may produce the expected output for a test input (even if none of these implementations are fault-free). In particular, the proposed approach derives a pseudo oracle for a test input by running the test input on n implementations of the supervised learning algorithm, and then using the common test output produced by a majority (determined by a percentage threshold) of these n implementations. The proposed approach includes techniques to address challenges in multiple-implementation testing (or generally testing) of supervised learning software: the definition of test cases in testing supervised learning software, along with resolution of inconsistent algorithm configurations across implementations. In addition, to improve dependability of supervised learning software during in-field usage while incurring low runtime overhead, The approach includes a multipleimplementation monitoring technique. The evaluations on the proposed approach show that multiple-implementation testing is effective in detecting real faults in real-world ML software (even popularly used ones), including 5 faults from 10 NaiveBayes implementations and 4 faults from 20 k-nearest neighbor implementations, and the proposed technique of multipleimplementation monitoring substantially reduces the need of running mul-

Multiple-implementation testing for XACML implementations.

Automatic XACML Requests Generation for Testing Access Control Policies.

Toward Systematic Testing of Access Control Policies

MUT: Human-in-the-Loop Unit Test Migration

On-line tracing of XACML-based policy coverage criteria

Conformance Checking of Access Control Policies Specified in XACML

EXAMINER-PRO: Testing Arm Emulators Across Different Privileges

Defining and Measuring Policy Coverage in Testing Access Control Policies.

Designing Fast and Scalable XACML Policy Evaluation Engines

Detection of Multiple-Duty-Related Security Leakage in Access Control Policies

Xengine: A Fast And Scalable Xacml Policy Evaluation Engine

A Fault Model and Mutation Testing of Access Control Policies

Automated Test Generation for Access Control Policies Via Change-Impact Analysis

Testing Access Control Policies

Using XACML to Define Access Control Policy in Information System

ACPT: A Tool for Modeling and Verifying Access Control Policies

XUAT-Copilot: Multi-Agent Collaborative System for Automated User Acceptance Testing with Large Language Model

Multiple-implementation testing of supervised learning software by oreoluwa alebiosu

MiTV: multiple-implementation testing of user-input validators for web applications.

Automated Extraction of Security Policies from Natural-Language Software Documents

Research on Multicloud Access Control Policy Integration Framework