Abstract:Data poisoning considers cases when an adversary manipulates the behavior of machine learning algorithms through malicious training data. Existing threat models of data poisoning center around a single metric, the number of poisoned samples. In consequence, if attackers can poison more samples than expected with affordable overhead, as in many practical scenarios, they may be able to render existing defenses ineffective in a short time. To address this issue, we leverage timestamps denoting the birth dates of data, which are often available but neglected in the past. Benefiting from these timestamps, we propose a temporal threat model of data poisoning with two novel metrics, earliness and duration, which respectively measure how long an attack started in advance and how long an attack lasted. Using these metrics, we define the notions of temporal robustness against data poisoning, providing a meaningful sense of protection even with unbounded amounts of poisoned samples when the attacks are temporally bounded. We present a benchmark with an evaluation protocol simulating continuous data collection and periodic deployments of updated models, thus enabling empirical evaluation of temporal robustness. Lastly, we develop and also empirically verify a baseline defense, namely temporal aggregation, offering provable temporal robustness and highlighting the potential of our temporal threat model for data poisoning.

What problem does this paper attempt to address?

The paper primarily focuses on addressing the issue of data poisoning attacks in the field of machine learning, particularly concerning the effectiveness of existing defense methods when faced with unbounded or large quantities of poisoned samples. Specifically, the paper makes the following contributions: 1. **Proposes a new threat model**: Existing data poisoning threat models mainly revolve around the number of poisoned samples. However, in real-world scenarios, attackers may inject a large number of malicious samples at a low cost, rendering defenses designed based on a limited number of poisoned samples ineffective. Therefore, this paper introduces the time dimension as a new consideration factor. By using the "birth date" of samples (i.e., the timestamp when the sample first becomes available), it defines two new metrics—"Earliness" and "Duration"—to evaluate attack behaviors. 2. **Defines temporal robustness**: Based on the two newly proposed metrics, the authors define the concept of temporal robustness against data poisoning attacks. This robustness measures the effectiveness of defense mechanisms even when faced with an unbounded number of malicious samples, as long as the attack behavior is temporally constrained. 3. **Constructs a benchmark testing framework**: To validate the proposed theories and methods, the paper designs a benchmark test set and evaluation protocol that simulates a continuous data collection process and periodic model updates. This allows for empirical evaluation of temporal robustness. 4. **Develops a baseline defense method**: The paper also proposes a simple yet effective defense strategy called "Temporal Aggregation" and demonstrates its verifiable temporal robustness. This method uses majority voting, combining models trained over multiple time periods to make predictions, thereby reducing the impact of malicious samples. Through these efforts, the paper aims to provide a new perspective and solution for the field of data poisoning defense, especially in scenarios where meaningful protection is still required despite the presence of an unbounded number of malicious samples.

Temporal Robustness against Data Poisoning

Oblivion: Poisoning Federated Learning by Inducing Catastrophic Forgetting.

Pick your Poison: Undetectability versus Robustness in Data Poisoning Attacks

Lethal Dose Conjecture on Data Poisoning

Autoregressive Perturbations for Data Poisoning

Stronger Data Poisoning Attacks Break Data Sanitization Defenses

On the Adversarial Risk of Test Time Adaptation: An Investigation into Realistic Test-Time Data Poisoning

Accumulative Poisoning Attacks on Real-time Data

DP2Dataset Protection by Data Poisoning

Exploring the Limits of Model-Targeted Indiscriminate Data Poisoning Attacks

Exploring Model Dynamics for Accumulative Poisoning Discovery

Data Poisoning Attacks on Regression Learning and Corresponding Defenses

Certified Defenses for Data Poisoning Attacks

Indiscriminate Data Poisoning Attacks on Neural Networks

Certified Robustness to Data Poisoning in Gradient-Based Training

Defending Against Adversarial Denial-of-Service Data Poisoning Attacks

Strong Data Augmentation Sanitizes Poisoning and Backdoor Attacks Without an Accuracy Tradeoff

With Great Dispersion Comes Greater Resilience: Efficient Poisoning Attacks and Defenses for Linear Regression Models

Test-Time Poisoning Attacks Against Test-Time Adaptation Models

De-Pois: An Attack-Agnostic Defense against Data Poisoning Attacks

Amplifying Membership Exposure via Data Poisoning