Abstract:Invariance learning methods aim to learn invariant features in the hope that they generalize under distributional shift. Although many tasks are naturally characterized by continuous domains, current invariance learning techniques generally assume categorically indexed domains. For example, auto-scaling in cloud computing often needs a CPU utilization prediction model that generalizes across different times (e.g., time of a day and date of a year), where `time' is a continuous domain index. In this paper, we start by theoretically showing that existing invariance learning methods can fail for continuous domain problems. Specifically, the naive solution of splitting continuous domains into discrete ones ignores the underlying relationship among domains, and therefore potentially leads to suboptimal performance. To address this challenge, we then propose Continuous Invariance Learning (CIL), which extracts invariant features across continuously indexed domains. CIL is a novel adversarial procedure which measures and controls the conditional independence between the labels and continuous domain indices given the extracted features. Our theoretical analysis demonstrates that CIL learns features that satisfy the invariant constraint with infinite samples. Empirical results on both synthetic and real-world datasets (including data collected from production systems) show that CIL consistently outperforms strong baselines among all the tasks.

Heredity and Invariance of Continuous Domains

The Heredity,invariance and Mapping Space of Quasicontinuous Domains

Consistently Semicontinuous Domains

Precontinuous Domains and Stable Mappings

A Full-Invariant Theorem and Some Applications

Continuity of Effect Algebra Operations in the Interval Topology

On a Characterization Theorem for Continuous Posets

The Fixed-point Sets of Scott Continuous Mapping on Consistent Domain

Some Characteristic Theorems on Consistently Connected Continuous Domains

Some Properties on Quasialgebraic Domain

Representations of domains via closure spaces in the quantale-valued setting

Further Results about Weight on Continuous Domains

Completely distributive enriched categories are not always continuous

The Scott Topology on Posets and Continuous Posets

Cartesian Closedness of The Category of Consistent L-domains

Discussion about Bounded Complete Domain

Directed Completions of Consistent FS-domains and Related Categorical Properties

Representations of Domains via CF-approximation Spaces

Topological representations for frame-valued domains via $L$-sobriety

Continuous Invariance Learning

Stable Mapping and Lawson Topology on Scott Domains