Abstract:Managers often believe that collecting more data will continually improve the accuracy of their machine learning models. However, we argue in this paper that when data lose relevance over time, it may be optimal to collect a limited amount of recent data instead of keeping around an infinite supply of older (less relevant) data. In addition, we argue that increasing the stock of data by including older datasets may, in fact, damage the model's accuracy. Expectedly, the model's accuracy improves by increasing the flow of data (defined as data collection rate); however, it requires other tradeoffs in terms of refreshing or retraining machine learning models more frequently. Using these results, we investigate how the business value created by machine learning models scales with data and when the stock of data establishes a sustainable competitive advantage. We argue that data's time-dependency weakens the barrier to entry that the stock of data creates. As a result, a competing firm equipped with a limited (yet sufficient) amount of recent data can develop more accurate models. This result, coupled with the fact that older datasets may deteriorate models' accuracy, suggests that created business value doesn't scale with the stock of available data unless the firm offloads less relevant data from its data repository. Consequently, a firm's growth policy should incorporate a balance between the stock of historical data and the flow of new data. We complement our theoretical results with an experiment. In the experiment, we empirically measure the loss in the accuracy of a next word prediction model trained on datasets from various time periods. Our empirical measurements confirm the economic significance of the value decline over time. For example, 100MB of text data, after seven years, becomes as valuable as 50MB of current data for the next word prediction task.

Limitations of Validity Intervals in Data Freshness Management

Time Series Data Validity.

The Case of Transparent Cache Invalidation in Web Applications

WiFresh: Age-of-Information from Theory to Implementation

Managing Periodically Updated Data in Relational Databases: A Stochastic Modeling Approach

Age-Delay Tradeoffs in Single Server Systems

Optimal Scheduling for Maximizing Information Freshness & System Performance in Industrial Cyber-Physical Systems

Minimizing Age of Processed Information in Wireless Networks

Statistical Age of Information: A Risk-Aware Metric and Its Applications in Status Updates

Auditing Interval-Based Inference

Fresh Multiple Access: A Unified Framework Based on Large Models and Mean-Field Approximations

Timely Communications for Remote Inference

A Spatiotemporal Framework for Information Freshness in IoT Uplink Networks

On the Analysis of Cache Invalidation With LRU Replacement

Time and the Value of Data

Optimizing Data Freshness in Time-Varying Wireless Networks with Imperfect Channel State

A Trade-Off Analysis between Sensor Quality and Data Intervals for Prognostics Performance

Optimizing Information Freshness over a Channel that Wears Out

Balancing Data Freshness and Distortion in Real-time Status Updating with Lossy Compression

The distribution of age-of-information performance measures for message processing systems

Real-Time Adaptive Abstraction and Approximation Using Validity Frames -- an Experience Report