Abstract:Model calibration and validation are critical in hydrological model robustness assessment. Unfortunately, the commonly used split‐sample test (SST) framework for data splitting requires modelers to make subjective decisions without clear guidelines. This large‐sample SST assessment study empirically assesses how different data splitting methods influence post‐validation model testing period performance, thereby identifying optimal data splitting methods under different conditions. This study investigates the performance of two lumped conceptual hydrological models calibrated and tested in 463 catchments across the United States using 50 different data splitting schemes. These schemes are established regarding the data availability, length and data recentness of continuous calibration sub‐periods (CSPs). A full‐period CSP is also included in the experiment, which skips model validation. The assessment approach is novel in multiple ways including how model building decisions are framed as a decision tree problem and viewing the model building process as a formal testing period classification problem, aiming to accurately predict model success/failure in the testing period. Results span different climate and catchment conditions across a 35‐year period with available data, making conclusions quite generalizable. Calibrating to older data and then validating models on newer data produces inferior model testing period performance in every single analysis conducted and should be avoided. Calibrating to the full available data and skipping model validation entirely is the most robust split‐sample decision. Experimental findings remain consistent no matter how model building factors (i.e., catchments, model types, data availability, and testing periods) are varied. Results strongly support revising the traditional split‐sample approach in hydrological modeling. Hydrological model calibration is a critical model building process that infers key model parameter values from observed system response data. Conventionally, this process requires the historical period to be split into a calibration period for tuning parameters and a validation period for testing model robustness (i.e., the split‐sample). Unfortunately, there is a lack of empirical evidence supporting how exactly to define the split‐sample. We designed an exhaustive and novel experiment comparing the range of possible split‐sampling schemes, including calibrating to older/recent years, calibrating to a short/long period, and calibrating to the full period of available system response data. Each scheme was evaluated based on performance, assessed in three different ways, in numerous post‐validation model testing periods for each of the 926 calibration case studies (two different hydrological models applied in 463 catchments). Results show that using older data for model calibration and then using newer data for validation, which is the typical practice in the literature, is an inferior choice and should be avoided. The results also show that calibrating to the full historical data and skipping model validation entirely is the most robust choice. Therefore, the split‐sample approach applied in this community for decades should be revised. A unique split‐sample experiment is performed across 463 catchments to provide guidance on split sample decision‐making in model calibration Calibrating models to the full available data period and skipping model validation entirely is the most robust choice Calibrating models to older data and then validating models on newer data, a very common approach in literature, is an inferior choice A unique split‐sample experiment is performed across 463 catchments to provide guidance on split sample decision‐making in model calibration Calibrating models to the full available data period and skipping model validation entirely is the most robust choice Calibrating models to older data and then validating models on newer data, a very common approach in literature, is an inferior choice

Improved Data Splitting Methods for Data-Driven Hydrological Model Development Based on a Large Number of Catchment Samples

On Lack of Robustness in Hydrological Model Development Due to Absence of Guidelines for Selecting Calibration and Evaluation Data: Demonstration for Data‐Driven Models

An R Package to Partition Observation Data Used for Model Development and Evaluation to Achieve Model Generalizability

On the Robustness of Conceptual Rainfall-Runoff Models to Calibration and Evaluation Data Set Splits Selection: A Large Sample Investigation

Achieving Robust and Transferable Performance for Conservation‐Based Models of Dynamical Physical Systems

A Robust Strategy to Account for Data Sampling Variability in the Development of Hydrological Models

Statistics for Sample Splitting for the Calibration and Validation of Hydrological Models

Time to Update the Split‐Sample Approach in Hydrological Model Calibration

Development of a Distributed Physics‐Informed Deep Learning Hydrological Model for Data‐Scarce Regions

Development of A Distributed Modeling Framework Considering Spatiotemporally Varying Hydrological Processes for Sub-Daily Flood Forecasting in Semi-Humid and Semi-Arid Watersheds

Assessing Methods to Disaggregate Daily Precipitation for Hydrological Simulation

Utilizing Satellite Surface Soil Moisture Data in Calibrating a Distributed Hydrological Model Applied in Humid Regions Through a Multi-Objective Bayesian Hierarchical Framework.

Influences of the Runoff Partition Method on the Flexible Hybrid Runoff Generation Model for Flood Prediction

Multi-objective parameter optimization of distributed hydrological models based on data-poor watersheds

Development of a Stepwise‐Clustered Multi‐Catchment Hydrological Model for Quantifying Interactions in Regional Climate‐Runoff Relationships

Comparing Statistical And Semi-Distributed Rainfall-Runoff Models For A Large Subtropical Watershed: A Case Study Of Jiulong River Catchment, China

Evaluation of Multi‐source Precipitation Data in a Watershed with Complex Topography Based on Distributed Hydrological Modeling

Can Sampling Techniques Improve the Performance of Decomposition-Based Hydrological Prediction Models? Exploration of Some Comparative Experiments

Data Decomposition, Seasonal Adjustment Method and Machine Learning Combined for Runoff Prediction: A Case Study

A Watershed Rainfall Data Recovery Approach with Application to Distributed Hydrological Models

Physically Based Distributed Hydrological Model Calibration Based on a Short Period of Streamflow Data: Case Studies in Four Chinese Basins