Abstract:High spatial resolution (1–5 m) remotely sensed datasets are increasingly being used to map land covers over large geographic areas using supervised machine learning algorithms. Although many studies have compared machine learning classification methods, sample selection methods for acquiring training and validation data for machine learning, and cross-validation techniques for tuning classifier parameters are rarely investigated, particularly on large, high spatial resolution datasets. This work, therefore, examines four sample selection methods—simple random, proportional stratified random, disproportional stratified random, and deliberative sampling—as well as three cross-validation tuning approaches—k-fold, leave-one-out, and Monte Carlo methods. In addition, the effect on the accuracy of localizing sample selections to a small geographic subset of the entire area, an approach that is sometimes used to reduce costs associated with training data collection, is investigated. These methods are investigated in the context of support vector machines (SVM) classification and geographic object-based image analysis (GEOBIA), using high spatial resolution National Agricultural Imagery Program (NAIP) orthoimagery and LIDAR-derived rasters, covering a 2,609 km2 regional-scale area in northeastern West Virginia, USA. Stratified-statistical-based sampling methods were found to generate the highest classification accuracy. Using a small number of training samples collected from only a subset of the study area provided a similar level of overall accuracy to a sample of equivalent size collected in a dispersed manner across the entire regional-scale dataset. There were minimal differences in accuracy for the different cross-validation tuning methods. The processing time for Monte Carlo and leave-one-out cross-validation were high, especially with large training sets. For this reason, k-fold cross-validation appears to be a good choice. Classifications trained with samples collected deliberately (i.e., not randomly) were less accurate than classifiers trained from statistical-based samples. This may be due to the high positive spatial autocorrelation in the deliberative training set. Thus, if possible, samples for training should be selected randomly; deliberative samples should be avoided.

SVDD-based one-class land-cover mapping using optimal training samples

Multiclass SVM based land cover classification with multisource data

Sparse and Stratified Sampling Method for One - Class Classification of Remote Sensing Images

Land-Cover Change Detection Using One-Class Support Vector Machine

Iterative Training Sample Expansion to Increase and Balance the Accuracy of Land Classification From VHR Imagery

Evaluation of Sampling and Cross-Validation Tuning Strategies for Regional-Scale Machine Learning Classification

On the Sampling Strategies for Evaluation of Joint Spectral-Spatial Information Based Classifiers

Landsat Tm Multi-Spectral Classification Using Support Vector Machine Method in Low-Hill Areas

Enhancing the performance of regional land cover mapping

Multiclass Approaches for Support Vector Machine Based Land Cover Classification

Boundary‐based Fuzzy‐SVDD for one‐class classification

Mapping Rice Planting Area from Landsat 8 Imagery Using One-Class Support Vector Machine

Integrating Multi-Temporal Spectral and Structural Information from ALOS/AVNIR-2 Images to Map Heterogeneous Land Use/Cover: A Hybrid Approach

Classification strategy for remote sensing data based on positive training samples

Dense v.s. Sparse: A Comparative Study of Sampling Analysis in Scene Classification of High-Resolution Remote Sensing Imagery

High Resolution Remotely Sensed Imagery Classification For Urban Environment Monitoring Based On Support Vector Machine

A novel optimization parameters of support vector machines model for the land use/ cover classification

A Method of Land Use/land Cover Change Detection from Remote Sensing Image Based on Support Vector Machines

Consistency Analysis Of Multi-Source Remotely Sensed Images For Land Cover Classification

Mapping of Land Cover Over Highly Heterogeneous Areas in Yunnan Province With Active and Passive Remotely Sensed Data

A new strategy based on multi-source remote sensing data for improving the accuracy of land use/cover change classification