Street-level Imagery Analytics and Applications

Fan Zhang,Jan Dirk Wegner,Bisheng Yang,Yu Liu

DOI: https://doi.org/10.1016/j.isprsjprs.2023.04.005

IF: 12.7

2023-01-01

ISPRS Journal of Photogrammetry and Remote Sensing

Abstract:Crop harvesting date is critical information for crop yield prediction, financial and logistic planning of grain market and downstream supply chain. Remote sensing has the potential to map harvesting date at regional scale. However, existing studies generally lack ground truth data, and have not fully utilized spectral and temporal information of satellite data. To address these gaps, we present a new approach named Field Rover to acquire large volumes of binary harvesting status (harvested VS. unharvested) ground truth data at regional scale on a weekly basis, by repeatedly using vehicle-mounted cameras to collect time series images for sampled fields and interpreting them with a deep learning approach. With these vehicle-derived ground truth data, we present a machine learning approach to upscale harvesting status and subsequently estimate harvesting date to each field in a study area based on a new satellite platform Planet SuperDove which provides daily 8-band surface reflectance at 3 m resolution. We acquired >200,000 vehicle images from September to November for two years (2021 and 2022), and the deep learning model was able to generate harvesting status for each image with an accuracy of 0.998, which can be treated as ground truth. From a time series of harvesting status derived from revisiting vehicle images, harvesting dates for >500 fields were obtained by a change detection approach. We then trained a remote sensing classification model using harvesting status ground truth, and applied it to generate a harvesting status map for each Planet SuperDove overpass day. The classification model achieved an accuracy of 0.96 and subsequently accurate harvesting date maps were obtained by a curve fitting approach. We found that the Planet SuperDove harvesting date agreed well with the Field Rover harvesting date ground truth (R2 = 0.84, RMSE ≈ 5.5 days) at the field level in two years. When focusing on 2022 when more Planet SuperDove satellites were launched, the remote sensing of the harvest date achieved an accuracy of R2 = 0.91, and RMSE ≈ 3.3 days. This study demonstrated the efficacy of using repeated vehicle images to acquire time-related agricultural ground truth data, as well as the efficacy of using vehicle-satellite integrative sensing to upscale ground truth data to the regional scale. We envision this new method can be applied to monitor other agricultural management practices and therefore effectively advance the monitoring and modeling of smart farming and sustainable agriculture.