Abstract:Accurate landslide displacement prediction is crucial for effective early warning systems to mitigate hazards. The importance of historical information varies with time during prediction due to the underlying landslide deformation mechanism. Despite advances in dynamic machine learning models like Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU), they struggle to fully capture and interpret the varying importance of historical information during prediction, resulting in decreased accuracy and limited understanding of landslide physical mechanisms. Additionally, these approaches rely on manual feature selection, which is independent of the learning process and therefore is prone to estimation errors. To address these challenges, we propose LiteTransNet, a Lightweighted Transformer Network tailored for landslide displacement prediction. Built on the revolutionary Transformer model for sequential data, LiteTransNet leverages localized self-attention to selectively focus on relevant timestamps and provides interpretable results through its attention heatmap. Furthermore, LiteTransNet performs end-to-end time series modeling without extensive feature engineering. We validate LiteTransNet on two landslides in China's Three Gorges Reservoir Area and demonstrate its improved accuracy over recurrent baselines utilizing feature selection methods. Notably, the attention heatmap generated by LiteTransNet provides interpretable insights into the model's temporal dependency learning. It uncovers the varying importance of historical information at different timestamps, showcasing how LiteTransNet's attention aligns with specific periods of the external environment that cause significant disruptions in the landslide system. Our findings reveal that these disruptions continuously impact displacement prediction until their effects fade or are replaced by subsequent intense disturbances, which traditional recurrent methods are unable to capture. Further experiment shows that LiteTransNet enhances efficiency through its streamlined architecture and its parallelization capability. Overall, LiteTransNet innovates landslide prediction by providing accuracy, interpretability, and efficiency, enhancing understanding of deformation mechanisms to facilitate effective early warning systems.

A Fast Parametric and Structural Transfer Leaky Integrator Echo State Network for Reservoir Computing

A Physics-Informed Spatial-Temporal Neural Network for Reservoir Simulation and Uncertainty Quantification

Reservoir parameters prediction based on spatially transferred long short-term memory network

Fractional-integer-order echo state network for time series prediction

Hybrid Backpropagation Parallel Reservoir Networks

TE-ESN: Time Encoding Echo State Network for Prediction Based on Irregularly Sampled Time Series Data.

LiteTransNet: an Interpretable Approach for Landslide Displacement Prediction Using Transformer Model with Attention Mechanism

Impact of time-history terms on reservoir dynamics and prediction accuracy in echo state networks

Feature-Based Echo-State Networks: A Step Towards Interpretability and Minimalism in Reservoir Computer

Long-Short Term Echo State Network for Time Series Prediction

Learning the Deterministically Constructed Echo State Networks.

An echo state network with interacting reservoirs for modeling and analysis of nonlinear systems

Multi-Reservoir Echo State Network with Five-Elements Cycle

fastESN: Fast Echo State Network

MP-CRJ: Multi-parallel cycle reservoirs with jumps for industrial multivariate time series predictions

An Integrated Reservoir Predictor Based on Spatiotemporal Information Transformation

LIAF-Net: Leaky Integrate and Analog Fire Network for Lightweight and Efficient Spatiotemporal Information Processing

Sim‐Net: Simulation Net for Solving Seepage Equation Under Unsteady Boundary

Porosity and Permeability Prediction Using a Transformer and Periodic Long Short-Term Network

Deep Echo State Network With Multiple Adaptive Reservoirs for Time Series Prediction

Extreme Learning Machine for Multivariate Reservoir Characterization