Abstract:Classical palaeoenvironmental reconstruction models often incorporate biological ideas and commonly assume that the taxa comprising a fossil assemblage exhibit unimodal response functions of the environmental variable of interest. In contrast, machine-learning approaches do not rely upon any biological assumptions but instead need training with large data sets to extract some understanding of the relationships between biological assemblages and their environment. To explore the relative merits of these two approaches, we have developed a two-layered machine-learning reconstruction model MEMLM (Multi Ensemble Machine Learning Model). The first layer applies three different ensemble machine-learning models (random forests, extra random trees, and LightGBM), trained on the modern taxon assemblage and associated environmental data to make reconstructions based on the three different models, while the second layer uses multiple linear regression to integrate these three reconstructions into a consensus reconstruction. We considered three versions of the model: (1) a standard version of MEMLM, which uses only taxon abundance data; (2) MEMLMe, which uses only dimensionally reduced assemblage information, using a natural language-processing model (GloVe), to detect associations between taxa across the training data set; and (3) MEMLMc which incorporates both raw taxon abundance and dimensionally reduced summary (GloVe) data. We trained these MEMLM model variants with three high-quality diatom and pollen training sets and compared their reconstruction performance with three weighted-averaging (WA) approaches (WA-Cla for classical deshrinking, WA-Inv for inverse deshrinking, and WA-PLS for partial least squares). In general, the MEMLM approaches, even when trained on only dimensionally reduced assemblage data, performed substantially better than the WA approaches in the larger training sets, as judged by cross-validatory prediction error. When applied to fossil data, MEMLM variants sometimes generated qualitatively different palaeoenvironmental reconstructions from each other and from reconstructions based on WA approaches. We applied a statistical significance test to all the reconstructions. This successfully identified each incidence for which the reconstruction is not robust with respect to the model choice. We found that machine-learning approaches could outperform classical approaches but could sometimes fail badly in the reconstruction, despite showing high performance under cross-validation, likely indicating problems when extrapolation occurs. We found that the classical approaches are generally more robust, although they could also generate reconstructions which have modest statistical significance and therefore may be unreliable. Given these conclusions, we consider that cross-validation is not a sufficient measure of transfer function performance, and we recommend that the results of statistical significance tests are provided alongside the downcore reconstructions based on fossil assemblages.

Testing peatland testate amoeba transfer functions: Appropriate methods for clustered training-sets

Achieving Robust and Transferable Performance for Conservation‐Based Models of Dynamical Physical Systems

Taxon selection using statistical learning techniques to improve transfer function prediction

Improved Data Splitting Methods for Data-Driven Hydrological Model Development Based on a Large Number of Catchment Samples

Can machine-learning algorithms improve upon classical palaeoenvironmental reconstruction models?

Testing for Equality of Rates of Evolution

Functional traits data for testate amoebae of Northern Holarctic realm

Development of pedotransfer functions for water retention in tropical mountain soil landscapes: spotlight on parameter tuning in machine learning

Metamorphic testing of machine learning and conceptual hydrologic models

Development of pedotransfer functions for tropical mountain soilscapes: Spotlight on parameter tuning in machine learning

Evaluating the Performance of Probabilistic Algorithms for Phylogenetic Analysis of Big Morphological Datasets: A Simulation Study

Testing for heterogeneous rates of discrete character evolution on phylogenies

Assessing the Adequacy of Morphological Models using Posterior Predictive Simulations

Cross-validation strategies for data with temporal, spatial, hierarchical, or phylogenetic structure

mbtransfer: Microbiome intervention analysis using transfer functions and mirror statistics

The Transferability of Hydrological Models under Nonstationary Climatic Conditions

Phylogenetic ANOVA: Group‐clade aggregation, biological challenges, and a refined permutation procedure

A Test of Species Distribution Model Transferability Across Environmental and Geographic Space for 108 Western North American Tree Species

Transferability of ecological forecasting models to novel biotic conditions in a long-term experimental study

A tale of too many trees: a conundrum for phylogenetic regression

Assessing Site Effects and Geographic Transferability when Interpolating Point Referenced Spatial Data: A Digital Soil Mapping Case Study