Abstract:Much biodiversity data is collected worldwide, but it remains challenging to assemble the scattered knowledge for assessing biodiversity status and trends. The concept of Essential Biodiversity Variables (EBVs) was introduced to structure biodiversity monitoring globally, and to harmonize and standardize biodiversity data from disparate sources to capture a minimum set of critical variables required to study, report and manage biodiversity change. Here, we assess the challenges of a 'Big Data' approach to building global EBV data products across taxa and spatiotemporal scales, focusing on species distribution and abundance. The majority of currently available data on species distributions derives from incidentally reported observations or from surveys where presence-only or presence-absence data are sampled repeatedly with standardized protocols. Most abundance data come from opportunistic population counts or from population time series using standardized protocols (e.g. repeated surveys of the same population from single or multiple sites). Enormous complexity exists in integrating these heterogeneous, multi-source data sets across space, time, taxa and different sampling methods. Integration of such data into global EBV data products requires correcting biases introduced by imperfect detection and varying sampling effort, dealing with different spatial resolution and extents, harmonizing measurement units from different data sources or sampling methods, applying statistical tools and models for spatial inter- or extrapolation, and quantifying sources of uncertainty and errors in data and models. To support the development of EBVs by the Group on Earth Observations Biodiversity Observation Network (GEO BON), we identify 11 key workflow steps that will operationalize the process of building EBV data products within and across research infrastructures worldwide. These workflow steps take multiple sequential activities into account, including identification and aggregation of various raw data sources, data quality control, taxonomic name matching and statistical modelling of integrated data. We illustrate these steps with concrete examples from existing citizen science and professional monitoring projects, including eBird, the Tropical Ecology Assessment and Monitoring network, the Living Planet Index and the Baltic Sea zooplankton monitoring. The identified workflow steps are applicable to both terrestrial and aquatic systems and a broad range of spatial, temporal and taxonomic scales. They depend on clear, findable and accessible metadata, and we provide an overview of current data and metadata standards. Several challenges remain to be solved for building global EBV data products: (i) developing tools and models for combining heterogeneous, multi-source data sets and filling data gaps in geographic, temporal and taxonomic coverage, (ii) integrating emerging methods and technologies for data collection such as citizen science, sensor networks, DNA-based techniques and satellite remote sensing, (iii) solving major technical issues related to data product structure, data storage, execution of workflows and the production process/cycle as well as approaching technical interoperability among research infrastructures, (iv) allowing semantic interoperability by developing and adopting standards and tools for capturing consistent data and metadata, and (v) ensuring legal interoperability by endorsing open data or data that are free from restrictions on use, modification and sharing. Addressing these challenges is critical for biodiversity research and for assessing progress towards conservation policy targets and sustainable development goals.

Standards recommendations for the Earth BioGenome Project

Earth BioGenome Project: Sequencing life for the future of life

Ethical, legal, and social issues in the Earth BioGenome Project

Scalable, accessible and reproducible reference genome assembly and evaluation in Galaxy

Indigenous peoples and local communities as partners in the sequencing of global eukaryotic biodiversity

The Earth BioGenome Project 2020: Starting the clock

Updates to the Alliance of Genome Resources Central Infrastructure

Updates to the Alliance of Genome Resources Central Infrastructure Alliance of Genome Resources Consortium

Minimum information about a single amplified genome (MISAG) and a metagenome-assembled genome (MIMAG) of bacteria and archaea

The European Reference Genome Atlas: piloting a decentralised approach to equitable biodiversity genomics

The changing face of genome assemblies: Guidance on achieving high‐quality reference genomes

Metadata harmonization-Standards are the key for a better usage of omics data for integrative microbiome analysis

Standard Guidelines for the Chromosome-Centric Human Proteome Project

Recommendations for extending the GFF3 specification for improved interoperability of genomic data

Building essential biodiversity variables (EBVs) of species distribution and abundance at a global scale

An International Effort Towards Developing Standards for Best Practices in Analysis, Interpretation and Reporting of Clinical Genome Sequencing Results in the CLARITY Challenge

EM and XRM Connectomics Imaging and Experimental Metadata Standards

A Global View of Standards for Open Image Data Formats and Repositories

A decade of GigaScience: 10 years of the evolving genomic and biomedical standards landscape

Integrated Bio-Search: challenges and trends for the integration, search and comprehensive processing of biological information

Towards complete and error-free genome assemblies of all vertebrate species