Abstract:Various galaxy merger detection methods have been applied to diverse datasets. However, it is difficult to understand how they compare. Our aim is to benchmark the relative performance of merger detection methods based on machine learning (ML). We explore six leading ML methods using three main datasets. The first dataset consists of mock observations from the IllustrisTNG simulations, which acts as the training data and allows us to quantify the performance metrics of the detection methods. The second dataset consists of mock observations from the Horizon-AGN simulations, introduced to evaluate the performance of classifiers trained on different, but comparable data to those employed for training. The third dataset is composed of real observations from the Hyper Suprime-Cam Subaru Strategic Program (HSC-SSP) survey. We also compare mergers and non-mergers detected by the different methods with a subset of HSC-SSP visually identified galaxies. For the simplest binary classification task (i.e. mergers vs. non-mergers), all six methods perform reasonably well in the domain of the training data. At the lowest redshift explored $0.1<z<0.3$, precision and recall generally range between sim 70 and 80 , both of which decrease with increasing $z$ as expected (by sim 5 for precision and sim 10 for recall at the highest $z$ explored $0.76<z<1.0$). When transferred to a different domain, the precision of all classifiers is only slightly reduced, but the recall is significantly worse (by sim 20-40 depending on the method). Zoobot offers the best overall performance in terms of precision and F1 score. When applied to real HSC observations, different methods agree well with visual labels of clear mergers, but can differ by more than an order of magnitude in predicting the overall fraction of major mergers. For the more challenging multi-class classification task to distinguish between pre-mergers, ongoing-mergers, and post-mergers, none of the methods in their current set-ups offer good performance, which could be partly due to the limitations in resolution and the depth of the data. In particular, ongoing-mergers and post-mergers are much more difficult to classify than pre-mergers. With the advent of better quality data (e.g. from JWST and Euclid), it is of great importance to improve our ability to detect mergers and distinguish between merger stages.

The Galaxy Cluster Merger Catalog: An Online Repository of Mock Observations from Simulated Galaxy Cluster Mergers

The GalMer database: galaxy mergers in the virtual observatory

Simulated Cluster Archive: A Computational Catalog of X-Ray Clusters in a Lambda-CDM Universe

The Three Hundred Project: a Large Catalogue of Theoretically Modelled Galaxy Clusters for Cosmological and Astrophysical Applications

Merging Cluster Collaboration: Optical and Spectroscopic Survey of a Radio-selected Sample of 29 Merging Galaxy Clusters

The Theoretical Astrophysical Observatory: Cloud-Based Mock Galaxy Catalogues

Harnessing the Hubble Space Telescope Archives: A Catalogue of 21,926 Interacting Galaxies

A catalogue of merging clusters of galaxies: cluster partners, merging subclusters, and post-collision clusters

Constructing a Galaxy Cluster Catalog in IllustrisTNG-300 using the Mulguisin Algorithm

Decoding the Early Universe: Exploring a Merger Scenario for the High-Redshift Cluster JKCS041 using Numerical Models

The Observational Consequences of Merging Clusters of Galaxies

Galaxy merger challenge: A comparison study between machine learning-based detection methods

The evolution of galaxy morphology from redshift z=6 to 3: Mock JWST observations of galaxies in the ASTRID simulation

Constructing a Galaxy Cluster Catalog in IllustrisTNG300 Using the Mulguisin Algorithm

Catalogue of model star clusters in the Milky Way and M31 galaxies

Mock Lightcones and Theory Friendly Catalogs for the CANDELS Survey

Creating Mock Catalogues of Stellar Haloes from Cosmological Simulations

Optical galaxy cluster mock catalogs with realistic projection effects: validations with the SDSS redMaPPer clusters

Merging galaxies in isolated environments I. Multiband photometry, classification, stellar masses, and star formation rates