Abstract:The continual evolution of photon sources and high-performance detectors drives cutting-edge experiments that can produce very high throughput data streams and generate large data volumes that are challenging to manage and store. In these cases, efficient data transfer and processing architectures that allow online image correction, data reduction or compression become fundamental. This work investigates different technical options and methods for data placement from the detector head to the processing computing infrastructure, taking into account the particularities of modern modular high-performance detectors. In order to compare realistic figures, the future ESRF beamline dedicated to macromolecular X-ray crystallography, EBSL8, is taken as an example, which will use a PSI JUNGFRAU 4M detector generating up to 16 GB of data per second, operating continuously during several minutes. Although such an experiment seems possible at the target speed with the 100 Gb s-1 network cards that are currently available, the simulations generated highlight some potential bottlenecks when using a traditional software stack. An evaluation of solutions is presented that implements remote direct memory access (RDMA) over converged ethernet techniques. A synchronization mechanism is proposed between a RDMA network interface card (RNIC) and a graphics processing unit (GPU) accelerator in charge of the online data processing. The placement of the detector images onto the GPU is made to overlap with the computation carried out, potentially hiding the transfer latencies. As a proof of concept, a detector simulator and a backend GPU receiver with a rejection and compression algorithm suitable for a synchrotron serial crystallography (SSX) experiment are developed. It is concluded that the available transfer throughput from the RNIC to the GPU accelerator is at present the major bottleneck in online processing for SSX experiments.

A Generic Data Acquisition Framework For High Performance 2D X-RAY Detectors

High Resolution X-Ray Medical Sequential Image Acquisition and Processing System Based on PCI Interface

RDMA data transfer and GPU acceleration methods for high-throughput online processing of serial crystallography images

The Spectral X-ray Imaging Data Acquisition (SpeXIDAQ) Framework

A Generic High Bandwidth Data Acquisition Card for Physics Experiments.

Design of a Common High-Performance Data Acquisition System for Physics Experiments at HIAF

Detector integration at HEPS: a systematic, efficient and high-performance approach

Real-time Data Acquisition and Processing System for MHz Repetition Rate Image Sensors

Dynamically Reconfigurable Data Readout of Pixel Detectors for Automatic Synchronization with Data Acquisition Systems

A distributed software system for integrating data-intensive imaging methods in a hard X-ray nanoprobe beamline at the SSRF

Detectors for the future of X-ray imaging

Data reduction and processing for photon science detectors

Real-Time Discrete SPAD Array Readout Architecture for Time of Flight PET

Design and Implementation of Distributed Data Acquisition Architecture in High Energy Physics

X-ray Single-Pixel Imaging with MPGD-based detectors

A Fast Readout Electronic System for Accurate Spatial Detection in Ion Beam Tracking for the Next Generation of Particle Accelerators

A Plugin-Based Software Framework for Data Acquisition and Online Processing

Development and optimization of a backscatter X-ray detection system based on pencil beam scanning

A Novel Hybrid Architecture for Ultra-Fast and High-Resolution Data Acquisition

High Dynamic Range X-ray Detector Pixel Architectures Utilizing Charge Removal

High Performance Data Acquisition System for IRFPA Testing