Abstract:The Gyrokinetic Toroidal Code at Princeton (GTC-P) is a highly scalable and portable particle-in-cell (PIC) code. It solves the 5D Vlasov-Poisson equation featuring efficient utilization of modern parallel computer architectures at the petascale and beyond. Motivated by the goal of developing a modern code capable of dealing with the physics challenge of increasing problem size with sufficient resolution, new thread-level optimizations have been introduced as well as a key additional domain decomposition. GTC-P's multiple levels of parallelism, including inter-node 2D domain decomposition and particle decomposition, as well as intra-node shared memory partition and vectorization have enabled pushing the scalability of the PIC method to extreme computational scales. In this paper, we describe the methods developed to build a highly parallelized PIC code across a broad range of supercomputer designs. This particularly includes implementations on heterogeneous systems using NVIDIA GPU accelerators and Intel Xeon Phi (MIC) co-processors and performance comparisons with state-of-the-art homogeneous HPC systems such as Blue Gene/Q. New discovery science capabilities in the magnetic fusion energy application domain are enabled, including investigations of Ion-Temperature-Gradient (ITG) driven turbulence simulations with unprecedented spatial resolution and long temporal duration. Performance studies with realistic fusion experimental parameters are carried out on multiple supercomputing systems spanning a wide range of cache capacities, cache-sharing configurations, memory bandwidth, interconnects and network topologies. These performance comparisons using a realistic discovery-science-capable domain application code provide valuable insights on optimization techniques across one of the broadest sets of current high-end computing platforms worldwide.

TWANG-PIC, a novel gyro-averaged one-dimensional particle-in-cell code for interpretation of gyrotron experiments

Gyrokinetic Particle Simulation of Microturbulence for General Magnetic Geometry and Experimental Profiles

A Computational Efficient Simulation Tool for Gyrotrons

Particle-In-Cell Simulation Of A Gyrotron Traveling Wave Amplifier

Particle in cell simulations of a ka-band helical waveguide gyrotron traveling wave tube amplifier

Gyro-average Method for Global Gyrokinetic Particle Simulation in Realistic Tokamak Geometry

Particle Simulation of a Ka-Band Gyrotron Traveling Wave Amplifier

Theoretical Analysis and PIC Simulation of a 140-Ghz Double Confocal Waveguide Gyro-TWA

Development of A 2.5-Dimensional Particle-In-Cell Code for Efficient High-Power Klystron Design

Explicit Structure-Preserving Geometric Particle-in-Cell Algorithm in Curvilinear Orthogonal Coordinate Systems and Its Applications to Whole-Device 6D Kinetic Simulations of Tokamak Physics

PIC Simulation of a Conventional W-band Gyrotron Oscillator

A Two-Dimension Beam Dynamics Simulation Code Base On Pic Method

Modern Gyrokinetic Particle-In-Cell Simulation of Fusion Plasmas on Top Supercomputers

Experiment and Simulation of a W-Band CW 30 kW Low-Voltage Conventional Gyrotron

Modeling betatron radiation using particle-in-cell codes for plasma wakefield accelerator diagnostics

Time-Dependent Nonlinear Theory and Numerical Simulation of 94 Ghz Complex Cavity Gyrotron

Detailed Investigation on Nonstationary Behavior in a Frequency-Tunable Gyrotron

Implementation of a Mesh refinement algorithm into the quasi-static PIC code QuickPIC

QuickPIC: a highly efficient fully parallelized PIC code for plasma-based acceleration

CBETor: a hybrid-kinetic particle-in-cell code for cross-beam energy transfer simulation

Field Theory and a Structure-Preserving Geometric Particle-in-cell Algorithm for Drift Wave Instability and Turbulence