SACRA-K: A Performance-Portable Numerical Relativity Code with Kokkos
SACRA-K: A Performance-Portable Numerical Relativity Code with Kokkos
Ming-Zhe Han, Kenta Kiuchi, Masaru Shibata
AbstractWe present SACRA-K (SimulAtor for Compact objects in Relativistic Astrophysics with Kokkos), a performance-portable numerical relativity code ported from the Fortran code SACRA-MPI to C++ with the Kokkos library. SACRA-K retains the physics and numerical methods of SACRA-MPI, namely a Baumgarte-Shapiro-Shibata-Nakamura (BSSN) spacetime evolution with Z4c constraint propagation and a box-in-box Berger-Oliger adaptive mesh refinement, together with the high resolution shock capturing scheme for the general relativistic hydrodynamics from NANASI, while gaining cross vendor portability. We validate the port against SACRA-MPI across binary black hole, black hole neutron star, and binary neutron star configurations. Across these tests, the waveform discrepancies are well below both the scatter among independent numerical relativity codes and the resolution dependent variation within a given code, and remain at or below the distinguishability threshold of current gravitational-wave detectors; the $π$ symmetry is preserved exactly at the bitwise level; and the gravitational wave phase of the binary neutron star merger exhibits second order convergence. In the smallest test configuration, SACRA-K runs about an order of magnitude faster on the NVIDIA A100 GPU cluster or the AMD MI300A APU cluster than the Fortran SACRA-MPI on the CPU cluster, and we measure its scaling up to 256 accelerator devices.