Reproducibility of density functional approximations: how new functionals should be reported

Susi Lehtola,Miguel A. L. Marques

DOI: https://doi.org/10.1063/5.0167763

2023-08-23

Abstract:Density functional theory is the workhorse of chemistry and materials science, and novel density functional approximations (DFAs) are published every year. To become available in program packages, the novel DFAs need to be (re)implemented. However, according to our experience as developers of Libxc [Lehtola et al, SoftwareX 7, 1 (2018)], a constant problem in this task is verification, due to the lack of reliable reference data. As we discuss in this work, this lack has lead to several non-equivalent implementations of functionals such as BP86, PW91, PBE, and B3LYP across various program packages, yielding different total energies. Through careful verification, we have also found many issues with incorrect functional forms in recent DFAs. The goal of this work is to ensure the reproducibility of DFAs: DFAs must be verifiable in order to prevent reappearances of the abovementioned errors and incompatibilities. A common framework for verification and testing is therefore needed. We suggest several ways in which reference energies can be produced with free and open source software, either with non-self-consistent calculations with tabulated atomic densities or via self-consistent calculations with various program packages. The employed numerical parameters -- especially, the quadrature grid -- need to be converged to guarantee a $\lesssim0.1\mu E_{h}$ precision in the total energy, which is nowadays routinely achievable in fully numerical calculations. Moreover, as such sub-$\mu E_{h}$ level agreement can only be achieved when fully equivalent implementations of the DFA are used, also the source code of the reference implementation should be made available in any publication describing a new DFA.

Computational Physics

What problem does this paper attempt to address?

The paper attempts to address the reproducibility issues encountered when implementing Density Functional Approximations (DFAs) in different software packages. Specifically:

- **Validation Issues**: Due to the lack of reliable reference data, the implementation of newly developed DFAs often varies across different packages. For example, functionals like BP86, PW91, PBE, and B3LYP have multiple non-equivalent implementations in different packages, leading to different total energy results.
- **Formal Errors**: Through careful validation, researchers have also found that many recently proposed DFAs contain formal errors.
- **Verifiability**: To prevent the recurrence of the above errors and incompatibilities, ensuring the verifiability of DFAs is crucial. Therefore, a common framework for validation and testing is needed.

The goal of the paper is to ensure the reproducibility of DFAs, meaning that DFAs must be verifiable to prevent the recurrence of the aforementioned errors and incompatibilities. To achieve this goal, the paper proposes several methods to generate reference energies, which can be implemented using freely available open-source software. These methods include using tabulated atomic densities for non-self-consistent calculations or performing self-consistent calculations through various packages. Additionally, to ensure that the total energy accuracy is less than 0.1 microhartree (µEh), the numerical parameters used (especially the integration grid) need to be converged. The paper also emphasizes that the source code of the reference implementation should be provided in any publication describing new DFAs.