GPU Accelerated Barycentric Treecodes and their Application to Kohn-Sham Density Functional Theory

Abstract

Electronic structure calculations are an integral step in the design and engineering of materials. Kohn-Sham Density Functional Theory (KS-DFT) is a computationally tractable first-principles formulation of electronic structure that is widely used to predict material properties. KS-DFT represents the electron density in terms of single-electron wavefunctions by replacing explicit electron-electron interactions with a mean-field interaction and an approximation of an exchange-correlation functional. The development of numerical methods for KS-DFT is an ongoing area of research; more efficient numerical methods will enable the simulation of larger and more challenging materials systems and improve KS-DFT's predictive capability.

The goal of this work was to develop an integral equation based numerical method for KS-DFT, both to investigate the feasibility and to explore any advantages of an integral equation approach compared to the preexisting numerical methods based on differential equations. We achieved this goal through the development of Treecode-Accelerated Green Iteration (TAGI).We used the method of Green's functions to convert the eigenvalue problem for the Kohn-Sham differential operator into a fixed-point problem for an equivalent integral operator. We developed real-space discretization techniques to numerically evaluate the integral operators with high accuracy, including adaptive mesh refinement schemes, a higher order Fejer quadrature rule, and singularity-subtraction schemes to weaken the Green's function singularities. Next, we leveraged fixed-point acceleration techniques to improve the convergence rates of the fixed-point iterations. Finally, we developed treecodes based on barycentric interpolation; these fast summation algorithms reduce the computational complexity of evaluating the discretized integral operators. We developed these barycentric treecodes to run efficiently on high performance computing architectures; in particular, parallelized the computations with a distributed memory Message Passing Interface (MPI) implementation, and accelerated the computations with Graphics Processing Units (GPUs). We achieved high GPU efficiency by leveraging an extra level of parallelism afforded by the barycentric approximations that is not present for previous treecodes based on multipole expansions.

We developed TAGI for two types of calculations; all-electron and Optimized Norm-Conserving Vanderbilt (ONCV) pseudopotential. In all-electron calculations, atomic nuclei are represented by singular Coulomb potentials, and wavefunctions are computed for each electron in the system. The singular nuclear potential, which gives rise to sharply varying wavefunctions near the nuclei, requires substantial mesh refinement to achieve high accuracy. In pseudopotential calculations, the singular nuclear potentials are replaced with smooth, non-singular pseudopotentials. These are generated by absorbing an atom's chemically inert core electrons into its nucleus, leaving only the chemically active valence electrons to be computed. These calculations require significantly less local refinement near the nuclei, enabling TAGI calculations of larger systems.

We demonstrated TAGI by computing the ground-state energy of a variety of molecules for all-electron and pseudopotential calculations. We showed TAGI's ability to systematically converge to chemical accuracy through the adaptive mesh refinement schemes. These calculations were performed on up to eight compute nodes containing a total of thirty two GPUs. The techniques developed in this work have enabled TAGI to calculate chemically accurate ground state energies of molecules containing up to a few hundred electrons in several hours (all-electron C6H6 in 4 hours, pseudopotential Si30H40 in 3 hours, and pseudopotential C60 in 8 hours). While TAGI does not outperform the more mature methods based on differential equations, this work constitutes a substantial proof of concept for treecode-accelerated integral equation based methods for KS-DFT and presents numerous opportunities for further improvement.