Describe the bug
The current ABACUS–DeepH interface has two independent classes of failure:
- Correctness: the atom-pair support of the ABACUS Hamiltonian is not always a subset of the overlap support. DeepH uses overlap support to construct the inference graph, so valid Hamiltonian edges can be absent from the graph and prediction output.
- Current-version I/O compatibility: current ABACUS develop output cannot be consumed by either public DeepH 0.2.2 or the current documented DeepH 1.0.6 path through DeepH-dock 0.9.12.
Versions checked on 2026-08-28:
1. Reproduced H/S support mismatch
ABACUS creates overlap pairs only for direct orbital overlap:
distance(i,j) < r_phi(i) + r_phi(j)
However, the nonlocal Hamiltonian can couple two orbitals through a third atom's projector:
<phi_i|beta_I> D_I <beta_I|phi_j>
Therefore, S_ij(R) == 0 does not imply H_ij(R) == 0. This follows directly from the different pair construction in overlap.cpp and nonlocal.cpp.
In the four-Si reproducer below, the NAO cutoff is 6 Bohr and the nonlocal-projector cutoff is 1.94 Bohr. Atoms 1 and 3 are 13.52296 Bohr apart, so they have no direct overlap, while each is 7 Bohr from atom 2 and couples through its projector.
At an atom-block threshold of 1e-8, current ABACUS produces:
H atom-block keys: 16
S atom-block keys: 14
H-only [0, 0, 0, 1, 3]: max|H|=5.48772265e-05 Ry, max|S|=0
H-only [0, 0, 0, 3, 1]: max|H|=5.48772265e-05 Ry, max|S|=0
These overlap blocks are structurally absent, not merely filtered numerical noise, so lowering the sparse threshold does not solve the problem.
This is deterministically harmful in the current converter: DeepH-dock first derives atom_pairs from overlap nonzeros, then extracts Hamiltonian blocks only for those overlap-derived pairs (source). Thus H-only pairs are discarded during conversion. Current DeepH documentation also permits only S/Sap graph construction during inference and states that overlap sparsity is indispensable, while training defaults to H and also supports HS (inference, training).
Possible consequences are silently missing predicted Hamiltonian blocks, incorrect bands/eigenvalues, and downstream generalized-eigenproblem failures.
2. Current ABACUS output is not readable by the documented DeepH path
Current ABACUS SCF output is:
running_scf.log
hrs1_nao.csr
hrs2_nao.csr # nspin=2
sr_nao.csr
DeepH-dock 0.9.12 still hard-codes:
running_scf.log
data-HR-sparse_SPIN0.csr
data-HR-sparse_SPIN1.csr
data-SR-sparse_SPIN0.csr
It also parses the legacy CSR header, while ABACUS v3.9.0.25+ writes the rich CSR header with spin metadata, UnitCell data, and labeled CSR sections. The current DeepH-dock documentation explicitly targets ABACUS 3.10 LTS and shows only the legacy layout (documentation).
For overlap-only inference, an actual calculation get_s run on e461ffd produced:
OUT.<suffix>/running_get_s.log
OUT.<suffix>/sr_nao.csr
DeepH-dock and public DeepH 0.2.2 expect case-sensitive running_get_S.log and SR.csr. The latest DeepH prediction guide also still says that ABACUS dumps SR.csr (documentation).
Public DeepH 0.2.2 has additional incompatibilities: it searches case-sensitively for lattice constant (Bohr) while current ABACUS prints Lattice constant (Bohr), rejects nspin=2, and applies a separate 1e-8 atom-block filter (parser).
This conflicts with the current ABACUS documentation claim that “DeepH v1.0.0+” reads the v3.9.0.25+ CSR format (ABACUS documentation). I could not find a current released converter that implements that contract.
3. Other interface limitations that should be part of the supported-version contract
- DeepH-dock cannot distinguish two ABACUS species of the same element that use different basis/pseudopotential settings; it requires standard element-symbol labels.
- Public DeepH 0.2.2 does not support collinear nspin=2. DeepH-dock added support for the legacy two-file layout, but current hrs1_nao.csr/hrs2_nao.csr cannot reach that path.
- Current ABACUS native-binary and NPZ out_hsr modes have no documented DeepH conversion path; only text CSR should be advertised as supported.
- Public DeepH 0.2.2 and the new JAX deepx-1.0.6 are separate implementations. The compatibility documentation should name exact ABACUS, DeepH-dock, and DeepH versions.
Expected behavior
- The graph used for training and inference must contain every Hamiltonian target/prediction edge:
H_target_keys <= graph_keys
- Current ABACUS SCF and get_s output should be readable by the converter advertised in the ABACUS documentation.
- The two projects should publish one exact compatibility matrix covering ABACUS version/commit, CSR format, DeepH-dock version, DeepH version, and nspin=1/2/4.
A robust fix would export or construct a dedicated graph-support/atom-pair artifact containing the union of direct-overlap and nonlocal-projector-mediated Hamiltonian pairs. This should be independent of numerical overlap values; changing the physical overlap matrix or only lowering a threshold is not sufficient.
To Reproduce
- Build ABACUS develop at e461ffd4246e6be41189d35a242bb44c4c50eca9 with LCAO, MPI, and OpenMP enabled.
- Create a case directory and copy these files from the ABACUS repository:
cp ../../tests/PP_ORB/Si_dojo_nsoc.upf .
cp ../../tests/PP_ORB/Si_dojo_6au_sz.orb .
- Use the following input files.
INPUT:
INPUT_PARAMETERS
suffix deeph_bridge4
calculation scf
nbands 10
symmetry 0
pseudo_dir ./
orbital_dir ./
gamma_only 0
ecutwfc 20
scf_thr 1e-8
scf_nmax 100
basis_type lcao
smearing_method gauss
smearing_sigma 0.002
mixing_type broyden
mixing_beta 0.7
mixing_gg0 0.0
out_hsr 1 8
ks_solver scalapack_gvx
STRU:
ATOMIC_SPECIES
Si 14 Si_dojo_nsoc.upf upf201
NUMERICAL_ORBITAL
Si_dojo_6au_sz.orb
LATTICE_CONSTANT
1.0
LATTICE_VECTORS
40.0 0.0 0.0
0.0 40.0 0.0
0.0 0.0 40.0
ATOMIC_POSITIONS
Direct
Si
0.0
4
0.325000000000 0.500000000000 0.500000000000 1 1 1
0.500000000000 0.500000000000 0.500000000000 1 1 1
0.651554445663 0.587500000000 0.500000000000 1 1 1
0.500000000000 0.500000000000 0.675000000000 1 1 1
KPT:
K_POINTS
0
Gamma
1 1 1 0 0 0
- Run with one OpenMP thread:
OMP_NUM_THREADS=1 /path/to/abacus
- Compare 4-by-4 atom blocks in hrs1_nao.csr and sr_nao.csr at 1e-8. The two H-only pairs and values shown above are obtained.
- Change the INPUT to calculation get_s and run again. The actual output names are running_get_s.log and sr_nao.csr, which the current DeepH-dock ABACUS validator does not recognize.
Environment
- OS: Ubuntu 22.04
- Compiler: GCC 11.4.0 through Open MPI mpicxx
- MPI: Open MPI 4.1.2
- Linear algebra: OpenBLAS, LAPACK, ScaLAPACK
- ABACUS build: MPI ON, OpenMP ON, LCAO ON, ELPA ON
- Runtime: CPU, one MPI rank, OMP_NUM_THREADS=1
The SCF converged in 84 iterations. The get_s reproduction completes without SCF in about 0.08 seconds.
Additional Context
The old warning in the public DeepH repository already states that the DeepH–ABACUS interface has an overlap sparsity-pattern bug (warning). The reproducer above identifies a concrete physical/source-code cause and shows that the warning is still applicable to the latest documented S/Sap inference workflow.
Related issue #3691 is not a duplicate. That issue concerned a spin/SOC dimension mismatch (50-by-50 H versus 25-by-25 S) and was resolved by setting the converted data as spinful. This report concerns missing atom-pair keys even when H and S have the same orbital dimension.
Suggested regression coverage:
- Assert H_target_keys <= graph_keys for the four-Si projector bridge case.
- End-to-end SCF and get_s conversion tests for nspin=1, nspin=2, and nspin=4.
- Pin exact ABACUS, DeepH-dock, and DeepH versions in CI and documentation.
Describe the bug
The current ABACUS–DeepH interface has two independent classes of failure:
Versions checked on 2026-08-28:
1. Reproduced H/S support mismatch
ABACUS creates overlap pairs only for direct orbital overlap:
However, the nonlocal Hamiltonian can couple two orbitals through a third atom's projector:
Therefore, S_ij(R) == 0 does not imply H_ij(R) == 0. This follows directly from the different pair construction in overlap.cpp and nonlocal.cpp.
In the four-Si reproducer below, the NAO cutoff is 6 Bohr and the nonlocal-projector cutoff is 1.94 Bohr. Atoms 1 and 3 are 13.52296 Bohr apart, so they have no direct overlap, while each is 7 Bohr from atom 2 and couples through its projector.
At an atom-block threshold of 1e-8, current ABACUS produces:
These overlap blocks are structurally absent, not merely filtered numerical noise, so lowering the sparse threshold does not solve the problem.
This is deterministically harmful in the current converter: DeepH-dock first derives atom_pairs from overlap nonzeros, then extracts Hamiltonian blocks only for those overlap-derived pairs (source). Thus H-only pairs are discarded during conversion. Current DeepH documentation also permits only S/Sap graph construction during inference and states that overlap sparsity is indispensable, while training defaults to H and also supports HS (inference, training).
Possible consequences are silently missing predicted Hamiltonian blocks, incorrect bands/eigenvalues, and downstream generalized-eigenproblem failures.
2. Current ABACUS output is not readable by the documented DeepH path
Current ABACUS SCF output is:
DeepH-dock 0.9.12 still hard-codes:
It also parses the legacy CSR header, while ABACUS v3.9.0.25+ writes the rich CSR header with spin metadata, UnitCell data, and labeled CSR sections. The current DeepH-dock documentation explicitly targets ABACUS 3.10 LTS and shows only the legacy layout (documentation).
For overlap-only inference, an actual calculation get_s run on e461ffd produced:
DeepH-dock and public DeepH 0.2.2 expect case-sensitive running_get_S.log and SR.csr. The latest DeepH prediction guide also still says that ABACUS dumps SR.csr (documentation).
Public DeepH 0.2.2 has additional incompatibilities: it searches case-sensitively for lattice constant (Bohr) while current ABACUS prints Lattice constant (Bohr), rejects nspin=2, and applies a separate 1e-8 atom-block filter (parser).
This conflicts with the current ABACUS documentation claim that “DeepH v1.0.0+” reads the v3.9.0.25+ CSR format (ABACUS documentation). I could not find a current released converter that implements that contract.
3. Other interface limitations that should be part of the supported-version contract
Expected behavior
A robust fix would export or construct a dedicated graph-support/atom-pair artifact containing the union of direct-overlap and nonlocal-projector-mediated Hamiltonian pairs. This should be independent of numerical overlap values; changing the physical overlap matrix or only lowering a threshold is not sufficient.
To Reproduce
INPUT:
STRU:
KPT:
Environment
The SCF converged in 84 iterations. The get_s reproduction completes without SCF in about 0.08 seconds.
Additional Context
The old warning in the public DeepH repository already states that the DeepH–ABACUS interface has an overlap sparsity-pattern bug (warning). The reproducer above identifies a concrete physical/source-code cause and shows that the warning is still applicable to the latest documented S/Sap inference workflow.
Related issue #3691 is not a duplicate. That issue concerned a spin/SOC dimension mismatch (50-by-50 H versus 25-by-25 S) and was resolved by setting the converted data as spinful. This report concerns missing atom-pair keys even when H and S have the same orbital dimension.
Suggested regression coverage: