Methods & reproducibility

How to use predictions: The 0.20 pKd MAE previously quoted here came from a PDBBind 2020 split randomised per complex, which left near-identical structures on both sides — it measured memorisation, not generalisation, and should be disregarded. On 516 PDB structures released after the training data, MAE is 1.35 raw / 1.22 after linear recalibration, against a 1.32 constant-predictor baseline, with Spearman ρ = 0.39. (These figures were revised upward in August 2026 after we found that our own benchmark had been carving pockets a quarter the size of the training ones; see Validation for the correction.) Treat the pKd column as a coarse triage prior: it sorts a library into rough tiers better than chance, but does not reliably order close analogs and is not an experimental K_i/K_d. See Validation.

Correction — the geometric pathway never functioned, and removing it does not hurt (August 2026).

Stage 6 was built as an SE(3)-equivariant EGNN over pocket Cα coordinates fused with ESM2-3B sequence embeddings. Perturbing the pocket geometry — shuffling residues, adding 10 Å of noise, zeroing every coordinate, scaling by 100× — changes the predicted pK_d by exactly 0.000000. Zeroing the ESM2 tensor moves it by −0.60, so the inputs are certainly being read.

Cause. A neighbour-gather bug in the EGNN message-passing layer. Neighbour features were broadcast along the wrong tensor axis (unsqueeze(2) where unsqueeze(1) was required), so every node's "neighbours" resolved to copies of the node itself. The displacement x_i − x_j was therefore identically zero on every edge, every inter-residue distance was 0, and geometry contributed nothing. A freshly initialised encoder shows the same exact-zero response, so this was never a learned behaviour — the equivariant pathway received no geometric signal in any training run, from the first.

An earlier version of this note gave the wrong cause, attributing it to a preprocessing misalignment between all-atom coordinates and per-residue embeddings, and saying the network had "learned to ignore" the input. Both were incorrect. We are recording the error rather than replacing it silently.

Does fixing it help? No measurable difference either way. With the gather corrected and verified Cα-aligned pockets, we trained matched models on identical data and schedule — one given real coordinates, one with residue order destroyed — repeated across three random seeds:

Seed	With Cα geometry	Geometry destroyed	Better arm
11	1.261	1.265	geometry
42	1.288	1.214	no geometry
7	1.267	1.290	geometry
mean	1.272 (sd 0.012)	1.256 (sd 0.032)	—
constant predictor	1.428		—

Every arm beats the constant baseline, so the models are genuinely learning and the comparison is meaningful. But the arms swap places by seed, and the gap between them (0.016 pK_d) is half the run-to-run spread within the no-geometry arm alone (0.032). Spearman behaves the same way: 0.414 ± 0.014 with geometry, 0.441 ± 0.041 without. There is no effect here to report in either direction.

A single-seed version of this section previously claimed the opposite — that destroying geometry produced a better model (1.213 vs 1.270). Replication across three seeds shows that result came entirely from one outlier run. We are recording the correction rather than replacing it silently.

Caveats: 3,077 complexes, ~1M parameters, three seeds, ~600 validation compounds. This does not show that pocket geometry is useless at full scale — only that on the data surviving locally, correctly implemented, it makes no measurable difference. Settling it properly needs the full ~19,000-structure corpus.

What this means for the published numbers: nothing changes. The external benchmark on the Validation page measured the model exactly as it runs in production, and the fix is opt-in so the shipped checkpoint reproduces its predictions bit-for-bit. MAE 1.22 and ρ = 0.39 stand as the current figures (revised upward in August 2026 for a benchmark preprocessing error unrelated to the geometry channel). They are the performance of a ligand-graph-plus-protein-sequence model — and on this evidence, that is all VectaBind has ever been.