A single small molecule can have thousands of atoms. Full attention, every atom comparing itself to every other atom, costs O(atoms squared), too expensive to run at every one of AF3's roughly 200 diffusion steps. AF3's atom-level attention (AtomTransformer in machine-learning terms, the step where an atom checks its own local 3D geometry in structural-biology terms) instead restricts each atom to a nearby neighborhood.
Each atom only compares itself to a local neighborhood, not the whole molecule, that is what keeps atom-level attention affordable.
Masking is not a separate mechanism from ordinary attention biasing. A disallowed pair just gets -10^10 added to its attention logit before the softmax, effectively -infinity, so it comes out as exactly zero after it.