Binding free energy calculations rely critically on a precise definition of the bound state and well-designed ligand restraints to ensure that binding free energy calculations converge rapidly and yield estimates of well-defined thermodynamic quantities. The distance-to-bound-configuration (DBC) is a single variable that can precisely delineate the bound state of a ligand including translational, rotational and conformational degrees of freedom and has been successfully used to capture binding modes with complex geometries. DBC is defined as the root-mean-square deviation (RMSD) of ligand coordinates in the frame of reference of the binding site. In the special case where the ligand features symmetry-equivalent atoms, a standard RMSD arbitrarily distinguishes equivalent poses, mixing equivalent and nonequivalent degrees of freedom, and preventing the precise delineation of the bound state ensemble, which negates the benefits of defining a flat-bottom binding restraint. To remedy this, we introduce a symmetry-adapted DBC coordinate where the RMSD is minimized over permutations of equivalent ligand atoms. This coordinate is implemented in a portable software library, the Collective Variables Module. We tested the approach by computing the absolute binding free energy of benzene to the engineered site of a mutant lysozyme (L99A/M102H) using alchemical free energy perturbation. We found that the symmetry-adapted restraint leads to well-behaved convergence of both the decoupling free energy in the binding site and the restrained free energy in the gas phase, recovering the affinity computed using a classic center-of-mass restraint. Thus, symmetry-adapted DBC seamlessly generalizes the benefits of DBC restraints to the case of symmetric ligands. The underlying symmetric RMSD coordinate can also be used for analyzing or biasing simulations in other contexts than affinity predictions.