Histone deacetylases (HDACs) have recently attracted considerable interest as targets in the treatment of cell proliferative diseases such as cancer. In the present work, a general framework is proposed for chemical groups that bind into the HDAC catalytic core. Based on this framework, a series of groups was selected for further investigation. A method was developed to rank the HDAC inhibitory potential of these moieties at the B3LYP/6-31G* level, making use of extra diffuse functions and of the PCM solvation model where appropriate. The resulting binding geometries indicate that very stringent constraints should be satisfied in order to have bidental zinc chelation, and even more so to have a strong binding affinity, which makes it difficult to predict the binding mode and affinity of such zinc-binding groups. The chemical hardness and the pK(a) were identified as important criteria for the binding affinity. Also, the hydrophilicity may have a direct influence on the binding affinity. The calculated binding energies were qualitatively validated with experimental results from the literature, and were shown to be meaningful for the purpose of ranking. Additionally, the insights gained from the present work may be useful for increasing the accuracy of QSAR models by providing a rational basis for selecting descriptors.