Alterations in genomic and non-genomic mechanisms can disturb homeostasis and cause severe human diseases. Histone deacetylases (HDACs) are epigenetic regulators which catalyze the removal of acetyl moieties from histones and non-histone proteins. Aberrant histone deacetylation, due to increased HDAC activity and expression, often correlates with pathological gene repression and neoplastic transformation. Therefore, intense efforts have been made to find small molecule inhibitors of HDACs (HDACIs). Such compounds indeed alter cellular signaling networks relevant for tumorigenesis, and several HDACIs are currently tested in clinical trials against different types of cancer. Although HDACs share a conserved deacetylase domain and an at least similar mechanism of catalysis, isoenzyme-specific HDACIs could be identified and certain HDACIs even evoke degradation of HDACs. Here, we summarize molecular actions of HDACs and of different classes of HDACIs. In addition, we review data obtained in clinical studies involving HDACIs and we discuss how such agents might be beneficial for the treatment of cancer.