(23S)-25-Dehydro-1alpha(OH)-vitamin D(3)-26,23-lactone (MK) is an antagonist of the 1alpha,25(OH)(2)-vitamin D(3) (1,25D)/human nuclear vitamin D receptor (hVDR) transcription initiation complex, where the activation helix (i.e. helix-12) is closed. To study the mode of antagonism of MK an hVDR mutant library was designed to alter the free molecular volume in the region of the hVDR ligand binding pocket occupied by the ligand side-chain atoms (i.e. proximal to helix-12). The 1,25D-hVDR structure-function studies demonstrate that 1) van der Waals contacts between helix-12 residues Leu-414 and Val-418 and 1,25D enhance the stability of the closed helix-12 conformer and 2) removal of the side-chain H-bonds to His-305(F) and/or His-397(F) have no effect on 1,25D transactivation, even though they reduce the binding affinity of 1,25D. The MK structure-function results demonstrate that the His-305, Leu-404, Leu-414, and Val-418 mutations, which increase the free volume of the hVDR ligand binding pocket, significantly enhance MK antagonist potency. Surprisingly, the H305F and H305F/H397F mutations turn MK into a VDR superagonist (EC(50) approximately 0.05 nm) but do not concomitantly alter MK binding affinity. Molecular modeling studies demonstrate that MK antagonism stems from its side chain energetically preferring a pose in the VDR ligand binding pocket where its terminal C26-methylene atom is far removed from helix-12. MK superagonism results from an energetically favored increase in interaction between Leu-404/Val-418 and C26, resulting in an increase in the stability and population of the closed, helix-12 conformer. Finally, the results/model generated, coupled with application of a VDR ensemble allosterics model, provide an understanding for the species specificity of MK.