Mapping the melatonin receptor. 5. Melatonin agonists and antagonists derived from tetrahydrocyclopent[b]indoles, tetrahydrocarbazoles and hexahydrocyclohept[b]indoles

J Med Chem. 1998 Feb 12;41(4):451-67. doi: 10.1021/jm970246n.


Tetrahydrocyclopent[b]indoles, tetrahydrocarbazoles, and hexahydrocyclohept[b]indoles have been prepared as melatonin analogues to investigate the nature of the binding site of the melatonin receptor. The affinity of analogues was compared in a radioligand binding assay using chicken brain membranes and agonist and antagonist potency measured in clonal Xenopus laevis melanophore cells. Comparison of the N-acyl-3-amino-6-methoxytetrahydrocarbazoles (2) with N-acyl-4-(aminomethyl)-6-methoxy-9-methyltetrahydrocarbazoles (9) showed that the latter have much higher binding affinities for the chicken brain receptor. Comparison of N-acyl-1-(aminomethyl)-7-methoxy-4-methyltetrahydrocyclopent[b]ind oles (10), 6-methoxytetrahydrocarbazoles (9), and N-acyl-10-(aminomethyl)-2-methoxy-5-methylhexahydrocyclohept[b]ind oles (11) showed that the tetrahydrocarbazoles had the highest binding affinity with the cyclohept[b]indoles and the cyclopent[b]indoles having rather lower affinities. All of these observations are in agreement with our postulated model of melatonin orientation at the binding pocket in which the 3-amidoethane side chain is in a conformation close to the 5-methoxyl group, as is shown in the X-ray crystallographic structure of 9m and in the energy-minimized computed structures. Separation of the enantiomers of members from each of these three systems was accomplished by chiral HPLC. It was found that in all cases the (-)-enantiomer had a higher binding affinity than the (+)-enantiomer. An X-ray crystallographic analysis of the two enantiomers of 9a showed that the (+)-enantiomer had the (R) absolute stereochemistry. Since the sign of the Cotton curves, determined from circular dichroism studies, was the same for all (+)-enantiomers, it is assumed that the absolute stereochemistry at these centers is identical. In the Xenopus melanophore assay, the tetrahydrocarbazoles 2 (R = H) were mainly weak antagonists, while those with R = OMe were agonists. The biological behavior of the tetrahydrocarbazoles 9 (R = H) depended on R1, some being agonists and some antagonists, whereas those with R = OMe were generally agonists. Variation of the R and R1 groups in compounds of type 9 produced both agonists and antagonists. The tetrahydrocylopentaindoles 10 had similar biological properties to the corresponding analogues of 9, but the hexahydrocycloheptaindoles 11 showed a much greater propensity to be antagonists. In all cases the (S)-enantiomers were found to be more potent agonists than the (R)-enantiomers.

Publication types

  • Research Support, Non-U.S. Gov't

MeSH terms

  • Animals
  • Binding, Competitive
  • Brain / metabolism
  • Carbazoles / chemical synthesis*
  • Carbazoles / chemistry
  • Carbazoles / pharmacology
  • Cell Membrane / metabolism
  • Chickens
  • Crystallography, X-Ray
  • Indicators and Reagents
  • Indoles / chemical synthesis*
  • Indoles / chemistry
  • Indoles / pharmacology
  • Melanophores / cytology
  • Melanophores / drug effects
  • Melanophores / physiology
  • Melatonin / metabolism
  • Models, Molecular
  • Molecular Conformation
  • Receptors, Cell Surface / agonists*
  • Receptors, Cell Surface / antagonists & inhibitors*
  • Receptors, Cytoplasmic and Nuclear / agonists*
  • Receptors, Cytoplasmic and Nuclear / antagonists & inhibitors*
  • Receptors, Melatonin
  • Structure-Activity Relationship
  • Xenopus laevis


  • Carbazoles
  • Indicators and Reagents
  • Indoles
  • Receptors, Cell Surface
  • Receptors, Cytoplasmic and Nuclear
  • Receptors, Melatonin
  • Melatonin