The activation of G-proteins by melatonin mt1 receptors was studied by measuring [35S]-guanosine-5'-(3-thiotriphosphate) ([35S]-GTPgammaS) binding to membranes prepared from Chinese hamster ovary (CHO) cells stably expressing human mt1 receptors. Melatonin stimulated [35S]-GTPgammaS binding in a concentration-dependent manner (pEC50, 8.77+/-0.02). The optimal (212+/-4%) increase over basal levels of binding (basal = 100%) was observed following incubation of membranes (12.5 microg protein/well) for 120 min at 30 degrees with [35S]-GTPgammaS (0.1 nM), in the presence of GDP (10 microM), NaCl (100 mM), and MgCl2 (10 mM). Melatonin analogues stimulated [35S]-GTPgammaS binding with a rank order (2-iodomelatonin > melatonin = S20098 > GR196429 > 6-chloromelatonin = 6-hydroxymelatonin >> N-acetylserotonin > or = GR135531 = mt1 luzindole = 5-HT = 0), which was identical to their affinities for the high affinity state of the receptor (correlation coefficient 0.94). All agonists evoked similar maximum increases in [35S]-GTPgammaS binding. EC50 values were 14- to 63-fold lower than binding affinities. The melatonin receptor antagonist luzindole (0.1-10 microM) evoked a parallel rightward shift in the melatonin concentration-response curve, with a pKB of 7.19+/-0.13, which is similar to its affinity in radioligand binding studies for human mt1 receptors. Stimulation of [35S]-GTPgammaS binding was abolished by pretreatment of cells with pertussis toxin (18 hr, 100 ng/mL) prior to preparation of membranes. Melatonin was without effect in CHO cells which lacked the mt1 receptor. Thus, melatonin and melatonin analogues stimulate [35S]-GTPgammaS binding with a profile which is consistent with binding to mt1 receptors causing activation of Gi/Go G-proteins.