5-Methoxytryptoline potently inhibits [3H]imipramine binding to membranes from the cerebral cortex and platelets. Since 5-methoxytryptoline, which appears to occur endogenously with particularly high levels in the human pineal gland, also inhibits 5-hydroxytryptamine (5-HT, serotonin) uptake, it should be considered as a putative endogenous ligand modulating 5-HT transport. As the 5-HT transporter complex comprises the imipramine and the substrate recognition sites, which interact allosterically, it was essential to define the mechanism of inhibition of [3H]imipramine binding by 5-methoxytryptoline. Human platelets show an active and saturable uptake of 5-HT and tryptamine. The uptake of both substrates appears to be mediated by the same carrier and it is inhibited by 5-methoxytryptoline at submicromolar concentrations. 5-HT and tryptamine inhibit [3H]imipramine binding in human platelets with a Hill slope for inhibition close to unity and IC50 values of 3,265 and 3,475 nM, respectively. This inhibition is, however, not competitive because both 5-HT and tryptamine significantly decrease the rate of [3H]imipramine-receptor dissociation. Although 5-methoxytryptoline potently inhibits [3H]imipramine binding (IC50 = 44 nM) in human platelets with a Hill slope of unity, it does not affect the receptor-ligand dissociation rate of [3H]imipramine even at concentrations up to 100 microM. The present experiments show that 5-methoxytryptoline, in spite of its chemical similarity to the indoleamine transporter substrates, interacts with the imipramine receptor through a mechanism of competitive inhibition. This conclusion is supported by a selective effect of 5-methoxytryptoline on the Kd of [3H]imipramine binding.(ABSTRACT TRUNCATED AT 250 WORDS)