Compounds from both the beta-carboline (BC) and xanthine groups have been suggested to be the natural ligands for benzodiazepine (BZ) receptors. In this study we examined the effects of several BC's and caffeine, 1,3,7-trimethylxanthine, on the binding of 3H-flunitrazepam (3H-FZ) and beta-3H-carboline-3-carboxylic acid ethyl ester (3H-BCCE) to the BZ receptors of rat and mouse brain. In mice, convulsion-producing doses of caffeine (120 mg/kg intravenously) and harmane (30 mg/kg intravenously) lowered the specific binding of 3H-FZ in vivo by 12-31%. A tremorogenic dose of harmaline (30 mg/kg intravenously) increased binding by 31%. Caffeine and harmane also slightly decreased the in vivo binding of 3H-BCCE, a compound that binds preferentially to the cerebellar type of BZ receptors. Harmaline stimulated the binding of 3H-BCCE only in the forebrain. Both harmaline and harmane increased by 41-111% the amount of 3H-BCCE that was distributed to the brain. In vitro BC's and caffeine displaced 3H-FZ from receptors in the rat brain with various Ki values (4.7 to 206.9 microM). The antagonism for BZ binding was competitive and in Scatchard analysis produced linear plots. Exceptions were harmaline and caffeine in the forebrain: both exhibited curvilinear plots for 3H-FZ binding. Harmaline increased the binding, and caffeine decreased it by altering the affinity of a subgroup of BZ receptors. In the hindbrain both harmaline and caffeine inhibited binding and produced linear plots. BC-induced tremor and convulsions unveil a large number of spare receptors in the brain, and these seem to be of the cerebellar type of BZ receptors. In addition, our results show that tremorogenic and convulsive BC's act differently on BZ receptors: during harmaline-induced tremor the affinity of some BZ receptors is increased, while harmane-induced convulsions are connected to direct occupation of BZ receptors.