Twelve ergolines (O-acylated lysergol and dihydrolysergol-I derivatives) were synthesized to study their antagonism of 5-HT responses in comparison with methylsergide and LY 53857 [6-methyl-1-(1-methylethyl)-8 beta-ergoline carboxylic acid 2-hydroxy-1-methylpropyl-ester hydrogen maleate] in cylindrical segments of the isolated rat tail artery. With regard to (9.10-didehydro-6-methyl-8 beta-ergoline)methyl R,S-2-methylbutyrate, the most potent new ergoline derivative, we examined the phenomenon of insurmountable antagonism to 5-HT by methylsergide. O-Acylated lysergol and dihydrolysergol-I derivatives competitively antagonized 5-HT-induced contractions with calculated pA2 values of 7.30 +/- 0.42 for the weakest and 8.42 +/- 0.35 for the most potent ergoline derivative in this series. N1-isopropyl substitution did not generally enhance 5-HT2 receptor affinities but lowered affinities for alpha 1 adrenoceptors in rat aorta. Methysergide and LY 53857 were insurmountable antagonists of 5-HT in rat tail artery. Preincubation with (9.10-didehydro-6-methyl-8 beta-ergoline)methyl R,S-2-methylbutyrate (1 mumol/l) partially prevented the depression of 5-HT-induced contractions caused by methysergide (1-10 nmol/l). Methysergide (100 nmol/l) abolished the protective effect of (9.10-didehydro-6-methyl-8 beta-ergoline)methyl R,S-2-methylbutyrate. (9.10-Didehydro-6-methyl-8 beta-ergoline)methyl R,S-2-methylbutyrate (1 mumol/l), concomitantly incubated with methysergide (30 nmol/l), partially restored the maximum response to 5-HT that had been depressed by methysergide (30 nmol/l). Partial restoration could not be mimicked by washout of methysergide.(ABSTRACT TRUNCATED AT 250 WORDS)