Polypeptides synthesized in eucaryotic cells with a C-terminal -Cys-Xaa-Xaa-Xaa (-CXXX) sequence are candidates for post-translational modifications that include the removal of the last 3 amino acids and the lipidation and methyl esterification of the cysteinyl residue. To characterize the methylation reaction in vitro, the peptide Leu-Ala-Arg-Tyr-Lys-Cys (LARYKC) and its S-isoprenylated and S-alkylated derivatives were synthesized and assayed as methyl-accepting substrates with subcellular fractions of rat tissues including liver microsomal membranes. While little or no peptide-specific methyltransferase activity was detected in the latter preparation using the unmodified hexapeptide, the C10, C15, and C20 isoprenylated derivatives were substrates with Km values of 389 microM for S-geranyl-LARYKC, 2.2 microM for S-farnesyl-LARYKC, and approximately 10.9 microM for S-geranylgeranyl-LARYKC. The methyl-acceptor activities of a variety of n-alkyl S-derivatives of LARYKC (C8, C10, C13, C15) were also tested; all of these compounds were poorer substrates than the S-geranyl derivative. This enzyme activity uses S-adenosyl-L-methionine as the methyl donor (Km = 2.1 microM) and can be inhibited by S-adenosylhomocysteine (Ki = 9.2 microM), a product of the methylation reaction. The S-farnesyl-LARYKC peptide can inhibit the carboxyl methylation of bovine retinal rod outer segment membrane proteins that was previously shown to occur at the alpha-carboxyl group of C-terminal cysteine residues, demonstrating that the same enzyme can methylate both peptides and proteins. These results suggest that the methyl esterification of proteins containing a C-terminal -CXXX sequence requires not only the removal of the 3 terminal amino acids, but the isoprenylation of the sulfhydryl group as well.