Transforming growth factor-beta1 (TGF-beta1) regulates a variety of cellular functions. In several types of cells, for example, it acts as a growth inhibitor and an inducer of apoptotic cell death. Although one of the important modulators in retinal vascular development and retinal neovascularization, the effects of TGF-beta1 on retinal microvascular cells are not fully defined. We have found that proliferation of both bovine retinal endothelial cells (EC) and pericytes was inhibited by TGF-beta1 in a concentration-dependent manner. However, only retinal EC lost viability after exposure to increasing concentrations of TGF-beta1 (up to 10 microg/ml) in the presence of 2% fetal bovine serum. Dying EC exhibited the morphological and biochemical characteristics of apoptosis. Fragmented nuclei and chromatin condensation were apparent after staining with the fluorochrome Hoechst 33258 and the reagent ApopTag; moreover, gel electrophoresis of DNA from TGF-beta1-treated EC demonstrated degradation of chromatin into the discrete fragments typically associated with apoptosis. The addition of anti-TGF-beta1 neutralizing antibody abolished the apoptotic cell death induced by TGF-beta1. Because not all the EC in a given culture died after exposure to TGF-beta1, we separated the apoptosis-sensitive cells from those resistant to TGF-beta1 -mediated apoptosis and determined the expression of several proteins associated with this apoptotic pathway. Apoptosis of EC mediated by TGF-beta1 was associated with a decreased level of the cyclin-dependent kinase inhibitor p21waf1/cip1, compared with that observed in the apoptosis-resistant cells. In contrast, the translation product of the tumor-suppressor gene p53 was increased in the TGF-beta1-treated apoptotic cells. Thus, we propose that p21waf1/cip1 and p53 function in distinct pathways that are protective or permissive, respectively, for the apoptotic signals mediated by TGF-beta1.