Endonexin (annexin IV) is a member of the annexin family of homologous proteins that bind membranes and aggregate vesicles in a calcium-dependent fashion. This study examines the lipid modulation and mechanism of the binding of endonexin to membranes using a fluorescence energy transfer assay to measure bovine endonexin binding to well-defined large unilamellar vesicles. The calcium sensitivity for endonexin-membrane binding is observed to be highly dependent on the types of membrane lipids present. As with most annexins, negatively charged lipids best promote endonexin binding to phosphatidylcholine (PC) containing membranes. However, a comparison of 11 different types of lipids reveals that other factors such as the type of ion contributing the charge and head-group size are also important. The concentrations of calcium required for half-maximal binding of endonexin to PC vesicles containing 30% phosphatidylserine (PS) or 30% phosphatidylinositol (PI), both lipids with net charge-1, are 48 +/- 6 and 114 +/- 19 microM, respectively, while half-maximal binding to 30% phosphatidylinositol bisphosphate (PIP2), with a greater net charge of -3 to -5, occurs at 65 microM calcium, similar to the calcium requirement for binding to PS. The apparent affinities of endonexin for seven different types of lipids parallel those reported for annexin V [Andree, H. A. M., Reutelingsperger, C. P. M., Hauptmann, R., Hemker, H. C., Hermans, W. T., & Willems, G. M. (1990) J. Biol. Chem. 265, 4923-4928], except for a greater preference of endonexin for membranes containing phosphatidic acid. Mixing PS and phosphatidylethanolamine (PE) or PS and PI in the same PC vesicle synergistically enhances endonexin-membrane binding, indicating that even lipids with no net charge such as PE may dramatically affect endonexin binding to mixed-lipid membranes. The maximum amount of endonexin able to bind to PS/PC vesicles at 1 mM calcium increases with mole % PS. A simple and general model that treats protein-membrane binding as a two-step process, with adsorption to a membrane surface followed by interaction with specific lipid molecules [Lentz, B. R., & Hermans, J. (1989) Biochemistry 28, 7459-7461], is extended to include the coupled binding of calcium with binding of specific lipid molecules. This extended model accurately predicts trends observed when protein and calcium titrations of endonexin binding to PS/PC vesicles are performed under a wide variety of conditions and suggests that 3-5 calcium ions and 9-18 PS molecules participate in each endonexin-membrane complex.(ABSTRACT TRUNCATED AT 400 WORDS)