Neurons and especially their synapses often project long thin processes that can invaginate neighboring neuronal or glial cells. These "invaginating projections" can occur in almost any combination of postsynaptic, presynaptic, and glial processes. Invaginating projections provide a precise mechanism for one neuron to communicate or exchange material exclusively at a highly localized site on another neuron, e.g., to regulate synaptic plasticity. The best-known types are postsynaptic projections called "spinules" that invaginate into presynaptic terminals. Spinules seem to be most prevalent at large very active synapses. Here, we present a comprehensive review of all kinds of invaginating projections associated with both neurons in general and more specifically with synapses; we describe them in all animals including simple, basal metazoans. These structures may have evolved into more elaborate structures in some higher animal groups exhibiting greater synaptic plasticity. In addition to classic spinules and filopodial invaginations, we describe a variety of lesser-known structures such as amphid microvilli, spinules in giant mossy terminals and en marron/brush synapses, the highly specialized fish retinal spinules, the trophospongium, capitate projections, and fly gnarls, as well as examples in which the entire presynaptic or postsynaptic process is invaginated. These various invaginating projections have evolved to modify the function of a particular synapse, or to channel an effect to one specific synapse or neuron, without affecting those nearby. We discuss how they function in membrane recycling, nourishment, and cell signaling and explore how they might change in aging and disease.