In vertebrate neurons, the axon initial segment (AIS) is specialized for action potential initiation. It is organized by a giant 480 Kd variant of ankyrin G (AnkG) that serves as an anchor for ion channels and is required for a plasma membrane diffusion barrier that excludes somatodendritic proteins from the axon. An unusually long exon required to encode this 480Kd variant is thought to have been inserted only recently during vertebrate evolution, so the giant ankyrin-based AIS scaffold has been viewed as a vertebrate adaptation for fast, precise signaling. We re-examined AIS evolution through phylogenomic analysis of ankyrins and by testing the role of ankyrins in proximal axon organization in a model multipolar Drosophila neuron (ddaE). We find giant isoforms of ankyrin in all major bilaterian phyla, and present evidence in favor of a single common origin for giant ankyrins and the corresponding long exon in a bilaterian ancestor. This finding raises the question of whether giant ankyrin isoforms play a conserved role in AIS organization throughout the Bilateria. We examined this possibility by looking for conserved ankyrin-dependent AIS features in Drosophila ddaE neurons via live imaging. We found that ddaE neurons have an axonal diffusion barrier proximal to the cell body that requires a giant isoform of the neuronal ankyrin Ank2. Furthermore, the potassium channel shal concentrates in the proximal axon in an Ank2-dependent manner. Our results indicate that the giant ankyrin-based cytoskeleton of the AIS may have evolved prior to the radiation of extant bilaterian lineages, much earlier than previously thought.