JIP3/UNC-16/dSYD is a MAPK-scaffolding protein with roles in protein trafficking. We show that it is present on the Golgi and is necessary for the polarized distribution of synaptic vesicle proteins (SVPs) and dendritic proteins in neurons. UNC-16 excludes Golgi enzymes from SVP transport carriers and facilitates inclusion of specific SVPs into the same transport carrier. The SVP trafficking roles of UNC-16 are mediated through LRK-1, whose localization to the Golgi is reduced in unc-16 animals. UNC-16, through LRK-1, also enables Golgi-localization of the μ-subunit of the AP-1 complex. AP1 regulates the size but not the composition of SVP transport carriers. Additionally, UNC-16 and LRK-1 through the AP-3 complex regulates the composition but not the size of the SVP transport carrier. These early biogenesis steps are essential for dependence on the synaptic vesicle motor, UNC-104 for axonal transport. Our results show that UNC-16 and its downstream effectors, LRK-1 and the AP complexes function at the Golgi and/or post-Golgi compartments to control early steps of SV biogenesis. The UNC-16 dependent steps of exclusion, inclusion and motor recruitment are critical for polarized distribution of neuronal cargo.