Even distribution of peroxisomes (POs) and lipid droplets (LDs) is critical to their role in lipid and reactive oxygen species homeostasis. How even distribution is achieved remains elusive, but diffusive motion and directed motility may play a role. Here we show that in the fungus Ustilago maydis ∼95% of POs and LDs undergo diffusive motions. These movements require ATP and involve bidirectional early endosome motility, indicating that microtubule-associated membrane trafficking enhances diffusion of organelles. When early endosome transport is abolished, POs and LDs drift slowly towards the growing cell end. This pole-ward drift is facilitated by anterograde delivery of secretory cargo to the cell tip by myosin-5. Modelling reveals that microtubule-based directed transport and active diffusion support distribution, mobility and mixing of POs. In mammalian COS-7 cells, microtubules and F-actin also counteract each other to distribute POs. This highlights the importance of opposing cytoskeletal forces in organelle positioning in eukaryotes.