The morphology of the superconducting flux line lattice (FLL) of Nb comprises gradual variations with various lock-in transitions and symmetry breaking rotations. We report a comprehensive small-angle neutron scattering study of the FLL in an ultrapure single crystal of Nb as a function of the orientation of the applied magnetic field. We attribute the general morphology of the FLL and its orientation to three dominant mechanisms; first, nonlocal contributions, second, the transition between open and closed Fermi surface sheets and, third, the intermediate mixed state between the Meissner and the Shubnikov phase.