We study the emergence of a yield stress in dense suspensions of non-Brownian particles by combining local velocity and concentration measurements using magnetic resonance imaging with macroscopic rheometric experiments. We show that the competition between gravity and viscous stresses is at the origin of the development of a yield stress in these systems at relatively low volume fractions. Moreover, it is accompanied by a shear-banding phenomenon that is the signature of this competition. However, if the system is carefully density matched, no yield stress is encountered until a volume fraction of 62.7 +/- 0.3%.