It was previously believed that larger metal nanoparticles behave as tiny mirrors that are pushed by the light beam radiative force along the direction of beam propagation, without a chance to be confined. However, several groups have recently reported successful optical trapping of gold and silver particles as large as 250 nm. We offer a possible explanation based on the fact that metal nanoparticles naturally occur in various non-spherical shapes and their optical properties differ significantly due to changes in localized plasmon excitation. We demonstrate experimentally and support theoretically three-dimensional confinement of large gold nanoparticles in an optical trap based on very low numerical aperture optics. We showed theoretically that the unique properties of gold nanoprisms allow an increase of trapping force by an order of magnitude at certain aspect ratios. These results pave the way to spatial manipulation of plasmonic nanoparticles using an optical fibre, with interesting applications in biology and medicine.