As first pointed out by Bardeen and Ginzburg in the early sixties, the amount of magnetic flux carried by vortices in superconducting materials depends on their distance from the sample edge, and can be smaller than one flux quantum, phi0 = h/2e (where h is Planck's constant and e is the electronic charge). In bulk superconductors, this reduction of flux becomes negligible at submicrometre distances from the edge, but in thin films the effect may survive much farther into the material. But the effect has not been observed experimentally, and it is often assumed that magnetic field enters type II superconductors in units of phi0. Here we measure the amount of flux introduced by individual vortices in a superconducting film, finding that the flux always differs substantially from phi0. We have observed vortices that carry as little as 0.001phi0, as well as 'negative vortices', whose penetration leads to the expulsion of magnetic field. We distinguish two phenomena responsible for non-quantized flux penetration: the finite-size effect and a nonlinear screening of the magnetic field due to the presence of a surface barrier. The latter effect has not been considered previously, but is likely to cause non-quantized penetration in most cases.