Dissymmetric, periodically nanostructured metal films can show non-reciprocal transmission of polarized light, in apparent violation of the Lorentz reciprocity theorem. The wave vector dependence of the extraordinary optical transmission in gold films with square and oblique subwavelength hole arrays was examined for the full range of polarized light input states. In normal incidence, the oblique lattice, in contrast to square lattice, showed strong asymmetric, non-reciprocal transmission of circularly polarized light. By analyzing the polarization of the input and the output with a complete Mueller matrix polarimeter the mechanisms that permits asymmetric transmission while preserving the requirement of electromagnetic reciprocity is revealed: the coupling of the linear anisotropies induced by misaligned surface plasmons in the film. The square lattice also shows asymmetric transmission at non-normal incidence, whenever the plane of incidence does not coincide with a mirror line.