Ferroelectric and magnetic materials are a time-honoured subject of study and have led to some of the most important technological advances to date. Magnetism and ferroelectricity are involved with local spins and off-centre structural distortions, respectively. These two seemingly unrelated phenomena can coexist in certain unusual materials, termed multiferroics. Despite the possible coexistence of ferroelectricity and magnetism, a pronounced interplay between these properties has rarely been observed. This has prevented the realization of multiferroic devices offering such functionality. Here, we report a striking interplay between ferroelectricity and magnetism in the multiferroic TbMn2O5, demonstrated by a highly reproducible electric polarization reversal and permanent polarization imprint that are both actuated by an applied magnetic field. Our results point to new device applications such as magnetically recorded ferroelectric memory.