It was recently observed that the ferroelectrics showing the strongest sensitivity to an applied magnetic field are spiral magnets. We present a phenomenological theory of inhomogeneous ferroelectric magnets, which describes their thermodynamics and magnetic field behavior, e.g., dielectric susceptibility anomalies at magnetic transitions and sudden flops of electric polarization in an applied magnetic field. We show that electric polarization can also be induced at domain walls and that magnetic vortices carry electric charge.