We present a concept and prototype of a memory element based on current driven magneto-impedance (MI) effect that stores the binary data (0, 1) as the orientation of the magnetization. The magnetization orientation in the surface layer with tilted anisotropy easy axis can be switched controllably between two stable states by applying current pulses of the appropriate sign, and can be detected by sensing the impedance. We demonstrated the functioning of a non-volatile magnetic memory with a read speed performance up to and above 2 GHz. A prototype of a memory element was realized on a short piece of amorphous microwire, as this material exhibits the highest MI effect, and the required anisotropy can be quite easily obtained. Nevertheless, this concept can be extended to other materials and geometries exhibiting MI effect and possessing a required magnetic anisotropy.