In this paper, we, respectively, introduce memristor, meminductor, and memcapacitor models. By employing these models, a simple chaotic circuit is designed, which only contains three memory devices in parallel: a memristor, a memcapacitor, and a meminductor. Complex dynamic behaviors are observed and investigated, including variable-boostable feature, chaos bursting phenomenon, phase diagram offsets, coexisting attractors, and transient and local sustained chaotic states. Furthermore, the presented chaotic system is realized by an experimental circuit, and abundant dynamic characteristics are confirmed by hardware experiments, which can be applied in engineering fields.