Growth-mediated feedback between synthetic gene circuits and host organisms leads to diverse emerged behaviors, including growth bistability and enhanced ultrasensitivity. However, the range of possible impacts of growth feedback on gene circuits remains underexplored. Here we mathematically and experimentally demonstrated that growth feedback affects the functions of memory circuits in a network topology-dependent way. Specifically, the memory of the self-activation switch is quickly lost due to the growth-mediated dilution of the circuit products. Decoupling of growth feedback reveals its memory, manifested by its hysteresis property across a broad range of inducer concentration. On the contrary, the toggle switch is more refractory to growth-mediated dilution and can retrieve its memory after the fast-growth phase. The underlying principle lies in the different dependence of active and repressive regulations in these circuits on the growth-mediated dilution. Our results unveil the topology-dependent mechanism on how growth-mediated feedback influences the behaviors of gene circuits.