Nanoscale metal inclusions in or on solid-state dielectrics are an integral part of modern electrocatalysis, optoelectronics, capacitors, metamaterials and memory devices. The properties of these composite systems strongly depend on the size, dispersion of the inclusions and their chemical stability, and are usually considered constant. Here we demonstrate that nanoscale inclusions (for example, clusters) in dielectrics dynamically change their shape, size and position upon applied electric field. Through systematic in situ transmission electron microscopy studies, we show that fundamental electrochemical processes can lead to universally observed nucleation and growth of metal clusters, even for inert metals like platinum. The clusters exhibit diverse dynamic behaviours governed by kinetic factors including ion mobility and redox rates, leading to different filament growth modes and structures in memristive devices. These findings reveal the microscopic origin behind resistive switching, and also provide general guidance for the design of novel devices involving electronics and ionics.