The cellular diversity of interneurons in the neocortex is thought to reflect subtype-specific roles of cortical inhibition. Here we ask whether perturbations to two subtypes--parvalbumin-positive (PV+) and somatostatin-positive (SST+) interneurons--can be compensated for with respect to their contributions to cortical development. We use a genetic cell fate switch to delete both PV+ and SST+ interneurons selectively in cortical layers 2-4 without numerically changing the total interneuron population. This manipulation is compensated for at the level of synaptic currents and receptive fields (RFs) in the somatosensory cortex. By contrast, we identify a deficit in inhibitory synchronization in vitro and a large reduction in cortical gamma oscillations in vivo. This reveals that, while the roles of inhibition in establishing cortical inhibitory/excitatory balance and RFs can be subserved by multiple interneuron subtypes, gamma oscillations depend on cellular properties that cannot be compensated for--likely, the fast signalling properties of PV+ interneurons.