Visually induced neuronal activity in V1 displays a marked gamma-band component which is modulated by stimulus properties. It has been argued that synchronized oscillations contribute to these gamma-band activity. However, analysis of Local Field Potentials (LFPs) across different experiments reveals considerable diversity in the degree of oscillatory behavior of this induced activity. Contrast-dependent power enhancements can indeed occur over a broad band in the gamma frequency range and spectral peaks may not arise at all. Furthermore, even when oscillations are observed, they undergo temporal decorrelation over very few cycles. This is not easily accounted for in previous network modeling of gamma oscillations. We argue here that interactions between cortical layers can be responsible for this fast decorrelation. We study a model of a V1 hypercolumn, embedding a simplified description of the multi-layered structure of the cortex. When the stimulus contrast is low, the induced activity is only weakly synchronous and the network resonates transiently without developing collective oscillations. When the contrast is high, on the other hand, the induced activity undergoes synchronous oscillations with an irregular spatiotemporal structure expressing a synchronous chaotic state. As a consequence the population activity undergoes fast temporal decorrelation, with concomitant rapid damping of the oscillations in LFPs autocorrelograms and peak broadening in LFPs power spectra. We show that the strength of the inter-layer coupling crucially affects this spatiotemporal structure. We predict that layer VI inactivation should induce global changes in the spectral properties of induced LFPs, reflecting their slower temporal decorrelation in the absence of inter-layer feedback. Finally, we argue that the mechanism underlying the emergence of synchronous chaos in our model is in fact very general. It stems from the fact that gamma oscillations induced by local delayed inhibition tend to develop chaos when coupled by sufficiently strong excitation.