Delay-period activity of prefrontal cortical cells, the neural hallmark of working memory, is generally assumed to be sustained by reverberating synaptic excitation in the prefrontal cortical circuit. Previous model studies of working memory emphasized the high efficacy of recurrent synapses, but did not investigate the role of temporal synaptic dynamics. In this theoretical work, I show that biophysical properties of cortical synaptic transmission are important to the generation and stabilization of a network persistent state. This is especially the case when negative feedback mechanisms (such as spike-frequency adaptation, feedback shunting inhibition, and short-term depression of recurrent excitatory synapses) are included so that the neural firing rates are controlled within a physiological range (10-50 Hz), in spite of the exuberant recurrent excitation. Moreover, it is found that, to achieve a stable persistent state, recurrent excitatory synapses must be dominated by a slow component. If neuronal firings are asynchronous, the synaptic decay time constant needs to be comparable to that of the negative feedback; whereas in the case of partially synchronous dynamics, it needs to be comparable to a typical interspike interval (or oscillation period). Slow synaptic current kinetics also leads to the saturation of synaptic drive at high firing frequencies that contributes to rate control in a persistent state. For these reasons the slow NMDA receptor-mediated synaptic transmission is likely required for sustaining persistent network activity at low firing rates. This result suggests a critical role of the NMDA receptor channels in normal working memory function of the prefrontal cortex.