Single photon sources can greatly benefit from specially designed structures that modify the properties of the photon emitter. Dielectric cavities are often discussed, but they require a compromise between the spectral width and Purcell factor. In this Letter, we introduce plasmonic cavities as promising alternatives. We first study how the emitter couples with the modes of such structures. We then show how a patch antenna configuration simultaneously presents a large Purcell factor, collection efficiency, and spectral width.