Photoacoustic imaging combines the high contrast of optical imaging with the spatial resolution and penetration depth of ultrasound. This technique holds tremendous potential for imaging in small animals and importantly, is clinically translatable. At present, there is no accepted standard physical phantom that can be used to provide routine quality control and performance evaluation of photoacoustic imaging instruments. With the growing popularity of the technique and the advent of several commercial small animal imaging systems, it is important to develop a strategy for assessment of such instruments. Here, we developed a protocol for fabrication of physical phantoms for photoacoustic imaging from polyvinyl chloride plastisol (PVCP). Using this material, we designed and constructed a range of phantoms by tuning the optical properties of the background matrix and embedding spherical absorbing targets of the same material at different depths. We created specific designs to enable: routine quality control; the testing of robustness of photoacoustic signals as a function of background; and the evaluation of the maximum imaging depth available. Furthermore, we demonstrated that we could, for the first time, evaluate two small animal photoacoustic imaging systems with distinctly different light delivery, ultrasound imaging geometries and center frequencies, using stable physical phantoms and directly compare the results from both systems.