We investigate the modulation of an optical field caused by its interaction with an ultrasound beam in a tissue mimicking phantom. This modulation appears as a modulation in the intensity autocorrelation, which is measured by a photon counting correlator. The factors contributing to the modulation are: 1. amplitude of vibration of the particles of the tissue, 2. refractive index modulation, and 3. absorption coefficient in the region of the tissue intercepted by the ultrasound beam and light. We show in this work that a significant part of the contribution to this modulation comes from displacement of the tissue particles, which in turn is governed by the elastic properties of the tissue. We establish, both through simulations and experiments using an optical elastography phantom, the effects of the elasticity and absorption coefficient variations on the modulation of intensity autocorrelation. In the case where there is no absorption coefficient variation, we suggest that the depth of modulation can be calibrated to measure the displacement of tissue particles that, in turn, can be used to measure the tissue elasticity.