We report on optical image contrast for a specific apertureless near-field optical microscope. We demonstrate that the main part of the optical image's contrast results from the sample's topography. The coupling mechanism is analyzed, and we show that the microscope can be regarded as an interferometer that sensitively detects near-field components. However, in the basic configuration the reference field of the interferometer is coupled to the topography. Finally, it is demonstrated that, by controlling the phase of the reference field, one can largely decorrelate the optical image from the topography.