We analyzed theoretically the spectral dependence of polarimetric sensitivity to temperature (KT) and the susceptibility of phase modal birefringence to temperature (dB/dT) in several birefringent photonic crystal holey fibers of different construction. Contributions to dB/dT related to thermal expansion of the fiber dimensions and that related to temperature-induced changes in glass and air refractive indices were calculated separately. Our results showed that dB/dT depends strongly on the material used for manufacturing the fiber and on the fiber's geometry. We demonstrate that, by properly designing the birefringent holey fiber, it is possible to reduce its temperature sensitivity significantly and even to ensure a null response to temperature at a specific wavelength. Furthermore, we show that the temperature sensitivity in a fiber with arbitrary geometry can be significantly reduced by proper choice of the glass used in the fiber's manufacture. We also measured the polarimetric sensitivity to temperature and identified its sign in two silica-air fibers. The experimental values are in good agreement with the results of modeling.