We present a physical model describing the radiance acquired by an infrared sensor over a rugged heterogeneous surface. This model predicts the radiance seen over complex landscapes like urban areas and provides an accurate analysis of the signal, as each component is available at ground and sensor level. Plus, it allows data comparison from different instruments. Two representative cases (natural and urban) are analysed to show the composition and the construction of the sensor signal and to highlight the importance of having a 3D model, especially for rugged surfaces where environment weights in the overall spectral domain.