Using fMR adaptation, we studied the effects of aging on the neural processing of passively viewed naturalistic pictures composed of a prominent object against a background scene. Spatially distinct neural regions showing specific patterns of adaptation to objects, background scenes, and contextual integration (binding) were identified in young adults. Older adults did not show adaptation responses corresponding to binding in the medial-temporal areas. They also showed an adaptation deficit for objects whereby their lateral occipital complex (LOC) did not adapt to repeated objects in the context of a changing background. The LOC could be activated, however, when objects were presented without a background. Moreover, the adaptation deficit for objects viewed against backgrounds was reversed when elderly subjects were asked to attend to objects while viewing these complex pictures. These findings suggest that the elderly have difficulty with simultaneous processing of objects and backgrounds that, in turn, could contribute to deficient contextual binding.