We investigated whether individuals who are good at recognizing previously presented items are also good at recognizing the context in which items were presented. We focused specifically on whether the relation between item recognition and context recognition abilities differs in younger and older adults. It has been hypothesized that context memory declines more rapidly in older adults due to an age-related deficit in associative binding or recollection. To test this hypothesis, younger and older adults were asked to remember lists of names and objects, as well as the context (i.e. their size, location, and color) that accompanied those items. Following presentation of each list, recognition tests for items and context were administered. Confirmatory factor analysis (CFA) models analyzing both item and context scores together provided no evidence of separate factors for item and context memory. Instead, the best-fitting model separated performance by item-type, regardless of context, and no differences were found in the structure of these abilities in younger and older adults. These findings are consistent with the limited previous latent variable research on context memory in aging suggesting that there is no context recognition memory ability separable from item memory in younger nor older adults. Instead, individual differences in recognition memory abilities may be specific to the domain of the studied stimulus.