The relative contribution of dominant and non-dominant language networks to recovery from aphasia is a matter of debate. We assessed with functional magnetic resonance imaging (fMRI) to what extent the left and right hemispheres are associated with recovery from aphasia after stroke. fMRI with three language tasks was performed in 13 aphasic stroke patients and in 13 healthy subjects. Severity of aphasia was examined within 2 months after stroke and after at least 1 year. Recovery of naming ability and scores on the Token Test were correlated with data from fMRI in the chronic phase. A breath-hold paradigm was used to investigate hemodynamic responsiveness. Overall language performance in the chronic phase correlated with higher relative activation of left compared to right perisylvian areas. Recovery of naming ability was positively correlated with activation in the left inferior frontal gyrus (IFG) for semantic decision and verb generation. Recovery on the Token Test was positively correlated with activation in both left and right IFG during semantic decision and verb generation. Hemodynamic response to the breath-hold task was similar in patients and controls. Our study suggests that in the chronic stage after stroke left IFG activity is associated with improvement of picture naming and sentence comprehension, whereas activity in the right IFG may reflect up-regulation of non-linguistic cognitive processing. Altered hemodynamic responsiveness seems an unlikely confounder in the interpretations of fMRI results.