Our aim was to examine how brain imaging in the initial phase of a stroke could predict both acute/subacute as well as chronic spatial neglect. We present the first voxel-wise longitudinal lesion-behaviour mapping study, examining acute/subacute as well as chronic performance in the same individuals. Acute brain imaging (acquired on average 6.2 days post-injury) was used to evaluate neglect symptoms at the initial (mean 12.4 days post-stroke) and the chronic (mean 491 days) phase of the stroke. Chronic neglect was found in about one-third of the patients with acute neglect. Analysis suggests that lesion of the superior and middle temporal gyri predict both acute/subacute as well as chronic neglect. At the subcortical level, the basal ganglia as well as the inferior occipitofrontal fasciculus/extreme capsule appear to play a significant role for both acute/subacute as well as chronic neglect. Beyond, the uncinate fasciculus was critically related to the emergence of chronic spatial neglect. We infer that individuals who experience spatial neglect in the initial phase of the stroke yet do not have injury to these cortical and subcortical structures are likely to recover, and thus have a favourable prognosis.