Our aim was to develop an accurate multispectral tissue segmentation method based on 3D feature maps. We utilized proton density (PD), T2-weighted fast spin-echo (FSE), and T1-weighted spin-echo images as inputs for segmentation. Phantom constructs, cadaver brains, an animal brain tumor model and both normal human brains and those from patients with either multiple sclerosis (MS) or primary brain tumors were analyzed with this technique. Initially, misregistration, RF inhomogeneity and image noise problems were addressed. Next, a qualified observer identified samples representing the tissues of interest. Finally, k-nearest neighbor algorithm (k-NN) was utilized to create a stack of color-coded segmented images. The inclusion of T1 based images, as a third input, produced significant improvement in the delineation of tissues. In MS, our 3D technique was found to be far superior to that based on any combination of 2D feature maps (P < 0.001). We identified at least two distinctly different classes of lesions within the same MS plaque, representing different stages of the disease process. Further, we obtained the regional distribution of MS lesion burden and followed its changes over time. Neuropsychological aberrations were the clinical counterpart of the structural changes detected in segmentation. We could also delineate the margins of benign brain tumors. In malignant tumors, up to four abnormal tissues were identified: 1) a solid tumor core, 2) a cystic component, 3) edema in the white matter, and 4) areas of necrosis and hemorrhage. Subsequent neurosurgical exploration confirmed the distribution of tissues as predicted by this analysis.