Mass spectrometry imaging (MSI) has become an important tool for 2D profiling of biological tissues, allowing for the visualization of individual compound distributions in the sample. Based on this information, it is possible to investigate the molecular organization within any particular tissue and detect abnormal regions (such as tumor regions) and many other biologically relevant phenomena. However, the large number of compounds present in the spectra hinders the productive analysis of large MSI datasets when utilizing standard tools. The heterogeneity of samples makes exploratory visualization (a presentation of the general idea of the molecular and structural organization of the inspected tissues) challenging. Here, we explore the application of various dimensionality reduction techniques that have been used extensively in the visualization of hyperspectral images and the MSI data specifically, such as principal component analysis, independent component analysis, non-negative matrix factorization, t-distributed stochastic neighbor embedding, and uniform manifold approximation and projection. Further, we propose a new approach based on a combination of structure preserving visualization with nonlinear manifold embedding of normalized spectral data. This way, we aim to preserve as much spatially overlapping signals as possible while augmenting them with information on compositional (spectral) variation. The proposed approach can be used for exploratory visualization of MSI datasets without prior deep chemical or histological knowledge of the sample. Thus, different datasets can be visually compared employing the proposed method. The proposed approach allowed for the clear visualization of the molecular layer, granular layer, and white matter in chimpanzee and macaque cerebellum slices.