Fluorescence molecular tomography (FMT) is a highly sensitive and noninvasive imaging modality for three-dimensional visualization of fluorescence probe distribution in small animals. However, the simplified photon propagation model and ill-posed inverse problem limit the improvement of FMT reconstruction. In this work, we proposed a novel K-nearest neighbor based locally connected (KNN-LC) network to improve the performance of morphological reconstruction in FMT. It directly builds the inverse process of photon transmission by learning the mapping relation between the surface photon intensity and the distribution of fluorescent source. KNN-LC network cascades a fully connected (FC) sub-network with a locally connected (LC) sub-network, where the FC part provides a coarse reconstruction result and LC part fine-tunes the morphological quality of reconstructed result. To assess the performance of our proposed network, we implemented both numerical simulation and in vivo studies. Furthermore, split Bregman-resolved total variation (SBRTV) regularization method and inverse problem simulation (IPS) method were utilized as baselines in all comparisons. The results demonstrated that KNN-LC network achieved accurate reconstruction in both source localization and morphology recovery in a short time. This promoted the in vivo application of FMT for visualizing the distribution of biomarkers inside biological tissue.