The current trend in high throughput screening is the utilization of more complex model systems that mimic both structural and functional properties of cellular processes in vivo. In this context, 3D cell culture models have emerged as effective systems to study tumor initiation and cancer behavior, where colony organization represents distinct phenotypic signatures that enable differentiation of cancer cells in culture using phase imaging and in the absence of clinical markers. If the colony organization can be classified into different phenotypes, it will enable rapid drug screening using phase microscopy. In this paper, we propose a novel method based on locality-constrained dictionary learning for the discrimination of aberrant colony organization in phase images, which encodes original SIFT (Scale-Invariant Feature Transform) features into high dimensional sparse codes with locality-preserving landmark points on the nonlinear manifold, and summarizes the sparse features at various locations and scales through spatial pyramid matching for robust representation. Experimental results demonstrate the significant improvement of performance, compared to the state-of-art in the field.