Pharmit (http://pharmit.csb.pitt.edu) is an open-source online resource that allows users to interactively search libraries of millions compounds as part of a structure-based drug discovery workflow. Here we describe the systems-level implementation decisions made in designing Pharmit that, when combined with novel sub-linear time search algorithms, allow it to screen millions of molecules in seconds. The key concepts are to maximize parallelism while minimizing intra-thread communication, optimize data layout for sequential processing, and efficiently manage memory allocation. We describe how these concepts are applied to the cheminformatic data inherent to Pharmit and discuss limitations and possible future directions.