Three-dimensional (3D) culture models, such as organoids, are flexible systems to interrogate cellular growth and morphology, multicellular spatial architecture, and cell interactions in response to drug treatment. However, new computational methods to segment and analyze 3D models at cellular resolution with sufficiently high throughput are needed to realize these possibilities. Here we report Cellos (Cell and Organoid Segmentation), an accurate, high throughput image analysis pipeline for 3D organoid and nuclear segmentation analysis. Cellos segments organoids in 3D using classical algorithms and segments nuclei using a Stardist-3D convolutional neural network which we trained on a manually annotated dataset of 3,862 cells from 36 organoids confocally imaged at 5 μm z-resolution. To evaluate the capabilities of Cellos we then analyzed 74,450 organoids with 1.65 million cells, from multiple experiments on triple negative breast cancer organoids containing clonal mixtures with complex cisplatin sensitivities. Cellos was able to accurately distinguish ratios of distinct fluorescently labelled cell populations in organoids, with ≤3% deviation from the seeding ratios in each well and was effective for both fluorescently labelled nuclei and independent DAPI stained datasets. Cellos was able to recapitulate traditional luminescence-based drug response quantifications by analyzing 3D images, including parallel analysis of multiple cancer clones in the same well. Moreover, Cellos was able to identify organoid and nuclear morphology feature changes associated with treatment. Finally, Cellos enables 3D analysis of cell spatial relationships, which we used to detect ecological affinity between cancer cells beyond what arises from local cell division or organoid composition. Cellos provides powerful tools to perform high throughput analysis for pharmacological testing and biological investigation of organoids based on 3D imaging.