To obtain a structural model of a macromolecular assembly by single-particle EM, a large number of particle images need to be collected, aligned, clustered, averaged, and finally assembled via reconstruction into a 3D density map. This process is limited by the number and quality of the particle images, the accuracy of the initial model, and the compositional and conformational heterogeneity. Here, we describe a structure determination method that avoids the reconstruction procedure. The atomic structures of the individual complex components are assembled by optimizing a match against 2D EM class-average images, an excluded volume criterion, geometric complementarity, and optional restraints from proteomics and chemical cross-linking experiments. The optimization relies on a simulated annealing Monte Carlo search and a divide-and-conquer message-passing algorithm. Using simulated and experimentally determined EM class averages for 12 and 4 protein assemblies, respectively, we show that a few class averages can indeed result in accurate models for complexes of as many as five subunits. Thus, integrative structural biology can now benefit from the relative ease with which the EM class averages are determined.