The lipid composition of cell membranes has increasingly been recognized as playing an important role in the function of various membrane proteins, including G Protein-Coupled Receptors (GPCRs). For instance, experimental and computational evidence has pointed to lipids influencing receptor oligomerization directly, by physically interacting with the receptor, and/or indirectly, by altering the bulk properties of the membrane. While the exact role of oligomerization in the function of class A GPCRs such as the μ-opioid receptor (MOR) is still unclear, insight as to how these receptors oligomerize and the relevance of the lipid environment to this phenomenon is crucial to our understanding of receptor function. To examine the effect of lipids and different MOR conformations on receptor oligomerization we carried out extensive coarse-grained molecular dynamics simulations of crystal structures of inactive and/or activated MOR embedded in an idealized mammalian plasma membrane composed of 63 lipid types asymmetrically distributed across the two leaflets. The results of these simulations point, for the first time, to specific direct and indirect effects of the lipids, as well as the receptor conformation, on the spatio-temporal organization of MOR in the plasma membrane. While sphingomyelin-rich, high-order lipid regions near certain transmembrane (TM) helices of MOR induce an effective long-range attractive force on individual protomers, both long-range lipid order and interface formation are found to be conformation dependent, with a larger number of different interfaces formed by inactive MOR compared to active MOR.