Accurate and efficient patient dose calculations are essential for treatment planning in magnetic resonance imaging guided radiotherapy (MRIgRT). Achieving reasonable performance for a space-angle discontinuous finite element method (DFEM) grid based Boltzmann solver (GBBS) with magnetic fields for clinical MRIgRT applications largely depends on how the transport sweep is orchestrated. Compared to classical Discrete Ordinates, DFEM in angle introduces increased angular degrees of freedom and eliminates ray-effect artifacts. However, the inclusion of magnetic fields introduces additional serial dependencies such that parallelization of the space-angle transport sweeps becomes more challenging. Novel techniques for the transport sweep and right-hand source assembly are developed, predicated on limiting the number of bulk material densities modeled in the transport sweep scatter calculations. Specifically, k-means clustering is used to assign sub-intervals of mass-density for each spatial element to execute the scatter-dose calculations using batched multiplication by pre-inverted transport sweep matrices. This is shown to be two orders of magnitude more efficient than solving each elemental system individually at runtime. Even with discrete material densities used in the transport sweep scatter calculations, accuracy is maintained by optimizing the material density assignments using k-means clustering, and by performing the primary photon fluence calculations (ray-tracing) using the underlying continuous density of the computed tomography (CT) image. In the presence of 0.5 T parallel and 1.5 T perpendicular magnetic fields, this approach demonstrates high levels of accuracy with gamma 1%/1 mm passing rates exceeding 94% across a range of anatomical sites compared to GEANT4 Monte Carlo dose calculations which used continuous densities. This deterministic GBBS approach maintains unconditional stability, produces no ray-effect artifacts, and has the benefit of no statistical uncertainty. Runtime on a non-parallelized Matlab implementation averaged 10 min per beam averaging 80 000 spatial elements, paving way for future development based on this algorithmically efficient paradigm.