Recent advances in electromagnetic nonreciprocity raise the question of how to engineer the nonreciprocal electromagnetic response with geometrical approaches. In this Letter, we examine this problem by introducing generalized electromagnetic continua consisting structured points, which carry extra degrees of freedom over coordinate transformation. We show that general nonreciprocal media have a unique time-varying Riemannian metric structure with local spinning components. It is demonstrated that the nonreciprocity can be alternatively identified as the torsion tensor of a Riemann-Cartan space, which could provide analytic expressions for the magneto-optical effect and the axionic magnetoelectric coupling. Our theory not only gives a deeper insight into the fundamental understanding of electromagnetic nonreciprocity but also provides a practical principle to geometrically design nonreciprocal devices through frame transformation.