We investigate the finite-size scaling of the boundary quantum geometric tensor (QGT) numerically close to the Anderson localization transition in the presence of small external magnetic fields. The QGT exhibits universal scaling and reveals the crossover between the orthogonal and unitary critical states in weak random magnetic fields. The flow of the QGT near the critical points determines the critical exponents. Critical distributions of the QGT are universal and exhibit a remarkable isotropy even in a homogeneous magnetic field. We predict universal and isotropic Hall conductance fluctuations at the metal-insulator transition in an external magnetic field.