Magnetic particle imaging(MPI) is a tracer-based tomographic imaging technique that allows the concentration of magnetic nanoparticles to be determined with high spatio-temporal resolution. System Matrix(SM)-based methods implement MPI through optimizing a under-/over-determined linear system constrainted the knowledgebased priors, and provide high imaging accuracy. However, the model-based SM acquired methods suffers from a non-negligible modeling error, while the measurement-based acquisition is time-consuming and often necessitates frequent repetition to ensure accuracy. In this paper, we introduced an anisotropic prior-guided transformer generative adversarial network(APT-GAN) for high-precision system Matrix generation in MPI. Specifically, a two-step strategy was adopted to eliminat the need for the repetitive measurement calibration process and ensure the generated SMs are not only realistic but also physically plausible. Fisrtly, we constructed the SM prior by improved anisotropy equilibrium model, then we developed the APT-GAN leveraging prior knowledge, featuring a generator with multiple cascaded generative transformer(GT) modules that include attention and feedforward mechanisms, and a discriminator based on an enhanced VGG architecture with integrated AvgPool2d layers. Experiments results demonstrate that our proposed method outperforms existing magnetization model in SM generating and generates high-quality MPI images.