We present a multi-scale approach of tumor modeling in order to predict its evolution during radiotherapy. Within this context we focus on three different scales of tumor modeling: microscopic (individual cells in a voxel), mesoscopic (population of cells in a voxel) and macroscopic (whole tumor), with transition interfaces between these three scales. At the cellular level, the description is based on phase transfer probabilities in the cellular cycle. At the mesoscopic scale we represent populations of cells according to different stages in a cell cycle. Finally, at the macroscopic scale, the tumor description is based on the use of FDG PET image voxels. These three scales exist naturally: biological data are collected at the macroscopic scale, but the pathological behavior of the tumor is based on an abnormal cell-cycle at the microscopic scale. On the other hand, the introduction of a mesoscopic scale is essential in order to reduce the gap between the two extreme, in terms of resolution, description levels. It also reduces the computational burden of simulating a large number of individual cells. As an application of the proposed multi-scale model, we simulate the effect of oxygen on tumor evolution during radiotherapy. Two consecutive FDG PET images of 17 rectal cancer patients undergoing radiotherapy are used to simulate the tumor evolution during treatment. The simulated results are compared with those obtained on a third FDG PET image acquired two weeks after the beginning of the treatment.