The purpose of this study was to evaluate the rate of change (RoC) in the size of the lumpectomy cavity (LC) before and during breast radiotherapy (RT) using cone-beam computed tomography (CBCT), relative to the initial LC volume at CT simulation (CTVLC) and timing from surgery. A prospective institutional review board-approved study included 26 patients undergoing breast RT: 20 whole breast irradiation (WBI) patients and six partial breast irradiation (PBI) patients, with surgical clips outlining the LC. The patients underwent CT simulation (CTsim) followed by five CBCTs during RT, once daily for PBI and once weekly for WBI. The distance between surgical clips and their centroid (D) acted as a surrogate for LC size. The RoC of the LC size, defined as the percentage change of D between two scans divided by the time interval in days between the scans, was calculated before (CTsim to CBCT1) and during RT (CBCT1 to CBCT5). The mean RoC of D for all patients before starting RT was -0.25%/day (range, -1.3 to 1.4) and for WBI patients during RT was -0.15%/day (range, -0.45 to 0.40). Stratified by median CTVLC, the RoC before RT for large CTVLC group (≥ 25.7 cc) was 15 times higher (-0.47%/day) than for small CTVLC group (< 25.7 cc) (-0.03%/day), p = 0.06. For patients undergoing CTsim < 42 days from surgery, the RoC before RT was -0.43%/day compared to -0.07%/day for patients undergoing CTsim ≥ 42 days from surgery, p = 0.12. For breast cancer RT, the rate of change of the LC is affected by the initial cavity size and the timing from surgery. Resimulation closer to the time of boost treatment should be considered in patients who are initially simulated within six weeks of surgery and/or with large CTVLC.