Pay-for-performance programs are one strategy used by health plans to improve the efficiency and quality of care delivered to beneficiaries. Under such programs, providers are often compared against their peers in order to win bonuses or face penalties in payment. Yet luck has the potential to affect performance assessment through randomness in the sorting of patients among providers or through random events during the evaluation period. To investigate the impact luck can have on the assessment of performance, we investigated its role in assigning penalties under Medicare's Hospital Readmissions Reduction Policy (HRRP), a program that penalizes hospitals with excess readmissions. We performed simulations that estimated program hospitals' 2015 readmission penalties in 1,000 different hypothetical fiscal years. These hypothetical fiscal years were created by: (a) randomly varying which patients were admitted to each hospital and (b) randomly varying the readmission status of discharged patients. We found significant differences in penalty sizes and probability of penalty across hypothetical fiscal years, signifying the importance of luck in readmission performance under the HRRP. Nearly all of the impact from luck arose from events occurring after hospital discharge. Luck played a smaller role in determining penalties for hospitals with more beds, teaching hospitals, and safety-net hospitals.