Little is known about the precise relationship between energy intake, overweight, sedentary lifestyle, and steps in the colorectal adenoma-carcinoma pathway. We studied these parameters within a case-control study. Patients with adenomas < 10 mm (n = 154) or > 10 mm (n = 208) were compared with polyp-free controls (n = 426) for determining factors associated with adenoma formation, i.e., observed for small and large adenomas, or with adenoma growth only. Colorectal cancer cases (n = 171) were compared with population controls (n = 309) to determine factors specific to the final stage, cancer. Exercise reduced the risk of cancer [odds ratio (OR) = 0.3, 95% confidence interval (CI) = 0.2-0.5 for high vs. low physical activity] but had little influence on adenomas. High energy intake increased the risk of cancer [OR for 5th vs. 1st quintile (OR5) = 1.6, 95% CI = 0.9-2.9, p = 0.02], but not of adenomas. High body mass index (BMI) significantly increased the risk of large adenomas (OR5 = 2.1, 95% CI = 1.2-3.5, p = 0.02 and OR5 = 1.7, 95% CI = 1.0-3.1, p = 0.25) for large and small adenomas vs. polyp-free controls. Neither height nor weight nor BMI influenced the risk of cancer. Results were unmodified when controlling for dietary risk factors and family history. Energy intake, a sedentary lifestyle, and high BMI were independently associated with a high risk of cancer itself or large adenomas, which indicates an effect on promotion of colorectal tumors. These findings suggest that preventive advice regarding these factors should be provided, even late in life, to decrease the risk of colorectal cancer.