Physical activity may have a role in many cancers, but little is known about its effect on esophageal and gastric adenocarcinoma risk. We investigated occupational physical activity and esophageal and gastric adenocarcinoma risk in a population-based, case-control study including 212 esophageal, 264 gastric cardia and 389 distal gastric cancer cases, and 1,330 controls in Los Angeles County. Lifetime occupational histories were obtained during in-person interviews, and total lifetime occupational activity (Total Activity Index) was calculated using US Census job codes classified as sedentary, or moderately or highly physically active. Average Annual Activity Index was a per-year Total Activity Index counterpart. Unconditional logistic regression was used to calculate odds ratios, 95% confidence intervals and trend tests adjusting for gender, race, age, birthplace, education, smoking, body mass index (BMI) and number of years worked. Esophageal adenocarcinoma risk tended to decrease with increasing Total Activity Index (OR = 0.67, 95% CI = 0.38,1.19 for highest versus lowest quartile), but neither gastric cardia nor distal gastric cancer was associated with the Total Activity Index. This inverse association held for esophageal adenocarcinoma (OR = 0.61, 95% CI = 0.38,0.99 for highest vs. lowest quartile) and modest associations were observed for gastric cardia (OR = 0.76, 95% CI = 0.49,1.18) and distal gastric cancer (OR = 0.77, 95% CI = 0.52,1.14) when based on Average Annual Activity Index before age 65 years. Analyses stratified by gender, race, age, BMI, education and years worked provided similar results. We found a modest protective effect of Total Activity Index on esophageal adenocarcinoma. Future studies with more complete information on occupational and recreational physical activity are needed to confirm and further investigate the suggested protective effect of physical activity on these tumor types.