It has been suggested that yoga has a positive effect on low back pain and function. The objective of this systematic review was to assess the effectiveness of yoga as a treatment option for low back pain. Seven databases were searched from their inception to March 2011. Randomized clinical trials were considered if they investigated yoga in patients with low back pain and if they assessed pain as an outcome measure. The selection of studies, data extraction and validation were performed independently by two reviewers. Seven randomized controlled clinical trials (RCTs) met the inclusion criteria. Their methodological quality ranged between 2 and 4 on the Jadad scale. Five RCTs suggested that yoga leads to a significantly greater reduction in low back pain than usual care, education or conventional therapeutic exercises. Two RCTs showed no between-group differences. It is concluded that yoga has the potential to alleviate low back pain. However, any definitive claims should be treated with caution.