To summarize safety data such as clinical adverse experiences in clinical trials with a moderate to long-term follow-up, we may use a measurement which accounts for the potential differences in the follow-up duration between treatment groups. The incidence rate, which uses the total person-time follow-up in a treatment group as the denominator, is one of these measures. When treatment comparisons are based on the difference of the incidence rates, it is of interest to construct confidence intervals for the rate differences. In this paper, we first discuss the assumptions and scenarios in which the exposure adjusted incidence rate may be appropriate. Then we review several methods of calculating confidence intervals for the difference of two incidence rates assuming that the number of events come from a Poisson distribution. The methods considered include Wald's method, the two-by-two table method, the Miettinen and Nurminen (MN) method, and the conditional MN method. For all but the MN method, explicit confidence intervals can be obtained. For the MN method, some numerical iterations are required. The properties of these methods are evaluated by simulations. The results show that the MN method outperforms the other three in terms of the coverage of the confidence interval, especially when the rates and therefore the number of events are small. Lastly, data from a clinical study are used to demonstrate the application of the methods.