The aim of this study was to identify factors influencing health care utilization behavior for children with mild or severe disease symptoms in rural Ghana. Between March and September 2008 a cross-sectional health care utilization survey was conducted and 8,715 caregivers were interviewed regarding their intended behavior in case their children had mild or severe fever or diarrhea. To show associations between hospital attendance and further independent factors (e.g. travel distance or socio-economic status) prevalence ratios were calculated for the four disease symptoms. A Poisson regression model was used to control for potential confounding. Frequency of hospital attendance decreased constantly with increasing distance to the health facility. Being enrolled in the national health insurance scheme increased the intention to attend a hospital. The effect of the other factors diminished in the Poisson regression if modeled together with travel distance. The observed associations weakened with increasing severity of symptoms, which indicates that barriers to visit a hospital are less important if children experience a more serious illness. As shown in other studies, travel distance to a health care provider had the strongest effect on health care utilization. Studies to identify local barriers to access health care services are important to inform health policy making as they identify deprived populations with low access to health services and to early treatment.