E. coli has two-component systems composed of histidine kinase proteins and response regulator proteins. For a given extracellular stimulus, a histidine kinase senses the stimulus, autophosphorylates and then passes the phosphates to the cognate response regulators. The histidine kinase in an orthodox two-component system has only one histidine domain where the autophosphorylation occurs, but a histidine kinase in some unusual two-component systems (unorthodox two-component systems) has two histidine domains and one aspartate domain. So, the unorthodox two-component systems have more complex phosphorelay mechanisms than orthodox two-component systems. In general, the two-component systems are required to promptly respond to external stimuli for survival of E. coli. In this respect, the complex multi-step phosphorelay mechanism seems to be disadvantageous, but there are several unorthodox two-component systems in E. coli. In this paper, we investigate the reason why such unorthodox two-component systems are present in E. coli. For this purpose, we have developed simplified mathematical models of both orthodox and unorthodox two-component systems and analyzed their dynamical characteristics through extensive computer simulations. We have finally revealed that the unorthodox two-component systems realize ultrasensitive responses to external stimuli and also more robust responses to noises than the orthodox two-component systems.