In their lead paper, Huerta, Casebeer and VanderPlaat argue that there are several key forces driving the development of health services delivery (HSD) networks, and propose a series of paradoxes and propositions to initiate this timely and essential dialogue. Ultimately, they submit that networks are likely to remain within the healthcare system to build system capacity and drive integration. Given this, they challenge us to further the dialogue and investigate these networks. While this peer commentary shares many of the lead author's perspectives, the generic nature of the discussion does not bring us to the relative complexities revealed in some HSD network practices. A Canadian child health network lens is used to re-examine the lead paper's conceptualization of network typologies and the proposed paradox of structure. We combine network practice and academic expertise to highlight the structural, governance and leadership tensions between traditional hierarchical public service organizations and the non-hierarchical nature of inter-organizational networks. Child health network leaders and members must examine and work with the challenges associated with importing traditional organizational cultures into an inter-organizationally networked context, while simultaneously maintaining these dual (or duelling) cultures.