Considering the brain as a complex network of interacting dynamical systems offers new insights into higher level brain processes such as memory, planning, and abstract reasoning as well as various types of brain pathophysiology. This viewpoint provides the opportunity to apply new insights in network sciences, such as the discovery of small world and scale free networks, to data on anatomical and functional connectivity in the brain. In this review we start with some background knowledge on the history and recent advances in network theories in general. We emphasize the correlation between the structural properties of networks and the dynamics of these networks. We subsequently demonstrate through evidence from computational studies, in vivo experiments, and functional MRI, EEG and MEG studies in humans, that both the functional and anatomical connectivity of the healthy brain have many features of a small world network, but only to a limited extent of a scale free network. The small world structure of neural networks is hypothesized to reflect an optimal configuration associated with rapid synchronization and information transfer, minimal wiring costs, resilience to certain types of damage, as well as a balance between local processing and global integration. Eventually, we review the current knowledge on the effects of focal and diffuse brain disease on neural network characteristics, and demonstrate increasing evidence that both cognitive and psychiatric disturbances, as well as risk of epileptic seizures, are correlated with (changes in) functional network architectural features.