Highly malignant brain tumors such as glioblastoma multiforme form complex growth patterns in vitro in which invasive cells organize in tenuous branches. Here, we formulate a chemotaxis model for this sort of growth. A key element controlling the pattern is homotype attraction, i.e., the tendency for invasive cells to follow pathways previously explored. We investigate this in two ways: we show that there is an intrinsic instability in the model, which leads to branch formation. We also give a discrete description for the expansion of the invasive zone, and a continuum model for the nutrient supply. The results indicate that both strong heterotype chemotaxis and strong homotype chemoattraction are required for branch formation within the invasive zone. Our model thus can give a way to assess the importance of the various processes, and a way to explore and analyze transitions between different growth regimes.