The demographic history of human populations in North Africa has been characterized by complex processes of admixture and isolation that have modeled its current gene pool. Diverse genetic ancestral components with different origins (autochthonous, European, Middle Eastern, and sub-Saharan) and genetic heterogeneity in the region have been described. In this complex genetic landscape, Algeria, the largest country in Africa, has been poorly covered, with most of the studies using a single Algerian sample. In order to evaluate the genetic heterogeneity of Algeria, Y-chromosome, mtDNA and autosomal genome-wide makers have been analyzed in several Berber- and Arab-speaking groups. Our results show that the genetic heterogeneity found in Algeria is not correlated with geography or linguistics, challenging the idea of Berber groups being genetically isolated and Arab groups open to gene flow. In addition, we have found that external sources of gene flow into North Africa have been carried more often by females than males, while the North African autochthonous component is more frequent in paternally transmitted genome regions. Our results highlight the different demographic history revealed by different markers and urge to be cautious when deriving general conclusions from partial genomic information or from single samples as representatives of the total population of a region.