Traditional measures of biodiversity, such as species richness, usually treat species as being equal. As this is obviously not the case, measuring diversity in terms of features accumulated over evolutionary history provides additional value to theoretical and applied ecology. Several phylogenetic diversity indices exist, but their behaviour has not yet been tested in a comparative framework. We provide a test of ten commonly used phylogenetic diversity indices based on 40 simulated phylogenies of varying topology. We restrict our analysis to a topological fully resolved tree without information on branch lengths and species lists with presence-absence data. A total of 38,000 artificial communities varying in species richness covering 5-95% of the phylogenies were created by random resampling. The indices were evaluated based on their ability to meet a priori defined requirements. No index meets all requirements, but three indices turned out to be more suitable than others under particular conditions. Average taxonomic distinctness (AvTD) and intensive quadratic entropy (J) are calculated by averaging and are, therefore, unbiased by species richness while reflecting phylogeny per se well. However, averaging leads to the violation of set monotonicity, which requires that species extinction cannot increase the index. Total taxonomic distinctness (TTD) sums up distinctiveness values for particular species across the community. It is therefore strongly linked to species richness and reflects phylogeny per se weakly but satisfies set monotonicity. We suggest that AvTD and J are best applied to studies that compare spatially or temporally rather independent communities that potentially vary strongly in their phylogenetic composition-i.e. where set monotonicity is a more negligible issue, but independence of species richness is desired. In contrast, we suggest that TTD be used in studies that compare rather interdependent communities where changes occur more gradually by species extinction or introduction. Calculating AvTD or TTD, depending on the research question, in addition to species richness is strongly recommended.