Small heat shock proteins (sHSPs), as one subclass of molecular chaperones, are important for cells to protect proteins under stress conditions. Unlike the large HSPs (represented by Hsp60 and Hsp70), sHSPs are highly divergent in both primary sequences and oligomeric status, with their evolutionary relationships being unresolved. Here the phylogenetic analysis of a representative 51 sHSPs (covering the six subfamilies: bacterial class A, bacterial class B, archae, fungi, plant, and animal) reveals a close relationship between bacterial class A and animal sHSPs which form an outgroup. Accumulating data indicate that the oligomers from bacterial class A and animal sHSPs appear to exhibit polydispersity, while those from the rest exhibit monodispersity. Together, the close evolutionary relationship and the similarity in oligomeric polydispersity between bacterial class A and animal sHSPs not only suggest a potential evolutionary origin of the latter from the former, but also imply that their oligomeric polydispersity is somehow a property determined by their primary sequences.