The 5'-nucleotide of small RNAs associates directly with the MID domain of Argonaute (AGO) proteins. In humans, the identity of the 5'-base is sensed by the MID domain nucleotide specificity loop and regulates the integrity of miRNAs. In Arabidopsis thaliana, the 5'-nucleotide also controls sorting of small RNAs into the appropriate member of the AGO family; however, the structural basis for this mechanism is unknown. Here, we present crystal structures of the MID domain from three Arabidopsis AGOs, AtAGO1, AtAGO2 and AtAGO5, and characterize their interactions with nucleoside monophosphates (NMPs). In AtAGOs, the nucleotide specificity loop also senses the identity of the 5'-nucleotide but uses more diverse modes of recognition owing to the greater complexity of small RNAs found in plants. Binding analyses of these interactions reveal a strong correlation between their affinities and evolutionary conservation.