Staphylococcus aureus is a commensal and pathogen of several mammalian species, particularly humans and cattle. We aimed to (i) identify S. aureus genes associated with host specificity, (ii) determine the relatedness of human and animal isolates, and (iii) identify whether human and animal isolates typically exchanged mobile genetic elements encoding virulence and resistance genes. Using a well-validated seven-strain S. aureus microarray, we compared 56 UK S. aureus isolates that caused infection in cows, horses, goats, sheep and a camel with 161 human S. aureus isolates from healthy carriers and community acquired infections in the UK. We had previously shown that human isolates are clustered into ten dominant and a few minor lineages, each with unique combinations of surface proteins predicted to bind to human proteins. We found that the animal-associated S. aureus clustered into ten lineages, with 61 % assigned to four lineages, ST151, ST771, ST130 and ST873, that were unique to animals. The majority of bovine mastitis was caused by isolates of lineage ST151, ST771 and ST97, but a few human lineages also caused mastitis. S. aureus isolated from horses were more likely to cluster into human-associated lineages, with 54 % of horse-associated S. aureus assigned to the human clusters CC1, CC8 and CC22; along with the presence of some multi-drug resistant strains, this suggests a human origin. This is the most comprehensive genetic comparison of human versus animal S. aureus isolates conducted, and because we used a whole-genome approach we could estimate the key genes with the greatest variability that are associated with host specificity. Several genes conserved in all human isolates were variable or missing in one or more animal lineages, including the well-characterized lineage specific genes fnbA, fnbB and coa. Interestingly, genes carried on mobile genetic elements (MGEs) such as chp, scn and sak were less common in animal S. aureus isolates, and bap was not found. There was a lot of MGE variation within lineages, and some evidence that exchange of MGEs such as bacteriophage and pathogenicity islands between animal and human lineages is feasible, but there was less evidence of antibiotic resistance gene transfer on the staphylococcal cassette chromosomes (SCC) or plasmids. Surprisingly, animal lineages are closely related to human lineages and only a handful of genes or gene combinations may be responsible for host specificity.