This paper proposes a method to match diffusion tensor magnetic resonance images (DT-MRIs) through the large deformation diffeomorphic metric mapping of vector fields, focusing on the fiber orientations, considered as unit vector fields on the image volume. We study a suitable action of diffeomorphisms on such vector fields, and provide an extension of the Large Deformation Diffeomorphic Metric Mapping framework to this type of dataset, resulting in optimizing for geodesics on the space of diffeomorphisms connecting two images. Existence of the minimizers under smoothness assumptions on the compared vector fields is proved, and coarse to fine hierarchical strategies are detailed, to reduce both ambiguities and computation load. This is illustrated by numerical experiments on DT-MRI heart images.