We have prepared two ultracold fermionic atoms in an isolated double-well potential and obtained full control over the quantum state of this system. In particular, we can independently control the interaction strength between the particles, their tunneling rate between the wells and the tilt of the potential. By introducing repulsive (attractive) interparticle interactions we have realized the two-particle analog of a Mott-insulating (charge-density-wave) state. We have also spectroscopically observed how second-order tunneling affects the energy of the system. This work realizes the first step of a bottom-up approach to deterministically create a single-site addressable realization of a ground-state Fermi-Hubbard system.