van der Waals (vdW) heterostructures are a central focus of materials science and condensed matter physics due to the novel physical phenomena and properties obtained by precisely stacking heterogeneous atomically thin layers. vdW heterostructures are expected to allow for the coherent manipulation of THz lattice vibrations and hence heat conduction due to the ability to precisely control chemical composition at the atomic scale, but little work has focused on thermal transport in these materials. Here, we report an ab initio study of thermal transport in vdW superlattices consisting of alternating transition metal dichalcogenide atomic layers. Our calculations show that the lattice vibrational spectrum and scattering rates can be precisely manipulated by the choice of each atomically thin layer, resulting in materials with novel properties such as large thermal anisotropies approaching 200 and ultralow cross-plane thermal conductivities comparable to those of amorphous materials. Our work demonstrates how coherent manipulation of phonons in vdW superlattices can expand the property space beyond that occupied by natural materials and suggests an experimental route to realize these properties.