Infrared four-wave mixing experiments performed upon deuterated amorphous silicon layers (a-Si:D) reveal profound differences in the dynamics of Si-D stretch vibrations compared to those of analogous Si-H vibrational modes in hydrogenated amorphous silicon (a-Si:H). Remarkably, transient-grating measurements of the population decay rate of the Si-D vibrations show single-exponential decay directly into collective modes of the a-Si host, bypassing the local bending modes of the defect into which the Si-H vibrations decay. Photon-echo measurements of the vibrational dephasing suggest at low temperature contributions from TO nonequilibrium phonons and at elevated temperatures elastic phonon scattering of TA phonons.