We characterize the impact of the modulator material on chirp, digital signal processing (DSP) algorithms and system-level performance in coherent digital optical links. We compare theoretically, in simulations and experimentally the lithium niobate (LiNbO3), indium phosphide (InP) and silicon (Si) integrated platforms. Distortions to vector diagrams are traced back to modulation physics, and are interpreted as quadrature crosstalk. In a back-to-back BPSK setup with an RF drive signal amplitude of 1.5Vπ, we measure chirp parameters α of ~0, 0.10 and 0.06 and error vector magnitude EVMRMS of 5.3%, 9.4% and 10.6% with the LiNbO3, InP and Si modulators respectively. Both α and EVMRMS are found to scale with the RF signal amplitude. In simulations, using a polynomial fit over a sinusoidal fit when pre-compensating the Si modulator transfer function slightly improves EVM (-0.6%). We also show that Si-related distortions can impact the efficiency of symbol timing recovery. In conclusion, phase and attenuation distortions in InP and Si modulators deteriorate the overall performance in coherent links, and cannot be neglected for large RF signal amplitudes. These results will benefit the optical communications community.