We describe and test theoretical principles for consistent integration of experimental and ab initio data from diverse sources into a single statistical mechanical model. The approach is based on the recently introduced concept of statistical distance between partition functions, uses a simple vector algebra formalism to describe measurement outcomes and coarse-graining operations, and takes advantage of thermodynamic perturbation expressions for fast exploration of the model parameter space. The methodology is demonstrated on a combination of thermodynamic, structural, spectroscopic, and imaging pseudoexperimental data along with ab initio-type trajectories, which are incorporated into models describing the behavior of a near-critical fluid, liquid water, thin-film mixed oxides, and binary alloys. We evaluate how different target data constrain the model parameters and how the uncertainty associated with incomplete target information and limited sampling of the system's phase space might influence the choice of optimal parameters.