We propose a double-channel angular-multiplexing polarization holographic imaging system with common-path and off-axis configurations. In the system, its input plane is spatially divided into three windows: an object window and two reference windows, and two orthogonal linear polarizers are attached, respectively, on the two reference windows; a two-dimensional cross grating is inserted between the input and output planes of the system. Thus the object beam passing through the object window and the two orthogonal polarized reference beams passing through the two reference windows can overlap each other at the output plane of the system and form a double-channel angular-multiplexing polarization hologram (DC-AM-PH). Using this system, the complex amplitude distributions of two orthogonal polarized components from an object can be recorded and reconstructed by one single-shot DC-AM-PH at the same time. Theoretical analysis and experimental results demonstrated that the system can be used to measure the Jones matrix parameters of polarization-sensitive or birefringent materials.