Light polarization is used in the animal kingdom for communication, navigation, and enhanced scene interpretation and also plays an important role in astronomy, remote sensing, and military applications. To date, there have been few photodetector materials demonstrated to have direct polarization sensitivity, as is usually the case in nature. Here, we report the realization of a carbon-based broadband photodetector, where the polarimetry is intrinsic to the active photodetector material. The detector is based on p-n junctions formed between two macroscopic films of single-wall carbon nanotubes. A responsivity up to ~1 V/W was observed in these devices, with a broadband spectral response spanning the visible to the mid-infrared. This responsivity is about 35 times larger than previous devices without p-n junctions. A combination of experiment and theory is used to demonstrate the photothermoelectric origin of the responsivity and to discuss the performance attributes of such devices.