On a 2-Relative Entropy

Entropy (Basel). 2021 Dec 31;24(1):74. doi: 10.3390/e24010074.

Abstract

We construct a 2-categorical extension of the relative entropy functor of Baez and Fritz, and show that our construction is functorial with respect to vertical morphisms. Moreover, we show such a '2-relative entropy' satisfies natural 2-categorial analogues of convex linearity, vanishing under optimal hypotheses, and lower semicontinuity. While relative entropy is a relative measure of information between probability distributions, we view our construction as a relative measure of information between channels.

Keywords: 2-category; Bayesian inference; discrete memoryless channel; functor; information theory; relative entropy; synthetic probability.