Agars and carrageenans are 1,3-alpha-1,4-beta-galactans from the cell walls of red algae, substituted by zero (agarose), one (kappa-), two (iota-), or three (lambda-carrageenan) sulfate groups per disaccharidic monomer. Agars, kappa-, and iota-carrageenans auto-associate into crystalline fibers and are well known for their gelling properties, used in a variety of laboratory and industrial applications. These sulfated galactans constitute a crucial carbon source for a number of marine bacteria. These microorganisms secrete glycoside hydrolases specific for these polyanionic, insoluble polysaccharides, agarases and carrageenases. This article reviews the microorganisms involved in the degradation of agars and carrageenans, in their environmental and taxonomic diversity. We also present an overview on the biochemistry of the different families of galactanases. The structure-function relationships of the family GH16 beta-agarases and kappa-caraggeenases and of the family GH82 iota-carrageenases are discussed in more details. In particular, we examine how the active site topologies of these glycoside hydrolases influence their mode of action in heterogeneous phase. Finally, we discuss the next challenges in the basic and applied field of the galactans of red algae and of their related degrading microorganisms.