We present two algorithms that extend existing HMM parameter adaptation algorithms (MAP and MLLR) by adapting the HMM structure. This improvement relies on a smart combination of MAP and MLLR with a structure optimization procedure. Our algorithms are semi-supervised: to adapt a given HMM model on new data, they require little labeled data for parameter adaptation and a moderate amount of unlabeled data to estimate the criteria used for HMM structure optimization. Structure optimization is based on state splitting and state merging operations and proceeds so as to optimize either the likelihood or a heuristic criterion. Our algorithms are successfully applied to the recognition of printed characters by adapting the HMM character models of a polyfont printed text recognizer to new fonts. Our experiments involve a total of 1,120,000 real and 3,100,000 synthetic character images and concern a set of 89 HMM models. A comparison of our results with those of state-of-the-art adaptation algorithms (MAP and MLLR) shows a significant increase in the accuracy of character recognition.