We present a general model using supervised learning and MAP estimation that is capable of performing many common tasks in automated skin lesion diagnosis. We apply our model to segment skin lesions, detect occluding hair, and identify the dermoscopic structure pigment network. Quantitative results are presented for segmentation and hair detection and are competitive when compared to other specialized methods. Additionally, we leverage the probabilistic nature of the model to produce receiver operating characteristic curves, show compelling visualizations of pigment networks, and provide confidence intervals on segmentations.