For any image guided surgery, independently of the technique which is used (navigation, templates, robotics), it is necessary to get a 3D bone surface model from CT or MR images. Such model is used for planning, registration and visualization. We report that graphical representation of patient bony structure and the surgical tools, interconnectively with the tracking device and patient-to-image registration are crucial components in such a system. For Total Shoulder Arthroplasty (TSA), there are many challenges, The most of cases that we are working with are pathological cases such as rheumatoid arthritis, osteoarthritis disease. The CT images of these cases often show a fusion area between the glenoid cavity and the humeral head. They also show severe deformations of the humeral head surface that result in a loss of contours. This fusion area and image quality problems are also amplified by well-known CT-scan artifacts like beam-hardening or partial volume effects. The state of the art shows that several segmentation techniques, applied to CT-Scans of the shoulder, have already been disclosed. Unfortunately, their performances, when used on pathological data, are quite poor [1, 2]. The aim of this paper is to present a new image guided surgery system based on CT scan of the patient and using bony structure recognition, morphological analysis for the operated region and robust image-to-patient registration.