Virtual haptic simulation of minimally invasive arthroscopic surgery becomes an extremely important training tool that allows the medical students to acquire necessary motor skills before they can approach actual patients. Normally, 3D simulation of the interior of a joint requires significant efforts from the software developers but yet remains not always photo realistic. In this paper, we propose a pioneering approach of using augmented real arthroscopic images for realistic and immersive image-driven visualization and haptic interaction within the surgical field as if it were actual three-dimensional scene where body parts displayed in the image act and feel as real 3D objects rather than their images.