Medical imaging research and clinical applications usually require combination and integration of various techniques ranging from image processing and analysis to realistic visualization to user-friendly interaction. Researchers with different backgrounds coming from diverse areas have been using numerous types of hardware, software, and environments to obtain their results. We also observe that students often build their tools from scratch resulting in redundant work. A generic and flexible medical imaging and visualization toolkit would be helpful in medical research and educational institutes to reduce redundant development work and hence increase research efficiency. This paper presents our experience in developing a Medical Imaging and Visualization Toolkit (BIL-kit) that is a set of comprehensive libraries as well as a number of interactive tools. The BIL-kit covers a wide range of fundamental functions from image conversion and transformation, image segmentation, and analysis to geometric model generation and manipulation, all the way up to 3D visualization and interactive simulation. The toolkit design and implementation emphasize the reusability and flexibility. BIL-kit is implemented in the Java language so that it works in hybrid and dynamic research and educational environments. This also allows the toolkit to extend its usage for the development of Web-based applications. Several BIL-kit-based tools and applications are presented including image converter, image processor, general anatomy model simulator, vascular modeling environment, and volume viewer. BIL-kit is a suitable platform for researchers and students to develop visualization and simulation prototypes, and it can also be used for the development of clinical applications.