Herbal medicine, including traditional Chinese medicine, has been used for the prevention, treatment, and cure of disorders or diseases for centuries. In addition to being used directly as therapeutic agents, medicinal plants are also important sources for pharmacological drug research and development. With the increasing consumption of herbal products intended to promote better health, it is extremely important to assure the safety and quality of herbal preparations. However, under current regulation surveillance, herbal preparations may not meet expectations in safety, quality, and efficacy. The challenge is how to assure the safety and quality of herbal products for consumers. It is the responsibility of producers to minimize hazardous contamination and additives during cultivation, harvesting, handling, processing, storage, and distribution. This article reviews the current safety obstacles that have been involved in traditional Chinese herbal medicine preparations with examples of popular herbs. Approaches to improve the safety of traditional Chinese medicine are proposed.