Due to the complexity of terrain in natural environments, the soft exoskeleton cannot adaptively adjust parameters to achieve the optimal performance. To this end, a design for a soft exoskeleton assistive force parameter optimization method on multi-terrains is presented in this paper. Firstly, the core control parameters are determined by analyzing the system's motion dynamics. Then, the collected data from inertial measurement unit (IMU) is transferred to the convolutional neural network (CNN) to recognize the certain terrain. In the meanwhile, the control parameters corresponding to the different terrains are optimized by the Bayesian algorithm. Finally, the optimal assistive force parameters are transferred to the system for improving the performance of the soft exoskeleton. The experiment is conducted on three participants, wherein the net metabolic rates of the subjects are compared with and without the assistive force. The final results show that the metabolic rates of the subjects reduce the average value of 19.6% on flat ground, 11.6% on walking uphill, and 12.7% on walking upstairs. The experimental results confirm the effectiveness of the proposed method.