Accurate shoulder joint angle estimation is crucial for analyzing joint kinematics and kinetics across a spectrum of movement applications including in athletic performance evaluation, injury prevention, and rehabilitation. However, accurate IMU-based shoulder angle estimation is challenging and the specific influence of key error factors on shoulder angle estimation is unclear. We thus propose an analytical model based on quaternions and rotation vectors that decouples and quantifies the effects of two key error factors, namely sensor-to-segment misalignment and sensor orientation estimation error, on shoulder joint rotation error. To validate this model, we conducted experiments involving twenty-five subjects who performed five activities: yoga, golf, swimming, dance, and badminton. Results showed that improving sensor-to-segment misalignment along the segment's extension/flexion dimension had the most significant impact in reducing the magnitude of shoulder joint rotation error. Specifically, a 1° improvement in thorax and upper arm calibration resulted in a reduction of 0.40° and 0.57° in error magnitude. In comparison, improving IMU heading estimation was only roughly half as effective (0.23° per 1°). This study clarifies the relationship between shoulder angle estimation error and its contributing factors, and identifies effective strategies for improving these error factors. These findings have significant implications for enhancing the accuracy of IMU-based shoulder angle estimation, thereby facilitating advancements in IMU-based upper limb rehabilitation, human-machine interaction, and athletic performance evaluation.