Recent research has indicated a relationship between skeletal muscle mass and insulin resistance in patients with type 2 diabetes mellitus (T2DM). However, no study has examined the relationship between skeletal muscle mass and insulin secretion in patients with Japanese T2DM. This study aimed to fill this research gap by investigating the relationship between skeletal muscle mass and clinical parameters of T2DM with special reference to the effect of sex or age on the relationship. We examined 138 consecutive T2DM patients who presented at a single center. Anthropomorphic measurement was conducted and skeletal muscle mass was determined by bioelectrical impedance analysis for calculating skeletal muscle index (SMI) as the ratio of appendicular muscle mass (AMM) to total body weight. Fasting plasma glucose (FPG) and glycosylated hemoglobin (HbA1c) were levels, and values of stimulated C-peptide immunoreactivity (CPR) were determined by glucagon stimulation testing. Statistical analysis showed that AMM was negatively correlated with age in T2DM patients, whereas SMI had no correlation with either FPG or HbA1c levels. On the other hand, SMI was found to be negatively correlated with the log-transformed stimulated CPR values in male patients <65 years (r = -0.40, p < 0.05) and in female patients <65 years (r = -0.40, p < 0.05). The results of multivariate analysis suggest a strong association between the log-transformed stimulated CPR value and SMI. These findings indicate that increased endogenous insulin secretion is associated with lower skeletal muscle mass in T2DM patients who are <65 years of age.