Quantitative performance and skill assessments are critical for evaluating the progress of surgical residents and the efficacy of different training programs. Current evaluation methods are subjective and potentially unreliable, so there is a need for objective methods to evaluate surgical performance. We identify a feasible method to measure kinematic data in the live operating room setting and to assess the repeatability of an analysis method based on a hierarchical decomposition of surgical tasks. We used an optoelectronic motion analysis system to acquire postural data and tool tip trajectories of one expert surgeon over a period of four months. To assess repeatability of performance measures, we created a hierarchical decomposition diagram describing the procedure in terms of surgical tasks, tool sequences and fundamental tool actions. From the kinematic data, we extracted characteristic measures of individual tool actions and compared these measured distributions using the Kolmogorov-Smirnov statistic. The comparisons of distributions show consistent performance over time by a trained surgeon and little effect from patient variability, and so are likely reliable measures of performance. An expanded set of reliable kinematic measures will form the basis for quantifying surgical skill and should be useful in validating surgical simulations for use in training, certifying surgeons and designing and evaluating new surgical tools.