Impact of study design on development and evaluation of an activity-type classifier

J Appl Physiol (1985). 2013 Apr;114(8):1042-51. doi: 10.1152/japplphysiol.00984.2012. Epub 2013 Feb 21.


Methods to classify activity types are often evaluated with an experimental protocol involving prescribed physical activities under confined (laboratory) conditions, which may not reflect real-life conditions. The present study aims to evaluate how study design may impact on classifier performance in real life. Twenty-eight healthy participants (21-53 yr) were asked to wear nine triaxial accelerometers while performing 58 activity types selected to simulate activities in real life. For each sensor location, logistic classifiers were trained in subsets of up to 8 activities to distinguish between walking and nonwalking activities and were then evaluated in all 58 activities. Different weighting factors were used to convert the resulting confusion matrices into an estimation of the confusion matrix as would apply in the real-life setting by creating four different real-life scenarios, as well as one traditional laboratory scenario. The sensitivity of a classifier estimated with a traditional laboratory protocol is within the range of estimates derived from real-life scenarios for any body location. The specificity, however, was systematically overestimated by the traditional laboratory scenario. Walking time was systematically overestimated, except for lower back sensor data (range: 7-757%). In conclusion, classifier performance under confined conditions may not accurately reflect classifier performance in real life. Future studies that aim to evaluate activity classification methods are warranted to pay special attention to the representativeness of experimental conditions for real-life conditions.

Publication types

  • Research Support, Non-U.S. Gov't

MeSH terms

  • Actigraphy* / instrumentation
  • Activities of Daily Living / classification*
  • Adult
  • Artificial Intelligence*
  • Biomedical Research / methods*
  • Equipment Design
  • Female
  • Humans
  • Logistic Models
  • Male
  • Middle Aged
  • Motor Activity*
  • Principal Component Analysis
  • Reproducibility of Results
  • Research Design*
  • Signal Processing, Computer-Assisted*
  • Surveys and Questionnaires
  • Time Factors
  • Transducers
  • Walking
  • Young Adult