GEARing smart environments for pediatric motor rehabilitation

Abstract

Background: There is a lack of early (infant) mobility rehabilitation approaches that incorporate natural and complex environments and have the potential to concurrently advance motor, cognitive, and social development. The Grounded Early Adaptive Rehabilitation (GEAR) system is a pediatric learning environment designed to provide motor interventions that are grounded in social theory and can be applied in early life. Within a perceptively complex and behaviorally natural setting, GEAR utilizes novel body-weight support technology and socially-assistive robots to both ease and encourage mobility in young children through play-based, child-robot interaction. This methodology article reports on the development and integration of the different system components and presents preliminary evidence on the feasibility of the system.

Methods: GEAR consists of the physical and cyber components. The physical component includes the playground equipment to enrich the environment, an open-area body weight support (BWS) device to assist children by partially counter-acting gravity, two mobile robots to engage children into motor activity through social interaction, and a synchronized camera network to monitor the sessions. The cyber component consists of the interface to collect human movement and video data, the algorithms to identify the children's actions from the video stream, and the behavioral models for the child-robot interaction that suggest the most appropriate robot action in support of given motor training goals for the child. The feasibility of both components was assessed via preliminary testing. Three very young children (with and without Down syndrome) used the system in eight sessions within a 4-week period.

Results: All subjects completed the 8-session protocol, participated in all tasks involving the selected objects of the enriched environment, used the BWS device and interacted with the robots in all eight sessions. Action classification algorithms to identify early child behaviors in a complex naturalistic setting were tested and validated using the video data. Decision making algorithms specific to the type of interactions seen in the GEAR system were developed to be used for robot automation.

Conclusions: Preliminary results from this study support the feasibility of both the physical and cyber components of the GEAR system and demonstrate its potential for use in future studies to assess the effects on the co-development of the motor, cognitive, and social systems of very young children with mobility challenges.

Keywords: Activity recognition; Body weight support; Decision making; Human-robot interaction; Pediatric rehabilitation.

Fig. 1

Different phases in the development of the GEAR system

Fig. 2

The GEAR environment system concept includes playground equipment, an open-area body weight support device, and socially assistive robots to maximize children’s learning. Kinect sensors, strategically placed around the play area, synchronously collect information about the child’s actions from different angles, and send it to a central server that interprets the scene and instructs the robots

Fig. 3

The GEAR system cyber component architecture

Fig. 4

Screenshots of the GEAR interface during a training session

Fig. 5

Comparison between the application of maximum likelihood (left) and smoothing (right) for estimating transition probabilities out of small data sets. Smoothing assigns small but nonzero probabilities to events that have not (yet) been observed, acknowledging the fact that the data set may be small and sparse

Fig. 6

Snapshots of a child within the GEAR system. The child, supported by the device, performs various and complex motor actions and interacts with the robots during exploration and manipulation of the objects of the enriched environment

Fig. 7

a. Overview of video representation framework. b. The two approaches for action classification: SVM with Majority Voting fusion (left), Multiple Instance Learning SVM (right). For illustration purposes, we assume three views per action instance. Frames are cropped to focus on the child

Fig. 8

a. The MDP model for CRI. Each of the arrows can be labeled by actions with its corresponding transition probabilities. b. The initial MDP (left), and the updated MDP after observing some transitions (right)

Fig. 9

Box Plots depicting number of looking instances per minute (a) and number of movements the child initiated towards the robot (b) from all sessions. The center box lines represent the median and the box edges the 25th and 75th percentiles. The whiskers show the range up to 1.5 times the interquartile range. c. Total number of completed ascending trials on the platform and staircase while following the robot

Fig. 10

Action classification results using the MI-SVM classification approach. Diagonal entries of confusion matrix show the percentage of correctly classified action instances per action class with respect to ground truth annotations. Results are averaged over five random training/testing splits

Fig. 11

Difference in rewards using the regular (subjects 1 & 2) and optimal policy (subject 3) between the first and the last session. There was a noticeable difference in subject 3 compared to the other two subjects where the performance remained relatively similar