Modeling habits as self-sustaining patterns of sensorimotor behavior

Abstract

In the recent history of psychology and cognitive neuroscience, the notion of habit has been reduced to a stimulus-triggered response probability correlation. In this paper we use a computational model to present an alternative theoretical view (with some philosophical implications), where habits are seen as self-maintaining patterns of behavior that share properties in common with self-maintaining biological processes, and that inhabit a complex ecological context, including the presence and influence of other habits. Far from mechanical automatisms, this organismic and self-organizing concept of habit can overcome the dominating atomistic and statistical conceptions, and the high temporal resolution effects of situatedness, embodiment and sensorimotor loops emerge as playing a more central, subtle and complex role in the organization of behavior. The model is based on a novel "iterant deformable sensorimotor medium (IDSM)," designed such that trajectories taken through sensorimotor-space increase the likelihood that in the future, similar trajectories will be taken. We couple the IDSM to sensors and motors of a simulated robot, and show that under certain conditions, the IDSM conditions, the IDSM forms self-maintaining patterns of activity that operate across the IDSM, the robot's body, and the environment. We present various environments and the resulting habits that form in them. The model acts as an abstraction of habits at a much needed sensorimotor "meso-scale" between microscopic neuron-based models and macroscopic descriptions of behavior. Finally, we discuss how this model and extensions of it can help us understand aspects of behavioral self-organization, historicity and autonomy that remain out of the scope of contemporary representationalist frameworks.

Keywords: habits; mental-life; meso-scale modeling; self-maintaining patterns-of-behavior; sensorimotor.

Figure 1

Non-linear functions used to calculate the node-density of a SM-state, and to scale the influence of nodes by their proximity to the current SM-state (Plot A) and by their weight (Plot B). See main text for details.

Figure 2

The influence of a single node. This plot shows the combined influence of single node, located at N_p = (0.5, 0.5) with N_v = (0, 0.1) in a hypothetical 2-motor, 0-sensor IDSM. The N_v is exactly vertical, so all horizontal motion is due to the attraction factor, and vertical motion is due to the velocity factor. See Equations (6–9) and main text for details.

Figure 3

Nodes with lower weights have less influence on system-dynamics. These plots show how the influence of a node decreases with its weight. Each plot shows the dynamics of the same in the same 2-motor, 0-sensor IDSM with four activated nodes, each given a weight (N_w) of 0, except for the circled node on the right, which has the weight indicated at the top of each plot.

Figure 4

Three snapshots of the 2-Motor IDSM as a fixed dynamical system. The left plot indicates the influence of the velocity term, the central plot indicates the influence of the attraction factor, and the right plot indicates the combination of the two. In the final plot, a randomly selected initial condition (star) is shown to have a trajectory (blue curve) that approaches the trained cycle of motor activity (gray circle).

Figure 5

Training and performance of an oscillatory behavior. The top plot shows the position of the robot, and the bottom three plots indicate SM-trajectories and the motor components of activated IDSM-nodes (arrows) for different time-periods in normalized SM-space. See main text for details.

Figure 6

Robot with two motors and two directional light sensors.

Figure 7

Training of phototactic and photophobic behaviors and the long term evolution of each of the trained behaviors. The square frames show the spatial trajectories taken by a robot trained with the behavior indicated to the left of the row, during the time indicated at the top of the column. Robots are relocated to a random position and assigned a random motor-state every 50 time-units. The light is fixed at the center of the arena. The bar chart shows the mean distance of the robot from the light for each behavior during each indicated time-period.

Figure 8

Spatial and sensorimotor trajectories of habits that have emerged from a randomly initialized IDSM. The spatial plots (Plot A) indicate the spatial trajectories taken by the agent during the last 25% of the trial indicated in the lower right corner. Plot (B) shows a PCA dimensional reduction projection of the sensorimotor trajectories for these same trajectories, with colors used to group those trials that have a similar spatial trajectory.

Figure 9

Exploration and re-visitation of sensorimotor regions in habits that have emerged from a randomly initialized IDSM. To generate this alternative view of the sensorimotor trajectories displayed in Figure 8, we subdivided the SM-space into a 10×10×10×10 lattice and assigned a region ID number to each hypercube in order that they were visited. We then plot the region ID number of the current SM-state against time. Colors correspond to those used in Figure 8.