Bayesian online learning of the hazard rate in change-point problems

Abstract

Change-point models are generative models of time-varying data in which the underlying generative parameters undergo discontinuous changes at different points in time known as change points. Change-points often represent important events in the underlying processes, like a change in brain state reflected in EEG data or a change in the value of a company reflected in its stock price. However, change-points can be difficult to identify in noisy data streams. Previous attempts to identify change-points online using Bayesian inference relied on specifying in advance the rate at which they occur, called the hazard rate (h). This approach leads to predictions that can depend strongly on the choice of h and is unable to deal optimally with systems in which h is not constant in time. In this letter, we overcome these limitations by developing a hierarchical extension to earlier models. This approach allows h itself to be inferred from the data, which in turn helps to identify when change-points occur. We show that our approach can effectively identify change-points in both toy and real data sets with complex hazard rates and how it can be used as an ideal-observer model for human and animal behavior when faced with rapidly changing inputs.

Figure 1

Cartoon illustrating a simple change-point problem. A. Data points (circles) are generated by adding random noise to an underlying mean (dashed line) that undergoes two change-points, at times 5 and 10. B. The number of points since the last change-point, or run-length, is plotted as a function of time for the same example. The run-length increases by one when there is no change-point (black line) and decreases to zero when there is one (dashed black line). C Schematic of the message-passing updating rule from (Adams and MacKay, 2007). Starting from time 1, all of the weight is on the node at run-length 0. At time 2 this node then sends messages to nodes at run-lengths 1 and 0. From there each node sends two messages, one increasing the run-length by one and the other back to the node at r_t = 0. Using these input messages, each node updates its weight, p(r_t|x_1:t).

Figure 2

Effect of changing the hazard rate on the mean of the predictive distribution generated by the algorithm in (Adams and MacKay, 2007) for a single data set. Each panel shows a different setting of the hazard rate h, as indicated. The top half of each panel shows the data (gray dots) along with the model’s predicted mean in black. The bottom half of each panel shows the logarithm of the run-length distribution, log p(r_t|x_1:t), with darker shades corresponding to higher probabilities. Clearly the predictions are heavily influenced by the choice of h and, without knowing the actual change-point locations, it is not obvious to tell which one is better matched to the data.

Figure 3

Inference of a constant hazard rate for a toy problem. A. The raw data (circles) and the predicted mean (solid line) plotted versus time. The actual change-point locations from the generative process are shown by the gray vertical lines. B. The marginal run-length distribution p(r_t|x_1:t) = Σ_at p(r_t, a_t|x_1:t) versus time. C. The marginal distribution over the number of change-points, p(a_t|x_1:t) = Σ_rt p(r_t, a_t|x_1:t), versus time. D. The maximum likelihood on-line estimate of the hazard rate (solid black line) quickly converges to the actual hazard rate (dashed black line).

Figure 4

Illustration of the generative model corresponding to the change-point hierarchy. See text for details.

Figure 5

Effect of pruning on the two-level case for Bernoulli, Gaussian, and Laplacian data (columns). A–C, Generative parameters (dashed line in A), data (gray circles in B and C), and inferences from a pruned (solid black lines) and unpruned (thick gray lines) models versus time. D–F, Number of nodes versus time for the pruned (black) and unpruned (gray) cases. G–I, Unpruned run-length distribution versus time. J–L, Pruned run-length distributions versus time. Horizontal lines indicate bin boundaries used for pruning.

Figure 6

Effect of pruning on a three-level hierarchy example. A The data (circles) are sampled from a Gaussian distribution whose mean changes every 2 time steps for t ≤ 10 and then remains constant afterwards. The predictive mean is shown for the unpruned (dashed black) and pruned (gray) model. B The model’s estimates of h (gray line unpruned, black line pruned) compared with the generative h (dashed black line). C The number of nodes for the unpruned (gray) and pruned (black) cases over time on a logarithmic scale. D Marginal low-level run-length distribution, $p(r_t^{(2)}\mid x_{1:t})$, in the unpruned case. This is to be compared with the pruned version of the same distribution in panel G. E and H Marginal high-level run-length distribution, $p(r_t^{(1)}\mid x_{1:t})$, in the unpruned and pruned cases respectively. F and I Marginal change-point count distribution, p(a_t|x_1:t), in the unpruned and pruned cases.

Figure 7

Bernoulli data example, similar to that in (Behrens et al., 2007). A. Binary data representing the presence (vertical black line) or absence (vertical white line) of reward at a given time. B. The probability of reward delivery, ρ_t, (gray dashed line) changes over time according to a change-point process and is well tracked by the algorithm (black line). C The actual hazard rate over time (dashed gray line) compared with the hazard rate inferred by the model (black line). D The low level run-length distribution, p(r⁽²⁾|x_1:t) computed by the model as a function of time. E The change-point count distribution, p(a_t|x_1:t), as a function of time. F The high level run-length distribution, p(r⁽¹⁾|x_1:t), as a function of time shows that the model has recognized the high level change-point at t = 200.

Figure 8

Gaussian example. A The data over time (thin gray line) are sampled from a Gaussian distribution whose mean and variance undergo change-points. The black line indicates the model’s estimate of the predictive mean. B The model’s estimate of the hazard rate (black line) compared with the true hazard rate of the generative process (dashed gray line). C The low level run-length distribution, $p(r_t^{(2)}\mid x_{1:t})$ over time. D The change-point count distribution, p(a_t − a_p|x_1:t) over time. E The high level run-length distribution, $p(r_t^{(1)}\mid x_{1:t})$.

Figure 9

Change-point analysis of daily GM stock returns, 1972–2009. A. Daily log return (R_t, (gray line) and the estimated scale factor (λ_t, black line) from equation 44 plotted as a function of time. B. The model’s estimate of the hazard rate, ${\stackrel{\sim }{h}}_t^{(1)}$, in units of change-points per year, versus time. C. Low level run-length distribution, $p(r_t^{(2)}\mid {\mathbf{x}}_{1:t})$, versus time (note that run-length is measured in years). Several change-points are evident, corresponding to abrupt changes in the variance of the return. D. Distribution over the number of change-points, p(a_t − a₀|x_1:t), versus time. E. High level run-length distribution, $p(r_t^{(1)}\mid {\mathbf{x}}_{1:t})$, versus time. The model identified one high level change-points around 2001, roughly at the time of Richard Wagoner’s appointment as CEO. This event was followed by a higher frequency of inferred change-points and therefore an increase in the estimated hazard rate.