Geometric anomaly detection in data

Abstract

The quest for low-dimensional models which approximate high-dimensional data is pervasive across the physical, natural, and social sciences. The dominant paradigm underlying most standard modeling techniques assumes that the data are concentrated near a single unknown manifold of relatively small intrinsic dimension. Here, we present a systematic framework for detecting interfaces and related anomalies in data which may fail to satisfy the manifold hypothesis. By computing the local topology of small regions around each data point, we are able to partition a given dataset into disjoint classes, each of which can be individually approximated by a single manifold. Since these manifolds may have different intrinsic dimensions, local topology discovers singular regions in data even when none of the points have been sampled precisely from the singularities. We showcase this method by identifying the intersection of two surfaces in the 24-dimensional space of cyclo-octane conformations and by locating all of the self-intersections of a Henneberg minimal surface immersed in 3-dimensional space. Due to the local nature of the topological computations, the algorithmic burden of performing such data stratification is readily distributable across several processors.

Keywords: persistent cohomology; singularities; stratification inference.

Fig. 1.

Annular neighborhood classes $A_x$ of several points $x$ in union of a hemisphere with a plane (both blue) along a circle (red). Regular points, which lie away from this red circle and from the boundaries, have $A_x$, which looks like a standard annulus. All points lying in the boundary have $A_x$, which resembles a half or quarter annulus, as depicted in the central panel. All points $x$ on the red circle itself have neighborhoods $A_x$, which resemble two annuli glued along two edges, as indicated in the right panel. The dimensions of ${\mathbf{\text{H}}}^1\left(A_x\right)$ count the number of independent loops in $A_x$, so from left to right, these are one, zero, and three, respectively.

Fig. 2.

Two-dimensional depiction of the 3D IsoMAP projection (4) of points sampled from the 24-dimensional conformation space of cyclo-octane. Points $x$ for which $\dim {\mathbf{\text{H}}}^1\left(A_x\right)>1$ have been colored red, and these clearly appear to cluster near the two embedded circles where the two surfaces intersect. We show the full set of points in A and B and additionally highlight the intersection points identified by our method separately in C and D, using the same perspectives as A and B, respectively. The perspective in B and D corresponds to a counterclockwise rotation around the $z$ axis ($<90^{○}$) and a counterclockwise rotation around the $x$ axis ($<45^{○}$) of the perspective in A and C.

Fig. 3.

Two-dimensional projections of points sampled from Henneberg’s minimal surface immersed in 3D space. Points $x$ for which $\dim {\mathbf{\text{H}}}^1\left(A_x\right)>1$ are shown in red, and these lie along the four self-intersections. Similarly, points $x$ for which $\dim {\mathbf{\text{H}}}^1\left(A_x\right)=0$ have been colored cyan and appear near the boundary. The perspective in B corresponds to a counterclockwise rotation around the $z$ axis ($<90^{○}$) and a counterclockwise rotation around the $x$ axis ($90^{○}$) of the perspective in A; we indicate the $x$, $y$, and $z$ axes to facilitate comparison.

Fig. 4.

Robustness with respect to the choice of local neighborhood size for the cyclo-octane dataset using local persistent cohomology (PCoh; purple line) and local PCA (orange). The horizontal axis represents local neighborhood size, while the vertical axis corresponds to the Hausdorff distance between the intersection points $S_r$ selected by each method with neighborhood radius $r$ on one hand and a set of ideal reference points $R\subset S_r$ on the other. This distance is defined to be the smallest $ϵ>0$ so that the union of radius $ϵ$ balls around points in $R$ contains all of the points in $S_r$. A–F illustrate the singularities detected at neighborhood radii corresponding to points a–f, respectively. The extreme points responsible for the steep increase in the Hausdorff distance for each method are indicated with black arrows.