A comparison of supervised classification methods for the prediction of substrate type using multibeam acoustic and legacy grain-size data

Abstract

Detailed seabed substrate maps are increasingly in demand for effective planning and management of marine ecosystems and resources. It has become common to use remotely sensed multibeam echosounder data in the form of bathymetry and acoustic backscatter in conjunction with ground-truth sampling data to inform the mapping of seabed substrates. Whilst, until recently, such data sets have typically been classified by expert interpretation, it is now obvious that more objective, faster and repeatable methods of seabed classification are required. This study compares the performances of a range of supervised classification techniques for predicting substrate type from multibeam echosounder data. The study area is located in the North Sea, off the north-east coast of England. A total of 258 ground-truth samples were classified into four substrate classes. Multibeam bathymetry and backscatter data, and a range of secondary features derived from these datasets were used in this study. Six supervised classification techniques were tested: Classification Trees, Support Vector Machines, k-Nearest Neighbour, Neural Networks, Random Forest and Naive Bayes. Each classifier was trained multiple times using different input features, including i) the two primary features of bathymetry and backscatter, ii) a subset of the features chosen by a feature selection process and iii) all of the input features. The predictive performances of the models were validated using a separate test set of ground-truth samples. The statistical significance of model performances relative to a simple baseline model (Nearest Neighbour predictions on bathymetry and backscatter) were tested to assess the benefits of using more sophisticated approaches. The best performing models were tree based methods and Naive Bayes which achieved accuracies of around 0.8 and kappa coefficients of up to 0.5 on the test set. The models that used all input features didn't generally perform well, highlighting the need for some means of feature selection.

Figure 1. Study area, acoustic data and ground-truth samples.

A: Bathymetry data with ground-truth samples overlaid, colours indicating the test and validation sets. B: Backscatter data with ground-truth samples overlaid, colours indicating the substrate class.

Figure 2. Exploration of training data.

A: Comparing the distributions of bathymetry values between training samples (dashed) and raster grid (solid). B: Comparing the distributions of backscatter values between training samples (dashed) and raster grid (solid). C: Comparing the distribution of bathymetry values between substrate classes. D: Comparing the distribution of backscatter values between substrate classes.

Figure 3. Comparing model performance on the test data.

The dashed lines represent the performance of the baseline model. The best performing models are to the top-right.

Figure 4. Output predictions from top three models and agreement between them.

A: NB2, B: RF2, C∶CT1, D: Agreement.