TY - JOUR
AU1 - Fincham, Jennifer, I
AU2 - Wilson,, Christian
AU3 - Barry,, Jon
AU4 - Bolam,, Stefan
AU5 - French,, Geoffrey
AB - Abstract Management of the marine environment is increasingly being conducted in accordance with an ecosystem-based approach, which requires an integrated approach to monitoring. Simultaneous acquisition of the different data types needed is often difficult, largely due to specific gear requirements (grabs, trawls, and video and acoustic approaches) and mismatches in their spatial and temporal scales. We present an example to resolve this using a convolutional neural network (CNN), using ad hoc multibeam data collected during multi-disciplinary surveys to predict the distribution of seabed habitats across the western English Channel. We adopted a habitat classification system, based on seabed morphology and sediment dynamics, and trained a CNN to label images generated from the multibeam data. The probability of the correct classification by the CNN varied per habitat, with accuracy above 60% for 85% of habitats in a training dataset. Statistical testing revealed that the spatial distribution of 57 of the 100 demersal fish and shellfish species sampled across the region during the surveys possessed a non-random relationship with the multibeam-derived habitats using CNN. CNNs, therefore, offer the potential to aid habitat mapping and facilitate species distribution modelling at the large spatial scales required under an ecosystem-based management framework. Introduction In the marine environment, ecosystem-based management (EBM), which has its origins in the Convention on Biological Diversity, is being implemented through current national and regional legislation such as the Common Fisheries Policy, the Marine Strategy Framework Directive and the Water Framework Directive (Hyder et al., 2015). EBM inherently requires an integrated management approach that recognizes the full array of interactions within an ecosystem, including humans, rather than considering single species or ecosystem services in isolation. To implement the concept, there is an implicit need to delineate and monitor marine habitats in terms of vulnerability to human impacts, identify the potential irreversibility of those impacts, and elucidate habitats critical to species for vital population processes (Pikitch et al., 2004). There is a need for monitoring surveys to become more integrated if data regarding the required features, of suitable quality and at the right spatial scales, are to be acquired (Knol, 2013). While monitoring an entire ecosystem is a logistical and financial challenge, integrated, multi-disciplinary monitoring approaches wherein data pertaining to multiple facets of the ecosystem are acquired, can lead to greater efficiencies over multiple, independent surveys (Hyder et al., 2015; Kupschus et al., 2016). However, specific gear requirements and/or mismatches in spatial and temporal scales often means that certain monitoring methods are incompatible with each other (Bean et al., 2017). This issue may be exemplified regarding the use of acoustic backscatter data for predicting or mapping seabed habitats (Lamarche and Lurton, 2018). Acquisition and quality of backscatter data for localized mapping or monitoring marine protected areas (MPAs) are different from those for fisheries stock assessments, which are typically conducted over the larger spatial scales of the International Council for Exploration of the Sea rectangles. Thus, comparable multibeam data cannot be effectively gathered when these survey types are integrated. In the western English Channel, a comprehensive spatial dataset of seabed acoustic data has been acquired largely through an ad hoc basis. Due to differences in system configurations and that the data were collected in an “unattended” manner (no raw data were collected or records made of multibeam setting changes), precise analyses or processing of the backscatter data were not possible. Utilization of these data for the creation of a habitat map suitable for management under EBM would inherently be a time-consuming and laborious task due to their patchy distribution and variable quality. As a result, a more pragmatic approach was taken to make best use of this opportunistically collected data wherein the machine learning method used the data “as is” and only utilized the backscatter information to improve the interpretation of the data. Machine learning, viewed as a subset of artificial intelligence, is the scientific study of algorithms and statistical models that computer systems use to perform a specific task without using explicit instructions, relying on patterns and inference instead. One group of such algorithms, convolutional neural networks (CNNs), is a class of deep neural networks that is commonly applied to analysing visual imagery to aid in the cataloguing and identification of large image repositories (Cires and Meier, 2012). In the terrestrial realm CNNs have been applied to identify images from camera trap videos and stills (Norouzzadeh et al., 2018) and successfully adopted for plankton identification, fisheries surveillance, and counting of sea birds from aerial images in the marine environment (French et al., 2015; ICES, 2018; Bowler et al., 2019). CNNs are theoretically the most suitable machine learning algorithm for mapping sea beds based on acoustic data due to their potential for image classification, their large learning capacity, and the ease in which they may be trained and run on available graphics processing unit architectures and opensource software (Hu et al., 1998; Krizhevsky et al., 2012; Simonyan and Zisserman, 2014; French et al., 2015; Lloffe and Szegedy, 2015). Furthermore, CNNs are trained on labelled image repositories such as ImageNet and LabelMe, which are readily available online. In this study, we first develop a partially automated method of classifying images generated from seabed multibeam data for the western English Channel using a CNN. We utilized a habitat classification based on seabed morphology and sediment dynamics, which allows habitats to be visualized and a CNN to be trained to identify those habitats. Second, to examine the potential utility of this approach for future application under EBM and to facilitate species distribution modelling at large spatial scales (Moriarty et al., 2020), we test the significance of the relationships between the abundance of demersal species and the habitats predicted using CNN. Methods Study area and data acquisition The study area covers the relatively shallow, tide-dominated waters of the western English Channel. The western, deeper part of this region opens to the Atlantic Ocean while the eastern, shallower part is connected to the tidal-dominated eastern English Channel and eventually to the North Sea (Coggan and Diesing, 2011). Demersal fauna (fish and shellfish) and multibeam acoustic data were collected from annual beam trawl surveys conducted by the RV Cefas Endeavour each spring for a 12-year period (2006–2018). The commercially rigged 4-m beam trawl (measured between inside edges of the shoes) to sample the fauna was fitted with a chain mat and flip-up ropes and polypropylene nets. Two gears were deployed per haul and recorded separately. One gear was fitted with a 40-mm cod-end liner and the other used a commercial 80-mm liner [Holmes (Cefas), pers. comm.]. The 40-mm liner is more appropriate for sampling epibenthic invertebrates while the catch composition of the 80-mm liner approximates that from commercial gears. The beams were deployed simultaneously and towed along the seabed for 0.9–3 nm and so the sample is the combination of the two gears. The finfish and macrobenthos were sorted, counted, weighed, and measured on board as described by Van Der Kooij et al. (2011) (Supplementary Table S1). Pelagic species such as sprat and mackerel incidentally caught (potentially during hauling) by the gear were removed; species thought to be unreliably identified—such as clingfish or seahorses—were grouped (Supplementary Table S1). Between 73 and 137 stations were sampled each year across 24 survey strata, each stratum representing a specific benthic habitat and environmental condition. The survey design was based on the location of fishing grounds (informed by local fishers), which were mapped over existing oceanographic, bathymetric, and geological survey maps to delineate the boundaries of the survey strata. The stations were selected based on random allocation within the strata allowing sampling across a broader distribution of benthic habitats (Van Der Kooij et al., 2011). Of the 1234 demersal samples, only 445 samples were spatially coincident with multibeam data (see below): these 445 were amenable for subsequent analyses (“The CCN Prediction” section). Habitat classification The multibeam data were transferred in real time to an offline Olex system, a navigating and charting system that records GPS and echosounder data. Backscatter (acoustic intensity of seafloor return echo depth) and bathymetry (intensity depth) data are recorded as a value between 0 and 100, creating a picture mosaic of the intensity of the multibeam energy reflected from the seafloor and a 3d seabed surface. The multibeam data were exported from Olex and stored as a space-delimited ASCII file with latitude, longitude, depth, and acoustic intensity as columns. The data were ordered by latitude, with a maximum possible data density ∼4 points per m2, resulting in one value per 0.23 m2 square cell. We adopted a habitat classification approach based on expert judgement that integrated the high-resolution seabed morphology and backscatter data in their broader geographical, oceanographic, and geological contexts (Wilson, 2016). The absence of ground truthing data prevents determining if a set of rock ridges, for example represents the bedding surface of a rock outcrop. However, using information regarding the broader geological context (morphology and lithology) overcomes this. Similarly, areas with contrasting backscatter can have inferences made about their genesis made based on the morphology of the feature and an understanding of the current regime of the area (for example elongated parallel patches of coarser material in an area of high currents can be interpreted as evidence for the sedimentary process of scouring). The classification then consists of two components: the substrate class describing the type of substrate, followed by a habitat descriptor, or modifier, describing the form of the habitat (Supplementary Table S3). The substrate classes are: Rock, Mud, Fine, Sand, Gravel, and Biogenic. Rock has its own seven modifiers, which differ from the modifiers of the other classes as rock erodes and shapes into different forms. Mud, Fine (fine sediments which are coarser than sand and finer than gravel), Sand, and Gravel have the same eight modifiers as these substrates form in the same way. Biogenic habitats have their own five modifiers: Complete, Most, Partial, Patch, and Streaked. Input data preparation for automated habitat classification Manual habitat classification was performed using the visual appearance of the multibeam data rendered by GIS software. Automated classification and segmentation have a rich history in computer vision (Martin et al., 2004) leading to contemporary approaches, many of which use CNNs (Krizhevsky et al., 2012). The multibeam data come in the form of a point cloud with each point having longitude, latitude, backscatter, and bathymetry. While recent neural network models have accepted point clouds as direct inputs (Uy et al., 2019), raster-based approaches are better established and the techniques for training them are well known. We therefore chose to convert the multibeam point cloud data into raster form so that it could be more easily processed by a CNN. Due to its size—over 100GB—we divided the dataset into a grid of cells, each of which can be processed individually on a desktop machine. Manual inspection of the multibeam point cloud data indicated that neighbouring points were typically 0.75 m apart; this was the effective spatial resolution of the data. We therefore chose 0.75 m per pixel as the target resolution for the rasterized data to preserve as much of the resolution of the point cloud data as possible. CNN networks normally include down sampling layers that scale the resolution of the interpolation down by a factor of 2. Consequently, they will perform most effectively when the input data have a resolution that is a multiple of the aggregate down sampling factor used within the network. To afford maximum flexibility for the design of the network, we used input images with a pixel size of a power of 2. Our original intention was to choose the size of the grid cells to allow efficient processing by the network directly. Increasing the size of input images to a CNN allows it to utilize a larger amount of surrounding context, potentially improving accuracy. The availability of context is limited however by the scanning procedure. As the survey vessel travels, it scans a “band” of the seabed. As can be seen from Figure 1, most of the seabed was not surveyed. Increasing the size of input images much beyond the width of the “band” is therefore unlikely to be beneficial. The width of the scanning band varies with depth and was generally 350 m at its widest. We chose an input size of 512 × 512 pixels, giving a physical size of 384 m × 384 m given the 0.75 m per pixel resolution with a total of 35 131 cells. This covers the width of the larger scanning bands while being sufficiently small that a batch of several images can be processed at one time while fitting within the 12 GB of available GPU memory during training. To reduce memory space, the rasterized images were quantized into 16-bit integers and stored in a Portable Network Graphics (PNG) format with point bathymetry values being stored to an accuracy of 3.4 mm.) Figure 1. Open in new tabDownload slide The process of aligning the habitat patches with the grid cells begins with the mask (a), which is skeletonized (b), and finally, paths are created, which are then overlaid with habitat patches (c). Figure 1. Open in new tabDownload slide The process of aligning the habitat patches with the grid cells begins with the mask (a), which is skeletonized (b), and finally, paths are created, which are then overlaid with habitat patches (c). Preparation of the multibeam data and the development of the CNN were carried out in Python. The multibeam data were interpolated and rasterized (Supplementary Figure 1) using a “K-nearest neighbours” regression model provided by SciKit-Learn 0.20 (Pedregosa et al., 2011). The trained regression model was then used to create a uniformly spaced grid by predicting backscatter and bathymetry values at uniformly spaced longitude and latitude points. To avoid edge artefacts during the rasterization process, we ensured that all points from the point cloud were within 6 m of the bounds of the cell. The rasterization process generated three output images per cell: one each for backscatter, bathymetry, and a mask image. The mask image identifies the regions within for which data are available. To rasterize the point data, the grid cells were grouped by the set of point data which covers each cell (“coverage” set hereafter). For each set of grouped cells, we used the coverage set to select the points within 6 m of the bounds of the grid cells. To show the areas covered by data, a mask was generated for each cell that covers pixels within 6 m of data points. Empty cells, for which no point cloud data were available, were filtered out and no images were generated for these cells. For each cell in the coverage set, rasterized images for both the backscatter and bathymetry were created. Ground truth data preparation To create the ground truth data (a total of 204 habitat examples; Table 1), the multibeam data were viewed using Olex to identify, outline, and count each habitat. The outlines were imported into QGIS mapping software, labelled with the relevant habitats, and saved as GEOJSON files. Examples of each habitat class and modifier were manually identified and labelled using multibeam data gathered in 2007, 2008, and 2010. These years were chosen to give examples of data collected with the original EM3002D dual-headed transducer and examples collected with the upgraded equidistant spacing. Once identified, outlines around each habitat example were drawn in QGIS 2.18.13 (QGIS Development Team, 2018) and saved as a shapefile in a WGS-84 co-ordinate system, labelled with the relevant habitat class and modifier and finally converted to a GEOJSON format. Table 1. Training results for all classes and modifiers: accuracy (%), number of training examples tested per class and modifier, and the total number of blocks classified per habitat. . Accuracy (%) . Training examples (n) . Classified blocks (n) . Class  Rock 84.0 69 12 334  Mud 87.4 18 2 844  Fine 84.8 38 8 042  Sand 76.4 52 8 158  Gravel 81.5 15 2 219  Biogenic 77.5 12 1 634  Overall class 82.2 Modifier (Rock)  Flat 72.7 4 3 825  Subcrop 76.4 11 3 736  Bedded 72.7 4 170  BedFractured 39.2 2 547  Fractured 39.2 1 173  Rubble 73.7 16 459  Sediment veneer 63.4 42 3 424 Modifier (mud, fine, sand, gravel)  Flat 85.3 37 13 161  Scoured 72.6 5 2 691  Streak 45.3 5 129  Bedforms 76.2 15 1 288  Stacked 66.6 2 254  Subcrop 72.1 11 2 160  Mega Ripple 68.4 30 1 583 Modifier (Biogenic)  Complete 19.0 2 106  Most 33.2 1 12  Partial 29.7 1 17  Patch 41.2 3 1 233  Streaked 90.4 5 266  Overall modifier 76.2 . Accuracy (%) . Training examples (n) . Classified blocks (n) . Class  Rock 84.0 69 12 334  Mud 87.4 18 2 844  Fine 84.8 38 8 042  Sand 76.4 52 8 158  Gravel 81.5 15 2 219  Biogenic 77.5 12 1 634  Overall class 82.2 Modifier (Rock)  Flat 72.7 4 3 825  Subcrop 76.4 11 3 736  Bedded 72.7 4 170  BedFractured 39.2 2 547  Fractured 39.2 1 173  Rubble 73.7 16 459  Sediment veneer 63.4 42 3 424 Modifier (mud, fine, sand, gravel)  Flat 85.3 37 13 161  Scoured 72.6 5 2 691  Streak 45.3 5 129  Bedforms 76.2 15 1 288  Stacked 66.6 2 254  Subcrop 72.1 11 2 160  Mega Ripple 68.4 30 1 583 Modifier (Biogenic)  Complete 19.0 2 106  Most 33.2 1 12  Partial 29.7 1 17  Patch 41.2 3 1 233  Streaked 90.4 5 266  Overall modifier 76.2 Open in new tab Table 1. Training results for all classes and modifiers: accuracy (%), number of training examples tested per class and modifier, and the total number of blocks classified per habitat. . Accuracy (%) . Training examples (n) . Classified blocks (n) . Class  Rock 84.0 69 12 334  Mud 87.4 18 2 844  Fine 84.8 38 8 042  Sand 76.4 52 8 158  Gravel 81.5 15 2 219  Biogenic 77.5 12 1 634  Overall class 82.2 Modifier (Rock)  Flat 72.7 4 3 825  Subcrop 76.4 11 3 736  Bedded 72.7 4 170  BedFractured 39.2 2 547  Fractured 39.2 1 173  Rubble 73.7 16 459  Sediment veneer 63.4 42 3 424 Modifier (mud, fine, sand, gravel)  Flat 85.3 37 13 161  Scoured 72.6 5 2 691  Streak 45.3 5 129  Bedforms 76.2 15 1 288  Stacked 66.6 2 254  Subcrop 72.1 11 2 160  Mega Ripple 68.4 30 1 583 Modifier (Biogenic)  Complete 19.0 2 106  Most 33.2 1 12  Partial 29.7 1 17  Patch 41.2 3 1 233  Streaked 90.4 5 266  Overall modifier 76.2 . Accuracy (%) . Training examples (n) . Classified blocks (n) . Class  Rock 84.0 69 12 334  Mud 87.4 18 2 844  Fine 84.8 38 8 042  Sand 76.4 52 8 158  Gravel 81.5 15 2 219  Biogenic 77.5 12 1 634  Overall class 82.2 Modifier (Rock)  Flat 72.7 4 3 825  Subcrop 76.4 11 3 736  Bedded 72.7 4 170  BedFractured 39.2 2 547  Fractured 39.2 1 173  Rubble 73.7 16 459  Sediment veneer 63.4 42 3 424 Modifier (mud, fine, sand, gravel)  Flat 85.3 37 13 161  Scoured 72.6 5 2 691  Streak 45.3 5 129  Bedforms 76.2 15 1 288  Stacked 66.6 2 254  Subcrop 72.1 11 2 160  Mega Ripple 68.4 30 1 583 Modifier (Biogenic)  Complete 19.0 2 106  Most 33.2 1 12  Partial 29.7 1 17  Patch 41.2 3 1 233  Streaked 90.4 5 266  Overall modifier 76.2 Open in new tab The GEOJSON files represent the ground truth annotations in vector form and required rasterizing prior to use for training the CNN. For each input image, an equivalent ground truth image was produced at 1/32 resolution, in keeping with the output resolution of the network. The ground truth maps were stored as PNG files using the red and green channels to separately store the class and modifier (see Table 1). Alignment with the path of vessel travel The performance of neural network-based classification and segmentation models can be improved by removing sources of irrelevant variation. While training our CNN using the input images (described in the “Input data preparation for automated habitat classification” and “Ground truth data preparation” sections) was successful, we found that superior performance was obtained when images aligned with the direction of travel of the survey vessel were used. This is because the method of sonar image capture results in strong artefacts in the backscatter imagery in the form of a strong central band along the direction of travel. The first step of the alignment process was to estimate the direction of the path of travel. The previously created mask (see the “Input data preparation for automated habitat classification” section) was scaled down by a factor of 32, to reduce the size of the image of the complete dataset to a manageable size (the reduction in resolution does not negatively impact this part of the pipeline). The SciPy v1.1.0 “binary_closing” function was applied with five iterations to close any gaps in the mask (Figure 1a). The downscaled mask was then skeletonized to 1-pixel width using the “skeletonize” function in Scikit-image v0.14.1. Junctions and pixels with more than two neighbours were removed (Figure 1b). The skeletonized mask was converted into a graph using the python package “networkx”, with a graph node created for each white pixel and edges connecting immediate neighbours. Continuous paths of nodes that have two incident neighbours were extracted from the graph. Habitat patches were aligned to these paths, such that the centre of the patch lies on the path (Figure 1c). This process resulted in 35 985 aligned patches of which around 2000 had corresponding habitat annotations in the GEOJSON ground truth files. The CNN The CNN development We used a segmentation network for CNN, the architecture of which was based on VGG-19, a 19-layer model (Simonyan and Zisserman, 2014). This process used blocks of convolutional layers with 3 × 3 kernels, followed by batch normalization (Lloffe and Szegedy, 2015; Long et al., 2015) and rectified linear unit (ReLU) activation (Agarap, 2019). The CNN was trained to accept two channel depth/intensity images as inputs and predicts habitat type at 1/32 resolution of that input. The resulting prediction grid had a resolution of 24 m × 24 m. The CNN training Using the 2000 image patches (“Alignment with the path of vessel travel” section), 1000 were used to train the CNN while the remaining 1000 were used for testing. This process was then repeated, but with the roles of the 1000 images used for training and testing reversed. Thus, those originally used for training became test images and those originally used for testing became training images. During training, data augmentation was used to artificially expand the training set by presenting the network with flipped, scaled, and rotated versions of the same image, artificially introducing additional variation exposing a greater variety of material dataset (He et al., 2019). The CNN was set to train repeatedly for 1000 epochs, each epoch being 1 complete run through with the training data set. The training data were supplied to the CNN in “batches” based on the memory available, using the Adam update rule with a learning rate of 1 × 10−4 (Kingma, 2015). The CCN predictions The CNN was then used to create “blocks” with the predicted habitat information for the remaining 384 unlabelled patches (size of 384 m × 384 m). Two blocks were created for each patch—a high-precision block with habitat class and modifier predicted at a 1 pixel resolution (each block containing multiple habitats) and a low-precision block with habitat and modifier predicted based on the most common habitat in the corresponding high-precision block. These blocks were saved in vector form as GEOJSON format for easy import into QGIS mapping software. The low-precision habitat blocks were overlaid with, and then joined to, the 445 sampling stations to provide one habitat class and modifier per demersal species sample. Randomization test of relationship between species densities and habitats To formally test the ability of the CNN classification to capture the variation in species abundance, we developed a non-parametric randomization test in R (R Core Team, 2018) to determine the statistical significance of the relationship between species abundance with one or more of the classified habitats. For each of the 100 faunal species sampled across the survey area (excluding those outlined in the methods), abundance at each of the 445 stations was used to calculate the observed percentage [HP from (1) below] of the stations in each habitat, and the observed percentage of individuals from each species (SP) in each habitat. For species with no habitat preference, station percentages would be comparable to habitat percentages. Thus, an intuitive measure of the difference between a species’ habitat percentages from the overall habitat percentages (D) is given by: D=∑habitatsSP-HP.(1) The observed value of D was compared with the null distribution, based on no relationship between species count and habitat, using a simple randomization test (Manly, 2006; see R function differ in Supplementary Material S1 for precise details). Results Training results The training and testing results of the CNN gave an overall accuracy of 82.2% for habitat classes, with the most accurately identified class being Mud (87.4%) and the least accurately identified being Biogenic (77.5%) (Table 1). The overall accuracy for habitat modifiers was 76.2. In general, the Biogenic modifiers did worse than the other modifiers (the first four of these all had accuracies below 42%—though the fifth (Streaked) modifier had the highest accuracy of all modifiers (90.4%). We also calculated the kappa statistic (Cohen, 1960) to demonstrate the difference between prediction accuracy from our model and that which would arise if classifications were made at random. For the class prediction, the random allocation prediction was 20.4% and the kappa statistic was 77.6%. For the modifier prediction, these two values were 17.9 and 71.0%, respectively. Thus, clearly, our predictions are much better than random allocation for both classes and modifiers. Predicted outputs Only 30 of the 44 potential class and modifier combinations were present across the western English Channel. Fine sediments, Rock, and Sand dominated with blocks identified for all three of these habitats across the entire area (Figure 2). Biogenic and Gravel habitats were less common but still widely distributed, while Mud habitats were observed mainly near the coast and the Isle of Scilly in the extreme north-west of the area (Figure 2). Figure 2. Open in new tabDownload slide A map of the study area, western English Channel, with locations of trawls for demersal species (black dots), and lines multibeam data classified coloured by habitats classified using the CNN: Mud (brown), Sand (yellow), Fine (orange), Gravel (grey), Biogenic (black), and Rock (green). Blocks where data were used as training data are indicated in red. Figure 2. Open in new tabDownload slide A map of the study area, western English Channel, with locations of trawls for demersal species (black dots), and lines multibeam data classified coloured by habitats classified using the CNN: Mud (brown), Sand (yellow), Fine (orange), Gravel (grey), Biogenic (black), and Rock (green). Blocks where data were used as training data are indicated in red. An example of output from high- and low-precision blocks is shown in Figure 3. As the low-precision predictions were created from the dominant habitat in the high-precision block, their spatial scales more closely match that of the area sampled by the beam trawl and, hence, were considered more appropriate for the non-parametric testing of species distribution. Of these low-precision blocks, the most dominant habitat class found across the 445 stations was Fine (27.6% of stations), closely followed by Rock (26.7%), and then Sand (22.7%), Mud (11.5%), Gravel (8.3%), and Biogenic (3.1%) (Table 2). The most common five habitat class and modifier combinations found at just under 55% of stations were: Fine Flat (18.9%); Sand Flat (11%); Rock Sediment Veneer (10.3%); Gravel Flat (7.4%); and Rock Subcrop (7.2%) (Table 2). Many habitats are seen across relatively few of the stations, particularly Biogenic and Gravel habitats, most of which are assigned to 1% or less of the stations. Figure 3. Open in new tabDownload slide Examples of high-precision predictions with multiple habitat polygons per block (a) and low-precision predictions with the dominant habitat per block predicted (b). In (a) and (b), there are a mixture of habitats including: Biogenic 4 (yellow), Sand Flat (green), Rock Subcrop (purple), and Fine Flat (blue). Figure 3. Open in new tabDownload slide Examples of high-precision predictions with multiple habitat polygons per block (a) and low-precision predictions with the dominant habitat per block predicted (b). In (a) and (b), there are a mixture of habitats including: Biogenic 4 (yellow), Sand Flat (green), Rock Subcrop (purple), and Fine Flat (blue). Table 2. The results of the CNN identification of habitats. Class . Station numbers (%) . Class modifier . Number of samples (%) . Biogenic 14 (3.15) Biogenic Complete 1 (0.22) Biogenic Patch 7 (1.57) Biogenic Streaked 6 (1.35) Fine 123 (27.64) Fine Bedforms 15 (3.37) Fine Flat 84 (18.88) Fine Mega Ripple 1 (0.22) Fine Scoured 13 (2.92) Fine Streak 2 (0.45) Fine Subcrop 8 (1.80) Gravel 37 (8.31) Gravel Flat 33 (7.42) Gravel Scoured 2 (0.45) Gravel Subcrop 2 (0.45) Mud 51 (11.46) Mud Flat 24 (5.39) Mud Scoured 25 (5.62) Mud Streak 1 (0.22) Mud Subcrop 1 (0.22) Rock 119 (26.74) Rock Bed Fractured 4 (0.90) Rock Bedded 5 (1.12) Rock Flat 26 (5.84) Rock Fractured 1 (0.22) Rock Rubble 5 (1.12) Rock Sediment Veneer 46 (10.34) Rock Subcrop 32 (7.19) Sand 101 (22.70) Sand Bedforms 5 (1.12) Sand Flat 49 (11.01) Sand Mega Ripple 18 (4.04) Sand Scoured 1 (0.22) Sand Streak 2 (0.45) Sand Subcrop 26 (5.84) Class . Station numbers (%) . Class modifier . Number of samples (%) . Biogenic 14 (3.15) Biogenic Complete 1 (0.22) Biogenic Patch 7 (1.57) Biogenic Streaked 6 (1.35) Fine 123 (27.64) Fine Bedforms 15 (3.37) Fine Flat 84 (18.88) Fine Mega Ripple 1 (0.22) Fine Scoured 13 (2.92) Fine Streak 2 (0.45) Fine Subcrop 8 (1.80) Gravel 37 (8.31) Gravel Flat 33 (7.42) Gravel Scoured 2 (0.45) Gravel Subcrop 2 (0.45) Mud 51 (11.46) Mud Flat 24 (5.39) Mud Scoured 25 (5.62) Mud Streak 1 (0.22) Mud Subcrop 1 (0.22) Rock 119 (26.74) Rock Bed Fractured 4 (0.90) Rock Bedded 5 (1.12) Rock Flat 26 (5.84) Rock Fractured 1 (0.22) Rock Rubble 5 (1.12) Rock Sediment Veneer 46 (10.34) Rock Subcrop 32 (7.19) Sand 101 (22.70) Sand Bedforms 5 (1.12) Sand Flat 49 (11.01) Sand Mega Ripple 18 (4.04) Sand Scoured 1 (0.22) Sand Streak 2 (0.45) Sand Subcrop 26 (5.84) Column 1 defines the habitat class, column 2 gives the number and percentage of stations per class, and columns 3 and 4 break this information down for the modifiers of that class. Open in new tab Table 2. The results of the CNN identification of habitats. Class . Station numbers (%) . Class modifier . Number of samples (%) . Biogenic 14 (3.15) Biogenic Complete 1 (0.22) Biogenic Patch 7 (1.57) Biogenic Streaked 6 (1.35) Fine 123 (27.64) Fine Bedforms 15 (3.37) Fine Flat 84 (18.88) Fine Mega Ripple 1 (0.22) Fine Scoured 13 (2.92) Fine Streak 2 (0.45) Fine Subcrop 8 (1.80) Gravel 37 (8.31) Gravel Flat 33 (7.42) Gravel Scoured 2 (0.45) Gravel Subcrop 2 (0.45) Mud 51 (11.46) Mud Flat 24 (5.39) Mud Scoured 25 (5.62) Mud Streak 1 (0.22) Mud Subcrop 1 (0.22) Rock 119 (26.74) Rock Bed Fractured 4 (0.90) Rock Bedded 5 (1.12) Rock Flat 26 (5.84) Rock Fractured 1 (0.22) Rock Rubble 5 (1.12) Rock Sediment Veneer 46 (10.34) Rock Subcrop 32 (7.19) Sand 101 (22.70) Sand Bedforms 5 (1.12) Sand Flat 49 (11.01) Sand Mega Ripple 18 (4.04) Sand Scoured 1 (0.22) Sand Streak 2 (0.45) Sand Subcrop 26 (5.84) Class . Station numbers (%) . Class modifier . Number of samples (%) . Biogenic 14 (3.15) Biogenic Complete 1 (0.22) Biogenic Patch 7 (1.57) Biogenic Streaked 6 (1.35) Fine 123 (27.64) Fine Bedforms 15 (3.37) Fine Flat 84 (18.88) Fine Mega Ripple 1 (0.22) Fine Scoured 13 (2.92) Fine Streak 2 (0.45) Fine Subcrop 8 (1.80) Gravel 37 (8.31) Gravel Flat 33 (7.42) Gravel Scoured 2 (0.45) Gravel Subcrop 2 (0.45) Mud 51 (11.46) Mud Flat 24 (5.39) Mud Scoured 25 (5.62) Mud Streak 1 (0.22) Mud Subcrop 1 (0.22) Rock 119 (26.74) Rock Bed Fractured 4 (0.90) Rock Bedded 5 (1.12) Rock Flat 26 (5.84) Rock Fractured 1 (0.22) Rock Rubble 5 (1.12) Rock Sediment Veneer 46 (10.34) Rock Subcrop 32 (7.19) Sand 101 (22.70) Sand Bedforms 5 (1.12) Sand Flat 49 (11.01) Sand Mega Ripple 18 (4.04) Sand Scoured 1 (0.22) Sand Streak 2 (0.45) Sand Subcrop 26 (5.84) Column 1 defines the habitat class, column 2 gives the number and percentage of stations per class, and columns 3 and 4 break this information down for the modifiers of that class. Open in new tab Relationship between habitats and species A statistically significant relationship (p < 0.05) for a habitat class was observed for 57 of the 100 species in the dataset, while 58 species exhibited a significant relationship for a combined habitat class and modifier (Supplementary Table S2). Examples of species with significant non-random distribution include the most common species in the survey area, poor cod (Trisopterus minutus), common dragonet (Callionymus lyra), and lesser spotted dogfish (Scyliorhinus canicular); specialist species such as the scallops (Aequipecten opercularis) and seahorses (Hippocampus spp.); and commercially important species in the area such as common sole (Solea solea), haddock (Melanogrammus aeglefinus), and Norway lobster (Nephrops norvegicus). Discussion Commensurate with the philosophy of an EBM approach in the marine environment is the need for fisheries science to have due consideration of both target and non-target species during assessments of the state of fisheries and fishing impacts on marine ecosystems (Moriarty et al., 2020). This notion has led to a need for greater understanding of and detailed information on the distribution of a broad spectrum of fish species across large spatial scales, such as large marine ecosystems or ecoregions (Kelley and Sherman, 2018). This study represents a step towards this, using multibeam data acquired through integrated surveys to allow mapping seabed habitats over a large spatial scale and assessing the relationship between the distributions of demersal faunal species and habitats. Specifically, we have demonstrated that application of CNNs allows this to be achieved using minimal resource compared to traditional mapping approaches. Habitat maps are commonly derived using data from grab samples, video techniques and samples of sediment processed for particle size distributions (Coggan and Diesing, 2011; Coggan et al., 2012). Rather than creating a habitat map, our novel approach involved using a CNN to undertake the habitat mapping technology to create a habitat variable, which can be used with survey data to improve the accuracy of modelling demersal species. This is particularly beneficial as species and ecosystem modelling often lack habitat variables despite evidence of the importance of habitats in species distributions (Marine Alliance for Science and Technology Scotland, 2018; Marine Ecosystem Research Programme, 2018). Based on the training data, the CNN achieved a good habitat prediction accuracy, above 70% on average for habitat class. Prediction accuracy for the modifiers was slightly lower, while that for the various biogenic habitats ranged from 19.9 to 85.7%. Our class kappa is greater than the 0.75 level suggested by Fleiss (1981) as “excellent” and our modifier kappa is close to this value. Clearly, while such thresholds are somewhat arbitrary, our values of kappa will be useful to compare with further habitat classification studies. The data allowed a large-scale description of the variability of seabed habitats across the western English Channel. The habitats the data discerned included were: muddy habitats in the inshore embayments (congruent to other studies, which report a muddy-sandy inshore environment with silt-clay; Rees et al., 1999; Stephens et al., 2011), rock and gravel along the east and south-east edges’ limits of the survey area (also in alignment with other studies which observed increased gravel content in these regions; Rees et al., 1999; Stephens et al., 2011), and a transitional zone in the middle of the English Channel of fine and sand sediments. Biogenic habitats are present throughout the study area (Diesing et al., 2009). CNNs have been used for similar purposes such as analysing video data for marine habitat mapping (Diegues et al., 2018), and in terrestrial mapping for land use detecting species of high biodiversity conservation interest (Guirado et al., 2017). We demonstrate that it is possible to process disparate multibeam data, collected during several multi-disciplinary surveys, to produce an image repository suitable for use with a corresponding demersal species dataset. The successes of the training results of the CNN were variable, but this could be improved in the future with an enhanced training dataset and real-world verification of select CNN results. The original multibeam data were deemed not suitable for mapping; however, when processed with the CNN and used in non-parametric tests with species distribution data, over 50% of the demersal species from the region show a statistically significant non-random distribution across habitats. Thus, a link between the benthic habitats of the western English Channel and many species of ecological and commercial importance has been demonstrated using this novel approach. Additional data would allow the approach to test whether the less commonly sampled species such as rays, skates, and rocklings show significant habitat relationships. Our approach allowed us to classify the large number of habitats in the absence of ground truth data, although we had to assume such habitats were a realistic representation of the nature of the seabed present. Misidentification of habitats (Howell et al., 2014) and disagreement between individual surveyors regarding classification, particularly at habitat boundaries and for transitional habitats (Stevens et al., 2004), are common sources of error in habitat mapping. Currently, there is no effective way to quality assure the images in the real world, i.e. outside of the training and testing for the CNN approach. Empirical particle size data obtained from grab sampling could, in future, be used to ground truth some of the blocks, allowing a verification of these predictions. The outcomes of the present analyses could be improved through further modelling of species counts against a wider suite of explanatory variables (including CNN habitat classification). However, this would assume that the errors from the CNN classification are at an acceptable level. These errors could arise from: (i) errors occurring at random; (ii) classification being close to the correct value; and (iii) classification is biased towards an incorrect classification. While certain errors are potentially more problematic than others, the classification-based errors arising from a bias towards a habitat would potentially be the most serious. Thus, further work is required to investigate the provenance of errors and examine their influence on subsequent further development of the CNN classifications. The training dataset for the CNN was based on 3 years (2007, 2008, and 2010) of multibeam data. As multibeam data quality may be affected by a range of factors including sea conditions, vessel speed, and water turbidity, limiting the training data set to 3 years may reduce the accuracy of identification in the years not included in the training data. Additional training data, including a range of habitats examples from multiple years and taken from different geographical areas, could therefore help improve the accuracy of the classification. This would be most beneficial in habitats such as Biogenic habitats where prediction accuracy was relatively low compared with more accurately predicted habitats such as mud, which are unlikely to be improved upon beyond the 90% accuracy. In conclusion, we have demonstrated that non-uniform, opportunistically acquired multibeam data from a number of integrated surveys may successfully be used to predict seabed habitats using CNN and used to explain demersal species distributions over large spatial scales. The multibeam data were obtained using widely available equipment, required minimal resources and did not intervene with primary survey operations. Given the current desire for developing predictive or explanatory models (Hyder et al., 2015) and the gaps in many current models (Marine Alliance for Science and Technology Scotland, 2018), our approach could be valuable in future fisheries research. Supplementary data Supplementary material is available at the ICESJMS online version of the manuscript. Data availability The data underlying this article will be shared on reasonable request to the corresponding author. Acknowledgements We acknowledge the support of Cefas, who funded this work under the project DP410 and the scientists and crew of the RV Cefas Endeavour for the collection of the survey data. Further thanks to Sven Kupschus and Jeroen van der Kooij for valuable discussion and advice during the development of this work and David Maxwell for commenting on the manuscript. References Agarap A. F. 2019 . Deep learning using rectified linear units (ReLU). arXiv preprint arXiv: 1803.08375. Bean T. P. , Greenwood N. , Beckett R. , Biermann L. , Bignell J. P. , Brant J. L. , Copp G. H. , et al. 2017 . A review of the tools used for marine monitoring in the UK: combining historic and contemporary methods with modeling and socioeconomics to fulfill legislative needs and scientific ambitions . Frontiers in Marine Science , 4 : 263 . Google Scholar Crossref Search ADS WorldCat Bowler E. , Fretwell P. T. , French G. , Mackiewicz M. 2019 . Using deep learning to count albatrosses from space. In IGARSS 2019-2019 IEEE International Geoscience and Remote Sensing Symposium, pp. 10099 – 10102 . IEEE. arXiv: 1907.02040v1 [cs.CV]. Cires D. , Meier U. 2012 . Multi-column deep neural networks for image classification. arXiv: 1202. 2745v1 [cs.CV]. Coggan R. , Diesing M. 2011 . The seabed habitats of the central English Channel: a generation on from Holme and Cabioch, how do their interpretations match-up to modern mapping techniques? Continental Shelf Research , 31 : S132 – S150 . Google Scholar Crossref Search ADS WorldCat Coggan R. , Froján C. R. B. , Diesing M. , Aldridge J. 2012 . Spatial patterns in gravel habitats and communities in the central and eastern English Channel . Estuarine, Coastal and Shelf Science , 111 : 118 – 128 . Google Scholar Crossref Search ADS WorldCat Cohen J. 1960 . A coefficient of agreement for nominal scales . Educational and Psychological Measurement , 20 : 37 – 46 . Google Scholar Crossref Search ADS WorldCat Diegues A. , Pinto J. , Ribeiro P. , Frias R. , Alegre D. C. 2018 . Automatic habitat mapping using convolutional neural networks. AUV 2018—2018 IEEE/OES Auton Underw Veh Work Proc. doi:10.1109/AUV.2018.8729787. Diesing M. , Coggan R. , Vanstaen K. 2009 . Widespread rocky reef occurrence in the central English Channel and the implications for predictive habitat mapping . Estuarine, Coastal and Shelf Science , 83 : 647 – 658 . Google Scholar Crossref Search ADS WorldCat Fleiss J. L. 1981 . Statistical Methods for Rates and Proportions , 2nd edn. John Wiley , New York . ISBN: 978-0-471-26370-8 Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC French G. , Fisher M. , Mackiewicz M. , Needle C. 2015 . Convolutional neural networks for counting fish in fisheries surveillance video. In Proceedings of the Machine Vision of Animals and their Behaviour (MVAB), pp. 7.1 – 7.10 . Guirado E. , Tabik S. , Alcaraz-Segura D. , Cabello J. , Herrera F. 2017 . Deep-learning convolutional neural networks for scattered shrub detection with Google earth imagery. arXiv preprint arXiv: 1706.00917. He K. , Zhang X. , Ren S. , Sun J. 2019 . Identity mappings in deep residual networks . Computer Vision , 9908 : 630 – 645 . Google Scholar OpenURL Placeholder Text WorldCat Howell K. L. , Bullimore R. D. , Foster N. L. 2014 . Quality assurance in the identification of deep-sea taxa from video and image analysis: response to Henry and Roberts . ICES Journal of Marine Science , 71 : 899 – 906 . Google Scholar Crossref Search ADS WorldCat Hu B. G. , Gosine R. G. , Cao L. X. , De Silva C. W. 1998 . Application of a fuzzy classification technique in computer grading of fish products . IEEE Transactions on Fuzzy Systems , 6 : 144 – 152 . Google Scholar Crossref Search ADS WorldCat Hyder K. , Rossberg A. G. , Allen J. I. , Austen M. C. , Barciela R. M. , Bannister H. J. , Blackwell P. G. , et al. 2015 . Making modelling count-increasing the contribution of shelf-seas community and ecosystem models to policy development and management . Marine Policy , 61 : 291 – 302 . Google Scholar Crossref Search ADS WorldCat ICES. 2018 . Report of the Workshop on Machine Learning in Marine Science (WKM-LEARN), 16–20 April 2018, ICES Headquarters, ICES, Copenhagen, Denmark. Kelley A. E. , Sherman K. 2018 . Trends of the Large Marine Ecosystem Assessment and Management Approach as Reflected in the Literature . Ocean and Coastal Management , 155 : 1 – 21 . Google Scholar Crossref Search ADS WorldCat Kingma D. P. , Salimans T. , Welling M. 2015 . Variational dropout and the local reparameterization trick. In Advances in Neural Information Processing Systems: Proceedings of the 28th International Conference on Neural Information Processing Systems, Vol 2 , pp. 2575 – 2583 . Knol M. 2013 . Making ecosystem-based management operational: integrated monitoring in Norway . Maritime Studies , 12 : 17 . Google Scholar Crossref Search ADS WorldCat Krizhevsky A. , Sutskever B. , Hinton G. 2012 . ImageNet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems, pp, 1097 – 1105 . Kupschus S. , Schratzberger M. , Righton D. 2016 . Practical implementation of ecosystem monitoring for the ecosystem approach to management . Journal of Applied Ecology , 53 : 1236 – 1247 . Google Scholar Crossref Search ADS WorldCat Lamarche G. , Lurton X. 2018 . Recommendations for improved and coherent acquisition and processing of backscatter data from seafloor-mapping sonars . Marine Geophysical Research , 39 : 5 – 22 . Google Scholar Crossref Search ADS WorldCat Lloffe S. , Szegedy C. 2015 . Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv: 1502.03167. Long J. , Shelhamer E. , Darrell T. 2015 . Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431 – 3440 . Manly B. F. J. 2006 . Randomization, Bootstrap and Monte Carlo Methods in Biology , 3rd edn. Chapman and Hall/CRC, London . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC Marine Alliance for Science and Technology Scotland. 2018 . Model summaries: MASTS. Retrieved from MASTS website: https://www.masts.ac.uk/research/marine-ecosystem-modelling/model-summaries/. Marine Ecosystem Research Programme. 2018 . dynamic ecosystem models: Marine Ecosystem Research Programme. Retrieved from MERP website: http://www.marine-ecosystems.org.uk/Research_outcomes/Model_Interactive. Martin D. , Fowlkes C. C. , Malik J. 2004 . Learning to detect natural image boundaries using local brightness, color, and texture cues . IEEE Transactions on Pattern Analysis and Machine Intelligence , 26 : 530 – 549 . Google Scholar Crossref Search ADS PubMed WorldCat Moriarty M. , Sethi S. A. , Pedreschi D. , Smeltz T. S. , McGonigle C. , Harris C. P. , Wolf N. , et al. 2020 . Combining fisheries surveys to inform marine species distribution modelling . ICES Journal of Marine Science , 77 : 539 – 552 . Google Scholar Crossref Search ADS WorldCat Norouzzadeh M. S. , Nguyen A. , Kosmala M. , Swanson A. , Palmer M. S. , Packer C. , Clune J. 2018 . Automatically identifying, counting, and describing wild animals in camera-trap images with deep learning . Proceedings of the National Academy of Sciences of the United States of America , 115 : E5716 – E5725 . Google Scholar Crossref Search ADS PubMed WorldCat Pedregosa F. , Varoquaux G. , Gramfort A. , Michel V. , Thirion B. , Grisel O. , Blondel M. , et al. 2011 . Scikit-learn: machine learning in python . Journal of Machine Learning Research , 12 : 2825 – 2830 . Google Scholar OpenURL Placeholder Text WorldCat Pikitch E. K. , Santora C. , Babcock E. A. , Bakun A. , Bonfil R. , Conover D. O. , Dayton P. , et al. 2004 . Ecosystem-based fisheries management . Science , 305 : 346 – 347 . Google Scholar Crossref Search ADS PubMed WorldCat QGIS Development Team. 2018 . QGIS Geographical Information System. Open Source Geospatial Foundation Project. QGIS Development Team.. http://q.gis.osgeo.org. R Core Team. 2018 . R: A Language and Environment for Statistical Computing . R Foundation for Statistical Computing , Vienna, Austria . Retrieved from ttps://www.R-project.org/. Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC Rees H. L. , Pendle M. A. , Waldock R. , Limpenny D. S. , Boyd S. E. 1999 . A comparison of benthic biodiversity in the North Sea, English Channel, and Celtic Seas . ICES Journal of Marine Science , 56 : 228 – 246 . Google Scholar Crossref Search ADS WorldCat Simonyan K. , Zisserman A. 2014 . Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv: 1409.1556. Stephens D. , Coggan R. , Diesing M. 2011 . Geostatistical modelling of surficial sediment composition in the North Sea and English Channel: using historical data to improve confidence in seabed habitat maps. In 2011 ICES Annual Science Conference, Gdańsk, Poland, pp. 1 – 8 . Stevens J. P. , Blackstock T. H. , Howe E. A. , Stevens D. P. 2004 . Repeatability of Phase 1 habitat survey . Journal of Environmental Management , 73 : 53 – 59 . Google Scholar Crossref Search ADS PubMed WorldCat Uy M. A. , Pham Q. H. , Hua B. S. , Nguyen T. , Yeung S. K. 2019 . Revisiting point cloud classification: a new benchmark dataset and classification model on real-world data. In Proceedings of the IEEE International Conference on Computer Vision, pp. 1588 – 1597 . Van Der Kooij J. , Kupschus S. , Scott B. E. 2011 . Delineating the habitat of demersal fish assemblages with acoustic seabed technologies . ICES Journal of Marine Science , 68 : 1973 – 1985 . Google Scholar Crossref Search ADS WorldCat Wilson C. 2016 . Methods of Interpreting Seabed Bathymetric Data for Use in the Planning of Environmental Monitoring Surveys . Cefas , Lowestoft . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC © International Council for the Exploration of the Sea 2020. All rights reserved. For permissions, please email: journals.permissions@oup.com This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model)
TI - Developing the use of convolutional neural networking in benthic habitat classification and species distribution modelling
JF - ICES Journal of Marine Science
DO - 10.1093/icesjms/fsaa208
DA - 2020-12-01
UR - https://www.deepdyve.com/lp/oxford-university-press/developing-the-use-of-convolutional-neural-networking-in-benthic-WzeAzT46nI
SP - 3074
EP - 3082
VL - 77
IS - 7-8
DP - DeepDyve
ER -