The application of artificial intelligence to benthic species identification

Project description:

Marine benthic ecosystems are chronically under-sampled particularly in environments >50m. Yet a rising level of anthropogenic threats makes data collection ever more urgent. Currently, modern underwater sampling tools, particularly Autonomous Underwater Vehicles (AUV) and Remotely Operated Vehicles (ROV), are able to collect vast image datasets, but cannot bypass the bottleneck formed by manual image annotation. Computer Vision (CV) can be a faster, more consistent, cost effective and a sharable alternative to manual annotation. The application of CV to benthic ecology is in its infancy. Recent research has shown some promising results, however there is a need for further development of both methods and tools available in order to bring CV into the tool box used in benthic biodiversity and ecological studies. This studentship will focus on the development and testing of an effective CV based image processing pipeline. It will test the application of both existing CV tools (for example using Matlab, Google’s Tensor Flow or R based algorithms) as well as novel methods including use of underwater hyperspectral image data, and hybrid CV models. The student will have a unique opportunity to expand their outlook into a highly multi-disciplinary domain. They will interact with ecologists, computer scientists, engineers, ocean scientists, and photographers developing a wide network beyond the supervisory team. Depending on their background the student may receive training in ecology and taxonomy, computer vision, machine learning, marine optics, Matlab, R and Python programming. A degree in either an ecological field, computer science field, or other highly numerate field e.g. MTH, engineering etc is required. We recognise that candidates are unlikely to have both ecological and programming skills. Thus we are looking for someone with a strong mathematical background and a demonstrable capacity to learn new skills and adapt their knowledge to new situations. Skills in use of statistical and / or computational models (for example one or more of the following - GLMS, GAMS, multivariate statistics, machine learning, convolutional neural networks) are essential.

Supplementary information: 

Manual analysis is a time consuming process which forms the current bottleneck in image based marine ecological sampling. In addition manual image annotation results are subject to observer bias [1] and results (format, taxonomic resolution and nomenclature) differ from one institution, project or individual annotator to another. This lack of standardisation makes merging and comparing datasets difficult [2] and the data quality is not always consistent. Artificial intelligence (AI) and computer vision (CV) provide potential means by which to both accelerate and standardise the interpretation of image data [1,3]. 

The aim of the studentship is to advance the use of CV in image analysis for marine ecological study and biodiversity assessments. 

The objectives each align with a planned theses chapter and peer reviewed publication 

1. Quantify the performance of selected CV algorithms in identification of benthic taxa from field acquired imagery. (Paper 1). 

2. Quantify change in performance of selected CV algorithms when used in serial on smaller pools of taxa following12. (Paper 2). 

3. Quantify the change in performance of selected CV algorithms when data from hyperspectral imagery is included as terms in the CV models. (Paper 3). 

4. Develop a novel pipeline to analyse seafloor imagery using CV methods. (Paper 4). 

Across 1-3 the project will quantify how performance varies with algorithm / implementation complexity (including benchmarking classification using classical techniques). To address O1 and O2 the student will use existing CV tools (e.g. Matlab, Google’s Tensor Flow or R- based algorithms) and an existing dataset of seafloor imagery acquired using both AUV and ROV. To address O3 the student will use Plymouth University’s TriOS sensor and prototype underwater hyperspectral camera and lighting to collect new image data in aquarium tests. To address O4 the student will draw upon their own findings (O1-3) to develop and test a CV based image processing pipeline accessible to ecologists. 

The project is intellectually challenging due to its multi-disciplinary nature that will require the student to take on many new skills offering a rewarding experience. The topic is highly relevant to national research priorities making output highly citable. The student will become a leader in the application of AI to environmental survey and monitoring, making the student highly employable and papers highly citable. We do not foresee any significant risks to completion of the project, which is desk and aquarium based. The seafloor image data required for use by the project have already been acquired and manually analysed by a single observer. In addition pilot studies in the application of CV to benthic image analysis [4], and in use of underwater hyperspectral imaging to discriminate between closely related marine species [unpublished data and 5], have already been successfully undertaken in Howell’s research group and have demonstrated the potential of these methods to improve CV performance. 

The student will be based in Howell’s research group where 2 PDRFs and 3 PhD students are undertaking benthic image analysis for ecological research as part of externally funded projects including NERC, industry, and Government Agency funding.


1. Culverhouse PF, Williams R, Reguera B, Herry V, González-Gil S (2003) Doexperts make mistakes? A comparison of human and machine identification ofdinoflagellates. Marine Ecology Progress Series 247:17-25. 

2. Howell KL, Davies JS, Allcock AL, Braga-Henriques A, Buhl-Mortensen P et al(2019) A framework for the development of a global standardised marine taxonreference image database (SMarTaR-ID) to support image-based analyses. BioRXivand in review at PlosOne. 

3. MacLeod N, Benfield M, Culverhouse P (2010) Time to automate identification.Nature 467:154-155. 

4. Piechaud N, Culverhouse PF, Hunt C, Howell KL. (2019) Automated Identificationof benthic epifauna from images using computer vision. Marine Ecology ProgressSeries. 615, 15-30. 

5. Schwarz JN, (2001). The use of high spectral resolution in-situ optical data formonitoring case II (coastal) water quality. PhD University of Southampton.