The application of artificial intelligence to benthic species identification

Primary supervisor: Dr Kerry Howell (University of Plymouth)

Secondary supervisor: Professor Graham D. Finlayson, (University of East Anglia). Email: G.Finlayson@uea.ac.uk

Additional supervisors:

Dr Phil Culverhouse, (University of Plymouth)

Dr Jill Schwarz (University of Plymouth)

Project description

Marine benthic ecosystems are chronically under-sampled, particularly in environments >50m. Yet a rising level of anthropogenic threats makes data collection ever more urgent. 

Currently, modern underwater sampling tools, particularly Autonomous Underwater Vehicles (AUV) and Remotely Operated Vehicles (ROV), are able to collect vast image datasets, but cannot bypass the bottleneck formed by manual image annotation. 

Computer Vision (CV) can be a faster, more consistent, cost-effective and a sharable alternative to manual annotation. 

The application of CV to benthic ecology is in its infancy. Recent research has shown some promising results, however, there is a need for further development of both methods and tools available in order to bring CV into the toolbox used in benthic biodiversity and ecological studies. 

This studentship will focus on the development and testing of an effective CV based image processing pipeline. It will test the application of both existing CV tools (for example using Matlab, Google’s Tensor Flow or R based algorithms) as well as novel methods including the use of underwater hyperspectral image data, and hybrid CV models. 

The student will have a unique opportunity to expand their outlook into a highly multi-disciplinary domain. They will interact with ecologists, computer scientists, engineers, ocean scientists, and photographers developing a wide network beyond the supervisory team. 

Depending on their background the student may receive training in ecology and taxonomy, computer vision, machine learning, marine optics, Matlab, R and Python programming. The student will spend periods of time at UEA and thus will benefit from interaction with two academic institutions. A degree in either an ecological field, computer science field or other highly numerate fields e.g. mathematics, engineering etc is required. 

We recognise that candidates are unlikely to have both ecological and programming skills. Thus we are looking for someone with a strong mathematical background and a demonstrable capacity to learn new skills and adapt their knowledge to new situations. 

Skills in use of statistical and/or computational models (for example one or more of the following - GLMS, GAMS, multivariate statistics, machine learning, convolutional neural networks) are essential.

Manual analysis is a time-consuming process which forms the current bottleneck in image-based marine ecological sampling 1,2,3. In addition, manual image annotation results are subject to observer bias4,5 and results (format, taxonomic resolution and nomenclature) differ from one institution, project or individual annotator to another. 

This lack of standardisation makes merging and comparing datasets difficult 6,7,8 and the data quality is not always consistent. Artificial intelligence (AI) and computer vision (CV) provide potential means by which to both accelerate and standardise the interpretation of image data 4,9,10,11.

The aim of the studentship is to advance the use of CV in image analysis for marine ecological study and biodiversity assessments.

The objectives each align with a planned theses chapter and peer-reviewed publication

  • Quantify the performance of selected CV algorithms in the identification of benthic taxa from field acquired imagery.
  • Quantify change in performance of selected CV algorithms when used in serial on smaller pools of taxa following12.
  • Quantify the change in performance of selected CV algorithms when data from hyperspectral imagery is included as terms in the CV models.
  • Develop a novel pipeline to analyse seafloor imagery using CV methods.

Across 1-3 the project will quantify how performance varies with algorithm/implementation complexity (including benchmarking classification using classical techniques). To address O1 and O2 the student will use existing CV tools (e.g. Matlab, Google’s Tensor Flow or R-based algorithms) and an existing dataset of seafloor imagery acquired using both AUV and ROV. 

To address O3 the student will use the University of Plymouth's TriOS sensor and prototype underwater hyperspectral camera and lighting to collect new image data in aquarium tests. To address O4 the student will draw upon their own findings (O1-3) to develop and test a CV based image processing pipeline accessible to ecologists.

The project is intellectually challenging due to its multi-disciplinary nature that will require the student to take on many new skills offering a rewarding experience. The topic is highly relevant to national research priorities making output highly citable. 

The student will become a leader in the application of AI to environmental survey and monitoring, making the student highly employable papers highly citable. We do not foresee any significant risks to completion of the project, which is desk and aquarium-based. 

The seafloor image data required for use by the project have already been acquired and manually analysed by a single observer. In addition pilot studies in the application of CV to benthic image analysis12, and in use of underwater hyperspectral imaging to discriminate between closely related marine species (Howell et al., in prep), have already been successfully undertaken in Howell’s research group and have demonstrated the potential of these methods to improve CV performance.

The student will be based in Howell’s research group where 1 PDRF and 3 PhD students are undertaking benthic image analysis for ecological research as part of externally funded projects including NERC, industry, and Government Agency funding.

References

  • Edgington DR, Cline DE, Davis D, Kerkez I, Mariette J (2006) Detecting, tracking and classifying animals in underwater video. Proc Oceans. IEEE
  • Beijbom O, Edmunds PJ, Roelfsema C, Smith J, Kline DI, Neal BP, Dunlap MJ, Moriarty V, Fan T-Y, Tan C-J (2015) Towards automated annotation of benthic survey images: Variability of human experts and operational modes of automation. PloS one 10:e0130312.
  • Schoening T, Durden J, Preuss I, Albu AB, Purser A, De Smet B, Dominguez-Carrió C, Yesson C, de Jonge D, Lindsay D (2017) Report on the Marine Imaging Workshop 2017. Research Ideas and Outcomes 3:e13820
  • Culverhouse PF, Williams R, Reguera B, Herry V, González-Gil S (2003) Do experts make mistakes? A comparison of human and machine identification of dinoflagellates. Marine Ecology Progress Series 247:17-25
  • Durden JM, Bett BJ, Schoening T, Morris KJ, Nattkemper TW, Ruhl HA (2016) Comparison of image annotation data generated by multiple investigators for benthic ecology. Marine Ecology Progress Series
  • Bullimore RD, Foster NL, Howell KL (2013) Coral-characterized benthic assemblages of the deep Northeast Atlantic: defining "Coral Gardens" to support future habitat mapping efforts. Ices Journal of Marine Science 70:511-522
  • Althaus F, Hill N, Ferrari R, Edwards L, Przeslawski R, Schönberg CH, Stuart-Smith R, Barrett N, Edgar G, Colquhoun J (2015) A standardised vocabulary for identifying benthic biota and substrata from underwater imagery: the CATAMI classification scheme. PloS one 10:e0141039
  • McClain CR, Rex MA (2015) Toward a Conceptual Understanding of β-Diversity in the Deep-Sea Benthos. Annual Review of Ecology, Evolution, and Systematics 46:623-642
  • MacLeod N, Benfield M, Culverhouse P (2010) Time to automate identification. Nature 467:154-155
  • Beijbom O, Edmunds PJ, Kline DI, Mitchell BG, Kriegman D (2012) Automated annotation of coral reef survey images. Proc Computer Vision and Pattern Recognition (CVPR), IEEE Conference on. IEEE
  • Favret C, Sieracki JM (2016) Machine vision automated species identification scaled towards production levels. Systematic Entomology 41:133-143
  • Piechaud N, Culverhouse PF, Hunt C, Howell KL. (submitted) Automated Identification of benthic epifauna from images using computer vision. Marine Ecology Progress Series.

Funding

This project has been shortlisted for funding by the ARIES NERC Doctoral Training Partnership. Undertaking a PhD with ARIES will involve attendance at training events.

ARIES is committed to equality & diversity, and inclusion of students of any and all backgrounds. All ARIES Universities have Athena Swan Bronze status as a minimum.

Applicants from quantitative disciplines who may have limited environmental science experience may be considered for an additional three-month stipend to take appropriate advanced-level courses.

Usually, only UK and EU nationals who have been resident in the UK for three years are eligible for a stipend. Shortlisted applicants will be interviewed on 26/27 February 2019.

For further information please see www.aries-dtp.ac.uk or contact us at aries.dtp@uea.ac.uk.