Optical Method:  Overview

Background

Optical systems are powerful tools for enumerating and observing plankton. They form one third of the triad of in situ tools along with nets/pumps and high-frequency acoustics. In the laboratory, optical systems provide a means of archiving and processing zooplankton samples collected with nets or pumps in the field. Organisms can be imaged on scales of micrometers to centimeters at resolutions that enable identification to family, genus, or in some cases - species. Optical imaging is a rapidly-developing field whose advances benefit from the availability of low-cost, high-resolution digital cameras and compact, computing systems.

Field Applications

In situ systems have the advantage of collecting images of zooplankton in their natural orientations and in association with predators, prey, or other particles from which behaviour may be inferred. Fragile gelatinous taxa, which would normally be damaged or destroyed by nets/pumps can be enumerated using in situ imaging systems. Systems with large enough sampling volumes, are also useful for the real-time imaging of zooplankton of low abundance (e.g. fish larvae, gelatinous plankton). Examples of in situ systems include: the video plankton recorder (VPR), underwater video profiler (UVP), in situ ichthyoplankton imaging system (ISIIS), shadowed image particle profiler and environmental recorder (SIPPER), zooplankton visualization system (ZOOVIS), lightframe on-sight key species investigation (LOKI) system, FlowCytoBot, and others. Technology for field applications is quickly evolving with new systems regularly coming into place. Newest advancements include the autonomous zooplankton sensing glider (Zooglider, Oman et al. 2018).

Lab Applications

Laboratory systems for processing plankton samples include the ZooScan, FlowCAM, flatbed scanners utilizing ZOOIMAGE software and ZooCam. A hybrid system called the Line-scanning Zooplankton Analyzer (LiZA) allows analysis of flow-through samples collected at sea. Newest advancements include another at-sea instrument, the Plankton Imager, a high-speed colour line scan-based imaging instrument connected to the ships water supply (Pitois et al. 2020)

Image Processing

Hardware is just a part of the imaging equation. With acquisition rates exceeding 10 Hz in some systems, it is quite possible to collect terabytes of data on a single cruise. Without software to enable rapid processing of images, optical systems would suffer from the same problem as net sampling … the accumulation of large amounts of unprocessed samples. Advances in computing power have allowed the development of segmentation algorithms, which isolate objects of interest from the background, combined with machine learning systems that utilize classification algorithms to train a computer to identify unknown objects based on training sets provided by a human. Many of these tasks are now performed in real-time, while at sea. This opens up a new dimension in sampling. No longer do we simply collect samples and then wait for months or years to see what they contain. Novel information can be provided to scientists at sea who can then modify their sampling design to capitalize on the new information. When used effectively, machines can perform repetitive tasks with ease, leaving humans to focus on correcting misclassifications and interpret patterns.

Image Classification / Machine Learning

[Section in progress: need a volunteer to write this. Felipe? ]  The use of image analysis combined with identification algorithms used in machine/deep learning and artificial intelligence is a rapidly growing field of research and can have a direct impact on the rate at which changes in plankton communities can be evaluated.

Sample method papers

Cheng K, Cheng X, Wang Y, Bi H, Benfield MC (2019) Enhanced convolutional neural network for plankton identification and enumeration. PLoS ONE 14(7): e0219570. https://doi.org/10.1371/ journal.pone.0219570

Ellen, J. S., Graff, C. A., & Ohman, M. D. (2019). Improving plankton image classification using context metadata. Limnology and Oceanography: Methods, 17(8), 439-461. https://doi.org/10.1002/lom3.10324

Zheng, H., Wang, R., Yu, Z., Wang, N., Gu, Z., & Zheng, B. (2017). Automatic plankton image classification combining multiple view features via multiple kernel learning. BMC bioinformatics, 18(16), 1-18. https://doi.org/10.1186/s12859-017-1954-8

Sample method papers & associated code

Kerr, T., Clark, J. R., Fileman, E. S., Widdicombe, C. E., & Pugeault, N. (2020). Collaborative deep learning models to handle class imbalance in FlowCam plankton imagery. IEEE Access, 8, 170013-170032.
https://gitlab.ecosystem-modelling.pml.ac.uk/tk359/flowcamclassification

Lumini, A., Nanni, L. and Maguolo, G. (2020), "Deep learning for plankton and coral classification", Applied Computing and Informatics, . https://doi.org/10.1016/j.aci.2019.11.004
https://github.com/LorisNanni/Deep-Learning-and-Transfer-Learning-Features-for-Plankton-Classification

Pastore, V.P., Zimmerman, T.G., Biswas, S.K. et al. Annotation-free learning of plankton for classification and anomaly detection. Sci Rep 10, 12142 (2020). https://doi.org/10.1038/s41598-020-68662-3
https://github.com/sbianco78/UnsupervisedPlanktonLearning