
Shape is an important physical property of an object that characterizes its appearance. It is often represented by the boundary of the object in an image or a video as in Figure 1. Using this representation as data, a variety of statistical analysis has been conducted. Particularly, the classification of shape is one of the most fundamental tasks in many application fields, ranging from medical imaging, and computer vision to bioinformatics. It can also be applied to environmental science and engineering.
Traditionally, various mathematical representations of shape were proposed and developed, which include (unordered) point clouds (Besl and McKay, 1992), a set of (ordered) finite points called landmarks (Dryden and Mardia, 2016), level sets (Malladi
However, there have been steady efforts to develop a framework for the representation of shape using its entire contour instead of a finite number of points. The continuous curve seems to be more natural when analyzing its shape. Additionally, there is an issue regarding how and where to choose landmarks if we represent shape by discrete points. As a result, functional representations of curves for shape have recently been developed. For instance, the shape of an alga in Figure 1 is represented as a planar curve, so regarded as two-dimensional functional data. Yet, we need to consider the invariance of an additional transformation as well as translation, scaling, and rotation when comparing shapes of two different continuous curves. This is re-parameterization, which is a smooth one-to-one transformation of the domain of curves. For instance, the two curves (middle and right) in Figure 2 have an identical shape no matter how we parameterize the curves.
Moreover, if we assume that the curve is closed such as in Figure 1, we need a closure condition such that the mapping from a domain to the curve describes the traversal of the shape with the same starting and ending points. To solve these challenges, mathematical frameworks under the Riemannian structure have been developed by Michor and Mumford (2006) and Srivastava
There has been an increasing need for studying algal species in water as the overgrowth of algae has a critical impact on the management of drinking water supply systems (Coltelli
Regular monitoring of algal blooms is an important task for ensuring the safety of water supply systems. The direct counting of algal cells using a microscope is a traditional method for monitoring the status of algal blooms, but it is a time-consuming process that requires intensive labor from researchers. Therefore, efforts to develop automated technology to reduce time and effort in algal cell identification have continued. Understanding the differences and similarities between algal shapes can help in the identification of algal genera. The algal cell image dataset used in this study was collected using a digital imaging flow cytometer and microscope (FlowCAM, Fluid Imaging Technologies, Yarmouth, ME, USA) provided by Korea Water Resources Corporation (K-water). The dataset includes a total of 2571 morphological images of six algal genera, including three cyanobacteria (Microcystis sp., Oscillatoria sp., and Anabaena sp.), two diatoms ( Fragilaria sp. and Synedra sp.), and a green alga (Pediastrum sp.) (Park
Many approaches for morphological identification of algae in watersheds have been developed based on its images, which were captured by some microscope devices such as a digital imaging flow cytometer and microscope (FlowCAM). Using the pixel values with some image analysis tools, many methods are proposed for the classification of algae, including the convolutional neural network (CNN) (Medina
However, the shape of algal species has seldom been studied although it is one of the most important features for identification. The shape of algal species, which is represented as a planar closed curve that is extracted from its microscope image, is suitable for classification. In this paper, we adopt and describe the elastic shape analysis framework with a novel functional representation called the square-root velocity function (SRVF). The benefits from using this representation and the elastic Riemannian metric are described along with the inherent geometry. Based on this mathematical framework, we can define the shape distance and further shape statistics when multiple sample curves are given. We then apply various well-known statistical classification methods to the dataset of algal shape. One group of the methods is based on pairwise shape distances such as the nearest neighbor classifiers. The other group is based on probability distributions for shape such as linear and quadratic discriminant analysis in the standard multivariate fashion.
In this paper, our contributions are as follows: (i) introducing various classification approaches for algae based on the elastic shape analysis framework, (ii) providing the experimental results from real environmental systems, and (iii) comparing and investigating the strengths and drawbacks of the presented approaches.
Under the elastic shape distance with the SRVF representation after removing translation, scaling, rotation and re-parameterization, we first evaluate the classification performance using the algal shape of
Since curves reside in non-linear manifold, it is complicated to build a probability model on their representation space. Even though the SRVF representation simplifies the elastic metric to the simple metric and the corresponding space becomes the unit Hilbert sphere, it is still non-Euclidean. Thus, we utilize a tangent space at a particular point on the sphere and a projection. Analytic expressions of some tools for this projection are well-known in differential geometry. Among many choices of the projection, we use the inverse-exponential map to project the SRVFs on the shape space into a tangent space produced at the sample mean shape. Since it is now a Euclidean space, we can use the standard principal component analysis (PCA) to reduce dimension of the data and construct a probability model with the projected shape representations.
We first apply the Gaussian distribution with low-dimensional principal components on the tangent space and classify some test shapes by the Gaussian likelihoods after estimating the mean and covariance for each species. Assuming both equal and unequal covariance structures as linear and quadratic discriminant analysis (LDA & QDA), we set these model-based classification procedures as baseline (Pal
The rest of this paper is organized as follows. Section 2 briefly reviews the geometric framework for elastic shape analysis of planar curves. Section 3 begins by describing various classification approaches. First a few nonparametric methods are based on the pairwise distances, such as the nearest neighbors and the nearest mean rules. The other model-based methods need the computation of some relevant statistics and the estimation of appropriate probability distributions. We then introduce two standard procedures and two additional ones, which rely on pairwise statistics and dimension reduction to different degrees. Section 4 provides empirical studies that show applications of these diverse procedures in shape classification for algal species. Section 5 provides a short discussion and lays out some directions for future work.
In this section, we briefly describe a Riemannian geometric framework for non-Euclidean space that shape of curves reside in. Among many functional representations and metrics of curves for shape, we adopt the square-root velocity function (SRVF) and the elastic Riemannian metric to compute the shape distance. More details of this elastic shape analysis framework are provided by Srivastava and Klassen (2016).
We represent the shape of an algae as an absolutely continuous, parameterized curve in ℝ2. For the closed curves that we used, they are denoted as where the domain
is a unit circle which implies that the starting and the ending points of
Once we set the functional representation for algal shape, we next need an appropriate distance between two curves
The distance between two curves given by
where metric, which facilitates the computation of distances between two curves (Srivastava
distance on the space of SRVFs is equivalent to a particular elastic distance on the space of curves. The
distance between SRVFs is invariant to rotation and re-parameterization (Kurtek
(Robinson, 2012). Finally, one can uniquely recover the curve
Since the SRVF representation . Thus, the distance between any two different
is an arc length of the great circle of the unit sphere, and is computed by
, where
However, we still need to consider the rotation and re-parameterization of the curves. We use the concept of equivalence class, defined by [. It means that the shape space
is a quotient space of the pre-shape space
under the action of the rotation and re-parameterization groups. The shape distance, or geodesic between two curves is defined as the distance between their equivalence classes, and is computed by
If we denote the minimizers of .
Under the space of SRVFs with the distance that is invariant to all shape-preserving transformations, we define sample statistics for the shape of curves, which include the sample mean and the sample covariance. Although we derived the pre-space of the SRVFs as the unit sphere where computations are relatively easy, this transformed space is not Euclidean. Thus, traditional vector calculus cannot be applied. We adopt the concept of the tangent space so that standard statistical procedures are applicable on this linearized space. To approximate the tangent space, we need to choose an appropriate point on the sphere for projection and a tool for moving SRVFs onto the tangent space. We use the exponential map and its inverse as the projection tool in differential geometry. Figure 3 illustrates these two maps with via a three-dimensional unit sphere and
at
. The exponential map
and the inverse exponential map (
for , and where || · || is the
norm and
For a projection point which determines the tangent space and also affects any result of subsequent statistical analysis, the sample mean is one of the most commonly used. Once we have a sample of curves, we convert the curves into normalized SRVFs , and then the sample mean is computed using the shape distance
given in
It is called the sample Karcher mean. While this mean is an entire equivalence class based on the definition, we proceed with subsequent analysis simply by selecting one element
Given sample SRVFs and their mean shape, we can define the Karcher covariance on a locally linearized tangent space. Let
We describe three distance-based methods and four different model-based procedures for the classification of shape data that involves nonlinear registration and resides in non-Euclidean space. The presented classification approaches are all based on the elastic shape analysis framework introduced in the previous section.
We first consider two well-known nonparametric classification methods: (1)
The non-elastic distance is the minimizer of
The nearest mean classifier first finds the Karcher sample mean of the training shapes in each class using
Based on the Karcher sample mean and covariance under the elastic shape analysis framework with the SRVF representation, we consider four model-based classification methods by estimating appropriate probability distributions on the tangent space: (1) linear and quadratic discriminant analysis (LDA & QDA) on a single space (Pal
The first two classifiers use probability models on a single tangent space as a baseline. Let
where
The next two model-based classifiers are called the aggregated pairwise classifiers on a single tangent space, but with multiple tangent PC subspaces. Similarly, we first map all training shapes into a tangent space at the overall mean
We can also use Σ̂
To sum up, there are three choices in the outlined classification procedures using the Gaussian models: (1) LDA vs. QDA, (2) single vs. aggregated likelihoods from pairwise comparisons, and (3) the dimensionality of the PC space. Traditionally, many multivariate problems with respect to the first choice have been dealt with. For the second choice, the aggregated procedures are expected to lead to more robust classification performance, however it could be computationally more expensive. Determining optimal tangent PC dimension is not trivial, and requires an extensive search across various classification problems or datasets. In general, we aim to achieve a low-dimensional Euclidean representation of shape data via tPCs with decent performance of classification. Through the following empirical studies with algal shape data, these classification approaches are compared and some practical considerations are addressed.
In this section, we provide the application of the classification methods previously described to shape data of algae in water. For the nearest classifiers, we focus on the overall performance and the benefit of using the elastic distance rather than the non-elastic one. For the models based on multivariate Gaussian distributions, we compare the overall accuracy as well as investigate the trend of the performance with various choices of the tPC dimension.
The dataset we applied for in this work was first obtained from images, which were collected using a FlowCAM in a project by the KoreaWater Resources Corporation (K-water) in 2015. As described in the motivating subsection, the images were captured and collected in the major rivers in South Korea, and were used for classification by Park
In the preprocess to extract a curve from an image of an object of interest, the first step was to segment its region by converting the original image to one with a binary mask. After choosing an appropriate threshold pixel value for each image, we replaced all pixel values to either one or zero. Then, we extracted all coordinate values (
Here, we present classification results of various approaches using the elastic shape from algal images. The dataset consists of two-dimensional closed curves of the (
When comparing the three distance-based classification procedures using the cross-validation with 20 random splits:
When comparing the four model-based classification procedures using the cross-validation with the identical 20 splits for the distance-based methods: LDA, QDA, pairwise LDA, and pairwise QDA; the highest average accuracy of QDA and pairwise QDA were 77.25% (with a standard deviation of 1.22%), and 77.55%(1.11%) when using the first 35, 56 tPCs, respectively as shown in Figure 7. As the number of tPCs increased, the average accuracy of LDA and pairwise LDA consistently increased while the accuracy of QDA and pairwise QDA increased up to a certain number of PCs, and then started to decrease. Overall, the QDA classification methods showed better performance than the LDA-type approaches. The discrepancy of their performance was large when a smaller number of tPCs were used. Moreover, the pairwise methods, by aggregating multiple likelihoods from pairwise PC spaces tended to have slightly more robust classification results for various choices of tPCs.
The overall classification accuracy of the model-based approaches was worse than that of the nearest neighbor classifiers based on the elastic distance. This might be caused by some characteristics that the probability models have: (1) classification by the global structure of the estimated distributions on the reduced dimensional space and (2) some distortion between an approximated tangent space and the original shape space.
We introduced several classification approaches for shape data based on the elastic shape analysis framework, and applied them to algal identification via their shape. Since we assumed the representation for algal shape as a continuous planar curve, we needed to overcome some challenges for analysis: (1) invariance properties of shape, (2) nonlinearity of its representation space, and (3) high dimensionality. Based on the elastic metric with the square-root velocity function, we could define the shape distance that led to better classification results. Based on the linearization of the data in tangent spaces and the dimension reduction by PCA on the tangent space, we could build probability models on the lower-dimensional Euclidean space to be used for shape classification.
However, there are some practical issues for the presented classification approaches for elastic shape. First, the
In addition, there is room for improvement of the model-based classification. A single projection at the overall mean shape might result in distorted likelihoods for classification. If we develop more local linearization or better intrinsic ways to build statistical models, we can enhance the classification performance for shape. For future work, we will seek alternatives to linearization and dimension reduction procedures that can improve classification performance. Further enhancements, beyond the shape of algae, can be developed in methodology to incorporate other features such as texture (pixel or voxel values), inside the object of interest. We will consider statistical models applicable to various types of data that represent morphological features of an object in images or videos.
This study was supported by K-water (Korea Water Resources Corportion). This work was supported by a National Research Foundation of Korea (NRF) grant funded by the Korean government (RS-2022-00167077) and a INHA UNIVERSITY Research Grant.
Average classification rates (%) over 20 random splits of three distance-based methods for algal shape
Nearest neighbor | Nearest mean | ||||||||
---|---|---|---|---|---|---|---|---|---|
83.36 (1.14) | 84.02 (1.08) | 84.03 (0.88) | 83.63 (0.90) | 83.57 (0.86) | 83.28 (0.86) | 82.91 (0.87) | 73.91 (1.10) | ||
76.92 (0.88) | 76.33 (0.82) | 75.72 (1.09) | 75.00 (0.96) | 74.24 (1.08) | 73.68 (1.10) | 72.80 (1.06) | 72.18 (1.13) |