Abstract
The labeling of Environmental Microorganisms (EM) which help decomposing pollutants, plays a fundamental role for establishing sustainable ecosystem. We propose an environmental microorganism classification engine that can automatically analyze microscopic images using Conditional Random Fields (CRF) and Deep Convolutional Neural Networks (DCNN). First, to effectively represent scarce training images, a DCNN pre-trained for image classification using a large amount of data is re-purposed to our feature extractor that distils pixel-level features in microscopic images. In addition, pixel-level classification results by such features can be refined using global features that describe the whole image in toto. Finally, our CRF model localizes and classifies EMs by considering the spatial relations among DCNN-based features, and their relations to global features. The experimental results have shown 94.2% of overall segmentation accuracy and up to 91.4% mean average precision of the results.
| Original language | English |
|---|---|
| Journal | Pattern Recognition |
| Volume | 77 |
| Pages (from-to) | 248-261 |
| Number of pages | 14 |
| ISSN | 0031-3203 |
| DOIs | |
| Publication status | Published - 01.05.2018 |
Funding
In this paper, we introduced an EM classification system. Considering the small training dataset problem, we adopt an approach where a DCNN pre-trained on large auxiliary image data is re-purposed and fine-tuned to a pixel-level feature extractor using EM images. The global features are used to support the classification and improve the segmentation quality by providing a long-range consistency between pixel labels. To overcome the noisy image problem, CRF is used to jointly localize and classify EMs by considering the spatial relations among pixel-level features, and their relations to global features. Experimental results validate the effectiveness of each of these three contributions (i.e., DCNN features, global features and CRF). From the architectural perspective, our system consists of two main parts, feature extraction based on DCNN features, and pixel-level classification using the local-global CRF. Both parts benefit from the above-mentioned contributions. The experimental results in Table 3 have justified the usefulness of DCNN features over other common features. Although the performance improvement is not so dramatic as DCNN features, Table 4 validates the advantage of the CRF over denseCRF, with the support of global features that give about 8% improvement, as seen from Table 3 . But, our EM classification system provides a freedom in choosing feature extraction and classification methods, so the DCNN-based method and the local-global CRF can be replaced with more advanced ones in the future. Regarding the technical improvement, we will use more advanced pairwise potentials, trained with a DCNN to connect the global node with the local nodes. This will allow to omit the assumption that only one EM present in the scene. Especially, rather than optimizing DCNN-based potentials and a CRF separately, we will learn ‘message estimators’ to efficiently perform their joint optimization [64] . Here, instead of explicitly compute potentials, message estimators only output messages required in a message passing algorithm (in our case Loopy Belief Propagation (LBP)). Sergey Kosov received his Diploma in Applied Mathematics from the Kirgiz-Russian Slavic University, Kirgyzstan in 2004, and M.Sc. degree in Computer Science from the Saarland University, Germany in 2008. From 2008 to 2013, he worked as a researcher in the Max Plank Institute for Informatics and Leibniz University, Germany. Currently he is an external Ph.D. student at Pattern Recognition Group in University of Siegen, Germany. His research interests include classification with conditional random fields and deep neural networks, motion estimation with optical ow, 3-D reconstruction as well as movie industry. Kimiaki Shirahama received his B.E., M.E. and D.E degrees in Engineering from Kobe University, Japan in 2003, 2005 and 2011, respectively. After working as an assistant professor in Muroran Institute of Technology, Japan, since 2013, he is working as a postdoctoral researcher at Pattern Recognition Group in University of Siegen, Germany. From 2013 to 2015, his research activity was supported by the Postdoctoral Fellowship of Japan Society for the Promotion of Science (JSPS), and is now supported within a project of German Federal Ministry of Education and Research (BMBF). His research interests include multimedia data processing, machine learning, data mining and sensor- based human activity recognition. He is a member of ACM SIGKDD, ACM SIGMM, the Institute of Image Information and Television Engineers in Japan (ITE), Information Processing Society of Japan (IPSJ) and the Institute of Electronics, Information and Communication Engineering in Japan (IEICE). Chen Li received his B.E. degree in the University of Science and Technology Beijing, China in 2008, M.Sc. degree in the Northeast Normal University, China in 2011, and Dr.-Ing. degree in the University of Siegen, Germany in 2016. From 2016 to 2017, he worked as a postdoctoral researcher in the Johannes Gutenberg University of Mainz, Germany. Currently, he is working as an associate professor in the Northeastern University, China. His research interests are machine learning, pattern recognition, microscopic and biomedical image analysis. Marcin Grzegorzek is Head of the Research Group for Pattern Recognition at the University of Siegen, Professor at the Department of Knowledge Engineering at the University of Economics in Katowice and Chairman of the Board of Data Understanding Lab Ltd. He studied Computer Science at the Silesian University of Technology, did his Ph.D. at the Pattern Recognition Lab at the University of Erlangen-Nuremberg, worked scientifically as Postdoc in the Multimedia and Vision Research Group at the Queen Mary University of London and at the Institute for Web Science and Technologies at the University of Koblenz-Landau, did his habilitation at the AGH University of Science and Technology in Kraków. He published more than 100 papers in pattern recognition, image processing, machine learning, and multimedia analysis and acted as examiner in 18 finalised doctoral procedures. For the time being, he runs eight externally funded research projects. For instance, Marcin coordinates the project CogAge ( www.cognitivevillage.de ) aiming at developing a user-friendly support system for elderly that applies machine learning algorithms for sensor-based health assessment.
