Article abstract

Journal of Agricultural and Crop Research

Research Article | Published November 2020 | Volume 8, Issue 11. pp. 260-271.

doi: https://doi.org/10.33495/jacr_v8i11.20.180

 

Data augmentation method for strawberry flower detection in non-structured environment using convolutional object detection networks

 



 

 

Umme Fawzia Rahim*

Hiroshi Mineno

 

Email Author
Tel: +8108054923240.

 

Graduate School of Integrated Science and Technology, Shizuoka University, 3-5-1 Johoku, Naka-ku, Hamamatsu, Shizuoka 432 – 8081, Japan.



……....…...………..........................…………....………............…............……...........……........................................................………...……..…....……....…

Citation: Rahim UF, Mineno H (2020). Data augmentation method for strawberry flower detection in non-structured environment using convolutional object detection networks. J. Agric. Crop Res. 8(11):260-271. doi: 10.33495/jacr_v8i11.20.180.

……....…...………..........................…………....………............…............……...........……........................................................………...……..…....……....…



 Abstract 


Deep learning has demonstrated significant capabilities for learning image features and presents many opportunities for agricultural automation. Deep neural networks typically require large and diverse training datasets to learn generalizable models. However, this requirement is challenging for applications in agricultural automation systems, since collecting and annotating large amount of training samples from filed crops and greenhouses is an expensive and complicated process due to the large diversity of crops, growth seasons and climate changes. This research proposed a new method for augmenting training dataset using synthesized images that preserves the background context and texture of the data object. A synthetic dataset of 1800 images was generated using a reference dataset and applying image processing techniques. As reference dataset 100 and for evaluating detection performance 230 real images of strawberry flowers were collected in greenhouses. Experimental results demonstrated that the suggested method provides improved performance when applied to the state-of-the-arts convolutional object detectors including Faster R-CNN, SSD, YOLOv3 and CenterNet for the task of strawberry flower detection in non-structured environment. The YOLOv3 w/darknet53 model achieved 46.84% boost in performance with average precision (AP) improved from 39.20% to 86.04% when applied augmentation using synthetic dataset. The AP of Faster R-CNN w/resnet50, SSD w/resnet50 and FPN and CenterNet w/hourglass52 models improved by 15.71, 18.42 and 22.24%, respectively. The Faster R-CNN w/resnet50 model provided most significant strawberry flower detection performance with AP 90.84%, which is higher than SSD w/resnet50 and FPN, YOLOv3 w/darknet53 and CenterNet w/hourglass52 models (88.56%, 86.04 % and 83.82%, respectively).

Keywords  Flower detection   deep convolutional neural network   data augmentation   synthetic dataset   

 

 

Copyright © 2020 Author(s) retain the copyright of this article.or(s) retain the copyright of this article.

This article is published under the terms of the Creative Commons Attribution License 4.0

 

 

  References 

 

Adamsen FJ, Coffelt TA, Nelson JM, Barnes EM, Rice RC (2000). Method for using images from a color digital camera to estimate flower number. Crop Sci. 40:704-709.

 

Aquino A, Millan B, Gutiérrez S, Tardáguila J (2015). Grapevine flower estimation by applying artificial vision techniques on images with uncontrolled scene and multi-model analysis. Comput. Electron. Agric. 119:92-104.

 

Bac CW, van Henten EJ, Hemming J, Edan Y (2014). Harvesting robots for high-value crops: State‐of‐the‐art review and challenges ahead. J. F. Robot. 31:888-911.

 

Bairwa N, Agrawal NK (2014). Counting of flowers using image processing. Int. J. Eng. Res. Technol. 3:775-779.

 

Dcunha S, Das J, Qu C (2017). Counting Apples and Oranges with Deep Learning : https://doi.org/10.1109/LRA.2017.2651944.

 

De Brabandere B, Neven D, Van Gool L (2017). Semantic instance segmentation with a discriminative loss function. arXiv Prepr. arXiv1708.02551.

 

Deng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L (2009). Imagenet: A large-scale hierarchical image database, in: 2009 IEEE Conference on Computer Vision and Pattern Recognition. Ieee, pp. 248-255.

 

Dias PA, Tabb A, Medeiros H (2018). Apple flower detection using deep convolutional networks. Comput. Ind. 99, 17–28. https://doi.org/10.1016/j.compind.2018.03.010.

 

Douarre C, Schielein R, Frindel C, Gerth S, Rousseau D (2018). Transfer learning from synthetic data applied to soil–root segmentation in x-ray tomography images. J. Imaging 4, 65.

 

Duan K, Bai S, Xie L, Qi H, Huang Q, Tian Q (2019). Centernet: Keypoint triplets for object detection, in: Proceedings of the IEEE International Conference on Computer Vision. pp. 6569-6578.

 

Dyrmann M, Mortensen AK, Midtiby HS, Jørgensen RN (2016). Pixel-wise classification of weeds and crops in images by using a Fully Convolutional neural network. Int. Conf. Agric. Eng. p. 6.

 

Everingham M, Van Gool L, Williams CKI, Winn J, Zisserman A (2010). The pascal visual object classes (voc) challenge. Int. J. Comput. Vis. 88:303-338.

 

Girshick R, Donahue J, Darrell T, Malik J (2015). Region-based convolutional networks for accurate object detection and segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 38:142-158.

 

He K, Zhang X, Ren S, Sun J (2016). Deep residual learning for image recognition, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 770-778.

 

Hočevar M, Širok B, Godeša T, Stopar M (2014). Flowering estimation in apple orchards by image analysis. Precis. Agric. 15:466-478. https://doi.org/10.1007/s11119-013-9341-6.

 

Huang J, Rathod V, Sun C, Zhu M, Korattikara A, Fathi A, Fischer I, Wojna Z, Song Y, Guadarrama S (2017). Speed/accuracy trade-offs for modern convolutional object detectors, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 7310-7311.

 

Kamilaris A, Prenafeta-Boldú FX (2018). Deep learning in agriculture: A survey. Comput. Electron. Agric. 147, 70–90. https://doi.org/10.1016/j.compag.2018.02.016.

 

Kapach K, Barnea E, Mairon R, Edan Y, Ben-Shahar O (2012). Computer vision for fruit harvesting robots–state of the art and challenges ahead. Int. J. Comput. Vis. Robot. 3:4-34.

 

Kaur R, Porwal S (2015). An optimized computer vision approach to precise well-bloomed flower yielding prediction using image segmentation. Int. J. Comput. Appl. p. 119.

 

Krizhevsky A, Sutskever I, Hinton GE (2012). Imagenet classification with deep convolutional neural networks, in: Advances in Neural Information Processing Systems. pp. 1097-1105.

 

LeCun Y, Bengio Y, Hinton G (2015). Deep learning. Nature. 521:436-444.

 

Lin TY, Dollár P, Girshick R, He K, Hariharan B, Belongie S (2017). Feature pyramid networks for object detection, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 2117-2125.

 

Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C-Y, Berg AC (2016). Ssd: Single shot multibox detector, in: European Conference on Computer Vision. Springer, pp. 21-37.

 

Mannin CD, Manning CD, Schütze H (1999). Foundations of statistical natural language processing. MIT press.

 

Namin ST, Esmaeilzadeh M, Najafi M, Brown TB, Borevitz JO (2018). Deep phenotyping : deep learning for temporal phenotype / genotype classification. Plant Methods. pp. 1-14. https://doi.org/10.1186/s13007-018-0333-4.

 

Oppenheim D, Edan Y, Shani G (2017). Detecting Tomato Flowers in Greenhouses Using Computer Vision. Int. J. Comput. Electr. Autom. Control Inf. Eng. 11:104-109.

 

Plebe A, Grasso G (2001). Localization of spherical fruits for robotic harvesting. Mach. Vis. Appl. 13:70-79.

 

Rahim UF, Mineno H (in press). Tomato Flower Detection and Counting in Greenhouses Using Faster Region-based Convolutional Neural Network. Journal of Image and Graphics.

 

Redmon J, Divvala S, Girshick R, Farhadi A (2016). You only look once: Unified, real-time object detection, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 779-788.

 

Redmon J, Farhadi A (2018). YOLOv3: An Incremental Improvement. Redmon J, Farhadi A (2017). YOLO9000: Better, faster, stronger. Proc. 30th IEEE Conf. Comput. Vis. Pattern Recognition, CVPR 2017 2017-Janua, pp. 6517-6525. https://doi.org/10.1109/CVPR.2017.690.

 

Ren S, He K, Girshick R, Sun J (2015). Faster r-cnn: Towards real-time object detection with region proposal networks, in: Advances in Neural Information Processing Systems. pp. 91-99.

 

Simonyan K, Zisserman A (2014). Very deep convolutional networks for large-scale image recognition. arXiv Prepr. arXiv1409.1556. Thorp KR, Dierig DA (2011). Color image segmentation approach to monitor flowering in lesquerella. Ind. Crops Prod. 34:1150-1159. https://doi.org/10.1016/j.indcrop.2011.04.002.

 

Tyagi AC (2016). Towards a second green revolution. Irrig. Drain. 65:388-389.

 

Ubbens J, Cieslak M, Prusinkiewicz P, Stavness I (2018). The use of plant models in deep learning: An application to leaf counting in rosette plants. Plant Methods 14:1-10. https://doi.org/10.1186/s13007 -018-0273-z.

 

Ward D, Moghadam P, Hudson N (2018). Deep leaf segmentation using synthetic data. arXiv Prepr. arXiv1807.10931.

 

Woebbecke DM, Meyer GE, Von Bargen K, Mortensen DA (1995). Color indices for weed identification under various soil, residue, and lighting conditions. Trans. ASAE 38:259-269.

 

Xu R, Li C, Paterson AH, Jiang Y, Sun S, Robertson JS (2018). Aerial Images and Convolutional Neural Network for Cotton Bloom Detection. Front. Plant Sci. 8:1-17. https://doi.org/10.3389/fpls.2017. 02235.

 

Zhao Y, Gong L, Huang Y, Liu C (2016). A review of key techniques of vision-based control for harvesting robot. Comput. Electron. Agric. 127:311-323.

 

Zhou X, Wang D, Krähenbühl P (2019). Objects as points. arXiv Prepr. arXiv1904.07850.