Fruit detection and 3D location using instance segmentation neural networks and structure-from-motion photogrammetry
| العنوان: | Fruit detection and 3D location using instance segmentation neural networks and structure-from-motion photogrammetry |
|---|---|
| المؤلفون: | Javier Ruiz-Hidalgo, Verónica Vilaplana, Jordi Gené-Mola, Eduard Gregorio, Josep Ramon Morros, Joan R. Rosell-Polo, Ricardo Sanz-Cortiella |
| المساهمون: | Universitat Politècnica de Catalunya. Departament de Teoria del Senyal i Comunicacions, Universitat Politècnica de Catalunya. GPI - Grup de Processament d'Imatge i Vídeo |
| المصدر: | Recercat. Dipósit de la Recerca de Catalunya instname UPCommons. Portal del coneixement obert de la UPC Universitat Politècnica de Catalunya (UPC) Repositorio Abierto de la UdL Universitad de Lleida |
| بيانات النشر: | Elsevier, 2020. |
| سنة النشر: | 2020 |
| مصطلحات موضوعية: | 0106 biological sciences, Computer science, Fruit -- Breeding, fruit detection, Point cloud, Enginyeria agroalimentària [Àrees temàtiques de la UPC], Horticulture, 01 natural sciences, False positive paradox, Teledetecció -- Aplicacions agrícoles, Structure from motion, Segmentation, Terrestrial remote sensing, Projection (set theory), Structure-from-motion, fruit location, Fructicultura, Artificial neural network, business.industry, Forestry, Pattern recognition, 04 agricultural and veterinary sciences, Remote sensing, Mask R-CNN, Computer Science Applications, Support vector machine, Photogrammetry, Enginyeria de la telecomunicació::Radiocomunicació i exploració electromagnètica::Teledetecció [Àrees temàtiques de la UPC], 040103 agronomy & agriculture, 0401 agriculture, forestry, and fisheries, Artificial intelligence, business, Agronomy and Crop Science, 010606 plant biology & botany |
| الوصف: | The development of remote fruit detection systems able to identify and 3D locate fruits provides opportunities to improve the efficiency of agriculture management. Most of the current fruit detection systems are based on 2D image analysis. Although the use of 3D sensors is emerging, precise 3D fruit location is still a pending issue. This work presents a new methodology for fruit detection and 3D location consisting of: (1) 2D fruit detection and segmentation using Mask R-CNN instance segmentation neural network; (2) 3D point cloud generation of detected apples using structure-from-motion (SfM) photogrammetry; (3) projection of 2D image detections onto 3D space; (4) false positives removal using a trained support vector machine. This methodology was tested on 11 Fuji apple trees containing a total of 1455 apples. Results showed that, by combining instance segmentation with SfM the system performance increased from an F1-score of 0.816 (2D fruit detection) to 0.881 (3D fruit detection and location) with respect to the total amount of fruits. The main advantages of this methodology are the reduced number of false positives and the higher detection rate, while the main disadvantage is the high processing time required for SfM, which makes it presently unsuitable for real-time work. From these results, it can be concluded that the combination of instance segmentation and SfM provides high performance fruit detection with high 3D data precision. The dataset has been made publicly available and an interactive visualization of fruit detection results is accessible at http://www.grap.udl.cat/documents/photogrammetry_fruit_detection.html. Dades primàries associades a l'article http://hdl.handle.net/10459.1/68505 This work was partly funded by the Secretaria d’Universitats i Recerca del Departament d’Empresa i Coneixement de la Generalitat de Catalunya (grant 2017 SGR646), the Spanish Ministry of Economy and Competitiveness (project AGL2013-48297-C2-2-R) and the Spanish Ministry of Science, Innovation and Universities (project RTI2018-094222-B-I00). Part of the work was also developed within the framework of the project TEC2016-75976-R, financed by the Spanish Ministry of Economy, Industry and Competitiveness and the European Regional Development Fund (ERDF). The Spanish Ministry of Educationis thanked for Mr. J.Gené’s pre-doctoral fellowships (FPU15/03355). We would also like to thank Nufri (especially Santiago Salamero and Oriol Morreres) and Vicens Maquinària Agrícola S.A. for their support during data acquisition, and Ernesto Membrillo and Roberto Maturino for their support in dataset labelling. |
| وصف الملف: | application/pdf |
| اللغة: | English |
| URL الوصول: | https://explore.openaire.eu/search/publication?articleId=doi_dedup___::f4ee93382d44e627c9603a171b62023e https://hdl.handle.net/10459.1/67802 |
| حقوق: | OPEN |
| رقم الأكسشن: | edsair.doi.dedup.....f4ee93382d44e627c9603a171b62023e |
| قاعدة البيانات: | OpenAIRE |
| الوصف غير متاح. |