Fruit detection and 3D location using instance segmentation neural networks and structure-from-motion photogrammetry
Visualitza/Obre
10.1016/j.compag.2019.105165
Inclou dades d'ús des de 2022
Cita com:
hdl:2117/175526
Tipus de documentArticle
Data publicació2020-01-13
Condicions d'accésAccés obert
Tots els drets reservats. Aquesta obra està protegida pels drets de propietat intel·lectual i
industrial corresponents. Sense perjudici de les exempcions legals existents, queda prohibida la seva
reproducció, distribució, comunicació pública o transformació sense l'autorització del titular dels drets
Abstract
The development of remote fruit detection systems able to identify and 3D locate fruits provides opportunities to improve the efficiency of agriculture management. Most of the current fruit detection systems are based on 2D image analysis. Although the use of 3D sensors is emerging, precise 3D fruit location is still a pending issue. This work presents a new methodology for fruit detection and 3D location consisting of: (1) 2D fruit detection and segmentation using Mask R-CNN instance segmentation neural network; (2) 3D point cloud generation of detected apples using structure-from-motion (SfM) photogrammetry; (3) projection of 2D image detections onto 3D space; (4) false positives removal using a trained support vector machine. This methodology was tested on 11 Fuji apple trees containing a total of 1455 apples. Results showed that, by combining instance segmentation with SfM the system performance increased from an F1-score of 0.816 (2D fruit detection) to 0.881 (3D fruit detection and location) with respect to the total amount of fruits. The main advantages of this methodology are the reduced number of false positives and the higher detection rate, while the main disadvantage is the high processing time required for SfM, which makes it presently unsuitable for real-time work. From these results, it can be concluded that the combination of instance segmentation and SfM provides high performance fruit detection with high 3D data precision. The dataset has been made publicly available and an interactive visualization of fruit detection results is accessible at http://www.grap.udl.cat/documents/photogrammetry_fruit_detection.html.
CitacióGené, J. [et al.]. Fruit detection and 3D location using instance segmentation neural networks and structure-from-motion photogrammetry. "Computers and electronics in agriculture", 13 Gener 2020, vol. 169, núm. 105165, p. 105165:1-105165:21.
ISSN0168-1699
Versió de l'editorhttps://www.sciencedirect.com/science/article/pii/S0168169919321507
Fitxers | Descripció | Mida | Format | Visualitza |
---|---|---|---|---|
COMPAG_2019_2054_Original_V0.pdf | 1,720Mb | Visualitza/Obre |