Visual object detection for autonomous transport vehicles in smart factories

YAZICI, AHMET; Eker, Onur; Cevikalp, Hakan; GENGEÇ, NAZLICAN; Yavuz, Hasan Serhan

doi:10.3906/elk-2008-62

Visual object detection for autonomous transport vehicles in smart factories

Nazlıcan GENGEÇ, (Kastamonu Üniversitesi, Mühendislik Mimarlık Fakültesi, Elektrik Elektronik Mühendisliği Bölümü, Kastamonu, Türkiye)

Onur EKER, (Eskişehir Osmangazi Üniversitesi, Mühendislik Mimarlık Fakültesi, Elektrik Elektronik Mühendisliği Bölümü, Eskişehir, Türkiye)

Hakan ÇEVİKAL, (Eskişehir Osmangazi Üniversitesi, Mühendislik Mimarlık Fakültesi, Elektrik Elektronik Mühendisliği Bölümü, Eskişehir, Türkiye)

Ahmet YAZICI, (Eskişehir Osmangazi Üniversitesi, Mühendislik Mimarlık Fakültesi, Bilgisayar Mühendisliği Bölümü, Eskişehir, Türkiye)

Hasan Serhan YAVUZ (Eskişehir Osmangazi Üniversitesi, Mühendislik Mimarlık Fakültesi, Elektrik Elektronik Mühendisliği Bölümü, Eskişehir, Türkiye)

Turkish Journal of Electrical Engineering and Computer Sciences

3 0

Yıl: 2021 Cilt: 29 Sayı: 4 Sayfa Aralığı: 2101 - 2115 Metin Dili: Türkçe DOI: 10.3906/elk-2008-62 İndeks Tarihi: 23-06-2022

Visual object detection for autonomous transport vehicles in smart factories

Öz:

Autonomous transport vehicles (ATVs) are one of the most substantial components of smart factories of Industry 4.0. They are primarily considered to transfer the goods or perform some certain navigation tasks in the factory with self driving. The recent developments on computer vision studies allow the vehicles to visually perceive the environment and the objects in the environment. There are numerous applications especially for smart traffic networks in outdoor environments but there is lack of application and databases for autonomous transport vehicles in indoor industrial environments. There exist some essential safety and direction signs in smart factories and these signs have an important place in safety issues. Therefore, the detection of these signs by ATVs is crucial. In this study, a visual dataset which includes important indoor safety signs to simulate a factory environment is created. The dataset has been used to train different fast-responding popular deep learning object detection methods: faster R-CNN, YOLOv3, YOLOv4, SSD, and RetinaNet. These methods can be executed in real time to enhance the visual understanding of the ATV, which, in turn, helps the agent to navigate in a safe and reliable state in smart factories. The trained network models were compared in terms of accuracy on our created dataset, and YOLOv4 achieved the best performance among all the tested methods.

Anahtar Kelime:

Belge Türü: Makale Makale Türü: Araştırma Makalesi Erişim Türü: Erişime Açık

[1] Muhuri PK, Shukla AK, Abraham A. Industry 4.0: A bibliometric analysis and detailed overview. Engineering Applications of Artificial Intelligence 2019; 78: 218-235. doi: 10.1016/j.engappai.2018.11.007
[2] Sales DO, Correa DO, Fernandes LC, Wolf DF, Osorio FS. Adaptive finite state machine based visual autonomous navigation system. Engineering Applications of Artificial Intelligence 2014; 29: 152-162. doi: 10.1016/j.engappai.2013.12.006
[3] Wali SB, Abdullah MA, Hannan MA, Hussain A, Samad S et al. Vision-based traffic sign detection and recognition systems: current trends and challenges. Sensors 2019; 19 (9): 2093. doi: 10.3390/s19092093
[4] Syu JL, Li HT, Chiang JS, Hsia CH, Wu PH et al. A computer vision assisted system for autonomous forklift vehicles in real factory environment. Multimedia Tools and Applications 2016; 76 (18): 18387-18407. doi: 10.1007/s11042- 016-4123-6
[5] Castelli F, Michieletto S, Ghidoni S, Pagello E. A machine learning-based visual servoing approach for fast robot control in industrial setting. International Journal of Advanced Robotic Systems 2017; 14 (6): 172988141773888. doi: 10.1177/1729881417738884
[6] Islam SOB, Lughmani WA, Qureshi WS, Khalid A, Mariscal MA et al. Exploiting visual cues for safe and flexible cyber-physical production systems. Advances in Mechanical Engineering 2019; 11 (12): 168781401989722. doi: 10.1177/1687814019897228
[7] Luo C, Yu L, Yang E, Zhou H, Ren P. A benchmark image dataset for industrial tools. Pattern Recognition Letters 2019; 125: 341-348. doi: 10.1016/j.patrec.2019.05.011
[8] Perez L, Rodríguez I, Rodríguez N, Usamentiaga R, García D. Robot guidance using machine vision techniques in industrial environments: a comparative review. Sensors 2016; 16 (3): 335. doi: 10.3390/s16030335
[9] Ren S, He K, Girshick R, Sun J. Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence 2017; 39 (6): 1137-1149. doi: 10.1109/tpami.2016.2577031
[10] Redmon J, Farhadi A. Yolov3: An incremental improvement, arXiv preprint arXiv:1804.02767, 2018.
[11] Bochkovskiy A, Wang CY, Liao HYM. YOLOv4: Optimal speed and accuracy of object detection, arXiv preprint arXiv:2004.10934, 2020.
[12] Liu W, Anguelov D, Erhan D, Szegedy C, Reed S et al. SSD: Single shot multiBox detector. In: European Conference on Computer Vision (ECCV 2016), Amsterdam, the Netherlands; 2016. pp. 21-37. doi: 10.1007/978-3-319-46448- 0_2
[13] Lin TY, Goyal P, Girshick R, He K, Dollar P. Focal loss for dense object detection. In: 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy; 2017. doi: 10.1109/iccv.2017.324
[14] Wang G, Ren G, Quan T. A traffic sign detection method with high accuracy and efficiency. In: Proceedings of the 2nd International Conference on Computer Science and Electronics Engineering; 2013.
[15] Wang G, Ren G, Wu Z, Zhao Y, Jiang L. A fast and robust ellipse-detection method based on sorted merging. The Scientific World Journal 2014; 1-15. doi: 10.1155/2014/481312
[16] Houben S. A single target voting scheme for traffic sign detection. In: 2011 IEEE Intelligent Vehicles Symposium (IV), Baden, Germany, 2011. pp. 124-129.
[17] Liang M, Yuan M, Hu X, Li J, Liu H. Traffic sign detection by ROI extraction and histogram features-based recognition. In: the 2013 international joint conference on neural networks (IJCNN), Dallas, TX, USA; 2013. pp. 1-8.
[18] Loy G, Barnes N. Fast shape-based road sign detection for a driver assistance system. In: 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)(IEEE Cat. No. 04CH37566), Sendai, Japan; 2004. pp. 70-75.
[19] Wang G, Ren G, Wu Z, Zhao Y, Jiang L. A robust, coarse-to-fine traffic sign detection method. In: the 2013 international joint conference on neural networks (IJCNN), Dallas, Texas, USA; 2013. pp. 1-5. [20] Dalal N, Triggs B. Histograms of oriented gradients for human detection. In: 2005 IEEE computer society conference
on computer vision and pattern recognition (CVPR’05), San Diego, CA, USA; 2005. pp. 886-893.
[21] Supreeth HSG, Patil CM. An approach towards efficient detection and recognition of traffic signs in videos using neural networks. In: 2016 International Conference on Wireless Communications, Signal Processing and Networking (WiSPNET), Chennai, India; 2016. pp. 456-459.
[22] Cevikalp H, Triggs B. Efficient object detection using cascades of nearest convex model classifiers. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA; 2012. pp. 3138-3145.
[23] Hussain SU, Triggs W. Feature sets and dimensionality reduction for visual object detection. In: Proceedings of the British Machine Vision Conference 2010, Aberystwyth, Wales, UK; 2010. doi:10.5244/c.24.112
[24] Lowe DG. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision 2004; 60 (2): 91-110. doi: 10.1023/b:visi.0000029664.99615.94
[25] Felzenszwalb P, McAllester D, Ramanan D. A discriminatively trained, multiscale, deformable part model. In: 2008 IEEE conference on computer vision and pattern recognition, Anchorage, Alaska, USA; 2008. pp. 1-8.
[26] Uijlings JRR, Van de Sande KEA, Gevers T, Smeulders AWM. Selective search for object recognition. International Journal of Computer Vision 2013; 104 (2): 154-171. doi: 10.1007/s11263-013-0620-5
[27] Cevikalp H, Triggs B. Polyhedral conic classifiers for visual object detection and classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Venice, Italy; 2017. pp. 261-269. doi: 10.1109/cvpr.2017.438
[28] Cevikalp H, Saglamlar H. Polyhedral Conic Classifiers for Computer Vision Applications and Open Set Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 2020; 1-1. doi: 10.1109/tpami.2019.2934455
[29] Cevikalp H, Triggs B. Visual object detection using cascades of binary and one-class classifiers. International Journal of Computer Vision 2017; 123 (3): 334-349. doi: 10.1007/s11263-016-0986-2
[30] Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. Communications of the ACM 2017; 60 (6): 84-90. doi: 10.1145/3065386
[31] Qian R, Zhang B, Yue Y, Wang Z, Coenen F. Robust chinese traffic sign detection and recognition with deep convolutional neural network. In: 2015 11th International Conference on Natural Computation (ICNC), Zhangjiajie, China; 2015. pp. 791-796.
[32] Girshick R, Donahue J, Darrell T, Malik J. Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA; 2014. pp. 580-587.
[33] Zuo Z, Yu K, Zhou Q, Wang X, Li T. Traffic signs detection based on faster R-CNN. In: 2017 IEEE 37th International Conference on Distributed Computing Systems Workshops (ICDCSW), Atlanta, GA, USA; 2017. pp. 286-288.
[34] Redmon J, Divvala S, Girshick R, Farhadi A. You only look once: Unified, real-time object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Las Vegas, NV, USA; 2016. pp. 779-788.
[35] Redmon J, Farhadi A. YOLO 9000: better, faster, stronger. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Venice, Italy; 2017. pp. 7263-7271.
[36] Zhang J, Huang M, Jin X, Li X. A real-time Chinese traffic sign detection algorithm based on modified YOLOv2. Algorithms 2017; 10 (4): 127. doi: 10.3390/a10040127
[37] He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA; 2016. pp. 770-778.
[38] Wang CY, Mark LHY, Wu YH, Chen PY, Hsieh JW et al. CSPNet: A new backbone that can enhance learning capability of CNN. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Virtual; 2020. pp. 390-391.
[39] Huang G, Liu Z, Van Der Maaten L, Weinberger KQ. Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Venice, Italy; 2017. pp. 4700-4708. doi: 10.1109/CVPR.2017.243
[40] He K, Zhang X, Ren S, Sun J. Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 2015; 37 (9): 1904-1916. doi: 10.1109/tpami.2015.2389824
[41] Liu S, Qi L, Qin H, Shi J, Jia J. Path aggregation network for instance segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Salt Lake City, UT, USA; 2018. pp. 8759-8768. doi: 10.1109/CVPR.2018.00913

APA	GENGEÇ N, Eker O, Cevikalp H, YAZICI A, Yavuz H (2021). Visual object detection for autonomous transport vehicles in smart factories. , 2101 - 2115. 10.3906/elk-2008-62
Chicago	GENGEÇ NAZLICAN,Eker Onur,Cevikalp Hakan,YAZICI AHMET,Yavuz Hasan Serhan Visual object detection for autonomous transport vehicles in smart factories. (2021): 2101 - 2115. 10.3906/elk-2008-62
MLA	GENGEÇ NAZLICAN,Eker Onur,Cevikalp Hakan,YAZICI AHMET,Yavuz Hasan Serhan Visual object detection for autonomous transport vehicles in smart factories. , 2021, ss.2101 - 2115. 10.3906/elk-2008-62
AMA	GENGEÇ N,Eker O,Cevikalp H,YAZICI A,Yavuz H Visual object detection for autonomous transport vehicles in smart factories. . 2021; 2101 - 2115. 10.3906/elk-2008-62
Vancouver	GENGEÇ N,Eker O,Cevikalp H,YAZICI A,Yavuz H Visual object detection for autonomous transport vehicles in smart factories. . 2021; 2101 - 2115. 10.3906/elk-2008-62
IEEE	GENGEÇ N,Eker O,Cevikalp H,YAZICI A,Yavuz H "Visual object detection for autonomous transport vehicles in smart factories." , ss.2101 - 2115, 2021. 10.3906/elk-2008-62
ISNAD	GENGEÇ, NAZLICAN vd. "Visual object detection for autonomous transport vehicles in smart factories". (2021), 2101-2115. https://doi.org/10.3906/elk-2008-62

APA	GENGEÇ N, Eker O, Cevikalp H, YAZICI A, Yavuz H (2021). Visual object detection for autonomous transport vehicles in smart factories. Turkish Journal of Electrical Engineering and Computer Sciences, 29(4), 2101 - 2115. 10.3906/elk-2008-62
Chicago	GENGEÇ NAZLICAN,Eker Onur,Cevikalp Hakan,YAZICI AHMET,Yavuz Hasan Serhan Visual object detection for autonomous transport vehicles in smart factories. Turkish Journal of Electrical Engineering and Computer Sciences 29, no.4 (2021): 2101 - 2115. 10.3906/elk-2008-62
MLA	GENGEÇ NAZLICAN,Eker Onur,Cevikalp Hakan,YAZICI AHMET,Yavuz Hasan Serhan Visual object detection for autonomous transport vehicles in smart factories. Turkish Journal of Electrical Engineering and Computer Sciences, vol.29, no.4, 2021, ss.2101 - 2115. 10.3906/elk-2008-62
AMA	GENGEÇ N,Eker O,Cevikalp H,YAZICI A,Yavuz H Visual object detection for autonomous transport vehicles in smart factories. Turkish Journal of Electrical Engineering and Computer Sciences. 2021; 29(4): 2101 - 2115. 10.3906/elk-2008-62
Vancouver	GENGEÇ N,Eker O,Cevikalp H,YAZICI A,Yavuz H Visual object detection for autonomous transport vehicles in smart factories. Turkish Journal of Electrical Engineering and Computer Sciences. 2021; 29(4): 2101 - 2115. 10.3906/elk-2008-62
IEEE	GENGEÇ N,Eker O,Cevikalp H,YAZICI A,Yavuz H "Visual object detection for autonomous transport vehicles in smart factories." Turkish Journal of Electrical Engineering and Computer Sciences, 29, ss.2101 - 2115, 2021. 10.3906/elk-2008-62
ISNAD	GENGEÇ, NAZLICAN vd. "Visual object detection for autonomous transport vehicles in smart factories". Turkish Journal of Electrical Engineering and Computer Sciences 29/4 (2021), 2101-2115. https://doi.org/10.3906/elk-2008-62