Hostname: page-component-848d4c4894-ttngx Total loading time: 0 Render date: 2024-05-17T00:56:54.531Z Has data issue: false hasContentIssue false
  • English
  • Français

Automatic weld joint type recognition in intelligent welding using image features and machine learning algorithms

Published online by Cambridge University Press:  02 January 2024

Satish Sonwane Open the ORCID record for Satish Sonwane [Opens in a new window]  and
Shital Chiddarwar Open the ORCID record for Shital Chiddarwar [Opens in a new window]
Satish Sonwane*
Affiliation:
Mechanical Engineering Department, Visvesvaraya National Institute of Technology, Nagpur, India
Shital Chiddarwar
Affiliation:
Mechanical Engineering Department, Visvesvaraya National Institute of Technology, Nagpur, India
*
Corresponding author: Satish Sonwane; Email: satish.sonwane@gmail.com
Get access Rights & Permissions [Opens in a new window]

Abstract

Welding is the most basic and widely used manufacturing process. Intelligent robotic welding is an area that has received much consideration owing to the widespread use of robots in welding operations. With the dawn of Industry 4.0, machine learning is substantially developing to alleviate issues around applying robotic welding intelligently. Identifying the correct weld joint type is essential for intelligent robotic welding. It affects the quality of the weldment and impacts the per-unit cost. The robot controller must change different welding parameters per joint type to attain the desired weld quality. This article presents an approach that uses image features like edges, corners, and blobs to identify different weld joint types using machine learning algorithms. Feature extractors perform the task of feature extraction. The feature extractor choice is crucial for accurate weld joint identification. The present study compares the performance of five feature extractors, namely (1) Histogram of gradients, (2) Local binary pattern, (3) ReLU3 layer, (4) ReLU4 layer, and (5) Pooling layer of ResNet18 Neural network applied to classifiers like Support Vector machines, K-Nearest Neighbor and Decision trees. We trained and tested the proposed model using the Kaggle Weld joint dataset (for Butt and Fillet Joints) and our in-house dataset (for Vee, lap, and corner joints). The experimental findings show that out of the 15 models, the pre-trained ResNet18 feature extractor with an Support Vector Machines classifier has excellent performance with a threefold recognition accuracy of 98.74% for the mentioned dataset with a computation time of 31 ms per image.

Keywords

intelligent robotic weldingweld joint type recognitionimage feature extractionwelding automationdeep features
Type
Research Article
Information
AI EDAM , Volume 37 , 2023 , e26
Check for updates
Copyright
© The Author(s), 2023. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Babić, BR, Nešić, N and Miljković, Z (2011) Automatic feature recognition using artificial neural networks to integrate design and manufacturing: Review of automatic feature recognition systems. Artificial Intelligence for Engineering Design, Analysis and Manufacturing 25(3), 289–304. https://doi.org/10.1017/S0890060410000545.CrossRefGoogle Scholar
Bao, J, Zheng, X, Zhang, JJ, Ji, X and Zhang, JJ (2018) Data-driven process planning for shipbuilding. Artificial Intelligence for Engineering Design, Analysis and Manufacturing 32(1), 122–130. https://doi.org/10.1017/S089006041600055X.CrossRefGoogle Scholar
Breiman, L, Friedman, JH, Olshen, RA and Stone, CJ (2017) Classification and Regression Trees. London: Routledge. https://doi.org/10.1201/9781315139470.CrossRefGoogle Scholar
Cai, W, Wang, J, Zhou, Q, Yang, Y and Jiang, P (2019) Equipment and machine learning in welding monitoring: A short review. In ACM International Conference Proceeding Series, Part F1476. New York: ACM, pp. 9–15. https://doi.org/10.1145/3314493.3314508.Google Scholar
Chen, Z, Chen, J and Feng, Z (2018) Welding penetration prediction with passive vision system. Journal of Manufacturing Processes 36, 224–230. https://doi.org/10.1016/j.jmapro.2018.10.009.CrossRefGoogle Scholar
Chen, S, Liu, J, Chen, B and Suo, X (2022) Universal fillet weld joint recognition and positioning for robot welding using structured light. Robotics and Computer-Integrated Manufacturing 74, 102279. https://doi.org/10.1016/j.rcim.2021.102279.CrossRefGoogle Scholar
Choudhury, B, Chandrasekaran, M and Devarasiddappa, D (2020) Development of ANN modelling for estimation of weld strength and integrated optimization for GTAW of Inconel 825 sheets used in aero engine components. Journal of the Brazilian Society of Mechanical Sciences and Engineering 42(6), 1–16. https://doi.org/10.1007/s40430-020-02390-7.CrossRefGoogle Scholar
Corke, P (2015) Robotics, vision and control_ fundamental algorithms in MATLAB. In Springer Tracts in Advanced Robotics, vol. 118. Heidelberg: Springer. https://doi.org/10.1007/s10472-014-9440-8.Google Scholar
Cortes, C and Vapnik, V (1992) Support-vector network. IEEE Expert-Intelligent Systems and Their Applications 7 (5), 63–72. https://doi.org/10.1109/64.163674.Google Scholar
Dalal, N and Triggs, B (2005) Histograms of oriented gradients for human detection. In 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), vol. 1. San Diego, CA: IEEE, pp. 886–893. https://doi.org/10.1109/CVPR.2005.177.Google Scholar
Donahue, J, Jia, Y, Vinyals, O, Hoffman, J, Zhang, N, Tzeng, E and Darrell, T (2014) DeCAF: A deep convolutional activation feature for generic visual recognition. 31st International Conference on Machine Learning, ICML 2014 2, 988–996.Google Scholar
Duque, DA, Prieto, FA and Hoyos, JG (2019) Trajectory generation for robotic assembly operations using learning by demonstration. Robotics and Computer-Integrated Manufacturing 57, 292–302. https://doi.org/10.1016/j.rcim.2018.12.007.CrossRefGoogle Scholar
Fan, J, Jing, F, Fang, Z and Tan, M (2017) Automatic recognition system of welding seam type based on SVM method. International Journal of Advanced Manufacturing Technology 92(1–4), 989–999. https://doi.org/10.1007/s00170-017-0202-8.CrossRefGoogle Scholar
Fan, J, Jing, F, Yang, L, Long, T and Tan, M (2019) A precise seam tracking method for narrow butt seams based on structured light vision sensor. Optics and Laser Technology 109, 616–626. https://doi.org/10.1016/j.optlastec.2018.08.047.CrossRefGoogle Scholar
Fang, Z and Xu, D (2009) Image-based visual seam tracking system for fillet joint. In 2009 IEEE International Conference on Robotics and Biomimetics, ROBIO 2009. Guilin: IEEE, pp. 1230–1235. https://doi.org/10.1109/ROBIO.2009.5420852.CrossRefGoogle Scholar
Gheysarian, A and Honarpisheh, M (2019) Process parameters optimization of the explosive-welded Al/cu bimetal in the incremental sheet metal forming process. Iranian Journal of Science and Technology - Transactions of Mechanical Engineering 43(2013), 945–956. https://doi.org/10.1007/s40997-018-0205-6.CrossRefGoogle Scholar
Guo, J, Zhu, Z, Sun, B and Yu, Y (2019) Principle of an innovative visual sensor based on combined laser structured lights and its experimental verification. Optics and Laser Technology 111, 35–44. https://doi.org/10.1016/j.optlastec.2018.09.010.CrossRefGoogle Scholar
Gyasi, EA, Handroos, H and Kah, P (2019) Survey on artificial intelligence (AI) applied in welding: A future scenario of the influence of AI on technological, economic, educational and social changes. Procedia Manufacturing 38(2019), 702–714. https://doi.org/10.1016/j.promfg.2020.01.095.CrossRefGoogle Scholar
He, Y, Li, D, Pan, Z, Ma, G, Yu, L, Yuan, H and Le, J (2020) Dynamic modeling of weld bead geometry features in thick plate GMAW based on machine vision and learning. Sensors (Switzerland) 20(24), 1–18. https://doi.org/10.3390/s20247104.CrossRefGoogle ScholarPubMed
He, D and Wang, L (1990) Texture unit, texture Spectrum, and texture analysis. IEEE Transactions on Geoscience and Remote Sensing 28(4), 509–512. https://doi.org/10.1109/TGRS.1990.572934.Google Scholar
He, Y, Yu, Z, Li, J, Ma, G and Xu, Y (2019) Fault correction of algorithm implementation for intelligentized robotic multipass welding process based on finite state machines. Robotics and Computer-Integrated Manufacturing 59, 28–35. https://doi.org/10.1016/j.rcim.2019.03.002.CrossRefGoogle Scholar
Kumar, G and Bhatia, PK (2014) A detailed review of feature extraction in image processing systems. In International Conference on Advanced Computing and Communication Technologies, ACCT. Rohtak: IEEE, pp. 5–12. https://doi.org/10.1109/ACCT.2014.74.Google Scholar
Las-Casas, MS, de Ávila, TLD, Bracarense, AQ and Lima, EJ (2018) Weld parameter prediction using artificial neural network: FN and geometric parameter prediction of austenitic stainless steel welds. Journal of the Brazilian Society of Mechanical Sciences and Engineering 40(1), 26. https://doi.org/10.1007/s40430-017-0928-0.CrossRefGoogle Scholar
Lei, Z, Shen, J, Wang, Q and Chen, Y (2019) Real-time weld geometry prediction based on multi-information using neural network optimized by PCA and GA during thin-plate laser welding. Journal of Manufacturing Processes 43, 207–217. https://doi.org/10.1016/j.jmapro.2019.05.013.CrossRefGoogle Scholar
Li, Y, Xu, D and Tan, M (2006) Welding joints recognition based on Hausdorff distance. Gaojishu Tongxin/Chinese High Technology Letters 16(11), 1129–1133.Google Scholar
Mahadevan, R, Jagan, A, Pavithran, L, Shrivastava, A and Selvaraj, SK (2021). Intelligent welding by using machine learning techniques. Materials Today: Proceedings 46, 7402–7410. https://doi.org/10.1016/j.matpr.2020.12.1149.Google Scholar
Munoz, D, “Weld-joint-segments,” | Kaggle, https://www.kaggle.com/datasets/derikmunoz/weld-joint-segments (accessed Jun. 29, 2022).Google Scholar
Mongan, PG, Hinchy, EP, O’Dowd, NP and McCarthy, CT (2020) Optimisation of ultrasonically welded joints through machine learning. Procedia CIRP 93, 527–531. https://doi.org/10.1016/j.procir.2020.04.060.CrossRefGoogle Scholar
Nixon, MS and Aguado, AS (2012) Feature extraction and image processing. In Feature Extraction & Image Processing for Computer Vision. Amsterdam: Elsevier. https://doi.org/10.1016/b978-0-12-396549-3.00003-3.Google Scholar
Ojala, T, Pietikainen, M and Harwood, D (1994) Performance evaluation of texture measures with classification based on Kullback discrimination of distributions. Proceedings of 12th International Conference on Pattern Recognition 1, 582–585. https://doi.org/10.1109/ICPR.1994.576366.CrossRefGoogle Scholar
Ojala, T, Pietikainen, M, Maenpaa, T, Pietikäinen, M, Mäenpää, T, Pietikainen, M, Maenpaa, T, Pietikäinen, M, Mäenpää, T, Pietikainen, M and Maenpaa, T (2002) Multiresolution grayscale and rotation invariant texture classification with local binary patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence 24(7), 971–987. https://doi.org/10.1109/TPAMI.2002.1017623.CrossRefGoogle Scholar
Reisgen, U, Mann, S, Middeldorf, K, Sharma, R, Buchholz, G and Willms, K (2019) Connected, digitalized welding production—Industrie 4.0 in gas metal arc welding. Welding in the World 63(4), 1121–1131. https://doi.org/10.1007/s40194-019-00723-2.CrossRefGoogle Scholar
Rodríguez-Gonzálvez, P and Rodríguez-Martín, M (2019) Weld bead detection based on 3D geometric features and machine learning approaches. IEEE Access 7, 14714–14727. https://doi.org/10.1109/ACCESS.2019.2891367.CrossRefGoogle Scholar
Shah, HNM, Sulaiman, M, Shukor, AZ, Kamis, Z and Rahman, AA (2018) Butt welding joints recognition and location identification by using local thresholding. Robotics and Computer-Integrated Manufacturing 51, 181–188. https://doi.org/10.1016/j.rcim.2017.12.007.CrossRefGoogle Scholar
Shao, WJ, Huang, Y and Zhang, Y (2018) A novel weld seam detection method for space weld seam of narrow butt joint in laser welding. Optics and Laser Technology 99, 39–51. https://doi.org/10.1016/j.optlastec.2017.09.037.CrossRefGoogle Scholar
Sharma, A, Sharma, K, Islam, A and Roy, D (2019) Effect of welding parameters on automated robotic arc welding process. Materials Today: Proceedings 26, 2363–2367. https://doi.org/10.1016/j.matpr.2020.02.507.Google Scholar
Sumesh, A, Rameshkumar, K, Mohandas, K and Babu, RS (2015) Use of machine learning algorithms for weld quality monitoring using acoustic signature. Procedia Computer Science 50, 316–322. https://doi.org/10.1016/j.procs.2015.04.042.CrossRefGoogle Scholar
Tarn, T and Chen, S (2014) Welding, Intelligence and Automation. In Robotic Welding, Intelligence and Automation. Cham: Springer. https://doi.org/10.1007/978-3-540-73374-4.Google Scholar
Tian, Y, Liu, H, Li, L, Yuan, G, Feng, J, Chen, Y and Wang, W (2021) Automatic identification of multi-type weld seam based on vision sensor with Silhouette-mapping. IEEE Sensors Journal 21(4), 5402–5412. https://doi.org/10.1109/JSEN.2020.3034382.CrossRefGoogle Scholar
Vakharia, V, Gupta, VK and Kankar, PK (2017) Efficient fault diagnosis of ball bearing using relief and random forest classifier. Journal of the Brazilian Society of Mechanical Sciences and Engineering 39(8), 2969–2982. https://doi.org/10.1007/s40430-017-0717-9.CrossRefGoogle Scholar
Wang, X, Han, TX and Yan, S (2009) An HOG-LBP human detector with partial occlusion handling. In 2009 IEEE 12th International Conference on Computer Vision. Kyoto: IEEE, pp. 32–39. https://doi.org/10.1109/ICCV.2009.5459207.CrossRefGoogle Scholar
Wang, B, Hu, SJ, Sun, L and Freiheit, T (2020) Intelligent welding system technologies: State-of-the-art review and perspectives. Journal of Manufacturing Systems 56, 373–391. https://doi.org/10.1016/j.jmsy.2020.06.020.CrossRefGoogle Scholar
Wang, Q, Jiao, W, Wang, P and Zhang, YM (2020) A tutorial on deep learning-based data analytics in manufacturing through a welding case study. Journal of Manufacturing Processes 63, 1–12. https://doi.org/10.1016/j.jmapro.2020.04.044.Google Scholar
Wang, Z, Jing, F and Fan, J (2018) Weld seam type recognition system based on structured light vision and ensemble learning. In Proceedings of 2018 IEEE International Conference on Mechatronics and Automation, ICMA 2018, 61573358. Changchun: IEEE, pp. 866–871. https://doi.org/10.1109/ICMA.2018.8484570Google Scholar
Wang, N, Zhong, K, Shi, X and Zhang, X (2020) A robust weld seam recognition method under heavy noise based on structured-light vision. Robotics and Computer-Integrated Manufacturing 61, 101821. https://doi.org/10.1016/j.rcim.2019.101821.CrossRefGoogle Scholar
Wuest, T, Weimer, D, Irgens, C and Thoben, KD (2016) Machine learning in manufacturing: Advantages, challenges, and applications. Production and Manufacturing Research 4(1), 23–45. https://doi.org/10.1080/21693277.2016.1192517.CrossRefGoogle Scholar
Xue, B, Chang, B, Peng, G, Gao, Y, Tian, Z, Du, D and Wang, G (2019) A vision based detection method for narrow butt joints and a robotic seam tracking system. Sensors (Switzerland) 19(5), 1144. https://doi.org/10.3390/s19051144.CrossRefGoogle Scholar
Yang, L, Liu, Y and Peng, J (2019) An automatic detection and identification method of weld beads based on deep neural network. IEEE Access 7, 164952–164961. https://doi.org/10.1109/ACCESS.2019.2953313.CrossRefGoogle Scholar
Zeng, J, Cao, GZ, Peng, YP and Huang, SD (2020) A weld joint type identification method for visual sensor based on image features and SVM. Sensors (Switzerland) 20(2), 471. https://doi.org/10.3390/s20020471.CrossRefGoogle ScholarPubMed
Zeng, J, Chang, B, Du, D, Peng, G, Chang, S, Hong, Y, Wang, L and Shan, J (2017) A vision-aided 3D path teaching method before narrow butt joint welding. Sensors (Switzerland) 17(5), 1099. https://doi.org/10.3390/s17051099.CrossRefGoogle ScholarPubMed
Zou, Y and Chen, T (2018) Laser vision seam tracking system based on image processing and continuous convolution operator tracker. Optics and Lasers in Engineering 105, 141–149. https://doi.org/10.1016/j.optlaseng.2018.01.008.CrossRefGoogle Scholar