Hostname: page-component-77c89778f8-n9wrp Total loading time: 0 Render date: 2024-07-21T13:36:36.091Z Has data issue: false hasContentIssue false
  • English
  • Français

FROM SKETCHES TO GRAPHS: A DEEP LEARNING BASED METHOD FOR DETECTION AND CONTEXTUALISATION OF PRINCIPLE SKETCHES IN THE EARLY PHASE OF PRODUCT DEVELOPMENT

Published online by Cambridge University Press:  19 June 2023

Sebastian Bickel ,
Stefan Goetz  and
Sandro Wartzack
Sebastian Bickel*
Affiliation:
Friedrich-Alexander-Universität Erlangen-Nürnberg
Stefan Goetz
Affiliation:
Friedrich-Alexander-Universität Erlangen-Nürnberg
Sandro Wartzack
Affiliation:
Friedrich-Alexander-Universität Erlangen-Nürnberg
*
Bickel, Sebastian, Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany, bickel@mfk.fau.de

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

The digitalization trend is finding its way more and more into product development, resulting in new frameworks to enhance product engineering. An integral element is the application of new techniques to existing data, which offers an enormous potential for time and cost savings, because duplicate work in product design and subsequent steps is avoided. The reduction of costs can be further increased through the application as early as possible in the product development process. One solution is outlined in this publication, where the source of available data is principle sketches from engineering design. These represent the basic solution for technical products in a simplified way and are often deployed in the early stages of the development process. This representation enables not only a search of similar sketches but also other fields of interest such as product optimization or the search of CAD-geometries. To utilize this data in a practical way, a procedure is presented which recognizes the symbols of the sketches and subsequently converts them into graphs. An exemplary dataset from different gearbox layouts is used to present the application opportunities by performing similarity searches with multiple input formats.

Keywords

Early design phasesMachine learningArtificial intelligenceobject detectiondeep learning
Type
Article
Information
Proceedings of the Design Society , Volume 3: ICED23 , July 2023 , pp. 1975 - 1984
Creative Commons
Creative Common License - CCCreative Common License - BYCreative Common License - NCCreative Common License - ND
This is an Open Access article, distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives licence (http://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is unaltered and is properly cited. The written permission of Cambridge University Press must be obtained for commercial re-use or in order to create a derivative work.
Copyright
The Author(s), 2023. Published by Cambridge University Press

References

Abu-Aisheh, Z., Raveaux, R., Ramel, J. Y., & Martineau, P. (2015). An exact graph edit distance algorithm for solving pattern recognition problems. In 4th International Conference on Pattern Recognition Applications and Methods 2015.CrossRefGoogle Scholar
Bailey, D, Norman, A, Moretti, G, North, P (1995). Electronic schematic recognition. Massey University, Wellington, New ZealandGoogle Scholar
Bickel, S., Schleich, B., & Wartzack, S. (2021). Detection and classification of symbols in principle sketches using deep learning. Proceedings of the Design Society, 1, pp. 1183–1192.CrossRefGoogle Scholar
Bickel, S., Schleich, B., & Wartzack, S. (2022). A New Projection Based Method for the Classification of Mechanical Components Using Convolutional Neural Networks. Proceedings of the Design Society, 2, pp. 1501–1510.CrossRefGoogle Scholar
Cordella, L. P., Foggia, P., Sansone, C., & Vento, M. (2001). An improved algorithm for matching large graphs. In 3rd IAPR-TC15 workshop on graph-based representations in pattern recognition, pp. 149–159.Google Scholar
Cordella, L. P., Foggia, P., Sansone, C., & Vento, M. (2004). A (sub) graph isomorphism algorithm for matching large graphs. IEEE transactions on pattern analysis and machine intelligence, 26(10), pp. 1367–1372.CrossRefGoogle ScholarPubMed
Deng, X., Li, T., Xu, Y., Cao, Y., Kong, C., Zhang, E., (2020). The Computer Vision-based Tolerancing Callout Detection Model. Procedia CIRP. 92, pp. 134–139. https://doi.org/10.1016/j.procir.2020.05.189CrossRefGoogle Scholar
Elyan, E., Moreno-García, C., Jayne, C., (2018). Symbols Classification in Engineering Drawings. International joint conference on neural networks 2018 (IJCNN), 8–13 July 2018, Rio de Janeiro, Brazil https://dx.doi.org/10.1109/IJCNN.2018.8489087.CrossRefGoogle Scholar
Ester, M., Kriegel, H. P., Sander, J., & Xu, X. (1996). A density-based algorithm for discovering clusters in large spatial databases with noise. In kdd, vol. 96, No. 34, pp. 226–231.Google Scholar
Fu, L., Kara, L., (2011). From engineering diagrams to engineering models: Visual recognition and applications. Computer-Aided Design. 43. pp. 278–292. https://doi.org/10.1016/j.cad.2010.12.011CrossRefGoogle Scholar
Goetz, S., Schleich, B., Wartzack, S. (2018). A new approach to first tolerance evaluations in the conceptual design stage based on tolerance graphs, Procedia CIRP, vol. 75, pp. 167–172 https://doi.org/10.1016/j.procir.2018.04.030.CrossRefGoogle Scholar
Ho-Quang, T., Chaudron, M. R., Samúelsson, I., Hjaltason, J., Karasneh, B., & Osman, H. (2014). Automatic classification of UML class diagrams from images. In 2014 21st Asia-Pacific Software Engineering Conference, vol. 1, pp. 399–406. IEEE.CrossRefGoogle Scholar
Kasturi, R., Bow, S. T., El-Masri, W., Shah, J., Gattiker, J. R. and Mokate, U. B. (1990). A system for interpretation of line drawings, in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 12, no. 10, pp. 978–992, https://dx.doi.org/10.1109/34.58870.CrossRefGoogle Scholar
Levenshtein, V. I. (1966). Binary codes capable of correcting deletions, insertions, and reversals. In Soviet physics doklady, vol. 10, No. 8, pp. 707–710.Google Scholar
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., Zitnick, C. L. (2014). Microsoft COCO: Common Objects in Context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds) Computer Vision – ECCV 2014. vol. 8693. Springer, Cham. https://doi.org/10.1007/978-3-319-10602-1_48CrossRefGoogle Scholar
Lupinetti, K., Giannini, F., Monti, M., & Pernot, J. P. (2016). Automatic extraction of assembly component relationships for assembly model retrieval. Procedia CIRP, 50, pp. 472–477.CrossRefGoogle Scholar
Mizanur Rahman, S., Bayer, J., & Dengel, A. (2021). Graph-Based Object Detection Enhancement for Symbolic Engineering Drawings. In International Conference on Document Analysis and Recognition, pp. 74–90. Springer, ChamGoogle Scholar
Moon, Y., Lee, J., Mun, D., & Lim, S. (2021). Deep Learning-Based Method to Recognize Line Objects and Flow Arrows from Image-Format Piping and Instrumentation Diagrams for Digitization. Applied Sciences, 11(21), 10054.CrossRefGoogle Scholar
Moreno-García, C. F., Elyan, E., & Jayne, C. (2019). New trends on digitisation of complex engineering drawings. Neural computing and applications, 31(6), pp. 1695–1712.CrossRefGoogle Scholar
Okazaki, A., Kondo, T., Mori, K., Tsunekawa, S. and Kawamoto, E. (1988). An automatic circuit diagram reader with loop-structure-based symbol recognition, in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 10, no. 3, pp. 331–341, https://dx.doi.org/10.1109/34.3898CrossRefGoogle Scholar
Roth, K. (2001). Konstruieren mit Konstruktionskatalogen: Band 2: Kataloge. 3., Aufl. 2000. Springer-Verlag Berlin Heidelberg https://doi.org/10.1007/978-3-642-17467-4CrossRefGoogle Scholar
Shcherban, S., Liang, P., Li, Z., & Yang, C. (2021). Multiclass Classification of UML Diagrams from Images Using Deep Learning. International Journal of Software Engineering and Knowledge Engineering, 31(11n12), pp. 1683–1698.CrossRefGoogle Scholar
Vaxiviere, P., & Tombre, K. (1992). Celesstin: CAD conversion of mechanical drawings. Computer, 25(7), pp. 46–54.CrossRefGoogle Scholar
VDI 2223 (2004). Systematic embodiment design of technical products. Düsseldorf, VDIGoogle Scholar
Wen, R, Tang, W, Su, Z (2016). A 2D engineering drawing and 3D model matching algorithm for process plant. In: Proceedings—2015 international conference on virtual reality and visualization, ICVRV 2015, pp 154–159Google Scholar
Yu, Y., Samal, A., & Seth, S. C. (1997). A system for recognizing a large class of engineering drawings. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(8), pp. 868–890.Google Scholar
Yun, D. Y., Seo, S. K., Zahid, U., & Lee, C. J. (2020). Deep neural network for automatic image recognition of engineering diagrams. Applied Sciences, 10(11), 4005.CrossRefGoogle Scholar