Investigating Application of Machine Learning in
Identification of Polygon Shapes for Recognition of
Mechanical Engineering Drawings
Riddhi Adhikari, Shreyash Gadgil, Abhilash Mane, Nilesh Raykar
Department of Mechanical Engineering
Sardar Patel College of Engineering
Mumbai, Maharashtra, India
email: nilesh_raykar@spce.ac.in
Abstract—This paper investigates the applications of Machine
Learning in recognition of 2D drawings of machine components.
Recognition of primitive geometric shapes such as polygons within
engineering drawings forms basic element of such approach.
Machine learning algorithms are used to identify 3 to 7 sided
polygons with random shapes and segmented edges. The
uncertainty induced by segmented edges poses a challenge for
predicting number of sides using statistical method such as
Machine Learning. Different types of datasets with varying
amount of uncertainty are used. The recognition of shape is
attempted with different set of feature sets such as coordinates of
points, slopes of lines and geometric parameters such as area,
perimeter and centroid. Three machine learning models namely,
Random Forest Classifier, K-Neighbors Classifier and Support
Vector Classifier are adopted. The performance of these models
for identification of polygons is discussed.
Keywords—Machine learning, Pattern Recognition, Drawing
Recognition, Random Forest.
I. INTRODUCTION
Engineering drawing is a universal language of
communication in all engineering disciplines. In mechanical
engineering field, the product design is expressed in a set of
manufacturing drawings which contain important information
critical for the quality of manufactured products. Many drawings
for existing products are available in hardcopy or 2D CAD
format. Implementation of modern manufacturing technologies
demand these drawings to be converted in to 3D CAD model.
Drawing Recognition can be defined as identification of various
elements in a drawing to enable interpretation of 2D drawings of
machine components in terms of their 3D CAD model. The
automation of drawing recognition process has been attempted
for more than two decades using different approaches.
Recognition of drawings using methods of machine learning
methods has been on rise in recent years.
Machine learning is one of the trending phenomena,
extending its arms in various disciplines; especially those
concerned with data sensitive issues. It aims at developing and
training models that can automatically and continuously
improve with time and experience and perform their tasks more
precisely. With the growing ability of networked and mobile
computing systems to gather vast amount of data sets, machine
learning allows us to make useful predictions and important
decisions from these data sets without any human intervention.
The study of machine learning is important both for addressing
fundamental scientific and engineering questions and for the
highly practical computer software it has produced and fielded
across many applications [1]. Some of the applications of
machine learning include diagnosis of faults in complex
systems, computer vision, language translation, speech
recognition and automatic speech controller, robot control,
pattern recognition, etc.
In mechanical engineering domain, the advent of automation
and due to the need of carrying out certain operations
independently without human involvement, Machine learning
has been extensively used for assisting certain types of
engineering processes as follows. For example, (i) Fault
diagnosis of a reciprocating diesel engine is carried out by
selecting the most appropriate case from the database after
calculating the similarity between the given case and the
previous cases in the database [2], (ii) Prediction of flank tool
wear in high speed machines is carried out using Random forests
model and a set of statistical data created from parameters like
cutting force, vibration and acoustic emissions collected from
milling tests [3], (iii) Planning of production processes is done
using Machine learning models which recognize features such
as holes, grooves, etc. on the parts [4], (iv) Characterization of
materials using Unsupervised clustering attempted by [5] where
thermal, electrical, physical, mechanical and chemical
compositions of the materials were the features considered for
the study.
In design engineering, selection of appropriate design
methods using Machine learning algorithms like Random
Forests, Support Vector Machines, Logistic Regression and
Naïve Bayes has been studied [6]. The model learns from the
case studies provided in the database which depict use of
different design methods used by other. The role of machine
learning in machine design is discussed by [7], [8], [9], [10].
Pattern Recognition and its use in recognizing Engineering
Drawings is one of the important applications of machine
learning. Wide ranges of applications of pattern recognition
have led to a lot of research in this domain. Symbol recognition,
one of the segments of pattern recognition, is actively is used for
logo recognition, interpretation of architectural plans,
understanding and validation of electrical schematics in logic
circuits for its conversion to an electronic format, and
recognition of different entities of engineering drawings.
The drawing recognition process involves multiple steps and
identification of geometrical shapes such as polygons within a
2D drawing is an important step. Although this problem has
been addressed sufficiently in literature, the detection of polygon
shape when its edges have multiple internal points is tricky. Such
situation appears when a drawing object is broken along its edge
for ease of dimensioning or fitting of another detail. The current
work investigates use of machine learning for identifying such
polygons within a drawing.
The paper is composed as follows. The next section of the
paper describes in brief different approaches used for drawing
recognition in Mechanical Engineering field, challenges
currently faced by researchers and the typical process adopted
for drawing recognition. This is followed by discussion about
work done and results obtained about classification of convex
polygon shapes using machine learning.
II. DIFFERENT APPROACHES ADOPTED FOR DRAWING
RECOGNITION AND CHALLENGES INVOLVED
A typical engineering drawing for a machine component is
shown in Fig.1. The drawing consists of different elements
which need to be identified before converting these to 3D model.
Drawing view
Dimension
Symbol
Polygon /
geometric shape
Text
Fig. 1 Typical drawing for a machine component.
A. Different Approaches for Drawing Recognition
Two important approaches of drawing recognition are -
Statistical approach and structural approach. In statistical
approach, each geometric element representing a drawing
component is represented as an n-dimensional feature vector and
classification is carried out by partitioning the feature space into
different classes [11]. Classification algorithms like Decision
trees, Random Forests, Support Vector Machines (SVM), K-
Nearest Neighbours (KNN), Naive Bayes, Principal Component
Analysis (PCA) are used for this purpose. In structural approach,
each symbol is represented by some geometric primitives and an
Drawing Recognition
Statistical Structural
Approach Approach
SVM KNN PCA
Random Decision Naïve
Forests trees Bayes
ideal model is built for each symbol using these primitives so
that the input can be matched to the available ideal models and
classified. Fig.2 represents the different approaches of Drawing
Fig. 2 Different approaches for drawing recognition.
Recognition.
Some of the challenges in the above mentioned approaches
are: incompetency of the system when used for large datasets,
segmentation of drawings, inability to recognize distorted
objects and high computational complexity. One solution that is
proposed is the definition of symbol signatures that allow to
index into the drawing to locate areas where the symbol is likely
to appear [11]. In order to deal with other problems, researchers
experimented with different symbol features like image,
centroids, moments invariants, etc.
B. Typical Process of Drawing Recognition
A lot of work has been done previously in pattern
matching and symbol recognition to avoid human involvement
in analyzing technical drawings. Pattern matching is used for
matching two different drawings and also for matching 2-D
drawings with its 3-D part. Symbol Recognition or pattern
recognition is of great aid in manufacturing processes for
detecting specific types of features. It also helps in converting
paper based drawings into its electronic version. Process of
Drawing Recognition is described in Fig.3.
1) Data Extraction: Recognition and extraction of useful
information from the drawings.
Spatial drawing data can be represented in two formats-
Raster and Vector. In raster format, data is represented in the
form of a grid of cells or pixels and each cell has some value
associated with it. In vector format, data consists of coordinates
and information about how these coordinates are joined. In
vector format, data can be represented at its original resolution
whereas; in raster format, the resolution is determined by the size
of cells. In raster format, processing of attributes associated with
the cell can be complicated for large datasets. Hence, vector
format is preferred over raster format. Moreover,
drawings and their merits and limitations are discussed in [12]
and [13]. As Engineering Drawings usually consist of various
entities like Drawing views, text, symbols giving information
related to manufacturing or materials and dimensions, extracted
data has to be segmented into these entities. First step in the
process of conversion of this data into 3-D CAD model is
identification and classification of 2-D polygonal shapes in the
views.

Engineering Drawing Engineering Drawing

Vector format (DXF, IGES) Raster format (JPG, GIF)

Conversion to
Vector format

Segmentation

Views Text Dimensions Symbols

Polygons Curves

Extract ExtractFeatu Extract Extract Extract

Features-1 res-2 Features-3 Features-4 Features-5

ML ML ML ML ML
Models-1 Models-2 Models-3 Models-4 Models-5

Shape Curve Text Dimension Symbol

Data Data Data Data Data

Data Synthesis, Contextualization

3D CAD Model Bill of Materials

Fig. 3 Process of Mechanical Engineering Drawing Recognition

2) Conversion of data into features

different versions of CAD software or geometric modeling
packages store drawing information in vector databases such as
Data Exchange Format (DXF), Standard for the Exchange of
Product model (STEP) and Initial Graphic Exchange
Specification (IGES) which is also aesthetically pleasing. If data
is available in raster format, it is converted into vector form
before its use in the further process. DXF is a widely used
structure for storing data. It is rigidly defined and writing
programs for extraction of information from these structures is
easier. Various methods of CADD data extraction from CADD
After extracting data, it has to be converted into significant
features through which machine learning models can view
technical drawings. These features are different for views, texts,
dimensions and symbols and should be invariant to translation,
scaling and rotation. Flusser [14], discusses three major
approaches to deal with this - Brute force, image normalization
and invariants. Brute force method requires different versions of
representatives of the classes, such as blurred, rotated, scaled
and deformed, to be included in the training set. This method is
temporally complex. Normalization method transforms the
object into standard position which reduces complexity while
classifying. Invariants approach focuses on representing the
object by a set of features which are not sensitive to some
deformations. One such feature discussed in [14] that is used for
representing 2-D drawings is Moment invariants. It describes 4-sided
different forms of moment invariants used in previous Internal points
researches, which can deal with both geometric as well as
radiometric distortions. Use of feature vectors has certain
limitations which are overcome by Graph Theory based
approach which is discussed in [15]. Graph Theory was a 3-sided polygon
transition between Statistical Pattern recognition and Structural 5-sided
Pattern Recognition.
3) Use of features in Machine Learning models.
Development of machine learning algorithms for pattern
recognition during the three periods- Pure, Impure and Extreme
period is described in [16]. In Pure period, exact and inexact 6-sided 7-sided
matching of graphs was carried out using optimal and
suboptimal algorithms. In Impure period, few machine learning Four datasets were created for training and testing of the
models like Nearest Neighbour classifier (NN), its derivatives as machine learning models. Each of the dataset is designed to pose
K-NN and (K-K’)-NN, clustering and Learning Vector a varying level of difficulty in terms numerically predicting the
Quantization were developed. These models require a set of number of sides.
labeled data for classification. Machine learning models used for • The first dataset (POLY-1) consists of 1000 polygons having
pattern recognition in the extreme period, consist of Support number of sides ranging from 3 to 7. In these polygons, each
Vector Machines (SVM), Kernel Principal Component Analysis
(KPCA) and the Multilayer Perceptrons (MLP).
Fig. 4 Typical 3 to 7 sided polygons with internal points on edges.
Neural Networks is used for recognition of various symbols
used in electrical drawings in [17]. Among many different neural side is made up of a single line element, that is there are no
network classifiers, the multi-layer perceptron (MLP) classifier mid-side nodes. The angle between adjacent edges is
with back propagation (BP) learning model has been controlled and lies in the range of 5 degree to 175 degree
successfully applied in many pattern recognition problems [17]. and the value of coordinates is stored up to 8 decimal points,
The advantage of Neural Networks is that it can adapt to the This dataset has no ambiguity for numerically determining
changes after training. Any of these classification models can be the exact number of sides.
applied to the extracted features for identification of views, texts,
symbols or dimensions. The results obtained from all these • The second dataset (POLY-2) consists of 1000 polygons
models can then be integrated for obtaining 3-D CAD model or with number of sides ranging from 3 to 7 and each side is
for preparing bill of materials. made up of multiple segments (maximum 3). But here the
angles between the edges are not controlled and coordinates
III. IDENTIFICATION OF POLYGONS USING MACHINE LEARNING are stored up to 4 decimals. This is the most challenging set
The purpose of this work is to train a machine learning since it is difficult to numerically identify the exact number
model to identify different polygonal shapes from an of sides if two edges are almost along a line. A typical
engineering drawing. This process constitutes one branch of the
drawing recognition work and it is highlighted in Fig. 3. Features
such as images, coordinates of vertices, slopes of lines,
centroids, moment invariants can be included in machine
learning model. Statistical or machine learning methods can
automatically learn from large numbers of training samples to
achieve reasonable recognition ability[18]. Due to numerous
types of objects that appear in various engineering drawings, it
is impossible to provide all forms of such objects.
A. Generation of Polygon Dataset 7-sided polygon with
large uncertainty in
In this work, datasets containing sequential coordinates of detecting number of sides
points laid along randomly generated convex polygons with 3 to
7 sides are used to train the model. Typical polygon shapes are
shown in Fig. 4. VisualLISP which is inbuilt feature of Fig. 5 Typical polygon with large uncertainity in detection of
AutoCAD was used to generate datasets of randomly sized number of sides.
polygons with randomly placed internal points along is edges.
Code is written to export the coordinates of all points in polygon of such type is shown in Fig. 5.
sequence in .csv format.
• Third dataset (POLY-3) also consists of 1000 polygons and For the POLY-1 where sides of polygon are single line
is similar to the second dataset except that the angle segments, all models except KNN could classify the polygons
between adjacent sides of polygon is controlled and lies in with 100% accuracy. For POLY-2, the accuracy by all models is
the range of 5 degree to 175 degree and the value of the poorest. The accuracy obtained from conventional code was
coordinates is stored up to 8 decimal points. Although this also found to be deteriorating because the edges with included
reduces false counting or dropping of edges, the internal angles of less than 5 degrees or greater than 175 degrees were
points can still lead to false result about number of sides. wrongly detected as one. The accuracy improved in POLY-3
when the included angles between the edges were constrained in
• The last dataset (POLY-4) has 2000 polygons and was the range of 5 to 175 degrees. It was observed from the POLY-
created by joining POLY-1 and POLY-3 datasets. Due to 4 dataset that machine learning models gave better results when
presence of more proportion of easily predictable shapes, it some ideal cases of polygons were included in the dataset under
is expected to perform better in terms of accuracy. consideration.
In these datasets, rows represent various polygons and the
columns provide information for each polygon. The first column TABLE I ACCURACY OF DIFFERENT MODELS
represents number of sides of the polygon, second column Dataset
represents number of points followed by columns that give x and
POLY-1 POLY-2 POLY-3 POLY-4
y coordinates of successive points alternatively.
With
Feature With
B. Feature Selection Model used
Set
internal
internal
No points, POLY-1
points,
Four different features were tried for testing the polygon internal no
control
+
recognition model which are as follows. points control POLY-3
on edge
on edge
angles
angles
i. Cartesian coordinates of successive points of the
Random
polygons Forest
Cartesian
100% 49% 61.2% 83.2%
coordinates
ii. Polar coordinates of successive points of the classifier
Random
polygons Forest
Polar
100% 48.4% 81.2% 90%
coordinates
iii. Slopes of successive line segments with respect to classifier
Random
x axis. Forest Slopes 100% 53.2% 79.2% 91.2%
iv. Geometric parameters for each polygon: total classifier
Random
number of points, number of approximately Geometric
forest 100% 49.2% 86.3 93.3%
collinear points, perimeter, area, ratio of area to classifier
parameters
perimeter, range of x-coordinates, range of y- Geometric
KNN 98.0% 45.6% 74.7% 84.5%
coordinates, area bound by x- and y- limits, ratio of parameters
area to bound-area, x-coordinate of centroid of SVC
Geometric
100% 47.6% 72.5% 79.8%
perimeter, x-coordinate of centroid of perimeter, y- parameters
coordinate of centroid of perimeter, radial distance Conventional
--- 100% 58.8% 100% 100%
Python code
of centroid of perimeter from origin.
C. Selection of Machine Learning Model
The work was performed with scikit-learn library in Python. The Random Forest Classifies gives relatively good
Considering the problem statement and the nature of datasets, a accuracy in detection of polygons. The Random forest classifier
classifier model was needed. Decision tree algorithm can be additionally gives graphical representation of the decision tree
applied but it has limitation of over fitting for small or medium (Fig.6) which is helpful in identifying important features in the
size datasets. Hence, ‘Random Forest Classifier’ was used as the dataset.
main machine learning model. Additionally, K Nearest
Neighbors classifier (KNN) and Support Vector Classifier
(SVC) models were tried for comparison.
The dataset was pre-processed with StandardScaler. The
dataset was split randomly with 75% and 25% data respectively
available for training and testing purpose.
IV. RESULTS
The accuracy score was obtained for all the cases. This
was compared with the accuracy obtained from the conventional Fig. 6 Typical partial decision tree for POLY-3 dataset.
code generated in python for recognizing the sides of a polygon.
The average accuracy scores obtained for all the cases is
mentioned in Table I.

It was observed that in general the mistake in identifying
number of sides is within +/-1 side as seen from a typical
confusion matrix (Fig. 7) for POLY-3.
Fig. 7 Typical Confusion matrix for POLY-3 dataset.
V. CONCLUSION
The published work on recognition of drawings in
Mechanical Engineering domain is observed to be substantial;
however, a comprehensive solution for this purpose is yet not
established.
The Machine Learning based approach was successfully
implemented to identify number of sides of convex polygons
with 3 to 7 sides and segmented edges. The accuracy obtained is
up to 93.3% for a well-structured set of randomly shaped
polygons. The accuracy is however low (53.2%) for poorly
shaped polygons.
The method developed in this study can be extended to
identify more complex geometric shapes within engineering
drawings for machine components.
Physical Prototyping,” J. Comput. Inf. Sci. Eng., vol. 18, no. 1, p. 011002, Nov.
2017.
[10] S. K. Chandrasegaran et al., “The evolution, challenges, and future
of knowledge representation in product design systems,” vol. 45, no. 2, pp. 204–
228, Feb. 2013.
[11] J. Lladós, E. Valveny, G. Sánchez, and E. Martí, “Symbol
Recognition: Current Advances and Perspectives,” 2001.
[12] M. F. A. Jabal, M. S. M. Rahim, N. Z. S. Othman, and Z. Jupri, “A
Comparative Study on Extraction and Recognition Method of CAD Data from
CAD Drawings,” in 2009 International Conference on Information
Management and Engineering, 2009, pp. 709–713.
[13] B. . Prabhu, S. Biswas, and S. . Pande, “Intelligent system for
extraction of product data from CADD models,” Comput. Ind., vol. 44, no. 1,
pp. 79–95, Jan. 2001.
[14] J. Flusser, “Moment Invariants in Image Analysis.,” Enformatika,
pp. 376–381, 2006.
[15] K. Riesen and H. Bunke, “IAM Graph Database Repository for
Graph Based Pattern Recognition and Machine Learning,” in Structural,
Syntactic, and Statistical Pattern Recognition. SSPR /SPR 2008. Lecture Notes
in Computer Science, 2008.
[16] M. Vento, “A long trip in the charming world of graphs for Pattern
Recognition,” Pattern Recognit., vol. 48, no. 2, pp. 291–301, Feb. 2015.
[17] T. Cheng, J. Khan, H. Liu, and D. Y. Y. Yun, “A SYMBOL
RECOGNITION SYSTEM.”
[18] L. Wenyin, W. Zhang, and L. Yan, “An interactive example-driven
approach to graphics recognition in engineering drawings,” Int. J. Doc. Anal.
Recognit., vol. 9, no. 1, pp. 13–29, Feb. 2007.

REFERENCES

[1]M. I. Jordan and T. M. Mitchell, “Machine learning: Trends, perspectives,

and prospects.,” Science (New York, N.Y.), vol. 349, no. 6245, American
Association for the Advancement of Science, pp. 255–60, 17-Jul-2015.
[2] Q. Huang, “Application of Artificial Intelligence in Mechanical
Engineering Qi Huang,” in 2nd International Conference on Computer
Engineering, Information Science & Application Technology (ICCIA 2017)
Application, 2017, vol. 74, no. Iccia, pp. 855–860.
[3] D. Wu, C. Jennings, J. Terpenny, S. Kumara, and R. X. Gao,
“Cloud-Based Parallel Machine Learning for Tool Wear Prediction,” J. Manuf.
Sci. Eng., vol. 140, no. 4, p. 041005, Feb. 2018.
[4] N. Ahmad and A. A. Haque, “Manufacturing feature recognition of
parts using DXF files,” 4th Int. Conf. Mech. Eng., no. April, p. VI-111-116,
2001.
[5] A. S. Sobh, S. A. Salem, R. Darwish, M. Hussein, and O. Karam,
“Unsupervised clustering of materials properties using hierarchical
techniques,” Int. J. Collab. Enterp., vol. 5, no. 1/2, p. 74, 2015.
[6] M. Fuge, B. Peters, and A. Agogino, “Machine Learning Algorithms
for Recommending Design Methods,” J. Mech. Des., vol. 136, no. 10, p.
101103, Aug. 2014.
[7] A. K. Noor, “AI and the Future of the Machine Design,” Mech. Eng.,
vol. 139, no. 10, p. 38, Oct. 2017.
[8] G. X. Gu, C.-T. Chen, and M. J. Buehler, “De novo composite
design based on machine learning algorithm,” Extrem. Mech. Lett., vol. 18, pp.
19–28, Jan. 2018.
[9] M. L. Dering, C. S. Tucker, and S. Kumara, “An Unsupervised
Machine Learning Approach to Assessing Designer Performance During