Smart Draw!
Doodle Recognition
Saad Ali Khan
Department of
Computer Science And
Information Technology
NED University
KARACHI, PAKISTAN
saadalikhan1996@gmail.com
Taimoor Tahir
Department of
Computer Science And
Information Technology
NED University
KARACHI, PAKISTAN
taimoor.t@gmail.com
Abstract—Smart Art is new kind of drawing tool
that allows the user to easily draw technical drawing
while ‘on the go’. Inspired by the Doodle, a hand
drawing project by Google, Smart Art takes the
enriched capabilities of AI especially deep learning and
help users in making technical and business diagrams
more professional while giving them the ease of
doodling The user can draw diagrams while in
meetings, in conference or in class and the application
gives smart suggestions to convert the hand illustrations
of shapes to more presentable form. In this paper, we
will discuss Autonomous conversion of hand drawn
technical illustrations to digital age using our smart
suggestion engine which recognizes the component of
diagram and suggest relevant professional
representation from our repository of images. Using this
tool, a user can draw specialized and business graphs on
Smart Art board that can be used further in reports
eliminating the hassle to re construct the same diagram
or illustration using a software for documentation
purpose. We have used Convolution NN for recognizing
the hand illustration and Firebase to retrieve the
suitable suggested image from repository
Keywords—CNN, insert, fast, innovative, creative
drawings, Predictive output
I. INTRODUCTION
Artificial Intelligence has been a hot word for the last
few decades, but the last few years haven't worked.
Our journey began in the 50s when the father of AI
(Alan Turing) published his first paper. started in the
second half. He developed an imitation game just to
make sure machines (computers) could really think
like humans. The game includes his three races:
male, female, and interrogator. Investigators are
segregated from men and women. And he had to ask
1
Saqib Mumtaz
Department of
Computer Science And
Information Technology
NED University
KARACHI, PAKISTAN
thesaqibmumtaz@gmail.com
Ali Raza Khatri
Department of
Computer Science And
Information Technology
NED University
KARACHI, PAKISTAN
ali.raza@gmail.com
them some questions to determine their gender, and
also found a way to instruct the machine. Moreover,
it is now known as AI. The child machine's structure
was composed of genetic material, and its usual
decisions depended on the experimenter's judgment
of various words. It can be said that pattern
recognition is a problem and machine learning is a
possible solution. The goal of machine learning is not
to make perfect guesses. Machine learning allows us
to use models, loss functions and multiple approaches
to determine which approach provides better image
processing. Image processing also depends on the
type of image processing. This is because we have to
use multiple loss functions that perform better than
others due to their unique properties. There are many
examples of machine learning improving our
lifestyles. Machine learning is an artificial
intelligence (AI) technique that gives systems the
ability to learn and improve naturally from
experience without being explicitly programmed.
Machine Learning Cynosure is about developing
computer programs that can approach data and use it
to obtain knowledge. The cognitive process begins
with investigation and detail. B. Patterns, direct
events, or guidelines to look for patterns in your data
and use the guides you've published to draw better
conclusions. The point is to allow computers to
automatically acquire knowledge and change steps
accordingly without human intervention or assistance
“Smart art” allows you to create hand-drawn shapes
for technical drawings, enhanced by apps. Hand-
drawn shapes are easy to use when demonstrating
concepts in a meeting or giving a lecture in a class,
but lack the presentation and professional look and
feel to document directly there. I can not do it. Not
only will it be difficult to remember the shapes later,
but you may be bad at drawing. Therefore, it takes
extra effort to properly document your drawings
with professional tools.
Here are some fresh ideas for bridging the gap
between drawing a concept on paper and turning it
into a full-fledged drawing. Smart Art Board allows
you to illustrate your ideas using Sketch Canvas and
Smart Art Engine, suggesting professional shape
images suitable for illustration. The user can then
replace the illustration with an improved version of
the app. Suggested shapes are symbols or
professionally drawn shapes. Users can select their
own versions of drawings as well as drawings
submitted by other artists. Smart Art is a new kind of
drawing tool that combines the magic of machine
learning with developer drawing to empower anyone
to create something visually and quickly. We used a
convolutional neural network to predict categories of
shape objects. That prediction evolves as you add
more detail. Beyond the limits of intellectual art!
The ability to recognize and classify hand-drawn
doodles has important implications for general
artificial intelligence development. For example,
research in the fields of ingenuity, foresight, and
pattern reputation, especially in subfields involving
optical character recognition (OCR), would greatly
benefit from the introduction of robust classifiers for
overly noisy recordings. There is a possibility. As the
motivation behind this test, let's focus on the type of
the complete doodle. For simpler reasons than in the
actual game, this effort is due to the wide range of
classes (345), the wide format of his doodles even
within a single taxonomy, and the wide variety of his
doodles across multiple classes. remains extreme due
to the confusing proximity of So let's create a multi-
class classifier where information is an intelligent art!
Doodle recognition and the result is the expected
classification of delimited items.
II. RELATED WORK
Similar work was done to create a powerful hand-
drawn forum. The sketch board should have a green
version that respects the sketch below and can
retrieve an equivalent drawing from memory. While
many researchers have worked to create good-
looking sketches or complete imperfect sketches,
researchers have also turned to image retrieval
systems that discover comparable drawings from
memory. have also been working on. At Stanford
University, Kristine and James [2] used
convolutional neural networks (CNNs) to reconstruct
noisy or incomplete sketches. A CNN classifier
2
creates guidelines for perfecting a drawing or
removing noise from a sketch. Shortage. Another
way to refine your sketch is to interpolate between
latent vectors. In this way, we can visualize how an
image transforms into another photo by visualizing
the interpolation reconstruction. With that in mind,
learn how the model uses high settings to create
more diverse sketches, taking into account the
spherically interpolated latent vectors that are later
compared to another model with low settings. I hope
that. [15] suggests that latent vectors encode the
conceptual power of sketches. From this, we can see
that models trained on low settings are capable of
drawing sketch analogies. For example, subtracting
the encoded cat head latent vector from the full cat
latent vector gives us a vector representing the frame.
Repeat this test to remove the complete cat frame.
We therefore find that this analogy allows us to
discover how versions organize their latent space to
represent different criteria of the generated
attractiveness complex. The SBIR (Sketch Based
Image Retrieval) algorithm can create the same
image with a drop of paint or a preset
look. This structure was primarily built entirely on
histograms of world colors, dimensional design, or
site contingencies. The addition of strongly colored
droplet positions modified in suggests the use of a
shadow rubric containing waves. This has now been
shown to be beneficial for a wider range of color
texture image searches. The SBIR representation is
the use of edge maps to extrapolate the sketch from
the drawing before matching. Bimbo and Pala
discuss their method in [4], which mainly relied on
the flexible addition of image placement to facet
plans within sketches to assess similarities.
Complementary and tensile deformation strength
criteria used in the fitting are used to calculate the
similarity. Link drawings and snapshots in date to
display subsequent images. Nevertheless, this type of
scheme has limited use in SBIR structures due to its
price
A later method [5] combines the world labeling of
hue (e.g., RGB histograms) and shape (e.g., edge
direction histograms) to decompose the side pieces
into filamentary representations that encode distance,
deflection, and comparison dimension connections
increase.
Similarly, Rajendran and Chang [6] recommend to
withdraw aspect designation for) from source
drawing question and vacation spot picture and
(compare their curvature and) path (histograms for
shape similarity. Method utilizes multiple scales to
account for) diverse degrees of element in cartoon
question. Shih and Chen [7] extract features that
includes invariant moments, histogram of area
guidelines and varieties of rework coefficients that
are robust to geometric deformation, in an effort to
describe representative gadgets in the venture of
trademark segmentation and retrieval. [8]
Employ angular-spatial distribution of pixels in the
abstract sketch to extract capabilities using the
Fourier transform. The extracted features are
rotational and scale invariant and strong against
translation. [9] this suggest an actual-time SBIR
machine that locates picks the usage of a sketched
depiction of object shape divide the photo into a
regular grid and compute descriptors (either EHD
[10] or the Structure Tensor [11]) from each cellular
— concatenating the ones to shape an international
sketch feature. As we later display, this offers
lemmatizes the invariance to modifications in
function, scale or orientation compared to our method
And some of different descriptors [12] within a bowl
device. A scalable method to SBIR became supplied
very lately in mind finder [13] through spatial
gridding of edge portions (edgels) distinguished
within the picture. The approach is productive,
however as with [14] presentations restrained energy
to interpretation accepting portrayals to round contain
a similar vicinity as the gadgets they delineate.
III. METHODOLOGY
Prescriptive Analytics envisions using the results of
machine learning and deep learning to predict future
events (Predictive Analytics) and go beyond the
limits of human interaction to initiate proactive
decision-making. A smart art app that lets you safely
paint on canvas exactly how you want it. Choose a
route based on current data (taking shape into
account and optimizing the route). It implements
convolutional neural networks (CNNs) and uses
libraries such as NumPy, Matplotlib, Sklearn, SciPy,
Keras and Tensorflow. beginning. Let's take a closer
look at the NumPy library. This is a general purpose
array processing package. Provides high-performance
multidimensional array objects and tools for
manipulating those arrays. Basic package for Python
scientific computing. NumPy packages are used to
perform various operations. A nd array (NumPy
array) is a multidimensional array used to store
values of the same data type. These arrays are
indexed like sequences and start at zero. Next,
Python's matplotlib library. Use it to draw charts,
histograms, bar graphs, and everything used in
training to check accuracy and performance. Then
move on to cufflinks and plots, which provide
interactivity to plots. increase. The Sklearn (Scikit-
learn) library provides various classification,
regression and clustering algorithms such as support
vector machines, random forests, gradient boosting,
3
k-means, DBSCAN, and works with the Python
numerical and scientific libraries NumPy and SciPy
Designed to work together. design.
SciPy is a free, open-source Python library for
scientific and engineering computing. SciPy is built
on top of the NumPy Array object and is part of the
NumPy stack, which includes tools like Matplotlib,
Pandas, SciPy, and an ever-growing set of libraries
for scientific computing. Keras is a high-level neural
network API written in Python and running in
Tensor-Flow. Designed with a focus on enabling
rapid experimentation. Being able to get results from
ideas with the least delay possible is the key to good
research. That's why we use Keras in our project.
Keras makes prototyping easy and fast (due to its
ease of use, modularity, and extensibility). It supports
both convolutional and recurrent networks, and a
combination of the two that runs seamlessly on CPUs
and GPUs. Tensorflow is an open-source artificial
intelligence library that uses data flow diagrams to
create models. This allows you to create large neural
networks with many layers. Tensorflow is primarily
used for classification, recognition, understanding,
discovery, prediction, and creation. A Smart Art
Doodle model known for recognizing and quickly
learning local features in images. First, we perform
some preprocessing of the data for this purpose. Let
us first discuss the dataset we use in our project and
how we perform different processing over it to
achieve the best results, for training we use 70% of
the total dataset and the dataset consist of 4 classes
for each class we have 300 shapes for training, the
batch size we required for our model is 32 while the
no. of epochs is 4. We divide image rows and
columns by 128, for our 4 classes we use 1 channel.
We need to convert our image data into float and
after that we need to divide it by 255 (for image to
feature vector). The labels for our classes are as
follows. For (class 0) it is 0:300, for (class 1) it is
300:600, for (class 2) it is 600:900 and for (class 3) it
is 900:1200. Firstly, we use 2D convolution over our
dataset and then we apply activation function. By
following the CNN Architecture, we apply flatten to
bring all levels of a multi-layered image down to one
plane. And finally we feed the result through dense
layer and all in it we have a dropout rate of 0.5 then
we compile all results. In order to stop the training
phase of data we need to add a callback function such
that it would stop the training and don’t get it over fit.
We visualize our data according to the loss and the
accuracy level we get in training and finally we do
plot and print the confusion matrix. Then we are
Saving and loading model and weights then we
serialize into JSON file format, then serialize
Weights into HDF5 file format then load and create
model then finally loads weights into new file format
(HDF5 format) models. As first we apply our
algorithm on the sample datasets.

Figure 4.1

Figure 4.2
Figure 4.11 Figure 4.12

Figure 4.3

Figure 4.4

Figure 4.5 Figure 4.6

IV. RESULT
In first phase we test our model on four shapes that
includes circle, square, rectangle and triangle and all
evaluation was based on dataset of 200 images for
each object among which we get the highest accuracy
Figure 4.7 Figure 4.8 over triangle which was 1 and overall accuracy for all
four object was 0.68. 80% of the images were taken
for training and 20% were taken for testing.
In second phase, we test our model on another 4
shapes named arrow, diamond, rectangle and circle.
And this time we achieve highest accuracy over
circle which is 0.8 and the overall accuracy of our
model was 0.81. Which is higher than the previous
result which was 0.68 as shown in table 5.1 and 5.2

Table 5.1

Then we apply on our own datasets which we make.

Figure 4.9 Figure 4.10

4
Table 5.2
V. CONCLUSION AND FUTURE WORK
In this A.I tool, the features we have added are
important for every technical person. But this is not
all stop over here. It can be further more enhanced in
future. We can move it toward electronic side also by
enhancement of circuit equipment's. We can make
this application a simulation tool as well in future.
Which will perform simulation for every type of
connection related to electronic, electrical fields as
well. We can also extend it up by moving this app
Towards business side as well. By which we can
have business charts, business ow charts, and all
needs of a business person for expressing his point of
view clearly.
Not only doing above mentioned things we can make
our neural network better and stronger by converting
it to DCNN (deep convolution neural network). By
applying different methods, we can also enhance our
model's accuracy level.
VI. ACKNOWLEDGEMENT
I would like to express my special thanks of gratitude
to my advisor and co- advisor (Dr. MUHAMMAD
UMER and MOONA KANWAL) who gave us this
golden opportunity to do this wonderful project
(SMART ART), which also helped us in doing a lot
of Research and we came to know about so many
new things We are really thankful to them. We are
over helmed in all humbleness and gratefulness to
acknowledge our depth to all those who have helped
us to put these ideas, well above the level of
simplicity and into something concrete. Any attempt
at any level can 't be satisfactorily completed without
the support and guidance of MY parents and friends.
Secondly, we would like to thank our parents and
friends who helped us a lot in finalizing this project
within the limited time frame who helped us a lot in
guiding us from time to time in making this project.
VII. CODE
5
https://github.com/thesaqibmumtaz/Smart_Art.git
VIII. REFERENCES
[1] Xingyuan Zhang, Yaping Huang, Qi Zou,
Yanting Pei, Runsheng Zhang, Song Wang. (8,
January,2019) A Hybrid Convolutional Neural
Network for Sketch Recognition
[2] Kristine Guo., James WoMa, & Eric Xu, Quick,
Draw! Doodle Recognition
(November 2016).
[3] Saya Fujino, Taichi Hatanaka, Naoki Mori,
Keinosuke Matsumoto, Evolutionary Deep Learning
based on Deep Convolutional Neural Network for
Anime Storyboard Recognition (22 May 2018).
[4] del Bimbo, A., Pala, P., 1997. Visual image
retrieval by elasticmatching of user sketches, in:
PAMI, pp. 121-132.
[5] ain, A. K., Vailaya, A., 1996. Image retrieval
using color andshape. in: Pattern Recognition, pp.
1233-1244.
[6] Rajendran, R., Chang, S., 2000. Image retrieval
with sketchesand compositions. In: ICME. pp. 717-
720.
[7] Shih, J.-L., Chen, L.-H., 2001. A new system for
trademark segmentation and retrieval. in: Image
Vision Computing, pp. 1011-1018.
[8] Chalechale, A., Naghdy, G., Mertins, A., 2005.
Sketch-based image matching using angular
partitioning. in: IEEE Trans. Systems,
Man, and Cybernetics, pp. 28-41.
[9] Eitz, M., Hildebrand, K., Boubekeur, T., Alexa,
M., 2009. A descriptor for large scale image retrieval
based on sketched featurelines. In: SBIM. pp. 29-38.
[10] Shih, J.-L., Chen, L.-H., 2001. A new system for
trademark segmentation and retrieval. in: Image
Vision Computing, pp. 1011-1018.
[11] Eitz, M., Hildebrand, K., Boubekeur, T., Alexa,
M., 2009. A descriptor for large scale image retrieval
based on sketched feature lines. In: SBIM. pp. 29-38.
[12] Shechtman, E., Irani, M., June 2007. Matching
local self-similarities across images and videos. In:
CVPR.
[13] Wang, C., Cao, Y., Zhang, L., 2011.Mindfinder:
A sketch based image search engine based on edge
index. In: Proc. Comp. Vision
and Pattern Recognition.
[14] Eitz, M., Hildebrand, K., Boubekeur, T., Alexa,
M., 2009. A descriptor for large scale image retrieval
based on sketched feature lines. In: SBIM. pp. 29-38.
[15] T. White. Sampling Generative Networks.
ArXiv e-prints, Sept. 2016.
[16] Krizhevsky, I. Sutskever, and G. E. Hinton,
“Imagenet classification with deep convolutional
neural networks,” in Advances in neural information
processing systems (NIPS), 2012, pp. 1097–1105.
[17] R. Girshick, J. Donahue, T. Darrell, and J.
Malik, “Rich feature hierarchies for accurate object
detection and semantic segmentation,” in
Proceedings of the IEEE Conference on Computer
Vision and Pattern Recognition (CVPR), 2014.
[18] J. Donahue, Y. Jia, O. Vinyals, J. Hoffman, N.
Zhang, E. Tzeng, and T. Darrell, “Decaf: A deep
convolutional activation feature for generic visual
recognition,” CoRR, vol.abs/1310.1531, 2013.