Comparing ML Algorithms using Scikit-learn

2020028377 문경태

Abstract.
Machine learning algorithms of various models have been developed over the years. Linear regression, logistic
regression, SVM, FDA and Graphical Model, we've learned, and we're able to deal with it. In addition, Python's
Scikit-learn library provides it for easy use. In this project, we will have time to use this to do machine learning
ourselves. The MNIST dataset is one of the most representative datasets of handwritten digits provided by the
Mixed National Institute of Standards and Technology. This dataset is often used to test and learn algorithms in
machine learning and deep learning. We will try machine learning ourselves using this dataset.

Problem Set.
The MNIST dataset contains 1797 black and white handwritten numeric images with 8*8 pixels in size. Each
image represents a number from 0 to 9, and a corresponding label (target) is provided for each image. Now
we're going to experiment with this data, and we're going to move a total of 1,797 two-dimensional data to one
dimension. At this point, the matrix of 8*8 changes to 64 columns.

Figure 1. MNIST DATA(8*8 pixel).

Experiments.
1. Logistic Regression.
A. To use logistic regression, we need to calculate each data as an independent variable when
designing the model. This model classifies each variable as a number from 0 to 9.
B. Before learning the model, separate the data into shapes that fit the learning. Place the classifier as
Logistic Regression, and proceed with the learning.
C. Learn the model, fit it. Calculate the time and accuracy it takes.

2. SVM.
A. The Support Vector Machine (SVM) is a model used for classification and regression. Given a set of
data in either category, the SVM algorithm creates a nonprobability binary linear classification
model that determines which category the new data belongs to based on the given data set.
B. Prior to learning, data with interclass features are collected and labeled for each data. At this point,
standardize the data.
C. Learn the data by finding the support vector and the decision boundaries. At this point, C of the
parameter determines the degree to which the data samples are allowed to be placed in different
classes, and gamma determines the curvature of the crystal boundary. The key is to coordinate these
two well.

3. Neural Network
A. It utilizes a multilayer perceptron (MLP) to classify the MNIST data. Here we add the Dense layer
to learn the model with multiple layers.
B. At this time, when activation is performed as a sigmoid function, a vanshing gradient occurs, so the
model is trained with Relu as activation.
Figure 2. MNIST DATA TRAINING.

Figure 3. MNIST DATA Prediction.

Experiments Result.
1. Logistic Regression.
A. It has 95% accuracy. If the pixel was larger (if it was a larger matrix than 8*8) We could have
had higher accuracy.
B. It took about 6.8 seconds to learn. If we had more data, it would have taken more time.

2. SVM.
A. It has an accuracy of about 97%. At this point, you can have less accuracy by giving more
(overfitting) or less (overfitting). Reducing the Gamma value results in less accuracy because the
curvature of the boundary is reduced.
B. It took about 0.05 seconds to learn. Given more data, it will take a long time to learn. However,
CPU overload is expected.

3. Neural Network.
A. It has an accuracy of about 97%. Since MLP was used at this time, you can expect higher
accuracy if you increase the hidden-layer.
B. It took about 0.22 seconds to learn. More data does not significantly increase learning time, but
CPU overload is also expected.

Figure 4. MNIST DATA Result of Logistic Regression.

Figure 5. MNIST DATA Result of SVM.

Figure 6. MNIST DATA Result of Neural Network.

Discussion.
1. Logistic Regression.
A. Each variable is assumed to be an independent variable and the model is trained, you can expect
high accuracy regardless of overfitting.
B. There is a disadvantage that pre-processing of data is required.
C. However, as the number of inputs increases, it is expected that it will take longer to learn the
model. Also, because it is one layer, you cannot classify the problem of using multiple layers,
such as XOR.
 2. SVM.
A. As expected, even with little data, you can see high performance. It is also very useful because
you can learn nonlinear decision boundaries.
B. However, not only can learning or predicting large datasets be slow, but the cost of calculating
increases. In addition, there is a disadvantage that data preprocessing is required.
C. Depending on the parameter values, the prediction may vary dramatically. In addition, a larger
number of inputs can lead to overfitting.

3. Neural Network.
A. By utilizing the Hidden-layer, it is suitable for high-dimensional data and complex problems. In
addition, automatically learning features from input data is one of the biggest advantages of
neural networks.
B. However, there is a disadvantage that the calculation cost is high and requires a lot of memory.
C. Very sensitive to the amount of input. Depending on the amount of input, it is very likely to be
overfitting, especially if the activation is sigmoid, which can result in vanshing gradient.

Conclusion.
In this study, We have covered various models for learning MNIST data. Among the three models, the neural
network showed the highest accuracy. But there is something more important than this. That's what problems
you have and which models you choose when you're dealing with machine learning. In order to know that, it is
necessary to understand how various algorithms of artificial intelligence produce these results and the process.
In order for this task to achieve successful learning achievement, it is also necessary to understand the principles
of artificial intelligence. To understand the difference, to know the principle by which the difference occurs.
That's the most important thing.