Professional Documents
Culture Documents
Table of Contents
1) Introduction......................................................2
2) Aims and Objectives.......................................2
2.1) Problem.................................................3
2.2) Solution.................................................3
3) Related Work...................................................3
3.1) Literature Review..................................3
4) Method.............................................................3
4.1) Artificial Neural Network.....................3
4.1.1) ANN Experiment 1............................4
4.1.2) Comparison between the confusion matrix and error rates with Random Forest Tree
4.1.3) ANN Experiment 2............................5
4.1.4) PCA....................................................5
4.1.5) PCA on ANN.....................................5
4.2) Gradient Boosting..................................6
4.2.1) Gradient Boosting Experiment 1.......6
4.3) Difference between ANN and Gradient Boosting 7
4.4) Advantages of ANN and Gradient Boosting 7
4.5) Dis Advantages of ANN and Gradient Boosting 7
5) Result and Discussion......................................8
5.1) Artificial Neural Network.....................8
5.2) Performing PCA on ANN.....................9
5.3) PCA on ANN Experiment 1..................9
5.4) Gradient Boosting Experiment 1.........10
6) Conclusion and Future Work.........................10
7) REFRENCES.................................................10
2
to nearby neurons when it receives a certain are first classified. Geographic location, product
number of impulses and their aggregate surpasses price variety, overall price level, overall quality of
a certain threshold value (Witkowska, 1999).
Artificial neural networks are commonly used to items, time of waiting for payment, service attitude,
handle pattern classification problems (ANNs). A and shopping environment were all included as
case in point is bankruptcy prediction, in which input factors to the network. For the output variable,
neural networks are used to identify whether or not there is only one possible measurement value.
a corporation is insolvent. Evaluation of the firm's (2012) Ogwueleka, Misra, Colomo-Palacios, and
creditworthiness is related to the bankruptcy
prediction issue since the bank makes the Fernandez ANNs are commonly used because of
decision after researching the firm's financial their modelling capabilities. ANN is a versatile,
situation (Witkowska, 1999). Throughout the non-parametric technique based on biological
1990s, Poland saw a surge in the number of neural circuits. The multilayer perceptron is an
commercial banks. These banks analyses and effective classification tool that is widely used in
determine a company's creditworthiness using
several forms of analysis. practice.
In ANN models, the input and output parameters Neural networks are used by banks to analyses
are the same as in linear models. The three basic credit and loan applications and determine the
components of these models are the input data possibility of default. ANN approaches for default
layer, hidden layer(s), and output measure(s) layer. prediction, which first appeared in 1990, generally
Each of these levels has nodes, which are connected
to nodes in nearby layers (s). The transformation used multilayer networks and achieved higher Type
function and the weighted summation functions I and Type II classification accuracy than MDA.
may both be found in the hidden layer (s). Both of When MDA, LR, and decision trees were used to
these functions link the data values in the input to forecast bank and thrift failure, the multilayer
the measurements in the output. The weighted network fared the best. The bank failure prediction
summation function is often used in a
feedforward/back propagation neural network based on KOHONEN's self-organizing maps was
model (Nazari and Alidadi, 2022). accurate (Li and Ma, 2010).
4.1.1) ANN Experiment 1
An artificial neural (ANN) is a piece of man-made
brainpower designed to mimic the functioning of
the human mind. ANNs are made up of caring units,
which include information sources and yields. The
data sources are what the ANN uses to provide the
best yield.
We altered the values of the following parameters to
acquire the best outcome while designing a solution
with the ANN Algorithm.
Figure 1: A simplified neural network model Probability
I used to be
A simple neural network is shown in Figure 1. A
neural network is an interconnected collection of Number of hidden neurons
nodes comprised of neurons in an input layer, one A multilayer network is one that incorporates buried
or more hidden layers, and an output layer, as neurons. In general, the number of hidden neurons,
shown in Figure 1. Nh, is proportional to the degree of complexity of
The input pattern is internally represented by the the class boundaries. As the class boundaries get
middle layer, which is placed in the center of the more intricate, the number of buried neurons should
input layer. Patterns are recognized at this layer by be increased. To avoid the problem of overfitting,
distilling the input's attributes and then passing the number of hidden neurons, Nh, should be
them on to the output layer. The output layer's maintained as low as possible.
choice on categorization for the input patterns is the During the studies, it was experimented with
result of the output layer. As a result, the distilling various hidden neuron values in order to get the
character layer is referred to as the intermediate highest level of accuracy. The number of neurons
layer. In this study, the input and output variables
5
Figure 4: Difference between ANN and RFT Error Rates Figure 8: PCA Plot
4.3) Difference between ANN and Gradient ii. Lots of versatility - can optimize on several
Boosting loss functions and offers numerous hyper parameter
Boosting is a type of feature choice. Consider adjustment possibilities, making the function fit
Adaboost, which is utilized in the Viola-Jones face quite adaptable.
detector. There are many characteristics at first, but iii. There is no need for data pre-processing; it
include them all in the final classifier might be frequently works well with category and numerical
time-consuming and inaccurate. Thus, boosting information as is.
picks features by first locating the best features and iv. Imputation is not necessary to handle
then increasing the weights of misclassified data missing data.
points such that the following round of feature (Gradient Boosting for Classification |
selection concentrates on them. The feature Paperspace Blog, n.d.)
detectors in this case are regarded as weak 4.5) Dis Advantages of ANN and Gradient
classifiers. Boosting
By linearly combining weak classifiers, a strong Dis-Advantages of ANN
classifier may be generated. Neural networks, on i. Because neural networks are a "black box,"
the other hand, are more difficult to understand and their capacity to clearly identify possible
train. It is difficult to establish the importance of causal linkages is restricted.
characteristics for example, in ANNs. As a result, ii. Neural network models may be more
neural networks are similar to learning black boxes. challenging to utilize in the field since they
It is difficult to extract meaningful simpler data need more processing resources.
models from them once they have learned. They iii. Neural network models are prone to
also learn significantly differently from boosting in overfitting. The creation of neural network
that they use a gradient-based learning method to models is empirical, and many
try to minimize a cost function on all training sets. methodological concerns have to be
The behavior of the total system is determined by resolved.
the learned weights and biases, and it is difficult to from (Tu, 1996)
derive meaningful models from such Dis-Advantages of Gradient Boosting
representations ( What Is the Difference between i. Gradient Boosting Models will continue to
Boosting and Artificial Neural Networks in Data improve in order to reduce all mistakes. This
Mining? - Quora, n.d.). can lead to overfitting by exaggerating
4.4) Advantages of ANN and Gradient outliers.
Boosting ii. Computationally costly - frequently
Except for the differences between them they have necessitates a large number of trees (>1000),
some advantages as well which can be time and memory consuming.
Advantages of Artificial Neural Network iii. Because of the approach's considerable
i. The development of neural network models flexibility, there are numerous parameters
necessitates less formal statistical training. that interact and substantially impact its
ii. Complex nonlinear interactions between behavior (number of iterations, tree depth,
independent and dependent variables can be regularization parameters, etc.). During
detected implicitly by neural network tuning, this necessitates a huge grid search.
models. iv. Less interpretive in nature, yet this may be
iii. All conceivable interactions between easily remedied with a variety of methods.
predictor variables may be detected using (Gradient Boosting for Classification |
neural network models. Paperspace Blog, n.d.)
iv. Various training techniques may be used to
build neural networks.
from (Tu, 1996)
Advantages of Gradient Boosting
i. Frequently gives exceptional forecasting
accuracy.
8