Logistic Regression Model

Logistic regression is a fundamental classification technique for modeling. It belongs to the

group of linear classifiers and is somewhat similar to polynomial and linear regression. It is
the go-to method for binary classification problems (problems with two class values). Two
libraries are used- 1.sklearn and 2.statsmodel.

Testing and Training data prediction

0 1

0.42965 0.57034
0
5 5

0.14420 0.85579
1
1 9

0.00588 0.99411
2
2 8

0.84656 0.15343
3
2 8

0.05857 0.94142
4
7 3

Train score = 0.8360037700282752

Test score = 0.8245614035087719

GRID Search CV for Logistic Regresssion

GridSearchCV(cv=3, estimator=LogisticRegression(), n_jobs=-1,

param_grid={'max_iter': [10000, 100000], 'penalty': ['l2',
'none'],
'solver': ['newton-cg', 'saga'],
'tol': [0.0001, 0.001]},
scoring='accuracy')

{'max_iter': 10000, 'penalty': 'none', 'solver': 'saga', 'tol': 0.001}

LogisticRegression(max_iter=10000, penalty='none', solver='saga',

tol=0.001)
GridSearchCV implements a “fit” and a “score” method. It also implements “score_samples”,
“predict”, “predict_proba”, “decision_function”, “transform” and “inverse_transform” if they
are implemented in the estimator used.

Prediction-
Train-

0 1

0.93683 0.06316
0
5 5

0.10921 0.89078
1
2 8

0.24901 0.75098
2
5 5

0.12895 0.87105
3
0 0

0.01966 0.98033
4
2 8

Test

0 1

0.42840 0.57159
0
2 8

0.17732 0.82267
1
1 9

0.00441 0.99558
2
2 8

0.85219 0.14780
3
7 3

0.06413 0.93587
4
0 0
Training data Accuracy = 0.8294062205466541
Testing data Accuracy = 0.8201754385964912

Intercept of the Final Model = The intercept for the model is : [0.20374458]

The coefficient for age is -0.006866145631087787

The coefficient for economic.cond.national is 0.7137612157406149
The coefficient for economic.cond.household is 0.18715575070817067
The coefficient for Blair is 0.6704027420462002
The coefficient for Hague is -0.7456694342646842
The coefficient for Europe is -0.18361059463352536
The coefficient for political.knowledge is -0.2556036248854115
The coefficient for gender is 0.1559313613896698

The coefficients for each of the independent attributes. The sign of a regression coefficient
tells whether there is a positive or negative correlation between each independent variable
the dependent variable. A positive coefficient indicates that as the value of the independent
variable increases, the mean of the dependent variable also tends to increase. A negative
coefficient suggests that as the independent variable increases, the dependent variable tends
to decrease.
economic.cond.national has more positive coeffiecient . A positive coefficient indicates that
as the value of the independent variable increases.

Feature: 0, Score: -0.00687

Feature: 1, Score: 0.71376
Feature: 2, Score: 0.18716
Feature: 3, Score: 0.67040
Feature: 4, Score: -0.74567
Feature: 5, Score: -0.18361
Feature: 6, Score: -0.25560
Feature: 7, Score: 0.15593