RESEARCH ARTICLE | JULY 25 2023

Sentiment analysis of distance learning policy during the

Covid-19 pandemic using naïve bayes algorithm 
Parasian D. P. Silitonga  ; Darsono Nababan; Emerson P. Malau; Alex Rikki

AIP Conference Proceedings 2798, 020008 (2023)

https://doi.org/10.1063/5.0154276

CrossMark

 
View Export
Online Citation

02 August 2023 12:43:12

 Sentiment Analysis of Distance Learning Policy During the
Covid-19 Pandemic Using Naïve Bayes Algorithm
Parasian D.P Silitonga1,a), Darsono Nababan2,b), Emerson P. Malau1,c), Alex
Rikki1,d)

Faculty of Computer Science, Universitas Katolik Santo Thomas, Medan, Indonesia

1
2
Department of Information and Technology, University of Timor, Kefamenanu, Indoensia
a)
Corresponding author: parasianirene@gmail.com
b)
darsono.nanban@unimor.ac.id
c)
malauemerson@gmail.com
d)
alexrikisinaga@gmail.com

Abstract. Sentiment analysis is the interpretation and classification of users' emotions (positive, negative, neutral) about
a subject in text data using text analysis Comments and opinions, especially those contained in social media, are a source
of data that can be used to measure the level of popularity of a program or product launched.. Social media is a means to

02 August 2023 12:43:12

convey aspirations directly, but every aspiration is from social media users. Everyone who expresses opinions on social
media contains positive, negative, and neutral sentiments. The implementation of the Ministry of Education and Culture's
policy on the implementation of distance learning policies during the pandemic COVID-19 received various responses
from the people of Indonesia. neutral as many as 894 comments, then 52 comments with negative sentiment, and 32
comments with positive sentiment with an accuracy value of 98.79%.

Keywords: Sentiment Analysis, Social Media, Covid-19, Education, Distance Learning

INTRODUCTION

The COVID-19 emergency period changed the education system in Indonesia in a very short time, requiring
every student to study remotely. Of course this encourages teachers and lecturers to innovate as well as policy
makers in this case the Ministry of Education and Culture of the Republic of Indonesia to make adjustments during
the COVID-19 pandemic by changing several rules by issuing circular letter number 2 of 2020 regarding the
prevention and handling of COVID-19 within the Ministry of Education and Culture, as well as circular letter
number 3 of 2020 regarding the prevention of COVID-19 in education units. In addition, a series of other policies
were issued by the Ministry of Education and Culture in response to the development of the spread of COVID-19,
such as the cancellation of the national exam (UN), adjustment of school exams and an online approach to the
student registration process. This has an impact on all lines of education in Indonesia, from elementary school to
higher education. The policies that emerged resulted in the emergence of controversies ranging from the new student
admission system (PSSB), the determination of a single tuition fee (UKT) at the tertiary level, readiness of schools
and higher education teachers in implementing distance learning received various positive and negative responses
from various One of the elements of society in social media is Twitter social media. For this reason, this study aims
to analyze the public's response to the Ministry of Education and Culture's policies in implementing distance
education (PJJ) during the COVID-19 pandemic [1].
The research that has been done by Henni Tuhuteru about public sentiment on large-scale social restrictions
(PSBB) with the Support Vector Machine method. . The data obtained are 1075 tweets and comments and are
separated between 350 training data and 725 comments testing data. The data is then classified using the Support

International Conference of SNIKOM 2021

AIP Conf. Proc. 2798, 020008-1–020008-7; https://doi.org/10.1063/5.0154276
Published by AIP Publishing. 978-0-7354-4598-7/$30.00

020008-1
Vector Machine (SVM) algorithm. The results of the study show positive sentiment by 28%, negative sentiment by
27%, and neutral sentiment by 45%. It can be concluded that public sentiment towards the plan to implement PSBB
in Ambon City based on comments on social media is quite balanced between positive and negative sentiments and
is dominated by comments with neutral sentiments [2].
Vyas et al (2018) used a different algorithm to find the accuracy of 450 customer feedback datasets in an effort
to increase customer trust. The data is divided into training data test data with a ratio = 0.66. The algorithm used to
test the data is Support Vector Machine, Decisision Tree and Naïve Bayes to find accuracy and precision with a
training dataset of 450 tweets, SVM achieves 79.08% accuracy in identifying sentiment [3]. Rizqiyah, Putri (2018)
conducted a study on the percentage of consumer comments on the gojek application using Naive Bayes. From the
research it was found that what factors require users to use the gojek application so that companies can design
application development according to consumer needs [4].
Laksana & Purwarianti, (2014) researched on automatic text classification from Twitter as part of the Bandung
city community complaint management system using the SVM, Naïve Bayes and Decisiion Tree methods with a
combination of multi label classification and power set labels. From this study resulted in a classification model with
features of word extraction of complaints [5].

METHODS

Sentimen Analysis
Liu (2015) suggests that sentiment is a subjective expression, opinion, behavior, orientation and emotion, or even
tone in the text. A collection of sentiments that are unstructured as well as scattered (scattered) if processed properly
will produce information that is not only significant but also meaningful [6][9].
Kumar and Sebastian (2012) suggest that sentiment analysis is a study of computing related to opinions and
oriented to natural language processing. [7][10]. According to Schneider (2005) Broadly speaking, sentiment

02 August 2023 12:43:12

analysis is grouped into 2 parts, namely [8]:
1. Coarse-grained sentiment analysis
An analysis process at the document level. Document classification process with 3 categories, namely:
Positive, Neutral and Negative.
2. Fined-grained sentiment analysis
The object that is classified is a sentence.

Tableau

Tableau is an application that can analyze data in the form of graphs, scatter plots, maps, and other visual forms
dynamically and easy to understand. Tableau can display millions of data from various data storage sources such as
local databases, servers, cloud, big data, and so on (Tableau, 2015).[6] The very fast data processing process makes
this application very flexible and easy to use by the public [12][13].
The research methodology was carried out in several stages starting from problem identification, problem
solving methods and evaluation process. The steps used in processing data from Twitter to get the results of
sentiment analysis using the Naïve Bayes algorithm are shown in the following figure 1:

020008-2
 Case

processing
Data Pre-
Crawling Folding
Data
data from dan
Cleaning
twitter Tokenizatio
n

processing
Classificati
on with Data

Data
Data and
Algorithm Classificati
Train Data
Naïve on
Bayes

Data
Analysis

Evaluation, Analysis
Precision and Data
F1-Measure

FIGURE 1. Research Methodology Workflow

The workflow in the image above is the overall workflow. The data processing technique uses Rapidminer tools
to process data to evaluate classification after data crawling through the Twitter API. Then the application uses
Tableau to visualize the data.
1. Data Pre-processing
The process of collecting data using the twitter API with the keyword @kemendikbud the processes carried

02 August 2023 12:43:12

out at the pre-processing stage are :
a. Data Cleaning irrelevant tweets so that they become relevant data.
b. Case folding is the process of converting words into lowercase letters.
c. Tokenization is dividing a sentence into several parts.
d. Stemming is the process of getting basic words by removing affixes and suffixes.

2. Data Processing
At the data processing stage, the classification process is carried out using the Naïve Bayes algorithm by
grouping the data into two parts, namely test data and training data. For the training data value is 75%.

3. Analysis
The analysis stage is carried out after the classification process with the Naïve Bayes algorithm so as to
produce a classification with positive sentiment, negative sentiment and neutral sentiment. To determine the
value of accuracy, precision and drawing conclusions based on the results of the analysis of the Naïve
Bayes algorithm. An example in a classification with a false statement that is classified as negative
sentiment. The following are the performance results of the classification.

TABLE 1. Evaluation Performance Classification

Category Positive Negative
Positive True Positive False
(TP Negative
(FN)
Negative False True Negative
Negative (FP) (TN)

020008-3
 To see the level of accuracy, the following equation is used:
TP + T
Accuracy = (1)
TP + FN +FP+T

Precision is the ratio of positive correct predictions of all selected items. To calculate the preccion value, the
equation is used:
TP
Precision = (2).
TP +FP
Recall
Is the ratio of true positive predictions compared to the overall true positive data, the equation to find out Recall is:
TP
Recall = (3)
TP +FN

RESULTS AND DISCUSSION

The data used in this study was taken from Twitter with the keywords @kemendikbud and #kemendikbud. The
number of data categories that have been cleaned are then labeled with sentiments consisting of positive, negative
and neutral which are the results of the classified test data and then presented with the level of accuracy of the truth
of each classification in the form of a percentage.
Pre-processing data from 991 tweets were grouped according to their respective categories, with 34 tweets in the
positive category, 53 in the negative category and 904 in the neutral category. The following is the result of
grouping after being labeled.

02 August 2023 12:43:12

TABLE 2. Example of twitter after grouping
Tweets Tweets Translate Sentiment
Jangan bayar spp sekalian :") Don't pay tuition fees at all :") Learning using the Negative
Belajar pake internet jg butuh internet also requires a lot of quota. Then people who
kuota banyak. Terus orang yg gak don't have a cellphone or laptop have to ...
punya hp atau laptop harus… Robot loses remotte. #MendikbudSalahUrus
Robot kehilangan remotte. This nation needs educated and intelligent people, not
#MendikbudSalahUrus people lying down and stupid :). the job prospects of Negative
Bangsa ini butuh org2 S1 graduation onlen is open to...
berpendidikan dan cerdas, bukan Going to the mall is fine, going on a tour is fine, Positive
org2 rebahan dan bodoh :). going to the market is also fine, it's not your turn to
prospek kerja S1 wisuda onlen study at school! #MendikbudSalahUrus
adalah buka to.. Member of Commission X of the House of
Ke mall boleh, pergi wisata boleh, Representatives Illiza Sa'aduddin Djamal assessed Positive
ke pasar juga boleh, giliran belajar that the discourse on making distance education
di sekolah tidak boleh! technology (PJJ) permanent was not a good idea.
#MendikbudSalahUrus At school, teachers & students do not only meet face
Anggota Komisi X DPR Illiza to face, but also the process of character building by
Sa’aduddin Djamal menilai, teachers for students to become obedient children.
wacana mempermanenkan Netral
teknologi pendidikan jarak jauh
(PJJ)
Di sekolah guru & siswa bkn cmn
bertatap muka, tp juga proses
pembentukan karakter oleh guru
kpd siswanya agar jd anak yg pat… Netral

020008-4
 By using the Naïve Bayes algorithm in sentiment classification, the performance vector values obtained are
shown in the table:
TABLE 3. Classification Analysis with Naïve Bayes Algorithm
True True Positive True Class
Neutral Negative Precision

Prediction 894 0 1 99,8 %

Neutral 0 32 0 100 %
Prediction

Positive 10 1 52 82,5 %
Prediction

Negative 98,89 % 96,97% 98,11%

Class
Recall

From the results of data analysis in the table above, with a total of 991 tweet comments, 32 comments were
found to be true positive, 52 comments were stated to be true negative, and 894 comments were stated to be true
neutral. From these results, the average percentage of each Recall Precision and F1-Measure is between 93.7% and
97.4%. The testing process with the Naïve Bayes algorithm testing the algorithm in this study using the RapidMiner
tool resulted in an accuracy of 98.79%. The following is a matrix for performance vector sentiment analysis using
the Naïve Bayes algorithm :

02 August 2023 12:43:12

FIGURE 2. Performance Vector Sentiment Analysis.

020008-5
To find out the suitability of the following analysis results are the results of calculations using Rapidminer with a
kappa accuracy result of 0.929% which is shown in the following figure :
TABLE 4. Reliability of Similarity in Measurement
True neutral True positive True negative Class precision
Pred. neutral 894 0 1 99,89%
Pred. positive 0 32 0 100 %
Pred. negative 10 1 52 82,54 %
Class recall 98,89 % 96,97% 98,11%

02 August 2023 12:43:12

FIGURE. 3 Graph with Sentiment Value Tableau

CONCLUSIONS

Based on comment data obtained from Twitter about the Ministry of Education and Culture's policies in the
application of distance education, the tendency of comments sent by netizens contains neutral comments. Twitter
users are neutral on the matter. Of the 991 test data, 894 comments with neutral sentiments, then 52 comments with
negative sentiments, and 32 comments with positive sentiments with an accuracy value of 98.79% using the
Rapidminer application. Here it can be concluded that social media users in Indonesia are neutral towards the
implementation of the Ministry of Education and Culture's policies in the implementation of the distance education
system (PJJ).

ACKNOWLEDGMENTS

Thanks to Universitas Katolik Santo Thomas for providing support, so that this research can be completed
properly. Thank you to the Institute for Research and Community Service, Universitas Katolik Santo Thomas,
Medan.

020008-6
 REFERENCES

1. Kementerian Pendidikan dan Kebudayaan Republik Indonesia. 2020. Surat Edaran Nomor 4 tahun
2020 Tentang Pelaksanaan Pendidikan dalam Masa Darurat Covid-19.
https://kemdikbud.go.id/main/blog/2020/03/mendikbud-terbitkan-se-tentang-pelaksanaan-pendidikan-dalam-
masa-darurat-covid-19.
2. H. Tuhuteru, ‘Analisis Sentimen Masyarakat Terhadap Pembatasan Sosial Berksala Besar Menggunakan
Algoritma Support Vector Machine’, Inf. Syst. Dev., vol. 5, no. 2, pp. 7–13, 2020.
3. V. Vyas and V. Uma, ‘An Extensive study of Sentiment Analysis tools and Binary Classification of tweets
using Rapid Miner’, Procedia Comput. Sci., vol. 125, pp. 329–335, 2018.
4. P. Algoritma, ‘Analisa Sentimen Menggunakan Naïve Bayes Untuk Mengetahui Presentase Komentar Pada
Aplikasi Go-Jek’, no. 128..
5. Laksana, J. and Purwarianti, A., 2014, August. Indonesian Twitter text authority classification for
government in Bandung. In 2014 International Conference of Advanced Informatics: Concept, Theory and
Application (ICAICTA) (pp. 129-134). IEEE..
6. Liu, B., 2012. Sentiment Analysis and OpinionMining. [e-book]. Chicago: Morgan & Claypool Publisher..
7. A. Kumar and T. M. Sebastian, ‘Sentiment Analysis: A Perspective on its Past, Present and Future’, Int. J.
Intell. Syst. Appl., vol. 4, no. 10, pp. 1–14, 2012.
8. Ling, Juen, I Putu Eka N. Kencana, and Tjokorda Bagus Oka. (2014) “Analisis Sentimen menggunakan
Metode Naïve Bayes Classifier dengan Seleksi Fitur Chi Square [Title in English: Sentiment Analysis Using
Naïve Bayes Classifier Method with Feature Selection of Chi.
9. Liu, B., & Zhang, L. (2012). A survey of opinion mining and sentiment analysis. In Mining text data (pp.
415-463). Springer, Boston, MA.
10. Liu, B. (2010). Sentiment analysis: A multi-faceted problem. IEEE Intelligent Systems, 25(3), 76-80.
11. Zhang, L., Wang, S., & Liu, B. (2018). Deep learning for sentiment analysis: A survey. Wiley
Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 8(4), e1253.

02 August 2023 12:43:12

12. Cambria, E., Schuller, B., Liu, B., Wang, H., & Havasi, C. (2013). Knowledge-based approaches to concept-
level sentiment analysis. IEEE intelligent systems, 28(2), 12-14.
13. Zhang, L., Ghosh, R., Dekhil, M., Hsu, M., & Liu, B. (2011). Combining lexicon-based and learning-based
methods for Twitter sentiment analysis. HP Laboratories, Technical Report HPL-2011, 89.

020008-7