Coaching Clinic:

Pemetaan Tema Penelitian Secara

Terstruktur untuk Menyusun Proposal
Penelitian Berkualitas
Bayu Erfianto, Agung Toto Wibowo, Nur Alamsyah dan Nur
Ghaniaviyanto Ramadhan
Agenda
Pendahuluan
Systematic Mapping Studies (SMS)

3
Course Outline Pendahuluan

1 Perkenalan

2 Goal of Literature Review

3 Methods in Literature Review

4 Systematic Mapping Study

4
Perkenalan
5
Agung Toto Wibowo Bayu Erfianto Nur Alamsyah Nur Ghaniaviyanto
Ramadhan
Goal of
Literature
Review

7
Type of Literature Review

Systematic Mapping Study

Systematic Literature Review

Systematic Mapping Study
• The Systematic Mapping Study
(SMS) in Literature Reviews
techniques are useful at the very
beginning of the literature review
as a brainstorming tool.
• The literature mapping is broadly
used to complement the Systematic
Literature Review (SLR)
Systematic Literature Review
• The SLR is a type of literature
review that collects and critically
analyzes multiple research
studies or papers through a
systematic process.
• The purpose of a SLR is to
provide an exhaustive summary
of the available literature
relevant to a research question.
Goal of Systematic
Mapping Study
Deeper knowledge in your knowledge field

Get insight about the current trends and

future challenges

Identify the most important authors

Identify the most important journals &

conferences
Comparison of SMS and SLR
Systematic Mapping Study (SMS) Systematic Literature Review (SLR)
Focus on classification, thematic Focus on establishing the state of the
analysis and identifying publication art

Identify research gaps Identify research gaps

Quality is not evaluated Quality is evaluated based on Meta
analysis
Reflects based on search query and State of the art is the outcome and
inclusion criteria quality as its major focus
Comparison of SMS and SLR
Systematic Mapping Study (SMS)
Covers a higher breadth. More
articles can be covered regardless of
lack of methodologies
Can structure a larger field
High level categories
Doesn’t go into details, which might
lead to wrong classification
Systematic Literature Review (SLR)
Mentions lack of methodological in
primary studies
Fewer studies can be included
Especially concerning e.g. research
methods and research approaches
Takes details into account.
Comparison of SMS and SLR
• Unlike the SLR, the systematic mapping study classifies
subfields of a research area and focuses on identifying and
classifying themes by collecting as many studies as possible
• A systematic mapping study is sometimes conducted as a
preliminary study before the SLR, performing a systematic
mapping study before an SLR provided valuable research
design criteria
• Systematic mapping reduces the preparation time for
subsequent research (especially PhD research), because it
provides an overview of research areas and identifies research
gaps
 General Methodology in SMS
Define RQ Define research questions (and goals)

Search in scientific databases and extract relevant contents/data (iterating the

Search in Database process in several stages).

Define Inclusion Define inclusion criteria

Define Exclusion Define exclusion criteria for your SLR

Conduct Analisys Conduct Bibliometric Analysis

PRISMA: Preferred Reporting Items for

Systematic Reviews and Meta-Analyses
 1
6

Our Methodology
 Study of Literature
• Exploration through research database: Scopus,
WoS, IEEE, etc.
• Extract metadata from research database
• Use bibliometric analysis to conduct systematic
mapping (including bibliometric analysis) to find
promising research topics and to confirm that our
selected topics area novel
• Continue to SLR: construct state-of-the art of
selected research topic to find and deeply
understand what methods are used by
researchers. The final step of SLR is to find the
potential research gaps and define research
problems.
Our Approach
• Purpose:
• to get an overview of research topics in a
particular scientific field and how far the coverage
is from past research to the present.
• Methodology :
• Conduct Protocol PRISMA with
Bibliometric Analysis
• Systematic Mapping Study with
Bibliometric dan Systematic Topic
Modeling (STM)
Our Methodology
Our Methodology
in SLR
Session one : Tea Break
21
Workshop Sesi 1:
Collecting from Research Database

22
General Goal of Our Methodology
• Build a classification scheme and structure a field of interest
• Structure of the type of research and results by categorizing a field / sub-
field
• Show frequencies of publications for categories that reflects the trend of
research in the field of interest.
• Determine coverage and relation of a certain field (subjects) in the field of
interest and provide a visual summary by mapping the results
• Extract the topics from the field of interest to obtain the trend and gaps
certain field (subjects)
• Combine the results to answer specific research questions.
Conducting PRISMA Protocol
• PRISMA (Preferred Reporting Items for
Systematic Reviews and Meta-Analyses).
• We adopted a review protocol consisting of
search terms, resources to be searched,
study selection criteria, and study selection
procedures
Identifying Research Question for SMS
The literature review (SMS) aims to
answer the following research
questions (Examples):
• RQ1. Bagaimana evolusi penelitian
Indoor Tag Localization ditinjau dari
dukungan serta populasi publikasi
dari lima tahun terakhir yaitu 2017-
2022?
• RQ2. Apa saja yang termasuk pada
area topik penelitian Indoor Tag
Localization?
• RQ3. Bagaimana trend topik dan
apa topik saat ini yang merupakan
promising research topic pada
Indoor Tag Localization ?
Step 1 PRISMA : Search Query
• Use research database: Scopus,
WoS, ScienceDirect, IEEE
• Use queries to collect publications
from database
• Limit by Year
• Limit by Journals
• Limit by Article Types
• Limit by Area (CS,EE,Med)
• Download database in CSV or TXT
format
Step 2: Study Selection
• Dari Scopus diperoleh sebanyak 438
paper, ScienceDirect diperoleh
sebanyak 21 paper dan IEEE diperoleh
311 paper sehingga jumlah total
paper yang diperoleh adalah 770
paper.
• Setelah digabungkan, dilakukan
penyaringan kembali secara manual.
• terhadap duplikasi judul
• author dan author-keyword
• paper yang tidak terdapat author
keyword dapat ditambahkan dari indexed
keyword
• hasil akhir paper yang diperoleh adalah
sebanyak 446
Step 3 PRISMA : Data Extraction
• Data extraction:
• The step of data extraction is to prepare spreadsheet for bibliometric
analysis regarding the query and keywords.
• The metadata required for further bibliometric analysis pertains more
to the document types: such as articles, conference proceedings, book
chapters, books, and other documents including reviews and technical
papers, author keywords, publication years
• See next slide
Extraction from Search Database
• Contoh File Scopus.csv
 Answering
RQ 1

• RQ1. Bagaimana evolusi

penelitian Indoor Tag
Localization ditinjau dari
dukungan serta populasi
publikasi dari lima tahun
terakhir yaitu 2017-2022?
• Gunakan database hasil
filtering
• Diproses dengan Statistik
dari file CSV (lihat halaman
berikutnya)
Step 3: Study Selection to Answer RQ 1
• Gunakan File Scopus.csv

• Klasifikasikan
Berdasarkan jenis
Publikasi vs. Tahun

• Buat Stacked Bar Plot

untuk melihat trend per
jenis publikasi per tahun
Alternative Tools – “Publish or Perish”
Alternative Tools – “Publish or Perish”
Alternative Tools – “Publish or Perish”
SOLISKAN:
Shalat, Istirahat
dan Makan
Pengantar Bibliometric Analysis

40
 Bibliometric Analysis
• Bibliometrics is the use of statistical methods to analyze books,
articles and other publications, especially in regard with scientific
contents
• In recent years, bibliometric analyses: co-citation network
analyses, and keyword co-occurrence network analysis have been
widely used to determine research trends
• Co-citation network analysis determines the structure of scientific
communications by analyzing the associations among citations
• Co-occurrence keyword network analysis allows to understand the
knowledge structure underlying a technical field by analyzing links
between keywords found in the literature and evolution of the
keyword
• Our approach only concerns with author keyword--occurrence as
unit of analysis. Scopus is used to collect publications on the topic
of interest and get an overview of the author-keyword used to
identify evolving subtopics and relations.
Bibliometric Analysis
• Using VOSViewer as a tool
(https://www.vosviewer.com)
• VOSviewer is a software tool for
constructing and visualizing
bibliometric networks.
• Bibliometric networks may
include journals, researchers, or
individual publications, and they
can be constructed based on
citation, bibliographic coupling,
co-citation, or co-authorship
relations
Bibliometric Analysis
• Example: Indoor Tag
Localisation Keyword map.
• What can be inferred from
those figures?
• Make sure that your topics /
subtopics of interests are
depicted and tends to the
quite hot and fresh research
topic
• RQ2. Apa saja yang termasuk
pada area topik penelitian
Indoor Tag Localization
Bibliometric Analysis
• RQ2. Apa saja yang
termasuk pada area
topik penelitian Indoor
Tag Localization
• Apakah topik tersebut
termasuk hot topics
Workshop Sesi 2:
Bibliometric Tools & Analysis

45
Bibliometric Analysis using VOSViewer
• We will conduct author-keyword
occurrence analysis as unit of
analysis yo answer RQ2.
• Open VOSViewer, and create
bibliometric network from
downloaded file of Scopus.csv

• Choose Co-occurrence with

author-keyword as unit analysis,
and choose full counting
Bibliometric Analysis using VOSViewer
• Define threshold of minimum
number of occurrences of a
keyword
• Verify selected keyword to
exclude unrelated keywords
Bibliometric Analysis using VOSViewer
• Co-occurrence of Author-Keyword
Network via Network Visualization
• How many research clusters exist ?
• What are relationship among sub-topics in
a research cluster, and inter-clusters?
• What method they used?
• What technology they used?
• What purpose they used?
Bibliometric Analysis using VOSViewer
• Co-occurrence of Author-Keyword
Network via Overlay Visualization
• What are the new research topics ?
• What are relationship among new sub-
topics in a research cluster, and inter-
clusters?
• What new method they used?
• What new technology they used?
• What new purpose they used?
Workshop Sesi 3 :
Penulisan Naskah untuk
Workshop sesi 1-2

50
Sampai jumpa esok hari

Have a great day

Agenda
Pengantar Systematic Topic Modeling
Berbasis Data Mining

53
Research Topic Analysis
• Systematic Mapping Study is to identify topics trend in selected research field
• Regarding RQ3, we conducted text mining to answer bagaimana trend topik dan apa
topik saat ini yang merupakan promising research topic pada Indoor Tag Localization ?
• Text mining, also known as knowledge discovery from text, relies on various text
analyses and processes to extract meaningful information from unstructured text data
to structured text modeling (STM-based)
• STM is an unsupervised learning method to determine and classify topics underlying
textual data
• The STM determines the distribution of words constituting a topic based on the
frequency of words in a document along with metadata
• The STM used in our approach is extended version of the Latent Dirichlet Allocation
(LDA), the most widely used topic modeling method
• We used Orange Data Mining Tool to conduct STM. It is visual-based STM an no need
for coding
Systematic Topic Modeling
• Orange as a tool for Topic Modeling: using CSV of bibliometric Scopus
style database
• Use only key-word author column + year
Systematic Topic Modeling
• Results of STM-based topic modeling using Orange
Systematic Topic Modeling
• STM generates proportion values for each topic
model
• We can plot year vs. proportion value to obtain
the linear trend of each topic
• What can be inferred from the plots ? Which plot
reflects the chance to determine promising
topics?

To answer RQ3: bagaimana trend topik dan apa topik saat

ini yang merupakan promising research topic pada Indoor
Tag Localization ?
Topic 8 is confirmed to our interest, and it is sill promising
Tool Alternatif – implementasi LDA pada python
Systematic Topic Modeling:
Workshop Sesi 4

59
Installation Orange
• Link: https://github.com/biolab/orange3-text
Research Topic Analysis using Text Mining
• Open Orange Data Mining Tol and draw Topic
Modeling Diagram
Research Topic Analysis using Text Mining
• Buat file Excel baru dengan dua
kolom : Author Keyword dan Year
• Pastikan tidak ada duplikasi atau
baris kosong atau unmeaning
keyword

• Click icon file pada diagram orange

dan pastikan hanya ada dua
kolom: Author-keyword dan Year
• Click pada icon corpus dan
pastikan juga hanya ada dua kolom
yang sama.
Research Topic Analysis using Text Mining
• Click icon Preprocess Text. Proses
ini bertujuan untuk menghilangkan
atau mendefinisikan regular
expression pada corpus. Biarkan
default seperti dari Orange Tool

• Icon World Cloud adalah Optional,

bertujuan untuk mengecek apakah
Corpus berhasil diproses
Research Topic Analysis using Text Mining
• Click icon Preprocess Text. Proses
ini bertujuan untuk menghilangkan
atau mendefinisikan regular
expression pada corpus. Biarkan
default seperti dari Orange Tool

• Icon World Cloud adalah Optional,

bertujuan untuk mengecek apakah
Corpus berhasil diproses
Research Topic Analysis using Text Mining
• Click icon Text Modeling dan pilih
algoritma LDA.
• Jika topic of interest anda muncul
dalam 10 Number of Topics, maka
pada Windows sebelah kanan
dapat dicari di Topic no berapa
topic of interest anda berada
• Jika tidak muncul, gunakan 15 sd.
20 Number of Topics dan amati
 Research Topic Analysis using Text Mining
• Click icon Data Table untuk melihat
output dari algoritma LDA. Pada
Data Table akan muncul Topic
Proportion dari setiap Keyword
berdasarkan Tahun.
• Click Save Data untu menyimpan
dalam file Excel.
• Selanjutnya adalah mengolah file
output STM ini dalam file excel
untuk mencari trend topic
berdasarkan topic proportion
 Research Topic Analysis using Text Mining
• Selanjutnya gunakan Linear
Regression Plot di Excel atau
Linear Regression Plot dengan
aplikasi lainnya.

• Misalnya Hasil prediksi Topic 1

dengan 95% Conf. Interval
menunjukkan bukan promising
topic krn mengalami penurunan
proportion
Pengantar Critical Review

68
Purpose of Critical Review
The critical review is a writing task that asks you to summarize and
evaluate a text.

The critical review can be of a book, a chapter, or a journal article.

Writing the critical review usually requires you to read the selected
text in detail and to read other related texts so you can present a fair
and reasonable evaluation of the selected text.
What is Meant by Critical?
• To be critical does not mean to criticize in a negative
manner.
• Rather, it requires you to question the information and
opinions in a text and present your evaluation or
judgement of the text
• Decide the strengths and weaknesses of a text
• Evaluating requires an understanding of not just
the content of the text, but also an understanding
of a text’s purpose, the intended audience, and
why it is structured the way it is.
Structure of a Critical Review

• Can be in a short (one page) or in a long form (four pages). Usually have similar
structure:
• Introduction:
• Opening sentences that announce the author(s) and the title, and
briefly explain the topic of the text.
• Summary
• Present a summary of the key points along with a limited number of
examples.
• Critique
• Balanced discussion and evaluation of the strengths, weakness and
notable features of the text.
• Conclusion and References
• Restate your overall opinion of the text and briefly present
recommendations.
Structure of a Critical Review - Introduction

Opening sentences that announce the author(s) and the title, and briefly
explain the topic of the text.

Present the aim of the text and summarize the main finding or key
argument.

Conclude the introduction with a brief statement of your evaluation of the

text. This can be a positive or negative evaluation or, as is usually the case,
a mixed response.
Structure of a Critical Review - Summary
Present a summary of the key points along with a limited number of
examples.

Briefly explain the author’s purpose/intentions throughout the text

and briefly describe how the text is organised.

The summary should only make up about a third of the critical

review.
Structure of a Critical
Review - Critique
• Balanced discussion and evaluation of the strengths, weakness
and notable features of the text. Remember to base your
discussion on specific criteria. Good reviews also include other
sources to support your evaluation (remember to reference).
• You can choose how to sequence your critique. Here are some
examples to get you started:
• Most important to least important conclusions you make
about the text.
• If your critique is more positive than negative, then
present the negative points first and the positive last.
• If your critique is more negative than positive, then
present the positive points first and the negative last.
Structure of a Critical Review - Critique
• You can choose how to sequence your critique. Here are some examples to get you started:
• If there are both strengths and weakness for each criterion you use, you need to decide
overall what your judgement is. For example, you may want to comment on a key idea in the
text and have both positive and negative comments. You could begin by stating what is good
about the idea and then concede and explain how it is limited in some way. While this
example shows a mixed evaluation, overall you are probably being more negative than
positive.
• In long reviews, you can address each criterion you choose in a paragraph, including both
negative and positive points. For very short critical reviews (one page or less), where your
comments will be briefer, include a paragraph of positive aspects and another of negative.
• You can also include recommendations for how the text can be improved in terms of ideas,
research approach; theories or frameworks used can also be included in the critique section.
Structure of a Critical Review
• Conclusion and References
• Conclusion
• This is usually a very short paragraph.
• Restate your overall opinion of the text.
• Briefly present recommendations.
• If necessary, some further qualification or explanation of your judgement can be
included. This can help your critique sound fair and reasonable.
• References
• If you have used other sources in you review you should also include a list of references
at the end of the review.
Summarizing and Paraphrasing
• Summarizing and paraphrasing are essential skills for academic writing and in particular, the critical
review.
• To summarize means to reduce a text to its main points and its most important ideas. The length of
your summary for a critical review should only be about one quarter to one third of the whole critical
review.
• The best way to summarise
1. Scan the text. Look for information that can be deduced from the introduction, conclusion, title, and headings.
What do these tell you about the main points of the article?
2. Locate the topic sentences and highlight the main points as you read.
3. Reread the text and make separate notes of the main points. Examples and evidence do not need to be
included at this stage. Usually they are used selectively in your critique.
4. Paraphrasing means putting it into your own words. Paraphrasing offers an alternative to using direct
quotations in your summary (and the critique) and can be an efficient way to integrate your summary notes.
Summarizing and Paraphrasing
• The best way to paraphrase
1. Review your summary notes
2. Rewrite them in your own words and in complete sentences
3. Use reporting verbs and phrases, e.g. 'The author describes…', 'Smith argues
that …'.
4. Use quotation marks if If you include unique or specialist phrases from the
text.
Some General Criteria for Evaluating Texts
• The following list of criteria and focus questions may be useful
for reading the text and for preparing the critical review.
1. Significance and contribution to the field
2. Methodology or approach
3. Argument and use of evidence
4. Writing style and text structure
Significance and contribution to the field
o What is the author's aim? To what extent has this aim been
achieved?
o What does this text add to the body of knowledge (contribution to
knowledge)? This could be in terms of theory, data and/or practical
application
o What relationship does it bear to other works in the field?
o What is missing/not stated?
o Is this a problem?
Methodology or Approach
o What approach was used for the research? For example, quantitative
or qualitative, analysis/review of theory or current practice,
comparative, case study, personal reflection, etc..
o How objective/biased is the approach?
o Are the results valid and reliable?
o What analytical framework is used to discuss the results?
Argument and Use of Evidence
o Is there a clear problem, statement or hypothesis?
o What claims are made? Is the argument consistent?
o What kinds of evidence does the text rely on?
o How valid and reliable is the evidence? How effective is the evidence
in supporting the argument?
o What conclusions are drawn? Are these conclusions justified?
Writing Style and Text Structure
o Does the writing style suit the intended audience? For example,
expert/non-expert, academic/non-academic, etc.
o What is the organizing principle of the text? Could it be better
organized?

Example: https://www.student.unsw.edu.au/sample-extracts
Systematic Literature Review

84
Process of Literature Review
• Involves 3 major stages:
• Planning the review,
• Conducting the review, and
• Reporting the review
• Stage 1 and 2 are iterative
Planning the Review
• Formulate the Problem
• SLR is conducted to answer the research
questions
• The selection of studies to be included in the
review, methodology for data extraction and
synthesis, and reporting, should all be geared
toward answering the research questions
• Identifying the appropriate research question
can be an iterative process
• using prereview mapping to help identify subtopics
within a proposed research question
Planning the Review
• Develop and Validate the Review Protocol
• is a preset plan that specifies the methods utilized in
conducting the review
• Necessary for enhancing the quality of review
because it reduces the possibility of researcher bias
in data selection and analysis
• Increases the reliability of the review because others
can use the same protocol to repeat the study for
cross-check and verification
• The review protocol should describe all the elements
of the review, including:
• a. the purpose of the study, b. research questions, c.
inclusion criteria, d. search strategies, e. quality
assessment criteria and screening procedures, f.
strategies for data extraction, g. synthesis, and h.
reporting
Conducting the Review
• Search the Literature
• The quality of literature review is highly dependent on the
literature collected for the review ("Garbage-in, garbage-
out")
• Channels for literature search.
• Major sources: (1) electronic databases; (2) backward searching;
and (3) forward searching
• Electronic databases constitute the predominant source of
published literature collections
• Web of Science, EBSCO, ProQuest, IEEE Xplore
• Google Scholar is a very powerful open access database that
archives journal articles as well as “gray literature,” such as
conference proceedings, thesis, and reports
• Backward search to identify relevant work cited by the articles
• Using the list of references at the end of the article
• Forward search to find all articles that have since cited the
articles reviewed
• Google Scholar and the ISI Citation Index allow forward search of
articles
Conducting the Review
• Search the Literature
• Keywords used for the search.
• The keywords for the search should be derived from the
research question(s).
• e.g. RQ: “what factors affect business continuity after a natural
disaster?” The domains are "business", "continuity", and "natural
disaster".
• Take the concepts in the search statement and extend them by
synonyms, abbreviations, alternative spellings, and related
terms
• The synonyms of "business" can be "enterprise" and "firm".
• The terms related to "continuity" are "impact", "recovery", and
"resilience" / "resiliency".
• "Natural disaster" can be further broken down into "flood",
"hurricane", "earthquake", "drought", "hail", "tornado", etc.
• Use of Boolean operators in the search ("AND" and "OR")
• ("business" OR "firm" OR "enterprise") AND ("continuity" OR
"impact" OR "recovery" OR "resilience" OR "resiliency") AND
("natural disaster")
Conducting the Review
• Search the Literature
• Keywords used for the search.
• The keywords for the search should be derived from the
research question(s).
• e.g. RQ: “what factors affect business continuity after a natural
disaster?” The domains are "business", "continuity", and "natural
disaster".
• Take the concepts in the search statement and extend them by
synonyms, abbreviations, alternative spellings, and related
terms
• The synonyms of "business" can be "enterprise" and "firm".
• The terms related to "continuity" are "impact", "recovery", and
"resilience" / "resiliency".
• "Natural disaster" can be further broken down into "flood",
"hurricane", "earthquake", "drought", "hail", "tornado", etc.
• Use of Boolean operators in the search ("AND" and "OR")
• ("business" OR "firm" OR "enterprise") AND ("continuity" OR
"impact" OR "recovery" OR "resilience" OR "resiliency") AND
("natural disaster")
Conducting the Review
• Search the Literature
• Things to consider when selecting the correct
keywords.
• First, researchers should strike a balance between the
degree of exhaustiveness and precision
• Second, researchers doing cross-country studies
should pay attention to the cultural difference in
terminology
• Third, notice the evolving vocabulary
• Fourth, to know whether the keywords can perform
sufficiency
• Last but not least, it is very important to document
the date of search, the search string, and the
procedure
Conducting the Review
• Search the Literature
• Sampling Strategies
• All literature searches are guided by some kind of
sampling logic and search strategies adopted by the
reviewers.
• The sampling and search strategies differ across various
types of literature reviews.
• Depending on the purpose of the review, the search can
be exhaustive and comprehensive or selective and
representative (e.g. gray literatures can be
included/excluded)
• Refining results with additional restrictions.
• Other practical criteria might include the publication
language, date range of publication, and source of
financial support
• Stopping rule.
• stop when repeated searches result in the same
references with no new results
Conducting the Review
• Screen for inclusion
• Used to decide whether it should be included for
data extraction and analysis
• Follow a two-stage procedure: a. start with a coarse
sieve through the articles for inclusion based on the
review of abstracts, b. followed by a refined quality
assessment based on a full-text review
• Criteria for inclusion/exclusion.
• Establish inclusion and exclusion criteria based on the
research question(s). Any studies unrelated to the
research questions(s) should be excluded.
• Criteria should be practical
• the criteria should be capable of classifying research, can
be reliably interpreted, and can result in the amount of
literature manageable for the review.
Conducting the Review
• Screen for inclusion
• Can be based on research design and methodology. For
instance:
• studies may be restricted to those carried out in certain
geographic areas (e.g., developed vs. developing countries),
• of certain unit of analyses (e.g., individual business vs. the
aggregate economy; individual household vs. the entire
community),
• Screening procedure.
• Many suggest at least two reviewers work independently to
appraise the studies matching the established review inclusion
and exclusion criteria
• Appraisal is commonly based on the abstracts of the studies,
could also read the conclusion
• Two reviewers should resolve the disagreement through
discussion or by a third party
• if in doubt, include the studies for further examination
• Finally, the list of excluded papers should be maintained for
record keeping, reproducibility, and crosschecking
Conducting the Review
• Assess quality
• for understating each study before proceeding to
the steps of comparing and integrating findings
• Quality standards differ across various types of
reviews.
• For example, quality assessment is not crucial for
some types of descriptive reviews and critical reviews.
• However, quality assessment is important for reviews
aiming for generalization, such as testing reviews.
• Quality assessment does not necessarily need to
be used as a yes-or-no cutoff, but rather serve as
a tool for reviewers to be aware of and
acknowledge differences in study quality
Conducting the Review
• Assess quality
• Criteria for quality assessment
• use recognized quality assessment tools, for example,
checklists, to evaluate research studies
• ranking result can be used in two ways:
• a. "weight" the study qualitatively by placing studies
into high, medium, and low categories
• b. "weight" each study quantitatively
• Quality assessment procedure
• Similar to the inclusion screening process,
recommended that two or more researchers perform
a parallel independent quality assessment, resolve
disagreement
Conducting the Review
• Extract data
• Often involve coding, especially for extending
reviews.
• Working in a team, need to code a few papers
together before splitting the task to ensure
everyone is on the same page and coding the
papers similarly
• important for researchers to review the entire
paper
Conducting the Review
• Analyze and synthesize data
• organize the data according to the review they have
chosen.
• combination of charts, tables, and a textual description
• Mixed method to combine qualitative and
quantitative study:
• Segregated design involves synthesizing qualitative and
quantitative studies separately according to their
respective synthesis traditions and textually combining
both results
• An integrated design, by contrast, analyzes and
synthesizes quantitative and qualitative research
together
• Contingent design is characterized by being a cycle of
research synthesis—a group of qualitative or quantitative
studies is used to answer one specific research question
(or sub-research question) and then those results will
inform the creation of another research question to be
analyzed by a separate group of studies
Conducting the Review
SLR Example
• Putrada, A. G., Abdurohman, M., Perdana, D., & Nuha, H. H.
(2022). Machine Learning Methods in Smart Lighting Towards
Achieving User Comfort: A Survey. IEEE Access.
• van Schaik, S. N., Masthoff, J., & Wibowo, A. T. (2019).
Package recommender systems: A systematic review. Intelligent
Decision Technologies, 13(4), 435-452.
Terima Kasih dan Mohon Maaf
101
Pustaka dan Referensi
• XIAO, Yu; WATSON, Maria. Guidance on conducting a
systematic literature review. Journal of planning education and
research, 2019, 39.1: 93-112.
• PARÉ, Guy, et al. Synthesizing information systems knowledge:
A typology of literature reviews. Information & Management,
2015, 52.2: 183-199.