Paper Code: JRSI-2.

3-A17
1. The research novelty: Rate the novelty and originality of the ideas or results presented
in the paper.
a. A pioneering piece of work. Striking novel ideas or results.
b. Significant original work and novel results.
c. Some interesting ideas and results on a subject well investigated.
d. Minor variations on a well investigated subject.
e. It has been said many times before.

2. Research methods: Research methods appropriateness and evidence adequacy (the

relation between method and problem).
a. Research method is easily identified and is appropriate to address the problem.
b. Research method is not easily identified, but only partially appropriate to
address the problem.
c. Research method is not clearly identified, but inappropriateness is not evident.
d. Research method does not answer the problem.
e. Research method cannot be identified.

3. Future Impact: Future impact on related subject of the research/society.

a. Future impact is clearly defined, and is relevant to varied research/ society.
b. Future impact is defined but not very clear; the likely impact is relevant to
some research/ society.
c. Future impact is not explicitly defined; the likely impact is limited.
d. Future impact is not defined; the likely impact is questionable.
e. Future impact cannot defined.

4. The Clarity of Research Question: The clarity of research question/ problem and
relevance of the literature overview.
a. Problem is easily identified in the introductory section, and clearly related to
literature review.
b. Problem can be identified, and related to literature review.
c. Problem is incompletely reported, and the relevance to literature review is not
clear.
d. Problem is incomprehensible, and literature review is vague.
e. Problem is not reported, and literature is poorly reviewed.

5. Scientific argument and discussion: - The strength of argument and discussion in

connecting research finding with original research question. – The summary and
discussions (indicating to what degree the problem has been solved).
a. Excellent discussion with good amount of evidence to support conclusion.
b. Good discussion with adequate evidence to support conclusion.
c. Standard arguments that only partially support the conclusion.
d. Conclusions are not supported by decent discussion.
e. Conclusions and/or discussions are absent.
6. Generally, is that the paper worth to be published?
a. Yes
b. No

7. Detailed Comment: If answer No. 6 is “Yes”, please justify your recommendation and
suggest improvements in technical content or presentation.
Permasalahan/research question tidak jelas, kenapa solusi yang ditawarkan adalah
crowdsourcing dan mengapa metode yang dipilih adalah extreme programming?
Metode penelitian tidak jelas, dimana letak extreme programmingnya?
Apakah ada kecocokan antara masalah (WO) dengan metode yang dipilih/xp?
Kesimpulan yang diperoleh, terlalu tiba-tiba, tanpa dijelaskan darimana kesimpulan
tersebut diperoleh. Kesimpulan tidak menjawab permasalahan, karena
permasalahannya tidak jelas didefisikan.
Perlu ditambahkan survey terhadap beberapa aplikasi sejenis, lalu dimasukkan
sebagai dasar requirement. Tunjukkan keunggulan aplikasi yang dibangun (atau
prospek) dibandingkan dengan aplikasi yang sudah ada.
Testing yang dilakukan tidak menjadi poin dalam kesimpulan. Testing yang
dilakukan juga tidak terkait dengan persoalan. Apalah memang testing ini perlu
dilakukan dalam rangka menjadi usulan solusi permasalahan? Perlu dicantumkan
secara eksplisit permasalahan apa yang membuat penelitian ini memerlukan testing
dan hasilnya memang menjawab permasalahan tersebut.
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________

8. Detailed Comment: If answer No. 6 is “No”, please justify your reason and suggest
improvements in technical content or presentation to the next paper.
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________