Dr Yogesh K.

Dwivedi
Professor of Digital Marketing & Innovation
Director of Emerging Market Research Centre
Co-Director of Research, School of Management
Swansea University, Wales, UK
Email: y.k.dwivedi@Swansea.ac.uk; ykdwivedi@gmail.com
Editor-in-Chief: International Journal of Information Management
Series Editor: Springer Series on Advances in Theory and Practice of Emerging Markets
Associate Editor: European Journal of Marketing
Associate Editor: Government Information Quarterly
Senior Editor: Journal of Electronic Commerce Research
• Outline…
– Expectations from a doctoral work
– General structure of the thesis (a brief overview)
– Developing a publication strategy
– Publishing scholarly research: A reflective guidance
– Theoretical contributions
– Reviewing
 Completing a PhD in Business and Management:
A Guide to Doctoral Students

Recommended Reading:
Dwivedi, Y.K., Ravishankar, M.N. and Simintiras,
A.C. (2015), Completing a PhD in business and
management. Journal of Enterprise Information
Management, 28 (5), 615 – 621.
 Background
• Despite the ever-growing number of PhD students all over the
world, there remain significant doubts about whether:
– entering students in business and management disciplines fully understand
the process of producing a PhD thesis,
– defending it and developing a coherent publication strategy.
• In many cases the root of the problems are systemic such as:
– the lack of adequate research infrastructure and processes,
– availability of capable research supervision and
– the broader culture of the school/university.
• The aim of this talk is to offer some reflections and guidance on
what it takes to successfully complete a doctoral research thesis
– based on my own supervision and doctoral examination experiences.
 Doctoral Supervision and
Examination
• 23 Successful completions
– E-Gov; Innovation Adoption; Business/IT Alignment; HRIS; KMS
in Public Sector; E-Business Assimilation in SMEs; Self-Service
Technologies; Mobile Government; Mobile Internet; Social
Commerce; IS/IT Project Success and Failure; Gamification in
Marketing
• 75 Examinations - UK & Abroad
• This talk will focus on three key issues:
– choosing a research problem;
– demonstrating rigour and quality; and
– developing a publication strategy.
 Choosing a Research Problem:
Novelty/Originality

• One of the most critical aspects of doctoral research is making sure

that the research focuses on a current and non-trivial problem.
– Selected research problem should not have been investigated earlier either in
a PhD project or in a journal (or conference) article.

– A doctoral thesis is expected to provide clear evidence of a research

contribution (i.e. it must add something new) to existing knowledge.

• If an issue has already been investigated in great depth (e.g. in a

PhD thesis or in research articles), then it would be hard to
demonstrate an original research contribution.
 Choosing a Research Problem:
Novelty/Originality
How can one find out if a particular research problem has been
investigated earlier?
• A thorough review of the extant literature is called for.
– A good approach would be to use the electronic library resources and
databases to familiarise oneself with the history of a research topic, its
development and the recent conversations amongst scholars in the field.
• A common mistake committed by a PhD student is to assume
that any problem he/she arrives at originally (but without
referring to much scholarly literature) is a worthwhile PhD
project.
• Would it be appropriate to have the following as the main
research question in a PhD project:
– Why do companies offshore IT work?
 Choosing a Research Problem:
Relevance
• Another point to be carefully considered is the relevance of the
research problem to practice or/and to the wider society.
• Some research problems can be novel but its relevance may be
unclear or unspecified in the thesis.
– This could annoy examiners (and journal reviewers).
– This can be achieved by doing a thorough review of the literature &
exploring current issues of significance for business or society.
• Research problem should meet both: novelty and relevance.
• Novel does not always refer to “blue-sky thinking”:
– A much studied topic could be novel as long as it can be differentiated
based on context, data types, source, theoretical perspective, etc.
Demonstrating Rigour & Quality:
Literature Review
• Literature review is one of the core elements and generally
forms the foundation for a research project.
– Its rigour is normally judged by its comprehensiveness, i.e.,
whether the review systematically identifies and synthesizes all
the scholarly literature relevant to the research topic.
– Author-centric approach to structure the literature review is a
poor approach
• It generates a long descriptive list of who has said what without
any deeper analysis and synthesis.
– Instead, the literature review chapters of a PhD thesis should
ideally be organised around ideas and concepts
• for further detail see Webster and Watson (2002)
Demonstrating Rigour & Quality:
Literature Review
• Not all universities subscribe to a wide-range of scholarly journals
– This clearly impacts on the quality of the literature review
• Examiners/reviewers are now very unlikely to look at a poor
literature review sympathetically on the grounds that a student had
no reasonable way of getting his/her hands on published work
• Keyword searching on electronic databases is the preferred method
• One should frequently access the web sites of the important journals
in relevant field and read published articles therein
– This will provide students an insight into the broader intellectual
conversations going on in the discipline
– May also help you to serendipitously identify a potentially relevant
theoretical slant or methodological approach that might never have
been found through keyword searching alone
Demonstrating Rigour & Quality:
Theoretical Perspective
• Awareness and understanding of a range of appropriate
theoretical perspectives is critical
– Examiners would like to see this demonstrated in a doctoral thesis.
• While one might focus on a particular theory/model in a thesis, it
is also important to show awareness of competing theories and
models.
– What are the strengths and weaknesses of other theories and models?
– How have they been used thus far in the literature?
– It is also crucial that a thesis logically and rationally justifies the choice of a
particular theory/model.
• Many PhD theses merely assert and claim that their theoretical
perspective is appropriate without seriously comparing and
contrasting it with other related approaches.
Demonstrating Rigour & Quality:
Methodology
• The thesis should openly and clearly discuss:

– the ontological and epistemological assumptions underpinning the

thesis

• It should also illustrate a general awareness:

– of the most often used research methods and

– show an in-depth understanding of the particular method adopted in
the thesis
• A detailed description of the research context and data collection
procedures is always appreciated by examiners.
 Demonstrating Rigour & Quality:
Methodology
• Data analysis is another aspect that requires careful crafting
– often one comes across data analysis sections where there is very
little explanation of how the data were actually analysed
– all one gets to see is a brief statement such as “data were analysed
using X software”.

• Care must be taken to ensure that the steps followed in analysing

the data are reported in as much detail as possible
• Many PhD students find this a very difficult section of the thesis to
write.
– A well-written, clearly argued and credible data analysis section often
creates a very good impression about the thesis (and the student) in
the examiners’ minds
Demonstrating Rigour & Quality:
Discussion
• A thesis is incomplete without a strong discussion chapter
– Many students run out of steam by the time they start the discussion
chapter.
– Although labelled “discussion”, this chapter often ends up summarizing the
results and doing very little else.
– In some other problematic cases, discussion chapters only detail the
practical contributions, limitations and suggest avenues for future research.
• It is very hard to evaluate the contribution of a PhD thesis if no
effort is made to explain and discuss the findings in the light of
prior literature.
– Discussion should tease out some abstract principles and contribute to both
the phenomenon studied and the theory(ies) used.
– A good discussion must clearly highlight the theoretical contributions of the
study.
Demonstrating Rigour & Quality:
Other Issues
• Examiners may also put off by:
– inconsistent citation and referencing styles,
– grammar and word-choice issues,
– formatting issues and
– disjointed writing.
• It is very important to follow a consistent citation and referencing
style throughout the thesis.
• Very often one comes across numerous typological, spelling and
grammatical errors in a submitted thesis which give examiners a
bad impression about the quality of the thesis.
• It is sensible to take the help of a proofreading or copyediting
service before submission.
 General Structure of a
Doctoral Thesis
• Chapter 1: Introduction
– Defining research problem and motivation, research aim and objectives,
scope, approach, potential contributions, chapter outlines
• Chapter 2: Literature Review
– Analysis and synthesis of existing literature, identification and discussion of
research gaps
• Chapter 3: Theoretical Basis and Hypotheses Development
– Identification and discussion of alternative theories/models relevant for
problem domain, selection and justification of appropriate one, limitations
of selected theory/model and way to address them, overview of proposed
model, hypotheses development
• Chapter 4: Methodology
– Philosophy (Ontology & Epistemology), methodology, methods, instrument
development and validation, analysis techniques and tools, ethical
considerations
General Structure of a
Doctoral Thesis
• Chapter 5: Results
– Missing data, normality, common method bias, reliability test,
descriptive statistics, measurement model/EFA, structural
model/regression analysis
• Chapter 6: Discussion
– Reliability and validity, hypotheses, model performance, theoretical
contribution, Implications to practice and/or Policy, Limitations and
future research direction
• Chapter 7: Conclusions
– A brief overview of chapters, key conclusions, concluding
thoughts/remarks
• References
• Appendices
 Developing a Publication
Strategy
• While the definition of what constitutes a “good” scholarly journal
varies from country to country, the increasing pressure to publish
is most keenly felt by PhD students in the academic job market.
– It is therefore crucial to develop a publication strategy while working
on their PhD thesis.
• To find an academic job in a research-intensive university, it is
advisable that at least one paper is under review in a high quality
peer-reviewed journal by the thesis submission time.
– This demonstrates the student’s potential to the short-listing panels
and signals ambition.
• Peer-reviewed conference papers too are useful in that they show
recruiters that a student has experience of writing research
articles for a scholarly audience.
 Developing a Publication
Strategy
• However, an excessive focus on publications could take the focus
away from completing and submitting one’s PhD thesis on time.
– This is a very difficult balance to strike for many PhD students, especially in
countries like the UK where full-time PhD students are normally expected to
submit their PhD thesis in less than four years.
• In choosing between the two, students are well-advised to follow
a path that is well-aligned with their personality and their career
objectives.
– One should devise a publication strategy that takes into account the wider
institutional context and expectations.
• Taking supervisors into confidence can help plan a realistic and
achievable publication strategy.
 Summary
• The chosen research problem should be original and relevant.
• A doctoral thesis must illustrate rigour in all aspects of the work
but particularly:
– in undertaking literature review,
– selecting and justifying theories and/or theoretical constructs,
– critical instance on selecting, justifying and applying research method and
analysis techniques,
– discussing research results, clearly highlighting both theoretical and practical
contributions and
– drawing careful conclusions.
• Research publication is vital for both successful completion of a
PhD and building a successful academic career.
– It is therefore crucial for PhD students to develop a publication strategy.
– However, a caution must be taken not to give excessive focus on publications
as that could take the focus away from submitting one’s PhD thesis on time.
Thank you!

Questions?
Publishing and Reviewing Scholarly Research:
A Reflective Guidance
Dr Yogesh K. Dwivedi
Professor of Digital Marketing & Innovation
Director of Emerging Market Research Centre
Co-Director of Research, School of Management
Swansea University, Wales, UK
Email: y.k.dwivedi@Swansea.ac.uk; ykdwivedi@gmail.com
Editor-in-Chief: International Journal of Information Management
Series Editor: Springer Series on Advances in Theory and Practice of Emerging Markets
Associate Editor: European Journal of Marketing
Associate Editor: Government Information Quarterly
Senior Editor: Journal of Electronic Commerce Research
Background
• Publish or perish
– High quality peer-reviewed articles have become central to career
progression
• National research quality assessments
– Research Excellence Framework (REF)
– The data feeds into various league tables that influence students’ choice of
institutions
• Publishing in high quality journals is:
– Challenging
– Effort intensive
– At times, emotionally draining
• This talk aims to offer some reflections on issues relevant for successfully
publishing rigorous and relevant scholarly research in high quality
journals
Selecting an Appropriate Journal
Avoid predatory journals (http://thinkchecksubmit.org/)
• Do you or your colleagues know the journal?
• Can you easily identify and contact the publisher?
• Is the journal clear about the type of peer review it uses?
• Are articles indexed in services that you use?
• Is it clear what fees will be charged?
• Do you recognise the editorial board?
– Have you heard of the editorial board members?
– Do the editorial board mention the journal on their own
websites?
• Is the publisher a member of a recognized industry initiative?
– Committee on Publication Ethics (COPE); Directory of Open Access Journals
(DOAJ) ; Open Access Scholarly Publishers’ Association (OASPA)
 General Structure of Survey Based
Positivist Submissions
• Consider journal style and requirements
Title
Abstract
Keywords:
1. Introduction
2. Literature Review
3. Theoretical background and hypotheses development
4. Methodology
5. Results
6. Discussion
This section should first provide a synthesis of results with existing work published on this topic
and then insert the following subsections
6.1 Theoretical contributions
6.2 Implications for practice (at least one page)
6.3 Limitations and Future Research Direction
7. Conclusions
This section should provide at least a para outlining key points emerging this
research
References
 Abstract
• A short summary of the complete content
• 150 to 300 words
• Should include:
– Research questions examined and reasons for examining it (why it was
considered important and relevant)
– How research was conducted?
– Key results
– Key conclusions and contributions
• Structured vs non-structured abstract
• Write an initial draft at start but rewrite it at the end
 Introduction
• Avoid presenting full literature review in it
• It’s uncommon to see subsections, figures and tables in this section
• Should not be more than two pages in length
• Introduction should succinctly:
– introduce/define/describe the core topic/concepts
– highlight the importance of the research topic
– define the research problem – should provide facts and figures to illustrate the
relevance of the topic
– articulate and outline the research problem
– state the research aims/objectives and/or research questions
– provide few lines of overview of the remaining sections

• Many submissions lack full problem definition and/or equate that with
research gap
– In order to succeed, a topic must be timely, relevant and novel/original
 Literature Review
• It is not uncommon to confuse literature review with theory development
• It is essential to provide clear and succinct review of relevant literature
• Many submissions preclude this section, which is generally not very well
taken by reviewers, AEs and Editors
• Author centric vs. concept centric review
• This section should end by clearly articulating research gap that was
specified in the introduction section
• It is important to include relevant literature from the target journals
• It is always helpful to present a table to summarise key issues emerging
from the literature along with relevant citations
 Literature Review:
Webster & Watson (2002)
Theoretical Background & Hypotheses
Development
• It is not good practice to just select a theory without exploring suitability of
alternatives choices
– Explore suitable alternatives (for example, TAM, UTAUT, UTAUT2, TPB, TRA)
– Provide suitable justification for selecting a specific theory or model - for example, why
it was considered appropriate to select TAM, UTAUT or TPB over other alternatives
• Hypotheses development
– Should not have too many hypotheses
– Hypothesis should not be presented without providing surrounding discussion to justify
proposed relationships
– Should define the constructs and discuss their importance, how that has been examined
in the previous literature, why that is relevant to your research problem, and how your
context is similar or different from previous studies, which then determines if the
nature of a hypothesis should be similar or different from previous studies
– Don’t present more than one hypothesis based on same theoretical discussion
– Should not try to measure more than one relationships using one hypothesis
 Methodology
• Should start by selecting and justifying method(s) considered
appropriate for undertaking the research in question
• Sample, sampling technique, sample size, data collection
tools/techniques, time for data collection
• Instrument/measurements – source, any modifications made and
any testing (pre or pilot) conducted
• Ethical issues should be briefly outlined
• Analysis tools and techniques – why SEM over PLS or regression?
 Results

• Respondents’ profile/demographic information

• Descriptive statistics and reliability
• Common method bias
• Measurement and structural models
– Model fit indices – use more than one
• χ²/df; CFI; GFI/AGFI; PNFI; RMSEA
• Significance and variance explained
 Discussion

• Synthesis of results with existing literature

– Comment on reliability and validity of method applied
– Discuss significant/non-significant relationships
– Examine and rationalise similarities and differences with the literature
– Comment on model performance
• Contribution to theory and/or literature
• Implications for practice
• Limitations and future research directions
 Conclusion
• Present key points and not the summary of different sections
– A paragraph consisting of salient points would be sufficient
NB
• Many submissions mix conclusion section with discussion
section
– This is not good practice and is generally not very well taken by the
reviewers and editorial team
• Avoid duplication of text between abstract, introduction and
conclusion
 General
Recommendations

• Resist the temptation to submit before it is ready –

papers submitted in a hurry almost always come back as
a reject
• Seek internal (critical) feedback as well as comments at
conferences and workshops
• ‘Readability’ of the manuscript is crucial in improving
your chances - reviewers and editors are generally
unsympathetic to poorly written papers
 General
Recommendations
• Most journals insist on ‘theoretical contributions’
• What is your ‘claim – reason – evidence’?
• Editors ask themselves two questions – is this a good quality
paper (rigour)? Is the paper going to be of interest to the
journal’s audience (relevance)?
 General
Recommendations

• Please resubmit when you get a R & R decision!

• Reviewers are always right – do not ignore even a single
comment of the review team
• If your papers have never been rejected, it means you are
aiming too low and you are under-selling your work
• When rejected don’t submit the same paper without revising
anything to a different journal!
 General
Recommendations
(Lyytinen et al. 2007, p. 320, EJIS)

• Engage and participate in reviewing pools and networks

• Collaborate with scholars who have published in, or edit, top-level
journals
• Attend elite journal workshops on publishing
• Follow systematically and actively research streams in top journals
• Improve method and analysis skills
• Improve writing skills and understanding of article genres
 References
• Consistent formatting
• Complete information – volume, issue, page numbers
• All citations should appear in reference list and vice versa
• Check references are current and not very old – at least some
references must be recent ones as AE/Editor often use
reference list to identify reviewers
• Some references should be from the target journal to
illustrate fit of submission
• Look at journal formatting guidelines
Recommended Readings:
• Lee, N., & Greenley, G. (2009). What makes a good article?
Generating an insightful manuscript. European Journal of
Marketing, 43(5/6), 577-582.
• Fawcett, S. E., Waller, M. A., Miller, J. W., Schwieterman, M. A.,
Hazen, B. T., & Overstreet, R. E. (2014). A trail guide to publishing
success: tips on writing influential conceptual, qualitative, and
survey research. Journal of Business Logistics, 35(1), 1-16.
• Webster, J., & Watson, R. T. (2002). Analyzing the past to prepare
for the future: Writing a literature review. MIS quarterly, xiii-xxiii.
• Whetten, D. A. (1989). What constitutes a theoretical contribution?.
Academy of management review, 14(4), 490-495.
Thank you!

Questions?
 Contribution to Theory
• Recommended Readings:
– Boer, H., Holweg, M., Kilduff, M., Pagell, M., Schmenner, R., & Voss, C.
(2015). Making a meaningful contribution to theory. International
Journal of Operations & Production Management, 35(9), 1231-1252.
– Corley, K. G., & Gioia, D. A. (2011). Building theory about theory
building: what constitutes a theoretical contribution?. Academy of
management review, 36(1), 12-32.
– Lee, N., & Greenley, G. (2008). The primacy of theory. European Journal
of Marketing, 42(9/10), 873-878.
– Meyer, K. E. (2015). Context in management research in emerging
economies. Management and Organization Review, 11(3), 369-377.
– Shepherd, D. A., & Suddaby, R. (2017). Theory building: A review and
integration. Journal of Management, 43(1), 59-86.
– Whetten, D. A. (1989). What constitutes a theoretical
contribution?. Academy of management review, 14(4), 490-495.
 What Are the Building Blocks of
Theory Development?
(Source: Whetten, 1989)
A complete theory must contain four essential element
• What
– Which factors (variables, constructs, concepts) logically should be considered as part of the
explanation of the social or individual phenomena of interest?
• How
– How are factors related? This involves using arrows to connect the boxes.
• Why
– Underlying reasons to justify the selection of factors and the proposed casual relationships
– This rationale constitutes the theory's assumptions - the theoretical glue that welds the model
together
– Why should colleagues give credence to this particular representation of the phenomena?
• Who, where, when
– These conditions place limitations on the propositions generated from a theoretical model.
– These temporal and contextual factors set the boundaries of generalizability, and as such
constitute the range of the theory
What Are the Building Blocks of Theory
Development?
(Source: Whetten, 1989)
• What and How describe; only Why explains.
– What and How provide a framework for interpreting patterns, or discrepancies,
in empirical observations.
– This is an important distinction because data, whether qualitative or
quantitative, characterize; theory supplies the explanation for the
characteristics.
– Therefore, we must make sure that what is passing as good theory includes a
plausible, cogent explanation for why we should expect certain relationships in
our data.
• Together these three elements provide the essential ingredients of a
simple theory: description and explanation.
• Although it is important for theorists to be sensitive to context, the
Who, Where, and When of a theory are typically discovered through
subsequent tests of the initial, rudimentary theoretical statement
(What, How, Why).
 What Is a Legitimate, Value-Added
Contribution to Theory Development?
(Source: Whetten, 1989)
• What and How
– Although, in principle, it is possible to make an important theoretical contribution by
simply adding or subtracting factors (Whats) from an existing model, this process
seldom satisfies reviewers.
– The additions or deletions typically proposed are not of sufficient magnitude to
substantially alter the core logic of the existing model.
– One way to demonstrate the value of a proposed change in a list of factors is to
identify how this change affects the accepted relationships between the variables
(Hows).
• Just as a list of variables does not constitute a theory, so the addition of a new
variable to an existing list should not be mistaken as a theoretical contribution.
– Relationships, not lists, are the domain of theory.
• "Science is facts, just as houses are made of stone.... But a pile of stones is not a
house, and a collection of facts is not necessarily science.“ (Poincare, 1983 as
quoted in Whetten, 1989)
 What Is a Legitimate, Value-Added
Contribution to Theory Development?
(Source: Whetten, 1989)
• Generally, it is insufficient to point out limitations in current conceptions
of a theory's range of application.
– For example, discovering that a mainstream personnel selection model has low
predictive validity in a military setting does not by itself constitute a theoretical
contribution.
– In addition, theorists need to understand why this anomaly exists, so that they
can revise the How and What of the model to accommodate this new
information.
• Applying an old model to a new setting and showing that it works as
expected is not instructive by itself.
– This conclusion has theoretical merit only if something about the new setting
suggests the theory shouldn't work under those conditions.
– In other words, it is preferable to investigate qualitative changes in the
boundaries of a theory (applications under qualitatively different conditions),
rather than mere quantitative expansions.
 What Is a Legitimate, Value-Added
Contribution to Theory Development?
(Source: Whetten, 1989)
• Proposed improvements addressing only a single element of an existing
theory are seldom judged to be sufficient.
– Therefore, a general rule of thumb is that critiques should focus on multiple
elements of the theory.
– This approach adds the qualities of completeness and thoroughness to
theoretical work
• Theoretical critiques should provide compelling evidence. This evidence
can be:
– logical (e.g., the theory is not internally consistent),
– empirical (its predictions are inconsistent with the data accumulated from
several studies), or
– epistemological (its assumptions are invalid- given information from another
field).
• In general, theoretical critiques should propose remedies or alternatives.
 References
• Consistent formatting
• Complete information – volume, issue, page numbers
• All citations should appear in reference list and vice versa
• Check references are current and not very old – at least some
references must be recent ones as AE/Editor often use
reference list to identify reviewers
• Some references should be from the target journal to
illustrate fit of submission
• Look at journal formatting guidelines
Thank you!

Questions?
Reviewing
• Recommended Readings:
– Eric Overby, PhD Seminar Module on How to Write a
Review, Georgia Institute of Technology, USA
http://www.prism.gatech.edu/~eoverby3/ReviewingMod
ule.html
– Lee, A. 1995. "Reviewing a Manuscript for Publication."
Journal of Operations Management 13(1) 87-92.
– Lee, N., & Greenley, G. (2009). Being a successful and
valuable peer reviewer. European Journal of
Marketing, 43(1/2), 5-10.
– Lepak, D. 2009. "Editor's Comments: What Is Good
Reviewing?" Academy of Management Review 34(3) 375-
381.
Benefits to Reviewers (Lee, 1995)

• Doing a review confers an insider’s view of the reviewing process

– The reactions of the other reviewers and the editor all contain potential
lessons for one’s own manuscripts to be submitted for publication
– Offers opportunity to gain access to invaluable bibliographies
• Reputation
– A track record of good reviews will enhance one’s reputation with editors
• Benefits to others
– Doing a review of a manuscript is a socially significant gesture.
• Benefits to one’s own school of thought
General Steps (Source: Eric Overby)

• Editor/Journal send you invite based on

- Your publication/citation, expertise, recommendation
• It is important to respond early - yes or no.
– You can say no but with a appropriate reason
• It is important to complete the review by the specified date.
– Timely submission of review is important
– Need more time – send email to editor/associate editor
• Generally all reviews are send to all reviewers involved
– Read the reviews from other reviewers and reflect on your comments,
those of the other reviewers, and those of the editor(s).
Structure and Elements of a Good
Review (Source: Eric Overby)

• An overall summary of the submission

• Outline strengths of the submission
– Topic, LR, theory building, method, results and where/how it does or
could make a contribution

• Overall judgement and recommendation

– Honest vs. polite
• Clearly outline concerns and major issues about the paper
 Concerns and Major Issues
(Source: Eric Overby)
– Contribution
• Does the paper push forward the knowledge frontier? Could it with
revision? I.e., are the authors "doing the right thing"?
– Alignment
• Does the paper accomplish what it says it will? Is there correspondence
between the motivation and what the authors do?
– Technical (empirics, modeling, logic, etc.)
• Are the data/measures appropriate, are the models/arguments correct,
is the evidence convincing, is the logic sound, etc. i.e., are the authors
"doing the thing right"?
– Be specific
• If necessary, include page numbers, passages, references to tables, etc.
 Concerns and Major Issues
(Source: Eric Overby)
• Indicate your level of certainty
– If you are not an expert, state that, and describe how/why you
don't understand.
• Provide suggestions, if you can
– But allow the authors some latitude so that they maintain control
over their paper.
• Be thorough
– If something is a big deal to you, don't leave it out. If the paper is
revised, you'll want the authors to have fixed that issue.
• If the paper is revised, subsequent reviews should follow the
same structure
– Try not to bring up new major points, unless changes to the paper
precipitate that.
Thank you!

Questions?