2.

Starting Research
 Outline
› Select topic
› Define Problem
› Formulate Objective
› Define Scope of the Study
› Define the Sigificance
› Methodology

2/17/2018
 Research Characteristics
› Originates with a question or problem in a specific area.
› Requires clear articulation of a goal or objective.
› Follows a specific plan or procedure (framework or model or
architecture).
› Often divides main problem into sub-problems to ease the
study.
› Guided by specific problem, question, or hypothesis.
› Accepts certain critical and significant assumptions and theory
during the research work.
› Requires preparation of datasets, running experiments,
analysis and interpretation of results

2/17/2018
 Planning checklist
› Pick a topic
› Identify the research question(s)
› Check the literature
› Identify your philosophical
stance
› Identify appropriate theories
› Choose the method(s)
› Design the study
– Unit of analysis?
– Target population?
– Sampling technique?
– Data collection techniques?
– Metrics for key variables?
– Handle confounding factors
› Critically appraise the design for
threats to validity\
› Get University/Institute approval
– Informed consent?
– Benefits outweigh risks?
› Recruit subjects / field sites
› Conduct the study
› Analyze the data
› Write up the results and publish
them
– Iterate

2/17/2018
 Finding Research Problem
› Don’t expect inspiration to strike, staring at a blank
piece of paper
– “Science is 99% perspiration and 1% inspiration”.
› Because it is largely the result of hard work, rather than an inspired flash of
insight.
› So…:
– Read the literature with a question in mind
› You must read journal articles and conference papers to find out what's
already known.
– Authors also often point out topics for future research.
– Talk to people - your project partner and supervisor - and
explain your ideas
– Write down your ideas in a working form

2/17/2018
 Selecting a Topic
› All research and writing begins with a topic
Theoretical Applicable
› First: Identify an area that suited to you or matters to you?
– Think: What is your research interest?
– Select a general topic that interests you and will maintain your
interest throughout the various stages of the study.
› List key words to help you find information about the topic.
› Some preliminary reading from encyclopedia, or other reference source, will
help to get an overview of the topic.
› Research is all about addressing an issue or asking and
answering a question or solving a problem, so…

2/17/2018
Selecting a Topic…
• Then, search for a problem: “Anything messy or difficult”
– Finding a good question/problem to address can actually be the hardest
part of the research.
• what expertise exists around you?
• What are your skills? Are you comfortable with programming?
– Using the general overview, begin to focus the topic into something you
can cover well.
• Keep in mind the budget & the expected length of the study
• Be original as much as you can, review as much literature as possible to avoid
duplication on a specific topic
• Sources of Research Problems
– Observation.
– Literature reviews.
– Professional conferences.
– Experts.
– Previous experience

2/17/2018
 Starting with research proposal
› Starting research proposal enables to:
– decide about the amount & depth of research required
› One of the Pitfalls in research is Easy to pick research
goals that are too ambitious
– Instead:
• Try to scope the research boundary such that durable
work can be done within the given time and cost
• Find out where the state of the art lies
• Look to the future work section of papers
• Can also redo bad work, properly

2/17/2018
 Start writing research proposal…

• Good research requires:

–The scope and delimitations of the work to be
clearly defined.
–The process to be clearly explained so that it can
be reproduced and verified by other
researchers.
–A thoroughly planned design that is as objective
as possible.

2/17/2018
 Research Proposal
Research proposals are a roadmap documents that describe the
intended research including:
› Background
› The Problem statement
 Research questions
› Objective of the study
› Scope of the study
› Methodology of the study
› Significance of the study
› Work Detail (work breakdown & timeline, Budget)
› References

2/17/2018
 Research Proposal: Preparation
• Ask yourself:
– Am I familiar with related research in this area
– Do I have a good understanding of the steps that will be involved in
achieving these goals
– Do I have the ability to successfully conduct each of these steps
– Am I sufficiently motivated and enthusiastic about all the steps in this
project
– Am I convinced that the results of this research will be useful to others
• At the end of research proposal preparation make sure that you
have answered the following questions:
– What you are planning to do ?
– Why ?
– What the difficulties are?
– Is it feasible for you ?
– Do you have a plan of how to do it ?
– Have you done your homework ?

2/17/2018
 Starting the Research Problem
› Once you’ve identified a
research problem:
– State the problem clearly and
completely with justification.
– Determine the feasibility of the
research.

• Identify sub-problems:
– Completely researchable units that are small in number.
– Add up to the total problem.
– Must be clearly tied to the interpretation of the datasets and
algorithms to be used.
• Formulate hypothesis or research questions
2/17/2018
 Research Questions
› A Research Question is a statement that identifies the phenomenon
to be studied.
– Should be stated as a question, e.g., "Is there a relationship between a person's
age and learning?"
› Consider the following research questions about a User Interface
design:
– Is the UI as good as or better than current practice?
– Which of several design alternatives is best?
– What are its performance limits and capabilities?
– What are its strengths and weaknesses?
– Does it work well for novices, for experts?
– How much practice is required to become proficient?
› A well-thought-out and focused research question leads directly into
specific, testable hypotheses.
– What predictions would you make about the phenomenon you are examining?
This will be the foundation of your application

2/17/2018
 Hypotheses
› Hypotheses are tentative, intelligent guesses as to the
solution of the problem. It is the statement we submit to
testing
– There is often a 1-1 correspondence between a subproblem and a
hypothesis.
– Hypotheses can direct later research activities since they can help
determine the nature of the research and methods applied.
› For example: Research Hypotheses
– often states there is a pattern, or difference, or trend among the
variables
› Null hypothesis is the opposite of the research hypothesis
– States there is no trend or difference

2/17/2018
 What type of question are you asking?
› Existence: › Relationship
– Does X exist – Are X and Y related?
– Do occurrences of X correlate with
› Description & Classificatio occurrences of Y?
– What is X like? – What are the steps as X evolves?           
– What are its properties? › Causality
– How can it be categorized? – Does X cause Y?
– How can we measure it? – Does X prevent Y?
– What are its components? – What causes X?
› Descriptive-Comparative – What effect does X have on Y?
– How does X differ fromHow Y?  › Causality-Comparative
– Does X cause more Y than does Z?
› Frequency and Distribution
– Is X better at preventing Y than is Z?
– How often does X occur?
– What is an average amount of X? – Does X cause more Y than does Z under one condition
but not others?
› Descriptive-Process
› Design
– How does X normally work?
– By what process does X happen? – What is an effective way to achieve X?
– What are the steps as X evolves?            – How can we improve X?

2/17/2018
 Putting the Question in Context

What will you accept

as a valid truth?

What methods are appropriate

for answering this question?

2/17/2018
 What will you accept as a knowledge?

2/17/2018
 Hypotheses…
› Developing research hypotheses is important since they
provide the focal point of the research.
- They indirectly determine the quality of data which is required for
the analysis.
- Hypothesis should be specific
- Hypothesis should be limited to the piece of research in hand
because it has to be tested.
- The role of the hypothesis is to guide the researcher and keep him
on the right track.
- Working with hypotheses arise as a result of prior thinking about
the subject.

2/17/2018
 Hypotheses…
› A hypothesis must be
 Specific
 Conceptually clear in terms of common definitions
 Testable (for verification or rejection) by available techniques and resources
 Related to a body of theory;
 Stated to provide direction for the research;
 Formulated as causal relationships;

› Some examples of hypothesis

 Improved training provision will create a more productive reliable and satisfied workforce
 Smoking and lung cancer are independent
As a general rule, working with hypothesis is another basic step of the research process in most research
problems --- but may not be always necessary.

2/17/2018
 Factors to consider to develop a strong
research question:
› Do I know the field and its literature well?

› What are the important research questions in my field?

› What areas need further exploration?

› Could my study fill a gap? Lead to greater understanding?

› Has a great deal of research already been conducted in this

topic area?

2/17/2018
 Factors…
› Has this study been done before? If so, is there room for
improvement?
› Is the timing right for this question to be answered?
› Is it a hot topic, or is it becoming obsolete?
› Would funding sources be interested?
› If you are proposing a service program, is the target community
interested?
› Most importantly, will my study have a significant impact on the
field?

2/17/2018
 Characteristics of a Good Research Question
(RQ)
1. Importance
So what?
Will the research make a difference?
2. Scope
while the research problem should not attempt to solve all the
health dilemmas of the world, neither should it be too small
3. Specific
should explicitly identify the variables and
populations of interest

2/17/2018
 Characteristics of a Good Research Question
(RQ)
4. Measurable
cannot be a value judgment
the concepts contained in the question must have an
approach or method that can be measured

5. Contexuality (placement within the framework of

prior knowledge; based on prior knowledge)
knowledge of previous research studies and their outcomes empowers the
researcher to write a hypothesis to accompany the research question

6. Feasibility

2/17/2018
 Characteristics of a Good Research Question
(RQ)
7. Realistic time frame
months vs. years, subject availability, ethical
issues
8. Budget
materials, travel, etc… are the costs reasonable?

9. Interest to the researcher

research is time consuming and demands substantial self
investment; passionate interest is likely to keep the
researcher committed to project completion

2/17/2018
 Objective of the Study
› The objectives of a research summarize what is to be achieved by the
study. These objectives closely related to the research problem.
› The general objective of a study states what researchers expect to achieve
by the study in general terms.
› It is possible (and advisable) to break down a general objective into smaller,
logically connected parts; called specific objectives.
– Specific objectives should systematically address the various research
questions. They should specify what you will do in your study, where and for what
purpose.
› Characteristics of the specific objective:
– It is unequivocal, in other words it is precise and supports only one interpretation.
– It describes an observable behavior on the part of the subject.
– It specifies, where appropriate, the special conditions in which this behavior is
manifested and the criteria which will make it possible to judge whether the
objective has been attained.

2/17/2018
 Objectives..

› Some guidelines in developing objectives:

– Objectives must be specific, concrete and achievable statements.
– The objectives should clearly fit to the statement of the problem
– The objectives must propose to do things as per the capability of
the design of the study.
– Objectives should be in their approximate order of importance.

2/17/2018
 Scope and Delimitation of the study
• Scope of the study states a general outline or coverage of the study. The scope
of research is the areas covered in the research.
– In this part, you will tell exactly what will be done & where the information
used in the study specifically came from.
– Scope defines the object to be considered, or a theory process, activity.
describing either future, current or past knowledge or statements of descriptive
activity, experience etc.
• The scope identifies the boundaries of the study in term of subjects, objectives,
facilities, area, time frame, and the issues to which the research is focused.
– Sample phrases that help express the scope of the study:
• The coverage of this study…… OR, The study covers the ……
• The study consists of ……….... OR, This study focus on……..
• For example, the scope may be gathering information from children between the
ages of five years to 18 years.
• The delimitations of this study would include the decision not to gather
information from college and above students.

2/17/2018
 Delimitations of the study
› The work that will not be undertaken is described as the
delimitations of the research.
› The delimitation of the study describes geographic location,
age, sex, population traits, population size, or other similar
considerations.
– Delimitation is used to make the study better & more feasible &
not just for the interest of the researcher. It also identifies the
constraints or weaknesses of your study which are not within the
control of the researcher.
– Current delimitations might provide avenues for valuable future
research direction.
› Sample phrases that expressed the delimitations of the study
– The study does not cover the……
– The researcher limited this research to……
– This study is limited to………

2/17/2018
 Significance of the study
› Many research problems have a kind of theoretical feel
about them. Such studies often need to be justified:
– What is the research project’s practical value?
› Without this justification, it will prove difficult to convince
others that the problem in question is worth study.
› The significance of the study should discuss the
importance of the proposed research and its relevance.
– The investigation might be relevant for theory, practice and future
research.
– The investigator should explain why it is important for the study to
be undertaken and indicate the likelihood of its contribution to the
advancement of knowledge.

2/17/2018
 Significance of the study…

› Why your research is important and what contributions will

it give to the field

› State how your finding CAN MAKE THE DIFFERENCE and

WHY it is IMPORTANT that the research be carried out

2/17/2018
 Research Methodology
› Methodology is a science of studying how research is done
scientifically
– A way to systematically solve the research problem by logically adopting
various steps
– Look at similar research to decide what kind of methodology is best for
your research question
› Methodologies are high-level approaches to conducting
research.
– The individual steps within the methodology might vary based on the
research being performed.
› Methodology helps to understand not only the products of
scientific inquiry but the process itself
– Aims to describe and analyze methods, throw light on their limitations
and resources, clarify their presuppositions and consequences, relating
their potentialities to the twilight zone at the ‘frontiers of knowledge’

2/17/2018
 Part of Research methodology

› Research design (model, architecture)

› Literature review
› System Development Tools
› Study Population and Sampling
› Dataset preparation: method of Data collection & preprocessing
› Evaluation mechanisms: Testing procedure
2/17/2018
 Research Methodology…
› The proposal should describe in detail the general research plan.
(may not necessarily be true for all types of research)
– Description of study area
– Description of study design
– Description of study participants
– Eligibility criteria ( if any)
– Determination of sample size (if any)
– Description of selection process (sampling method)
– Methods of data collection
– Description of the expected outcome and explanatory variables… (if any)
– How data quality is ensured
– Operational definition
– Presentation of the data analysis methods
2/17/2018
 Choose a Method…
› Exploratory
Used to build new theories where
we don’t have any yet
– E.g. What do CMM level 3
organizations have in common?
– E.g. What are the experiences of
developers who have adopted
Ruby?
› Descriptive
Describes sequence of events and
underlying mechanisms
– E.g. How does pair programming
actually work?
– E.g. How do software immigrants
naturalize?
2/17/2018
› Causal
Determines whether there are causal
relationship between phenomena
– E.g. Does tool X lead to software
with fewer defects?
– E.g. Do requirements raceability
tools help programmers find
information more rapidly
› Explanatory
Adjudicates between competing
explanations (theories)
– E.g. Why does software inspection
work?
– E.g. Why do people fail to document
their requirements?
 Research Design
› Research design is considered as a "blueprint" for research.
› It is:
– a plan for selecting the sources and types of information used to
answer research questions
› WHAT data to collect from WHICH source?
– a framework for specifying the relationships among the study
variables
› WHICH one is the dependent and independent variables?
– a blueprint that outlines each procedures
› HOW to experiment, analyze and interpret the results?

› The best design depends on the research question as well as the

orientation of the researcher.

2/17/2018
 Commonly used research designs
• Quantitative Research Design:
– when the problem requires numerical measurements of trends,
characteristics or attributes of the subject matter;
– Analysis leads researcher to:
• depict what is typical and atypical among the data;
• show the degree of difference or relationship between two or more variables;
• determine the likelihood that the findings are real for the population as opposed
to having occurred only by chance in the sample.

• Qualitative Research Design :

– when the problem focuses on the meanings, perceptions, symbols or
description of the subject matter.
– Analysis leads researcher to:
• observe behaviors, situations, interactions and environments;
• scrutinize these observations for patterns and categories;
• answer research questions based on what can be deduced from the findings.

2/17/2018
 Commonly used research designs
› Qualitative research design, such as Descriptive design
– E.g. Case Study, Focus Groups
– Descriptive Design is the simplest method of scientific inquiry for
qualitative research undertaking
› Describe behavior and mental processes
› Most widely used, Survey method ask people’s opinion
› Quantitative research design, such as: Experimental design
– E.g. True Experiment; Quasi-Experiment
– Experimental design is one of the example of quantitative research design
in which, the researcher actively tries to change the scenario, algorithms,
parameters, or datasets.
– Experimental Design tests relationship of two or more variables
› Allows conclusions about cause-and-effect
› Quantitative measures of behavior compared in different
conditions created by researchers

2/17/2018
 Literature Review..
› A literature review is a necessity
– Without this step, you won’t know if your problem has been solved or what
related research is already underway
› Literature review has mainly two parts:
– Conceptual review: Start with concept discussion from general, broad, textbook and
the Internet
– Related works review: Move toward specialized, recent publications (journal articles,
conference papers) reporting related works with the current topic
› When performing review of related works:
– Start searching professional journals.
– Begin with the most recent articles you can find.
– Keep track of relevant articles in a bibliography.
– Don’t be discouraged if work on the topic is already underway.
› Related works review is a coherent discussion of previous related
works, concerning problem solved, methods used, performance
achieved & future research direction recommended

2/17/2018
 Why Literature Search?
› To make a base for research.
› To gain knowledge in the relevant field
› To know the progress already made in the relevant field
(State of the Art)
› To make sure that you are in right path
› To identify the gap and to get the motivation for
the study
› To relate your results and publish in relevant
journals

2/17/2018
 How do woe select appropriate topic.

2/17/2018
Execution of the project
› It is a very important step in the research process.
› If the execution of the project proceeds on correct lines, the
data to be collected would be adequate and dependable.
› Data collection may involve:
 Field survey / experiments
 Laboratory experiments
 Opinion poll
 Observation
 Document scanning
 Questionnaires
 Interview, etc.

41
Execution of the project
› Data Analysis
 Quantitative data – quantitative figures
 Qualitative data – analytical thinking
 Hypothesis testing
› Generalizations and Interpretation of the results

42
Reporting
› Finally, the researcher has to prepare the report of what
has been done.
› The result should be Communicated to others
Peers
Policy makers/Developers
Extension workers
General public
› The style and content varies depending on to whom the
research is written to.

43
 Steps in Research Problem Formulation

Problem

2/17/2018
 Summary
› This section addresses;
– selecting a topic
– Define Problem
– Formulate Objective
– Define Scope of the Study
– Methodology and design

2/17/2018
 Assignment: Article Evaluation
› Journal Paper or Thesis report Review
– Select an article that interest you
– Set evaluation criteria:
› Clarity and understandability of the problem statement
› originality of the study;
› significance of the study;
› completeness of literature review;
› adequacy of methodology;
› clear presentation of results;
› appropriateness of conclusion & recommendation;
› adequacy of references;
› overall organization.
– Read thoroughly the article and evaluate as per the criteria set
– Report your evaluation result

2/17/2018
 References
› Tedre & Sutinen: Three traditions of computing: what
educators should know
› Ioannidis: Why Most Published Research Findings Are
False.
› Sjoberg et al: Building Theories in Software Engineering

2/17/2018
2/17/2018
3. Reading research
paper/article

2/17/2018
 Reading…
› Comprehension
– What is the research problem the paper attempts to address
– What are the claimed contributions of the paper
– How do the authors substantiate their claims?
– What are the conclusions?
– Evaluation
– Is the research problem significant?
– Are the contributions significant?
– Are the claims valid?
– Are the authors aware of
the relation of their work to existing literature2?
– Is the paper addressing a well-known open problem?

2/17/2018
 Evaluation
– Is the research problem significant?
› Is the work scratching minor itches?
› Are the authors solving artificial problems (aka straw man)?
› Does the work enable practical applications, deepen
understanding, or explore new design space?
– Are the contributions significant?
› Is the paper worth reading?
› Are the authors simply repeating the state of the art?
› Are there real surprises? Are the authors aware of the relation of
their work to existing literature2?
› Is the paper addressing a well-known open problem?

2/17/2018
 Evaluation….
– Are the claims valid?
› Have the authors been cutting corners (intentionally or
unintentionally)?
› Has the right theorem been proven? Errors in proofs?
Problematic experimental setup? Confounding factors?
Unrealistic, artificial benchmarks? Methodological
misunderstanding?
› Do the numbers add up?
› Are generalizations valid?
› Are the claims modest enough3?

2/17/2018
 Reading…
– Synthesis
– What is the crux the research problem?
– What are some alternative approaches to address the
research problem?
– Is there an alternative way to substantiate the claim of the
authors?
– What is a good argument against the case made by the
authors?
– Can the research results be strengthened?
– Can the research results be applied to another context?
– What are the open problems raised by this work?

2/17/2018
 How to Read a Paper
By: S. Keshav

2/17/2018
 The Three-Pass Approach
› The key idea is that you should read the paper in up
to
three passes.
› Each pass accomplishes specific goals and builds
upon the previous pass:
– The first pass gives you a general idea about the paper.
– The second pass lets you grasp the paper’s content, but
not its details.
– The third pass helps you understand the paper in depth

2/17/2018
 The first pass
› The first pass is a quick scan to get a bird’s-eye view of
the paper.
› You can also decide whether you need to do any more
passes.
› This pass should take about five to ten minutes and
consists of the following steps:
1. Carefully read the title, abstract, and introduction
2. Read the section and sub-section headings, but ignore
everything else
3. Read the conclusion
4. Glance over the references, mentally ticking off the
ones you’ve already read
2/17/2018
 The first pass…
› At the end of the first pass, you should be able to answer
the five Cs:
1. Category: What type of paper is this? A measurement paper? An
analysis of an existing system? A description of a research
prototype?
2. Context: Which other papers is it related to? Which theoretical
bases were used to analyze the problem?
3. Correctness: Do the assumptions appear to be valid?
4. Contributions: What are the paper’s main contributions?
5. Clarity: Is the paper well written

2/17/2018
 The second pass
› In the second pass, read the paper with greater care, but
ignore details such as proofs.
› It helps to jot down the key points, or to make comments in
the margins, as you read.
1. Look carefully at the figures, diagrams and other
illustrations in the paper. Pay special attention to graphs.
– Are results shown with error bars, so that conclusions are
statistically significant?
2. Remember to mark relevant unread references for further
reading

2/17/2018
 The second pass…
› The second pass should take up to an hour.
› After this pass, you should be able to grasp the
content of the paper.
› You should be able to summarize the main thrust of
the paper, with supporting evidence, to someone
else.
› This level of detail is appropriate for a paper in
which you are interested, but does not lie in your
research specialty.

2/17/2018
 The Third Pass
› The key to the third pass is to
› attempt to virtually re-implement the paper (making the same assumptions as the authors – recreate
the work)
› re-creation helps to identify a paper’s innovation and also it hidden failings and assumptions

› This pass requires grate attention to detail.

› This pass takes one to two hours for experienced reader
› At the end of this pass
› The reader should be able to reconstruct the entire structure of the paper form memory
› The reader should be able to identify the strong and weak points
› The reader should be able to pinpoint implicit assumptions, missing citations to relevant work, and
potential issues with experimental or analytical techniques.
› You should also jot down ideas for future work

2/17/2018
 Review article
( Michael Bieber, New Jersey institure
of technology

2/17/2018
 Review Article…
› Introduction
– Objectives, article domain, audience
– Types of the article: conceptual/empirical
– Brief summary
– State the problem addressed, solution proposed and evidence
presented for the solutions
– Results
– Briefly summarize observations, conclusions and findings
– Objectives, article domain, audience

2/17/2018
 Review Article…
– Contributions
An article can make a contribution to the research field in many ways
– Describe each contribution clearly in a separate paragraph or bullet point.
– Discuss why the contribution is important
– How confident are you in the article’s results, and why?
– Are its ideas really new, or do the authors simply repackage old ideas and perhaps
give them a new name?
– Does it provide a new way to look at a problem?
– Does it bring together or "synthesize" several concepts (or frameworks, models,
etc.) together in an insightful way that has not been done before?
– Does it provide new solutions?
– Does it provide new results?
– Does it identify new issues?
– Does it provide a comprehensive survey or review of a domain?
– Does it provide new insights?
– Alternatively, if you believe the article makes no contributions, explain why
clearly.

2/17/2018
 Review…
› Foundation
– good research often is built on theories and frameworks that other
researchers have developed? (Which theoretical foundations are
used? In what way: references, citations)
› Analysis
– What has changed since the article was written?
– How do it's lessons, ideas and theories still apply?
– To what extent has its issues been resolved?
› Critique
– State how well (or badly) the authors did the research and
presented the research results

2/17/2018
 Review…
› Critique…
– Does it build upon the appropriate foundation (i.e., upon appropriate
prior research)?
– Did the authors choose the correct approach, and then execute it
properly?
– How confident are you in the article's results, and why?
– Are its ideas really new, or do the authors simply repackage old ideas
and perhaps give them a new name?
– Do the authors discuss everything they promise in the article's
introduction and outline?
– What are the article's shortcomings (faults) and limitations (boundaries)?
– Did it discuss all of the important aspects and issues in its domain (topic
area)?

2/17/2018
 Review…
› Critique …
– In what way should the article have made a contribution, but then
did not?
– Do the authors make appropriate comparisons to similar events,
cases or occurrences?
– How complete and thorough a job did the authors do? Do the
authors include an adequate discussion, analysis and
conclusions? Did they justify everything adequately? Did they
provide enough background information for the intended audience
to understand it? For you to understand it?
– Were there adequate and appropriate examples and illustrations?

2/17/2018
 Article Review…
› Further Critique
Critique conceptual
1. Logical consistency
2. Coherence
3. Substance
4. Focus
Critique Empirical
1. Clarity (is the purpose or argument clear?)
2. Theoretical grounding (grounded in more basic theory)
3. Design of the research investigation
4. Measurement
5. Analysis
6. discussion and Conclusion

2/17/2018
 Review…
› Impact
– The impact the article has had (do citation analysis)
› Questions
– List insightful questions of your own , arising from the reviewed
article
› Bibliography
– List the full bibliographic references

2/17/2018
 Sample templates for paper review (Fong 2009)

› Summary/Abstract
1. Motivation
2. Contribution
3. Methodology
4. Conclusion
› Critique
1. Title of first critique
2. Title of second critique
3. Title of third critique (optional)
› Synthesis
1. Title of the first synthesis idea
2. Title of the second synthesis idea

2/17/2018
 Article review Template
› Introduction
› Brief summary
› Results
› Contributions
› Foundation
› Analysis
› Critique
› Impact
› Questions
› Bibliography

2/17/2018
 References
› S. Keshav (2007) . How to Read a Paper. ACM SIGCOMM Computer Communication Review,
37(3).
› Amanda Stent (). How to Read a Computer Science Research Paper
› Philip W. L. Fong(2009). Reading a Computer Science Research Paper.
› Ian Parberry A Guide for New Referees in Theoretical Computer Science
› Michael Ernst; How to write a technical paper.
https://homes.cs.washington.edu/~mernst/advice/write-technical-paper.html
› Robin Murphy Reviewing Papers: A Student Guide.
http://web.archive.org/web/20080414163138/http://www.csee.usf.edu/~murphy/Students/reviewin
g.html
› How to Do Research: http://www.ifs.tuwien.ac.at/~silvia/research-tips/
› Advice on Research and Writing. http://www.cs.cmu.edu/afs/cs.cmu.edu/user/mleone/web/how-
to.html

2/17/2018
2/17/2018
4. Research Proposal Writing
 What is Research Proposal

› A research proposal is an outline of proposed works required by

many universities and institutions, serves as a useful basis for the
evaluation of a project as well as a guide line for the researcher.

› After proper and complete planning of a research, the plan should be

written down. The research proposal is the detailed plan of study.

› The term "research proposal" indicates that a specific course of

action will be followed.

2/17/2018
 What is Research…
› This assumes a longer preparatory period of reading,
observation, discussion, and incubation.
› Read everything that you can in your area of interest.
› Figure out what the important and missing parts of our
understanding are.
› Figure out how to build/discover those pieces.
› Live and breath the topic.
› Talk about it with anyone who is interested.
› Then just write the important parts as the proposal

2/17/2018
 What is Research …

› The written proposal:

– Forces the students to clarify their thoughts and to think about all
aspects of the study;

– Is a necessary guide if a team is working on the research;

– Is essential if the study involves research on human subjects or on

experimental animals, in order to get the institution’s ethical approval;

– Is an essential component submitted for funding.

2/17/2018
 What is Research Proposal…
› A well-thought out and well-written proposal can be
judged according to three main criteria.
– Is it adequate to answer the research question(s), and
achieve the study objective?
– Is it feasible in the particular set-up for the study?
– Does it provide enough detail that can allow another
investigator to do the study and arrive at comparable results?

2/17/2018
 Components of a Research Proposal
1. Title 6. Hypotheses /Questions
2. Summary/Abstract 7. Conceptual framework
3. Introduction/Background 8. Objective/Aim of the study
4. Statement of the problem 9. Significance of the study
5. Literature review 10.Research methods, materials
and procedures

2/17/2018
 Title

› A title ought to be well studied, and to give, as far as its limits

permit, a definite and concise indication of what is to come.

› The title of your research proposal should state your topic

exactly in the smallest possible number of words.

› All words in the title should be chosen with great care, and
association with one another must be carefully managed.

2/17/2018
 Title

› A good title is defined as the fewest possible words that

adequately describe the contents of the study.

› Title is a label; it is not a sentence.

› Titles should almost never contain abbreviations.

2/17/2018
 Abstract
› Motivation:
– Why do we care about the problem and the results?
› Problem statement:
– What problem are you trying to solve?
– What is the scope of your work (a generalized approach, or for a specific
situation)?
› Methodology/Approach
– How did you go about solving or making progress on the problem? Did you
use simulation, analytic models, prototype construction, or analysis of field
data for an actual product?
– What was the extent of your work (did you look at one application program
or a hundred programs in twenty different programming languages?)
– What important variables did you control, ignore, or measure?

2/17/2018
 Abstract…
› Results:
– What's the answer? Specifically, most good computer architecture papers
conclude that something is so many percent faster, cheaper, smaller, or
otherwise better than something else.
– Put the result there, in numbers. Avoid vague, hand-waving results such
as "very", "small", or "significant."
› Conclusions:
– What are the implications of your answer?
– Are your results general, potentially generalizable, or specific to a
particular case?
– How does this work add to the body of knowledge on the topic?
– Are there any practical or theoretical applications from your findings or
implications for future research?

2/17/2018
 Introduction/Background

› The introduction is the part of the proposal that provides

readers with the background information for the research
proposal.

› Its purpose is to establish a framework for the research, so

that readers can understand how it is related to other
research.

2/17/2018
 Introduction/Background
› Be sure to include a hook at the beginning of the introduction.

› This is a statement of something sufficiently interesting to motivate your

reader to read the rest of the proposal; it is an important/interesting
scientific problem that your study either solves or addresses.

› The introduction should cite those who had the idea or ideas first, and
should also cite those who have done the most recent and relevant work.

› You should then go on to explain why more work is necessary (your work,
of course.)

2/17/2018
 Statement of the Problem

› Most research proposals, whether designed for master's

theses or doctoral dissertations, may be considered as
responses to a problem.

› A problem might be defined as the issue that exists in the

literature, theory, or practice that leads to a need for the study.

› The prospective researcher should think on what caused the

need to do the research (problem identification).

2/17/2018
 Statement of the Problem

› The question that he/she should ask him/herself is:

– Are there questions about this problem to which answers

have not been found up to the present?

› The research problem should be stated in such a way

that it would lead to analytical thinking on the part of
the researcher

2/17/2018
 Statement of the Problem…
› Effective problem statements answer the question “Why does
this research need to be conducted?”

› A well-articulated statement of the problem establishes the

foundation for everything to follow.

› In subsequent sections of the proposal, there should be no

surprises, such as categories, questions, variables or data
sources that come out of nowhere

2/17/2018
 Literature Review

› Literature review is not a compilation of every work written

about a topic.

› It is not simply a list of sources reviewed separately for

their own merit.

› A literature review is a description of the literature relevant

to a particular field or topic.

2/17/2018
 Literature Review
› It gives an overview of
– what has been said,
– who the key writers are,
– what are the prevailing theories and hypotheses,
– what questions are being asked, and
– what methods and methodologies are appropriate and
useful.
› As such, it is not in itself primary research, but
rather it reports on other findings.

2/17/2018
 Questions and/or Hypotheses

› Hypotheses and questions are linked to the speculative

proposition of the problem statement, can be inferred from
the overall conceptual framework of a study, and are of
critical importance to data analysis and interpretation.

2/17/2018
 Questions and/or Hypotheses
› In research studies, the term hypothesis implies a derivation, within a
hypothetic-deductive theoretical system of a particular assertion or prediction.

› The hypothesis is subject to test, i.e., confirmation or rejection on empirical

grounds.

› The term question implies an interrogative statement that can be answered

by data, which is logically related to the same conceptual framework, but
which does not necessarily stem from that framework through logical
deduction.

2/17/2018
 Questions and/or Hypotheses
› Questions are most often used in qualitative inquiry, although
their use in quantitative inquiry is becoming more prominent.
› Hypotheses are relevant to theoretical research and are
typically used only in quantitative inquiry.
› A research question poses a relationship between two or more
variables but phrases the relationship as a question; a
hypothesis represents a declarative statement of the relations
between two or more variables.
› Deciding whether to use questions or hypotheses depends on
factors such as the purpose of the study, the nature of the
design and methodology of the research.

2/17/2018
 Questions and/or Hypotheses

› Make a clear and careful distinction between the

dependent and independent variables and be certain they
are clear to the reader.

› Hypotheses are thus tentative statements that should

either be acknowledged or rejected by means of research.

2/17/2018
 Conceptual framework
› Every research activity is conceptualized and will be
carried out within some contextual framework.

› This contextual framework is in part conceptual, in part

valuation, and in part practical (or operational), and all of
these factors must typically be considered.

› A conceptual framework is described as a set of broad

ideas and principles taken from relevant fields of enquiry
and used to structure a subsequent presentation.

2/17/2018
 Conceptual framework
› When clearly articulated, a conceptual framework has
potential usefulness as a tool to support research and,
therefore, to assist a researcher to make meaning of
subsequent findings.
› Such a framework should be intended as a starting point
for reflection about the research and its context.
› The framework is a research tool intended to assist a
researcher to develop awareness and understanding of the
situation under scrutiny and to communicate this.

2/17/2018
 Objective/aim of the Study
› The objectives of a research delineate the ends which the inquirer
seeks to bring about as a result of completing the research
undertaken.

› An end state to be achieved in relation to the problem.

› Objectives should be closely related to the statement of the problem.

› After statement of the primary objective, secondary objectives may

be mentioned.

2/17/2018
 Objective/aim of the Study
› Objectives should be
– Simple (not complex),
– Specific (not vague),
– Stated in advance (not after the research is done), and
– Stated using “action verbs” that are specific enough to be measured.
› A well worded objective will be SMART, i.e. Specific, Measurable, Attainable,
Realistic, and Time-bounded.
› Commonly, research objectives are classified into general objectives and
specific objectives.
› The general and specific objectives are logically connected to each other and
the specific objectives are commonly considered as smaller portions of the
general objectives.

2/17/2018
 Objective…
› General objective
– What exactly will be studied?
– General statements specifying the desired outcomes of the proposed project
› Specific objectives
– Specific statements summarizing the proposed activities and including
description of the outcomes and their assessment in measurable terms
– It identifies in greater detail the specific aims of the research project, often
breaking down what is to be accomplished into smaller logical components
– Specific objectives should systematically address the various aspects of the
problem as defined under ‘Statement of the Problem’ and the key factors that
are assumed to influence or cause the problem. They should specify what you
will do in your study, where and for what purpose

2/17/2018
 Objective…

› The formulation of objectives will help you to:

– Focus the study (narrowing it down to essentials);
– Avoid the collection of data which are not strictly
necessary for understanding and solving the problem
you have identified; and
– Organize the study in clearly defined parts or phases.

2/17/2018
 Significance of the Study

› Why your research is important and what contributions will

it give to the field

› State how your finding CAN MAKE THE DIFFERENCE and

WHY it is IMPORTANT that the research be carried out

2/17/2018
 Methods, material and procedures
› The methods or procedures section is really the heart of the
research proposal.
› You must decide exactly how you are going to achieve your stated
objectives: i.e., what new data you need in order to shed light on
the problem you have selected and how you are going to collect
and process this data.
› The activities should be described with as much detail as possible,
and the continuity between them should be apparent.
› Indicate the methodological steps you will take to answer every
question, to test every hypothesis illustrated in the
Questions/Hypotheses section or address the objectives you set.

2/17/2018
 Methods, material and procedures…
› The proposal should describe in detail the general research plan.
(may not necessarily be true for all types of research)
– Description of study area
– Description of study design
– Description of study participants
– Eligibility criteria ( if any)
– Determination of sample size (if any)
– Description of selection process (sampling method)
– Methods of data collection
– Description of the expected outcome and explanatory variables… (if any)
– How data quality is ensured
– Operational definition
– Presentation of the data analysis methods

2/17/2018
 Study design
› The type of research design chosen depends on:
– the type of problem;
– the knowledge already available about the problem; and
– the resources available for the study.

› Generally, there are two main categories of research design:

observational study, and experimental or intervention study.
› In the observational study, the researchers stand apart from events
taking place in the study. They simply observe and record.
› In the experimental or intervention study, the researches introduce an
intervention and observe the events which take place in the study.

2/17/2018
 Sampling………(for quantitative Studies)
› Sampling is the process of selecting a number of study units
from a defined study population.
› Often research focuses on a large population that, for practical
reasons, it is only possible to include some of its members in
the investigation.
› You then have to draw a sample from the total population. In
such cases you must consider the following questions:
– What is the study population you are interested in from which we want to
draw a sample?
– How many subjects do you need in your sample?
– How will these subjects be selected?

2/17/2018
 Study population and sampling
• A finite group is called population whereas a non-finite (infinite) group
is called universe
 Study population: Subset of general population defined by the
eligibility criteria.
 Actual sample studied is selected from the study population
• A sample is a subset of study population of
Population objects such as organizations, businesses,
individuals, households, and so forth.

• Sampling enables researchers to make

Sample estimates of some unknown characteristics of
the population in question

2/17/2018
 Reasons for Sampling
• Budget and time constraints
 In case of large populations it may not be
possible to investigate within the given budget
and allotted time
• High degree of accuracy and reliability
 Sampling may sometimes produce more accurate results than
taking a census as in the latter, there are more risks for making
interviewer and other errors due to the high volume of persons
contacted and the number of census takers, some of whom may
not be well-trained
 It depends on the extent to which the selected sample is
representative of population

2/17/2018
 The Sampling Process
Plan procedure for
selecting sampling units

Select Sampling Method:

3 Probability or non-probability Determine sample size 5
Sampling method

2 Select a Sampling Frame Select actual sampling units 6

Define the Target

1 Conduct experiments 7
population
107
2/17/2018
 Defining the Target Population
 The target population is that complete group whose relevant
characteristics are to be determined through the sampling
– The target population is the collection of elements or objects that
possess the information sought by the researcher and about which
inferences are to be made.
A target population may be, for example,
– all pre-college students in Addis Ababa,
– all post-graduate students enrolled in Ethiopian Universities,
– Printed real-life/historical Amharic document images for document
image retrieval
The target group should be clearly delineated if possible, for
example, do all pre-college students include only primary and
secondary students or also students in other specialized
educational institutions?

2/17/2018
 Probability and Non-Probability Sampling
In general there are two sampling techniques.
Probability Sampling – Every element in the population under study
has a non-zero probability of selection to a sample, and every
member of the population has an equal probability of being selected

 Includes Simple Random Sampling, Systematic Sampling, stratified sampling,

Cluster Sampling, Multistage Area Sampling, Internet Sampling, etc.

Non-Probability Sampling – An arbitrary means of selecting sampling

units based on subjective considerations, such as personal judgment
or convenience. It is less preferred to probability sampling
 Include Convenience Sampling, Purposive Sampling, Snowball Sampling, etc.

2/17/2018
 Classification of Sampling Techniques
Sampling Techniques

Nonprobability Probability
Sampling Techniques Sampling Techniques

Convenience Judgmental Quota Snowball

Sampling Sampling Sampling Sampling

Simple random Systematic Stratified Cluster Other Sampling

Sampling Sampling Sampling Sampling Techniques

2/17/2018
 Non-Probability sampling: Convenience
Sampling
 This is a sampling technique which selects those sampling
units most conveniently available at a certain point in, or
over a period, of time
 Convenience sampling attempts to obtain a sample of convenient
elements. Often, respondents are selected because they happen
to be in the right place at the right time.
Major advantages of convenience sampling is that it is
quick, convenient and economical
 A major disadvantage is that the sample may not be
representative
Convenience sampling is best used for the purpose of
exploratory research and supplemented subsequently with
probability sampling

2/17/2018
 Non-Probability Sampling: Quota Sampling
This is a sampling technique in which the researcher ensures
that certain characteristics of a population are represented in
the sample to an extent which is he or she desires
Quota Sampling has advantages and disadvantages:
 Advantages include the speed of data collection, less cost, the
element of convenience, and representativeness (if the subgroups
in the sample are selected properly)
 Disadvantages include the element of subjectivity (convenience
sampling rather than probability-based which leads to improper
selection of sampling units)

2/17/2018
 Quota sampling…
Quota sampling may be viewed as two-stage restricted judgmental
sampling.
– The first stage consists of developing control categories, or quotas, of
population elements.
– In the second stage, sample elements are selected based on
convenience or judgment.

Population Sample
composition composition
Control
Characteristic Percentage Percentage Number
Gender
Male 48 48 480
Female 52 52 520
100 100 1000
2/17/2018
 Non-Probability Sampling: Snowball Sampling
• In snowball sampling, an initial group of respondents
(individuals or organizations) are selected, usually at random by
probability methods.
– After the necessary information is being gathered from initial
respondents, they are asked to identify additional respondents who
belong to the target population of interest.
– Subsequent respondents are selected based on the referrals.
Example: Through a sample of 500 individuals, 20 players are
identified which, in turn, identify a number of other players
 The advantage of snowball sampling is that smaller sample sizes and
costs are necessary; a major disadvantage is that the second group of
respondents suggested by the first group may be very similar and not
representative of the population with that characteristic

2/17/2018
 Snowball sampling…
More systematic versions of snowball sampling can reduce the
potential for bias .For example, “respondent-driven sampling”
gives financial incentives respondents to recruit peers.

2/17/2018
 Probability Sampling: Simple Random
Sampling
› Simple Random Sampling is a technique which ensures that
each element in the population has an equal chance of being
selected for the sample
 Example: random-digit telephone dialing

› Simple random sampling is the most widely-used probability

sampling method, probably because
 it is easy to implement and easy to analyze.

› Sampling With Replacement and Without Replacement

 Sampling with replacement: select a random sample where a population
element can be chosen more than once.
 Sampling without replacement: select a random sample where a
population element can be chosen only one time.

2/17/2018
 Probability Sampling: Stratified Sampling
 A two-step process in which the population is partitioned into
subpopulations, or strata.
– The strata should be mutually exclusive and collectively exhaustive in that every
population element should be assigned to one and only one stratum and no
population elements should be omitted.
– Next, elements are selected from each stratum by a random procedure, usually
Simple Random Sampling. This is a technique in which simple random
subsamples are drawn from within different strata that share some common
characteristic
 Example: students in BDU can be divided based on study fields
(business, Technology, medical) & from each group, select sample
students using simple random sampling, whereby the size of the
sample for each group is determined by that group’s overall strength
– Stratified Sampling has the advantage of giving more representative samples and
less random sampling error;
– the disadvantage is that, it is more complex and information on the strata may be
difficult to obtain

2/17/2018
 Stratified sampling….
› The elements within a stratum should be as homogeneous as
possible, but the elements in different strata should be as
heterogeneous as possible.
› The stratification variables should also be closely related to the
characteristic of interest.
› Finally, the variables should decrease the cost of the stratification
process by being easy to measure and apply.
› In proportionate stratified sampling, the size of the sample drawn
from each stratum is proportionate to the relative size of that stratum
in the total population.
› In disproportionate stratified sampling, the size of the sample from
each stratum is proportionate to the relative size of that stratum and
to the standard deviation of the distribution of the characteristic of
interest among all the elements in that stratum

2/17/2018
 Stratified sampling….

Draw a sample from each stratum

2/17/2018
 Probability Sampling: Cluster Sampling
› Cluster sampling is a sampling technique used when "natural"
but relatively homogeneous groupings are evident in a statistical
population.
– The target population is first divided into mutually exclusive and
collectively exhaustive subpopulations, or clusters.
– Then a random sample of clusters is selected, based on a probability
sampling technique such as Simple Random Sampling.
› For each selected cluster, either all the elements are included in
the sample (one-stage) or a sample of elements is drawn
probabilistically (two-stage).
› Elements within a cluster should be as similar as possible, but
elements in different clusters should be as dissimilar as possible.
› In probability proportionate to size sampling, the clusters are
sampled with probability proportional to size.
– In the second stage, the probability of selecting a sampling unit in a
selected cluster varies inversely with the size of the cluster

2/17/2018
 Types of Clusters
Cluster Sampling

One-Stage Two-Stage Multistage

Sampling Sampling Sampling

Simple Cluster Probability

Sampling Proportionate
to Size Sampling

2/17/2018
Advantage and Disadvantage of Basic Sampling Techniques
Technique Strengths Weaknesses
Non-probability Sampling
Convenience sampling Least expensive, least time- Selection bias, sample not representative, not
consuming, most convenient recommended for descriptive or causal research

Quota sampling Sample can be controlled for Selection bias, no assurance of

certain characteristics representativeness
Snowball sampling Can estimate rare characteristics Time-consuming

Probability sampling
Simple random Easily understood, results Difficult to construct sampling frame, expensive,
sampling (SRS) projectable lower precision no assurance of
representativeness, Can decrease
representativeness
Stratified sampling Include all important Difficult to select relevant,
subpopulations, precision stratification variables, not feasible to stratify on
many variables, expensive
Cluster sampling Easy to implement, cost effective Imprecise, difficult to compute and interpret
2/17/2018
results
 Issues in Sample Design and Selection
› Accuracy
– Samples should be representative of the target population (less
accuracy is required for exploratory research than for conclusive
research)
› Resources
– Time, money and individual or institutional capacity are very
important considerations due to the limitation on them. Often,
these resources must be “traded” against accuracy
› Availability of Information
– Often information on potential sample participants in the form of
lists, directories etc. is unavailable (especially in developing
countries) which makes some sampling techniques (e.g.
systematic sampling) impossible to undertake
› Geographical Considerations
– The number and dispersion of population elements may determine
the sampling technique used (e.g. cluster sampling)

2/17/2018
 Evaluation and Testing
• Why Evaluation? Any systems needs validation & verification
– Validation
• Concerned with building the right system
• Make sure it’s what the user wants
– Verification
• Concerned with building the software right
• Make sure it works
• It provides the ability to measure the difference between
systems/tools/algorithms
–How well do our system (search engine, Recognizer, …) works?
–Is system A better than B? Under what conditions?
• Evaluation drives what to study
–Identify techniques/algorithms that work well and do not work
–What is the best component for the different modules integrated within
the system:
• Identify the different components of MM search engine?

2/17/2018
 Evaluation criteria
What are the main evaluation measures to check the
performance of an IR system?
• Efficiency
– Time and space complexity of the data structure and algorithms used in
developing the system
• Speed in terms of running time
• Memory space requirement

• Effectiveness
–How is a system capable of achieving its intended objective?
–Is system X better than other systems?
–User satisfaction: How “good” are the output generated by the system? is
that as per users need?
–To make sure the rate of Relevance of results to meet information need
of users

2/17/2018
 Assignment

Proposal Development

2/17/2018
2/17/2018
 Future Work

2/17/2018