CH 2 The Research Process

1
Points to recall
 Definition of research
 Eight features of research
 Purpose /objectives of research
 Motivation and significance of research
 Theory and hypothesis
 Types of research
 Ethical issues in research

2
Research Process (natural way of
viewing the process in CS)
 Realize there is a problem
 Look to see if anyone has solved it
 Develop a plan to solve it
 Solve it
 Evaluate your solution
 Disseminate your solution
 Topics- The Research Process
 Step 1 -Problem- Selection (title) & formulation of Research Problem
 Step 2 -Literature survey

(may also include development of objectives and working hypotheses)

 Step 3 -Research design

( Pilot (quick) study ) (Step 4- Developing Instruments)

 Research Execution
 Step 5- Data collection, exp., analysis, Testing hypotheses
 Step 6-, solution design, Interpretation & generalization
 Step 7- Preparation & writing of the report.
 NOTE: The above steps are not exhaustive, nor mutually exclusive,
but a series of closely related, continuously overlapping and
interdependent nonlinear steps/ actions.
4
Research Process (general)
Problem

Data
Liter., Theory Interpretation/
solution
design/Eval

Research
Design Data Collection &
(Qnt./Qlt./ Analysis/experimen
Design sc. tation
Develop
Instrument/
procedure
5
6
 Design Research process

Metrics, Analysis
How to Knowledge

Knowledge

Disciplinary
Inference

Knowledge
Identify Define Theory Design & Demonstration Evaluation Communica-
Problem & Objectives of Development Find Suitable tion
Motivate a Solution context Observe How
Effective, Scholarly
Define Problem What would a Artifact Use Artifact to efficient Publications
Show Better Artifact Solve problem
Importance/ Accomplish? Iterate Back to Professional
supported by lit. design Publications

Process Iterations

7
Classical phases – experimental reseach

8
Other variants

9
 Where can we get a research problem – topic?
From supermarket?

10
Step 1: Problems/Questions- Topic
 Finding –stating /formulating and- closing by RQ or Hypothesis

 A question/problem occurs or is posed to the researcher for

which that researcher has no and can find no answer(s).

 Example ;
 Some mobiles are being used widely? Sentiment analysis
 While we search on the web there are a number of junks
responded?
 In some places accidents are prevalent than others?

11
Cont…
 Research begins with a problem.
 This problem need not be Earth-shaking.

 Identifying this problem can actually be the hardest part of

research.
 In general, good research projects should:
 Address an important question.
 original and significant
 Advance knowledge.

12
General Sources of Research Problems
 Personal observation of the environment
 Problems / opportunities
 Literature reviews and future work recommendations
 Suggestions of previous researchers
 Professional conferences/ Experts
 Domain experts and professionals talking on conferences
Some frameworks-idea generation
 Proposing if a new way of doing/new approach / etc… will
work (Feasibility Study: Here is a new idea, is it possible ?)

 Comparative study

 What would happen if ….type of study (simulation)

 Hybridization /Integration study (combining methods, tools

techniques…

14
Feasibility Study: Here is a new idea, is it
possible ?
 • Is it possible to solve a specific kind of problem … effectively ?
 computer science perspective (Turing test, …)
 engineering perspective (build efficiently; fast — small)
 economic perspective (cost effective; profitable)
 • Is the technique new / novel / innovative ?
 compare against alternatives
 • Proof by construction
 build a prototype, often by applying on a “CASE”
 • Conclusions
 primarily qualitative; "lessons learned“,
 quantitative: - economic perspective: cost – benefit,
 engineering perspective: speed - memory footprint

15
16
17
 Hybridization
• Sometimes two approaches can be combined, in order to obtain a hybrid solution
(hybridization). Hybrid solutions can be
• Symbiotic (measuring the conditions in the environment and switching from one
approach to the other, so that each approach is active all the time while the conditions
are such that it provides better performance compared to the other approach) or
• Synergistic (creating a new approach, which, for each particular solution element
takes the better solution element of two different approaches).
Thus- Finding Research problems
(topics)
 Everywhere
 Whatever arouses interest, tweaks curiosity, raises questions
but no answer or answers exist but dispute arises on validity
 Extremely important to distinguish between PERSONAL and
RESEARCHABLE problem
 Personal problems are real but not researchable
 Researchable problems fit the requirement of the scientific
method

19
What is NOT a Research Problem?
 Not Self-Enlightenment
 Example, “ the problem of this research is to learn more about the way
the XXXX system is developed”
 Not Comparing Data
 Example “This research project will compare the increase in the
number of women students over 10 years from 1990 to 2000
with the men students over the same time span.”
 Problems that Result in a Yes or No Answer
 Example, “ Is homework beneficial to children?”

20
 SIX guidelines to select and
formulate an important and
useful research problem

21
1. Look Around You

 In many disciplines, questions that need answers –

phenomena that need explanation - are everywhere.
 Concentrate on smaller problems – continually ask
questions about what you hear and see.
 Why does such–and–such happen? What makes such–and–
such tick? (The reasons for somebody’s behaviour)

22
2. Read the Literature
 What things are already known – don’t reinvent the wheel –
also tells what is NOT known in the area – in other words,
what still needs to be done.
 Research project might
a) Address the suggestions for future research that another
researcher has offered
b) Replicate a research project in a different setting or with a
different population
c) Consider how various subpopulations might behave
differently in the same situation
d) Apply an existing perspective or explanation to a new
situation

23
Cont…
e) Explore unexpected or contradictory findings in previous
studies
f) Challenge research findings that seem to contradict what
you know or believe to be true.

Other advantages of Literature

 Provides theoretical base on which to build a rationale for your
study
 Provides potential research methodologies and methods of
measurement
 Help you interpret your results and relate them to what is
already known in the field

24
3. Attend Professional Conferences
 Many researchers have great success finding new research projects at
national and regional conferences.
 Learn “what is hot and what is not” in their field
 In our case
 Sentiment analysis
 M-systems , Security, privacy in wireless communication
 Social networking
 Network/system security, performance and configuration
 Big data, web mining
 Method tailoring (in software development)
 SOA
 Localization
 Novice researchers can make contacts with experts in their field, ask
questions, share ideas, exchange e-mail addresses with more experienced
and knowledgeable individuals

25
4. Seek the Advice of Experts

 Another simple yet highly effective strategy for

identifying a research problem is simply to ask an
expert:
a) What needs to be done?
b) What burning questions are still out there?
c) What previous research findings seemingly don’t
make sense?

26
5. Choose a Topic that Motivates You

 Reading literature, attend conferences, talk with experts,

will uncover a number of potential research problems
 Pick just one, based on what you want to learn more about
 Must believe that it is worth your time and effort.

 Saying “You’re going to be married to it, so you might as

well enjoy it.”

27
6. Choose a Topic That Others Will Find
Interesting and Worthy of Attention
 Want to share findings with a larger audience, not only end
with thesis.
 Describe what you have done at a regional or national
conference, publish an article in a professional journal, or
both.
 Future employers, too, are also interested in your thesis
topic if in your research, you are pursuing an issue of
broad scientific or social concern or, more generally, a hot
topic in your field.

28
Stating a Research Problem
 The heart of any research project is the problem.
 At every step in the process, successful researchers ask
themselves:
 What am I doing? For what purpose am I doing it?
 Such questions can help focus your efforts toward achieving
your ultimate purpose for gathering data: to resolve the
problem.
 Researchers get off to a strong start when they begin with an
unmistakably clear statement of the problem.
 Answer these two basic questions
 What?
 Why?

29
Thus
 After identifying a research problem, therefore, you must
articulate it in such a way that it is carefully phrased and
represents the single goal of the total research effort.
 So once you’ve identified a research problem:
 •State that problem clearly and completely.
 •Determine the feasibility of the research.
 Identify subproblems:
 •Completely researchable units.
 •Small in number.
 •Add up to the total problem.
 •Must be clearly tied to the interpretation of the data.

30
The Problem Statement
 Bad habits – try to state a research problem by jotting down
meaningless groups of words, verbal fragments - no help in
seeing the problem clearly
 Examples of half-statements, mere verbal blobs that only hint
at the problem but do not state it
 A) …..this research will study software metrics and the
quality of software
 B) …… subsidise ICT industry
 C)…… ICT promotes English in school
 D) ….is on QoS in computer network

31
Cont…
 The fragments demonstrate that the researcher either
cannot or will not think in terms of specific, researchable
goals

 But must limit the area of study to a manageable size

 Example, metric and quality,
 must limit what metrics? which quality attribute? and more
importantly what domain of the software you want to
investigate?
 By specifying the domain you are narrowing down the metric
and the quality attribute

32
Example: Metric and Quality
 A clear problem statement
 “This research will investigate the effect of module size on the
understandability of program for a Science subject educational
software”
 In this statement it is clear that
 The metric: module size
 The quality attribute: understandability
 The domain: program, Science subject, educational software

33
 Example : describe the basic problem of
your research?

 Through /the purpose of/ this ___________(survey,

historical, experimental, ethnographic, case) study /attempt/
____(is, was, will be made) to ________ (understand,
describe, develop, discover) the ________(central focus) for
_______(unit of analysis: person, process, groups, site).

 At this stage in the research, the _______(central focus) will

generally be defined as ________(provide a general research
questions of the central focus).

34
Research Questions and hypothesis
 Hypotheses, are statements the researcher makes about the
expected outcomes of relationships among variables.
 There are two forms: (a) null and (b) alternative.
 A null hypothesis represents the traditional approach
 It makes a prediction that in the general population, no
relationship or no significant difference exists between groups
on a variable. The wording is, “There is no difference (or
relationship)” between the groups
 Example
 There is no significant difference between the effects of
verbal cues, rewards, and no reinforcement in terms of social
interaction for children with autism and their siblings.

35
Cont…
 The second form, popular in journal articles, is the
alternative or directional hypothesis. (there is also
non directional)
 The investigator makes a prediction about the
expected outcome, basing this prediction on prior
literature and studies on the topic that suggest a
potential outcome
 Example:
 Publicly traded firms will have higher growth rates than
privately held firms

36
Cont…
 In cases where you don’t have hypothesis you may need
to formulate research questions representing the problem
in a form of question.
 Here is example;
 Very weak (in an empirical sense): Is the new technique any
good?
 Weak : Is the new technique better than our competitors?
 Better : Is the new technique faster than our competitors?
 Better still: Is the new technique faster than our competitors
within one hour of use?
 Even better : If error rates are kept under 2%, is the new
technique faster than our competitors within one hour of use?

37
Cont…
 To eliminate redundancy, write only research questions or
hypotheses—not both—unless the hypotheses build on the
research questions.

 Choose the form based on tradition, recommendations

from an adviser or faculty committee, or whether past
research indicates a prediction about outcomes

38
Evaluation of a research problem (some questions to ask):
– Is the problem in line with my goal/expectations and
the expectation of others?
– Will the solution of the problem advance knowledge?
– What is the value of potential outcome? (who are the
beneficiaries?)
– Do I possess or can I acquire the necessary skills,
abilities and background knowledge to study the
problem? (Researcher’s capability and interest)
– Will data be accessible?
– Do I have access to the necessary resources (time,
money, tools, equipments, laboratory, subjects, etc)
to conduct the investigation?

39
Step 2: Literature, Theory

What is the purpose of literature review?

40
Topic
 Definition
 Benefits
 Sources to look in to
 Searching strategies
 Reading /evaluating /organizing
 Some more tips
 Revisiting objectives/RQ/hypothesis

41
Literature Review (LR)
 In general, students don’t understand the purpose of
investigating literature
 Its function – the more one knows the more knowledgeably one
can approach the problems
 Re+view = look again at what others have done in areas that are
similar, though not necessarily identical to, one’s own area of
investigation
 As a researcher, you should know the literature about your topic
very, very well
 Many benefits from reviewing
 Conceptual and
 Empirical works

42
Benefits of Literature Review
 It can offer new ideas, perspectives, and approaches that may not
have occurred to you
 It can inform you about other researchers who conduct work in
this area – individuals whom you may wish to contact for advice
or feedback
 It can show you how others have handled methodological and
design issues in studies similar to your own
 It can reveal sources of data that you may not have known existed
 It can introduce you to measurement tools that other researchers
have developed and used effectively
 Available literature/theory is reviewed to determine if there is
already a solution to the problem. The existing solution might
require some revision or even be discarded.

43
Cont…
 It can reveal methods of dealing with problematic
situations that may be similar to difficulties you are facing
 It can help you interpret and make sense of your findings
and, ultimately, help you tie your results to the work of
those who have preceded you
 It will bolster your confidence that your topic is one worth
studying, because you will find that others have invested
considerable time, effort, and resources in studying it

44
Cont…
 For MSc. And PhD, the second last benefit is of particular
value
 Theses are presumed to be original investigations into
unexplored area
 Candidates thinking a particular problem to be unoccupied
territory has been so well tilled by others – practically
farmed out

 Important - you should know where others have been and

what activities they have been engaged

45
Where to look at?

46
Where to Begin searching
 Library catalog – Locate books relevant to your research
topic – may be general textbooks in your discipline or
collections of articles written by a variety of experts in the
field
 Indexes and Abstracts – Begin with periodical in your
academic area – Computer Science/software engineering/
IT
 Online databases - Access to the literature which contain
enormous collections of citations or abstracts related to
various subjects and disciplines

47
 In addition to using the library catalog, indexes and abstracts, and
online databases, the following strategies are suggested
 Looking at government publications
 Surfing the World Wide Web, (Google scholar, Research Gate ,
University websites…and http://sci-hub.tw/
 Using the citations and reference lists of those who have gone before
you – track down any references that you see cited by three or more
other researchers because such references are clearly influencing
current work in your field and should not be overlooked. Whenever
possible, go to the original source and read it yourself since most
authors misrepresent the work of a particular researcher in the same,
particular way; apparently, they are reading one another’s
descriptions of that researcher’s work rather than reading the
researcher’s own words!
 Be systematic and thorough
 Relate bibliography to problem

48
Conducting a Literature Search
1. Write the problem in its entirety at the top of the page or
computer screen
2. Write down each subproblem in its entirety as well
3. Identify the important words and phrases in each subproblem
4. Translate these words and phrases into specific topics that
you must learn more about. These topics become your
“agenda” as you read the literature
5. Go to the source /library to seek out resources related to your
agenda
6. Read!

49
Evaluating, Organising, and
Synthesizing the Literature
 Too many literature reviews do nothing more than report what
other people have done and said
 We learn nothing new from such a review; we’d better off reading the
original articles for ourselves
 In a good literature review, the researcher doesn’t merely
report the related literature
 He or she also evaluates, organises, and synthesizes what
others have done
 Do not only read other people’s work but you must also critically
evaluate their methods and conclusions

50
Evaluating, Organising, and
Synthesizing the Literature
 Never take other people’s conclusions at face value; determine
for yourself whether their conclusions are justified based on
the data presented
 In addition to evaluating what you read, you must also
organize the ideas you encounter during your review
 The subproblems within your main problem should, in many cases,
provide a general organisational scheme you can use
 Looking at how other authors have organised literature reviews
related to your topic may be helpful as well

51
Evaluating, Organising, and
Synthesizing the Literature
 Finally, and perhaps most importantly, you must synthesize
what you have learned from your review
 In other words, you must pull together the diverse perspectives and
research results you have read into a cohesive whole

 Here are some examples of what you might do:

 Compare and contrast varying theoretical perspectives
on the topic
 Show how approaches to the topic have changed over
time

52
Cont…
 Describe general trends in research findings
 Identify discrepant or contradictory findings, and suggest
possible explanations for such discrepancies
 Identify general themes that run throughout the literature
 When you write a literature review that does such
things, you have contributed something new to the
knowledge in the field even before you have conducted
your own study
 In fact, a literature review that makes such a
contribution is often publishable in its own right

53
Writing LR - Guidelines
1. Get the proper psychological orientation
2. Have a plan
3. Emphasise relatedness
4. Give credit where credit is due
5. Review the literature. Don’t reproduce it!
6. Summarise what you have read
7. Remember that your first draft will almost certainly
NOT be your last draft
8. Ask others for advice and feedback

54
Don’t Reproduce the Literature!
 Review the literature. Don’t reproduce it – writing LR is
one of the most challenging works– requires that you keep
a clear focus
 Not simply quoting long passages or cite at length the
words or ideas of others
 Students consider LR as merely a conventional filler –
something that everyone does

55
 Don’t Reproduce the Literature!
 As important as what others say about their research, and perhaps even
more important, is what you say about their research
 Your emphasis should always be on how a particular idea or research
finding relates to your own problem – something that only you can
discuss
 Advice – guidelines
1. Present your own discussion
2. Paraphrase (precis (short statement of the main points of a speech or
piece of writing), resume (summary), give a synopsis, an epitome
(person or thing that is the perfect example of a quality or type))
3. Use short, direct quotations if necessary
4. Long quotations are a last resort. Use them only for a very good
reason – for instance, when the specific words that an author uses are
as important as the ideas that the author presents

56
Thus
 A literature review is a necessity.
 Without this step, you won’t know if your problem has been
solved or what related research is already underway.
 When performing the review:
 Start searching professional journals.
 Begin with the most recent articles you can find.
 Keep track of relevant articles in a bibliography.
 Don’t be discouraged if work on the topic is already
underway.

Taken from: Research Concepts by Chris Jones and Xiaoping Jia (minor modifications by Nayda
Santiago)
Literature Review Pitfalls
 Be very careful to check your sources when doing
your literature review.
 Many trade magazines are not peer reviewed.
 Professional conferences and journals often have each article
reviewed by multiple people before it is even recommended
for publication.
 The IEEE and ACM digital libraries are good places to start
looking for legitimate research.

Taken from: Research Concepts by Chris Jones and Xiaoping Jia (minor modifications by Nayda
Santiago)
Literature Review Pitfalls (cont.)
 The Internet can be a good source of information. It is
also full of pseudo-science and poor research.
 Make sure you verify the claims of any
documentation that has not been peer reviewed by
other professionals in the computing industry.

Taken from: Research Concepts by Chris Jones and Xiaoping Jia (minor modifications by Nayda
Santiago)
Development/revisiting of objectives,
Research questions and working
hypotheses

60
Cont….
 After comprehensive literature review there may be a need
to revisit problem statements, objectives, hypothesis
and/RQ

 If you do not have objectives yet it is time now to craft in

general and specific objectives form
 General objective is just one sentence in an a deliverable
oriented format and agreed with the title
 Specific objectives are list of action statement collective
achievement of which will enable the attainment of the general
objective.

61
Developing objectives and Working Hypothesis …cont’d

Some guidelines in developing objectives:

– Objectives must be specific, concrete and
achievable statements;
– The objectives should clearly fit to the
statement of the problem
– The objectives must propose to do things as
per the capability of the design of the study.
– Objectives should be in their approximate
order of importance.

62
Developing objectives and Working Hypothesis …cont’d

A hypothesis must be
– Specific
– Conceptually clear in terms of common definitions
– Testable (verification or rejection) by available
techniques and resources;
– Related to a body of theory;
– Stated to provide direction for the research;
– Formulated as causal relationships;

63
Cont…
 Often PhD dissertations/thesis fail to make explicit their
hypothesis / thesis.
 Sometimes the reader can hardly “find” them implicit in a
section of “contributions” of the dissertation.

 Research question
 In cases where we don’t have a hypothesis, a problem
statement should also end with a research question
 Putting a the problem in a question form so that it guides the
research process.
 How many RQ
 You may have 3-5
 Some says research may only have one
64
RQ example

65
Step 3: Research Design
 If you fail to plan, you planned to fail !
 It is the entire design/plan of the research project
 It is said as the research proposal
 It involves deciding on all aspects of the research process, of course
focusing on methodology
 Includes planning in detail all the steps of the experimental phase.
 In engineering research it often includes the design of a prototype /
system architecture.
 That includes
 Philosophical assumption, General Research methodology
(approach)-Specific method, Data collection techniques, Data
analysis, and solution development, Evaluation/validation
 Publication outlet if possible
 It matters to get your proposal accepted or rejected
66
Cont…
 Research methodologies reflect high-level approaches in
conducting research.
 The individual steps within the methodology might vary
based on the research problem being performed.
 There are three broad approaches to be used in research design
(alone or in combination)
 Quantitative.
 Qualitative.
 Design science

 More on chapter 3

67
Research Design helps in many ways
Design of research …cont’d

– It sets up the framework for adequate tests of

relationships among study variables.
– It tells what observations to make/what
variables to measure, how to make
them/measure them, and how to analyze the
data;
– It suggests how many observations to make
and the type of statistical analysis to use;
68
• The preparation of the research design,
appropriate for a particular research problem,
involves usually the consideration of the
following.

• The time available for research

• the finance available for the purpose
• the availability and skills of the researcher and his
staff (if any)
• Identifying the study variables
• Identifying study subjects / experimental units,
• Planning the means of obtaining the information
and reasoning leading to selection.
• Sampling

69
Step 4: Develop
Instruments/procedures
 Can also be part of the research design
Instrument
 Two possibilities ; adopting from literature and crafting based on
objectives /RQ
 There are different data collection Methods/sources

1. From human data source : Questionnaire , Interview

(structured and unstructured interview, Observation
2. From non-human source- Extracting methods
1. Computer log files, Web files
2. Document analysis
3. Daily transaction records, Speech files
4. Camera (for security, face recognition etc…
70
Cont…
 For survey and other qualitative researches
 Develop your instrument to measure your variables
 You can get some instrument from published articles
 One variable can be measured by multiple questions
 Check the instrument if it is useful for your context: add new
questions or remove irrelevant ones
 Use statistical tools to check instrument validity
 Undertake pilot testing before you use your instrument in the
actual survey
 Statistical tools can be used like Cronbach Alpha and
Reliability Tests

71
Cont…
 Procedures
 For most experimental and design researches
 Data sources

How? How many ?

 How to make the data/ requirement ready?
Process and techniques
 How to make and measure the design/analysis?
 Etc…

 More on Chapter 3

72
Step 5: Data Collection and Analysis
 Data Collection
 From human or non-human sources
 Identify systematically and in a rational manner
 Follow and adhere to the research design
 Train your data collectors if you use data collectors
 Monitor data collection in the field
 Wrong data will result wrong conclusions
 Check your data if it is taken from databases for reliability

73
Cont…
 Analysis
 The data that were gathered are analyzed in ascertaining their meaning.
 Quantitative researches Involves statistical analyses, for the most part.
 The specific statistical tests and procedures depend on the nature of the
underlying data.
 Machine learning approaches /experiments
 In Design researches -Identify design inputs and evaluation criteria,
Implementation of methods (e.g. prototyping) and auxiliary tools (e.g.
simulation)
 Qualitative researchers use words to understand and describe the
phenomena
 Coding /pattern matching /narration

 More on chapter four

74
 Step 6: Result/solution design Interpretation
 The researcher discuss/interprets the newly analyzed data or provide
solutions (implementations) and suggests a conclusion.
 Interpretation means identifying relationship between variables/and
or other works to draw conclusions
 Provide lessons from designing a solution or doing experimentations
 Evaluation of the results
 Meaning of Testing hypothesis if any – Reject or Accept
 Provide explanations for accepting or rejecting / how novel or new
is the result….

 Example :The Data for SW success shows low figures, below our
expectation. What it tells us! Does it mean the SW failed? What
causes the Failure? How do we overcome those Failures?
75
Cont…
 What did your analysis/experiment/design show?
 Discussion in light of Literature, Research objectives and
Research questions.

76
Cont…
 The evaluation /testing in experimental and design research
will indicate further work too.
 This may lead the researcher to cycle back to an earlier step
in the process and begin again with a new hypothesis.
 This is one of the self-correcting mechanisms associated
with the scientific method.

 More details in chapter four

77
Step 7-Proposal /Report writing
 Last step?????????
 There are basically two main documents in the process of
a research.
 A proposal
 A final report (conference paper, journal article, thesis,
dissertation…)
 Slight tolerable difference on the format content and
structure.

78
Cont…

Will be covered in the chapter 5

79
Final Words

80
Review questions
 Where can we get a research problem – topic?
 What is the purpose of reviewing literature?
 Explain the major steps in a research process?

81