Professional Documents
Culture Documents
1
Research Methodology Empirical cycle
• Types of research • Hypothesis
• Conceptual research design • Experiment
– Research objective • Evaluation
– Research questions
– Definition of concepts
• Research planning “is it true? (and why?)
• Design
• Simulation/implementation The engineer, and more generally the designer, is concerned with how
things ought to be - how they ought to be in order to attain goals, and to
• Evaluation function … With goals and "oughts" we also introduce into the picture
the dichotomy between normative and descriptive. Natural science has
“is it possible?” (and how?) found a way to exclude the normative and to concern itself solely with
how things are … Artificial things can be characterized in terms of
functions, goals and adaptation.
Note: In CS, “implementation” can range from model to Research methods: simulation and decomposition
proof-of-concept prototype to full implementation
Nov 2006 Hans Weigand 9 Nov 2006 Hans Weigand 10
theory-oriented practice-oriented
Nov 2006 Hans Weigand 11 Nov 2006 Hans Weigand 12
2
Research Methodology Research objective
• Start exploring the project context
• Types of research
• Formulate research objective
• Conceptual research design
– useful (added-value)
– Research objective
– feasible
– Research framework
– clear
– Research questions
• The objective is to … by …
– Definition of concepts
• Research planning your contribution your overall result
3
Example: vd Heuvel (2002) Research questions
“a methodology for objectifying legacy systems”
1. Problem definition Explorative, desk research • The questions that you state yourself for getting at
your objective
2. Related research Literature review
• Main question/subquestions
3. Design of solution Meta-modeling
• Requirements:
4. Validation of - Logical consistency metamodel
– Effectiveness (do you reach your objective?)
proposed solution - Implementability prototype
– Efficiency (balance between goal and means)
- Plausibility field experiment
5. Assessment of – Steering function
• What type of knowledge is required?
research results • What material needs to be gathered?
• Research material
– People, documents, observation research research research
– Data sources, knowledge sources question I question II question III
• Research strategy
– Survey, experiment, case study, grounded
theory approach, desk research, … r esear ch met hod A r esear ch met hod B
(e.g. case st udy) (e.g. mat hemat ical
• Time planning pr oof ) r esear ch met hod C
(e.g. exper iment ing)
Arrows represent your choices
Nov 2006 Hans Weigand 23 Nov 2006 Hans Weigand 24
4
Concluding remarks on research
What is IS/CS research?
methodology
• Research needs preparation (GIGO)
• Be aware of the choices that you make Is CS a science?
• During execution, the framework should
always indicate the purpose of what you are IS CS an engineering discipline?
doing at that moment. Is it a formal science, branch of mathematics?
• As research is a creative process, adapting
the framework on the way is natural and
should always be possible.
Nov 2006 Hans Weigand 25 Nov 2006 Hans Weigand 26
Wieringa/Heerkens March/Smith
• Characterizes CS as design science • CS/IS research should integrate design and
• Design is aimed at solving some problem (action “natural science research”
problem, world problem) • Design is aimed at creating things
• The design itself is not research (contra Popper), • Basic design activities: build and evaluate
but the other steps in the intervention cycle give
rise to knowledge problems that can be addressed • Natural science is aimed at developing theory –
by scientific research not restricted to natural phenomena, can also apply
• Sometimes solving a knowledge problem requires to artifacts (contra Simon)
solving an action problem • Basic science activities: theorize and justify
model
theory problem world problem
met hod
-lack of t heor y
-lack of validit y action problem
action problem
inst ant iat ion -f alsif icat ion at t empt
(wor king pr ogr am) -anomaly
5
Ramesh et al Why mathematics?
• Based on empirical study of publications • Conceptual clarity
– like precise definitions in Law
• Focus in CS is on formulating (80%) rather
– Could perhaps be reached also by UML diagram
than describing or evaluating
• As a research method
• Research methods: mainly conceptual/ – to validate certain claims
mathematical • For the development of a (predictive) theory
• Low reference to other discipline (some – not really different from e.g. game theory in Economics
mathematics) – Example: relational algebra, complexity theory, ..
Pragmatism
Not in the sense of opportunism, but concerned with
Critical theory
action rather than with being Ch. Peirce
“There is no such thing as genuine knowledge and fruitful • Knowledge is not neutral - be critical about
understanding except as the offspring of doing… Only by assumptions such as the technical
wrestling with the conditions of het problem at first hand, imperative to improve efficiency
seeking and finding his own way, does he think“
John Dewey • Knowledge must be legitimated by
“Thinking is not different in kind from the use of natural
materials and energies ..” consensus and democratic discourse
Knowledge of a thing is knowledge of what you can do with it. • Knowledge is not neutral – science should
Tries to bridge empirical science and design science. serve social goals such as emancipation
6
Research as communication So …
• Communicate! (publish, discuss, review,…)
• Researchers communicate via papers, • Be clear! (Grice’s conversational maxims)
presentations, discussions • Take communication partners seriously!
• In communicating, they make descriptive – Read related research
– Try to understand
and normative statements (validity claims)
– Refer
• These claims may be challenged by others, – Consider their goals and expectations
leading to discussion • Don’t “drop an idea”; try to convince!
• Discussion is as essential as research • Justify your claims!
activities itself
Nov 2006 Hans Weigand 37 Nov 2006 Hans Weigand 38
Maxim of Quantity:
1. Make your contribution to the conversation as informative as necessary. How to justify your claims?
2. Do not make your contribution to the conversation more informative than
necessary.
Maxim of Quality:
"Ik ben er nog steeds van overtuigd dat
1. Do not say what you believe to be false.
zoals ik het doe je het moet doen want
2. Do not say that for which you lack adequate evidence.
anders zou ik het niet doen."
Maxim of Relevance:
Be relevant (i.e., say things related to the current topic of the conversation).
Maxim of Manner:
1. Avoid obscurity of expression.
2. Avoid ambiguity.
3. Be brief (avoid unnecessary wordiness).
4. Be orderly.
Literature
How to justify your claims? • Designing a research project – P. Verschuren, H.
Doorewaard. Utrecht, 1999.
• Grounding (empirical, simulation, …) • Desiging Information Technology in the postmodern age – R.
Coyne, 1997
• Careful reasoning • Qualitative methods in management research – E.
Gummesson
• Examples (not just for explanation, but to put your claim to the test)
• The sciences of the artificial. H. Simon, 1967
• Delimitation • Research in Computer Science: an empirical study
-Ramesh et al, 2004
• Be clear about the status of your claim. • Design and natural science research on information
technology - March, Smith, 1995