System Engineering

Research Methods
original paper by: Gerrit Muller
(https://www.sciencedirect.com/science/article/p
ii/S1877050913001166)

Presentation by Oka Mahendra

 Background
• Chalenge in system engineering research:
the application happen "in the field"
• The field can be an industrial company or
a government agency.
• Researchers need methods to research in
the field; methods to try-out ideas, collect
data, analyze, and evaluate.
Industrial problem to validate
research
Research Relevancy
Spectrum of Research Methods
 Collecting Data
• Loogbook
– Entries always need a date/time stamp and then the
what, how, why, when, where, and who key words
• Observation template
– Support note taking
• Structured Data Collection
– to facilitate further research
– collect background information, understand its
accuracy and possible constraints
 Extracting data from other people
• Interviews: free format, standardized
• Surveys
– Questionnaire
– Presentation data
– Evaluation
• Structured reports
 Questionnaire,
presentation, and evaluation
 Producing Artifacts
• Validating the claim by producing:
– prototype,
– proof-of-concept implementations,
– models, or simulations
• Artifacts form:
– sketches,
– block diagrams,
– spreadsheets, and
– models.
 Validation Challenges and Pitfalls
• Challenges:
– formulate a proper claim,
– measure or observe relevant data for evaluation, and
– analyze and interpret properly
• Important to scrutinize:
– Quality and interpretation of data
– Validity and applicability of literature
– Cogency of arguments and reasoning
– Assessment of the final result
– Generalization of the outcome
– Pitfall observed in practice: respondent, model,
researcher, circumstances
 Conclusions
• Research methods steps:
– a clear articulation of the current situation and
problem
– transform the problem into objectives
– improve the current situation: new ideas, methods, or
techniques
– evaluate the effectiveness objectively:
• articulating a claim or hypothesis with criteria
• observe, measure, and collect data
• Research in system engineering is handicapped
by the uniqueness of every field situation