Research Methods / Research Methods I

PhD@DEI / PDCTI / PDEB

2021/2022

Luís Paquete
University of Coimbra
Introduction

Teacher of RM 1/RM (PDCTI/PDEB): Teachers of RM (PhD@DEI):

● Luís Paquete ● Luís Paquete (module 1)

E-mail: paquete@dei.uc.pt E-mail: paquete@dei.uc.pt
Office: D.2.18 Office: D.2.18
Attendance: Thu, 9h-14h, zoom Attendance: Thu, 9h-14h, zoom

● Licínio Roque (module 2)

E-mail: lir@dei.uc.pt
Office: D.3.10

Use UCstudent to access material and to mark your presence in the classes
Zoom link for classes: https://dei.uc.pt/vc/phd
Slack RM2021:
https://join.slack.com/t/rm2021-group/shared_invite/zt-wkfsrec5-YXqSSsCkgHw8S_G52FjXnw
Introduction

Goals (RM1 and RM – module 1): to acquire knowledge and skills on experimental
methods for the empirical assessment of computing systems in terms of performance,
availability, security, etc.

Topics:
1. Experiments in computer science
2. Sampling and measurements
3. Exploratory data analysis
4. Linear regression
5. Confidence intervals
6. Hypothesis testing
7. Contigency tables
Introduction

Assessment RM 1 / RM Assessment RM (PhD@DEI):

(PDCTI/PDEB):
Module 1
● Written test and Exam (50%) ● Written test and Exam (25%)
● Group assignment (50%) ● Group assignment (25%)

Module 2
● Assignments (50%)

Requirement: Both modules with positive grade

Introduction

Assignment

● Three milestones:

9 November: Exploratory data analysis and linear regression

23 November: Pre-registration of hypotheses

20 December: Hypothesis testing
● Oral defense in 5 January
● The output of the each milestone is a written report
● Up to 3 students per group (ASAP!)
Introduction

Programming language R

● Interpreted programming language for statistical computing and graphics

● Free under GNU GPL v2 licence
● Command line interface, GUI (RStudio), Jupyter
● 17473 available packages (April 2021)
● Avaliable at http://www.r-project.org (see also https://cran.r-project.org)
● See Introduction to R at UCstudent and follow the tutorial
Introduction

Bibliography:

● D.J.Lilja, Measuring computer performance, Cambridge University Press, 2002

● P. Cohen, Empirical methods in artificial intelligence, MIT Press, 1995
● R. Jain, The art of computer systems performance analysis, Wiley, 1991
● D.G. Montgomery, Design and analysis of experiments, Wiley, 2001
● C.C. McGeoch, A guide to experimental algorithmics, Cambrigde University Press, 2012
● J.-Y. Le Boudec, Performance evaluation of computer and communication systems, EPFL,
2021 (http://perfeval.epfl.ch)
● NIST/SEMATECH, Engineering Statistics Handbook:
https://www.itl.nist.gov/div898/handbook/