Experimentation

1) Experiments on People
2) Experiments without using people

 Experiment starts when you start manipulating something, example,
testing website is done in its own environment.
 Survey can be used to collect data after experiment though it, itself
is an experiment.
 Process involves> Identify issues> Review literature what other
people think> Come up with your own hypothesis questions>
Analyse > Accept or reject> Report> Analyse> conduct survey>
verify independent; button and dependent things; the outcomes
what are you going to change.
 Question Types: What is the impact, and what is the impact of of X
on Y> Descriptive; What is happening
 When to use experimentation; Testing algorithms, ideas in a
simulated world; giving different information, testing parameters,
lab experimentation such as usability testing
 Algorithmic Experiments: Similar principles:
a) Base case idea, control and experimental
algorithms
b) Few ethical issues if not experimenting
personalized data.
c) Simulation involves representation of the world,
can simulate many objects such as more details or
less details, Useful when real world experiments
impractical; large infrastructure changes and
check how many people use it. For instance,
building a freeway cannot be done.
 Example of Project: Where people were going; like short
distances? Driving themselves and driver 4 seater, 7 seaters, were
they travelling for shopping, recreation or work etc. Result was as
everybody gets to micro cars; type of vehicle, travel time changed;
so, everybody will get to their destination on time.
 Takeaways: Sit down and think about careful design, make a
change and what is the effect of change, what external factors can
affect your objects, think about validity, form research questions
carefully, check how simulations and experimentation relate to the
real world.

 In simulation: you have control on everything

 Classic experimentation; things that you are changing; independent
variable example is the treatment; sugar pills etc.

It shouldn’t be all people with extreme views. experiments of sleep derivation. making sure that the group of people are representing people of what you are looking for. the other group that doesn’t get the treatment. The next people know what they are going to test. leaves the users to focus on putting more diligent efforts. Leaving a significance on your experiment for example the popular things is favoured. sugar pills etc. b) External validity: Doing surveys and the medias effect on the research. Selection bias. You get them divided into two.  Testing effect: Interface testing.  You should be able to judge the validity of experiment: a) Internal validity: Things that impact the outcome.  Statistical regression effect: When you choose your sample.  Perceiver Effect: This can help you will actually help leaves a good effect though it is useless. the group that gets treatments. when people are already familiar with one version and they test the second version of interface. Testing in lab. Dependent variable is what you are measuring for example.  Diffusion: You did testing and you tell other people what the testing was. Example of testing effect.  Demand Effect: You need to try hard. logic testing. real pill vs sugar pill  Random samples such as testing sample of University students. abstraction is an idealization  Internal Validity:  Selection Bias  History/Context  Maturation  Testing Effect  Drop-outs  Instrumentation  Statistical Regression Effect  Diffusion  Compensating Effect  Experimenter Expectations  Placebo Effect  Experimenter Demand Effect  External Validity:  Generalizability: Population .  Is your lab effect going to evoke people to work. you don’t want to involve only IT people. So the experimenter needs to be neutral. Example. example is the treatment. they feel more comfortable. some other thing that interferes with one.  Generalizability: Generalizable result out of the small group. it should be average.

 Is it realistic  Do Subject react naturally  Does your abstraction/ theory map reality accurately  2 Models:  Theoretical: Does your theory model work in the real world?  Empirical: What happens to X when you change Y  Experimentation: Information Security and Personal Privacy Examples Case study  Goal: Identify the target group based on age. demographics and high risk  Risks: a) Wi-Fi Hotspot Awareness b) Social Media Privacy Settings c) Encryption of Data d) Use of tunnels e) Difference between HTTP and HTTPS f) Mobile security & applications g) Downloading from untrusted sources h) Fake Calls to obtain account pin i) Email security  Target Groups:  Non-Technical People  Elderly People and Youth  Metropolitan People  Rural  Customers  Middle age people  International users with different awareness levels Experiment  Having a testing room with their own laptops  Connect to internet  Email notification: Spammed .

 Open some application and payment page with so much detail. models.  A Dummy phone call asking their banking details. methods. Compare two methods such as survey and interviews. conclusion: this is better. What will you change and how will you measure?  Scoping Review: How they researched on the topic you are covering: compare a contrast. how they find out what they find out. What other people. Don’t talk about the topic too much . did they use experiments. asking open your bank app. Model 1 Theoretical approach 1 test their model. ways and what is my topic (3-5). or resetting password. how many people did they covered.