BUS3003 W5L3 LECTURE 11 (SLIDES 15) 13/03

(WK5L2 12/03 NO VIDEO)

ETHICAL CONSIDERATIONS

INFORMED CONSENT

- Doesn't have to be physically signing a form – can be “by clicking next you agree to
partake”
- For an experiment, it has to speak on the nature of the treatment – what will yo ube
exposed to in the experimental condition and what will or will not be available to an
individual in the control group (brings element of deception)
- Researcher may decide to withhold certain information – has to go through an ethics
committee for approval and has to be debriefed at the end
- How groups will be made – ideally it should be random assignment but telling them and
using random assignment also minimises some resentful feelings (you can walk away
feeling that there wasn't a bias attached to it – you had an equal chance as the next person
to be part of the experiment
- If someone chooses not to participate or they want to withdraw, are there any available
alternative treatments
- Compensation - both groups or only one – raises flag bc are participants really willing to
participate in the study if that compensation wasn’t provided (a lucky draw is a work
around)

All of these need to be mapped out for informed consent, in an experiment and to minimise any
risks (they know what they are signing up for) and you are protecting yourself as a researcher

VOLUNTARY PARTICIPATION

- Aligns somewhat to compensation

- Want people to volunteer – no forced participation
- Acknowledge possible issues around threats to external validity to come into play – bc a
certain type of individual is more likely to participate (those who go the extra mile and
you can argue that this gives you a sample that is a lot similar in some ways)

PROTECTION FROM HARM

 - Informed consent is the starting ground to protecting participants – clearly outline what
experiment entails bc they know what they are signing up for
- Can be physical (strenuous)or mental – experiment forces them to reflect on negative
feelings, memories or emotions
- Also extends beyond experiment, refer them to resources if you know that the constructs
that the researcher is exploring could illicit some uncomfortable feelings, have some
support and debrief, so that they are not left uncontained
- Word questions carefully and sensitively
- Run a pilot study – to get feedback and improve the actual main study

ANONYMITY AND CONFIDENTIALITY

- Difference between the two
- Anonymity – what we want as a researcher, there should be no way for you to link
someone’s individual response to who they are (no personally identifiable information)
- Confidentiality – more around how do you safeguard the info that you do have, people
who are not involved in the research should not have access to raw data, don't hold on to
info that is no longer useful – get rid of it
- Should be able to practically answer how would you ensure that these two things are
achieved, separately

DECEPTION

- Has to be strongly justified

- 3 conditions (all must be present) where deception is acceptable/appropriate
- 1. no other non-deceptive method exists to study the phenomenon
- 2. study should have significant potential to contribute
- 3. it shouldn’t cause significant harm or emotional distress

When you justify this, you have to show that: Violations are sometimes justified by arguing that
benefits to society outweighs risks to participants

CASE STUDY

What is the research strategy (and how do you know that/justify answer)?
 - Words like causes, effects, leads to = may be an experiment bc it is looking to draw cause
and effect conclusions and manipulation and intervention of the IV is going to cause
something/change in the DV
- Make reference to the case study to substantiate
- Some researchers are relaxed in their writing, and it could’ve been correlational or
descriptive they can still use words like causes
- It is an experiment bc “randomised control trial”

What is the research design (and how do you that/justify answer)?

Remember for research design, you always comparing whatever you’re seeing to the 4 elements:
Manipulation, Measurement, Comparison & Control – to what extent are all 4 of these present
and what does it mean in terms of the design?

- Between groups, pre, post test experiment

- Manipulation of single IV – four-week mindfulness program
- Measurment of DV - pre and 2 post test
- Comparison of control and experiment group
- Control

Key here is to unpack each one and then reference the case study/contextualise you answers

If all four examples are not present, discuss those that are and mention those that are not present
and why it is not a true experiment

What is the design notation?

- Read up and go to hotseats for this!!!!!!!!!

BUS3003 WK6L1 LECTURE 12 (SLIDES 16) 18/03

VARIATIONS OF EXPERIMENTAL DESIGNS

 - Experiment = all 4 elements present (manipulation, measurement, comparison and
control
- true experiment = control is achieved by random assignment, if not, it may be a quasi-
experimental design
- An experimental design always involved comparison of scored from at least two
treatment conditions
- Within experimental designs there are two variations:
o Independent Groups Design (between-subjects):
o Two groups: experimental and control
o Participants cannot belong to both (independent)
&
o Repeated Measures Design (within-subjects):
o same participants in every condition of your experiment or at every time period
that you are measuring (so you could either be exposing participants to more than
one treatment or condition)

o What is key between these two variations of experimental design is that there is
always a comparison across treatments or conditions
o Summary: in experimental designs we are always comparing participants, if they
are different (belong to different groups) = independent/between-subjects, if they
are the same across each intervention/treatment and then you’ll compare which
one is most effective for example = repeated measures

BETWEEN-SUBJECTS DESIGN / INDEPENDENT GROUPS DESIGN

- Obtaining scores from two groups – control and experimental

- In exam/test:
- First is it experiment or not? If not, is it non-experimental or quasi-experimental?
- Then is it between-subjects or within-subjects?
- Is there a pre-test and a post-test or is there just one?

Distinguishing Features of Between-Subjects

- Compares scores of different groups of individuals

- Independence of scores (no overlap of group scores’, groups kept separately no
contamination for true results from the intervention)
Advantages

- Once between-subjects experiment is executed, it is simple to analyse the data and run
the analyses
- If we have random assignment, measurements that we see/record are uncontaminated
by other treatment factors/explanations for what you are observing (controlling for the
threats to internal validity)
- Enables you to make clear comparisons between the two treatment conditions (bearing in
mind that the control group is receiving nothing, and you are observing/controlling as
much as possible for individual differences amongst the two groups)

Disadvantages

- Used probability sampling, then randomly assigned individuals to the two groups and we
can say with a fair amount of confidence that we’ve held the variables constant
(minimised external factors influencing, group equivalence), but you can’t be 100%
certain that you’ve controlled for everything, there are risks for individual differences
that you are unaware of or is unaccounted for (especially in new research that is
underdeveloped with little research/theory)
- You need to recruit many participants bc you need different participants for
experimental and control groups and for random assignment and group equivalence to
work well

Threats to Internal Validity in Between-Subjects Design

- Remember that it must be applicable or relevant to the case study

- Yes, we use random assignment to eliminate some systematic variation, individual
differences but due to the nature of the design there could still be some threats at play
(such as:)
o Subject related bias: individuals in the experiment or control essentially catch
wind of the intention or design of the experiment, participants are altering their
behaviour in some way, picking up on cues and you may see that it is present
when scores are under/over-inflated depending on the group (think double-blind
experiment where both the researcher and the participant don’t know which group
they’ve been assigned to)
o Compensatory rivalry: similar to above but they are behaving in a particular way,
participants feel competitive that they aren’t receiving the experimental treatment
and report high levels as the experimental group without actually needing the
experimental treatment
o Resentful demoralisation: applicable to control group: they feel resentful that they
are not receiving the experimental treatment and they do not provide responses
that truly reflect = under-inflation of how they truly show up
 o Compensation (or compensatory equalisation): you could compensate both groups
which equalises the threat to internal validity, but this is not always the case,
sometimes only control group is compensated bc researchers feel bad that they
don’t receive anything, and this kind of compensation can result in an inaccurate
reflection of scores
Solution = double blind experiment or random compensation (lucky draw)
Above 4 points = Experimental effects or social interaction threats
o Differential attrition or participant dropout: this is problematic when if there is an
unequal rate of dropouts between the two groups bc individual differences are the
introduced bc the groups are no longer equivalent (what you may have controlled
for in the beginning has now gone out of the window or void) - you would
disclose this in your discussion chapter, limitations, acknowledgements

WITHIN-SUBJECTS DESIGN / REPEATED MEASURES DESIGN

- Same participants being exposed to every treatment condition, which means that
participants are serving as their own control group – there scores at intervention 1 will be
measured against 2 and against 3 (no need to be concerned about group equivalence)

Distinguishing Features

- Own control groups

- Assess when no treatment was given and then when treatment was administered
- Same participants across multiple treatment conditions
Advantages

- Bc you are using the same participants across treatments, you essentially need fewer
participants
- Group equivalence is not a concern, individual differences have been controlled for bc
you are just comparing the same people to themselves

Disadvantages

- Confounding environmental variables (differences): you need to ensure that the

surroundings/setup/nature of experiment (e.g.: venue) is the same across all treatment
conditions which is difficult to achieve
- Confounding time-related variables: bc measurements happens over-time things can
happen between measurements

- History = external event that influences participant scores (the whole group not
individually)
- Maturation = how do people naturally mature and evolve over time
- Testing effects = fatigue effect = mental or physical, and practice effect = for example
treatments are too close together
- Implementation = how instruments are used and changes in a study (a survey or
observation) - speaks to standardisation

In a test or exam, you must argue and show how it is applicable.

BUS3003 WK6L2 LECTURE 13 (SLIDES 17) 19/03

NON-EXPERIMENTAL AND QUASI-EXPERIMENTAL DESIGNS

- Sometimes conducting a true experiment is not feasible due to logistics and practical
nature of the study

Definitions:

- Non-experimental: Aim to demonstrate a relationship between two variables by

comparing different groups of scores but make no attempt to minimise threats to internal
validity or explain the relationship.
- Groups not randomly assigned, there may not be manipulation, several elements of the
FOUR defining features (of a true experiment) may not be present – deducing a cause-
and-effect conclusion may not be possible
- E.g.: differential research design, post-test only non-equivalent control group design, pre-
test post-test design.

- Quasi-experimental: Attempts to limit/control threats to internal validity (compared to

non-experimental)
- Random assignment may not be used but you may have used matching, counterbalancing,
there may still be a control group – this is so that you can draw some cause-and-effect
conclusions, but it is still limited
- E.g. pre-test post-test non-equivalent control group design, time series design

- In an exam question: how do you strengthen the research design? You want to strengthen
it to such an effect that you can draw cause-and-effect conclusions
i.e.: how do you make it as similar as possible to a true experiment that lacks random
assignment (maybe you introduce matching for example etc.)

Non-Equivalent Group Design:

- Differential Research Design:

- You could still have two independent groups (between subjects), participants don’t
belong to both, but the way they were formed was not through random assignment
 - Compare pre-existing groups (example: male and female employees or groups based on
years of experience; you cannot assign these groups), however, participants are not
randomly assigned

- Post-test only non-equivalent control group design:

- You don’t have a pre-test and a pre-test is important and beneficial bc it gives you a
baseline score of what participants were reporting before intervention
- A baseline also allows you to identify if there are any individual differences before you
intervene

Threats to internal validity

- Individual differences bc we don’t have equivalent groups which could be confounding

variables that could explain the observed effect and not necessarily your intervention
- There is still ambiguity in your conclusions – it is not clear
- One way to strengthen the design is to introduce the pre-test – gives you baseline and
indication of any individual differences
- So, this was previously just a post-test non-equivalent design, but with the introduction of
a pre-test, it now becomes a pre-post-test non-equivalent between-subjects design

Pre-Post Design (within groups)

 -
- Could be within groups – same participants assessed at more than one point in time

Time-Series Design

-
- Quite a number of observations before and after the intervention
- Becomes a Quasi-experimental design bc if you are assessing participants several times
before and after the intervention, you can still to a certain degree, discuss or conclude
 how the construct does change over time and there is a limit to the cause and effect nature
of it but it is a lot stronger than just having one observation before and after
- Key: how many observations are before and after the intervention
- Here, there are still elements present (manipulation, comparison and measurement), the
only ones that are missing is random assignment and no control group (no control = not a
true experiment)

NOTE: Exam tip – a non-experiment would have the bare minimum, no random assignment,
only a post-test

Threats to Validity (time-series – when you have several observations before and after)
- History
- Maturation
- Instrumentation

Threats to Internal Validity (Non-equivalent and pre-post designs)

- Selection bias = always applicable to a non-equivalent and only a post-test design bc

groups are not equivalent – individual differences that you have not accounted for and
controlled for which could affect the outcome you are testing
- Testing effects = they could pick up or become familiar, they could also become tired
- Mortality/Attrition = only applicable to non-equivalent (not time-series) if the dropout
rate between the groups is considerable different bc it introduces more variation of results
being due to chance, of individuals differences becoming heightened

Summary
 - This shows how you moved from a non-experimental design to a quasi-experimental
design
- Takeaway from this slide: within non-experimental and within quasi-experimental you
could have between-subjects design or a within-subjects design and within those what
would it look like
- Remember for within-subjects design you are looking at the same participants, there isn’t
another group
- You introduce more observations for a within-subjects design to go from non-
experimental to quasi-experimental
- You introduce a pre-test for a between-subjects design to from non-experimental to
quasi-experimental

BUS3003 WK6L3 LECTURE 14 (SLIDES 18) 20/03

PRACTICAL EXAMPLES/CASE-STUDY APPLICATIONS – GOOD TEST PREP