You are on page 1of 12


PSY1016 Week 8
Non-experimental Designs

Experimental Designs
Experiments involve the manipulation of an Independent Variable Generally laboratory-based
Examine cause-effect relationships High control Low ecological validity

Not always an option and, even where possible, not always the best option


Non-experimental Designs
Common data collection methods
Naturalistic observation Surveys Questionnaires

Common study designs

Correlational/Cross-sectional studies Longitudinal studies

Advantages of Non-Experimental Designs

Naturalistic research setting
Higher ecological validity
Variables are examined in more realistic, complex environment

Necessary where variable manipulation is impossible or unethical

Research using quasi-independent variables (e.g. Gender, ethnicity) Research into the effects of divorce on children


Advantages of Non-Experimental Designs

Better for certain types of research question
Assessing relationships between multiple variables and their complex interactions
Mediators and moderators

Identifying predictor variables

E.g. Which variables best predict school grades?

Developing & refining measures

Concurrent validity Conceptual structure (e.g. how many factors does a scale have?)

Assessing change over time

Which factors most determine memory loss in old age?

Non-experimental Designs: Data Collection

Naturalistic observation Surveys


Naturalistic observation
Observation of individual or group in their natural setting
Descriptive Can produce quantitative or qualitative data Researcher aims to limit their effect on the participant(s)
Avoid reactivity i.e. changes in participant behaviour due to being observed

Naturalistic observation
Amato (1989) observed caretakers of children in public places in California
43% had male caretakers Females more involved in caretaking in restaurants, males in playgrounds

Graham & wells (2001) observed aggressive behaviours in a Canadian bar

75% of the incidents involved only males 33% of incidents occurred outside the bar Also conducted interviews with bar customers:
Triggers for aggression included problems with bar staff, rowdy behaviour, and interpersonal relationship problems


Attempts to describe populations from samples Questions may focus on:
The British Social Attitudes Survey has monitored and interpreted attitudes towards social, economic, political, and moral issues annually since 1983

The Youth Risk Behavior Survey assess the health risk and health protective behaviours of American adolescents (e.g. drug and alcohol use, exercise, diet, etc.)

age, gender, income, ethnicity, etc.

The usual issues with respect to self-report measures apply
Social desirability Memory errors Interpretation of responses Clarity of questions Leading questions

If results are to be generalised, the sample must be large and representative


Non-experimental Designs
Cross-sectional/Correlational Longitudinal

Correlational/Cross-sectional Design
Essentially involves taking measurements at a single point in time with the aim of assessing relationships between variables
The minimum requirement is two variables To gain more insight it is common to include more variables
Introduce controls (similar to the use of control procedures in experimental designs) Assess intervening variables


Correlational studies: Interpretation

Correlational studies: Interpretation

Imagine a study finds a positive relationship between male baldness and marriage duration Could be interpreted as:
Long marriages make men bald Going bald keeps your wife happy Male baldness and long marriages are both strongly associated with a third variable: age


Correlational Studies: Control

The previous example demonstrates one of the reasons it is common to include extra variables that might be related to your variables of interest
The researcher can partial out the influence of age by including the variable in the study A partial correlation between baldness and marriage duration can control for the effect of age on both variables
This should reveal the true relationship between baldness and marriage duration (i.e. probably none, making age a mediator variable)

Issues in Analysis: internal reliability

Reliability of measures refers to their
Internal consistency Stability over time

Internal reliability can be measured using a statistic called Cronbachs alpha

It is an average of all possible split-half reliability tests It is generally considered acceptable if a scale has a Cronbachs alpha of at least .7 The issue for cross-sectional studies is that the reliability of the measures used limits the maximum correlation that can be found


Issues in Analysis: internal reliability

For example:
Imagine we find that the Cronbachs alpha for Stigma Scale (AQ-27) is .80 This means that the maximum correlation co-efficient you could find for the relationship between Stigma and any other variable would be .80 If the relationship is actually stronger than that, we wont be able to detect it We may be systematically underestimating the strength of its relationships with other variables

Issues in Analysis: internal reliability

Lets suppose that we are conducting a study to examine the relationship between Mental Illness Stigma and Authoritarianism. We conduct a reliability analysis and find that
Cronbachs alpha for Stigma Scale is .80 Cronbachs alpha for Authoritarianism is .70

The maximum correlation we could find between them can be calculated as the square root of their product


Issues in Analysis: internal reliability

In this example the maximum correlation coefficient we could find is:
(.8 x .7) = .56 = .75

Whatever the relationship between these two variables is in reality, this (.75) is the maximum we could detect In fact, all estimates of the relationship between the variables will be underestimates compared to what we could find with perfectly reliable measuring instruments

Issues in Analysis: internal reliability

However, it is possible to make an adjustment to your detected correlation co-efficient so that it takes account of the scales imperfect reliability
Take the correlation co-efficient (r) you found and divide it by maximum r
E.g. If we found that stigma and authoritarianism were positively related at r = .4 We divide this by the maximum r (.75) To give us a figure based on perfectly reliable measures
.40 / .75 = .53



Longitudinal Designs
The essential feature of a longitudinal survey is that it provides repeated observations over time on a set of variables for the set of persons belonging to the survey In some longitudinal designs, the same sample (or panel) is followed over time In other designs, sample members are rotated or completely replaced (some refer to these as repeated cross-sectional designs)

Longitudinal Designs
Panel (prospective) studies
Uses the same participants at each point in time Can reveal causal pathways

Retrospective studies
Participants provide data on past and current events

Useful for child development research and studies of aging But there are substantial costs and logistical issues



Longitudinal Designs: example

The National Child Development Study (NCDS)
17,000 people born in Britain in one week in 1958 Conducted eight follow-up sweeps or waves from 1965 to 2008 Added genetic data from 2002-2004 to examine effect on development of common traits and diseases Have analysed data related to health and education, as well as the effects of social changes such as the decline in marriage, the rise in women in employment, and decreased social mobility

Longitudinal Designs: validity

Threats to internal/external validity have been identified
History: changes can occur that are not related to measured variables Instrumentation: changes can occur in terms of the validity of the measure being used
What does gay mean in 1950? 1980? 2010?

Mortality: drop-out can occur non-randomly Testing: when using the same participants the equivalent of experimental order effects may occur