AS Psychology

Validity, Ethical Concerns, Variables, & Self Reports

9/22/2023
 • The fundamental concept to keep in mind
when creating any assessment is validity.
Validity refers to whether a test measures
what it aims to measure. For example, a
valid driving test should include a practical
driving component and not just a theoretical
What is test of the rules of driving. A valid language
test for university entry, for example, should
validity? include tasks that are representative of at
least some aspects of what actually happens
in university settings, such as listening to
lectures, giving presentations, engaging in
tutorials, writing essays, and reading texts.
 Validity
• Validity sits upon a spectrum. For example:
• Low Validity: Most people now know that the standard IQ test does not
actually measure intelligence or predict success in life.
• High Validity: By contrast, a standard pregnancy test is about 99%
accurate, meaning it has very high validity and is, therefore, a very
reliable test.
 Validity

Validity refers to
whether a measure
actually measures Types of validity Face validity
what it claims to be include
measuring.

External validity Internal validity

(Temporal and (Construct and
Ecological validity) Concurrent validity)
 Face validity is a measure of whether it
looks subjectively promising that a tool
measures what it's supposed to

Face
Validity
Face validity refers to whether a scale
“appears” to measure what it is supposed
to measure. That is, do the
questions seem to be logically related to
the construct under study.
 Face validity
e.g., It might be observed that people with higher scores in exams
are getting higher scores on an IQ questionnaire; you cannot be
sure that these are directly linked, but on the surface, it appears
that exam scores are a reasonable indication of IQ scores, so your
measure shows good face validity
e.g., a personality scale that measures emotional intelligence
should have questions about self-awareness and empathy. It
should not have questions about math or chemistry.
 Face validity

One common way to assess face

validity is to ask a panel of experts to
examine the scale and rate its
If a scale, or a test, doesn’t have face
appropriateness as a tool for measuring
validity, then people taking it won’t be
the construct. If the experts agree that
serious.
the scale measures what it has been
designed to measure, then the scale is
said to have face validity.
 Internal Validity

• Internal validity refers to whether or not the results of an experiment are

due to the manipulation of the independent variable. For example, a
researcher wants to examine how temperature affects willingness to help,
so they have research participants wait in a room.
 Internal validity

• There are different rooms, one has the temperature set at normal, one at
moderately warm, and the other at very warm.

• During the next phase of the study, participants are asked to donate to a
local charity before taking part in the rest of the study. The results showed
that as the temperature of the room increased, donations decreased.
 Internal validity

• On the surface, it seems as though the study has internal validity: room
temperature affected donations. However, even though the experiment
involved three different rooms set at different temperatures, each room
was a different size. The smallest room was the warmest and the normal-
temperature room was the largest.
 Internal validity

• Now, we don’t know if the donations were affected by room temperature

or room size. So, the study has questionable internal validity.

• Another way internal validity is assessed is through inter-rater

reliability measures, which help bolster both the validity and reliability of
the study.
 Construct validity

• Construct validity – asks whether a measure successfully measures the

concept it is supposed to (e.g. does a questionnaire measure IQ, or
something related but crucially different?).
 Concurrent Validity
• Concurrent validity – asks whether a measure is in agreement with pre-
existing measures that are validated to test for the same [or a very similar]
concept (gauged by correlating measures against each other).
• In essence, Concurrent validity is a method of assessing validity that
involves comparing a new test with an already existing test, or an already
established criterion.
 Concurrent Validity
• For example, a newly developed math test for the SAT will need to be
validated before giving it to thousands of students. So, the new version of
the test is administered to a sample of college math majors along with the
old version of the test.
• Scores on the two tests are compared by calculating a correlation between
the two. The higher the correlation, the stronger the concurrent validity of
the new test.
 External Validity

• External validity is a measure of whether data can be generalized to other

situations outside of the research environment they were originally
gathered in.

• This creates a big problem regarding external validity. Can we say that
what happens in a lab would be the same thing that would happen in the
real world?
 External Validity
• For example, a study on mindfulness involves the researcher randomly
assigning different research participants to use one of three mindfulness
apps on their phones at home every night for 3 weeks. At the end of three
weeks, their level of stress is measured with some high-tech EEG
equipment.
• This study has external validity because the participants used real apps
and they were at home when using those apps. The apps and the home
setting are realistic, so the study has external validity.
 Temporal Validity
• Temporal validity – this is high when research findings successfully apply
across time (certain variables in the past may no longer be relevant now or
in the future).
• e.g. Changes in attitude towards gender roles over time could lower the
temporal validity of data from past experiments when applied to modern-
day research.
 Ecological Validity
• Ecological validity – whether data is generalizable to the real world, based
on the conditions research is conducted under and procedures involved.
• e.g. Laboratory research can exert a high degree of control over
extraneous variables that would otherwise vary in a natural environment,
so results might be considered too ‘artificial’ and thus possess low
ecological validity.
 External Validity

• The external validity of an experiment can be assessed and improved by

replicating a study at different times and places and obtaining similar
results. For example, confidence in the generalizability [and in turn
external validity] of results is increased when research is successfully
replicated across different cultures.
 Variables

• An independent variable (IV) is a variable that is manipulated by a

researcher to investigate whether it consequently brings change in another

variable. This other variable, which is measured and predicted to be

dependent upon the IV, is therefore named the dependent variable (DV).
 Variables

• For example, in an experiment examining the effect of fatigue on short-

term memory, there are two groups ‘fatigued’ and ‘non-fatigued’. The

fatigued group ran for 10 minutes without stopping prior to being tested.

Both groups are given a list of words to recall immediately after reading

the list.
 Variables
• IVs and DVs only occur in experiments, as a cause and effect is predicted
between the two (i.e. that changes in the IV will directly lead to changes
in the DV).

• IVs and DVs do not feature in correlation studies, as correlation studies

look for a relationship between co-variables, cause and effect is therefore
not established as the variables are predicted to change in response to each
other.
 Variables

• A confounding variable is an “extra” variable that you didn’t account for. They

can ruin an experiment and give you useless results. They can suggest there is a

correlation when in fact there isn’t. They can even introduce bias. That’s why it’s

important to know what one is, and how to avoid getting them into your

experiment in the first place.

 Variables

• Extraneous variable (EV) is a general term for any variable, other than the

IV, that might affect the results (the DV). Where EVs are important

enough to cause a change in the DV, they become confounding variables.

 Question

At target practice, Tim shoots 15 bullets at the

bull's eye. The bullets produce a pattern like this
Tim's shooting is:

a) Reliable, not valid.

b) Valid, not reliable.
c) Neither reliable nor valid.
d) Both reliable and valid
 Question

Tim's shooting is:

a) Reliable, not valid.

b) Valid, not reliable.
c) Neither reliable nor
valid.
d) Both reliable and
valid.
 Ethics

• There are four ethical principles which are the main domains of responsibility for

consideration by researchers within the code; respect, competence, responsibility and

integrity.

• Researchers in psychological research should consider the following when they plan

and run research

 Ethics
• Consent; have participants given informed consent? If the participant is under
16 years old, has informed consent been given by their parents or guardians?

• Deception: have the participants been deceived in any way? If so, could this
have been avoided?

• Debriefing: have the participants been debriefed? Have they been given the
opportunity to ask questions?
 Ethics
• Withdrawal from the investigation: have the participants been informed
of their right to withdraw from the research at any point, including the
right to withdraw their data at a later date?
• Anonymity and Confidentiality: participants have a right to remain
anonymous in publication of the research and confidentiality should be
maintained except in exceptional circumstances where harm may arise to
the participant or someone associated with the research or participant.
 Ethics
• Protection of participants: researcher must protect participants from
both physical and psychological harm
 Self Reports
• Self-report techniques describe methods of gathering data where
participants provide information about themselves without interference
from the experimenter.

• Such techniques can include questionnaires, interviews, or even diaries,

and ultimately will require giving responses to pre-set questions.
 Strengths of Self Reports
• Participants can be asked about their feelings and cognitions (i.e.
thoughts), which can be more useful than simply observing behavior
alone.

• Scenarios can be asked about hypothetically without having to physically

set them up and observe participants’ behavior.
 Weaknesses of Self Reports
• Gathering information about thoughts or feelings is only useful if
participants are willing to disclose them to the experimenter.

• Participants may try to give the ‘correct’ responses they think researchers
are looking for (or deliberately do the opposite), or try to come across in
most socially acceptable way (i.e. social desirability bias), which can lead
to giving untruthful responses.
Questions?