Social Psychology of Language

Methods in Attitudinal
and Perceptual Research

Dr Erin Carrie
GM442; e.carrie@mmu.ac.uk; 0161 247 3930; @erincarrie
Outline

• Methodologies
• Research Design
• Methods for collecting data
• Methods for analysing data
• Tasks
Methodologies
Limitations

• Eliciting and measuring language attitudes is very different from

eliciting and measuring language use.

• Language attitudes are not directly observable.

• There are certain limitations when relying on self-reported data.

Direct and Indirect Methods

• When attempting to elicit language attitudes directly, we have to

be aware of motivated response distortions.
• e.g., engaging in ‘impression management’ (Tedeschi 1981)
• e.g., responding to ‘demand characteristics’ (Orne 1961)

• These are also known as social desirability and acquiescence bias.

Direct and Indirect Methods

• Indirect methods are often more elaborate and sophisticated.

• Attitudes elicited using these methods are more reliable because

they are less influenced by motivated response distortions.
Methodological Approaches

• Most language attitude studies take a quantitative approach.

• e.g., eliciting numerical data using questionnaires or tasks

• Some language attitude studies (also) take a qualitative approach.

• e.g., eliciting (additional) data via questionnaires or interviews

• For this unit, I recommend taking an experimental approach.

• social and linguistic variables (independent)
• attitude as a variable (dependent)
Initial Discussion

• Are you planning to take a QUAN, QUAL or mixed approach?

• Which tools or techniques will you employ; e.g., questionnaires,

tasks, interviews, observations, …?

• Which social and which linguistic variables are you investigating?

(social = optional, linguistic = essential)

• Do you intend to measure attitudes directly or indirectly?

Research Design
Research Design: Step 1

• Choosing a particular context of interest (e.g., accents in language

teaching, accents in court proceedings, accents in the workplace).

AND/OR

• Choosing particular social or linguistic variables of interest (e.g.,

ethnicity, social class, gender, or a particular accent or phoneme).
Research Design: Step 2

• Getting ethical approval (via EthOS).

• Complete application online:

• https://www2.mmu.ac.uk/research/staff/ethics-and-governance/ethics/
• Participant information sheet*
• Participant consent form*

*not necessary to include these as attachments for questionnaires

but essential when conducting interviews, observations, etc.
Research Design: Step 3

• Considering your role as the researcher.

• If you are collecting data in person…

• How involved will you be in the data collection process?
• Pros and cons of collecting data yourself.
• How might audio/visual cues influence responses?
Research Design: Step 4

• Designing the research instrument.

• Questionnaires and tasks need careful consideration to ensure

that the data they collect are both valid and reliable.

• They can be administered either in person or online.

• Pros/cons?
• e.g., researcher influence/control, manual input of data
Research Design: Step 4

• Designing the research instrument.

• Interviews should be carefully designed to ensure that the data

collected have sufficient coverage and depth.

• An interview guide or protocol can be used, taking into account

the order in which themes and questions are presented, as well
as the wording of questions.
Which research instrument would you employ for…?

• Doing a rating task (e.g., matched/verbal guise test)

• Answering demographic questions (e.g., age/gender categories)
• Rating suitability for a job (e.g., yes/no questions)
• Talking about experiences of language-based bias and prejudice
• Eliciting reactions (via ratings) to audio and visual stimuli
• Doing a consciousness-raising activity involving group discussions
• Observing attitudes in a real-life context (e.g., workplace setting)
• Discursively eliciting stereotypes of who uses certain slang terms
Methods for Collecting Data
Research Design: Step 1

• Contextualising the study.

• Title – plain language, not too explicit

• Information and consent – plain language, not too explicit
• Clear instructions
• Brief – inform participants of their role or the scenario (priming)
Research Design: Step 2

• Choosing a matched, verbal or conceptual guise design.

• Matched guise = the same speaker performing two(+) guises.

• the importance of ‘filler’ voices

• Verbal guise = different speakers performing each guise.

• the importance of matching speakers’ profiles

• Conceptual guise – pros and cons?

Issues with the MGT (Garrett 2010: 57-59)

• The salience question

• The perception question
• The accent-authenticity question
• The mimicking-authenticity question
• The community-authenticity question
• The style-authenticity question
• The neutrality question
Research Design: Step 3

• Selecting speech varieties and speakers.

• When using a verbal guise design, we must select speakers with

comparable profiles (e.g., age, gender, ethnicity, socioeconomic
status), so as not to introduce extraneous variables.

• Also, when using a verbal guise design, we must control for

features such as voice quality, speech rate, and intonation.
Research Design: Step 4

• Making suitable recordings of speakers.

• Reading aloud a pre-prepared text?

• style implications (Labov 1972)
• topic, social and cultural schemata

• Spontaneous speech?
• more representative of everyday speech
• more lexical, syntactic and morphological variation
Research Design: Step 4

• McKenzie (2008) – map task; ‘factually neutral’ stimulus speech

Research Design: Step 5

• Considering the content and wording of questions.

• Avoiding leading and loaded questions/statements, and being

mindful of positive and negative framing of questions/statements.

• This is especially important in interviews!

Leading and Loaded Questions

• Do you agree that attitudes towards regional dialects are

changing and that they’re changing for the better?
• What do you dislike about your own speech?
• Surely you like the way you speak, don’t you?
• What prejudice have you experienced based on your speech?
• It is true that people with second-language accents find it more
difficult to get a job in the UK.
• When did you stop being proud of your own accent/dialect?
Research Design: Step 6

• Likert scales typically range from ‘1’ to ‘5’ (Likert 1932).

strongly disagree disagree neither agree agree strongly agree

nor disagree

○ ○ ○ ○ ○

• Some studies use +/- with ‘0’ as neutral point.

Research Design: Step 6

• Magnitude estimation scales can be used instead of other scales.

not friendly -------------------------------------------------------------------------------- friendly

• They often have either 80 or 100 points on the scale.

• precise value selected on online survey
• precise value established using overlay on paper-based survey
Research Design: Step 6

• Semantic differential scales traditionally range from ‘1’ to ‘7’

(Osgood et al. 1957).

not friendly ○ ○ ○ ○ ○ ○ ○ friendly

• One semantic trait should be selected and its bipolar opposite;

e.g., ‘confident-insecure’ vs. ‘confident-not confident’.

• Some studies use +/- with ‘0’ as neutral point.

• If not, need to consider ‘I don’t know’ or ‘I’m not sure’ option.
Research Design: Step 7

• Choosing traits and their bipolar opposites.

• Traits are usually elicited in a pilot study with several participants,

and using open questions, to ensure that they are representative
and meaningful to the sample population (Garrett 2010).

• Binary, contradictory opposites (e.g., ‘not correct’) can be used

instead of less binary, affixed negations (e.g., ‘incorrect’) to help
control for multiple interpretations (Zimmer 1964).
Research Design: Step 8

• Ensuring that the data reflect the evaluative content of the scales.
• Take the following examples:

not friendly -------------------------------------------------------------------------------- friendly

not arrogant -------------------------------------------------------------------------------- arrogant

not kind -------------------------------------------------------------------------------- kind

not boring -------------------------------------------------------------------------------- boring

• Which extremes are worded positively/negatively?

• Which extremes represent positive/negative evaluations?
Research Design: Step 9

• Considering potential left/right biases.

• explain format clearly and subsequently transpose scales

• Considering order effects (e.g., questions, speakers and traits).

• influence reduced via manual or automatic randomisation

• Considering respondent fatigue or lack of attention.

• carefully select speakers/traits/questions and look for outliers
Research Design: Step 10

• It is useful to take the perspective of a participant in your study.

• Would I understand the questions?

• Would I interpret them as they are intended to be interpreted?
• Would I respond in appropriate ways?
Research Design: Step 10

• Piloting the questionnaire and/or interview.

• Not only important for eliciting relevant traits.

• Also useful for testing the research instrument:

• How long did it take respondents to complete?
• Were the instructions clear and easy-to-follow?
• Were the questions worded appropriately?
• Did respondents encounter any difficulties?
• In what way(s) could my interviewing style be enhanced?
Methods for Analysing Data
Analysing Data: Step 1

• Creating data sets prior to software analysis (e.g., SPSS/NVivo).

• For example, a quantitative data set containing:

• all demographic and social information about respondents
• respondents’ ratings on the chosen scales

• For example, a qualitative data set containing:

• written/typed responses to open-ended questions
• transcribed audio-recorded responses during interviews
Analysing Data: Step 2

• An example of nominal data would be gender categories; e.g.,

attributing the value of ‘1’ to males and ‘2’ to females.
• no ranking between groups; numerical data is meaningless
• An example of ordinal data would be a five-point scale from, e.g.,
‘always’ – ‘frequently’ – ‘sometimes’ – ‘rarely’ – ‘never’.
• ranking is important; numerical data is meaningless
• An example of scalar data would be a scale covering a particular
age range; e.g., ‘0’ to ‘75’.
• fixed zero point; numerical data is meaningful
Analysing Data: Step 3

• Calculating mean evaluations and standard deviations.

• Testing for significant findings between groups/speakers/traits:
• t-tests allow you to compare two mean scores
• ANOVAs allow you to compare more than two mean scores
• chi-square tests allow you to compare categorical data
• correlations allow you to compare ordinal/scalar data

• Conducting a content analysis of any qualitative data.

• pre-determined or emergent themes
• transcribing, pre-coding and coding, as well as querying data
Discussion

Would the following be examples of nominal/ordinal/scalar data?

• Length of residence in Manchester (measured in months)

• Age categories (e.g., 18-25, 26-45, 46-65, 66+)
• Ethnic categories (e.g., Black British, White British, British Asian)
• Ratings on a Likert scale (strongly agree – strongly disagree)
Discussion

Which statistical test would you use to find out…?

• …whether one speaker receives a significantly higher mean score

for status or solidarity?
• …whether there was a strong tendency for participants to rate
male speakers as both ‘gay’ and ‘effeminate’?
• …whether males or females are significantly more likely to rate a
speaker as employable?
• …whether speakers of a certain ethnicity tend to rate a speaker
significantly more positively than those of other ethnicities?
Tasks
Tasks

In pairs or small groups, …

• Construct Likert and semantic differential scales for eliciting

‘attitudes towards Indian accents in the call service industry’.

• Design a priming study investigating racial bias and attitudes

towards Standard Southern British English and Multicultural
Urban British English (using audio and visual stimuli).