Lecture:

Statistics in Cognitive Science

Hypothesis Testing
Logical Reasoning

• Deductive Reasoning

• Inductive Reasoning

• Abductive Reasoning

2 06/11/2023 Statistics in Cognitive Science - Lecture

Deductive Reasoning

3 06/11/2023 Statistics in Cognitive Science - Lecture

Deductive Reasoning

4 06/11/2023 Statistics in Cognitive Science - Lecture

Deductive Reasoning

5 06/11/2023 Statistics in Cognitive Science - Lecture

Deductive Reasoning

6 06/11/2023 Statistics in Cognitive Science - Lecture

Deductive Reasoning

7 06/11/2023 Statistics in Cognitive Science - Lecture

Deductive Reasoning

8 06/11/2023 Statistics in Cognitive Science - Lecture

Inductive Reasoning

1. I observed that the grass got wet every time when it rained.

2. ?

9 06/11/2023 Statistics in Cognitive Science - Lecture

Inductive Reasoning

1. I observed that the grass got wet every time when it rained.

2. When it rains, the grass gets wet.

10 06/11/2023 Statistics in Cognitive Science - Lecture

Abductive Reasoning

1. I observed that the grass got wet every time when it rained.

2. The grass is wet.

3. ?

11 06/11/2023 Statistics in Cognitive Science - Lecture

Abductive Reasoning

1. I observed that the grass got wet every time when it rained.

2. The grass is wet.

3. It probably rained.

12 06/11/2023 Statistics in Cognitive Science - Lecture

Statistical Inference

• We apply data analysis to infer properties of an underlying distribution of

probability (First sentence on Wikipedia)

• We draw a sample (our observed data set) and assume that it is drawn from a
population

• We infer properties of this population through inferential statistics performed

on our sample

13 06/11/2023 Statistics in Cognitive Science - Lecture

Significance Decisions / Hypothesis Testing

• Research Hypothesis
• Null and Alternative Hypothesis (H0 & H1)

14 06/11/2023 Statistics in Cognitive Science - Lecture

H0 & H1

• H0 = assumption about the population; „there‘s no difference“

The Null-Hypothesis

• H1 = „something is happening“
The Alternative-Hypothesis

15 06/11/2023 Statistics in Cognitive Science - Lecture

But why this complicated setting?
Falsifiability

• H0 = „All swans are white“

• H1 = „Some swans are black“

17 06/11/2023 Statistics in Cognitive Science - Lecture

H0 & H1

If the probability of a result occurring under H 0 is low, then

we reject H0 in favor of H1

18 06/11/2023 Statistics in Cognitive Science - Lecture

Probability

• Probability of events is measured on a scale of 0 to 1

• p represents our calculated probability

• Probability distribution is a histogram with columns measuring the likelihood

of occurrence

19 06/11/2023 Statistics in Cognitive Science - Lecture

Make a one- or two-tailed prediction

20 06/11/2023 Statistics in Cognitive Science - Lecture

Make a one- or two-tailed prediction

21 06/11/2023 Statistics in Cognitive Science - Lecture

Make a one- or two-tailed prediction

22 06/11/2023 Statistics in Cognitive Science - Lecture

Significance Decisions / Hypothesis Testing

• Research Hypothesis
• Null and Alternative Hypothesis (H0 & H1)
• Make statistical assumptions
• Decide which test is appropriate
• Set significance level
• Run your test
• Make a decision

23 06/11/2023 Statistics in Cognitive Science - Lecture

Reject or fail to reject the null hypothesis

• We reject the null hypothesis if our p-value lies under the pre-defined
significance level (α = .05).

• If we fail to reject the null hypothesis, we nonetheless do not conclude that

the null hypothesis is necessarily true

• Solution: Collect more data, or calculate an equivalence test

24 06/11/2023 Statistics in Cognitive Science - Lecture

Reject or fail to reject the null hypothesis

25 06/11/2023 Statistics in Cognitive Science - Lecture

Errors in statistical decision-making

Error Types Null hypothesis is

True False

Correct inference Type II error

Don‘t reject True negative False negative
Decision (1-α) (β)
about Null Type I error Correct inference
hypothesis is False positive True positive
Reject
(α) (1-β)

26 06/11/2023 Statistics in Cognitive Science - Lecture

Errors in statistical decision-making

Error Types Null hypothesis is

True False

Correct inference Type II error

Don‘t reject True negative False negative
Decision (1-α) (β)
about Null Type I error Correct inference
hypothesis is False positive True positive
Reject
(α) (1-β)

27 06/11/2023 Statistics in Cognitive Science - Lecture

Type I Error

28 06/11/2023 Statistics in Cognitive Science - Lecture

Errors in statistical decision-making

Error Types Null hypothesis is

True False

Correct inference Type II error

Don‘t reject True negative False negative
Decision (1-α) (β)
about Null Type I error Correct inference
hypothesis is False positive True positive
Reject
(α) (1-β)

29 06/11/2023 Statistics in Cognitive Science - Lecture

Type II Error

30 06/11/2023 Statistics in Cognitive Science - Lecture

Errors in statistical decision-making

Error Types Null hypothesis is

True False

Correct inference Type II error

Don‘t reject True negative False negative
Decision (1-α) (β)
about Null Type I error Correct inference
hypothesis is False positive True positive
Reject
(α) (1-β)

31 06/11/2023 Statistics in Cognitive Science - Lecture

Type II Error

32 06/11/2023 Statistics in Cognitive Science - Lecture

Errors in statistical decision-making

Error Types Null hypothesis is

True False

Correct inference Type II error

Don‘t reject True negative False negative
Decision (1-α) (β)
about Null Type I error Correct inference
hypothesis is False positive True positive
Reject
(α) (1-β)

33 06/11/2023 Statistics in Cognitive Science - Lecture

Type I Error

34 06/11/2023 Statistics in Cognitive Science - Lecture

Design an Experiment

1. Define research hypothesis

2. Operationalization
3. Make a one- or two-tailed prediction
4. Set Null (H0) and Alternative (H1) Hypothesis
5. Set significance level

Pick the right statistical test

35 06/11/2023 Statistics in Cognitive Science - Lecture

Design an Experiment

1. Do Cognitive Science students differ in general intelligence from the

normal population?
2. Measure IQ of 9 students
3. CS students do not have average intelligence
4. H0: µ = 100; H1: µ ≠ 100
5. α = .05

36 06/11/2023 Statistics in Cognitive Science - Lecture

What’s out there?

37 06/11/2023 Statistics in Cognitive Science - Lecture

One-Sample t-Test

• Let’s say we know the population mean µ = 100

• If we now collect data and want to know whether our sample is significantly
different from this population, what can we do?

38 06/11/2023 Statistics in Cognitive Science - Lecture

What do we have?

• µ = 100
• our sample size N = 9
• our sample mean
• our sample standard deviation
• our standard error of the mean

39 06/11/2023 Statistics in Cognitive Science - Lecture

What can we do with this?

• We can calculate, how many se our sample mean is away from the population
mean µ (or how far away is our difference from 0)

• How many degrees of freedom does it have?

• N-1

40 06/11/2023 Statistics in Cognitive Science - Lecture

Is t just like z?

• No!

• unlike z, t is not normally distributed (unless N > 120)

• It’s generally broader than a normal distribution

• Student‘s t (William Gossett)

• We have t tables that give us critical values

41 06/11/2023 Statistics in Cognitive Science - Lecture

 Density Plot
Density Plot

Example: z-distribution vs t-distribution Densit

df = 8

42 06/11/2023 Statistics in Cognitive Science - Lecture

 Density Plot

Example: z-distribution vs t-distribution

df = 8

43 06/11/2023 Statistics in Cognitive Science - Lecture

t-Table

44 06/11/2023 Statistics in Cognitive Science - Lecture

Example: t-distribution

df = 8

45 06/11/2023 Statistics in Cognitive Science - Lecture

How to report?

The average IQ of Cognitive Science students (M = 109) is significantly

different from the average IQ of the general population, t(8) = 3, p = .017.

46 06/11/2023 Statistics in Cognitive Science - Lecture

t-Table

47 06/11/2023 Statistics in Cognitive Science - Lecture

Example: t-distribution

df = 8

48 06/11/2023 Statistics in Cognitive Science - Lecture

How to report?

The average IQ of Cognitive Science students (M = 109) is significantly higher

than the average IQ of the general population, t(8) = 3, p = .009.

49 06/11/2023 Statistics in Cognitive Science - Lecture

Thanks for listening!