Week 5

Does Clickbait Work?

- Hypothesis Testing

UTS CRICOS 00099F

STATISTICAL
INFERENCE

UTS CRICOS 00099F

Poll

The Law of Large Numbers states that:

1. The distribution of the sample means approaches a normal distribution as the

sample size gets larger, regardless of the shape of the population distribution.

2. As a company gets larger, it becomes more difficult to sustain its previous growth
rates.

3. The larger the sample size, the closer the sample mean will be to the mean
population.
Law of Large Numbers (LLN)

States that if the same experiment is repeated independently a large number of

times, the average of the results of the trials must be close to the actual
expected value

• The sample mean becomes closer to the expected value as more trials are
performed.

• It is the reason casinos are profitable businesses!

LLN - Example
LLN - Example
Central Limit Theorem (CLT)

States that the distribution of the sample means approaches a normal

distribution (bell curve) as the sample size gets larger, regardless of the shape
of the population distribution.
CLT - Example

▪ Fast food chain with presence in all US

counties
▪ CEO believes their average market
share across counties is 50%
▪ She wants to know if this is true

▪ Sample 30 counties and obtain an

average of 45% market share
CLT - Example
CLT - Example
CLT - Example
Central Limit Theorem (CLT)
HYPOTHESIS
TESTING

UTS CRICOS 00099F

Estimating Population Characteristics

Parameter Statistic

▪ Population Mean ▪ Sample Mean

▪ Population ▪ Sample Standard
Standard Deviation Deviation
Hypothesis Testing

▪ Evaluate two mutually exclusive statements about a parameter:

▪ Null Hypothesis: initial conjecture about a population parameter

▪ Alternative Hypothesis: it’s the opposite of what is stated in the Null Hypothesis

“We either reject the Null or fail to reject it”

• We never “accept” the Null Hypothesis

Hypothesis Testing

▪ Null Hypothesis

▪ Alternative Hypothesis

▪ One-tailed vs Two-tailed Test

▪ Significance Level

▪ Test-statistic
Hypothesis Testing - Example
▪ iPhone 11 current mean battery life:
1,000 min (16.7 hours)
▪ Director of manufacturing wants to
introduce a new battery technology that
can increase mean battery life
▪ Apple produces a small batch of phones
with the new battery technology
▪ Sample mean battery life: 1,200 min (20
hours)
Hypothesis Testing - Example
▪ Null Hypothesis (𝐻0 ): Average battery life <= 1,000 min

▪ Alternative Hypothesis (𝐻𝑎 ): Average battery > 1,000 min

▪ One-tailed Test

▪ Significance Level: 0.01

▪ Test-Statistic ?
HT- Example
HT- Example
HT- Example
HT- Example
Hypothesis Testing - Example

𝑆𝑖𝑔𝑛𝑎𝑙 = 1,200 − 1,000 𝑆𝑖𝑔𝑛𝑎𝑙 = 1,200 − 1,000

Hypothesis Testing - Example

𝑆𝑖𝑔𝑛𝑎𝑙 1,200 −1,000 𝑆𝑖𝑔𝑛𝑎𝑙 1,200 −1,000

= = 1.54 = = 3.85
𝑁𝑜𝑖𝑠𝑒 130 𝑁𝑜𝑖𝑠𝑒 52
Hypothesis Testing - Example

𝑆𝑖𝑔𝑛𝑎𝑙 1,200 −1,000 𝑆𝑖𝑔𝑛𝑎𝑙 1,200 −1,000

= = 1.54 = = 3.85
𝑁𝑜𝑖𝑠𝑒 130 𝑁𝑜𝑖𝑠𝑒 52

Test Statistic
Hypothesis Testing - Example

𝑆𝑖𝑔𝑛𝑎𝑙 1,200 −1,000 𝑆𝑖𝑔𝑛𝑎𝑙 1,200 −1,000

= = 1.54 = = 3.85
𝑁𝑜𝑖𝑠𝑒 130 𝑁𝑜𝑖𝑠𝑒 52
p-value = 6.20% p-value = 0.006%
Why Normal Distributions?
Distribution of the Sample Mean

▪ Sample mean is Normally Distributed (Central Limit Theorem):

▪ Its mean equals the population mean:

▪ Its standard deviation equals the population standard deviation divided by the square root of the

sample size:
Distribution of the Sample Mean

68.27%

95.45%

99.73%
Hypothesis Testing – Known Standard Deviation

▪ Signal-to-Noise ratio is called z-statistic:

68.27%

95.45%

99.73%
Hypothesis Testing – Unknown Standard Deviation

▪ We don’t know the population (real)

standard deviation:
Hypothesis Testing – Unknown Standard Deviation
▪ Signal-to-Noise ratio is called

student’s t-statistic:

▪ T-distribution with n-1 degrees of

freedom (df)
Hypothesis Testing – Difference in Means
▪ Signal-to-Noise ratio is the student’s t-

statistic:

Signal

Noise
T-test for Difference in Means - Example

▪ A/B testing

Source: https://waat.eu/blog/a-b-testing-tools-comparison-google-optimize-vs-optimizely/
Type I and Type II Errors

Source: Camm, J. D., Cochran, J. J., Fry, M. J., & Ohlmann, J. W. (2020). Business analytics. Cengage AU
Type I and Type II Errors – Trade-off

Source: http://grasshopper.com/blog/the-errors-of-ab-testing-your-conclusions-can-make-things-worse/
DECISION-MAKING
BIASES

UTS CRICOS 00099F

Confirmation Bias

▪ Tendency to “cherry-pick” evidence that

confirms our existing beliefs
▪ Evaluating evidence is costly so we use
mental shortcuts to make the process
efficient
▪ It protects our self-esteem
Source: https://fs.blog/2017/05/confirmation-bias
IN-CLASS ACTIVITY

UTS CRICOS 00099F

In-class Activity

Problem: Does Clickbait Work?

Criteria: Student’s t-test for difference in means

Alternatives: 1%, 5%, 10% Significance Levels

Data: A/B testing data