Question 1:

Read the case study below and answer the questions that follow.

Case study

You are the newly appointed Senior Director of Organisational Effectiveness at Capitec Bank.

You theorise that the ‘outcome’ goals of Organisational Effectiveness, including market

growth and job creation can be improved through improved Employee Engagement at every

level of the company. You remember from your 3rd year courses (Organisational Effectiveness

and Research in Organisation and Business Context) that when it comes to Employee

Engagement, it is not enough to just survey employees. After all, conducting a survey is just a

good first step. To test your theory, you develop and implement a play-at-work intervention

programme at randomly selected Capitec branches across the Eastern Cape.

The intervention consisted of different single-player and multi-player games and the games

catered for individuals who enjoy sport-like, creative, thinking, board-based and puzzle games.

The games that were implemented were chosen after consulting with experts in the field of

Industrial Psychology. All employees at the selected branches are mandated to complete the

questionnaires and participate in the intervention. You randomly assign 6 branches to the

experimental group and 8 branches to the control group. You administer a questionnaire to both

the experimental and the control group prior to introducing the intervention. The intervention

runs over three weeks where only the experimental group had the opportunity to play during

their lunch break for one hour. After the three weeks both the experimental and the control

group completed the same questionnaire again (post-test). The questionnaire assesses

employee’s psychological detachment, work enjoyment, employee performance, and

workplace boredom of work teams as indicators of Employee Engagement. All the dependent
variable items were measured on a six-point Likert scale: 1 (Strongly disagree), 2 (Disagree),

3 (Slightly disagree), 4 (Slightly agree), 5 (Agree), 6 (Strongly agree).

1.1 Did you use a “true” or a “quasi” experimental research design? Motivate your
answer. (2)
True experiment
Random assignment to experimental and control group
1.2 What are the dependant and independent variables in your study? (2)
IV = play-at-work intervention
DV = Employer Engagement
1.3 Name the specific research design that you implement and illustrate the design
diagrammatically. (3)
Pre test post test control group design
OXO
O O
1.4 State your null and alternative hypotheses using statistical notation. (1)
There is no difference is scores in employer engagement for groups who attended the
intervention and groups who did not.
There will be a difference is scores in employer engagement for groups who attended
the intervention and groups who did not.
H₀: μ₁ = μ₂
H₁: μ₁ ≠ μ₂

1.5 Evaluate the internal validity of your research design. Include in your answer possible
confounding variables that could have provided alternative explanations for your
findings. (4)

Internal validity is high

Testing = Testing becomes a problem if simply taking the pretest changes the participants
behaviour

Regression towards the mean = Likely to occur whenever participants are selected
because they score extremely high or low on some characteristic. When they are re-tested,
their scores tend to change in the direction of the mean.

History/maturation
1.6 Evaluate the external validity of your study. To what extent will you be able to
generalize your findings? (4)

External validity is low. No random selection of population. Can only generalise to

bank employers at best.

1.7 Specify the advantages and disadvantages of using pretesting in your study. (4)
Adv
Sometimes necessary to select a particular type of participant.
Assess whether the groups were in fact equivalent to begin with.
Can measure the extent of change in a participant.
Necessary when there is a possibility that participants may drop out of the experiment – mortality.
Mortality rates can make groups non-equivalent.
Use of a pretest makes it possible to assess the effects of mortality.
DisAdv
Time-consuming and awkward.
Can sensitise participants to what you are studying, enabling them to figure out your hypothesis.
The pretest may be a demand characteristic – any cue which tells participants what the researcher
is trying to find in the experiment.
1.8 What sampling strategy did you use to select your sample? (2)
Convenient probability sampling
1.9 What type of scale is used to indicate gender and race in your study? (1)
Nominal/ordinal scale
1.10 What type of scale is used to indicate employer engagement in your study? (1)
Ratio scale

Using the data in the Jamovi file, determine if your intervention was successful in improving
Employee engagement.

1.11 What variable do you need to include in your analysis? (2)

Employer engagement variable and the grouping variable

1.12 What statistical test do you need to conduct to determine if the experimental group and
the control group significantly differed on their mean employee engagement scores?
Independent sample/2sample t-test (1)
1.13 State the design requirements and assumptions for this test. (4)
There are 2 independent groups
1 continuous variable on which the 2 groups are compared
Random sampling
Normal distribution of scores
Homogeneity of variance

1.14 State the decision rule. (1)

Reject H₀ if ɑ ≤ 0.05
Do not reject H₀ if ɑ > 0.05
1.15 Find and report the observed value (i.e., conduct the analysis). (6)

Group Descriptives

Group N Mean Median SD SE

EmployeeEngagement Experimental 432 126 128 22.0 1.06

Control 553 122 127 24.1 1.02

Homogeneity of Variances Tests

F df df2 p

EmployeeEngagement Levene's 6.93 1 983 0.009

Variance ratio 0.834 431 552 0.047

Note. Additional results provided by moretests

Independent Samples T-Test

Effect
Statistic df p
Size

EmployeeEngagement Student's t 2.30 ᵃ 983 0.021 Cohen's d 0.148

Mann- Rank biserial
111144 0.061 0.0695
Whitney U correlation

ᵃ Levene's test is significant (p < .05), suggesting a violation of the assumption of equal variances

1.16 Make a decision about H₀. (1)

Fail to reject H₀.

1.17 Follow the standard APA practice for writing and reporting results, interpret and report
the results of the analysis, include the effect size. (6)

Leven’s test is significant ( F = 6.93, p < 0.05) suggesting that the experimental and
control group are not homogenous therefore the Mann-Whitney U test was used. The
Mann-Whitney test indicated that the employee engagement for the experimental
group (Mdn = 128) was not different from the control group (Mdn = 127), U=
111144, p = 0.06.

1.18 Briefly explain what the Levene’s F-test is and why it is important to use the test.
(5)

The Levene’s F-test is the homogeneity of variance test to determine if the two groups
come from the same population. If the p-value for the F-test is > 0.05, then we assume
equal variance and we can assume that the 2 groups are identical and can be compared
to each other. If the p-value for the F-test is < 0.05, then we assume unequal variance.
We assume that the 2 groups are not equal and an alternative test Mann-Whitney U/non-
parametric test is conducted.

Using the same data, evaluate whether the intervention was successful in improving employee

job performance.

1.19 What variables are required to conduct this analysis? (2)

Pre test post test job performance

1.20 What test will you use to determine if there is a difference between these variables? (1)
Dependent sample/1 sample t-test

1.21 State the design requirements for this test. (3)

There is a set of continuous scores representing the sample
There is a hypothetical distribution of scores with which the sample is being compared
Sample was randomly drawn from the population
Scores are normally distributed

1.22 State the null and alternate hypothesis for this analysis. (1)

Job performance will be the same after attending the intervention

Job performance will not be the same after attending the intervention
H₀: μ = 0
H₁: μ ≠ 0
1.23 Report the t-statistic and make a decision about the null hypothesis. (3)

t = -97.8, df = 984, reject Ho

1.24 Display the relevant outputs to substantiate and prove your decision. (4)

Descriptives

N Mean Median SD SE

PreEmployeePerformance 985 20.6 20.0 3.27 0.104

PostEmplyeePerform 985 76.2 80.0 18.82 0.600

Paired Samples T-Test

statisti Effec
df p
c t Size

PreEmployeePerforman PostEmplyeePerfor Student' 98 < .00 Cohen' -

-97.8
ce m st 4 1 sd 3.12

1.25 Interpret and report the results of the analysis, what is your conclusion about your
intervention? Remember to follow APA format for reporting the results. (6)

A paired samples t-test was used to determine if employee performance improved after
participating in a play-at-work intervention. The results from the pre-test (M = 20.6, SD =
3.27) and post-test (M = 76.2, SD = 18.82) employee performance scale indicate that
participating in the play-at-work intervention resulted in an improvement in employee
performance, t(984) = 97.8, p < .001. Thus showing that the paly-at-work intervention was
successful in improving employee performance.