Regression tests

Regression tests are used to test cause-and-effect relationships. They look for the
effect of one or more continuous variables on another variable.

Predictor variable Outcome variable Research question example

Simple linear  Continuous  Continuous What is the effect of income on longevity?

regression  1 predictor  1 outcome

Multiple linear  Continuous  Continuous What is the effect of income and minutes of
regression  2 or more  1 outcome exercise per day on longevity?
predictors

Logistic  Continuous  Binary What is the effect of drug dosage on

regression the survival of a test subject?

Comparison tests
Comparison tests look for differences among group means. They can be used to
test the effect of a categorical variable on the mean value of some other
characteristic.

T-tests are used when comparing the means of precisely two groups (e.g. the
average heights of men and women). ANOVA and MANOVA tests are used when
comparing the means of more than two groups (e.g. the average heights of children,
teenagers, and adults).

Predictor variable Outcome variable Research question example

Paired t-test  Categorical  Quantitative What is the effect of two different test
 1 predictor  groups come from prep programs on the average exam
the same scores for students from the same
population class?

Independent  Categorical  Quantitative What is the difference in average exam

t-test  1 predictor  groups come from scores for students from two different
different schools?
populations

ANOVA  Categorical  Quantitative What is the difference in average pain

 1 or more  1 outcome levels among post-surgical patients
given three different painkillers?
 Predictor variable Outcome variable Research question example

predictor

MANOVA  Categorical  Quantitative What is the effect of flower

 1 or more  2 or more outcome species on petal length, petal width,
predictor and stem length?

Correlation tests
Correlation tests check whether two variables are related without assuming
cause-and-effect relationships.

These can be used to test whether two variables you want to use in (for example) a
multiple regression test are autocorrelated.

Predictor variable Outcome variable Research question example

Pearson’s r Continuous Continuous How are latitude and temperature related?

Choosing a nonparametric test

Non-parametric tests don’t make as many assumptions about the data, and are
useful when one or more of the common statistical assumptions are violated.
However, the inferences they make aren’t as strong as with parametric tests.

Predictor variable Outcome variable Use in place

of…

Spearman’s r  Quantitative  Quantitative Pearson’s r

Chi square test  Categorical  Categorical Pearson’s r

of
independence
Sign test  Categorical  Quantitative One-sample
t-test
Kruskal–Wallis  Categorical  Quantitative ANOVA
H  3 or more
groups
 Predictor variable Outcome variable Use in place
of…

ANOSIM  Categorical  Quantitative MANOVA

 3 or more  2 or more
groups outcome
variables

Wilcoxon  Categorical  Quantitative Independent

Rank-Sum test  2 groups  groups come t-test
from different
populations

Wilcoxon  Categorical  Quantitative Paired t-test

Signed-rank  2 groups  groups come
test from the same
population
Cronbach's Alpha (α) using SPSS Statistics

Introduction Reliability

Cronbach's alpha is the most common measure of internal consistency ("reliability"). It is most
commonly used when you have multiple Likert questions in a survey/questionnaire that form a scale
and you wish to determine if the scale is reliable. If you are concerned with inter-rater reliability, we
also have a guide on using Cohen's (κ) kappa that you might find useful.

Introduction Validity
 Assumptions
(Homogeneity) Levene's test is used to test if k samples have equal variances. Equal
variances across samples is called homogeneity of variance. {If the p-value for the Levene
test is > 0,05, then the variances are not significantly different from each other, thus there is
homogeneity, but if p < 0,05, then there is a significant difference between the variances,
and thus there is not homogeneity} {θέλω p > 0,05 και όχι μικρότερο όπως συνήθως}
(Normality/Normal distribution) Shapiro–Wilk test [is more appropriate method for small
sample sizes (<50 samples) although it can also be handling on larger sample size] AND
Kolmogorov–Smirnov test [is used for n ≥50]. For both of those tests, null hypothesis states
that data are taken from normal distributed population. {θέλω p > 0,05 και όχι μικρότερο
όπως συνήθως}
(No significant outliers) in SPSS (explore / Box-whisker diagram OR Boxplot) μόνο τους
αστερίσκους και όχι τους κύκλους

Parametric Tests
Student’s t-test (both independent and paired) assumes that both groups of data are
sampled from populations that follow a normal distribution and that both populations have
the same variance. In specific assumes both normality and equal variances.
One sample: [RESULTS] Depression score was statistically significantly lower than the
population normal depression score, t(39) = -2.381, p = .022.
Independent t-test: [RESULTS] This study found that overweight, physically inactive male
participants had statistically significantly lower cholesterol concentrations (5.80 ± 0.38
mmol/L) at the end of an exercise-training programme compared to after a calorie-
controlled diet (6.15 ± 0.52 mmol/L), t(38)=2.428, p=0.020.
Dependent T-Test: [RESULTS] You might report the statistics in the following format:
t(degrees of freedom) = t-value, p = significance level. In our case this would be: t(19) = -
4.773, p < 0.0005. Due to the means of the two jumps and the direction of the t-value, we
can conclude that there was a statistically significant improvement in jump distance
following the plyometric-training programme from 2.48 ± 0.16 m to 2.52 ± 0.16 m (p <
0.0005); an improvement of 0.03 ± 0.03 m.

Welch’s t-test / Welch’s ANOVA (comparing means between two independent groups
without assuming equal population variances) assumes that both groups of data are
sampled from populations that follow a normal distribution, but it does not assume that
those two populations have the same variance. Thus, assumes normality but not equal
variances.
[We show that the Welch’s t-test provides a better control of Type 1 error rates when the
assumption of homogeneity of variance is not met, and it loses little robustness compared
to Student’s t-test when the assumptions are met.
{https://www.rips-irsp.com/articles/10.5334/irsp.82/}]

Brown-Forsythe test (is a statistical test for the equality of group variances based on
performing an Analysis of Variance (ANOVA) on a transformation of the response variable).

McNemar test (is used to determine if there are differences on a dichotomous dependent
variable between two related groups) It can be considered to be similar to the paired-
samples t-test, but for a dichotomous rather than a continuous dependent variable.
However, unlike the paired-samples t-test, it can be conceptualized to be testing two
different properties of a repeated measure dichotomous variable. The McNemar test is used
to analyze pretest-posttest study designs, as well as being commonly employed in analyzing
matched pairs and case-control studies. If you have more than two repeated
measurements, you could use Cochran's Q test.

Cochran's Q test (is used to determine if there are differences on a dichotomous dependent
variable between three or more related groups) It can be considered to be an extension of
McNemar's test. Cochran's Q test is commonly used to analyse longitudinal study designs,
as well as being used when analysing participants that have undergone multiple different
trials (a.k.a. treatments/conditions).

ANOVA [F-test] (measures one dependent variable) is an omnibus test.

[post hoc test]
Tukey's honestly significant difference (HSD) - If your data met the assumption of
homogeneity of variances.
Games Howell test (If your data did not meet the homogeneity of variances assumption)

MANOVA (measures two or more dependent variables)

ANCOVA (can be thought of as an extension of the one-way ANOVA to incorporate a

covariate)

MANCOVA (can be thought of as an extension of the one-way MANOVA to incorporate a

covariate or an extension of the one-way ANCOVA to incorporate multiple dependent
variables) This covariate is linearly related to the dependent variables and its inclusion into
the analysis can increase the ability to detect differences between groups of an independent
variable.
-+-+-+-+-+-+-+-+-
ANOVA:
One-way (independent samples): The one-way analysis of variance (ANOVA) is used to
determine whether there are any statistically significant differences between the means of
two or more independent (unrelated) groups (although you tend to only see it used when
there are a minimum of three, rather than two groups). For example, you could use a one-
way ANOVA to understand whether exam performance differed based on test anxiety levels
amongst students, dividing students into three independent groups (e.g., low, medium and
high-stressed students). Also, it is important to realize that the one-way ANOVA is an
omnibus test statistic and cannot tell you which specific groups were statistically
significantly different from each other; it only tells you that at least two groups were
different.
[RESULTS] There was a statistically significant difference between groups as determined by
one-way ANOVA (F(2,27) = 4.467, p = .021). A Tukey post hoc test revealed that the time to
complete the problem was statistically significantly lower after taking the intermediate (23.6
± 3.3 min, p = .046) and advanced (23.4 ± 3.2 min, p = .034) course compared to the
beginners course (27.2 ± 3.0 min). There was no statistically significant difference between
the intermediate and advanced groups (p = .989).

Repeated Measures (dependent samples): Where the participants are the same in each
group. This usually occurs in two situations: (1) when participants are measured multiple
times to see changes to an intervention; or (2) when participants are subjected to more than
one condition/trial and the response to each of these conditions wants to be compared.
[RESULTS]
A repeated measures ANOVA with a Greenhouse-Geisser correction determined that mean
CRP concentration differed statistically significantly between time points (F(1.298, 11.663) =
26.938, P < 0.0005). Post hoc analysis with a Bonferroni adjustment revealed that CRP
concentration was statistically significantly decreased from pre-intervention to three months
(0.39 (95% CI, 0.24 to 0.54) mg/L, p < .0005), and from pre-intervention to post-intervention
(0.68 (95% CI, 0.34 to 1.02) mg/L, p = .001), but not from three months to post-intervention
(0.29 (95% CI, -0.01 to 0.59) mg/L, p = .054).

Two-way: The two-way ANOVA compares the mean differences between groups that have
been split on two independent variables (called factors). The primary purpose of a two-way
ANOVA is to understand if there is an interaction between the two independent variables on
the dependent variable. For example, you could use a two-way ANOVA to understand
whether there is an interaction between gender and educational level on test anxiety
amongst university students, where gender (males/females) and education level
(undergraduate/postgraduate) are your independent variables, and test anxiety is your
dependent variable. Alternately, you may want to determine whether there is an interaction
between physical activity level and gender on blood cholesterol concentration in children,
where physical activity (low/moderate/high) and gender (male/female) are your
independent variables, and cholesterol concentration is your dependent variable. The
interaction term in a two-way ANOVA informs you whether the effect of one of your
independent variables on the dependent variable is the same for all values of your other
independent variable (and vice versa). For example, is the effect of gender (male/female) on
test anxiety influenced by educational level (undergraduate/postgraduate)? Additionally, if a
statistically significant interaction is found, you need to determine whether there are any
"simple main effects", and if there are, what these effects are (we discuss this later in our
guide).
[RESULTS] A two-way ANOVA was conducted that examined the effect of gender and
education level on interest in politics. There was a statistically significant interaction
between the effects of gender and education level on interest in politics, F (2, 52) = 7.315, p
= .002. Simple main effects analysis showed that males were significantly more interested in
politics than females when educated to university level (p = .002), but there were no
differences between gender when educated to school (p = .465) or college level (p = .793).
Three-way: The three-way ANOVA is used to determine if there is an interaction effect
between three independent variables on a continuous dependent variable (i.e., if a three-
way interaction exists). As such, it extends the two-way ANOVA, which is used to determine
if such an interaction exists between just two independent variables (i.e., rather than three
independent variables).

Η διαφορά μεταξύ τα-Τεστ και ΑΝΟΒΑ είναι οι ΑΜ. Στο τ-Τεστ είναι πάντα 1, ενώ στο
ΑΝΟΒΑ είναι 2 και πάνω ΑΜ. Στο two-way μπορώ να δω και interaction των ΑΜ.

Non-parametric Tests
The Mann–Whitney U test ALSO CALLED Wilcoxon rank-sum test (is applied to
independent samples) assumes equal variances but not normality.

The Wilcoxon signed-rank test (is applied to matched or dependent samples).

Kruskal-Wallis H Test also called the "one-way ANOVA on ranks" (It is considered the
nonparametric alternative to the one-way ANOVA, and an extension of the Mann-Whitney
U test to allow the comparison of more than two independent groups) It is a rank-based
nonparametric test that can be used to determine if there are statistically significant
differences between two or more groups of an independent variable on a continuous or
ordinal dependent variable. It is important to realize that the Kruskal-Wallis H test is an
omnibus test statistic and cannot tell you which specific groups of your independent
variable are statistically significantly different from each other; it only tells you that at least
two groups were different. Since you may have three, four, five or more groups in your
study design, determining which of these groups differ from each other is important. You
can do this using a post hoc test.

Friedman Test (alternative to the one-way ANOVA with repeated measures) It is used to
test for differences between groups when the dependent variable being measured is
ordinal. It can also be used for continuous data that has violated the assumptions necessary
to run the one-way ANOVA with repeated measures (e.g., data that has marked deviations
from normality).

The Kaplan-Meier method also known as the "product-limit method" (used to estimate the
probability of survival past given time points (i.e., it calculates a survival distribution)
Furthermore, the survival distributions of two or more groups of a between-subjects factor
can be compared for equality.