One-Way ANOVA: Comparing Three or More Groups

To compare the means of two groups we use a t-test. But what if we have three, four, or more groups? This is where one-way ANOVA (Analysis of Variance) comes in.

Why not multiple t-tests?

Running several pairwise t-tests inflates the Type I error rate. With $\alpha = .05$ per test, the chance of finding at least one false significant result across three comparisons climbs to about 14% (1 − .95³, a rough figure that treats the tests as independent).

The F statistic

ANOVA compares between-group variance to within-group variance:

When the differences between groups are large relative to the random variation within groups, $F$ grows and the $p$ value shrinks.

This ratio is the classic F-test (Fisher’s ANOVA): it pools the group variances and so assumes they are equal. Welch’s ANOVA compares the same group means but weights each group j by $n_j / s_j^2$ (its size over its variance) and adjusts the denominator degrees of freedom, so it does not need that assumption (Welch, 1951). Both test the same null hypothesis: all group means are equal.

Assumptions

The classic F-test can mislead when the group variances differ, especially when the group sizes differ too. Welch’s ANOVA keeps the Type I error rate under control in that case and loses little power when the variances are equal, so Delacre et al. (2019) recommend it as the default. Letting Levene’s test choose between the two is no remedy: tests of equal variances often lack the power to detect variance differences that do exist (Delacre et al., 2019).

A quick look with Python

from scipy import stats

group_a = [82, 85, 88, 90, 86]
group_b = [78, 81, 84, 80, 83]
group_c = [91, 94, 89, 95, 92]

# Welch's ANOVA (SciPy 1.15 or later); leave out equal_var for the classic F-test
f_stat, p_value = stats.f_oneway(group_a, group_b, group_c, equal_var=False)
print(f"Welch F = {f_stat:.2f}, p = {p_value:.4f}")

When you find a significant result ($p < .05$), move on to post-hoc tests to see which groups differ: Games-Howell after Welch’s ANOVA (Games & Howell, 1976), Tukey HSD after the classic F-test.

Run it in Basira

Try the steps above on Basira — assumption checks, post-hoc tests, and effect size are computed for you. Open K > 2 Independent Groups (Analysis Studio → Group Comparisons). Smart Mode checks normality in each group (Shapiro-Wilk) and the equality of variances (Levene’s test): it picks the classic F-test with Tukey HSD, Welch’s ANOVA with Games-Howell, or, when normality fails, the Kruskal-Wallis test with Dunn’s test. Its choice between the F-test and Welch’s ANOVA is the two-step approach advised against above. To report Welch’s ANOVA every time, use Manual Mode: select One-Way ANOVA, set Variance to Welch’s (Unequal Variance) and choose Games-Howell under Post-Hoc. Manual Mode starts with the assumption checks and the descriptive table switched off: tick Normality and Homogeneity under Assumptions and Descriptives under Output as well.