One-Way ANOVA: Comparing Three or More Groups

To compare the means of two groups we use a t-test. But what if we have three, four, or more groups? This is where one-way ANOVA (Analysis of Variance) comes in.

Why not multiple t-tests?

Running several pairwise t-tests inflates the Type I error rate. With $\alpha = .05$ per test, the chance of finding at least one false significant result across three comparisons climbs to about 14% (1 − .95³, a rough figure that treats the tests as independent).

The F statistic

ANOVA compares between-group variance to within-group variance:

F = \frac{\text{Between-group variance}}{\text{Within-group variance}} = \frac{MS_{\text{between}}}{MS_{\text{within}}}

When the differences between groups are large relative to the random variation within groups, $F$ grows and the $p$ value shrinks.

This ratio is the classic F-test (Fisher’s ANOVA): it pools the group variances and so assumes they are equal. Welch’s ANOVA compares the same group means but weights each group j by $n_j / s_j^2$ (its size over its variance) and adjusts the denominator degrees of freedom, so it does not need that assumption (Welch, 1951). Both test the same null hypothesis: all group means are equal.

Box plot showing the score distributions of three methods — Figure 1 — Score distributions of three different methods. ANOVA tests whether their means are equal.

Assumptions

The classic F-test can mislead when the group variances differ, especially when the group sizes differ too. Welch’s ANOVA keeps the Type I error rate under control in that case and loses little power when the variances are equal, so Delacre et al. (2019) recommend it as the default. Letting Levene’s test choose between the two is no remedy: tests of equal variances often lack the power to detect variance differences that do exist (Delacre et al., 2019).

A quick look with Python

from scipy import stats

group_a = [82, 85, 88, 90, 86]
group_b = [78, 81, 84, 80, 83]
group_c = [91, 94, 89, 95, 92]

# Welch's ANOVA (SciPy 1.15 or later); leave out equal_var for the classic F-test
f_stat, p_value = stats.f_oneway(group_a, group_b, group_c, equal_var=False)
print(f"Welch F = {f_stat:.2f}, p = {p_value:.4f}")

When you find a significant result ( $p < .05$ ), move on to post-hoc tests to see which groups differ: Games-Howell after Welch’s ANOVA (Games & Howell, 1976), Tukey HSD after the classic F-test.

Run it in Basira

Try it in Basira — assumption checks, post-hoc tests, and effect size are computed for you. Open K > 2 Independent Groups (Analysis Studio → Group Comparisons). Smart Mode chooses the test by the following rule:

Basira's Smart Mode runs Welch's ANOVA with Games-Howell comparisons by default. It makes the Kruskal-Wallis H test the primary test in only two cases: the dependent variable is on an ordinal scale (declared as ordinal in the dataset or, if nothing is declared, holding at most 7 distinct whole numbers), or a group is strongly non-normal (|skewness| > 2 or |excess kurtosis| > 7). The Shapiro-Wilk and Levene tests are reported but do not decide the choice. The other test is always shown as a sensitivity row, and Basira says so when the two disagree.

As of Basira 2026.10.29 (checked 24 October 2026).

Try it with sample data

Start from a ready sample dataset or upload your own, and let Basira produce the F statistic, post-hoc tests, and effect size for you.

Open