F-Test for Effects

The first thing that is usually of interest is do determine if we have evidence to suggest that the average response differs among the groups. The test procedure is as follows:

1. The null hypothesis is that all the group means are the same, that is, , and the alternative hypothesis is that not all the group means are equal.
2. The test statistic is F=MS(Between Groups)/MS(Within Groups). The numerator and denominator can be found from the ANOVA table computed by all statistics packages.
3. The p-value is the probability of observing a value of the test statistic larger than the one observed in part 2. We use an F-distribution with k-1 and n-k degrees of freedom to find this probability.
4. Conclusion. If we have a small p-value it means that the data we observed would be unlikely to occur if the null hypothesis was correct. So this means the data are inconsistent with the null hypothesis and we have evidence against the null hypothesis and for the alternative that the group means differ. A large p-value would imply that the data are consistent with the null hypothesis and provide no evidence to conclude that the means differ. We cant conclude they are the same, only that there is no evidence that they differ from each other.