Familywise Error in Multiple Comparisons: Disentangling a Knot through a Critique of O'Keefe's Arguments against Alpha Adjustment
Title:
Familywise Error in Multiple Comparisons: Disentangling a Knot through a Critique of O'Keefe's Arguments against Alpha Adjustment
Author:
Matsunaga, Masaki
Appeared in:
Communication methods and measures
Paging:
Volume 1 (2007) nr. 4 pages 243-265
Year:
2007-12-01
Contents:
This essay discusses a statistical problem of multiple comparisons and argues for adjusting familywise alpha under certain conditions. The principle of familywise correction dictates that, upon performing multiple tests within a "family," alpha should be adjusted (i.e., lower than the conventional .05 level) so that overall Type I error inflation is controlled (e.g., Curran-Everett, 2000; Tukey, 1991). This correctional procedure recently has been disputed by O'Keefe (2003), to whose argument the present critique is counterposed. Both pros and cons of familywise alpha adjustment are discussed. First, arguments against familywise correction are reviewed with discussions of five interrelated problems. Second, an alternative perspective advocating the use of familywise procedures under certain circumstances is explicated. Notions of the scope of a null hypothesis and the statistics-theory linkage are highlighted as the grounds of this argument. Finally, an alternative approach to the multiple-comparison problem—false discovery rate control (FDR; Benjamini & Hochberg, 1995, 2000) is introduced as a promising third possibility to the "to Bonferroni or not to Bonferroni" debate. The guideline for the use of FDR and traditional familywise alpha adjustment is also discussed.
Journal description