Files
Abstract
A number of multiple comparison procedures (MCPs) have been studied in the posttest only design. But for the multi-sample pretest-posttest design has received little attention regarding multiple comparison tests. The Bryant-Paulson MCP has been suggested for analysis of covariance for last three decades, but no study has been found that examined this procedures Type I error rate and statistical power. Dayton has recently proposed a new approach to multiple pairwise comparisons that has a greater true-model rate than several traditional MCPs in context of posttest only design. In the present study the statistical power and Type I error rate of five traditional MCPs and the true model rate of six selected MCPs are compared for completely randomized multi-sample pretest-posttest design. Five traditional MCPs examined are: the Shaffer (SFP), Hochberg (HBP), Bryant-Paulson (BPP), modified Hayter (MHP), and Bonferroni (BNP) procedures. Along with these five traditional MCPs, a new MCP is also considered in the present study: the Dayton model-testing procedure (MTP). Data were generated and manipulated under four data conditions: number of population compared, populations mean patterns, correlation between the covariate and dependent variable, and sample sizes. A total of 198 combinations of number of groups (3, 4, and 5), pattern of sample size per group and per correlation among covariate and dependent variable (.3, .5, and .8), and population mean configurations were manipulated using the computer simulated data. Based on the results of this study, the following conclusions were drawn: 1) the MHP and BNP maintained familywise error rate under the nominal alpha. For most of the conditions, the SFP, HBP, and BNP maintained familywise error rate two standard error below the nominal level; 2) with regards to any-pair, per-pair, and all-pairs power, the MHP outperformed the SFP, HBP, BPP, and BNP. The BPP was the second best procedure in terms of any-pair power. Regarding the per-pair and all-pairs power, the SFP was the second best procedure for J = 3 and 4 and HBP was the second best when J = 5; 3) the MTP consistently had the best true-model rate compared to the traditional MCPs; 4) for additional analyses when sample size of 20 and 25 per group were selected, the values of statistical power and true-model rate were increased for each MCP compared to the initial analyses when the original sample sizes were considered. The conclusion regarding the performance of MCPs did not change with the increased sample sizes.