Within-class grouping: evidence versus conjecture.

Abrami, Philip C. ; Chambers, Bette ; Lou, Yiping 等

In 1996, we published a quantitative review of the literature, using established and widely used methods (for example, Hedges and Olkin, 1985), on the effects of within-class grouping on pupil achievement and other outcomes (Lou et al., 1996). The purpose of the review, like many other reviews, was to identify the nature and overall size of the phenomenon and to determine whether any significant variability in findings could be explained. To do so, we assembled and relied on the evidence from a collection of empirical studies and did not place overriding emphasis on a single study.

The mean weighted effect size (ES) of within-class versus no grouping on pupil achievement was +0.17 and was based on 103 independent effect sizes extracted from 51 published and unpublished studies involving a total of 16,073 pupils. The ES is a standardised mean difference and can be interpreted like a z-score. Seventy-four of the ESs were greater than zero (favouring within-class grouping), five were equal to zero, and twenty-four ESs were below zero (favouring whole-class instruction). We also reported that the findings were statistically heterogeneous or variable and proceeded to explore which of 26 study features accounted for the variability.

A recent reply by Prais (1998) raises three concerns about our article: a) teachers should not decide simply whether to group or not to group pupils but consider how to divide their time among a variety of teaching styles including within-class grouping; b) there may be other organisational features of classrooms that are more important than within-class grouping; and c) we have "seriously mis-summarised our findings" (p. 83). In this abbreviated rejoinder (see also Abrami et al., 1998) we will deal with the former two concerns together before dealing with the final, more serious charge.

Teaching styles and classroom organisation

Our review was not intended to explore whether and which combination of teaching styles places pupils at greatest advantage academically. Such a review, or collection of reviews, would need either to synthesise studies which contrasted different combinations of teaching styles directly or to synthesise studies where the styles comparison could be achieved indirectly (for example, if each primary study employed a common comparison method such as 'traditional', whole-class instruction). Since our meta-analysis established that within-class grouping can be used successfully to promote pupil achievement, attitudes, and self-concept, it would be useful in such an assemblage of reviews. Consequently, we urge Prais (1998) or others to undertake the appropriate research integrations or primary investigations.

We also urge Prais (1998) to adopt more rigorous means than selective citation and conjecture to make criticisms about the possible limitations of our research and the import of other teaching methods and practices. To wit, criticisms should consider the collection of studies and follow from the empirical evidence. Doing otherwise is a denigration of social and educational research and potentially harmful to generations of pupils.

Specifically, Prais (1998) relies heavily on a discussion paper by Luxton and Last (1997) to inform his opinion. According to Prais (1998), Luxton and Last (1997) report on ". . . the main organisational features of schooling judged as important by English teachers and school inspectors following a systematic programme of observations of Continental classes. . . ." (p. 86). We question whether such a report provides sufficient and rigorous evidence to conclude whether features of teaching style or classroom organisation are responsible for enhanced pupil outcomes. For example, what were the statistical and experimental controls used to limit threats to internal validity and eliminate rival explanations? Were objective measures of achievement and other indices of student progress included?

Furthermore, Prais (1998) argues that the Continental emphasis on whole-class teaching is aided by the following features: a) flexibility in age of entry to school; b) additional teaching time for pupils with difficulties; and c) greater clarity on the essential elements of the syllabus. We wonder whether these features, individually or in combination, have been shown to influence objectively measured pupil outcomes - and not subjective impressions of same - through either primary research which statistically or experimentally control for extraneous influences or through research syntheses.

Finally, we do not believe these features are antithetical to within-class grouping. Indeed, Prais notes (1998) that additional time for pupils with difficulty ". . . might be considered as coming close to 'within-class grouping'" (p. 86). Indeed, we concluded: ". . . the best within-class grouping practices combine the physical placement of students into groups with the adaptation of instructional methods and materials for small-group learning" (Lou et al., 1996). Imagine, therefore, how much better Continental education would be if the practice of within-class grouping was more widespread.

Who's mis-summarised the findings?

A published claim that researchers have mis-summarised their findings is a strong criticism. It hints at incompetence and may even raise questions among some readers about the ethical intent of the authors. It attempts to discredit rather than merely disagree. Needless to say, we were troubled by this accusation and show below that it is without substance.

Prais (1998) offers numerous criticisms of our meta-analysis. Chief among these are: a) the overall achievement effect is small and practically useless; b) the authors overemphasise the average effect size and under-emphasise the variability in the findings; c) certain within-class grouping practices may harm pupils of low ability; d) the authors did not go far enough in their analysis of explanatory characteristics; and e) the authors conducted a series of univariate analyses rather than a single multivariate analysis. We discuss each of these in turn.

A useless finding?

We did not attempt to undertake an analysis of the costs to teachers of implementing small-group instruction versus the benefits to pupils. We were careful to limit our review to the effects of grouping on pupil outcomes, not teacher attitudes or workload.

Consequently, we did not speculate, as Prais (1998) does, about possible increases in the burden on teachers in attending simultaneously to the needs of many distinct groups within a class rather than the whole class. Nevertheless, we object to the subtle implication that teachers may be unwilling to accept the challenge of any teaching method which benefits pupils because teachers may need to work harder.

Furthermore, we can imagine situations where the demands of whole-class instruction far outweigh the demands of small-group instruction. For example, the needs of pupils for teacher assistance appear to us to be much greater when taught in a whole-class format with individual seatwork than when teachers use within-class grouping and rely on the active involvement of classmates for constructive dialogue and explanation. In whole-class instruction, the teacher provides special assistance to individuals who request it. In small-group instruction, the teacher provides special assistance to groups who request it. We invite the reader to do the mathematics on which demands are greater.

Finally, no objective means exist for judging the importance or practical utility of an effect size. Judgements of importance depend too heavily on contextual issues including such factors as the emphasis placed on pupil learning, the dedication of teachers to excellence, teacher adaptability and enthusiasm for innovation, administrative support for change, and so on.

A highly variable finding?

We anticipated that the findings of our quantitative review would not be uniform across studies. Our analysis and discussion of the 26 study features which might explain the achievement variability consumes most of the text and five tables of the review. We carefully identify the conditions which promote positive impacts of small group instruction and, therefore, conditions which detract from the effectiveness of small group instruction. We urge the reader to consult Lou et al. (1996) for the details.

In contrast, it would appear that Prais (1998) is more interested in dealing with the exception rather than the rule. His focus seems more on using the variability in study findings to argue for the widespread use of whole-class instruction. Because the findings are not wholly consistent he seems eager to suggest that they are completely inconsistent, to wit "almost anything can happen" (p. 84), that no pattern favouring small group instruction has emerged and that ". . . a teacher adopting class-grouping is engaged in a very worrying form of Russian roulette with her pupils' prospective attainments!" (p. 85).

Do the variable findings, therefore, argue for the widespread use of whole-class instruction? By extension and to continue the exaggeration, almost anything can happen with whole-class instruction and a teacher adopting whole-class instruction is engaged in a very worrying form of Russian roulette with her pupils' prospective attainments.

While the average effect size does not tell us everything important about the effectiveness of small-group instruction it does tell us something. This lesson seems to have been lost on Prais (1998) who has seriously mis-summarised the findings.

A harmful finding?

Prais (1998) claims that our findings for pupils of low ability are puzzling and inconsistent with other findings which suggest that ". . . low-ability homogeneous groups lose a 'great deal of ground'" (p. 85). This is a misrepresentation of our findings.

There were 53 study findings which compared small-group instruction versus whole-class instruction for pupils of differing relative abilities. For many of these study findings there was either insufficient detail to determine the manner of group ability composition or no data which directly compared homogeneous versus heterogeneous groups for pupils with different relative abilities (for example, a study might compare heterogeneous groups to whole-class instruction but provide no data on homogeneous groups). For these study findings we asked simply whether any form of within-class grouping (that is, both homogeneous and heterogeneous) was superior to whole-class instruction and for which pupils. Although the size of the effect varied, we found that within-class grouping benefited pupils of all relative abilities. Bright pupils gained significantly when placed in small groups for learning; pupils of medium ability gained significantly when placed in small groups for learning; and slow pupils gained significantly when placed in small groups for learning.

We next turned our attention to those study findings where there was a direct comparison of relatively homogeneous ability groups with relatively heterogeneous ability groups and there was also evidence for each type of relative pupil ability (that is, high, medium, and low). For these data, we were not interested in determining whether grouping pupils was superior to not grouping pupils, since we had settled this by our prior comparison, but were interested instead in which type of grouping was best and for which pupils. Thus, for these comparisons we did not include any data from pupils who learned via whole-class instruction.

The overall mean effect size (ES = +0.12) significantly favoured homogeneous grouping over heterogeneous grouping but the findings were variable. Low ability pupils performed best in heterogeneous groups; medium ability pupils performed best in homogeneous groups; and high ability pupils learned equally well regardless of group composition.

A superficial finding?

We are confident that our review goes well beyond any other review or primary study on within-class grouping. Nevertheless, any integrative review, even one as exhaustive as ours, is limited by the nature of the evidence. A review, after all, is an assemblage of historical evidence which allows for critical comparisons from between and within studies. The design of the review and to a certain extent its quality is dependent on the quality and comprehensiveness of the work that the primary researchers produced. A review only answers questions which the collection of evidence allows. Consequently, a reviewer's obligation is to point towards directions for future research as a way of highlighting both what is known as well as what is not yet known.

With regard to the directions for future research on within-class grouping, we wrote: "In undertaking this review, we attempted to explore the instructional and learning processes which distinguish whole-class instruction from small-group instruction and among heterogeneous and homogeneous small groups. However, the complexity of these processes and the paucity of evidence limited the extent to which we were successful. Now that we have moved closer to determining whether within-class grouping is effective, when it is effective, and with whom it is effective, it is time to devote greater energy to understanding why it is effective. We hope this review sets the stage for such inquiry" (Lou et al., 1996).

We appreciate Prais' (1998) concern for what we do not yet know. But we also recognise and appreciate that there is much that we do know. We hope that Prais comes to think likewise and encourage him to keep what is known in mind as he undertakes those rigorous investigations to explore carefully his conjectures.

An incomplete finding?

Prais (1998) also notes that we performed a series of univariate analyses for each of the explanatory features rather than a single multivariate analysis. The implication of this criticism is that we might have learned something wholly different had we approached the statistical analyses differently.

Our decision to report a series of univariate analyses was both substantive and practical. Reporting the results for each explanatory feature separately provides the most thorough presentation of the data since each explanatory factor is examined separate from the others. It also eliminates the influence of collinearity and means that evidence is neither discarded nor details lost to meet the data requirements of a multivariate analysis. In addition, including the complete results of both univariate and multivariate analyses in a single article would have made the published paper exceptionally long.

However, a multivariate analysis addresses concerns about suppressor variables and can be useful in identifying the most parsimonious model of the data. Consequently, we decided some time ago to see whether we could overcome the methodological challenges that a multivariate analysis presented and have already submitted the work for publication (Lou et al., under review).

The multivariate analysis supports and extends the univariate analyses. We identify the combination of factors which greatly enhance pupil learning in small groups (ES = 1.64). In contrast, when all these conditions are absent, learning in small groups may be less (ES = -0.02).

Conclusion

Lou et al. (1996) quantitatively integrated the empirical evidence on within-class grouping. Overall, they found positive effects of small-group instruction on pupil learning (as well as pupil attitudes and pupil general self-concept). They also found the achievement effects to be variable and used 26 study features to explore the variability. They argued that: "To be maximally effective, within-class grouping practices require the adaptation of instruction methods and material for small group learning" (Lou et al., 1996, p. 423).

Prais (1998) claimed that we seriously mis-summarised our findings. He concluded that: "Better ways forward are likely to be found by encouraging organisational reforms, on the lines (just listed) that are virtually universal in high-attaining Continental countries, and which serve to encourage more whole-class teaching and so bring the whole class forward together" (Prais, 1998, p. 87).

In this abbreviated rejoinder, we have spoken to each of Prais' (1998) concerns. We have refuted the serious accusation that we had mis-summarised our findings. In particular, our findings are: useful; not so variable as to be meaningless; provide evidence of beneficial effects for pupils of all relative abilities; are thorough and detailed; and provide a rather complete picture of the available evidence. In contrast, we believe that Prais (1998) has relied too heavily on conjecture and selective citation to offer a view of within-class grouping which is a serious mis-summarisation of the findings and a grave disservice to educators and pupils.

REFERENCES

Abrami, P.C., Lou, Y., Chambers, B., Poulsen, C. and Spence, J. C. (1998), 'Within-class grouping: evidence versus conjecture', Centre for the Study of Learning and Performance, Concordia University, Montreal, Quebec, Canada, October.

Hedges, L.V. and Olkin, I. (1985), Statistical Methods in Meta-Analysis, Orlando, FL., Academic Press.

Lou, Y., Abrami, P.C., and Spence, J.C. (under review), 'Best practices for within-class grouping', Centre for the Study of Learning and Performance, Concordia University, Montreal, Quebec, Canada.

Lou, Y., Abrami, P.C., Spence, J.C., Poulsen, C., Chambers, B. and d'Apollonia, S. (1996), 'Within-class grouping: a meta-analysis', Review of Educational Research, 66, 423-58.

Luxton, R. and Last, G. (1997), 'Under-achievement and pedagogy', National Institute Discussion Paper No. 112 (forthcoming in Teaching Mathematics and Its Applications).

Prais, S.J. (1998), 'Raising schooling attainments by grouping pupils within each class', National Institute Economic Review, 165, 83-8.